Microscopic image acquisition focusing layering processing method

Abstract

The invention discloses a microscopic image acquisition focusing layering processing method, which comprises the following steps of: identifying a layer image with the highest definition in a first unit scanning area along a Z-axis direction by adopting a microscope through a hill climbing method to serve as a microscopic image, and calculating a Z-axis coordinate of the layer image to serve as a standard layer Z coordinate of a next unit scanning area; and segmenting a small number of image layers in a subsequent unit scanning area by taking the Z coordinate of the standard layer as a reference, identifying an image layer with the highest definition from the image layer as a microscopic image until the microscopic image of each unit scanning area of the slide is obtained, and splicing to obtain the microscopic image of the whole slide view. According to the method, starting from the second unit scanning area, the number of the layers needing to be segmented and the number of the layers and the images recognized after segmentation are greatly reduced, so that the handling capacity and the requirement for the computing power are greatly reduced, meanwhile, the imaging speed is increased, the structural features of the specimen in the slide can be accurately restored, and the method has good application prospects.

Description

Technical Field

[0001] The invention belongs to the field of microscopic image processing, and in particular relates to a microscopic image acquisition, focusing and layered processing method. Background Art

[0002] In clinical practice, it is difficult to directly capture clear images with existing technology, and three-dimensional cells are mostly focused by cross-sectional scanning after focusing the microscope. Figure 1 The existing stitching technology is to scan the microscopic image in units of a continuous single objective lens field of view, determine the focus elevation (Z direction) range h, estimate the coordinate position of the clear layer, and then perform layered scanning in the positive Z direction within h with a unit distance of m to determine the clarity of the scanned layer and compare it with the clarity of the estimated clear layer. When the clarity ratio exceeds p, the coordinates of the position of the layer (X1, Y1, Z1) are determined; immediately perform layered scanning in the negative Z direction of the estimated clear layer with a unit distance of m to determine the clarity of the scanned layer and compare it with the clarity of the estimated clear layer. When the clarity ratio exceeds p, the coordinates of the position of the layer (X2, Y2, Z2) are determined; at this time, (|Z1-Z2|) is used as the elevation and defined as h'. Within h', with a unit distance of m', the above method steps are repeated to continuously determine and select the clear layer, and finally n layers are divided and the clearest layer in the area is selected as the final image of the position. When using this method for microscopic imaging, characteristic cells are usually selected for positioning, presenting a local area cell image. If you want to present a clear image of the entire film, you need to cut each area into layers, and then splice and present the clear images. This method can effectively present clear cell images, but it requires strong computing power and storage as support. At the same time, image scanning and processing takes a long time and is inefficient, thus limiting the imaging speed. Summary of the invention

[0003] In view of the above-mentioned deficiencies in the prior art, the technical problem to be solved by the present invention is to provide a method for focusing and layering processing of microscopic image acquisition.

[0004] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0005] A microscopic image acquisition focusing layering processing method comprises the following steps:

[0006] S1. Establish a spatial rectangular coordinate system with the slide plane as the XOY plane and the elevation direction as the Z axis direction;

[0007] S2, determining a scanning path of the microscope, taking a single field of view of the microscope as a unit for scanning an area, and scanning the glass slide along the scanning path;

[0008] S3, photographing the first unit scanning area of ​​the scanning path with a microscope, dividing the focus area of ​​the microscope into n layers along the Z-axis direction, identifying the layer image with the highest definition along the Z-axis direction as the microscopic image of the first unit scanning area by the hill climbing method, and calculating the Z-axis coordinate of the layer image as the standard layer Z coordinate of the second unit scanning area;

[0009] S4, photographing the next unit scanning area of ​​the scanning path using a microscope, and dividing 2m+1 layers along the Z axis direction based on the standard layer Z coordinate of the unit scanning area, 2m+1<n;

[0010] S5. Identify the layer image with the highest definition from the 2m+1 layer images as the microscopic image of the unit scanning area, and calculate the Z-axis coordinate of the layer image as the standard layer Z coordinate of the next unit scanning area;

[0011] S6. Repeat steps S4 and S5 to obtain microscopic images of each unit scanning area of ​​the slide, and stitch them together to obtain a microscopic image of the entire slide field of view.

[0012] Furthermore, in the step S1, a corner point of the glass slide is used as the origin of the spatial rectangular coordinate system, and the length direction and width direction of the glass slide are used as the X-axis direction and Y-axis direction of the XOY plane, respectively.

[0013] Furthermore, in the step S2, when scanning the glass slide, a "battlement-like" movement method is used in the X-axis direction and the Y-axis direction to determine the scanning path of the microscope.

[0014] Furthermore, in the step S4, the method of dividing the focus area of ​​the recognition image into 2m+1 layers along the Z axis direction based on the Z coordinate of the standard layer is:

[0015] Take the Z coordinate of the standard layer as the standard layer, start from the standard layer and step upward with a predetermined step value and divide it in sequence to form m layers, start from the standard layer and step downward with a predetermined step value and divide it in sequence to form m layers, and obtain 2m+1 layers including the standard layer.

[0016] Furthermore, the value of m is 2 or 3.

[0017] Furthermore, the predetermined step value is 50 nm to 500 nm.

[0018] Furthermore, in the step S3 and the step S5, after identifying the layer image with the highest clarity, the center position coordinates of the layer image are obtained as image coordinates, and the Z coordinate in the center position coordinates is the standard layer Z coordinate of the next unit scanning area; in the step S6, the positions of each microscopic image are arranged according to the X coordinate and Y coordinate in the image coordinates, and splicing is performed.

[0019] Furthermore, in the step S3 and the step S5, the center position coordinates of the layer image are obtained by using a microbubble positioning method.

[0020] Furthermore, in the step S5, the method for identifying the layer image with the highest definition is:

[0021] Presetting a convolutional neural network model for predicting the clarity of an image, and training the convolutional neural network model to obtain a clarity recognition model;

[0022] Each layer image is input into the clarity recognition model to obtain the clarity prediction value of each layer image, and the layer image with the highest clarity prediction value is output as the layer image with the highest clarity.

[0023] Furthermore, in the step S5, the following formula is also used to calculate the clarity factor of each layer image:

[0024]

[0025] Where σ represents the clarity factor; N represents the number of all pixels in the area; x i represents the gray level of the i-th pixel; μ represents the average gray level of the image;

[0026] After calculating the clarity factor of each layer image, it is determined whether the layer image output by the clarity recognition model is consistent with the layer image with the largest clarity factor value. If they are inconsistent, the layer image output by the clarity recognition model is marked.

[0027] In the present invention, starting from the second unit scanning area, the layers of each area are divided into only 5 or 7 layers, from which the clearest images can be identified and screened. The number of layers that need to be segmented and the number of layer images identified after segmentation are greatly reduced, thereby greatly reducing the processing volume and the requirements for computing power, while improving the imaging speed. It can be used for scanning and processing of slides such as blood smears, and truly restore the structural characteristics of the specimens on the slides. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0029] Figure 1 Schematic diagram of using the hill climbing method to identify the layer image with the highest clarity.

[0030] Figure 2 The flowchart is a method for processing microscopic image acquisition focus layers according to an embodiment of the present invention.

[0031] Figure 3 This is a schematic diagram of scanning the entire slide and stitching together to form a microscopic image of the entire slide field of view in this embodiment.

[0032] Figure 4 Schematic diagram of scanning the entire slide using the "battlement-style" movement method.

[0033] The accompanying drawings in the specification are numeraled as follows:

[0034] Glass slide-100; single field of view area-200; first unit scanning area-201; second unit scanning area-202; scanning path-300. DETAILED DESCRIPTION

[0035] The following describes the implementation methods of the present invention through specific examples. The illustrations provided in the following embodiments are only used to schematically illustrate the basic concept of the present invention. The following embodiments and features in the embodiments may be combined with each other without conflict.

[0036] See also Figure 2 and Figure 3 An embodiment of the microscopic image acquisition focusing layering processing method of the present invention comprises the following steps:

[0037] S1. A spatial rectangular coordinate system is established with the plane where the glass slide 100 is located as the XOY plane and the elevation direction as the Z axis direction. Generally, a corner point of the glass slide 100 is used as the origin of the spatial rectangular coordinate system, and the length direction and width direction of the glass slide 100 are used as the X axis direction and Y axis direction of the XOY plane, respectively. In this embodiment, a spatial rectangular coordinate system is established with the corner point of the lower left corner of the glass slide 100 as the origin, the length direction of the glass slide 100 as the X direction, the width direction of the glass slide 100 as the Y direction, and the elevation direction as the Z axis direction.

[0038] S2, determine the scanning path 300 of the microscope, based on a single field of view area 200 of the microscope lens (i.e., the area corresponding to one field of view of the microscope lens, corresponding to Figure 3 The glass slide 100 is scanned along the scanning path 300. Figure 4 In this embodiment, when scanning the glass slide 100, a "battlement-type" movement method is used to determine the scanning path 300 of the microscope in the X-axis direction and the Y-axis direction. Figure 4The arrow direction of the scanning path 300 is the moving direction during scanning.

[0039] S3. Take the unit scanning area corresponding to the starting position of the scanning path 300 as the first unit scanning area 201, use a microscope to shoot the first unit scanning area 201 of the scanning path 300, and divide the focus area into n layers along the Z-axis direction. The spacing between adjacent layers is generally 50nm~500nm, which can be determined according to the actual situation of the specimen in the glass slide 100. The layer image with the highest clarity is identified along the Z-axis direction by the hill climbing method as the microscopic image of the first unit scanning area 201, that is, the final image of the first unit scanning area 201; the position of the layer with the highest clarity found is the focus position of the microscope with the highest clarity. Afterwards, it is also necessary to calculate the Z coordinate of the layer image with the highest clarity, and use it as the standard layer Z coordinate of the second unit scanning area 202. Among them, the hill climbing method is a commonly used method for identifying the layer image with the highest clarity. The principle can be referred to the records and Figure 1 , I will not go into details here.

[0040] According to the actual situation of the specimen in the slide 100, since the imaging distance of the specimen varies only within a very small range, the change in the sharpest focus position of two adjacent unit scanning areas is very small. Therefore, the sharpest focus position of the previous unit scanning area (that is, the Z coordinate of the layer image with the highest clarity) can be directly used as the center to float up and down within a small range to find the layer image with the highest clarity in the next unit scanning area. Its Z coordinate is the sharpest focus position of the unit scanning area, thereby greatly reducing the number of segmented layers.

[0041] In this embodiment, after identifying the layer image with the highest definition in the first unit scanning area 201, the center position coordinates (X a1 , Y a1 , Z a1 ) as the image coordinates of the microscopic image corresponding to the first unit scanning area 201. For example, the center position coordinates of the layer image can be obtained by using a microbubble positioning method. The microbubble positioning method is a commonly used coordinate positioning method and will not be described in detail here. The Z coordinate Z in the center position coordinates a1 That is, the standard layer Z coordinate of the second unit scanning area 202. a1 and Y coordinate Y a1 It is used for positioning the image, so as to locate the position of the microscopic image of the first unit scanning area 201 during subsequent image stitching.

[0042] S4, using a microscope to photograph the next unit scanning area of ​​the scanning path 300, and taking the standard layer Z coordinate of the unit scanning area as a reference, dividing 2m+1 layers along the Z axis direction, 2m+1<n. Wherein, taking the standard layer Z coordinate as a reference, the method of dividing the focus area of ​​the recognition image into 2m+1 layers along the Z axis direction is:

[0043] The Z coordinate of the standard layer is used as the standard layer, starting from the standard layer, stepping upward with a predetermined step value and sequentially splitting to form m layers, starting from the standard layer, stepping downward with a predetermined step value and sequentially splitting to form m layers, and obtaining 2m+1 layers including the standard layer. The step value generally ranges from 50nm to 500nm, which can be determined according to the actual situation of the specimen in the slide 100. In combination with the implementation situation, in order to simplify the calculation, the value of m is generally 2 or 3 to find a clear image.

[0044] In this embodiment, m=2. For example, after obtaining the microscopic image of the first unit scanning area 201, the second unit scanning area 202 is photographed. At this time, according to the coordinate Y a1 The standard layer of the second unit scanning area 202 is determined, and the layer image of the standard layer is first obtained as the first layer image. Then, based on the standard layer, a layer image can be obtained by moving up by one step value from the standard layer as the second layer image, and then by moving up by one step value as the third layer image, and then a layer image can be obtained by moving down by one step value from the standard layer as the fourth layer image, and then by moving down by one step value as the fifth layer image, thereby obtaining five layer images. Of course, depending on the implementation, m=3 can sometimes be taken, so that three layers are divided above and below the standard layer, thereby obtaining seven layer images.

[0045] S5. Identify the layer image with the highest definition from the 2m+1 layer images as the microscopic image of the unit scanning area, and calculate the Z-axis coordinate of the layer image with the highest definition as the standard layer Z coordinate of the next unit scanning area. In this embodiment, a convolutional neural network model is used to identify the layer image with the highest definition, and the specific method is as follows:

[0046] A convolutional neural network model for predicting the clarity of an image is pre-set, and the convolutional neural network model is trained to obtain a clarity recognition model. During recognition, each layer image is input into the clarity recognition model to obtain a clarity prediction value of each layer image, and the layer image with the highest clarity prediction value is output as the layer image with the highest clarity.

[0047] In order to test the accuracy of the clarity recognition model, in this step, the clarity factor of each layer image is calculated according to the grayscale of the image and it is determined whether the layer image output by the clarity recognition model is consistent with the layer image with the largest clarity factor value. If they are consistent, it can be considered that the layer image output by the clarity recognition model is the image with the highest clarity, that is, the clarity recognition model output is accurate. If the layer image output by the clarity recognition model is inconsistent with the layer image with the largest clarity factor value, it is considered that the layer image output by the clarity recognition model may not be the image with the highest clarity. The layer image output by the clarity recognition model is marked and can be manually verified later. If it is confirmed that the layer image output by the clarity recognition model is not the image with the highest clarity, the clarity recognition model can be improved. The formula for calculating the clarity factor of each layer image is as follows:

[0048]

[0049] Where σ represents the clarity factor; N represents the number of all pixels in the area; x i represents the gray level of the i-th pixel; μ represents the average gray level of the image.

[0050] Since the prediction method using the convolutional neural network model is more accurate, the layer image output by the convolutional neural network model is always used as the microscopic image of the unit scanning area. The calculation formula of the clarity factor is only used to assist in verifying the output of the convolutional neural network model and will not be used as the main basis for judgment.

[0051] For example, for the five layer images of the second unit scanning area 202, the five layer images are first input into the clarity recognition model during recognition, and the clarity recognition model outputs a layer image as the microscopic image of the second unit scanning area 202; thereafter, the clarity factors of the five layer images are calculated respectively through the formula, and the layer image with the largest clarity factor is compared with the layer image output by the clarity recognition model to see whether they are consistent, so as to further determine whether the output of the clarity recognition model is accurate.

[0052] Of course, in this step, the center position coordinates (X a2 , Y a2 , Z a2 ) as the image coordinates of the microscopic image corresponding to the second unit scanning area 202. The Z coordinate Z in the center position coordinates a2 This is the standard layer Z coordinate of the third unit scanning area. a2 and Y coordinate Y a2 It is used for positioning the image, so as to locate the position of the microscopic image of the second unit scanning area 202 during subsequent image stitching.

[0053] S6, repeating steps S4 and S5 to obtain microscopic images of each region of the glass slide 100, and stitching together to obtain a microscopic image of the entire field of view of the glass slide 100. For example, for the third unit scanning area, according to the coordinate Z a2 Determine the standard layer of the third unit scanning area, and obtain the 5 layer images of the third unit scanning area by stepping up and down the standard layer, and then obtain the microscopic image of the third unit scanning area and the center position coordinates (X a3 , Y a3 , Z a3 ), Z coordinate Z a3 This is the standard layer Z coordinate of the fourth unit scanning area. a2 and Y coordinate Y a2 It is used for image positioning, so as to locate the position of the microscopic image of the third unit scanning area during subsequent image stitching.

[0054] After that, the above operation is repeated for each subsequent unit scanning area in sequence until the scanning of the entire area of ​​the glass slide 100 is completed, and the microscopic images of all the unit scanning areas of the glass slide 100 are obtained. Finally, the positions of the microscopic images are arranged according to the X coordinates and Y coordinates in the image coordinates of each microscopic image, and they are spliced ​​to obtain a clear microscopic image of the entire field of view of the glass slide 100.

[0055] In this embodiment, only the first unit scanning area 201 needs to use the hill climbing method to identify the clearest image. Starting from the second unit scanning area 202, the processing of subsequent unit scanning areas no longer divides the layers of each area into n layers and then identifies and selects the clearest image, but only divides them into 5 layers, and identifies and selects the clearest image in the 5 layers. Compared with the existing method, when this embodiment processes clear images of the same number of fields of view, the number of required segmentations and the number of identifications after segmentation are reduced, and the processing volume is reduced by about n / 5 times of the existing method, which greatly reduces the requirements for computing power and improves the imaging speed. The method of this embodiment is used to scan and process blood smears (formed by applying blood on a glass slide 100 in a specific method), which can truly restore various cell characteristics, perfectly present the delicate texture, color and internal details of the cells, including the clear reproduction of cell lines, cytoplasm, particles, etc., simulate manual focusing, accurately restore cell pathological characteristics, and provide a judgment basis for early screening of blood diseases (such as acute promyelocytic leukemia) and infectious diseases.

[0056] The above embodiments only express the preferred implementation modes of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for those of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for focusing and layering microscopic image acquisition, It is characterized in that The following steps are involved: S1. Establish a spatial rectangular coordinate system with the slide plane as the XOY plane and the elevation direction as the Z axis direction; S2, determining a scanning path of the microscope, taking a single field of view of the microscope as a unit for scanning an area, and scanning the glass slide along the scanning path; S3, photographing the first unit scanning area of ​​the scanning path with a microscope, dividing the focus area of ​​the microscope into n layers along the Z-axis direction, identifying the layer image with the highest definition along the Z-axis direction as the microscopic image of the first unit scanning area by the hill climbing method, and calculating the Z-axis coordinate of the layer image as the standard layer Z coordinate of the second unit scanning area; S4, using a microscope to photograph the next unit scanning area of ​​the scanning path, and taking the standard layer Z coordinate of the unit scanning area as a reference, dividing 2m+1 layers along the Z axis direction, 2m+1<n; S5. Identify the layer image with the highest definition from the 2m+1 layer images as the microscopic image of the unit scanning area, and calculate the Z-axis coordinate of the layer image as the standard layer Z coordinate of the next unit scanning area; S6. Repeat steps S4 and S5 to obtain microscopic images of each unit scanning area of ​​the slide, and stitch them together to obtain a microscopic image of the entire slide field of view.

2. The microscopic image acquisition focusing layering processing method according to claim 1, Features: In the step S1, a corner point of the glass slide is used as the origin of the spatial rectangular coordinate system, and the length direction and width direction of the glass slide are used as the X-axis direction and Y-axis direction of the XOY plane respectively.

3. The microscopic image acquisition focusing layering processing method according to claim 2, Features: In the step S2, when scanning the glass slide, a "battlement-like" movement method is used to determine the scanning path of the microscope in the X-axis direction and the Y-axis direction.

4. The microscopic image acquisition focusing layering processing method according to claim 1, Features: In the step S4, the method of dividing the focus area of ​​the recognition image into 2m+1 layers along the Z axis direction based on the standard layer Z coordinate is: Take the Z coordinate of the standard layer as the standard layer, start from the standard layer and step upward with a predetermined step value and divide it in sequence to form m layers, start from the standard layer and step downward with a predetermined step value and divide it in sequence to form m layers, and obtain 2m+1 layers including the standard layer.

5. The microscopic image acquisition focusing layering processing method according to claim 4, Features: The value of m is 2 or 3.

6. The microscopic image acquisition focusing layering processing method according to claim 4, Features: The predetermined step value is 50 nm to 500 nm.

7. The microscopic image acquisition focusing layering processing method according to claim 1, Features: In the step S3 and the step S5, after identifying the layer image with the highest definition, the center position coordinates of the layer image are obtained as image coordinates, and the Z coordinate in the center position coordinates is the standard layer Z coordinate of the next unit scanning area; in the step S6, the positions of each microscopic image are arranged according to the X coordinate and Y coordinate in the image coordinates, and splicing is performed.

8. The microscopic image acquisition focusing layering processing method according to claim 7, Features: In the step S3 and the step S5, the center position coordinates of the layer image are obtained by using the microbubble positioning method.

9. The microscopic image acquisition focusing layering processing method according to any one of claims 1 to 8, It is characterized in that In the step S5, the method for identifying the layer image with the highest definition is: Presetting a convolutional neural network model for predicting the clarity of an image, and training the convolutional neural network model to obtain a clarity recognition model; Each layer image is input into the clarity recognition model to obtain the clarity prediction value of each layer image, and the layer image with the highest clarity prediction value is output as the layer image with the highest clarity.

10. The microscopic image acquisition focusing layering processing method according to claim 9, Features: In the step S5, the following formula is also used to calculate the clarity factor of each layer image: Where σ represents the clarity factor; N represents the number of all pixels in the area; x i represents the gray level of the i-th pixel; μ represents the average gray level of the image; After calculating the clarity factor of each layer image, it is determined whether the layer image output by the clarity recognition model is consistent with the layer image with the largest clarity factor value. If they are inconsistent, the layer image output by the clarity recognition model is marked.

Images