Focusing searching method used in nasal secretion smear scanning process

Abstract

The invention relates to the technical field of pathological sections, and provides a focus searching method for a nasal secretion smear scanning process, which comprises the following steps of: dividing a delimited scanning area into M rows and N columns of small scanning blocks; obtaining a focus searching point list according to the slide global image and the coordinates of all the small scanning blocks; a focus searching point is taken from the list in sequence, the camera is moved to the f point to shoot pictures, and the definition of each picture is calculated; the picture with the highest definition is taken, if the detected cell number is larger than R, the z-axis reference position is adjusted to the z-axis position corresponding to the picture, and the rough focus searching process is ended; finding a focus f from the list in sequence, moving the camera to the point f, and moving the height of the camera to a z1 position to shoot a picture; if the detected number of the cells is greater than R, the camera shoots pictures within z1 + / -m * q with the step of m, and the definition of each picture is calculated; and taking the z-axis position corresponding to the picture with the highest definition as the focus of the point until all the focuses are found.

Description

Technical Field

[0001] The invention relates to the technical field of pathological sections, and in particular to a focus-finding method used in a nasal secretion smear scanning process. Background Art

[0002] The nasal secretion smear examination mainly checks for neutrophils and eosinophils under a microscope, which can provide an important reference for the diagnosis of rhinitis. The nasal secretion smear is scanned into a digital image with a pathology slice scanner, and then the cells in it are detected using a deep learning method, which can greatly reduce the workload of the pathologist and speed up the diagnosis. The quality of the scan has an important impact on the final diagnosis. At present, conventional scanners need to judge the scanning area and focus through a global map, but the cells of nasal secretions are sparsely distributed on the glass slide, and may be blank on the global map. No obvious focus can be found, resulting in focus failure. In addition, the glue used to seal the glass slide will produce bubbles, which are similar in shape to cells and will interfere with the focus of the cells. In summary, the current scanner's focus search method cannot meet the scanning requirements of nasal secretion smears. Summary of the invention

[0003] In order to solve the above problems, the present invention provides the following technical solutions:

[0004] The present invention provides a focus finding method for use in a nasal secretion smear scanning process, the focus finding method comprising:

[0005] (1) Take a global image of the slide and define a scanning area frame, and divide the defined scanning area into M rows and N columns of scanning blocks, with some overlap between adjacent blocks to ensure normal splicing;

[0006] (2) Obtaining a focus search list F based on the global image of the slide and the coordinates of all scanned blocks, where list F is sorted by the grayscale values ​​of the foreground pixels in the scanned blocks;

[0007] (3) Take a focus point f from the list F in order, move the camera to point f, and take pictures between z0±n*p with a step of n, and calculate the clarity of each picture; where z0 is the z-axis reference position, p is half of the number of pictures taken minus 1, that is, 2p+1 is the total number of pictures taken;

[0008] For example, when the step n is 120 and p is 15, 31 images of [z0-120x15,z0-120x14,…,z0+120x14,z0+120x15] are taken and the clarity of the 31 images is calculated;

[0009] (4) Take the image with the highest definition and use the cell detection model to detect cells. If the number of cells detected is greater than R, adjust the z-axis reference position to the z-axis position corresponding to the image, and the coarse focus process ends; otherwise, return to step (3); R is preferably 3;

[0010] (5) Take a focus point f from the list F in order, move the camera to point f, move the camera to the height of z1 and take a picture;

[0011] (6) The image is tested for cells by the cell detection model. If the number of cells detected is greater than 3, the camera takes pictures between z1±m*q with a step of m, and calculates the clarity of each picture; otherwise, return to step 5; where n>m, q is half of the number of pictures taken minus 1, that is, 2q+1 is the total number of pictures taken;

[0012] For example, when the step m is 10 and q is 10, take 21 images of [z1-10x10,z1-10x9,…,z1+10x9,z1+10x10] and calculate the clarity of the 21 images;

[0013] (7) The z-axis position corresponding to the image with the highest resolution is taken as the focus of the point, and the process returns to step (5) until all the focuses are found.

[0014] Furthermore, the scanned blocks within the scanned area are scanned in an S-shaped order, and the scanning order is (1,1)>(2,1)>...>(M,1)>(M,2)>(M-1,2)>...(1,2)>(1,3)>(2,3).

[0015] Furthermore, the specific process of obtaining the focus list F in step (2) is as follows:

[0016] (2-1) Use the cvtColor function of opencv to convert the global image into a grayscale image G;

[0017] (2-2) Each pixel in the grayscale image G is a foreground pixel (cell) with a grayscale value S = 255-G. Since the value of each pixel in the grayscale image ranges from 0 to 255, if there is a cell on this pixel (foreground pixel), then the value of this pixel is far from 255. If there is nothing on this pixel (background pixel), then the value of this pixel is close to 255. Using 255-pixel grayscale value, the larger the result, the farther away from 255, and the more likely it is a cell.

[0018] (2-3) For all scanned blocks, the grayscale value of the foreground is calculated as follows: the grayscale mean of the foreground pixel of the scanned block is expressed as mean(S[y:y+h,x:x+w]), where xy and wh are the xy coordinates and width and height of the scanned block respectively;

[0019] (2-4) Initialize a binary image B, set the height of the binary image to M, the width to N, and the initial value to 1. When the value of the binary image B is 1, it means that it can be used as a focus search, and when the value of the binary image B is 0, it means that it cannot be used as a focus search, where each pixel in the binary image is a scanned small block;

[0020] (2-5) Call OpenCV's distanceTransform function to calculate the distance from each point with a value of 1 in the binary image B to the nearest point with a value of 0;

[0021] (2-6) Among the points whose distance is greater than k, take the point with the highest grayscale value of the foreground pixel and add it to the list F. Set the row and column coordinates of the point to (u, v). Set all the points between [u-2:u+2,v-2:v+2] in B to 0, and they can no longer be used as search points. Among them, 0≤k≤3 indicates the minimum interval between search points.

[0022] (2-7) Repeat steps (2-5) and (2-6) until there are no points in the binary image that can be used as focus points.

[0023] Furthermore, the calculation process of the clarity in step (3) is:

[0024] (3-1) Use the cvtColor function of opencv to convert the captured color image into a grayscale image;

[0025] (3-2) Use the resize function of opencv to reduce the grayscale image to 0.25 of its original size;

[0026] (3-3) Use the Sobel function of opencv to calculate its gradient dx, dy in the x and y directions;

[0027] (3-4) Use the magnitude function of opencv to calculate the gradient value M, and take the mean of the gradient value M as the clarity of the image.

[0028] Furthermore, the method for determining z1 in step (5) is:

[0029] Take out the points in list F that have obtained the fine-focus z value and put them into list F1. If list F1 is empty, return the reference position z0, and z1=z0. If list F1 is not empty, calculate the distance between the current point and the points in list F1, and take the z value of the point closest to it as the reference position z1 of the current point.

[0030] 6. A focus-finding method for nasal secretion smear scanning according to claim 1, characterized in that the calculation process of the clarity in step (6) is:

[0031] (6-1) According to the detected cell position and size, the corresponding area is cut out from the captured image;

[0032] (6-2) Convert the captured image into a grayscale image;

[0033] (6-3) Reduce the grayscale image proportionally to 0.25 of its original size;

[0034] (6-4) Calculate its gradient dx, dy in the x and y directions;

[0035] (6-5) Calculate the gradient value. If the cell is a neutrophil, use the mean of the gradient value as the clarity of the image. If the cell is an eosinophil, increase the weight of the eosinophil and use the mean of the gradient value multiplied by 2 as the clarity of the image.

[0036] The present invention has the following beneficial effects:

[0037] (1) The present invention divides the scanning area frame defined by the global image of the glass slide into scanning blocks, sorts the cells in the scanning blocks according to their confidence scores, and performs coarse focusing and fine focusing in order of confidence scores, thereby speeding up the focusing speed. During the focusing process, the shortest distance between each focus point can also be set according to needs. Even if the cell area is not clearly visible on the global image, the method can still find the area where some cells are located and focus on them;

[0038] (2) The present invention uses a cell detection model to focus the image, eliminating interference from impurities and bubbles, etc., so that the cells can be accurately focused and a clear nasal secretion scan can be obtained. This can be used to construct a machine learning method to automatically count the number of each cell, thereby improving the efficiency and accuracy of diagnosis; and it can also quickly eliminate cell-free focus and speed up the entire focus-finding process. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a focus search flow chart of the present invention.

[0040] Figure 2 It is a schematic diagram of the scanning area defined in the present invention.

[0041] Figure 3 It is a focus diagram of the focus search list F in the present invention. Figure 4 This is a diagram of cell detection results during the focus-finding process of the present invention. DETAILED DESCRIPTION

[0042] The specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings. It should be pointed out that the embodiments are only specific explanations of the invention and should not be regarded as limitations of the invention. The purpose of the embodiments is to enable those skilled in the art to better understand and reproduce the technical solutions of the present invention. The protection scope of the present invention shall still be based on the scope defined by the claims.

[0043] like Figure 1 As shown, the present invention provides a focus finding method for use in a nasal secretion smear scanning process, the focus finding method comprising:

[0044] S1, take a global image of the slide and define the scanning area frame, such as Figure 2 As shown, the designated scanning area is divided into scanning blocks of M rows and N columns, and there is partial overlap between adjacent blocks to ensure normal splicing; the scanning blocks in the scanning area are scanned in an S-shaped order, and the scanning order is (1,1)>(2,1)>...>(M,1)>(M,2)>(M-1,2)>...(1,2)>(1,3)>(2,3).

[0045] S2, obtain the focus search list F according to the global image of the slide and the coordinates of all scanned small blocks, such as Figure 3 As shown, the list F is sorted by the grayscale value of the foreground pixel points (cells) in the scanned small block; the specific process of obtaining the focus list F is: (2-1) using the cvtColor function of opencv to convert the global image into a grayscale image G;

[0046] (2-2) Each pixel in the grayscale image G is a foreground pixel (cell) with a grayscale value S = 255-G. Since the value of each pixel in the grayscale image ranges from 0 to 255, if there is a cell on this pixel (foreground pixel), then the value of this pixel is far from 255. If there is nothing on this pixel (background pixel), then the value of this pixel is close to 255. Using 255-pixel grayscale value, the larger the result, the farther away from 255, and the more likely it is a cell.

[0047] (2-3) For each scanned block, the grayscale value of its foreground is calculated as follows: the grayscale mean of the foreground pixel of the scanned block is expressed as mean(S[y:y+h,x:x+w]), where xy and wh are the xy coordinates and width and height of the scanned block respectively;

[0048] (2-4) Initialize a binary image B, set the height of the binary image to M, the width to N, and the initial value to 1. When the value of the binary image B is 1, it means that it can be used as a focus search, and when the value of the binary image B is 0, it means that it cannot be used as a focus search, where each pixel in the binary image is a scanned small block;

[0049] (2-5) Use the distanceTransform function of OpenCV to calculate the distance from each point with a value of 1 to the nearest point with a value of 0 in the binary image B;

[0050] (2-6) Among the points whose distance is greater than k, take the point with the highest foreground score and add it to the list F; that is, let the row and column coordinates of the point be (u, v), and set all the points between [u-2:u+2,v-2:v+2] in B to 0, that is, the points within the radius k of the point can no longer be used as the focus; where 0≤k≤3 indicates the minimum interval between the focus points;

[0051] (2-7) Repeat steps (2-5) and (2-6) until there are no points in the binary image that can be used as focus points.

[0052] S3, sequentially take a focus point f from the list F, move the camera to point f, and take pictures between z0±n*p with a step of n, and calculate the clarity of each picture; where z0 is the z-axis reference position; p is half of the number of pictures taken minus 1, that is, 2p+1 is the total number of pictures taken; for example, when the step n is 120 and p is 15, 31 pictures [z0-120x15, z0-120x14, …, z0+120x14, z0+120x15] are taken, and the clarity of the 31 pictures is calculated;

[0053] The calculation process of clarity is:

[0054] (3-1) Use the cvtColor function of opencv to convert the captured color image into a grayscale image;

[0055] (3-2) Use the resize function of opencv to reduce the grayscale image to 0.25 of its original size;

[0056] (3-3) Use the Sobel function of opencv to calculate its gradient dx, dy in the x and y directions;

[0057] (3-4) Use the magnitude function of opencv to calculate the gradient value M, and take the mean of the gradient value M as the clarity of the image.

[0058] S4, take the image with the highest definition, and use the cell detection model to detect cells. If the number of cells detected is greater than R, adjust the z-axis reference position to the z-axis position corresponding to the image, and the coarse focus process ends; otherwise, return to step S3; R is preferably 3;

[0059] S5, select a focus point f from the list F in order, move the camera to point f, move the camera to a height of z1 and take a picture; wherein z1 is determined as follows:

[0060] Take out the points in list F that have obtained the fine-focus z value and put them into list F1. If list F1 is empty, return the reference position z0, and z1=z0. If list F1 is not empty, calculate the distance between the current point and the points in list F1, and take the z value of the point closest to it as the reference position z1 of the current point.

[0061] S6, the image is detected by the cell detection model. If the number of cells detected is greater than 3, the camera takes pictures between z1±m*q with a step of m, and calculates the clarity of each picture; otherwise, return to step S5; where n>m, q is half of the number of pictures taken minus 1, that is, 2q+1 is the total number of pictures taken;

[0062] For example, when the step m is 10 and q is 10, 21 images of [z1-10x10, z1-10x9, …, z1+10x9, z1+10x10] are taken and the clarity of the 21 images is calculated; the clarity calculation process is:

[0063] (6-1) According to the detected cell position and size, the corresponding area is cut out from the captured image;

[0064] (6-2) Convert the captured image into a grayscale image;

[0065] (6-3) Reduce the grayscale image proportionally to 0.25 of its original size;

[0066] (6-4) Calculate its gradient dx, dy in the x and y directions;

[0067] (6-5) Calculate the gradient value. If the cell is a neutrophil, use the mean of the gradient value as the clarity of the image. If the cell is an acidophil, increase the weight of the acidophil, and multiply the mean of the gradient value by 2 as the clarity of the image. Figure 4 This is the cell detection result diagram. In the diagram, the eosinophils are detected in the red box, and the neutrophils are detected in the green box. It can be clearly seen that the detected cells are very clear, and there are no impurities or bubbles in the diagram. The z-axis position corresponding to the image with the highest clarity is taken as the focus of the point, and the process returns to step S5 until all the focuses are found.

[0068] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0069] It should be noted that the technical features not described in detail in the present invention can be implemented by any existing technology.

Claims

1. A focusing method for nasal secretion smear scanning, characterized in that: The focus finding method comprises: (1) Taking a global image of the slide and defining a scanning area frame, and dividing the defined scanning area into scanning blocks of M rows and N columns, with some overlap between adjacent blocks; (2) Obtaining a focus search list F based on the global image of the slide and the coordinates of all scanned blocks, wherein the list F is sorted by the grayscale values ​​of the foreground pixels in the scanned blocks; (3) Take a focus point f from the list F in order, move the camera to point f, and take pictures between z0±n*p with a step of n, and calculate the clarity of each picture; where z0 is the z-axis reference position, and p is half of the number of pictures taken minus 1; (4) Take the image with the highest resolution and use the cell detection model to detect cells. If the number of cells detected is greater than R, adjust the z-axis reference position to the z-axis position corresponding to the image, and the coarse focus search process ends; otherwise, return to step (3); (5) Take a focus point f from the list F in order, move the camera to point f, move the camera to the height of z1 and take a picture; (6) The image is used to detect cells through the cell detection model. If the number of cells detected is greater than R, the camera takes pictures between z1±m*q with a step of m, and calculates the clarity of each picture; otherwise, return to step 5; where n>m, q is half of the number of pictures taken minus 1; (7) The z-axis position corresponding to the image with the highest resolution is taken as the focus of the point, and the process returns to step (5) until all the focuses are found.

2. A focusing method for nasal secretion smear scanning according to claim 1, characterized in that: The scan blocks within the scan area are scanned in an S-shaped sequence.

3. A focusing method for nasal secretion smear scanning according to claim 1, characterized in that: The specific process of obtaining the focus list F in step (2) is as follows: (2-1) Convert the global image into a grayscale image G; (2-2) Each pixel in the grayscale image G is a foreground pixel with a grayscale value S = 255-G; (2-3) For each scanned block, the grayscale value of the foreground is calculated as follows: the grayscale mean of the foreground pixel of the scanned block is expressed as mean(S[y:y+h,x:x+w]), where xy and wh are the xy coordinates and width and height of the scanned block respectively; (2-4) Initialize a binary image B, set the height of the binary image to M, the width to N, and the initial value to 1. When the value of the binary image B is 1, it means that it can be used as a focus search, and when the value of the binary image B is 0, it means that it cannot be used as a focus search, where each pixel in the binary image is a scanned small block; (2-5) Calculate the distance between each point with a value of 1 and the nearest point with a value of 0 in the binary graph B; (2-6) Among the points whose distance is greater than k, take the point with the highest grayscale value of the foreground pixel and add it to the list F. Set the row and column coordinates of the point to (u, v). Set all the points between [u-2:u+2,v-2:v+2] in B to 0, so that they can no longer be used as focus points; where 0≤k≤3; (2-7) Repeat steps (2-5) and (2-6) until there are no points in the binary image that can be used as focus points.

4. A focusing method for nasal secretion smear scanning according to claim 1, characterized in that: The calculation process of clarity in step (3) is: (3-1) Convert the captured color image into a grayscale image; (3-2) Reduce the grayscale image proportionally to 0.25 of its original size; (3-3) Calculate its gradient dx, dy in the x and y directions; (3-4) Calculate the gradient value M, and take the average of the gradient values ​​M as the clarity of the image.

5. A focusing method for nasal secretion smear scanning according to claim 1, characterized in that: The method for determining z1 in step (5) is: Take out the points with fine-focus z values ​​from list F and put them into list F1. If list F1 is empty, return the reference position z0, and z1=z0. If list F1 is not empty, calculate the distance between the current point and the point in list F1, and take the z value of the point with the closest distance as the reference position z1 of the current point.

6. A focusing method for nasal secretion smear scanning according to claim 1, characterized in that: The calculation process of clarity in step (6) is: (6-1) According to the detected cell position and size, the corresponding area is cut out from the captured image; (6-2) Convert the captured image into a grayscale image; (6-3) Reduce the grayscale image proportionally to 0.25 of its original size; (6-4) Calculate its gradient dx, dy in the x and y directions; (6-5) Calculate the gradient value. If the cell is a neutrophil, the mean of the gradient value is used as the clarity of the image; If the cells are eosinophils, the value obtained by multiplying the mean of the gradient values ​​by 2 is used as the clarity of the image.

Images