Method for selecting suitable high-power image area from low-power image for blood cell microscopy

Abstract

The application provides a method for selecting a suitable high-power image area from a low-power image for blood cell microscopy, which comprises the following steps: S1, pre-training a classification model; S2, obtaining a user-inputted blood smear low-power image; S3, preliminarily cutting the image according to the region average gray value of the inputted image; S4, segmenting the cut image; S5, classifying and judging the segmented image according to the classification model; S6, screening the image classified as normal; S7, taking the center part of the region with the most dense normal image as an initial output region; S8, calculating the number of white blood cells in the initial output region by using a watershed algorithm; if the number of white blood cells meets a set threshold, entering S9; S9, outputting the selected image region and giving the distribution coordinates of the image region in the blood smear image in S2, so as to automatically determine a suitable high-power image area.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of artificial intelligence medical examination, and particularly relates to a method for judging the uniformity of cell distribution. BACKGROUND

[0002] In blood cell microscopic examination, especially bone marrow cell microscopic examination, firstly, the low-power image of cells on the whole slide is observed by using the low-power objective (10X or 20X or 40X) of a microscope; after the information of the whole slide is understood, a suitable area is selected at a position close to the tail of the slide and where the blood sample is thin, and then the objective of the microscope is switched to a high-power objective (60X or 100X) so that the high-power objective with a small depth of field can be precisely focused on the area to obtain a clear high-power image of cells, thereby classifying and counting different types of white blood cells. Generally, more than 200 white blood cells need to be observed in the high-power image.

[0003] The work of blood cell microscopic examination in the past is completed by manual operation. In the field of artificial intelligence medical examination, we need to realize the function of automatically selecting a suitable area for shooting a high-power image from a low-power image through an algorithm. SUMMARY

[0004] In order to overcome the above technical defects, the purpose of the present application is to provide a method for selecting a suitable area for shooting a high-power image from a low-power image for blood cell microscopic examination, which comprises:

[0005] Step S1: a classification model is trained in advance through steps S1.1-S1.4, and the trained classification model is used to screen out scanning images of normal blood smears;

[0006] Step S1.1: a large number of whole slide low-power scanning picture samples of blood smears are obtained; for example, 20-40 whole slide low-power scanning picture samples of representative blood smears are obtained;

[0007] Step S1.2: the whole slide scanning picture samples in step S1.1 are divided according to a specific size (for example, a square region with a size of 600 pixels x 600 pixels, 800 pixels x 800 pixels or 1000 pixels x 1000 pixels is selected), and the divided images are divided into three types of dense, normal and sparse according to the uniformity of red blood cell distribution, and the uniformity of red blood cell distribution is used as a standard for selecting a suitable high-power image area;

[0008] Step S1.3: The three types of each segmented image, i.e., dense, normal and sparse, are separated according to the Otsu threshold method by dynamically adjusting the separation gray threshold value to separate the gray scale of the image into foreground (foreground is red blood cells and white blood cells) and background (background is blood smear background) according to the gray level, so that the gray value difference between the two parts is maximum, and the gray difference within each part is minimum. After Otsu threshold method transformation, the foreground is white and the background is black. Then, the position of the white blood cells in the foreground of the segmented image is determined according to the HSV value range of the white blood cells, and the corresponding position is changed to black, and finally changed to a binary image with white red blood cell area and black white blood cell and background area;

[0009] The main formula of the Otsu threshold method is:

[0010]

[0011]

[0012] wherein, is the inter-class variance of the foreground and the background, k is the gray scale separation threshold value of the Otsu method, k * is the optimal threshold value of the gray scale separation of the Otsu method, L is the gray scale level of the image, m G is the average gray scale value of the entire image, m1(k) and P1(k) are the average gray scale value of the pixels classified into the foreground and the probability of the pixels being classified into the foreground, respectively;

[0013] Step S1.4: The images processed in S1.2-S1.3 are made into a data set, and the data set is input into a residual network combined with a feature pyramid classification structure for classification learning to obtain a corresponding classification model file (.pt format) after training is completed;

[0014] Step S2: Obtain the low-power image of the blood smear input by the user;

[0015] Step S3: Divide the input image into several areas (for example, 25 parts) with equal areas according to the width of the input image, and calculate the average gray scale value of each area from left to right until the average gray scale value of a certain area is greater than a certain threshold value (for example, 200), then cut off the part left to the area in the input image, and the remaining part of the input image is used as the input of the next step;

[0016] Step S4: The cut image in step S3 is segmented according to the same size in step S1.2;

[0017] Step S5: The segmented image in step S4 is input into the trained classification model in step S1.4 in turn, and the probability value of the red blood cell distribution in each image judged by the classification model is obtained, so as to classify and judge the segmented image;

[0018] Step S6: Since there is a certain error in the classification model, from the perspective of algorithm stability, it is necessary to further screen the images classified as normal in the classification according to the proportion of red blood cell images and the average gray value of the images. For example, first, the images with a red blood cell proportion less than 40% are screened out, and the remaining images are sorted in ascending order of average gray value. Then, the top 25% of the sorted images are screened out to obtain a collection of images classified as normal after screening.

[0019] Step S7: Obtain the distribution position of the images classified as normal after screening in step S3. Perform connected component analysis on the distribution position. Define the region where the images classified as normal after screening are continuously distributed in the images obtained in step S3 as a connected domain. The largest area in all connected domains is the most dense region, the second largest is the second dense region, and so on. Select the center part of the same size as in step S1.2 in the most dense region as the initial output region.

[0020] Step S8: Use the watershed algorithm to calculate the number of white blood cells in the initial output region obtained in step S7. If the number of white blood cells meets the set threshold, proceed to step S9. If the number of white blood cells does not meet the set threshold, expand the range of the initial output region in step S7 until it exceeds the image range of the blood smear in step S3 or the image size is greater than a certain threshold. If the number of white blood cells meets the set threshold, proceed to step S9. If the number of white blood cells still does not meet the threshold or the cell distribution in the output range is uneven, select the second dense region of the images classified as normal, repeat this step, and so on.

[0021] For example, in the watershed algorithm, first, obtain a binary image according to the HSV value of the white blood cells. Perform dilation operation and distance transformation operation on the binary image to obtain the dilation image and distance conversion image. In the dilation operation, the kernel is 3 pixels x 3 pixels, and the iteration is 3 times. In the distance transformation, the distance is the L2 distance from the pixel with a value of 255 to the nearest pixel with a value of 0. Then, perform binary processing (threshold value is 0.1 times the maximum gray value of the distance conversion image) on the distance conversion image and mark it as foreground. Subtract the foreground part of the dilation image from the unknown region and mark the remaining part as background. Perform watershed transformation on the marked image to obtain the boundary of the white blood cells in the image and count the white blood cells.

[0022] Step S9: Output the selected image region and give its distribution coordinates in the blood smear image in step S2 to determine the region suitable for shooting high-magnification images.

[0023] After adopting the above technical scheme, compared with the prior art, the following beneficial effects are obtained:

[0024] The purpose of this invention is to provide a method for determining the uniformity of cell distribution, thereby enabling the selection of regions suitable for high-magnification imaging from low-magnification cell images. This invention can accurately select suitable regions for high-magnification imaging from low-magnification cell images, and is not affected by subjective human factors, thus greatly improving the efficiency of blood cell microscopy. Attached Figure Description

[0025] Figure 1 This is a network structure that combines residual networks with feature pyramids;

[0026] Figure 2 This is a flowchart illustrating whether the number of white blood cells in the initial output region meets a threshold.

[0027] Figure 3 This is an example image showing the suitable region for capturing high-magnification images, as provided by the algorithm. Detailed Implementation

[0028] The advantages of the present invention are further illustrated below with reference to the accompanying drawings and specific embodiments. Those skilled in the art should understand that the following detailed description is illustrative rather than restrictive and should not be construed as limiting the scope of protection of the present invention.

[0029] This embodiment provides a method for selecting a suitable region for high-magnification image capture from a low-magnification image for blood cell microscopy, comprising steps S1-S9:

[0030] Step S1: The classification model is trained in advance through steps S1.1 to S1.4. The trained classification model is used to screen out normal blood smear scan images.

[0031] Specifically, the training method for step S1 includes steps S1.1 to S1.4:

[0032] Step S1.1 Obtain a large number of low-magnification scan images of the entire blood smear.

[0033] For example, a blood smear is placed in a designated area on the microscope stage. A low-power lens is selected, the camera is activated, and the microscope automatically scans the entire blood smear, saving all the low-power images acquired for later use. The above steps are repeated to collect approximately 20-40 representative low-power images of the blood smear.

[0034] Step S1.2 divides the full-scan image sample from step S1.1 into a specific size (e.g., a square area of ​​800 pixels × 800 pixels), and divides the segmented image into three types: dense, normal, and sparse, based on the uniformity of red blood cell distribution. The uniformity of red blood cell distribution is used as the standard for selecting a suitable high-magnification image area.

[0035] Step S1.3, each of the three types of segmented images, dense, normal and sparse, is separated according to the otsu threshold method by dynamically adjusting the separation gray threshold to separate the gray scale of the image into foreground (red blood cells and white blood cells) and background (blood smear background) according to the gray level, and to maximize the difference between the foreground and the background, and to minimize the difference in gray scale within the foreground and the background. After otsu threshold method conversion, the foreground is white and the background is black; then determine the position of the white blood cells in the foreground of the segmented image according to the HSV value range of the white blood cells, and change the corresponding position to black, and finally change to a binary image with white red blood cell area and black white blood cell and background area.

[0036] The formula of the otsu threshold method is:

[0037]

[0038]

[0039] wherein, is the inter-class variance of the foreground and the background, k is the gray scale separation threshold of the otsu method, k * is the optimal threshold of the gray scale separation of the otsu method, L is the gray scale of the image, m G is the average gray scale value of the entire image, m1(k) and P1(k) are the average gray scale value of the pixels classified into the foreground and the probability of the pixels being classified into the foreground, respectively.

[0040] Step S1.4, the images processed by S1.2-S1.3 are made into a data set, and the data set is input into a classification structure (such as shown in Figure 1 ) combined with the residual network of the feature pyramid for classification learning to obtain the corresponding classification model file (.pt format) after training is completed.

[0041] Step S2: obtaining the low-power image of the blood smear input by the user.

[0042] Step S3: dividing the input image into 25 areas of equal area according to the width of the input image, and calculating the average gray scale value of each area from left to right until the average gray scale value of a certain area is greater than 200, then cutting off the part to the left of the area in the input image, and the remaining part of the input image is used as the input of the next step.

[0043] Step S4: obtaining the cut image in step S3 and dividing it according to the same size (e.g. 800 pixels x 800 pixels) in step S1.2.

[0044] Step S5: The segmented images in step S4 are sequentially input into the trained classification model in step S1.4 to obtain the probability values of the distribution of red blood cells in each image as dense, normal, and sparse determined by the classification model, thereby classifying and judging the segmented images.

[0045] Step S6: Further screening is performed on the images classified as normal in the classification judgment according to the proportion of red blood cell images and the average gray value of the images, thereby obtaining a collection of images classified as normal after screening.

[0046] Step S7: The distribution position of the images classified as normal after screening in step S3 is obtained, and a connected component analysis is performed on the distribution position. The region of the images classified as normal after screening that is continuously distributed in the images obtained in step S3 is defined as a connected component, and the largest area in all connected components is the densest region, the second largest area is the second densest region, and so on. A central part of the densest region that is the same size as that in step S1.2 (for example, 800 pixels x 800 pixels) is selected as an initial output region.

[0047] Step S8: As shown in Figure 2 , a marker-based watershed algorithm is used to calculate the number of white blood cells in the initial output region obtained in step S7. (1) If the number of white blood cells meets the set threshold value, it is further determined whether the white blood cells are uniformly distributed in the output range. (1.1) If the white blood cells are uniformly distributed, step S9 is entered. (1.2) If the white blood cells are not uniformly distributed, the second densest region of the images classified as normal is selected to repeat the step. (2) If the number of white blood cells does not meet the set threshold value, the range of the initial output region in step S7 is expanded until it exceeds the image range of the blood smear in step S3 or the image size is greater than a certain threshold value. (2.1) If the number of white blood cells meets the set threshold value and the white blood cells are uniformly distributed, step S9 is entered. (2.2) If the number of white blood cells still does not meet the threshold value or the white blood cells are not uniformly distributed in the output range, the second densest region of the images classified as normal is selected to repeat the step, and so on.

[0048] In the watershed algorithm, a binary image is first obtained according to the HSV value of the white blood cells. The binary image is subjected to dilation operation and distance transformation operation to obtain an expanded image and a distance conversion image. In the dilation operation, the kernel is 3 pixels x 3 pixels, and the iteration is performed 3 times. In the distance transformation, the distance is the L2 distance from the pixel with a value of 255 to the nearest pixel with a value of 0. Then, the distance conversion image is subjected to binary processing (the threshold value is 0.1 times the maximum gray value of the distance conversion image) and marked as foreground. The part of the expanded image that is subtracted from the foreground is marked as unknown region, and the remaining part is marked as background. The marked image is subjected to watershed transformation to obtain the boundary of the white blood cells in the image, and white blood cell counting is performed based on the boundary.

[0049] If all the dense regions have been traversed and no region satisfying the condition is found, the step is terminated.

[0050] Step S9: output the selected image region and give its distribution coordinates in the blood smear image in step S2, thereby determining the region suitable for taking high-power images, as shown in Figure 3 .

[0051] It should be noted that the embodiments of the present application have better implementation, and do not limit the present application in any form, and any skilled person in the art can change or modify the above disclosed technical content into equivalent effective embodiments, as long as it does not deviate from the technical solution of the present application, and any modification or equivalent change and modification of the above embodiments according to the technical essence of the present application still belongs to the scope of the technical solution of the present application.

Claims

1. A method for selecting a suitable area for taking a high power image from a low power image for blood cell microscopy, characterized in that, The application relates to a blood smear image screening method based on a classification model. Step S1: a classification model is trained in advance through steps S1.1-S1.4, and the trained classification model is used for screening normal blood smear scanning images; S1.1: a large number of full-scan picture samples of blood smears are acquired; S1.2: the full-scan picture samples in step S1.1 are segmented according to a specific size, and the segmented images are divided into three types of dense, normal and sparse according to the uniformity of red blood cell distribution, and the uniformity of red blood cell distribution is used as a standard for selecting a suitable high-magnification image area; S1.3: each segmented image of the three types of dense, normal and sparse is separated from the background according to the otsu threshold method by dynamically adjusting the separation gray threshold, the foreground is red blood cells and white blood cells, the background is the blood smear background, the gray value difference between the foreground and the background is maximized, and the gray difference in the foreground and the background is minimized, after the otsu threshold method transformation, the foreground is white, and the background is black; The position of the white blood cells in the foreground of the segmented image is determined according to the HSV value range of the white blood cells, the corresponding position is changed to black, and finally the binary image is changed to a white red blood cell area and a black white blood cell and background area; S1.4: the images processed through S1.2-S1.3 are made into a data set, the data set is input into a classification structure of a residual network combined with a feature pyramid for classification learning, and a corresponding classification model file after training is obtained; Step S2: a low-magnification image of a blood smear input by a user is acquired; Step S3: the input image is equally divided into several areas with equal areas according to the width of the input image, the average gray value of each area is calculated from left to right until the average gray value of a certain area is greater than a specific threshold, then the part left of the area in the input image is cropped, and the remaining part of the input image is used as the input of the next step; Step S4: the image cut in step S3 is segmented according to the same size in step S1.2; Step S5: the segmented image in step S4 is input into the trained classification model in step S1.4 in sequence, the probability value of the red blood cell distribution in each image judged by the classification model is obtained, and the segmented image is classified and judged; Step S6: the image classified as normal in the classification judgment is further screened according to the proportion of the red blood cell image and the average gray value of the image, and a collection of the screened images classified as normal is obtained; Step S7: the distribution position of the screened images classified as normal in step S3 is obtained, the distribution position is analyzed by a connected domain, the continuously distributed area of the screened images classified as normal in step S3 is defined as a connected domain, the largest area in all the connected domains is the densest area, the second largest area is the second densest area, and so on, and the center part of the densest area with the same size in step S1.2 is selected as an initial output area. Step S8: Count the number of white blood cells in the initial output region obtained in step S7 using a marker-based watershed algorithm; (1) if the number of white blood cells meets the set threshold, further determine whether the white blood cell distribution in the output range is uniform, (1.1) if the white blood cell distribution is uniform, proceed to step S9, (1.2) if the white blood cell distribution is not uniform, select the second dense region of the picture classified as normal to repeat this step; (2) if the number of white blood cells does not meet the set threshold, expand the range of the initial output region in step S7 until it exceeds the image range of the blood smear in step S3 or the image size is greater than a certain threshold, (2.1) if the number of white blood cells meets the set threshold and the white blood cell distribution is uniform, proceed to step S9, (2.2) if the threshold of the number of white blood cells is still not met or the cell distribution in the output range is not uniform, select the second dense region of the picture classified as normal to repeat this step; if all dense regions have been traversed and no region meeting the conditions is found, terminate this step; Step S9: output the selected image region and give its distribution coordinates in the blood smear image in step S2, thereby determining the region suitable for shooting high-magnification images.

2. The method for selecting a suitable area for taking a high power image from a low power image for blood cell microscopy according to claim 1, wherein, In step S1.3, the formula of the otsu threshold method is: wherein, is the inter-class variance of the foreground and background, k is the gray scale partition threshold value of the otsu method, k * is the optimal threshold value of the gray scale partition of the otsu method, L is the gray scale of the image, m G is the average gray scale value of the entire image, m1(k) and P1(k) are the average gray scale value of the pixels classified to the foreground and the probability of the pixels being classified to the foreground, respectively.

3. The method for selecting a suitable area for taking a high power image from a low power image for blood cell microscopy according to claim 1, wherein, In step S8, in the watershed algorithm, first obtain a binary image according to the HSV value of the white blood cells, perform dilation operation and distance transformation operation on the binary image to obtain a dilated image and a distance conversion image, then perform binary processing on the distance conversion image and mark it as foreground, mark the part of the dilated image minus the foreground as unknown region, and mark the remaining part as background, perform watershed transformation on the marked image to obtain the boundary of the white blood cells in the image, and count the white blood cells based on this.

Images