Cell focusing method and device, storage medium, electronic equipment and film reading machine

Abstract

The invention provides a cell focusing method and device, a storage medium, electronic equipment and a film reading machine. The method comprises the following steps: controlling a biological sample carrier and an imaging device to move horizontally relative to each other so as to enable a region of interest in a biological sample to enter a visual field of the imaging device, controlling the biological sample carrier and the imaging device to move vertically relative to each other, and controlling the imaging device to shoot a plurality of original sample images at different height positions; under the condition that the existence of the leukocyte is identified, acquiring a definition value of a cell nucleus of the leukocyte in the current original sample image, and taking the definition value as a definition value of the current original sample image; and performing focusing shooting based on the definition value of each original sample image to obtain a focus image of the target leukocyte. According to the method, the problem of unclear focusing caused by sharpness value jump points due to introduction of substances such as platelets and small dye residues can be avoided, the sharpness value jump points are reduced, the focusing is clearer, and meanwhile, the influence of the sharpness value jump points on the focusing speed of the target white blood cells is reduced.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a cell focusing method, device, storage medium, electronic equipment and film reader. Background Art

[0002] Manual microscopy is a method of using a microscope to perform microscopic analysis on human blood, body fluids, pathological cell sections, etc. This method is widely used in hospital clinical examination departments. It is a key means of directly diagnosing and identifying diseases and is the gold standard for cell morphology analysis. However, the traditional method of manually controlling the microscope to find the observation area and control the focus by doctors in the clinical examination department has the disadvantages of slow speed, unclear focus, high error rate, and high time cost. It currently does not meet the needs of clinical and clinical examination departments for rapid disease examination.

[0003] In recent years, with the rapid development of artificial intelligence technology and medical equipment, microscope autofocus technology has begun to develop rapidly, which can achieve rapid focusing of cell targets. Compared with traditional manual microscopy methods, it has greatly improved the efficiency of cell morphology analysis and solved some of the pain points of clinical doctors.

[0004] In the cell morphology analysis of the sample, a low-power microscope is generally used to find the white blood cell monolayer in the sample carrier and obtain multiple white blood cell coordinates, and then switch to a high-power microscope to retrieve the white blood cell coordinates obtained under the low-power microscope, and focus on and photograph individual white blood cells in sequence. The current autofocus technology focuses on and photographs individual white blood cells based on the clarity values ​​of all substances in the image. However, the number of small dye residues in different images obtained during the focusing process is different and relatively random; and the platelet area in the image is small. For unclear images during the focusing process, the number of platelets that can be identified in different images is also different; focusing on and photographing individual white blood cells based on the clarity values ​​of all substances in the image is prone to unclear focus caused by clarity value jumps. Summary of the invention

[0005] In order to solve the above problems, the present invention provides a cell focusing method, device, storage medium, electronic equipment and film reader.

[0006] In a first aspect, an embodiment of the present invention provides a cell focusing method, comprising:

[0007] Controlling the relative horizontal movement of the biological sample carrier and the imaging device so that the region of interest in the biological sample enters the field of view of the imaging device, and controlling the relative vertical movement of the biological sample carrier and the imaging device, and controlling the imaging device to capture multiple original sample images at different height positions;

[0008] In the case where white blood cells are identified to exist in the current original sample image, a clarity value of a cell nucleus of the white blood cells in the current original sample image is obtained as the clarity value of the current original sample image;

[0009] Based on the clarity value of each original sample image, the focus image of the target leukocyte is obtained by focusing and shooting.

[0010] In some implementations, when white blood cells are identified in the current original sample image, target white blood cells are determined from the white blood cells, and the clarity value of the nucleus of the target white blood cells in the original sample image is obtained as the clarity value of the current original sample image.

[0011] In some implementations, the method further includes: identifying whether white blood cells exist in the original sample image, comprising:

[0012] Convert the current original sample image into an HSV format image;

[0013] Extracting saturation channel information from the HSV format image, performing image segmentation based on a preset saturation threshold, and obtaining a leukocyte nucleus segmentation image;

[0014] Based on the leukocyte nucleus segmentation image, identify whether there are leukocytes in the current original sample image.

[0015] In some implementations, identifying whether white blood cells exist in the current original sample image based on the white blood cell nucleus segmentation image includes:

[0016] Perform morphological operations on the leukocyte nucleus segmentation image to obtain a morphologically processed image;

[0017] Detecting contours in the morphologically processed image to obtain contour information, wherein the contour information includes: the number of contours, the area of ​​each contour, the contour index, and the maximum contour area among all contours;

[0018] It is identified whether white blood cells exist in the current original sample image according to the contour information.

[0019] In some implementations, performing morphological operations on the leukocyte nucleus segmentation image to obtain a morphologically processed image includes:

[0020] Using the structure element of the first size, the leukocyte nucleus segmentation image is corroded a preset number of times to obtain an eroded image;

[0021] The erosion-processed image is dilated a preset number of times using the structure element of the second size to obtain a dilated image as a morphologically processed image.

[0022] In some implementations, identifying whether white blood cells exist in the current original sample image according to the contour information includes:

[0023] If the number of contours is 0, there are no white blood cells in the current original sample image;

[0024] If the number of contours is not 0, determine whether there are white blood cells in the current original sample image based on the maximum contour area;

[0025] If the maximum value of the contour area is less than the white blood cell screening threshold, there are no white blood cells in the current original sample image;

[0026] If the maximum value of the contour area is greater than or equal to the leukocyte screening threshold, leukocytes exist in the current original sample image.

[0027] In some implementations, when it is recognized that white blood cells exist in the current original sample image, determining the target white blood cells from the white blood cells includes:

[0028] Get the width and height of the current original sample image, and get the centroid coordinates of the contours whose contour area is greater than or equal to the white blood cell screening threshold; use the vertical coordinates of the centroids of these contours to subtract height / 2, and the horizontal coordinates to subtract width / 2 to obtain the offset of the centroid of each contour in the x-direction and y-direction relative to the center of the field of view. If the offset of the centroid in the x-direction and y-direction relative to the center of the field of view is less than the offset threshold, store the corresponding contour index in the first list;

[0029] Check whether the first list is empty:

[0030] If the first list is empty, it is determined that there are no white blood cells within the central area of ​​the visual field;

[0031] If the first list is not empty, it is determined that white blood cells exist within the central area of ​​the visual field, and the target white blood cells are determined from the white blood cells within the central area of ​​the visual field.

[0032] In some implementations, determining the target leukocyte from the leukocytes present within the central area of ​​the field of view includes:

[0033] If the number of contours corresponding to the first list is 1, the current contour index is recorded, and the white blood cell corresponding to the current contour index is determined as the target white blood cell;

[0034] If the number of contours corresponding to the first list is greater than 1, traverse the contour indexes in the first list and calculate the distance from the centroid of each contour to the center of view to sort them, and determine the candidate contours corresponding to the minimum two distances;

[0035] If the difference between the distances of the two candidate contours is within a preset range, determining the candidate contour with a larger area among the two candidate contours as the target white blood cell;

[0036] If the difference between the distances of the two candidate contours is not within a preset range, the candidate contour with the smaller distance is determined to be the target white blood cell.

[0037] In some implementations, when it is determined that there are no white blood cells within the central area of ​​the field of view, determining the target white blood cells outside the central area of ​​the field of view includes:

[0038] Get the width and height of the original sample image, get the centroid coordinates of the contours whose contour area is greater than or equal to the white blood cell screening threshold, use the ordinate of the centroid of these contours to subtract height / 2, and the abscissa to subtract width / 2 to obtain the offset of the centroid of each contour in the x-direction and y-direction relative to the center of the field of view. If the offset of the centroid in the x-direction and y-direction relative to the center of the field of view is greater than or equal to the offset threshold, store the corresponding contour index in the second list;

[0039] Check whether the second list is empty:

[0040] If the second list is empty, it is determined that there are no white blood cells outside the central area of ​​the visual field;

[0041] If the second list is not empty, it is determined that there are leukocytes outside the central domain of the visual field, and the target leukocytes are determined from the leukocytes outside the central domain of the visual field.

[0042] In some implementations, determining the target leukocytes from the leukocytes existing outside the central field of view includes:

[0043] If the number of contours corresponding to the second list is 1, the current contour index is recorded, and the white blood cell corresponding to the current contour index is determined as the target white blood cell;

[0044] If the number of contours corresponding to the second list is greater than 1, the contour indexes in the current second list are traversed and the distances from the centroid of each contour to the center of the field of view are sorted to determine the two candidate contours with the smallest distances. If the difference between the distances of the two candidate contours is within a preset range, the candidate contour with the larger area among the two candidate contours is determined to be the target white blood cell; if the difference between the distances of the two candidate contours is not within the preset range, the candidate contour with the smaller distance is determined to be the target white blood cell.

[0045] In some implementations, obtaining a clarity value of a nucleus of a target leukocyte in a current original sample image includes:

[0046] Traversing the contour indexes of all contours in the morphologically processed image, if the contour index is not the contour index corresponding to the target leukocyte, filling the corresponding contour in the morphologically processed image to obtain a cell nucleus mask image of the target leukocyte;

[0047] Set a zero matrix background mask image of the same size as the cell nucleus mask image of the target leukocytes;

[0048] Combine the original sample image with the cell nucleus mask image of the target leukocyte and the zero matrix background mask image to perform an AND operation to obtain a cell nucleus color image of the target leukocyte;

[0049] Based on the cell nucleus color image of the target leukocyte, a clarity value of the cell nucleus of the target leukocyte is acquired.

[0050] In some implementations, when it is identified that there are no white blood cells in the current original sample image, the clarity value of the global red blood cells in the current original sample image is obtained as the clarity value of the current original sample image.

[0051] In some implementations, a Sobel variant algorithm is used to obtain a clarity value of a cell nucleus of a white blood cell in a current original sample image, including:

[0052] Converting the color image of the cell nucleus of the leukocyte into a grayscale image format to obtain a first image;

[0053] Processing the first image by dividing it into blocks to obtain a plurality of non-overlapping first image blocks;

[0054] Extracting the gradient features of each first image block in the x direction and the y direction based on the Sobel operator;

[0055] Extracting gradient features in two diagonal directions for each first image block, and adding the gradient features in the two diagonal directions to obtain a constant gradient value;

[0056] Taking the absolute value of the gradient features of each first image block in the x direction and the y direction, and then fusing them according to a set ratio to obtain a fused image;

[0057] Calculate the sum of all non-zero elements of the gradient features in the fused image, and then divide it by the number of non-zero elements of the gradient features to obtain the average value of the gradient features in the x-direction and the y-direction as the clarity value of the current first image block;

[0058] Adding the constant gradient eigenvalue of the current first image block to the definition value, to obtain evaluation values ​​of the gradient eigenvalues ​​in the x direction, y direction and two diagonal directions of the current first image block;

[0059] Average the evaluation values of the first image blocks in the first image to obtain an average gradient value, which is used as the clarity value of the current original sample image.

[0060] In some implementations, the process of dividing the first image into blocks to obtain multiple first image blocks includes:

[0061] Divide the first image into blocks with a third size, and divide it into multiple first image blocks with the third size. During the division process, the area that does not meet the third size is filled to the third size through a filling operation to obtain the first image blocks with the third size.

[0062] In some implementations, an FFT variant algorithm is used to obtain the clarity value of global red blood cells in the current original sample image, including:

[0063] Obtain a color image of global red blood cells from the original sample image;

[0064] Convert the color image of global red blood cells into a grayscale image format to obtain a second image;

[0065] Convert the second image from unsigned integer data to single-precision type data to obtain a third image;

[0066] Divide the third image into multiple third image blocks, and randomly select two of the third image blocks;

[0067] Perform a discrete Fourier transform on the two third image blocks to convert them to the frequency domain to obtain corresponding frequency domain images;

[0068] Center the frequency domain image;

[0069] Create a matrix of L×L with element values of 255, and set the grayscale values of the pixel points on the circle with a radius of L / 2 in the matrix to 0;

[0070] Use the matrix to complete the filtering operation on the low-frequency information in the central region of the frequency domain image;

[0071] Perform an inverse Fourier transform on the frequency domain image to convert the frequency domain image into a two-channel two-dimensional image;

[0072] Calculate the amplitude two-dimensional image through the real part and the imaginary part of the two-dimensional image;

[0073] Perform a logarithm operation on each element in the amplitude two-dimensional image and obtain an intermediate image;

[0074] Multiply the intermediate image by a preset value to obtain a final image;

[0075] Calculate the average value of all non-zero grayscale value elements in the final image, and obtain the average value of the high-frequency component amplitude as the clarity value of the current original sample image.

[0076] In a second aspect, an embodiment of the present invention provides a cell focusing device, comprising:

[0077] A control module, used to control the relative horizontal movement of the biological sample carrier and the imaging device so that the region of interest in the biological sample enters the field of view of the imaging device, and to control the relative vertical movement of the biological sample carrier and the imaging device, and to control the imaging device to capture multiple original sample images at different height positions;

[0078] An acquisition module, for acquiring, when recognizing that white blood cells exist in the current original sample image, a clarity value of a cell nucleus of the white blood cells in the current original sample image as a clarity value of the current original sample image;

[0079] The focusing module is used to perform focusing shooting based on the clarity value of each original sample image to obtain a focus image of the target leukocyte.

[0080] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by at least one processor, the method described in the first aspect is implemented.

[0081] In a fourth aspect, an embodiment of the present invention provides an electronic device, comprising a memory and at least one processor, wherein the memory stores a computer program, and when the computer program is executed by the at least one processor, the method described in the first aspect is implemented.

[0082] In a fifth aspect, an embodiment of the present invention provides a film viewing machine, comprising the electronic device described in the first aspect.

[0083] Beneficial effects:

[0084] The present invention obtains the original sample images at different height positions by shooting, and when the white blood cells are identified in the original sample images, obtains the cell nucleus clarity value of the white blood cells in the original sample images as the clarity value of the current original sample image, and focuses and shoots based on the clarity values ​​of each original sample image to obtain the focus image of the target white blood cells. Since the present invention focuses and shoots based on the clarity values ​​of the cell nuclei of all or part of the white blood cells, the problem of unclear focus caused by the introduction of platelets, small dye residues and other substances causing the clarity value jump point can be avoided, the clarity value jump point can be reduced, and the focus of the target white blood cells can be clearer. At the same time, the influence of the clarity value jump point on the focus speed of the target white blood cells is reduced, and the focus speed of the target white blood cells is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and should not be regarded as limiting the scope.

[0086] Figure 1 is a flow chart of a cell focusing method provided by an embodiment of the present invention;

[0087] Figure 2 is an example of an original sample image provided by an embodiment of the present invention;

[0088] Figure 3 is an example of a morphologically processed image after corrosion and expansion provided by an embodiment of the present invention;

[0089] Figure 4 is a first clarity value variation curve diagram provided by an embodiment of the present invention;

[0090] Figure 5 is a second definition value variation curve diagram provided by an embodiment of the present invention;

[0091] Figure 6 is a third definition value variation curve diagram provided by an embodiment of the present invention;

[0092] Figure 7 It is a schematic diagram of a cell focusing device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0093] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. The components of the embodiments of the present invention generally described and represented in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

[0094] Embodiment 1

[0095] This embodiment provides a cell focusing method, such as Figure 1 As shown, it includes steps S101 to S103:

[0096] Step S101, control the biological sample carrier and the imaging device to move horizontally relative to each other so that the region of interest in the biological sample enters the field of view of the imaging device, and control the biological sample carrier and the imaging device to move vertically relative to each other, and control the imaging device to capture multiple original sample images at different height positions.

[0097] Step S102: when it is identified that white blood cells exist in the current original sample image, the clarity value of the nucleus of the white blood cells in the current original sample image is obtained as the clarity value of the current original sample image.

[0098] Step S103: focusing and photographing based on the clarity values ​​of the original sample images to obtain a focus image of the target white blood cells.

[0099] In the method of this embodiment, in the process of relative vertical movement between the biological sample carrier and the imaging device, the imaging device is controlled to capture the original sample images at different height positions. When the white blood cells are identified in the original sample image, the clarity value of the nucleus of the white blood cells in the original sample image is obtained as the clarity value of the original sample image for focusing and shooting, and a focus image of the target white blood cells is obtained. Since this embodiment focuses on and shoots the target white blood cells based on the clarity value of the nucleus of all or part of the white blood cells in the original sample image, the problem of unclear focus caused by the introduction of platelets, small dye residues and other substances causing the jump point of the clarity value can be avoided, the jump point of the clarity value can be reduced, the focus of the target white blood cells can be made clearer, and the influence of the jump point of the clarity value on the focus speed of the target white blood cells can be reduced, and the focus speed of the target white blood cells can be improved. In addition, the heights of the white blood cells and the red blood cells in the biological sample carrier are different, and the maximum value point of the focus curve of the white blood cells and the maximum value point of the focus curve of the red blood cells do not overlap. The focus on and shoot of the target white blood cells based on the clarity value of the nucleus of all or part of the white blood cells in the original sample image can improve the clarity of the target white blood cells in the focus position image.

[0100] In practice, if the target white blood cells are focused on according to the white blood cell clarity value, if the white blood cell cytoplasm is ruptured, it is easy to cause misjudgment of the segmentation of the entire white blood cells when obtaining the image clarity value, causing the clarity value to jump, thereby resulting in unclear focus on the target white blood cells. The clarity value of the white blood cell nucleus has higher stability and whether the white blood cell nucleus is clear is of great significance for analyzing cell details to identify cells.

[0101] It should be noted that, to control the relative movement between the biological sample carrier and the imaging device, the biological sample carrier can be driven to move alone (method one), or the imaging device can be driven to move alone (method two), that is, this embodiment can drive the sample carrier and the imaging device to move relative to each other through any of the two driving modes, and the relative movement includes relative vertical movement or relative horizontal movement. This embodiment is not limited to this, and for ease of understanding, this embodiment is described using method one as an example.

[0102] Since the focal height of each white blood cell nucleus is different, if focusing is performed based on the global white blood cell nucleus clarity value, it cannot be guaranteed that a single white blood cell is the clearest. Therefore, in the present embodiment, when the white blood cells are identified in the current original sample image, the target white blood cells are further determined from the white blood cells, and the clarity value of the nucleus of the target white blood cells in the original sample image is obtained, thereby obtaining a focus image with high clarity of a single target white blood cell, so that the focus of the target white blood cell is clearer. Furthermore, controlling the relative horizontal movement of the biological sample carrier and the imaging device so that the region of interest in the biological sample enters the field of view of the imaging device, and controlling the relative vertical movement of the biological sample carrier and the imaging device, and controlling the imaging device to capture multiple original sample images at different height positions, may include:

[0103] Step S101a, controlling the biological sample carrier to move horizontally relative to the imaging device so that the region of interest in the biological sample enters the field of view of the imaging device;

[0104] Step S101b, controlling the biological sample carrier to move two steps continuously in the vertical direction based on the first step length, taking an original sample image in each step, and judging the direction of the focus according to the change trend of the clarity values ​​of the two original sample images: if the change trend is an increasing trend, the movement direction is close to the focus direction, and if the change trend is a decreasing trend, the movement direction is away from the focus direction;

[0105] Step S101c, perform coarse focusing in the direction of the focus, control the biological sample carrier to move a number of steps based on the second step length or the dynamic step length in the direction close to the focus, and capture at least one original sample image for each step; determine the clarity turning curve interval where the clarity value of the captured original sample image changes from an increasing trend to a decreasing trend, and then determine the vertical height position interval corresponding to the focus as the focus fine focusing interval; wherein the dynamic step length determines the distance from the focus according to the size of the clarity value to set a large step length or a small step length for focusing;

[0106] Step S101d: In the focus fine focusing interval, the biological sample carrier is controlled to move a certain number of steps based on the third step length, and at least one original sample image is captured for each step, wherein the image with the largest clarity value is the focus image of the target leukocytes. The third step length is smaller than the second step length.

[0107] In this embodiment, the biological sample carrier is a glass slide. The biological sample can be, for example, peripheral blood, which forms a blood film on the glass slide, and the objects of cell morphology analysis are red blood cells, white blood cells, platelets, etc. in the peripheral blood. The sample can also be, for example, bone marrow, which forms a bone marrow smear on the glass slide. Bone marrow examination generally includes various types of cells at different maturation stages, such as erythrocytes, granulocytes, lymphocytes, monocytic cells, plasma cells, and others such as megakaryocytes, reticular cells, phagocytes, endothelial cells, adipocytes, etc.

[0108] In some implementations, the method of this embodiment further includes: a step of identifying whether white blood cells exist in the original sample image, which specifically includes steps S102a to S102c:

[0109] Step S102a, converting the current original sample image into an HSV format image;

[0110] Since the original cell focus image taken by the camera is unsigned character data, when preprocessing the original sample image, the camera unsigned character data needs to be converted into opencv Mat format (OpenCV is used in this embodiment), and then the original cell focus image is further converted from BGR (B: Blue G: Green R: Red) format image to HSV (H: Hue, S: Saturation, V: Brightness) format image. If OpenCV is not used), the initial format of the original cell focus image can be RGB format, etc.

[0111] Step S102b, extracting saturation channel information from the HSV format image, performing image segmentation based on a preset saturation threshold, and obtaining a leukocyte nucleus segmentation image;

[0112] Since there is an obvious difference in saturation values ​​between red blood cells and white blood cell nuclei, the segmentation effect is better using a saturation threshold. Therefore, this embodiment performs threshold segmentation by extracting saturation channel information in an HSV format image.

[0113] In practical applications, since the saturation values of white blood cell nuclei, platelets, and staining residues are generally greater than a certain saturation threshold, while the saturation values of red blood cells and white blood cell cytoplasm are generally less than this saturation threshold, the preset saturation threshold can be determined based on experimental statistics. If the saturation value of a pixel point in the s-channel is greater than the saturation threshold, the corresponding saturation value is converted to 255 (255 appears as white in the image); if the saturation value of a pixel point in the s-channel is less than the saturation threshold, the corresponding value is converted to 0 (0 appears as black in the image). Finally, a white blood cell nucleus segmentation image Thresh_img is obtained, where white represents the area where white blood cell nuclei may exist, and black represents the remaining area. In this way, the white blood cell nucleus segmentation image can completely distinguish the parts of white blood cell nuclei, platelets, and staining residues from red blood cells.

[0114] Step S102c: Based on the white blood cell nucleus segmentation image, identify whether there are white blood cells in the current original sample image.

[0115] In some implementation manners, identifying whether there are white blood cells in the original sample image based on the white blood cell nucleus segmentation image further includes steps S102c-1 to S102c-4:

[0116] Step S102c-1: Perform morphological operations on the white blood cell nucleus segmentation image Thresh_img to obtain a morphologically processed image, where the morphological operations include erosion and dilation.

[0117] Specifically, performing morphological operations on the white blood cell nucleus segmentation image to obtain a morphologically processed image includes:

[0118] Perform erosion operations on the white blood cell nucleus segmentation image Thresh_img a preset number of times using a structural element of the first size to obtain an eroded image.

[0119] Perform dilation operations on the eroded image ImgA a preset number of times using a structural element of the second size to obtain a dilated image as the morphologically processed image.

[0120] In an example, the first size is 3×3 and the second size is 5×5. The 3×3 corrosion morphological structure element is used to perform two corrosion operations on the white blood cell nucleus segmentation image Thresh_img. That is, all the pixels of the white blood cell nucleus segmentation image Thresh_img are judged from left to right and from top to bottom to see if they need to be eliminated. Specifically, a certain pixel point on the white blood cell nucleus segmentation image Thresh_img is taken as the base point to judge whether the values ​​of the eight surrounding pixels are all 255. If not, the pixel value of the base point is changed to 0 (if so, the current base point remains unchanged). After two consecutive corrosion operations, some noise spots, small dye residues and normal platelets on the white blood cell nucleus segmentation image Thresh_img can be filtered out, and the corrosion-processed image ImgA is obtained. The eroded image ImgA is expanded three times using a 5×5 structural element. Specifically, a certain pixel on the eroded image ImgA is taken as the base point to determine whether there is at least one 255 in the eight surrounding pixel values. If so, the pixel value of the base point is changed to 255. The purpose is to compensate for the loss of the white blood cell nucleus caused by corrosion. After the expansion operation, the morphologically processed image ImgB is obtained. The mask acquisition of one or more white blood cell nuclei can be completed using this image, that is, all white blood cell nuclear substances and background are set to different pixel point values ​​to distinguish the two. Figure 2 and Figure 3 The original color sample image and the morphologically processed image (grayscale image) after corrosion and expansion are shown respectively.

[0121] Step S102c-2: Detect contours in the morphologically processed image to obtain contour information, which includes: the number of contours, the area of ​​each contour, the contour index, and the maximum contour area among all contours.

[0122] Step S102c-3, determining whether there are white blood cells in the current original sample image according to the contour information;

[0123] In some implementations, determining whether white blood cells exist in the current original sample according to the profile information further includes:

[0124] If the number of contours is 0, there are no white blood cells in the current original sample image;

[0125] If the number of contours is not 0, whether there are white blood cells in the current original sample image is determined based on the maximum value of the contour area: if the maximum value of the contour area is less than the white blood cell screening threshold, there are no white blood cells in the current original sample image; if the maximum value of the contour area is greater than or equal to the white blood cell screening threshold, there are white blood cells in the current original sample image.

[0126] When the number of contours is 0, it indicates that no white blood cells are detected at the current focus position, so the target white blood cells are directly focused and photographed according to the clarity value of the global red blood cells in the original sample image. If the number of contours is not 0, the maximum contour area and contour index are obtained.

[0127] The white blood cell screening threshold can be calculated through a large number of cell data statistical experiments. This value can screen out white blood cells. The maximum value of the contour area is compared with the white blood cell screening threshold. If it is less than the threshold, it is determined that there are no white blood cells in the image, and the clarity of the global red blood cells is calculated.

[0128] In one example, contour detection is performed on the morphologically processed image ImgB, and the contours of the area with a pixel value of 255 (white contours) are detected and the number of contours and the area of ​​each contour are counted. The outer contour may contain a smaller inner contour. There is a nucleolus inside the nucleus of a normal white blood cell. Saturation threshold segmentation is used, and the nucleolus will be identified and displayed as an inner contour. However, when identifying the target white blood cells, if the inner contour is counted at the same time, if the focus target selected later is the inner contour, then the outer contour will be completely filled, and the selected focus target will be filled and disappear. Therefore, the contour detection only counts the outer contour but not the inner contour. After the focus target is selected later, all contours except the focus target will be filled with black. Among them, when the coordinates of the inner and outer contours are obtained, if it is determined that the coordinates of contour A are within the coordinates of contour B, contour A is determined to be the inner contour.

[0129] When performing area statistics on the maximum values ​​of each contour area, the contour area maximum value and the contour index corresponding to the contour area maximum value are saved, where the contour index is the contour number. When looking for the contour area maximum value, the contours can be numbered from left to right starting from the upper left corner of the image to determine the contour area maximum value and its number.

[0130] The position of the target leukocyte is derived from the coordinates obtained under a low-power microscope. This application locks and focuses on the target leukocyte according to the coordinates of the target leukocyte under a high-power microscope. Under normal circumstances, the target leukocyte is located in the center of the field of view and is closer to the center of the field of view than other leukocytes. Therefore, when there are leukocytes in the current original sample image, the target leukocyte is determined from the leukocytes. First, the target leukocyte is determined within the center of the field of view, including:

[0131] Get the width and height of the current original sample image, and get the centroid coordinates of the contours whose contour area is greater than or equal to the white blood cell screening threshold; use the vertical coordinates of the centroids of these contours to subtract height / 2, and the horizontal coordinates to subtract width / 2 to obtain the offset of the centroid of each contour in the x-direction and y-direction relative to the center of the field of view. If the offset of the centroid in the x-direction and y-direction relative to the center of the field of view is less than the offset threshold, store the corresponding contour index in the first list;

[0132] Check whether the first list is empty:

[0133] If the first list is empty, it is determined that there are no white blood cells within the central area of ​​the visual field;

[0134] If the first list is not empty, it is determined that white blood cells exist within the central area of ​​the visual field, and the target white blood cells are determined from the white blood cells within the central area of ​​the visual field.

[0135] in,

[0136] Identify target leukocytes from the leukocytes present in the central area of ​​the field of view, including:

[0137] If the number of contours corresponding to the first list is 1, the current contour index is recorded, and the white blood cell corresponding to the current contour index is determined as the target white blood cell;

[0138] If the number of contours corresponding to the first list is greater than 1, traverse the contour indexes in the first list and calculate the distance from the centroid of each contour to the center of view to sort them, and determine the candidate contours corresponding to the minimum two distances;

[0139] If the difference between the distances of the two candidate contours is within a preset range, determining the candidate contour with a larger area among the two candidate contours as the target white blood cell;

[0140] If the difference between the distances of the two candidate contours is not within a preset range, the candidate contour with the smaller distance is determined to be the target white blood cell.

[0141] In a specific example, all contours are traversed and it is determined whether the area of ​​each contour satisfies or exceeds the white blood cell screening threshold. If so, the centroid coordinates (CenterX, CenterY) of the contour are obtained, where the origin of the centroid coordinates is the upper left corner of the image, and the width and height of the original BGR image are obtained at the same time, and the horizontal and vertical coordinates of the centroid are respectively subtracted by width / 2 and height / 2 to obtain the offset of the centroid relative to the center of the field of view. If the offsets in the x and y directions relative to the center of the field of view are both less than the offset threshold (the threshold can be obtained by converting the relevant mechanical error), the corresponding contour index is stored in the first list ListA as a candidate for the target white blood cell. The number of contours that meet the conditions in the first list ListA is determined. If the number of contours that meet the conditions is 0, it is determined that there are no white blood cells in the central domain of the field of view, and it is directly determined whether there are white blood cells outside the central domain of the field of view, and the target white blood cells are determined based on the contour information. Focus shooting is performed based on the clarity of the nucleus of the target white blood cells to obtain the target focus image. If the number of contours that meet the conditions is 1, the current contour index is recorded. If the number of contours that meet the conditions is greater than 1, the contour indexes that meet the conditions are traversed and the distance from the contour centroid to the center of the field of view is calculated and sorted. The target closest to the center of the field of view and the second closest target are found. If the difference between the center distance of the closest target and the second closest target is within 5 pixels, the area size of the two is calculated. The larger one is used as the target reference and the clarity value is calculated based on the nucleoplasm of the target and the current contour index is recorded. If the difference between the center distance of the closest target and the second closest target is within 5 pixels, the cell coordinates obtained under the low power microscope correspond to two white blood cells. At this time, one of them can be selected as the target white blood cell. Among them, the larger the contour area, the more stable the calculation clarity. If the difference between the center distance of the closest target and the second closest target is not within 5 pixels, the target contour closest to the center of the field of view is used as the target white blood cell for the clarity calculation of the nucleus.

[0142] Normally, the target leukocyte is located in the center of the visual field, but due to mechanical error, the target leukocyte may deviate from the center of the visual field and be located outside the center of the visual field. In order to avoid mechanical error, this embodiment also provides a target selection strategy when the target deviates from the center of the visual field.

[0143] When it is determined that there are no leukocytes within the central area of ​​the visual field, determining the target leukocytes outside the central area of ​​the visual field may include:

[0144] Get the width and height of the original sample image, get the centroid coordinates of the contours whose contour area is greater than or equal to the white blood cell screening threshold, use the vertical coordinates of the centroids of these contours to subtract height / 2, and the horizontal coordinates to subtract width / 2 to obtain the offset of the centroid of each contour in the x-direction and y-direction relative to the center of the field of view. If the offset of the centroid in the x-direction and y-direction relative to the center of the field of view is greater than or equal to the offset threshold, store the corresponding contour index in the second list;

[0145] Check whether the second list is empty:

[0146] If the second list is empty, it is determined that there are no white blood cells outside the central area of ​​the visual field;

[0147] If the second list is not empty, it is determined that there are leukocytes outside the central area of ​​the visual field, and the target leukocytes are determined from the leukocytes outside the central area of ​​the visual field.

[0148] It should be noted that, in practical applications, since the existence of white blood cells in the original sample image has been identified before, the second list will not be empty in principle.

[0149] Further, determining the target leukocytes from the leukocytes existing outside the central area of ​​the field of view may include:

[0150] If the number of contours corresponding to the second list is 1, the current contour index is recorded, and the white blood cell corresponding to the current contour index is determined as the target white blood cell;

[0151] If the number of contours corresponding to the second list is greater than 1, the contour indexes in the current second list are traversed and the distances from the centroid of each contour to the center of the field of view are sorted to determine the two candidate contours with the smallest distances. If the difference in the distances between the two candidate contours is within a preset range, the candidate contour with the larger area among the two candidate contours is determined to be the target white blood cell; if the difference in the distances between the two candidate contours is not within the preset range, the candidate contour with the smaller distance is determined to be the target white blood cell.

[0152] In a specific example, if the number of contours that meet the conditions in the first list ListA is 0, it is determined that there are no white blood cells in the central domain of the visual field, and it is necessary to further determine whether there are white blood cells outside the central domain of the visual field. The following scheme is adopted: traverse all contours and determine whether the contour area meets or exceeds the white blood cell screening threshold. If it meets, obtain the centroid coordinates (CenterX, CenterY) of the contour and simultaneously obtain the width and height of the original BGR image, and use the horizontal and vertical coordinates of the centroid to subtract width / 2 and height / 2 respectively to obtain the offset of the centroid relative to the center of the visual field. If the offsets relative to the center in the x and y directions are both greater than or equal to the offset threshold (the threshold can be obtained based on the conversion of relevant mechanical errors), the corresponding contour index is stored in the second list ListB. If the second list ListB is empty, the global red blood cell clarity calculation is directly selected. If the number of contours in the second list ListB is 1, the current contour index is recorded; if the number of contours that meet the conditions is greater than 1, the contour index that currently meets the conditions is traversed and the distance from the contour centroid to the center of the visual field is calculated and the distance is sorted. Find the target closest to the center. If the distance between the two targets is within 5 pixels, calculate the size of their contour areas. The target with the larger contour area is selected and the clarity value is calculated based on the nucleus and cytoplasm of the target and the current contour index is recorded.

[0153] In some implementations, obtaining the clarity value of the nucleus of the target white blood cell in the current original sample image may include steps S102d-1 to S102d-4:

[0154] Step S102d-1, traversing the contour indexes of all contours in the morphologically processed image, if the contour index is not the contour index corresponding to the target white blood cell, filling the corresponding contour in the morphologically processed image to obtain a cell nucleus mask image of the target white blood cell;

[0155] Step S102d-2, setting a zero matrix background mask image of the same size as the cell nucleus mask image of the target leukocyte;

[0156] Step S102d-3, performing an AND operation on the original sample image, the cell nucleus mask image of the target leukocyte, and the zero matrix background mask image to obtain a cell nucleus color image of the target leukocyte;

[0157] Step S102d-4: based on the color image of the nucleus of the target leukocyte, obtain the clarity value of the nucleus of the target leukocyte.

[0158] Before calculating the clarity, the nucleus component of the target white blood cells needs to be extracted. In a specific example, all contour indexes are traversed. If the index is not the target white blood cell, the corresponding contour is filled in the morphologically processed image ImgB, that is, the pixel value in the non-target contour is converted from 255 to 0, so that the contours other than the target contour become the same black as the background, and finally the nucleus mask image ImgC of the focused target is obtained, and then the zero matrix image background mask image ImgD with the same image size as ImgC is set, and then the original image is combined with ImgC and ImgD to perform an AND operation to obtain the nucleus color image ImgE, that is, to obtain a BGR color image of the nucleus component of the target white blood cells only. ImgE is further used to calculate the clarity, and the focus shooting of the target white blood cells is completed by calculating the clarity of the white blood cell nucleus.

[0159] In this embodiment, when it is identified that there are no white blood cells in the current original sample image, the clarity value of the global red blood cells in the current original sample image is obtained as the clarity value of the current original sample image, and the focus is taken according to the clarity value of each original sample image to obtain a focus image of the target white blood cells.

[0160] The number of small dye residues in different images obtained during the focusing process is different and relatively random; and the area of ​​platelets in the image is small. For unclear images during the focusing process, the number of platelets that can be identified in different images is also different; platelets and other components participating in the clarity calculation will easily cause jump points. Choosing to perform clarity calculation on the global red blood cells in the image can improve the accuracy of the clarity calculation of the focus position. In addition, the red blood cells in the target image are more numerous and larger in area than other components such as platelets. Choosing to perform clarity calculation on red blood cells will obtain a larger clarity value, and the clarity value changes significantly, which is convenient for faster judgment of the focus direction and more stable calculation.

[0161] Since the logic of red blood cells and white blood cells is different, red blood cells do not have too much texture information but are more about overall clarity, and removing the inherent diffuse aperture around red blood cells, so more consideration is given to measuring whether the high-frequency information of the remaining image components of red blood cells after removing some diffuse aperture components meets the clarity condition. White blood cells focus on the fact that the white blood cell nucleus has more texture and particles, so more attention is paid to the amount of gradient components to evaluate clarity. In this embodiment, different clarity functions are selected for the calculation of the clarity value of the white blood cell nucleus and the calculation of the clarity value of the global red blood cells.

[0162] In this embodiment, one of the Sobel variant algorithm, information entropy algorithm, energy function, L1 / L2 regularization, Scharr algorithm, and Laplace algorithm can be used to calculate the clarity of the nucleus of the white blood cells. In this embodiment, one of the FFT variant algorithm, information entropy algorithm, energy function, L1 / L2 regularization, Scharr algorithm, and Laplace algorithm can be used to calculate the clarity of the global red blood cells.

[0163] Specifically, this embodiment adopts the Sobel variant algorithm to obtain the clarity value of the nucleus of the white blood cells in the current original sample image, which may include:

[0164] Converting the color image of the cell nucleus of the leukocyte into a grayscale image format to obtain a first image;

[0165] Processing the first image by dividing it into blocks to obtain a plurality of non-overlapping first image blocks;

[0166] Extracting the gradient features of each first image block in the x direction and the y direction based on the Sobel operator;

[0167] Extracting gradient features in two diagonal directions for each first image block, and adding the gradient features in the two diagonal directions to obtain a constant gradient value;

[0168] Taking the absolute value of the gradient features of each first image block in the x direction and the y direction, and then fusing them according to a set ratio to obtain a fused image;

[0169] Calculate the sum of all non-zero elements of the gradient features in the fused image, and then divide it by the number of non-zero elements of the gradient features to obtain the average value of the gradient features in the x-direction and the y-direction as the clarity value of the current first image block;

[0170] Adding the constant gradient feature value and the definition value of the current first image block to obtain evaluation values ​​of the gradient feature values ​​in the x direction, y direction and two diagonal directions of the current first image block; and

[0171] The evaluation values ​​of the first image blocks in the first image are averaged to obtain an average gradient value, which is used as the clarity value of the current original sample image.

[0172] The first image is divided into blocks to obtain a plurality of first image blocks, including:

[0173] The first image is divided into multiple first image blocks of the third size by block processing according to the third size. During the division process, the area that does not meet the third size is filled to the third size by a filling operation to obtain the first image block of the third size.

[0174] In a specific example, the process of using the Sobel variant algorithm to obtain the clarity value of the global red blood cells in the current original sample image is as follows:

[0175] Step A: converting the color image of the nucleus of white blood cells from the BGR image format to the grayscale image format to obtain the first image FMAT1.

[0176] Step B: Since the Sobel operator is used for detailed texture analysis of white blood cell nuclei (it is not possible to analyze the trend of the nucleus from the horizontal and vertical directions alone, as the span of the nucleus is quite special and more texture change details can be extracted), it is necessary to perform multi-directional angle feature extraction. The specific steps are as follows:

[0177] In the first step, the first image FMAT1 is divided into multiple non-overlapping 3×3 first image blocks according to the image length and width information. If the image length and width do not meet the conditions for extracting 3×3, padding is required. For example, if the image size is 5×5, then there is a difference of one column of small blocks when extracting non-overlapping 3×3 image blocks in the horizontal direction. Therefore, padding operation is performed on the right side to change the image size to 6×6, and the padding value is 0.

[0178] The second step is to extract the image: Based on the Sobel operator, the gradient information of the image in the x and y directions is extracted respectively. The specific calculation formula of the Sobel operator is as follows:

[0179] X direction: Gx = [-1, 0, +1; -2, 0, +2; -1, 0, +1] * A,

[0180] Y direction: Gy=[-1,-2,-1;0,0,0;+1,+2,+1]*A,

[0181] Among them, Gx and Gy represent the gradient feature information in the horizontal direction and the vertical direction extracted from the image A respectively, and A represents the gray value of FMAT1.

[0182] The third step is to extract the gradient feature information in the horizontal and vertical directions from the first 3×3 image block.

[0183] Step 4: Extract the gradient information in the dual diagonal direction for the first 3×3 image block: subtract the grayscale value of the lower right corner from the grayscale value of the upper left corner to extract the left diagonal gradient feature information, and then subtract the grayscale value of the lower left corner from the grayscale value of the upper right corner to extract the right diagonal feature information. Add the left and right diagonal gradient values ​​to obtain the constant gradient value Cbias.

[0184] Step C, taking the absolute value of Gx and Gy to prevent some absolute values ​​from being negative and not satisfying the image grayscale value range.

[0185] Step D: Gx and Gy are fused into image Gimg at a ratio of 50%.

[0186] Step E, calculate the sum of all non-zero elements of the gradient features in the fused image Gimg, and divide it by the number of non-zero elements to obtain the average value of the gradients in the x and y directions to obtain the clarity value Cclear.

[0187] Step F: Add Cclear and Cbias to obtain the evaluation value Di of the gradient value in the x, y and diagonal directions of the current first image block.

[0188] Step G, adding the Di value of each first image block and dividing it by the total number of first image blocks to obtain the average gradient value Fclear, thereby completing the clarity analysis of the cell nucleus.

[0189] Compared with the traditional Sobel algorithm, which only focuses on the gradient eigenvalues ​​in the x and y directions, a clarity value calculation in three directions is designed, namely: the gradient eigenvalue statistics in the x, y and image diagonal directions. In this way, the clarity transformation of the cell nucleus texture can be reflected as much as possible.

[0190] In this embodiment, the FFT variant algorithm is used to obtain the global red blood cell clarity value in the current original sample image, which may include:

[0191] Obtaining a global red blood cell color image based on the original sample image;

[0192] Converting the color image of the global red blood cells into a grayscale image format to obtain a second image;

[0193] Convert the second image from unsigned integer data to single-precision data to obtain a third image;

[0194] Dividing the third image into a plurality of third image blocks, and randomly selecting two of the third image blocks;

[0195] Performing discrete Fourier transform on the two third image blocks and converting them into frequency domain to obtain corresponding frequency domain images;

[0196] Center the frequency domain image;

[0197] Create an L×L matrix with an element value of 255, and set the grayscale value of the pixel point where the circle with a radius of L / 2 in the matrix is ​​located to 0;

[0198] Use this matrix to complete the filtering operation of low-frequency information in the central area of ​​the frequency domain image;

[0199] Perform inverse Fourier transform on the frequency domain image to convert the frequency domain image into a dual-channel two-dimensional image;

[0200] Calculate the amplitude two-dimensional image by the real part and imaginary part of the two-dimensional image;

[0201] Perform a logarithm operation on each element in the amplitude two-dimensional image and obtain an intermediate image;

[0202] Multiplying the intermediate image by a preset value to obtain a final image; and

[0203] Calculate the average value of all non-zero grayscale elements in the final image, and obtain the average value of the high-frequency component amplitude as the clarity value of the current original sample image.

[0204] In a specific example, the process of calculating the clarity of global red blood cells using the FFT variant algorithm is as follows:

[0205] Step a, obtaining a color image of global red blood cells according to the original sample image, converting the color image of global red blood cells into a grayscale image format, and obtaining a second image FIMG2;

[0206] The specific process of obtaining a global red blood cell nucleus color image includes:

[0207] Step a1, converting the original sample image from the BGR image format to the HSV image format, obtaining the S channel information therein, so as to be used later to distinguish red blood cells, platelets and dye residues.

[0208] Step a2 uses the S saturation channel obtained from the HSV image to perform threshold segmentation. The threshold setting used for segmentation can be obtained based on experimental statistics. If the saturation value of the S channel pixel is greater than the threshold, the corresponding value is converted to 255 (appearing as white on the image). If the number of S channel pixel points is less than the threshold, the corresponding value is converted to 0 (appearing as black on the image). Finally, the red blood cell segmentation image Thresh_img is obtained;

[0209] Step a3, invert Thresh_img, that is, convert 0 to 255, and 255 to 0, to obtain image Thresh_img2;

[0210] Step a4, using the image Thresh_img2, the zero matrix image background mask image, and the original sample image, extract the red blood cell color image final_img excluding platelets and dye residues.

[0211] Step b, converting FIMG2 from an 8-bit unsigned integer to a 32-bit single-precision type to obtain a third image FIMG3;

[0212] Step c, select FIMG2 in blocks, the width and height of the original BGR image are W and H respectively, randomly select two non-overlapping third image blocks of size W / 4*H / 4 on the image, perform discrete Fourier transform on these two image blocks and convert them into frequency domain to obtain frequency domain image;

[0213] The specific calculation formula is as follows:

[0214]

[0215] Where, F(u,v) represents the frequency domain image, u represents the real part, v represents the imaginary part, M represents the width of the image, N represents the height of the image, x, y represent the x-axis and y-axis coordinates of the time domain image, f(x, y) represents the time domain image, and j represents the imaginary unit;

[0216] In the FFT variant algorithm, the block selection of FIMG2 can effectively improve the operation speed.

[0217] Step d, centering the frequency domain image, that is, moving the low-frequency area of ​​the image to the central area of ​​the image to facilitate filtering operations;

[0218] Step e, design a matrix with a length and width of 64 (L=64) and an element value of 255, and set the pixel value of the pixel point where the circle with a radius of 32 in the matrix is ​​located to 0;

[0219] Step f, filtering the matrix within the 64×64 range of the central area of ​​the frequency domain image, that is, using the matrix designed in step e to complete the filtering operation of the low-frequency information in the central area. The purpose of the filtering operation is to remove the diffuse aperture around the red blood cells to improve the clarity of the red blood cells in a single image, and to reduce the clarity jump point in the process of determining the focus position;

[0220] Step g, performing inverse Fourier transform on the frequency domain image, converting the image into a two-channel two-dimensional image, namely, a real part and an imaginary part, and completing the calculation of the amplitude two-dimensional image through the real part and the imaginary part;

[0221] Step h, performing a logarithm operation on each element of the amplitude two-dimensional image, and obtaining an intermediate image Res1 with a base of 2;

[0222] Step i, multiplying the intermediate image Res1 by a preset value 20 to obtain a final image Res2;

[0223] Step j, calculate the sum of all non-zero elements in the final image Res2, and divide it by the number of non-zero elements to obtain the average value of the high-frequency component amplitude, and complete the calculation of clarity.

[0224] Compared with the traditional FFT algorithm, the FFT variant algorithm in this embodiment performs a window setting filtering strategy to remove the aperture effect inherent in the outer edge of the red blood cells.

[0225] It should be understood that since the focusing process is from far away from the focus to near the focus and then far away from the focus again, there may be the following two situations: one is that there are white blood cells in the target field of view, but the white blood cell nucleus (far away from the focus) may not be detected and located under certain clarity; the other is that there are no white blood cells in the target field of view (large dye residues are mistakenly identified as white blood cells under low-power microscope, and their coordinate information is obtained. Under high-power microscope, the target is located according to the coordinates obtained under low-power microscope. The large dye residue may have flowed out of the field of view, so there will be no white blood cells in the target field of view).

[0226] In some implementations, during the focusing process, the clarity values ​​of the multiple original sample images may all be obtained based on the clarity value of the target white blood cell nucleus, or may all be obtained based on the clarity value of the global red blood cells, or may be both. Therefore, the method of this embodiment also includes:

[0227] If the clarity values ​​of the multiple original sample images are all obtained based on the clarity value of the target leukocyte nucleus, the focus position of the target leukocyte is determined based on steps S101a to S101d to obtain a focus image of the target leukocyte;

[0228] If the clarity values ​​of the multiple original sample images are all obtained based on the clarity value of the global red blood cells, the focus position of the target white blood cells is determined based on steps S101a to S101d to obtain a focus image of the target white blood cells;

[0229] If the clarity values ​​of multiple original sample images include clarity values ​​obtained based on target white blood cells and clarity values ​​obtained based on global red blood cells, the clarity value change curve corresponding to the clarity value of the global red blood cells is removed, and the focus position of the target white blood cells is determined based on steps S101a to S101d to obtain a focus image of the target white blood cells.

[0230] Figure 4 A first clarity value variation curve diagram is provided for obtaining the clarity value of the original sample image based on the global red blood cell clarity value and then based on the cell nucleus clarity value of the white blood cells; Figure 5 A second clarity value variation curve diagram is provided, each of which is based on the clarity value of the cell nucleus of the target leukocyte to obtain the clarity value of the original sample image. Figure 6 A third clarity value change curve diagram is provided, which is a clarity value of the original sample image obtained based on the clarity value of the global red blood cells. The ordinate represents the clarity value, and the abscissa represents the sequence number of the original sample image. It can be seen that Figure 4The global red blood cell clarity value and the white blood cell nucleus clarity value are quite different in magnitude. In this case, the curve of the global red blood cell clarity value is discarded, and the focus position is determined based on the change trend of the target white blood cell nucleus clarity value. The first point of the original sample image clarity value determined by the target white blood cell nucleus clarity value is taken as the starting point, and the focus position of the target white blood cell is determined based on steps S101a to S101d, thereby obtaining a focus image of the target white blood cell. Figure 5 The clarity value of the nucleus of the target leukocyte in the image shows a certain trend of change. In this case, the focus position is determined according to the clarity value of the nucleus of the target leukocyte. It should be understood that although Figure 5 There may be individual clarity value jump points in the image, but the change trend of the clarity value of the nucleus of the target white blood cells can still be accurately determined from the overall curve. Figure 6 The clarity value of the global red blood cells also shows a certain trend of change. In this case, the focus position is determined according to the clarity value of the global red blood cells.

[0231] The method of this embodiment speeds up the focusing speed of target leukocytes in two aspects:

[0232] On the one hand, focusing and photographing the target leukocytes according to the clarity values ​​of all leukocyte nuclei in the original sample image can avoid the introduction of platelets, small dye residues and other substances that cause clarity value jumps, avoid affecting the focusing speed due to clarity value jumps, and improve the focusing speed;

[0233] On the other hand, the calculation speed of the image clarity value is improved, overcoming the problem of low calculation speed of the clarity values ​​of all substances in the image and instead calculating the clarity values ​​of some substances in the image, thereby improving the calculation speed of the clarity value.

[0234] In general, the cell focusing method of this embodiment has a faster and clearer real-time focusing speed, and can be applied to the focusing of cells in peripheral blood smears or bone marrow smears at various magnifications such as 20x, 40x, 50x, and 100x.

[0235] Embodiment 2

[0236] This embodiment provides a cell focusing device, such as Figure 7 As shown, including:

[0237] The control module 201 is used to control the relative horizontal movement of the biological sample carrier and the imaging device so that the region of interest in the biological sample enters the field of view of the imaging device, and to control the relative vertical movement of the biological sample carrier and the imaging device, and to control the imaging device to capture multiple original sample images at different height positions;

[0238] The acquisition module 202 is used to acquire the clarity value of the nucleus of the white blood cells in the current original sample image as the clarity value of the current original sample image when the white blood cells are identified in the current original sample image;

[0239] The focusing module 203 is used to perform focusing shooting based on the clarity value of each original sample image to obtain a focus image of the target white blood cells.

[0240] The specific implementation methods and examples of each module in the device of this embodiment are detailed in Example 1, which will not be repeated here, and this embodiment at least has all the beneficial effects of Example 1.

[0241] Embodiment 3

[0242] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by at least one processor, the method of the first embodiment is implemented.

[0243] Embodiment 4

[0244] This embodiment provides an electronic device, including a memory and at least one processor. The memory stores a computer program, and when the computer program is executed by the at least one processor, the method of the first embodiment is implemented.

[0245] The computer-readable storage medium may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0246] The processor can be an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller unit (MCU), a microprocessor or other electronic components. In the several embodiments provided in the embodiments of the present invention, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device and method embodiments described above are merely illustrative.

[0247] Embodiment 5

[0248] This embodiment provides a film viewing machine, including the electronic device of the fourth embodiment.

[0249] It should be noted that, in this article, the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. The terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0250] Although the embodiments disclosed in the present invention are as above, the above contents are only embodiments adopted for facilitating the understanding of the present invention and are not intended to limit the present invention. Any technician in the technical field to which the present invention belongs can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in the present invention, but the patent protection scope of the present invention shall still be subject to the scope defined in the attached claims.

Claims

1. A cell focusing method, It is characterized in that include: Controlling the relative horizontal movement of the biological sample carrier and the imaging device so that the region of interest in the biological sample enters the field of view of the imaging device, and controlling the relative vertical movement of the biological sample carrier and the imaging device, and controlling the imaging device to capture multiple original sample images at different height positions; In the case where white blood cells are identified to exist in the current original sample image, a clarity value of a cell nucleus of the white blood cells in the current original sample image is obtained as the clarity value of the current original sample image; Based on the clarity value of each original sample image, the focus image of the target leukocyte is obtained by focusing and shooting.

2. The cell focusing method according to claim 1, It is characterized in that In the case where white blood cells are identified in the current original sample image, target white blood cells are determined from the white blood cells, and the clarity value of the nucleus of the target white blood cells in the original sample image is obtained as the clarity value of the current original sample image.

3. The cell focusing method according to claim 2, It is characterized in that Also includes: Identify whether white blood cells exist in the current raw sample image, including: Convert the current original sample image into an HSV format image; Extracting saturation channel information from the HSV format image, performing image segmentation based on a preset saturation threshold, and obtaining a leukocyte nucleus segmentation image; Based on the leukocyte nucleus segmentation image, identify whether there are leukocytes in the current original sample image.

4. The cell focusing method according to claim 3, It is characterized in that Based on the leukocyte nucleus segmentation image, identify whether there are leukocytes in the current original sample image, including: Perform morphological operations on the leukocyte nucleus segmentation image to obtain a morphologically processed image; Detecting contours in the morphologically processed image to obtain contour information, wherein the contour information includes: the number of contours, the area of ​​each contour, the contour index, and the maximum contour area among all contours; It is identified whether white blood cells exist in the current original sample image according to the contour information.

5. The cell focusing method according to claim 4, It is characterized in that The step of performing morphological operations on the leukocyte nucleus segmentation image to obtain a morphologically processed image includes: Using the structure element of the first size, the leukocyte nucleus segmentation image is corroded a preset number of times to obtain an eroded image; The erosion-processed image is dilated a preset number of times using the structure element of the second size to obtain a dilated image as a morphologically processed image.

6. The cell focusing method according to claim 4, It is characterized in that Identifying whether there are white blood cells in the current original sample image according to the contour information includes: If the number of contours is 0, there are no white blood cells in the current original sample image; If the number of contours is not 0, determine whether there are white blood cells in the current original sample image based on the maximum contour area; If the maximum value of the contour area is less than the white blood cell screening threshold, there are no white blood cells in the current original sample image; If the maximum value of the contour area is greater than or equal to the leukocyte screening threshold, leukocytes exist in the current original sample image.

7. The cell focusing method according to claim 4, It is characterized in that When it is recognized that white blood cells exist in the current original sample image, target white blood cells are determined from the white blood cells, including: Get the width and height of the current original sample image, and get the centroid coordinates of the contours whose contour area is greater than or equal to the white blood cell screening threshold; use the vertical coordinates of the centroids of these contours to subtract height / 2, and the horizontal coordinates to subtract width / 2 to obtain the offset of the centroid of each contour in the x-direction and y-direction relative to the center of the field of view. If the offset of the centroid in the x-direction and y-direction relative to the center of the field of view is less than the offset threshold, store the corresponding contour index in the first list; Check whether the first list is empty: If the first list is empty, it is determined that there are no white blood cells within the central area of ​​the visual field; If the first list is not empty, it is determined that white blood cells exist within the central area of ​​the visual field, and the target white blood cells are determined from the white blood cells within the central area of ​​the visual field.

8. The cell focusing method according to claim 7, It is characterized in that Identify target leukocytes from the leukocytes present in the central area of ​​the field of view, including: If the number of contours corresponding to the first list is 1, the current contour index is recorded, and the white blood cell corresponding to the current contour index is determined as the target white blood cell; If the number of contours corresponding to the first list is greater than 1, traverse the contour indexes in the first list and calculate the distance from the centroid of each contour to the center of view to sort them, and determine the candidate contours corresponding to the minimum two distances; If the difference between the distances of the two candidate contours is within a preset range, determining the candidate contour with a larger area among the two candidate contours as the target white blood cell; If the difference between the distances of the two candidate contours is not within a preset range, the candidate contour with the smaller distance is determined to be the target white blood cell.

9. The cell focusing method according to claim 7, It is characterized in that When it is determined that there are no leukocytes within the central area of ​​the visual field, target leukocytes are determined outside the central area of ​​the visual field, including: Get the width and height of the original sample image, get the centroid coordinates of the contours whose contour area is greater than or equal to the white blood cell screening threshold, use the vertical coordinates of the centroids of these contours to subtract height / 2, and the horizontal coordinates to subtract width / 2 to obtain the offset of the centroid of each contour in the x-direction and y-direction relative to the center of the field of view. If the offset of the centroid in the x-direction and y-direction relative to the center of the field of view is greater than or equal to the offset threshold, store the corresponding contour index in the second list; Check whether the second list is empty: If the second list is empty, it is determined that there are no white blood cells outside the central area of ​​the visual field; If the second list is not empty, it is determined that there are leukocytes outside the central area of ​​the visual field, and the target leukocytes are determined from the leukocytes outside the central area of ​​the visual field.

10. The cell focusing method according to claim 9, It is characterized in that The method of determining the target leukocytes from the leukocytes existing outside the central area of ​​the field of view comprises: If the number of contours corresponding to the second list is 1, the current contour index is recorded, and the white blood cell corresponding to the current contour index is determined as the target white blood cell; If the number of contours corresponding to the second list is greater than 1, the contour indexes in the current second list are traversed and the distances from the centroid of each contour to the center of the field of view are sorted to determine the two candidate contours with the smallest distances. If the difference between the distances of the two candidate contours is within a preset range, the candidate contour with the larger area among the two candidate contours is determined to be the target white blood cell; if the difference between the distances of the two candidate contours is not within the preset range, the candidate contour with the smaller distance is determined to be the target white blood cell.

11. The cell focusing method according to claim 4, It is characterized in that Get the clarity value of the nucleus of the target white blood cells in the current original sample image, including: Traversing the contour indexes of all contours in the morphologically processed image, if the contour index is not the contour index corresponding to the target leukocyte, filling the corresponding contour in the morphologically processed image to obtain a cell nucleus mask image of the target leukocyte; Set a zero matrix background mask image of the same size as the cell nucleus mask image of the target leukocytes; Combine the original sample image with the cell nucleus mask image of the target leukocyte and the zero matrix background mask image to perform an AND operation to obtain a cell nucleus color image of the target leukocyte; Based on the cell nucleus color image of the target leukocyte, a clarity value of the cell nucleus of the target leukocyte is acquired.

12. The cell focusing method according to any one of claims 2 to 11, It is characterized in that When it is identified that there are no white blood cells in the current original sample image, the clarity value of the global red blood cells in the current original sample image is obtained as the clarity value of the current original sample image.

13. The cell focusing method according to claim 11, It is characterized in that The Sobel variant algorithm is used to obtain the clarity value of the nucleus of the white blood cells in the current original sample image, including: Converting the color image of the cell nucleus of the leukocyte into a grayscale image format to obtain a first image; Processing the first image by dividing it into blocks to obtain a plurality of non-overlapping first image blocks; Extracting the gradient features of each first image block in the x direction and the y direction based on the Sobel operator; Extracting gradient features in two diagonal directions for each first image block, and adding the gradient features in the two diagonal directions to obtain a constant gradient value; Taking the absolute value of the gradient features of each first image block in the x direction and the y direction, and then fusing them according to a set ratio to obtain a fused image; Calculate the sum of all non-zero elements of the gradient features in the fused image, and then divide it by the number of non-zero elements of the gradient features to obtain the average value of the gradient features in the x-direction and the y-direction as the clarity value of the current first image block; Adding the constant gradient eigenvalue of the current first image block to the definition value, to obtain evaluation values ​​of the gradient eigenvalues ​​in the x direction, y direction and two diagonal directions of the current first image block; The evaluation values ​​of the first image blocks in the first image are averaged to obtain an average gradient value as the clarity value of the current original sample image.

14. The cell focusing method according to claim 13, It is characterized in that The first image is processed into blocks to obtain a plurality of first image blocks, including: The first image is divided into multiple first image blocks of the third size by block processing according to the third size. During the division process, the area that does not meet the third size is filled to the third size by a filling operation to obtain the first image block of the third size.

15. The cell focusing method according to claim 12, It is characterized in that The FFT variant algorithm is used to obtain the global red blood cell clarity value in the current original sample image, including: Obtaining a global red blood cell color image based on the original sample image; Converting the color image of the global red blood cells into a grayscale image format to obtain a second image; Convert the second image from unsigned integer data to single-precision data to obtain a third image; Dividing the third image into a plurality of third image blocks, and randomly selecting two of the third image blocks; Performing discrete Fourier transform on the two third image blocks and converting them into frequency domain to obtain corresponding frequency domain images; Center the frequency domain image; Create an L×L matrix with an element value of 255, and set the grayscale value of the pixel point where the circle with a radius of L / 2 in the matrix is ​​located to 0; Use the matrix to complete the filtering operation of low-frequency information in the central area of ​​the frequency domain image; Perform inverse Fourier transform on the frequency domain image to convert the frequency domain image into a dual-channel two-dimensional image; Calculate the amplitude two-dimensional image by the real part and imaginary part of the two-dimensional image; Perform a logarithm operation on each element in the amplitude two-dimensional image and obtain an intermediate image; Multiply the intermediate image by a preset value to obtain the final image; Calculate the average value of all non-zero grayscale elements in the final image, and obtain the average value of the high-frequency component amplitude as the clarity value of the current original sample image.

16. A cell focusing device, It is characterized in that include: A control module, used to control the relative horizontal movement of the biological sample carrier and the imaging device so that the region of interest in the biological sample enters the field of view of the imaging device, and to control the relative vertical movement of the biological sample carrier and the imaging device, and to control the imaging device to capture multiple original sample images at different height positions; An acquisition module, for acquiring, when recognizing that white blood cells exist in the current original sample image, a clarity value of a cell nucleus of the white blood cells in the current original sample image as a clarity value of the current original sample image; The focusing module is used to perform focusing shooting based on the clarity value of each original sample image to obtain a focus image of the target leukocyte.

17. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by at least one processor, the method according to any one of claims 1 to 15 is implemented.

18. An electronic device, It is characterized in that The method comprises a memory and at least one processor, wherein a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the method according to any one of claims 1 to 15 is implemented.

19. A film reading machine, It is characterized in that An electronic device comprising the electronic device described in claim 18.

Images