Cell counting method based on computer vision

Abstract

The present application relates to a computer vision-based cell counting method, comprising the following steps: S1, collecting an RGB original image E0 of a dyed section sample, and processing into a gray image E1 and a blue channel image E18 respectively; S2, performing image processing on the original image E0 to obtain a dyed cell image E6, and counting the dyed cells; S3, performing image processing on the gray image E1 to obtain an image E16 containing all dyed and non-dyed cells, and counting all cells; S4, performing image processing on the blue channel image E18 to obtain an image E21, counting the dyed cells according to given different demarcated regions, calculating the number of pixel areas, and accurately marking the peripheral contour of the region; and S5, visually displaying the above images and counts for intuitive display and cell counting.

Description

Technical Field

[0001] This invention relates to computer machine vision, specifically a cell counting method based on medical images using computer vision. Background Technology

[0002] The purpose of controlled learning systems in machine learning is to efficiently classify large amounts of data to facilitate the mining and application of this data. Pattern recognition enables machines to automatically identify objects. As a discipline, computer vision primarily focuses on extracting 'information' from images to build artificial intelligence systems. It has extremely broad application prospects in medical image processing, especially in cell detection and counting, and is highly valued by researchers worldwide.

[0003] Currently, in clinical diagnosis, the number and proportion of various abnormal cells are mainly determined subjectively by the human eye. This method is not only prone to significant errors and subjectivity, but also time-consuming, labor-intensive, and places a heavy burden on clinical medical personnel, resulting in high labor costs. With the widespread research into deep learning, medical cell detection and counting have gradually become a hot research topic. However, deep learning requires a large number of labels, which is expensive, and models are difficult to build with large sample sizes. Therefore, there is an urgent need for a lower-cost method for detecting and counting abnormal cells that can truly facilitate clinical diagnosis. Summary of the Invention

[0004] To address the shortcomings of the aforementioned technologies, the present invention aims to provide a computer vision-based method for counting cells in medical images.

[0005] The technical solution adopted by this invention to achieve the above objectives is: a cell counting method based on computer vision, comprising the following steps:

[0006] S1. Collect the original RGB image E0 of the stained slice sample and process it into grayscale image E1 and blue channel image E18 respectively.

[0007] S2. Perform image processing on the original image E0 to obtain the stained cell image E6, and count the stained cells;

[0008] S3. Perform image processing on grayscale image E1 to obtain image E16 containing all stained and unstained cells, and count all cells.

[0009] S4. Perform image processing on the blue channel image E18 to obtain image E21. Count the stained cells according to the given different demarcation regions, calculate the number of pixel areas, and accurately delineate the outer contour of the region.

[0010] S5. Visualize the above images and counts for intuitive display and cell counting.

[0011] The given different demarcation areas are areas that have been roughly selected by humans;

[0012] The calculation of pixel area is as follows: calculate the total pixel area of ​​stained cells within the region, and classify each demarcation region by size level in combination with the area threshold;

[0013] The precise delineation of the outer contour of the demarcation region is achieved by automatically drawing and displaying the smallest bounding box for the stained cells within the current demarcation region on the image.

[0014] Obtain the cell count map of the first-level region (E22), the cell count map of the second-level region (E23), and the area ratio map of the first-level region and the second-level region (E24);

[0015] S4 further includes classifying stained cells according to different staining degrees, and counting stained cells in different demarcation regions according to different staining degrees. A graph E25 showing the proportions of strongly stained, moderately stained, and weakly stained cells within the first-level and second-level regions is obtained.

[0016] The original stained image E0 includes stained abnormal cells and unstained cells, and the colors differ after staining.

[0017] The image processing steps performed on the original image E0 sequentially include: median filtering, threshold segmentation, binarization, and morphological closing operation to obtain the stained cell image E6.

[0018] It also includes further optimization processing of the stained cell map E6, such as inversion, watershed segmentation, and inversion.

[0019] The image processing of grayscale image E1 includes: performing erosion and morphological cap filtering on grayscale image E1, adjusting grayscale contrast, binarizing, deleting interference noise, inverting colors, multiplying with the binarized image E13 in step S3, and superimposing with the image E8 after watershed segmentation in step S2 to obtain image E16 containing all stained and unstained cells.

[0020] The image processing of the blue channel image E18 to obtain E21 includes: basic global thresholding, morphological closing operation, erosion and color inversion operation to obtain image E21.

[0021] The counting of stained cells and the counting of all cells are both done by counting the centroids of each connected region in the figure, and obtaining the counting graph of all stained cells E10 and the counting graph of all cells E17 respectively.

[0022] The process of classifying and counting stained cells in the demarcation area according to different staining degrees involves counting the centroids of each connected region classified by different staining degrees within the current demarcation area, obtaining the number of strongly stained, moderately stained, and weakly stained cells in different demarcation areas, as well as the proportion of these cells (Figure E25).

[0023] The present invention has the following beneficial effects and advantages:

[0024] 1. Clinical medical slice images were collected, raw data were gathered, and the problems of deep learning requiring a large number of labels, high cost, and difficulty in building models were solved. A low-cost method that truly promotes cell detection and counting in medical images was proposed.

[0025] 2. It can solve the problems of the current time-consuming and subjectively influenced manual counting of captured cells.

[0026] 3. Improved the efficiency and accuracy of cell counting in images in clinical medicine.

[0027] 4. Visualize the images and counting results for a more intuitive presentation. Attached Figure Description

[0028] Figure 1 This is a flowchart of the method of the present invention;

[0029] Figure 2 These are the effect diagrams of key steps in the method of this invention;

[0030] Figure 3 This is the original image E0 of the clinical medical slide sample obtained by the method of this invention;

[0031] Figure 4 This is the grayscale image E1 of the method of the present invention;

[0032] Figure 5 This is the grayscale adjustment image E2 of the method of the present invention;

[0033] Figure 6 This is the median filtering diagram E3 of the method of the present invention;

[0034] Figure 7 This is the threshold segmentation diagram E4 of the method of the present invention;

[0035] Figure 8 This is the binarized image E5 of the method of the present invention;

[0036] Figure 9 This is the closing operation diagram E6 of the method of the present invention;

[0037] Figure 10 This is the inverted color image E7 of the method of this invention;

[0038] Figure 11This is the watershed segmentation diagram E8 of the method of the present invention;

[0039] Figure 12 This is the inverse color image E9 of the method of the present invention;

[0040] Figure 13 This is Figure E10, showing the total number of stained cells according to the method of this invention.

[0041] Figure 14 This is the bottom cap filtering diagram E11 of the method of the present invention;

[0042] Figure 15 This is the grayscale contrast image E12 of the method of the present invention;

[0043] Figure 16 This is the binarized image E13 of the method of the present invention;

[0044] Figure 17 This refers to the deletion of the small white area image E14 by the method of the present invention;

[0045] Figure 18 This is the white small-area diagram E15 of the method of the present invention;

[0046] Figure 19 Figure E16 shows the addition process of the method of the present invention;

[0047] Figure 20 This is a diagram E17 showing all cell counts using the method of the present invention;

[0048] Figure 21 This is the blue channel diagram E18 of the method of this invention;

[0049] Figure 22 This is the basic global threshold segmentation map E19 of the method of the present invention;

[0050] Figure 23 This is the closed operation diagram E20 of the method of the present invention;

[0051] Figure 24 This is the inverse color image E21 of the method of the present invention;

[0052] Figure 25 This is the first-level zone counting diagram E22 of the method of the present invention;

[0053] Figure 26 This is the second-level zone counting diagram E23 of the method of the present invention;

[0054] Figure 27 This is the area ratio diagram of the first-level zone and the second-level zone, E24;

[0055] Figure 28 This is a diagram showing the proportions of strongly stained, moderately stained, and weakly stained cells in the first and second level regions, E25. Detailed Implementation

[0056] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0057] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0058] 1. Methods and Steps

[0059] This invention provides a cell counting method based on computer vision. For example... Figure 1 The flowchart shown is a method of the present invention. Figure 2 The diagram shows the effect of key steps in the method of the present invention. The method includes the following steps:

[0060] Step 1: Use the Macody Digital Slicing Assistant system to extract the original image E0 of the medical slide sample;

[0061] Step 2: Obtain grayscale image E1 by transforming the original image E0 to grayscale.

[0062] Step 3: Adjust the grayscale of grayscale image E1 to obtain grayscale adjusted image E2;

[0063] Step 4: Perform median filtering on the grayscale adjustment image E2 to obtain the median filtered image E3;

[0064] Step 5: Perform threshold segmentation on the median filtered image E3 to obtain the threshold segmentation image E4;

[0065] Step 6: Binarize the threshold segmentation image E4 to obtain the binarized image E5;

[0066] Step 7: Perform a closing operation on the binarized graph E5 to obtain the closed graph E6;

[0067] Step 8: Invert the colors of the closed operation graph E6 to obtain the inverted color graph E7. Perform watershed segmentation on the inverted color graph E7 to obtain the watershed segmentation graph E8.

[0068] Step 9: Invert the colors of the watershed segmentation image E8 to obtain the inverted image E9;

[0069] Step 10: Count the centroids of the connected regions in the inverted color map E9 to obtain the count map of all stained cells E10;

[0070] Step 11: Erosion and morphological bottom cap filtering are applied to the grayscale image E1 to obtain the bottom cap filtered image E11;

[0071] Step 12: Adjust the grayscale contrast of the bottom cap filter image E11 to obtain the grayscale contrast image E12;

[0072] Step 13: Binarize the grayscale contrast image E12 to obtain the binarized image E13;

[0073] Step 14: Remove small white areas from the binarized image E13 to obtain image E14 with small white areas removed;

[0074] Step 15: Invert the colors of the deleted white small area image E14 and multiply it with the binarized image E13 to obtain the white small area image E15;

[0075] Step 16: Add the white small area map E15 and the watershed segmentation map E8 to obtain the added map E16;

[0076] Step 17: Count the centroids of the connected regions in the summation diagram E16 to obtain the total cell count diagram E17 (including all stained and unstained cells);

[0077] Step 18: Extract the blue channel from the original image E0 to obtain the blue channel image E18;

[0078] Step 19: Perform basic global thresholding on the RGB image E18 to obtain the basic global thresholding image E19;

[0079] Step 20: Perform morphological opening and closing operations on the basic global threshold segmentation map E19 to obtain the closing operation map E20;

[0080] Step 21: Perform erosion and inversion operations on the closed operation image E20; obtain the inverted color image E21;

[0081] Step 22: Count the centroids of the connected regions in the inverted color map E21 to obtain the first-level region counting map E22 and the second-level region counting map E23 respectively;

[0082] Step 23: Calculate the area of ​​the first-level area counting map E22 and the second-level area counting map E23 to obtain the area ratio map E24 of the first-level area and the second-level area;

[0083] Step 24: Perform color threshold segmentation on the first-level and second-level regions to obtain the cell proportion map E25 of strong, moderate, and weak staining in the first-level and second-level regions.

[0084] The following is Figure 1 The following key steps will be explained in detail:

[0085] Step 1: Image Acquisition. In this invention, the method primarily uses the Macody Digital Slicing Assistant System to extract original images of clinical medical slide samples from the Affiliated Hospital of China Medical University. This method places strict requirements on the clinical authenticity of the samples, emphasizing practicality and effectiveness. The original sample image E0 is as follows: Figure 3 As shown.

[0086] Step 2: Obtaining grayscale image E1 by grayscale transformation of the original image E0 includes: converting the original clinical medical color image E0 into grayscale image E1 using grayscale transformation, wherein grayscale image E1 is as follows: Figure 4 As shown.

[0087] Step 3 involves adjusting the grayscale of image E1 to obtain an adjusted grayscale image E2.

[0088] Among them, grayscale adjustment image E2 is as follows Figure 5 As shown.

[0089] Step 4 involves performing median filtering on the grayscale adjusted image E2 to obtain a median-filtered image E3. The median-filtered image E3 is as follows: Figure 6 As shown.

[0090] Step 5 involves performing threshold segmentation on the median filtered image E3 to obtain a threshold segmentation image E4. The threshold segmentation image E4 is as follows: Figure 7 As shown.

[0091] Step 6 involves binarizing the threshold segmentation image E4 to obtain a binarized image E5. The binarized image E5 is as follows: Figure 8 As shown.

[0092] Step 7 involves performing a closing operation on the binarized graph E5 to obtain the closed graph E6. The closed graph E6 is as follows: Figure 9 As shown.

[0093] In step 8, the closed operation graph E6 is inverted to obtain an inverted graph E7. Watershed segmentation is then performed on the inverted graph E7 to obtain a watershed segmentation graph E8. The inverted graph E7 is as follows: Figure 10 As shown, the watershed segmentation diagram E8 is as follows: Figure 11 As shown.

[0094] Step 9 involves inverting the colors of the watershed segmentation image E8 to obtain an inverted color image E9. The inverted color image E9 is as follows: Figure 12 As shown.

[0095] Step 10 involves counting the centroids of connected regions in the inverted color map E9 to obtain a total stained cell count map E10. The total stained cell count map E10 is as follows: Figure 13 As shown.

[0096] Step 11 involves eroding and performing morphological bottom-cap filtering on the grayscale image E1 to obtain a bottom-cap filtered image E11. The bottom-cap filtered image E11 is as follows: Figure 14 As shown.

[0097] Step 12 involves adjusting the grayscale contrast of the bottom cap filter image E11 to obtain a grayscale contrast image E12. The grayscale contrast image E12 is as follows: Figure 15 As shown.

[0098] Step 13 involves binarizing the grayscale contrast image E12 to obtain a binarized image E13. The binarized image E13 is as follows: Figure 16 As shown.

[0099] Step 14 involves deleting small white areas from the binarized image E13 (with a white area deletion threshold of 300) to obtain image E14 with small white areas deleted. Image E14 with small white areas deleted is as follows: Figure 17 As shown.

[0100] Step 15 involves inverting the colors of the deleted white small-area image E14 and multiplying it with the binarized image E13 to obtain the white small-area image E15. The white small-area image E15 is as follows: Figure 18 As shown.

[0101] In step 16, the white small area map E15 and the watershed segmentation map E8 are summed.

[0102] Obtain the summation processing diagram E16. The summation processing diagram E16 is as follows: Figure 19 As shown.

[0103] Step 17 involves counting the centroids of the connected regions in the summation processing diagram E16 to obtain a total cell count diagram E17 (including all stained and unstained cells). The total cell count diagram E17 is as follows: Figure 20 As shown.

[0104] Step 18 involves extracting the blue channel from the original image E0 to obtain a blue channel image E18. The blue channel image E18 is as follows: Figure 21 As shown.

[0105] Step 19 involves performing basic global thresholding on the blue channel image E18 to obtain a basic global thresholding image E19. The basic global thresholding image E19 is as follows: Figure 22 As shown.

[0106] Step 20 involves performing morphological opening and closing operations on the basic global threshold segmentation map E19 (opening threshold is 2, closing threshold is 8, and the opening and closing thresholds are the pixel radii) to fill gaps, resulting in a closing operation map E20. The closing operation map E20 is as follows: Figure 23 As shown.

[0107] Step 21 involves performing erosion and color inversion operations on the closed operation image E20 to obtain the inverted color image E21. The inverted color image E21 is as follows: Figure 24 As shown.

[0108] Step 22 involves counting the centroids of connected regions in the inverted color map E21 to obtain a first-level region counting map E22 and a second-level region counting map E23. The first-level region counting map E22 is as follows: Figure 25 As shown, the second-level zone counting diagram E23 is as follows. Figure 26 As shown.

[0109] Step 23 involves calculating the area of ​​the first-level area counting map E22 and the second-level area counting map E23 to obtain the area ratio map E24 between the first-level and second-level areas. The area ratio map E24 between the first-level and second-level areas is shown below. Figure 27 As shown.

[0110] Step 24 involves performing color threshold segmentation on the first-level and second-level regions to obtain a cell proportion map E25 showing the proportions of strongly stained (brownish-black), moderately stained (brownish-yellow), and weakly stained (pale yellow) cells in each region. The cell proportion map E25 for strongly stained, moderately stained, and weakly stained cells in the first-level and second-level regions is shown below. Figure 28 As shown.

[0111] 2. Experimental Environment and Parameter Settings

[0112] All experiments were conducted on an Intel(R) Core(TM) i5-5200U CPU with 4.00GB of RAM at a 2.20GHz processor. Cell counting experiments were primarily programmed using Matlab, with Matlab 2014b as the experimental environment. The LoG algorithm filtering employed a Gaussian kernel with a standard deviation of 1.1 and a size of 7*7.

[0113] 3. Experimental Analysis and Results Presentation

[0114] In this embodiment, clinical medical breast cancer tissue samples were selected for image acquisition and processing, and the images were counted for visual and intuitive display.

[0115] Experimental Objective 1

[0116] For clinical cell counting, after slides are stained, raw images are acquired. These images show cells in two colors: black and blue. The goal is to distinguish stained cancer cells (black) from unstained normal glial cells (blue). This invention utilizes computer vision and image processing techniques, employing a series of image processing steps including grayscale conversion, histogram manipulation, median filtering, binarization, color-based thresholding, morphological processing, watershed segmentation, and connected component counting. Ultimately, the machine indirectly distinguishes between stained (black) and unstained (blue) cells. The number of stained cells was 647, and the total number of all cells, including both stained and unstained cells, was 2982. Therefore, the number of unstained cells was 2335, resulting in a stained-to-unstained cell ratio of 647:2335.

[0117] Experimental Objective 2

[0118] For cell detection and counting in clinical medical images, these images contain first-level and second-level regions. For example, the first-level region is called the in situ region, and the second-level region is called the infiltrative region. Human observation requires significant effort from medical personnel. According to clinical requirements, it is necessary to use machines to detect the number of first-level and second-level regions and determine their area ratio. This invention utilizes computer vision and image processing technology, employing a series of image processing steps including normalization, RGB color image processing, basic global threshold segmentation, morphological processing, connected region counting, and ghosting removal. Ultimately, the machine detects the number of first-level and second-level regions and obtains their area ratio. The machine detected one first-level region and 14 second-level regions, resulting in an area ratio of 2.777:1.

[0119] Experimental Objective 3

[0120] For cell counting in clinical medical images, there are first-level and second-level regions. These regions contain strongly positive cells (stained as brownish-black), moderately positive cells (stained as brownish-yellow), and weakly positive cells (stained as pale yellow). Due to the large number and density of cells, visual observation requires significant effort from medical personnel and is easily influenced by subjective judgment. According to clinical medical requirements, it is necessary to use machines to determine the proportion of each of the three different staining levels in the first-level and second-level regions, thereby achieving a deeper understanding of the patient's condition. This invention utilizes computer vision and image processing technology, employing a series of image processing steps including grayscale conversion, histogram manipulation, median filtering, binarization, color-based thresholding, morphological processing, watershed segmentation, and connected region counting. Ultimately, the machine detects the proportion of each of the three different staining levels in the first-level and second-level regions. The machine determined the proportions of the number of stained cells in the first and second grade regions to be 0.35326:0.38587:0.26087; and 0.29701:0.52564:0.17735, respectively.

[0121] To address the problems in the three clinical medical images mentioned above, this invention employs computer vision and image processing methods to automatically identify and count the captured stained cells, enabling the machine to assist in counting and display the data visually, thereby meeting the requirements of image processing practice.

[0122] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should be considered within the scope of protection of the present invention.

Claims

1. A computer vision-based cell counting method, characterized in that, The method comprises the following steps: S1, acquiring an RGB original image E0 of a stained section sample by using a Macodyl digital section assistant system, and processing the original image into a gray image E1 and a blue channel image E18 respectively; S2, performing image processing on the original image E0 to obtain a stained cell image E6, and counting the stained cells; the image processing on the original image E0 to obtain the stained cell image E6 comprises median filtering, threshold segmentation, binarization, morphological closing operation, color inversion, and watershed segmentation; S3, performing image processing on the gray image E1 to obtain an all-cell image E16 containing stained and unstained cells, and counting all the cells; the image processing on the gray image E1 comprises erosion and morphological bottom-hat filtering, adjusting the gray contrast, binarization processing, deleting interference noise points, color inversion, and performing product processing with the binarized image E13 in step S3 and superposition processing with the image E8 processed by the watershed segmentation in step S2, to obtain the all-cell image E16 containing stained and unstained cells; S4, performing image processing on the blue channel image E18: basic global threshold segmentation, morphological closing operation, erosion and color inversion operation, to obtain an image E21; counting the stained cells in a region roughly framed by a person, calculating the total pixel area of the stained cells in the region, and classifying the demarcated regions in different size grades in combination with an area threshold; automatically outlining and displaying a minimum bounding box for the stained cells in the current demarcated region on the image; obtaining a first-grade region stained cell count image E22, a second-grade region stained cell count image E23, and a first-grade region and second-grade region area ratio image E24; classifying the stained cells according to different staining degrees, classifying and counting the stained cells according to color threshold segmentation in different demarcated regions, and obtaining a cell proportion image E25 of strongly stained, moderately stained and weakly stained cells in the first-grade region and the second-grade region; S5, visualizing and displaying the above images and counts for intuitive display and cell counting.

2. The computer vision-based cell counting method of claim 1, wherein, The original image E0 after staining includes stained abnormal cells and unstained cells, and the colors are different after staining.

3. The computer vision-based cell counting method of claim 1, wherein, The counting of the stained cells and the counting of all the cells are both counting the centroids of each connected region in the image, to obtain all the stained cell count image E10 and the all-cell count image E17 respectively; The classification and counting of the stained cells in the demarcated region according to different staining degrees are counting the centroids of each connected region of different staining degrees in the current demarcated region, to obtain the number and proportion of strongly stained, moderately stained and weakly stained cells in the different demarcated regions, and the number proportion image E25.

Images