A method for automatic detection of cells

Abstract

The application relates to the technical field of medical image processing, in particular to a cell automatic detection method, which uses an active learning and semi-supervised learning algorithm framework and a sampling strategy on the basis of YOLOV7, selects samples with poor sample recognition effects to artificial experts for labeling through an active learning strategy, and selects samples with good sample recognition effects and generated pseudo labels as real values to expand the labeled samples according to a semi-supervised learning strategy; in the selection strategy, the information entropy and the similarity information of the pseudo labels are fused to calculate the uncertainty of the pseudo labels for specific classification scenes, compared with other machine learning methods, the YOLOV7 target detection framework based on deep learning can more accurately detect and identify cells with various sizes and various morphological perspectives, and effectively avoids the missed detection and low recognition rate caused by the problems of the traditional cell morphology algorithm in the perspective and the cell size.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing, and particularly relates to a cell automatic detection method. BACKGROUND

[0002] Current cell detection mainly relies on manual microscopy by experts, so the standards are not uniform, the detection process is time-consuming and laborious, and the price is high; and cell automatic detection technology is divided into artificial feature method and deep learning method, wherein the artificial feature method detects and identifies cells by database comparison, which requires a large feature database to be established manually, which is costly; the deep learning method avoids the shortcomings of manual feature extraction required by the artificial feature method, but relies on large-scale labeled data sets to train the model, and due to the high cost of labeling, it is also difficult to obtain so many labeled data in practice. Moreover, during detection, there are situations such as missed detection, incomplete recognition, low accuracy, long detection time, etc. SUMMARY

[0003] The present application aims to provide a cell automatic detection method, and proposes a cell automatic detection method to improve the technical problems of excessive reliance on large-scale labeled data sets in existing cell automatic detection technology based on deep learning, such as missed detection, incomplete recognition, low accuracy, and long detection time.

[0004] To achieve the above-mentioned purpose, the present application provides a cell automatic detection method, comprising the following steps:

[0005] Step 1: Collect a large number of cell microscopic images as a sample set, and each picture is a sample;

[0006] Step 2: Select a small number of samples in the sample set and perform manual labeling and definition, wherein the sample set with labels is L set, all remaining sample sets without labels are U set, the initial empty pseudo-label set is P set, the initial empty unlabeled sample candidate set is UP set, the sample set collected through active learning strategy is E set, and the semi-supervised sampling set is F set;

[0007] Step 3: Taking P, L, and F data sets as training samples, a semi-supervised detection model is trained through a YOLO V7 target detection model;

[0008] Step 4: Determine whether the F set is empty, if yes, jump to step 11 for execution; if no, continue to execute;

[0009] Step 5: First, update the unlabeled sample candidate set UP = UP U F, then empty all samples in the F set, and add the samples selected by the semi-supervised learning strategy to the UP set;

[0010] Step 6: Select s1 samples from U set as E set by YOLO model and active learning strategy S1, perform manual labeling, and update U set, where L = L U E, U = U / E;

[0011] Step 7: Build Faster R-CNN model, train Faster R-CNN classifier through L set expanded in the previous step, and obtain detection model;

[0012] Step 8: Detect the unlabelled sample candidate set UP by the trained Faster R-CNN model in step 7 to obtain the prediction result label1;

[0013] Step 9: Detect all samples in UP set using the YOLO model obtained in step 3 to obtain the prediction result label2;

[0014] Step 10: Select samples with the same prediction results obtained in steps 8 and 9 as p1 set, and supplement them to the pseudo-label set P;

[0015] Step 11: Find s2 samples from U set according to the YOLO model obtained in step 3 and semi-supervised learning sampling strategy S2 for use in the next training of YOLO model;

[0016] Step 12: Update U set to U = U / F;

[0017] Step 13: Determine whether U set is empty, if yes, the program ends; if not, return to step 3 and continue running.

[0018] In step 2, the L set contains 10 categories of white blood cells WBC, single fungal spore SMB, germinating fungal spore MMB, fungal spore group TMB, epithelial cells SPC, red blood cells RBC, clue cells XSC, trichomonas D, fungal hyphae MS and bacillus XJ.

[0019] In the first training, only the L set has sample data, and the YOLO model is the main detection model, and the final iteration result is the final cell detection model.

[0020] The automatic cell detection method obtains the prediction values of all unlabelled samples in the sample pool U by the YOLO model, including the probabilities of all cell categories and the bounding box.

[0021] The automatic cell detection method uses the information entropy measurement method to represent the uncertainty of the pseudo-label result, thereby selecting samples with large information content.

[0022] The cell automatic detection method introduces similarity information, and fuses the information entropy and similarity information of the pseudo label to calculate the uncertainty of the pseudo label for a specific classification scene.

[0023] The application provides a cell automatic detection method, which uses an algorithm framework and a sampling strategy of active learning and semi-supervised learning on the basis of YOLO V7, selects samples with poor recognition effect by the active learning strategy to be labeled by an artificial expert, and then selects samples with good recognition effect and generated pseudo labels as real values to expand the labeled samples according to the semi-supervised learning strategy; in the selection strategy, the information entropy and similarity information of the pseudo label are fused to calculate the uncertainty of the pseudo label for a specific classification scene, the quality of the selected samples is higher, the redundant samples are reduced, the artificial labeling time is less, the model accuracy is higher, the detection performance is better, compared with other machine learning methods, the YOLO V7 target detection framework based on deep learning can more accurately detect and identify cells of various sizes and various morphological angles, effectively avoiding the missed detection and low recognition rate caused by the size and angle of the traditional cell morphology algorithm. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0025] Figure 1 is a flowchart of a cell automatic detection method of the present application.

[0026] Figure 2 is a scale size comparison diagram of epithelial cells and molds in the specific embodiment of the present application.

[0027] Figure 3 is a comparison diagram of white blood cells and red blood cells in the specific embodiment of the present application.

[0028] Figure 4 is a comparison diagram of white blood cells and red blood cells in the specific embodiment of the present application. DETAILED DESCRIPTION

[0029] The embodiments of the present application will be described in detail below, and examples of the embodiments are shown in the drawings, wherein the same or similar reference signs represent the same or similar elements or elements with the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and are intended to explain the present application, and cannot be understood as a limitation of the present application.

[0030] Referring to Figure 1 The application provides a cell automatic detection method, comprising the following steps:

[0031] Step 1: Collect a large number of cell microscopic images as a sample set, and each picture is a sample;

[0032] Step 2: Select part of the samples in step 1 for manual labeling to obtain a labeled sample set defined as L set, all remaining unlabeled sample sets are defined as U set, define an initial empty pseudo-label set as P set, an initial empty unlabeled sample candidate set as UP set, and a sample set collected through an active learning strategy as E set, and a sample set collected through semi-supervised sampling as F set;

[0033] Step 3: Use P, L and F data sets as training samples to train a semi-supervised detection model through a YOLO V7 target detection model, wherein only the L set has sample data during the first training, the YOLO model is the main detection model, and the final iteration result is the final model;

[0034] Step 4: Determine whether the F set is empty, if the F set is empty, it indicates that no sample is extracted through semi-supervised learning, at this time, the UP data is the same as the data in the last cycle, so step 11 is directly executed; if not, continue to execute;

[0035] Step 5: First, update the unlabeled sample candidate set UP=F, then empty all samples in the F set, add the samples selected through the semi-supervised learning strategy to the UP set, so as to perform rescreening on these samples and predicted results in the subsequent step;

[0036] Step 6: Select s1 samples from the U set as the E set through the YOLO model and the active learning strategy S1, perform manual labeling, and update the sample pool U, wherein L=L∪E and U=U / E, s1 samples with poor recognition effect and large difference from the training samples are selected through the active learning strategy, and are labeled by an artificial expert, so that the number of training set samples can be effectively expanded;

[0037] Step 7: Build a Faster R-CNN model, train the Faster R-CNN classifier through the L set expanded in the last step, and obtain a detection model;

[0038] Step 8: Detect the unlabeled sample candidate set UP through the Faster R-CNN model trained in the last step to obtain a predicted result label1;

[0039] Step 9: Detect all samples in the UP set through the YOLO model obtained in step 3 to obtain a predicted result label2;

[0040] Step 10: In order to improve the accuracy of the prediction of the YOLO model, a Faster R-CNN model is additionally introduced to make a second prediction on the prediction result of the YOLO model, because the training data of the Faster R-CNN model is all manually labeled, therefore, the same sample obtained in the two steps of step 8 and step 9 is regarded as the result closest to the manual labeling, and is defined as a p1 set and is supplemented to the pseudo-label set P;

[0041] Step 11: according to the YOLO model obtained in step 3, combined with the semi-supervised learning sampling strategy S2, s2 samples are searched from the sample pool U for use in the next training of the YOLO model;

[0042] Step 12: U is updated as U=U / F;

[0043] Step 13: whether U is an empty set is judged, if yes, at this time, the labeling work on the unlabeled sample set U set has been completed, and the final detection model has been trained, so the program ends; if not, return to step 3 to continue running, and the YOLO model is trained through continuous loop iteration, and finally a high-precision cell detection model can be obtained with less labeling cost.

[0044] The samples that can be detected in the application include but are not limited to blood, urine, feces, bone marrow and leukorrhea and the like, and the leukorrhea leukocytes are taken as an example here.

[0045] Specifically, the active learning and semi-supervised learning sampling strategy used in the application is:

[0046] 1. The prediction value of all unlabeled samples in the U pool is obtained through the YOLO model, which includes the probability of all cell categories and the boundary box;

[0047] 2. In the active learning and semi-supervised learning algorithm framework, the selection strategy is crucial. From the geometric level, the sample with the greatest uncertainty of the prediction result is often located near the classification boundary, and the cell detection in the scene of the application involves 10 classifications, therefore, the information entropy is more easily extended to the current scene, and the information entropy measurement method is adopted to represent the uncertainty of the pseudo-label result, so as to select the sample with large information, and the algorithm is specifically expressed by the formula:

[0048] Information entropy:

[0049]

[0050] Wherein p m (y|x i ) represents the model prediction probability of the sample under the c class, and H(y|x i) represents the information entropy of the sample, the greater the value, the more chaotic the sample, the higher the uncertainty, num(U) is a constant representing the number of categories, and the sum of the information entropy of all categories of each picture is calculated.

[0051] Similarity:

[0052]

[0053] Where i represents the labeled sample, j represents the unlabeled sample, represents the labeled category probability, represents the predicted category probability, C is a constant representing the number of categories, num(U) represents the number of labels in U, R i represents the similarity between the i-th sample in the U pool and the labeled sample.

[0054] Finally, the information entropy and similarity information are combined, so it is set to:

[0055]

[0056] As shown in the above formula, the information entropy contains the independent information of the sample itself, and the similarity information is also introduced considering the difference between the pseudo-label sample and the labeled sample. By comparing itself and the labeled data, the comprehensive results of the pseudo-label are sorted, wherein The greater the value, the worse the sample recognition, and the smaller the value, the better the recognition effect; therefore, in the active learning sampling strategy, by selecting s1 samples with the largest value, the artificial expert is labeled, so as to improve the accuracy of the model, and in the semi-supervised learning, s2 samples with the smallest value are selected, and the pseudo-label result is considered to be close to the artificial labeling, and are sent to the labeled training sample set L.

[0057] The technical scheme provided by the application has the beneficial effects that: no sample staining is required, reducing the workload and cost; the sample quantity can be increased through collaborative active learning and semi-supervised learning under the condition of a small number of labeled samples, and the model can be optimized; cells with large scale difference and different perspectives can be recognized, and high recognition speed and accuracy can be achieved in a complex environment with target density.

[0058] The application also combines specific embodiments to explain the reasons for the accuracy difference of different detection methods, as shown in Figure 2 The volume of epithelial cells is about 10-20 times the size of mold, and the huge difference will cause the trained model to ignore small mold cells.

[0059] As Figure 3As shown, because the leucorrhea needs to stand for a period of time on the slide, the sample will evaporate to cause cell morphology damage and cell stacking, and the cells in the liquid sample are always in motion, so different visual angle forms will be presented, and various reasons cause the traditional cell morphology recognition method to be unable to recognize or recognize incorrectly the broken cells and the cells with different visual angles, finally leading to low model recognition accuracy.

[0060] Further, please refer to Figure 4 It can be understood that the present application has advantages compared with the existing deep learning detection method.

[0061] As Figure 4 As shown, the present application can recognize a plurality of cell categories, has good recognition accuracy in a complex scene with various cell categories, a large number of quantities and obvious environmental changes, and has a very good recognition speed of 100 ms per image. The left picture is a traditional recognition method with less recognition categories, easy to miss detection and low accuracy, and the right picture is the detection method of the present application.

[0062] The above only discloses one preferred embodiment of the present application, and of course cannot limit the scope of the right of the present application, and those skilled in the art can understand that all or part of the processes of the above-mentioned embodiments are implemented, and equivalent changes made according to the claims of the present application still belong to the scope covered by the present application.

Claims

1. A method for automatic detection of cells, characterized in that, The method comprises the following steps: Step 1: Collect a large number of cell microscopic images as a sample set, with each picture as a sample; Step 2: Select a small number of samples in the sample set and perform manual labeling and definition, wherein the obtained labeled sample set is L set, the remaining all unlabeled sample set is U set, the initial empty pseudo-label set is P set, the initial empty unlabeled sample candidate set is UP set, the sample set collected through active learning strategy is E set, and the semi-supervised sampling set is F set; Step 3: Taking P, L and F data sets as training samples, a semi-supervised detection model is trained through a YOLO V7 target detection model; Step 4: Determine whether the F set is empty, if yes, jump to step 11 for execution; if no, continue to execute; Step 5: First, update the unlabeled sample candidate set UP = UP∪F, then empty all samples in the F set, and add the samples selected by the semi-supervised learning strategy to the UP set; Step 6: Select s1 samples from the U set as the E set through the YOLO model and the active learning strategy S1, perform manual labeling, and update the U set, wherein L = L∪E, U = U / E; Step 7: Build a Faster R-CNN model, train the Faster R-CNN classifier through the L set expanded in the last step, and obtain a detection model; Step 8: Detect the unlabeled sample candidate set UP through the Faster R-CNN model trained in step 7 to obtain a prediction result label1; Step 9: Detect all samples in the UP set using the YOLO model obtained in step 3 to obtain a prediction result label2; Step 10: Select samples with the same prediction results obtained in steps 8 and 9 as p1 set, and supplement them to the pseudo-label set P; Step 11: According to the YOLO model obtained in step 3, find s2 samples from the U set by combining the semi-supervised learning sampling strategy S2, for use in the next training of the YOLO model; Step 12: Update the U set to U = U / F; Step 13: Determine whether the U set is empty, if yes, the program ends; if no, return to step 3 for continuous operation.

2. The automatic cell detection method according to claim 1, wherein the L set in step 2 comprises 10 categories of white blood cells WBC, single mold spore SMB, mold spore germination MMB, mold spore group TMB, epithelial cells SPC, red blood cells RBC, clue cells XSC, trichomonas D, mold hyphae MS and bacillus XJ.

3. The automatic cell detection method according to claim 1, wherein only the L set has sample data during the first training, the YOLO model is the main body of the detection model, and the final training iteration result is the final cell detection model used by the automatic cell detection method.

4. The automatic cell detection method according to claim 1, wherein the automatic cell detection method obtains the prediction value of all unlabeled samples in the sample pool U through the YOLO model, including the probability of each category of cells and the bounding box.

5. The automatic cell detection method according to claim 1, wherein ​ ​ ​ The cell automatic detection method adopts a measurement method of information entropy to represent the uncertainty of the pseudo label result, so as to select samples with large information quantity.

6. The cell automatic detection method of claim 5, wherein the cell automatic detection method introduces similarity information, and fuses the information entropy and the similarity information of the pseudo label to calculate the uncertainty of the pseudo label according to a specific classification scene. ​

Images