Intelligent detection method and device for long-tail blood cells, storage medium and electronic equipment

Abstract

The invention discloses an intelligent detection method and device for long-tail blood cells, a storage medium and electronic equipment, and relates to the cross technical field of artificial intelligence and blood cell detection.The method comprises the steps that training data are detected through a blood cell detection initial model, and a classification detection result is obtained; pre-training a single cell representation learning model, and respectively representing the obtained detection result and the real segmentation result; gaussian distribution of classes is established according to representation of a real result, and long-tail sample data is enhanced through an adaptive sampling probability mechanism; and establishing a class-guided posterior regularization mechanism, aligning class distribution obtained by the blood cell detection initial model with real class distribution, and training the blood cell detection initial model again in combination with target detection loss so as to detect the blood cell image. Therefore, by adopting the method, the long-tail distribution problem in fine-grained blood cell target detection can be effectively solved, and meanwhile, the detection precision of the model on long-tail blood cells is improved.

Description

Technical Field

[0001] The present invention relates to the cross - technical field of artificial intelligence and blood cell detection, and in particular, to an intelligent detection method, device, storage medium and electronic device for long - tailed blood cells. Background Art

[0002] In clinical examinations, blood cell detection occupies an important position. According to the different blood collection sites, it is mainly divided into bone marrow and peripheral blood detection. The bone marrow is the main place for hematopoiesis and has a variety of blood cell subtypes. The peripheral blood is the main source of daily blood cell detection. The changes in the morphology and quantity of these blood cells are important bases for evaluating the health status of the body and diagnosing blood diseases. In recent years, blood cell detection methods have been widely studied. The existing blood cell detection methods can be roughly divided into manual microscopy, rule - based morphological segmentation, and deep - learning - based methods. Among them, deep neural networks have been widely used in blood cell detection due to their efficient automatic feature extraction ability. Currently, two - stage object detection networks (RCNN), single - stage object detection models (such as SSD, YOLO), and VIT are widely used in blood cell detection models.

[0003] However, the clinical demand for the detection of multiple blood cell subtypes and the existence of some rare blood cell categories in the data collection process pose higher requirements for deep - learning - based blood cell detection networks, and it still faces many challenges: (1) It is difficult to distinguish multiple cell subtypes: There are more than 50 blood cell subtypes detected clinically, corresponding to various states in the life process of cell maturation, which means that different types of cells are generally very similar; when the same type of cell is in different parts of the human body, due to different external environments, there will also be differences in the same type of cell, which requires relevant practitioners in the field after professional learning to accurately judge. (2) The data shows a long - tail distribution: Most categories of blood cells only appear in specific disease states, so the number of collected samples is far less than that of common categories, resulting in a long - tail distribution of the overall data; this distribution feature makes a small number of categories have a large number of samples, which are called head categories, while most cell category samples are relatively scarce, which are called tail categories. This will cause the head categories to form gradient suppression on the tail categories during the training process of the model, and the tail categories themselves also lack diversity.

[0004] Therefore, it is necessary to provide a blood cell detection method that simultaneously considers the suppression of tail categories by head categories and the lack of diversity of tail categories, suppresses the detection preference of the model for different categories, and thus conducts effective multi - category blood cell detection. Summary of the Invention

[0005] The object of the present invention is to provide an intelligent detection method, device, storage medium and electronic device for long-tail blood cells, which can effectively solve the long-tail distribution problem in multi-class blood cell target detection and improve the detection accuracy of the model for long-tail blood cells.

[0006] To achieve the above object, the present invention provides an intelligent detection method for long-tail blood cells, including the following steps:

[0007] S1. Obtain blood cell images of all bone marrow or peripheral blood samples, introduce a classical object detection model for localization and classification, fine-tune using blood cell picture training data to obtain an initial blood cell detection model, and use this initial blood cell detection model to detect the training data to obtain detection results;

[0008] S2. According to the data labels, crop single-cell samples from the blood cell images, pre-train a single-cell representation learning model, and based on this pre-trained single-cell representation model, represent the classification detection results obtained by the initial model and the true single-cell segmentation results respectively;

[0009] S3. Establish a Gaussian distribution of classes, and generate data through an adaptive sampling probability mechanism to enhance long-tail sample data;

[0010] S4. Establish a class-guided posterior regularization mechanism to align the class distribution obtained by the initial blood cell detection model with the true class distribution, combine the object detection loss, train the initial blood cell detection model to obtain a final blood cell detection model, and then detect the blood cell images to obtain recognition results.

[0011] Preferably, the pre-trained single-cell representation learning model uses a balanced contrast learning model to train the cropped single-cell samples to obtain the trained backbone network and its parameters.

[0012] Preferably, establish a Gaussian distribution of classes and generate data as follows:

[0013] x c =μ c +σ c ⊙∈;

[0014]

[0015] In the formula, x c is the feature vector of the newly generated c-th class of cells, used to enhance long-tail sample data; μ c , σ c are the total mean and variance of the c-th class of cells in all batches, that is, the Gaussian distribution of classes; m is a hyperparameter used to calculate the total mean and variance of the c-th class of cells in all batches; is the sample X of the t-th batcht The mean and variance of the cell feature vectors of the c-th class; ∈ is the random perturbation term of the variance.

[0016] Preferably, the adaptive sampling probability mechanism further groups each category, calculates the total loss of each category within the group in the validation set, and then updates the sampling probabilities of each group of categories, avoiding the situation where some tail categories have no loss due to too few samples when updating the sampling probabilities of each category through the loss in the validation set.

[0017] Preferably, a category-guided posterior regularization mechanism is established, including obtaining the class means obtained by the initial model and the true result class means after fusing newly generated samples, then calculating the inter-class differences of each, estimating the probability distribution of the underlying features using Gaussian mixture distribution, and obtaining the class distribution loss through the KL divergence loss to evaluate the category-guided effect.

[0018] Preferably, the final blood cell detection model includes using the total loss function to constrain the training of the initial blood cell detection model; among them, the expression of the total loss function is as follows:

[0019] L total = L det + L crl ;

[0020] In the formula, L total is the total loss, L det , L crl are respectively the loss of the original object detection model and the loss of the feature distribution.

[0021] Preferably, the pre-trained single-cell representation learning model includes being constrained and trained using balanced supervised contrast loss, as follows:

[0022]

[0023] In the formula, |B y | is the total number of samples of class y, z i is the feature vector of a single cell sample, z p is the positive example class, z k is the negative example class, c y is the class prototype of the positive example class of class y samples, B j is the total number of negative example samples of class y, and c j is the class prototype of the negative example class of class y samples.

[0024] An intelligent detection device for long-tail blood cells, comprising:

[0025] An image acquisition module for acquiring blood cell images to be detected;

[0026] A blood cell detection module, which is used to obtain blood cell detection frame positioning and classification data by using a preset long-tail blood cell detection network;

[0027] A result display module, which is used to crop individual cells from the preprocessed blood cell image according to the blood cell detection frame positioning and classification data, and obtain and display the individual blood cell categories in the image to be detected.

[0028] A computer-readable storage medium stores computer instructions for executing an intelligent detection method for long-tail blood cells.

[0029] An electronic device, characterized in that it includes at least one processor, and the processor is communicatively connected to a memory;

[0030] Wherein, the memory stores instructions executable by the processor, and the instructions are used to execute an intelligent detection method for long-tail blood cells.

[0031] Therefore, the present invention adopts the above-mentioned intelligent detection method, device, storage medium and electronic device for long-tail blood cells, and has the following technical effects:

[0032] (1) By establishing a Gaussian distribution of classes and an adaptive sampling probability mechanism, new data is generated to provide multi-class blood cell data samples for the model, which can not only supplement the diversity of tail classes, but also improve the generalization of overall blood cell detection.

[0033] (2) Based on the class-guided posterior regularization mechanism, the class distribution obtained by guiding the initial blood cell detection model is aligned with the true class distribution, and a new loss is calculated on the basis of the class distribution, which can not only alleviate the gradient suppression of classes with a large number of samples on rare classes, but also reduce the influence of some sample noises.

[0034] The technical solution of the present invention will be further described in detail below with reference to the drawings and embodiments. Description of the Drawings

[0035] Figure 1 is a flowchart of an intelligent detection method for long-tail blood cells;

[0036] Figure 2 is a data processing flowchart of an intelligent detection device for long-tail blood cells. Detailed Embodiments

[0037] The present invention can be more specifically explained through the following embodiments. The purpose of disclosing the present invention is to protect all changes and improvements within the scope of the present invention. The present invention is not limited to the following embodiments.

[0038] Embodiment 1 provides an intelligent detection method for long-tailed blood cells, which is as follows:

[0039] For the definition of the blood cell detection problem, it is as follows:

[0040] Given the input blood cell image X = {x i |i ∈ R}, where x i is the i-th picture and R is the number of pictures; the task is to determine the bounding box coordinates and corresponding categories of each cell in the input image, and crop B = {B i |i ∈ N}, where B i represents a set of cropped C i class blood cells and N is the total number of categories.

[0041] As Figure 1 shown, in this embodiment, a target detection model (initial blood cell detection model) fine-tuned with blood cell data is used to detect long-tailed blood cells, and the specific process is as follows:

[0042] S1. Pass the input blood cell image X through the target detection model D θ fine-tuned with blood cell data to obtain the preliminary detection result B det = D θ (X), and at the same time cut out the cells in the corresponding data through the true labels to obtain B gt . In other embodiments, the target detection model D θ can also use different target detection models.

[0043] S2. Use the Balanced Contrastive Learning for Long-Tailed Visual Recognition (BCL) model for pre-training, and the data used for pre-training should be the same as the data for training the entire model. Here, the cut-out B gt is used for pre-training. After the pre-training is completed, select the backbone network and parameters of this model as the pre-trained single-cell learning representation model D ptm , and then use D ptm to extract the feature vectors F det = D ptm (B det ) of the preliminary detection result samples and the feature vectors F gt = D ptm (B gt ) of the true result samples.

[0044] Among them, the pre-trained single-cell representation model is constrained by calculating the balanced supervised contrastive loss L bcl , as follows:

[0045]

[0046] Wherein, |B y | is the total number of samples of class y, and z i is the feature vector of a single cell sample; z p is the positive example class, that is, the feature vectors of other cells belonging to the same cell class as y; z k is the negative example class, that is, the feature vectors of other cells not belonging to the same cell class as y; c y is the class prototype of the positive example class of samples of class y, and B j is the total number of negative example samples of class y, and c j is the class prototype of the negative example class of samples of class y. The goal of L bcl is to guide the model to still be able to train a balanced single-cell representation space under the long-tailed data distribution, represent the differences between each class, and at the same time reduce the influence of the long-tailed distribution, so that the feature vectors of different classes of cells can be evenly distributed in the computational space.

[0047] S3. According to the feature vectors F det = D ptm (B det ) of the preliminary test result samples and the feature vectors F gt = D ptm (B gt ) of the true result samples, calculate the mean of their respective feature vectors according to the cell class, and calculate the class prototypes Z det and Z gt .

[0048] In this process, to increase the intra-class diversity, in this embodiment, by establishing a Gaussian distribution of the class, the feature vectors F gt of the true result samples are processed to generate new sample data to enhance the features of the tail classes, as follows:

[0049] x c = μ c + σ c ⊙ ∈;

[0050]

[0051]

[0052] Wherein, x c is the feature vector of the newly generated c-th class of cells; μ c , σ c are the total mean and variance of the c-th class of cells in all batches; m is a hyperparameter used to calculate the total mean and variance of the c-th class of cells in all batches, and 0.01 is selected in the experiment; For the X samples of the t-th batch t is the mean and variance of the cell feature vectors of the c-th class; ∈ is the random perturbation term of the variance, and its value range is [0,1]. In a training cycle, μ c and δ c will be continuously iterated; after that, calculate the feature vector of the original real sample x t and the mean of the feature vectors of the sampled real result samples x c to obtain the class prototype Z gt of each type of sample after supplementation, which is used to calculate the inter-class difference in subsequent steps.

[0053] To effectively use the generated virtual features and avoid underfitting or overfitting, an effective adaptive sampling probability mechanism is adopted here, that is, if the generated virtual features can improve the performance of the corresponding class, the sampling probability is increased, otherwise it will be decreased, as follows:

[0054] First, initialize the sampling probability:

[0055]

[0056] In the formula, S c is the scaling factor estimated each time, and N c is the number of samples of the c-th class.

[0057] Secondly, in each training cycle, use the density-based clustering algorithm to divide multiple similar classes into the same group, and update the sampling probability p c in units of the group. Among them, the distance calculation formula of the density-based clustering algorithm is:

[0058]

[0059] In the formula, d ij is the distance between class i and class j, μ i , σ i are the mean and variance of the feature vectors of the i-th class, and μ j , σ j are the mean and variance of the feature vectors of the j-th class.

[0060] Then, after each training cycle, use the loss of multiple classes within the group on the validation set as the standard to guide the update of the sampling probability. The update of the probability p c is as follows:

[0061] When the loss in the validation set increases, p c = min(1, p c ·α), and appropriately increase the sampling probability of this class; otherwise, p c = max(0, p c·β), appropriately reduce the sampling probability of this category. Among them, α and β are hyperparameters responsible for controlling the proportion of increase or decrease in the sampling probability each time, and the values here are 1.1 and 0.9.

[0062] S4. To better guide the alignment of the class prototype distribution of the model detection results with the true class distribution, in this embodiment, a posterior regularization mechanism based on category guidance is adopted to align the class prototype distribution of the detection results with the true distribution, and then to detect long-tailed blood cells, specifically as follows:

[0063] First, to more detailedly compare the class prototype distribution of the detection results with the true class distribution, the inter-class difference is used for comparison, and then a clearer decision boundary is obtained, as follows:

[0064]

[0065] In the formula, are the inter-class differences of the true distribution and the detection result distribution respectively, Z i and Z k are the class prototypes of Z det and Z gt respectively. If the number of class prototypes is n, the number of vectors of the inter-class difference is

[0066] Secondly, since the existing inter-class differences are high-dimensional vectors for discriminating different classes, then next, it is necessary to learn the underlying feature distribution of different decision boundaries through a density estimator Gaussian mixture distribution (GMM). In this embodiment, two different estimators, P gt and Q pd .

[0067] Then, by calculating the KL divergence, the two feature distributions are measured, and the loss L crl of the feature distribution is obtained:

[0068]

[0069] In the formula, P gt and Q pd are the GMM estimators corresponding to the true result class distribution and the detection result class distribution respectively, and are the inter-class differences of the true result and the detection result respectively.

[0070] In summary, combining the object detection loss, the total loss L total = L det + L crl is used to update the initial model for blood cell detection, guide the model training, and then realize the detection of long-tailed blood cells, where L det is the loss of the original object detection model.

[0071] Another embodiment provides an intelligent detection device for long-tail blood cells, and the specific processing flow is as Figure 2 shown, including:

[0072] An image acquisition module for acquiring blood cell images to be detected.

[0073] A blood cell detection module for obtaining blood cell detection frame localization and classification data by using a preset long-tail blood cell detection network.

[0074] A result display module for cropping individual cells from the preprocessed blood cell images according to the blood cell detection frame localization and classification data, obtaining the individual blood cell categories in the images to be detected and displaying them.

[0075] Yet another embodiment provides a computer-readable storage medium storing computer instructions for executing an intelligent detection method for long-tail blood cells.

[0076] Another embodiment provides an electronic device including at least one processor, and the processor is communicatively connected to a memory;

[0077] wherein, the memory stores instructions executable by the processor for executing an intelligent detection method for long-tail blood cells.

[0078] Therefore, the present invention adopts the above intelligent detection method, device, storage medium and electronic device for long-tail blood cells, effectively solves the long-tail distribution problem in multi-class blood cell target detection, and improves the detection accuracy of the model for long-tail blood cells.

[0079] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that: they can still modify or equivalently replace the technical solutions of the present invention, and these modifications or equivalent replacements cannot make the modified technical solutions deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. An intelligent detection method for long-tail blood cells, characterized in that: The following steps are involved: S1. Obtain blood cell images of all bone marrow or peripheral blood samples, introduce a classic target detection model for positioning and classification, use blood cell image training data for fine-tuning, obtain an initial blood cell detection model, and use the initial blood cell detection model to detect the training data to obtain the detection results; S2. According to the data labels, single cell samples are cut out from the blood cell images, a single cell representation learning model is pre-trained, and based on the pre-trained single cell representation model, the classification detection results obtained by the initial model and the true single cell segmentation results are represented respectively; S3. Establish the Gaussian distribution of the class based on the representation of the real single-cell segmentation results, generate data through an adaptive sampling probability mechanism, and enhance the long-tail sample data; S4. Establish a category-guided posterior regularization mechanism to align the class distribution obtained by the initial blood cell detection model with the true class distribution, combine the target detection loss, train the initial blood cell detection model, obtain the final blood cell detection model, and then detect the blood cell image to obtain the recognition result.

2. The intelligent detection method for long-tail blood cells according to claim 1, characterized in that: The pre-trained single-cell representation learning model uses a balanced contrast learning model to train the cropped single-cell samples to obtain the trained backbone network and its parameters.

3. The intelligent detection method for long-tail blood cells according to claim 1, characterized in that: Establish a Gaussian distribution of the class and generate data as follows: x c =μ c +s c ⊙∈; In the formula, x c is the feature vector of the newly generated c-th cell, μ c , σ c is the total mean and variance of all batches of c-th type cells, m is a hyperparameter, is the t-th batch sample X t The mean and variance of the c-th cell feature vector, ∈ is the random perturbation term of the variance.

4. The intelligent detection method for long-tail blood cells according to claim 1, characterized in that: The adaptive sampling probability mechanism includes further grouping each category, calculating the total loss of each category in the group in the validation set, and then updating the sampling probability of each group of categories.

5. The intelligent detection method for long-tail blood cells according to claim 1, characterized in that: A category-guided posterior regularization mechanism is established, which includes obtaining the class mean obtained by the initial model and the real result class mean after fusing the newly generated samples, and then calculating the respective inter-class differences. The probability distribution of the underlying features is estimated using the Gaussian mixture distribution, and the class distribution loss is obtained through the KL divergence loss to evaluate the category guidance effect.

6. The intelligent detection method for long-tail blood cells according to claim 1, characterized in that: The final blood cell detection model includes utilizing a total loss function to constrain the initial blood cell detection model training; Among them, the expression of the total loss function is as follows: L total =L det +L crl ; Where, L total is the total loss, L det , L crl They are the loss of the original target detection model and the loss of feature distribution respectively.

7. The intelligent detection method for long-tail blood cells according to claim 1, characterized in that: The pre-trained single-cell representation learning model includes constrained training using a balanced supervised contrastive loss as follows: In the formula, |B y | is the total number of samples of category y, z i is the feature vector of a single cell sample, z p is the positive class, z k For the counterexample class, c y is the class prototype of the positive class of class y samples, B j is the total number of counterexample samples of category y, c j It is the class prototype of the counterexample class of class y samples.

8. An intelligent detection device for long-tailed blood cells, characterized in that: include: An image acquisition module, used for acquiring the blood cell image to be detected; A blood cell detection module is used to obtain blood cell detection frame positioning and classification data using a preset long-tail blood cell detection network; The result display module is used to crop single cells from the pre-processed blood cell image according to the blood cell detection frame positioning and classification data, obtain the single blood cell category in the image to be detected and display it.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to execute the intelligent detection method for long-tail blood cells according to any one of claims 1 to 7.

10. An electronic device, characterized in that: The device comprises at least one processor, wherein the processor is communicatively connected to the memory; Wherein, the memory stores instructions executable by the processor, and the instructions are used to execute the intelligent detection method for long-tail blood cells as described in any one of claims 1-7.

Images