Method and apparatus for automatic detection of blood cells in blood smear images

Abstract

This application proposes an automatic detection method and device for blood cells in blood smear images. It combines a local teacher model and an external teacher model to generate high-quality pseudo-labels for the originally extremely sparse labeled data, guiding the model to learn complete foreground concepts and effectively alleviating the problem of a large number of missed detections caused by insufficient labeling. It uses the text features of blood cell category names as domain-invariant anchors to guide the detector to generate visual representations with strong generalization, achieving excellent generalization performance in multiple visual domains. It introduces inter-text contrastive learning and inter-visual contrastive learning to promote clearer alignment between text features and visual features from multiple dimensions, improving generalization while ensuring discriminability.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to an automatic detection method and device for blood cells in blood smear images. Background Technology

[0002] Microscopic examination of blood smears is a crucial step in hematological diagnosis, but traditional manual microscopic methods have inherent limitations, being inefficient and labor-intensive. Blood cell target detection algorithms can automatically locate and count various blood cells in blood smear images, reducing the workload of laboratory personnel and providing diagnostic suggestions to hematologists. However, existing blood cell detection technologies face several challenges in practical applications. First, existing detection models can only detect a few cell types, limiting their application scenarios. Second, due to the vast number of blood cells in blood smears, the data used to train blood cell detectors is often extremely sparsely labeled, leading to a large number of missed detections. Finally, due to differences in staining conditions and slide preparation techniques among different hospitals, domain bias often exists between blood smears prepared by different hospitals; that is, a detector trained using data from one hospital may perform poorly on smear images from other hospitals, limiting its practicality in medical practice. Existing work has used language guidance to improve the domain generalization ability of detectors in everyday image domains, but due to the high similarity between blood cells, directly transferring this to blood cell detection tasks leads to a decrease in the detector's discrimination ability. Summary of the Invention

[0003] This application proposes an automatic detection method and device for blood cells in blood smear images, which can solve one of the problems existing in the background art.

[0004] To achieve the above objectives, this application adopts the following technical solution:

[0005] In a first aspect, an automatic detection method for blood cells in blood smear images is provided, including:

[0006] In the pseudo-label generation stage, the blood smear image is processed using a local teacher model to predict a first bounding box without a class; the grid point cue covering the entire image and the blood smear image are processed using an external teacher model to obtain a segmentation mask for each cell, and the bounding box of the segmentation mask is taken to obtain a second bounding box; based on the obtained first bounding box and second bounding box, clutter box elimination is performed to obtain pseudo bounding boxes; and a classifier is used to predict the class of the single-cell sub-image cropped from the pseudo bounding boxes to obtain pseudo labels.

[0007] Furthermore, during the detection phase, the regional features of the blood smear image are extracted; for category prediction, the regional features are mapped to the text feature dimension to obtain the target visual features; and, using the pseudo-label as supervision, the similarity between the target visual features and the category name text features is used to determine the category detection result of the blood cells.

[0008] Based on the above technical solutions, high-quality pseudo-labels are generated for the originally extremely sparse labeled data by combining local teacher models and external teacher models, guiding the model to learn complete foreground concepts and effectively alleviating the problem of a large number of missed detections caused by insufficient labeling. The text features of blood cell category names are used as domain-invariant anchors to guide the detector to generate visual representations with strong generalization, achieving excellent generalization performance in multiple visual domains. The introduction of inter-text contrastive learning and inter-visual contrastive learning promotes clearer alignment between text features and visual features from multiple dimensions, improving generalization while ensuring discriminability.

[0009] In one possible design approach of the first aspect, based on the obtained first bounding box and second bounding box, noise box elimination is performed to obtain a pseudo bounding box, specifically including:

[0010] For each second bounding box, the average gray value of its masked region and unmasked region are calculated respectively. If the average gray value of the masked region is higher than the average gray value of the unmasked region, the second bounding box is excluded to eliminate noise boxes caused by oversegmentation of the background and gaps.

[0011] Calculate the area of ​​all second bounding boxes. If a second bounding box with a smaller area is detected to be completely surrounded by a second bounding box with a larger area, discard the larger second bounding box to eliminate clutter caused by adjacent over-segmentation; and,

[0012] If a second bounding box is completely surrounded by any first bounding box, then the second bounding box is removed to avoid eliminating clutter caused by over-segmentation of the cell nucleus.

[0013] In one possible design approach of the first aspect, the regional features of the blood smear image are extracted by sequentially using a visual encoder and a path aggregation network (PAN) to extract the regional features.

[0014] In one possible design approach of the first aspect, the similarity sim between the target visual features and the category name text features is calculated as follows:

[0015]

[0016] in For normalization function, and A learnable scaling factor. The category name text features extracted by the text encoder. Visual features of the target extracted by the YOLO backbone network.

[0017] In one possible design of the first aspect, the automatic detection method for blood cells in blood smear images further includes:

[0018] Calculate the pairwise similarity between all categories of original text features to obtain the original similarity matrix. ;

[0019] Will All off-diagonal elements are multiplied by a scaling factor. The target similarity matrix is ​​obtained. :

[0020]

[0021] in It is the identity matrix;

[0022] The original text features are fine-tuned using an adapter, and the text similarity matrix between the fine-tuned features is calculated. ;

[0023] Using KL divergence loss for supervision near Distribution:

[0024]

[0025] in Used to transform a similarity matrix into a probability distribution;

[0026] Furthermore, the original text features are replaced with the finely tuned text features to obtain the category name text features.

[0027] In one possible design of the first aspect, the adapter is a learnable module comprising, in sequence, a linear downsampling layer, a nonlinear activation layer, and a linear upsampling layer.

[0028] In one possible design of the first aspect, the automatic detection method for blood cells in blood smear images further includes:

[0029] Flatten the visual features of all regions within a training batch;

[0030] Calculate the pairwise similarity between the flattened features to obtain the visual similarity matrix;

[0031] The region is assigned a category label based on the intersection-union ratio (IUU) between the feature extraction region and the labeled region. Regions of the same category are paired regions, while regions of different categories are unpaired regions.

[0032] Furthermore, the infoNCE loss is used to maximize the similarity between paired regions while minimizing the similarity between unpaired regions, making the clustering of visual representations clearer.

[0033] In one possible design of the first aspect, the categories of the blood cell detection dataset include primitive erythrocytes, early erythroblasts, intermediate erythroblasts, late erythroblasts, mature erythrocytes, cell mitosis, primitive granulocytes, early granulocytes, intermediate granulocytes, late granulocytes, band granulocytes, segmented neutrophils, eosinophils, basophils, immature lymphocytes, mature lymphocytes, immature monocytes, mature monocytes, immature plasma cells, mature plasma cells, megakaryocytes, naked nucleus megakaryocytes, fragmented cells, and platelets.

[0034] In one possible design of the first aspect, the local teacher model is a Region Boosting Network (RPN), Faster-RCNN, YOLO, or DINO, the external teacher model is a basic segmentation model (SAM), SegFormer, Mask2Former, or SEEM, and the classifier is a ResNet-50 model.

[0035] In a second aspect, an electronic device is provided, comprising: a processor, and a memory coupled to the processor, the memory for storing a computer program; the processor for executing the computer program stored in the memory such that the electronic device performs the automatic detection method for blood cells in a blood smear image as described in any possible implementation of the first aspect. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a flowchart of the overall method provided in the embodiments of this application. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0039] It should be noted that although functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification and the above-mentioned figures are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0040] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0041] Before introducing the embodiments of this application, a brief description of the current stage of technical research of this application will be given:

[0042] In the process of blood smear-assisted diagnosis, existing blood cell target detection technologies have the following drawbacks:

[0043] The range of detectable categories is limited. Existing publicly available blood cell testing datasets contain a very limited number of categories; for example, BCCD, LISC, and WBCDD each contain fewer than five categories, and they do not include clinically important categories such as plasma cells, erythroblasts, and megakaryocytes, thus restricting the application scenarios of detectors trained on them. Based on this situation, this embodiment constructs a large-scale dataset containing 24 categories for training, enabling blood cell detectors to be used in a wider range of applications.

[0044] Blood cell detection datasets are often extremely sparsely labeled, leading to a large number of missed detections. Because the number of blood cells in a blood smear is so vast, it's impractical to label all blood cells in the smear image. Annotators often only label a few representative cells, resulting in extremely sparse initial labeling. During training, only a few labeled cells are considered foreground, while most unlabeled cells are treated as background, severely misleading the detector's learning of foreground concepts and causing numerous missed detections. This embodiment proposes a dual-teacher pseudo-label automatic generation strategy, combining the advantages of external and local teacher models to generate high-recall, high-quality pseudo-labels, significantly alleviating the missed detection problem.

[0045] Significant domain bias exists between blood smear images obtained from different hospitals, and traditional closed-set detectors exhibit poor cross-domain performance. In real-world medical scenarios, due to differences in imaging equipment, staining procedures, magnification, and lighting conditions among different institutions, unavoidable domain differences exist in the blood smear images they produce. Traditional blood cell detection methods follow a closed-set setting, essentially learning the decision boundaries between predefined categories rather than true semantic representations. This fundamentally limits their generalization ability to out-of-distribution data. When faced with data from domains not seen during training, their detection performance is often very poor. This embodiment employs a language-guided detector to improve generalization across different visual domains, leveraging the cross-domain invariance of text features to bridge the gap between features from different visual domains.

[0046] Directly applying language guidance to blood cell detection tasks faces the problem of decreased discriminative power. Extracting textual features from blood cell category names as a classifier, allowing visual representations to cluster around them, can improve generalization, but it also reduces the detector's discriminative ability. This manifests in two ways: firstly, the high similarity between blood cell category names makes the text cluster centers very close, thus reducing classification performance; secondly, the visual features of different blood cell categories are also very similar, further hindering clear clustering of visual representations around textual features. To address this issue, this embodiment proposes a multi-contrast learning strategy to promote clearer alignment between textual and visual features, improving the detector's ability to discriminate fine-grained blood cell categories.

[0047] To address the aforementioned technical deficiencies, one embodiment of this application is as follows: Figure 1 As shown, a practical blood cell detection algorithm based on a dual-teacher model and discriminative language guidance is provided, specifically including:

[0048] 1. Dataset Construction. In this embodiment, to expand the detector's detectable range, a large-scale blood cell detection dataset containing 24 categories, called UniBCD-24, is constructed. These categories include primitive erythrocytes, early erythroblasts, intermediate erythroblasts, late erythroblasts, mature erythrocytes, mitotic cells, primitive granulocytes, early myelocytes, intermediate myelocytes, late myelocytes, band granulocytes, segmented neutrophils, eosinophils, basophils, immature lymphocytes, mature lymphocytes, immature monocytes, mature monocytes, immature plasma cells, mature plasma cells, megakaryocytes, naked megakaryocytes, fragmented cells, and platelets. This dataset uses digital images of blood smears from Xiangya Hospital of Central South University, annotated by professional hematologists. Due to the vast number of blood cells, the original annotations are extremely sparse. Therefore, this embodiment designs an effective dual-teacher pseudo-label generation strategy to provide strong supervision for the detector.

[0049] 2. Dual-Teacher Pseudo-Label Generation Strategy. Traditional sparse labeled object detection methods use the original partially labeled data to train a local teacher model, and then use the local teacher model to predict pseudo-labels for unlabeled objects. However, in extremely sparse scenes (such as blood smear images), the local teacher model cannot learn a complete foreground concept. Therefore, this embodiment introduces a basic segmentation model as an external teacher model to provide the detector with complete foreground prior knowledge. Furthermore, to better integrate the advantages of the two teacher models, this embodiment designs a hierarchical filtering fusion method to generate the final high-quality pseudo-labels. Specifically, the design is as follows:

[0050] 2.1 In this embodiment, the Original Labeled Region Cueing Network (RPN) is used as the local teacher model to predict unclassed bounding boxes for unlabeled cells. The advantage of the local teacher is that it is relatively accurate in predicting nucleated cells and has a decent recall rate. However, because it cannot learn a complete foreground concept, its recall rate for the large number of red blood cells and platelets is extremely low.

[0051] 2.2 In this embodiment, the Segmentation Base Model (SAM) is selected as the external teacher model to generate pseudo-boundaries for the large number of blood cells. Since the outlines of blood cells are clear and distinct, using SAM for panoramic segmentation of a blood smear can segment all blood cells in the smear image. Specifically, 32 evenly distributed points are selected along each edge of the image to form a 32×32 grid of points covering the entire image. These points are then fed into SAM along with the image to achieve panoramic segmentation, obtaining the segmentation mask for each cell. Subsequently, in this embodiment, bounding boxes are taken from these bounding boxes. The advantage of the external teacher model is that it exhibits extremely high recall for all types of cells. However, due to the inherent limitations of the segmentation task, it causes a large amount of redundancy during the prediction process, which can be categorized into the following four types: 1) Background oversegmentation: Bounding boxes are generated for large areas of blank background, because blank background areas are also segmented; 2) Adjacent oversegmentation: A large additional bounding box appears outside two intersecting red blood cell boxes, because the segmentation model treats two connected red blood cells as a single unit; 3) Nucleus oversegmentation. The nuclei of some nucleated cells will be highlighted separately because the nucleus differs significantly in color from the cytoplasm. SAM will treat them as a separate segmentation target and assign a mask accordingly. 4) Oversegmentation of gaps. Gaps between compact cells will be highlighted because gaps differ significantly from blood cells. During panoramic segmentation, these gaps will be segmented separately and assigned a bounding box.

[0052] 2.3 Hierarchical Fusion Filtering Method. To combine the advantages of the two teacher models, this embodiment proposes a hierarchical fusion filtering method to generate the final pseudo-box labels. The process can be divided into the following three steps:

[0053] 2.3.1 Gray-scale-based filtering. For each pseudo-boundary generated by SAM, the average gray-scale value of its masked and unmasked regions is calculated separately. If the average intensity of the masked region is higher than that of the unmasked region, the bounding box is determined to correspond to a bright intercellular gap or blank background area rather than a real cell, and is thus excluded. This operation can effectively eliminate noisy bounding boxes caused by oversegmentation of the background and gaps.

[0054] 2.3.2 Bounding Box Filtering. First, calculate the area of ​​all SAM boxes. If a smaller box is detected to be completely surrounded by a larger box, discard the larger outer box. This operation can effectively eliminate noisy boxes caused by over-segmentation of adjacent boxes.

[0055] 2.3.3 Fusion Filtering with RPN Boxes. If a SAM box is completely surrounded by any RPN box, the box is removed to avoid false detections in the nucleus region. This operation effectively eliminates extraneous boxes caused by oversegmentation of the nucleus.

[0056] 2.4 After obtaining the pseudo-boundary box, this embodiment uses a classification model to assign a pseudo-class to it. The specific process is as follows: First, a ResNet-50 model is trained as a classifier based on the single-cell sub-image cropped from the original annotation; then, the classifier is used to predict the class of the single-cell sub-image cropped based on the pseudo-boundary box.

[0057] 3. Language-Guided Detector Based on Multiple Contrast Learning. Traditional blood cell detectors follow a closed-set setting, essentially learning only the decision boundaries between predefined categories, resulting in poor cross-domain performance. This embodiment designs a language-guided detector based on multiple contrast learning, called C-YOLO-World. It uses text features as domain-invariant stable anchors, causing visual features to cluster around them, thereby enabling the detector to learn highly generalizable visual representations. In addition, to avoid the decrease in discrimination ability caused by language guidance, this embodiment introduces a multiple contrast learning strategy to widen the gap between text classifiers and promote clearer clustering. Specifically, the design is as follows:

[0058] 3.1 For the region features extracted by the visual encoder and the path aggregation network PAN, a regression head predicts the coordinate offset of the bounding box, and a projection layer maps the region features to the text feature dimension. Unlike traditional closed-set detectors that directly use a linear mapping layer to predict the target category, this embodiment uses the cosine similarity between the category name and the target visual features as the classifier. For the category name text features extracted by the CLIP text encoder... And the target visual features extracted by the YOLO backbone network The similarity between them can be calculated using the following formula:

[0059]

[0060] in For normalization function, and This is a learnable scaling factor. During inference, the category name with the highest similarity to blood cells is selected as its category. In this paradigm, visual representations continuously move closer to the corresponding text features in the feature space (similarity between categories increases, and the angle in the feature space decreases). Text features are invariant and stable across different visual domains and can serve as "stable anchors." The visual representations learned through this alignment inherit the domain generalization ability of text features, thereby improving the detector's cross-domain generalization ability.

[0061] 3.2 Inter-text Comparative Learning. Blood cell category names naturally exhibit high similarity, resulting in very high similarity between text features encoded by the CLIP text encoder. This leads to extremely close classifier distances, weakening the classification head's discriminative performance. This embodiment proposes an inter-text comparative learning strategy without compromising the original semantic relationships between categories. The implementation details are as follows: First, calculate the pairwise similarity between the original text features of all categories to obtain the original similarity matrix. Then All off-diagonal elements are multiplied by a scaling factor. The process of obtaining the target similarity matrix can be represented as follows:

[0062]

[0063] in The identity matrix is ​​then used. The original text features are then fine-tuned using an adapter, which is a learnable module containing a linear downsampling layer, a non-linear activation layer, and a linear upsampling layer. The text similarity matrix between the fine-tuned features is then calculated. Finally, KL divergence loss is used for supervision. near The distribution of this process can be represented as:

[0064]

[0065] in This is used to transform the similarity matrix into a probability distribution. In this embodiment, fine-tuned text features replace the original text features, and the similarity calculated with visual features is used as the classifier. Under this paradigm, the similarity between fine-tuned text features learns towards the scaled similarity, effectively widening the gap between classifiers for different categories of text, thereby improving discrimination performance and enhancing classification ability.

[0066] 3.3 Visual Contrast Learning. The visual features of different blood cell categories are also very similar, further hindering clear clustering of visual representations around text features. This embodiment introduces a visual contrast learning method to amplify the differences in visual features between different categories. The implementation details are as follows: First, all region visual features within a training batch are flattened, and then pairwise similarity is calculated to obtain a visual similarity matrix. The intersection-union ratio (IUU) of the feature extraction region and the labeled region is used as the basis for assigning category labels to regions. Regions of the same category are paired regions, while regions of different categories are unpaired regions. Finally, the infoNCE loss is used to maximize the similarity between paired regions while minimizing the similarity between unpaired regions, making the clustering of visual representations clearer and thus improving discrimination performance.

[0067] According to the technical solution of this embodiment, compared with the prior art, this embodiment mainly makes the following key improvements:

[0068] A. Construction of a large-scale blood cell detection dataset: This embodiment constructs a large-scale detection dataset containing 24 blood cell categories, which greatly expands the detection range of the blood cell detector and makes it applicable to a wider range of medical practice scenarios.

[0069] B. Introduction of a Dual-Teacher Pseudo-Label Generation Strategy: This embodiment addresses the extremely sparse labeling of the blood cell detector dataset by using both a local teacher model and an external teacher model to generate pseudo-labels. A three-step filtering fusion method is designed to combine the advantages of both teacher models. Compared to existing sparsely labeled object detection methods that only use local teachers, this method introduces an external teacher model to provide more complete foreground prior knowledge and uses the local teacher model to calibrate the external teacher model. This results in pseudo-labels with higher recall and more completeness, significantly improving the false negative problem.

[0070] C. First application of language guidance to blood cell target detection task: Unlike traditional methods that use fixed linear mapping layers to classify blood cells, this embodiment uses the similarity between text features and target region features as the classification basis, and uses language guidance as a domain-invariant anchor point to encourage the learning of visual representations with stronger domain generalization, so that the detector can also show excellent detection performance when faced with blood smear images prepared by different hospitals.

[0071] D. Multiple Contrast Learning Supervision Ensures Detector Discriminative Ability: To better adapt language guidance to fine-grained blood cell classification, this embodiment proposes a multiple contrast learning strategy to promote better alignment between text features and visual features. This method effectively widens the gap between text classifiers through KL divergence loss and makes visual features more clearly clustered through infoNCE loss, thereby improving the discriminative ability of the language guidance detector and reducing confusion and illusions.

[0072] Compared with existing technologies, this embodiment, especially in the area of ​​automated blood cell detection, has the following significant advantages:

[0073] A. Reduce reliance on expert resources and improve diagnostic efficiency: This embodiment significantly reduces reliance on expert resources by using a fully automated blood cell detection algorithm, thereby improving the efficiency of blood smear microscopic examination in hematology clinical practice.

[0074] B. Expanding the detectable range of blood cell detectors: This embodiment constructs a large-scale detection dataset containing 24 blood cell categories, covering previously unseen but clinically important categories such as plasma cells, erythroblasts, and megakaryocytes, thus broadening the practical application scenarios of blood cell detectors.

[0075] C. Achieving high recall with minimal labeling cost: This embodiment combines local and external teacher models to generate high-quality pseudo-labels for the originally extremely sparse labeled data, guiding the model to learn complete foreground concepts, effectively alleviating the problem of a large number of missed detections caused by insufficient labeling, and helping the detector to achieve more accurate counting and analysis.

[0076] D. Introducing Language Guidance to Improve Cross-Domain Detection Performance: To address the domain bias caused by differences in slide preparation from different hospitals, this embodiment uses the textual features of blood cell category names as domain-invariant anchor points to guide the detector in generating highly generalized visual representations. It achieves excellent generalization performance in multiple visual domains (especially unseen domains), which is of great significance for the practical application and promotion of the detector.

[0077] E. Enhance the ability to distinguish fine-grained blood cell categories: By introducing inter-textual and inter-visual contrastive learning, we promote clearer alignment between textual and visual features from multiple dimensions, thereby improving generalization while ensuring discriminability.

[0078] The effectiveness of the method in this embodiment is illustrated below using experimental data.

[0079] Table 1: Comparison results of different detection methods on the UniBCD-24 dataset under different supervised paradigms. The first column represents existing sparsely labeled detection methods, the second column represents closed-set detection methods, the third column represents text hint detection methods, and the fourth column represents the text hint detection method based on multiple contrastive learning proposed in this embodiment. AP represents average precision, reflecting the accuracy of detection; AR represents average recall, reflecting the ability to detect all elements without missing any. , and The recall rates for red blood cells, nucleated cells, and nucleated blood cells are shown respectively. The results demonstrate that the recall rates of all detection methods are significantly improved after adopting the dual-teacher pseudo-label generation strategy proposed in this embodiment, especially for red blood cells and blood cells, exceeding the recall rates of existing sparse labeling detection methods. The C-YOLO-World proposed in this embodiment also outperforms other detection methods.

[0080] Table 1

[0081]

[0082] Table 2: Comparison of cross-domain generalization ability on unseen domain datasets. WBCDD, LISC, and Raabin-det are publicly available blood cell detection datasets, whose category settings are covered by the dataset of this invention, but their visual domains differ from those of this invention's dataset; therefore, they are used to evaluate generalization ability when facing unseen domains. CST-YOLO, MFDS-DETR, and DINO are existing closed-set blood cell detection methods, while MM-GDINO and YOLO-World are existing general-domain text prompt detection methods. The C-YOLO-World proposed in this invention achieves the best performance among these methods, proving the effectiveness of this invention.

[0083] Table 2

[0084]

[0085] Alternative solution to this embodiment:

[0086] For the local teacher model, this embodiment uses the Region Boosting Network (RPN), which can be replaced by detectors such as Faster-RCNN, YOLO, and DINO.

[0087] For the external teacher model, this embodiment uses the basic segmentation model SAM, which can be replaced by segmentation models such as SegFormer, Mask2Former, and SEEM.

[0088] For the language bootstrap detector architecture, other language bootstrap detectors can be used as alternatives, such as GLIP, DetCLIP, Grounding DINO, etc.

[0089] For the text encoder, this embodiment uses the pre-trained CLIP text encoder, but it can be replaced by text encoders such as bioCLIP, ALIGN, and LLaMa.

[0090] The practical blood cell detection algorithm based on the dual-teacher model and discriminative language guidance in this embodiment can be used for automatic detection, localization, classification, and counting of blood cells in blood smear images, thereby assisting in hematological medical diagnosis.

[0091] This application also provides an electronic device, including: a processor, and a memory coupled to the processor, the memory being used to store a computer program; the processor being used to execute the computer program stored in the memory, so that the electronic device performs the method as described in any of the above embodiments.

[0092] Electronic devices can be computing devices such as desktop computers, laptops, handheld computers, and cloud servers. These electronic devices may include, but are not limited to, processors and memory.

[0093] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the electronic device, connecting various parts of the device via various interfaces and lines.

[0094] The memory can be used to store the computer program, and the processor implements various functions of the electronic device by running or executing the computer program stored in the memory and calling the data stored in the memory.

[0095] The memory may primarily include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function, etc.; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0096] This application also provides a storage medium, which is a computer-readable storage medium. The computer program is stored in the computer-readable storage medium, and when executed by a processor, the computer program can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0097] This application also provides a computer program product, including: a computer program or instructions that, when the computer program or instructions are run on a computer, cause the computer to perform any of the above possible implementation methods.

[0098] The above description is the preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications are also considered to be within the scope of protection of this application.

Claims

1. A method for automatic detection of blood cells in a blood smear image, characterized by, The method described is based on a large-scale blood cell testing dataset containing 24 categories, including: In the pseudo-label generation stage, a local teacher model (Region Enhancement Network, RPN) is used to process the blood smear image to predict a first bounding box without a class. An external teacher model (SAM) is used to process grid point cues covering the entire image and the blood smear image to obtain a segmentation mask for each cell. A bounding box is then taken from the segmentation mask to obtain a second bounding box. Based on the obtained first and second bounding boxes, noise box removal is performed to obtain pseudo-bounding boxes. Finally, a classifier is used to predict the class of the single-cell sub-image cropped from the pseudo-bounding boxes to obtain pseudo-labels. During the detection phase, the regional features of the blood smear image are extracted; for category prediction, the regional features are mapped to the text feature dimension to obtain the target visual features; and, using the pseudo-label as supervision, the similarity between the target visual features and the category name text features is used to determine the category detection result of the blood cells. Based on the obtained first and second bounding boxes, noise box removal is performed to obtain pseudo bounding boxes, specifically including: For each second bounding box, the average gray value of its masked region and unmasked region are calculated respectively. If the average gray value of the masked region is higher than the average gray value of the unmasked region, the second bounding box is excluded to eliminate noise boxes caused by oversegmentation of the background and gaps. Calculate the area of ​​all second bounding boxes. If a second bounding box with a smaller area is detected to be completely surrounded by a second bounding box with a larger area, discard the larger second bounding box to eliminate clutter caused by adjacent over-segmentation; and, If a second bounding box is completely surrounded by any first bounding box, then the second bounding box is removed to eliminate clutter caused by oversegmentation of the cell nucleus. The similarity sim between the target visual features and the category name text features is calculated in the following way: wherein is a normalization function, and is a learnable scaling factor, is the class name text feature extracted by the text encoder, is the target visual feature extracted by the YOLO backbone network; The automatic detection method for blood cells in blood smear images further includes: Calculate the pairwise similarity between all categories of original text features to obtain the original similarity matrix. ; Will All off-diagonal elements are multiplied by a scaling factor. The target similarity matrix is ​​obtained. : in It is the identity matrix; The original text features are fine-tuned using an adapter, and the text similarity matrix between the fine-tuned features is calculated. ; Using KL divergence loss for supervision near Distribution: in Used to transform a similarity matrix into a probability distribution; and The original text features are replaced with the finely tuned text features to obtain the category name text features; The automatic detection method for blood cells in blood smear images further includes: Flatten the visual features of all regions within a training batch; Calculate the pairwise similarity between the flattened features to obtain the visual similarity matrix; The region is assigned a category label based on the intersection-union ratio (IUGR) between the feature extraction region and the labeled region. Regions of the same category are paired regions, while regions of different categories are unpaired regions. The infoNCE loss is used to maximize the similarity between paired regions while minimizing the similarity between unpaired regions, making the clustering of visual representations clearer.

2. The automatic detection method for blood cells in blood smear images as described in claim 1, characterized in that, The regional features of the blood smear image are extracted by sequentially using a visual encoder and a path aggregation network (PAN).

3. The automatic detection method for blood cells in blood smear images as described in claim 1, characterized in that, The adapter is a learnable module comprising, in sequence, a linear downsampling layer, a nonlinear activation layer, and a linear upsampling layer.

4. The automatic detection method for blood cells in blood smear images as described in claim 1, characterized in that, The categories of the blood cell test dataset include primitive erythrocytes, early erythrocytes, intermediate erythrocytes, late erythrocytes, mature erythrocytes, cell mitosis, primitive granulocytes, early granulocytes, intermediate granulocytes, late granulocytes, band granulocytes, segmented neutrophils, eosinophils, basophils, immature lymphocytes, mature lymphocytes, immature monocytes, mature monocytes, immature plasma cells, mature plasma cells, megakaryocytes, naked nucleus megakaryocytes, fragmented cells, and platelets.

5. An electronic device, characterized in that, The electronic device includes: a processor, and a memory coupled to the processor. The memory is used to store computer programs; The processor is configured to execute the computer program stored in the memory, so that the electronic device performs the automatic detection method for blood cells in blood smear images as described in any one of claims 1-4.

Images