A method, system and medium for automatic detection and counting of bone marrow megakaryocytes

Abstract

The application discloses a kind of marrow megakaryocyte automatic detection and counting method, system and medium, the present application method includes the local image obtained by segmenting marrow smear panorama image carries out cell filtering and extracts cell boundary box set;Active sample selection is generated to cell boundary box set guide mask;Under the guidance of guide mask, sample active screening is carried out;Pre-training mask auto-encoder MAE is obtained based on the image sample of active screening;The encoder weight of pre-trained mask auto-encoder MAE is introduced into marrow megakaryocyte detection classification model based on mask auto-encoder MAE, and marrow megakaryocyte detection classification model is trained end to end to be used to realize marrow megakaryocyte detection and counting of marrow megakaryocyte image data set of target domain T.The present application aims to realize marrow megakaryocyte automatic identification, solve marrow megakaryocyte interclass imbalance problem, improve the classification precision and recognition ability of marrow megakaryocyte.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to a method and system for automatic detection and counting of megakaryocytes, and a medium. BACKGROUND

[0002] Megakaryocytes are an important component of bone marrow, and have various morphologies and subtypes. Accurate classification of megakaryocytes is crucial for diagnosing blood diseases such as myelodysplastic syndrome (MDS). However, automatic classification of megakaryocytes faces many challenges. Traditional methods of megakaryocyte classification usually rely on expert manual annotation, which is tedious, time-consuming, and inconsistent. Although deep learning techniques have significantly improved the accuracy and efficiency of medical image tasks in recent years, these methods still have certain limitations, especially in the classification of megakaryocyte images. The main problems are as follows: (1) Bone marrow smear images contain a large number of megakaryocytes, and accurate labeling of each megakaryocyte requires a large amount of manpower, especially when distinguishing between multiple megakaryocyte subtypes. The large number and variety of megakaryocytes limit the popularization of automatic classification systems in practical applications. (2) Megakaryocyte images often have a long-tail distribution, meaning that some megakaryocyte types appear less frequently in images. This leads the model to focus on processing more common cell types during training, while ignoring rare cell types. This class imbalance problem makes the model perform poorly when faced with minority classes, affecting classification accuracy and diagnostic effectiveness. (3) Megakaryocyte images are usually simple, with limited image information and a large background, making it difficult to extract effective features through traditional image processing methods. Therefore, how to effectively extract key features from low-information content images is the key to improving the accuracy of automatic megakaryocyte classification.

[0003] In recent years, self-supervised learning, as a new unsupervised learning method, has become an effective means to solve the above problems by learning the potential features in the data without labeled data. Masked Autoencoder (MAE) model proposed by He K, Chen X, Xie S, et al. Masked autoencoders are scalable vision learners [C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2022: 16000-16009. through partial masking of the input image, then training the model to reconstruct only through the information of the unmasked part, so as to realize self-supervised learning. This method has been proven to have good performance in computer vision tasks, especially in image representation learning and pre-trained models. However, although the self-supervised learning method has been successful in many tasks, its application to the classification of megakaryocyte images still faces many challenges, mainly in the following aspects: (1) Low information content of megakaryocyte images: Compared with other medical images, megakaryocyte images have less information content, especially in traditional classification methods, it is difficult to extract effective features from simple megakaryocyte images. (2) Influence of class imbalance: Megakaryocyte image datasets usually have a significant class imbalance phenomenon, with fewer image samples of some megakaryocyte types and more samples of common megakaryocyte types. This class imbalance phenomenon exacerbates the bias in the model training process, making the model perform poorly when dealing with minority classes. SUMMARY

[0004] The technical problem to be solved by the present application is to provide a method and system for automatic detection and counting of bone marrow megakaryocytes, and a medium, which aims to realize automatic recognition of bone marrow megakaryocytes, solve the problem of class imbalance of bone marrow megakaryocytes, and improve the classification accuracy and recognition ability of bone marrow megakaryocytes.

[0005] To solve the above technical problems, the technical scheme adopted by the present application is:

[0006] A method for automatic detection and counting of bone marrow megakaryocytes, comprising the following steps:

[0007] S1, filtering the local image obtained by segmenting the bone marrow smear panoramic image to extract a set of cell boundary boxes;

[0008] S2, performing active sample selection on the cell bounding box set to generate a guidance mask;

[0009] S3, performing sample active screening on the local image under the guidance of the guidance mask;

[0010] S4, pre-training a masking autoencoder MAE based on the image samples obtained by active screening;

[0011] S5, importing the encoder weight of the pre-trained masking autoencoder MAE into the bone marrow megakaryocyte detection classification model based on the masking autoencoder MAE, and performing end-to-end training on the bone marrow megakaryocyte detection classification model to realize bone marrow megakaryocyte detection and counting on the bone marrow megakaryocyte image data set of the target domain T.

[0012] Optionally, step S1 comprises:

[0013] S1.1, normalizing the local image segmented from the bone marrow smear panoramic image to standardize the pixel value;

[0014] S1.2, performing blur processing on the normalized local image to smooth the image and reduce noise;

[0015] S1.3, using a K-means clustering algorithm to perform preliminary pixel segmentation on the blurred local image, and grouping the pixels into a predetermined number of color clusters;

[0016] S1.4, performing color cluster merging on the local image after preliminary pixel segmentation to reduce the number of color clusters;

[0017] S1.5, converting the local image after color cluster merging to HSV color space;

[0018] S1.6, generating a cell mask for capturing cell color regions based on a predefined HSV value range, and merging the cell masks by logical OR operation to identify cell regions;

[0019] S1.7, applying connected component analysis CCA to label the cell regions in the mask, and filtering out irrelevant regions according to the minimum size and filling rate to generate a cell bounding box set.

[0020] Optionally, generating a mask for capturing cell color regions based on a predefined HSV value range in step S1.6 comprises: generating two single-color masks for capturing purple and dark blue regions respectively based on a predefined HSV value range in advance, and then performing logical OR operation on the two single-color masks to obtain a composite mask as the cell mask for capturing cell color regions.

[0021] Optionally, step S2 comprises:

[0022] S2.1, normalizing the bone marrow megakaryocyte images in the bone marrow megakaryocyte image dataset of the target domain T;

[0023] S2.2, using the pre-trained feature extraction model with shared parameters to extract high-dimensional features from the cell boundary box corresponding region of interest image in the normalized cell boundary box set and the bone marrow megakaryocyte images in the bone marrow megakaryocyte image dataset of the target domain T respectively;

[0024] S2.3, using the K-means clustering algorithm to cluster the high-dimensional features and counting the number of samples contained in each cluster;

[0025] S2.4, dynamically calculating the distance threshold value based on the density of samples:

[0026] ,

[0027] In the above formula, is the distance threshold value, and the subscript represents the th cluster, and respectively represent the lower limit and upper limit of the Euclidean distance of the samples in the cluster, represents the number of samples in the th cluster, represents the maximum number of samples in all clusters;

[0028] S2.5, filtering each cluster based on the distance threshold value , including: in each cluster, calculating the Euclidean distance between each sample in the cluster and the cluster center :

[0029] ,

[0030] If the Euclidean distance between the sample and the cluster center exceeds the distance threshold value , the sample is considered as an invalid sample and is deleted from the current cluster , obtaining the filtered cluster sample set:

[0031] ,

[0032] wherein, represents the original sample set of the th cluster, represents the filtered sample set of the th cluster;

[0033] S2.6, recombine the filtered sample set of each cluster into a filtered and optimized data set:

[0034]

[0035] wherein, is the filtered and optimized data set, is the total number of clusters obtained by clustering;

[0036] S2.7, the region range of the mask generated by the masked autoencoder MAE under the guidance of the filtered and optimized data set is taken as the obtained guide mask.

[0037] Optionally, the sample active screening of the local image under the guidance of the guide mask in step S3 includes: S3.1, based on the guide mask generated in step S2, the image block in the local image that overlaps with the region of the guide mask is taken as the region of interest; S3.2, high-dimensional feature extraction is performed on the image block of the region of interest, and the extraction method is consistent with the feature extraction method adopted in step S2.2, to ensure the consistency of the feature space; S3.3, the Euclidean distance between the feature vector of the image block of the region of interest and the high-confidence cluster center corresponding to the guide mask is calculated, and the samples with smaller distance from the high-confidence cluster center are screened out; S3.4, the screened samples are retained to form a new high-confidence sample subset as the result of sample active screening.

[0038] Optionally, when the masked autoencoder MAE is pre-trained based on the image samples obtained by active screening in step S4, the reconstruction error is used as a loss function to optimize the model parameters of the masked autoencoder MAE, and the function expression of the reconstruction error is:

[0039] ,

[0040] wherein, is the reconstruction error, represents the original pixel value of the i-th occluded pixel in the input image, represents the reconstructed pixel value of the i-th occluded pixel by the masked autoencoder MAE, represents the set of occluded pixels, is the total number of the set of occluded pixels, represents the mean square error MSE.

[0041] Optionally, when the pre-trained encoder weights of the masking autoencoder MAE are imported into the bone marrow megakaryocyte detection model for automatic detection and positioning of the bone marrow megakaryocytes, and into the bone marrow megakaryocyte classification model for classification and counting of the bone marrow megakaryocytes, the bone marrow megakaryocyte detection model used includes a bone marrow megakaryocyte detection model MAE-RT-DETR based on a real-time detection transformer RT-DETR and obtained by replacing the encoder part in the original RT-DETR model with the encoder weights of the masking autoencoder MAE, and a bone marrow megakaryocyte classification model MAE-ViT based on a visual transformer ViT and obtained by replacing the encoder part in the original ViT model with the encoder weights of the masking autoencoder MAE.

[0042] In addition, the present application also provides a bone marrow megakaryocyte automatic detection and counting system comprising a microprocessor and a memory connected to each other, wherein the microprocessor is programmed or configured to execute the bone marrow megakaryocyte automatic detection and counting method.

[0043] In addition, the present application also provides a computer readable storage medium having a computer program or instructions stored therein, wherein the computer program or instructions are programmed or configured to execute the bone marrow megakaryocyte automatic detection and counting method by a processor.

[0044] In addition, the present application also provides a computer program product comprising a computer program or instructions, wherein the computer program or instructions are programmed or configured to execute the bone marrow megakaryocyte automatic detection and counting method by a processor.

[0045] Compared with existing technologies, the present invention has the following advantages: (1) By introducing self-supervised learning and active sample selection strategies, the present invention significantly reduces the reliance on a large amount of manually labeled data. Especially in the process of labeling megakaryocyte images, it reduces the workload of experts, significantly reduces labeling costs and time, and improves data utilization efficiency. (2) By dynamically adjusting the threshold, the present invention prioritizes the selection of tail data samples for training, which significantly improves the classification accuracy of the model when facing a few types of megakaryocytes. This strategy effectively addresses the long-tail distribution problem in megakaryocyte image datasets, improves the detection and classification capabilities of rare megakaryocyte types, and makes up for the neglect of rare megakaryocyte categories by existing methods. (3) The present invention combines efficient cell filtering methods with active sample selection strategies based on clinical prior knowledge, which effectively improves the overall performance of the model. By processing image noise, extracting features, and performing cluster analysis, background interference is minimized, ensuring that the model can extract more representative features from complex and low-information-content megakaryocyte images, thereby improving the accuracy of detection and classification. (4) Compared with other cell detection and classification algorithms, the present invention has higher detection accuracy and better detection effect. (5) This invention can not only identify common megakaryocyte types, but also provide efficient classification results for some uncommon or low-frequency megakaryocyte types. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of the basic process of the method in an embodiment of the present invention.

[0047] Figure 2 This is a schematic diagram illustrating the principle of the method in an embodiment of the present invention.

[0048] Figure 3 This is a schematic diagram illustrating the principle of active sample selection in an embodiment of the present invention. Detailed Implementation

[0049] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] In view of the low information quantity of the bone marrow megakaryocyte image and the class imbalance problem, the application proposes an innovative self-supervised learning framework, which further improves the classification accuracy of the megakaryocyte image by introducing an active sample selection strategy in the pre-training stage. Specifically, the application guides data selection through clinical prior knowledge and introduces a dynamic threshold strategy in the process of self-supervised learning, preferentially selects samples of minority classes for training, thereby effectively addressing the class imbalance problem and improving the model's recognition ability for rare megakaryocyte types. As shown in Figure 1 and Figure 2 The bone marrow megakaryocyte automatic detection and counting method of the embodiment faces the following steps:

[0051] S1, performing cell filtering on the local image obtained by segmenting the bone marrow smear panoramic image to extract a cell bounding box set;

[0052] S2, performing active sample selection on the cell bounding box set to generate a guide mask;

[0053] S3, performing sample active screening on the local image under the guidance of the guide mask;

[0054] S4, pre-training a masked autoencoder MAE based on the image samples obtained by active screening;

[0055] S5, importing the encoder weight of the pre-trained masked autoencoder MAE into a bone marrow megakaryocyte detection classification model based on the masked autoencoder MAE, and performing end-to-end training on the bone marrow megakaryocyte detection classification model to realize bone marrow megakaryocyte detection and counting on the bone marrow megakaryocyte image data set of the target domain T.

[0056] The local image in the embodiment is obtained by segmenting the bone marrow smear panoramic image. The panoramic image of the clinical bone marrow smear is obtained, and each image is segmented into a local image. After deleting the images with large blank areas, the bone marrow megakaryocytes in the images are labeled by doctors with rich experience as the data set for full supervision learning. The remaining unlabeled images are used for data training of the self-supervised model. Specifically, a total of 35 original bone marrow smear panoramic images are collected in the embodiment, covering various types of bone marrow megakaryocyte subtypes. Each panoramic image is segmented into a local image of about 1280x768 pixels. Based on these local images, images with large blank areas or poor quality are removed, and finally 20,000 images are selected as a data set for training and verification. After being labeled by professional doctors, 7577 bone marrow megakaryocytes are labeled. After the labeling is completed, the images are divided into a training set, a test set and a verification set according to a ratio of 7:2:1, and the remaining more than 10,000 unlabeled images are used as data for self-supervised learning.

[0057] In the embodiment, step S1 includes:

[0058] S1.1, normalizing the local image segmented from the bone marrow smear panoramic image to standardize the pixel values;

[0059] S1.2, performing blur processing on the normalized local image to smooth the image and reduce noise;

[0060] S1.3, using a K-means clustering algorithm to perform preliminary pixel segmentation on the blurred local image, grouping the pixels into a predetermined number of color clusters;

[0061] S1.4, performing color cluster merging on the preliminary pixel segmented local image to reduce the number of color clusters;

[0062] S1.5, converting the color cluster merged local image to HSV color space;

[0063] S1.6, generating a cell mask for capturing the color area of cells based on a predefined HSV value range, and merging the cell masks by logical OR operation to identify the cell area;

[0064] S1.7, applying connected component analysis CCA to label the cell area in the mask, and filtering out irrelevant areas according to the minimum size and fill rate to generate a set of cell bounding boxes.

[0065] The Gaussian blur function can effectively reduce image noise and improve image quality. Therefore, the blur processing in step S1.2 of the embodiment refers to Gaussian blur processing, and the function expression of the Gaussian blur processing is:

[0066] ,

[0067] wherein, is the RGB value of the pixel in the local image after Gaussian blur processing, is the standard deviation of the RGB values of each pixel in the specified window region of the local image, and represent the pixel coordinates, respectively. In the embodiment, the blur kernel size used in the Gaussian blur processing is 3x3, i.e. is the standard deviation of the RGB values of each pixel in the 3x3 window region of the local image, which smoothes the image and reduces noise through Gaussian blur. This denoising process can enhance the accuracy of subsequent region extraction and segmentation, thereby improving the cell detection effect.

[0068] In this embodiment, step S1.3, which uses the K-means clustering algorithm to perform preliminary pixel segmentation on the blurred local image, refers to clustering based on the distance between each pixel in the local image and the cluster center of the color cluster. The calculation function expression for the distance between each pixel in the local image and the cluster center of the color cluster is as follows:

[0069] ,

[0070] ,

[0071] in, for and distance, For the local image, the first The RGB values ​​of each pixel. For the first The RGB values ​​of the center of each color cluster. Indicates the first The set of all pixels in each color cluster; K-means clustering is an unsupervised algorithm that divides pixels into different color clusters by minimizing the distance between a sample and its corresponding cluster center, thereby simplifying image content and enhancing the separation of cells from the background. As an optional implementation, the first step of clustering in this embodiment initially segments the image into 20 color clusters to represent the main color regions in the image. The second step merges these 20 clusters into 10 clusters to reduce the number of clusters and enhance the prominence of cell regions. Through the above steps, the processed image can effectively reduce background interference while retaining key cell information, thus making the visual separation of cells from the background more obvious. This method helps to simplify image content, facilitates subsequent feature extraction and classification, and improves the accuracy of the model.

[0072] In this embodiment, the processed image is converted to the HSV color space to more intuitively define colors and facilitate the detection of specific regions. Step S1.6 of this embodiment, generating a mask for capturing cell color regions based on a predefined HSV value range, includes: pre-generating two monochrome masks for capturing purple and dark blue regions respectively based on the predefined HSV value range; then performing a logical OR operation on the two monochrome masks to obtain a composite mask, which is used as the cell mask for capturing cell color regions. These cell regions can be extracted as regions of interest for subsequent processing.

[0073] Next, step S1.7 applies connected component analysis CCA to label the cell regions in the mask and generates a set of cell bounding boxes by filtering out irrelevant regions according to the minimum size and fill rate. Specifically, for each connected region, its bounding box is calculated and regions that do not meet the minimum size requirement or have a fill rate lower than 70% are filtered out. This step can remove background regions and only keep relevant cell regions. For each image, a set of n bounding boxes is finally generated These bounding boxes are used for further bone marrow megakaryocyte detection and classification.

[0074] As shown in Figure 3 , the embodiment step S2 includes:

[0075] S2.1, normalizing the bone marrow megakaryocyte images in the target domain T bone marrow megakaryocyte image dataset to eliminate channel bias; in the active sample selection process, the target domain T represents the image dataset to be analyzed. In order to ensure the stability and consistency of clustering, the target domain data is first normalized. The normalization process is as follows:

[0076] ,

[0077] In the above formula, is the normalization result of , is the c-th channel i-th row j-th pixel point, and respectively represent the mean and standard deviation of the channel . This formula standardizes the pixel value to zero mean and unit standard deviation, effectively eliminating channel bias.

[0078] S2.2, using a pre-trained feature extraction model with shared parameters to extract high-dimensional features from the cell boundary box corresponding region of interest image in the normalized cell boundary box set and the bone marrow megakaryocyte image in the target domain T bone marrow megakaryocyte image dataset; wherein the feature extraction model can be selected according to actual needs, for example, as an optional implementation, a pre-trained ResNet model is used in the embodiment to extract high-dimensional features of the normalized image;

[0079] S2.3, using K-means clustering algorithm to cluster high-dimensional features and counting the number of samples contained in each cluster; K-means algorithm updates the cluster center by iteration to minimize the squared distance between the samples in the cluster and the cluster center. For each cluster , the calculation method of the cluster center is as follows:

[0080] ,

[0081] In the above formula, Represents the feature vector of an image. Cluster The objective function clusters data points by assigning each data point to its nearest cluster center and recalculating the cluster centers. This process continues until convergence or the stopping criterion is met.

[0082] S2.4, Dynamically calculate the distance threshold based on sample density:

[0083] ,

[0084] In the above formula, Distance threshold, subscript Indicates the first Clusters, and Let represent the lower and upper bounds of the Euclidean distance between samples within the cluster, respectively. Indicates the first The number of samples in each cluster This represents the maximum number of samples in all clusters. After clustering, the density of each sample is calculated based on the Euclidean distance between the feature vector of each bone marrow megakaryocyte image and the center of its cluster. To address the long-tail distribution problem in the dataset, the distance threshold is dynamically adjusted based on the sample density. Specifically, the threshold varies depending on the number of samples and the differences in cluster distribution. This method sets a higher threshold for higher-density clusters and a lower threshold for lower-density clusters (i.e., minority cell types), thus prioritizing tail data for training and improving the model's ability to classify and detect rare cell types. Through this proactive sample selection process, this invention effectively addresses the class imbalance problem in the data and prioritizes minority class samples during the pre-training stage, optimizing the model's learning process and improving the accuracy of blood cell detection and classification.

[0085] S2.5, for each cluster based on a distance threshold Filtering is performed, including: within each cluster, calculating the value of each sample within the cluster. With cluster center Euclidean distance :

[0086] ,

[0087] If the sample With cluster center Euclidean distance Exceeding the distance threshold Then the sample Considered an invalid sample from the current cluster The original sample set of the i-th cluster is represented as

[0088] ,

[0089] The original sample set of the i-th cluster is represented as The filtered sample set of the i-th cluster is represented as The filtered sample set of the i-th cluster is represented as

[0090] S2.6, the filtered sample set of each cluster is recombined into a filtered and optimized data set:

[0091]

[0092] The filtered and optimized data set is represented as The total number of clusters obtained by clustering is represented as , thereby dynamically setting the distance threshold based on the sample density to identify and preferentially select samples belonging to long-tail data to obtain filtered samples;

[0093] S2.7, the region range of the mask generated by the masked autoencoder MAE under the guidance of the filtered and optimized data set is used as the obtained guide mask.

[0094] The sample active screening of the local image under the guidance of the guide mask in step S3 of the embodiment includes: S3.1, based on the guide mask generated in step S2, the image block in the local image that overlaps with the region of the guide mask is taken as the region of interest; S3.2, high-dimensional feature extraction is performed on the image block of the region of interest, and the feature extraction method is consistent with the feature extraction method adopted in step S2.2, to ensure the consistency of the feature space; S3.3, the Euclidean distance between the feature vector of the image block of the region of interest and the high-confidence cluster center corresponding to the guide mask is calculated, and samples with smaller distance from the high-confidence cluster center are screened out; S3.4, the screened samples are retained to form a new high-confidence sample subset as the result of sample active screening.

[0095] In step S4 of the embodiment, the masked autoencoder (MAE) is pre-trained using the masked autoencoder (MAE) self-supervised large model, and the input of the model is the processed unlabeled bone marrow megakaryocyte image. In step S4 of the embodiment, the reconstruction error is used as the loss function to optimize the model parameters of the masked autoencoder MAE, and the function expression of the reconstruction error is:

[0096] ,

[0097] wherein,​​ is the reconstruction error, represents the original pixel value of the i-th occluded pixel in the input image, represents the reconstructed pixel value of the i-th occluded pixel by the Masked Autoencoder (MAE), represents the set of occluded pixels, is the total number of the set of occluded pixels, represents the mean square error (MSE).

[0098] In the upstream stage of the present embodiment, a Masked Autoencoder (MAE) is used for self-supervised pre-training. By masking part of the input image and reconstructing these areas, the MAE learns the low-level feature representation of the input data and extracts the potential structural information in the megakaryocyte image. Specifically, the megakaryocyte image is input into the MAE model, and the reconstruction error is used as the loss function to optimize the model parameters. In the downstream stage, the encoder weights of the above MAE pre-trained model are imported into the bone marrow megakaryocyte detection model for supervised learning to achieve the megakaryocyte detection and classification tasks. In step S5 of the present embodiment, when the pre-trained encoder weights of the Masked Autoencoder (MAE) are imported into the bone marrow megakaryocyte detection model for automatic detection and positioning of bone marrow megakaryocytes, and imported into the bone marrow megakaryocyte classification model for classification and counting of bone marrow megakaryocytes, as Figure 2As shown, the adopted bone marrow megakaryocyte detection model includes a bone marrow megakaryocyte detection model MAE-RT-DETR based on a real-time detection transformer RT-DETR and obtained by replacing the encoder part in the original RT-DETR model with the encoder weight of a masked autoencoder MAE, and a bone marrow megakaryocyte classification model MAE-ViT based on a vision transformer ViT and obtained by replacing the encoder part in the original ViT model with the encoder weight of a masked autoencoder MAE. In this embodiment, the pre-trained masked autoencoder MAE is used to detect and classify megakaryocytes by using the bone marrow megakaryocyte detection model MAE-RT-DETR and the bone marrow megakaryocyte classification model MAE-ViT as downstream models; the MAE-RT-DETR model is trained to realize automatic detection and positioning of megakaryocytes; and the MAE-ViT model is trained to realize classification and counting of megakaryocytes. For panoramic images of bone marrow smears, this method can automatically detect and count megakaryocytes, provide a diagnosis reference for doctors in the clinic, and achieve the purpose of computer-aided diagnosis. MAE-ViT is used for image classification tasks, and through fine-tuning of pre-trained features, high-level semantic information of megakaryocyte categories is learned. MAE-RT-DETR is used for target detection in images, and by integrating the tasks of positioning and classifying bone marrow megakaryocyte regions into an end-to-end optimization problem, the position and category of megakaryocytes are directly output. Through the combination of self-supervised learning of MAE and supervised fine-tuning of MAE-ViT and MAE-RT-DETR, bone marrow megakaryocytes can be efficiently extracted and recognized from large-scale image data. During the training process, a cross-entropy loss function is used for the classification task, and an IoU (Intersection over Union) loss function is used for the target detection task. Through the end-to-end training process, the model is gradually optimized, so that the accuracy of megakaryocyte detection and classification is significantly improved. The bone marrow megakaryocyte image to be detected is input into the trained bone marrow megakaryocyte detection model MAE-RT-DETR and the bone marrow megakaryocyte classification model MAE-ViT, and the corresponding megakaryocyte image inference result is obtained.

[0099] To verify the automatic detection and counting method for bone marrow megakaryocytes in the embodiment, the test set is input into the data set trained in step four to obtain a prediction result, and the prediction result and the true label are compared to obtain the detection accuracy of the model. The experimental results are as follows: when only ViT is used for classification, the model accuracy is 72.5%; after introducing MAE self-supervised pre-training, the model accuracy of MAE-ViT is improved to 80.6%; on this basis, after adding the cell screening method based on clinical prior knowledge proposed in the embodiment, the accuracy is improved to 82.4%; finally, after adding the active sample selection strategy in the embodiment, the model accuracy is further improved to 83.9%. Compared with other methods, the method in the embodiment performs better in the classification task: the ResNet (He K, Zhang X, Ren S, et al. Deep residual learning for image recognition [C] / / Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770-778.) method has an accuracy of 75.7%, the Moco v3 (He K, Fan H, Wu Y, et al. Momentum contrast for unsupervised visual representation learning [C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2020: 9729-9738.) method has an accuracy of 75.1%, and the MIXMIM (Chen K, Liu Z, Hong L, et al. Mixed autoencoder for self-supervised visual representation learning [C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2023: 22742-22751.) method has an accuracy of 78.6%. In terms of detection task, when only the self-supervised pre-trained MAE-RT-DETR model (with IoU 50 as the index) is used, the performance is 65.0%; and after adding the active learning method in the embodiment, the IoU 50 index rises to 66.2%, further proving the superiority of the method in the embodiment.

[0100] In summary, considering the low information content of bone marrow megakaryocyte images and the problem of class imbalance, in order to make full use of the effective information in the sample and reduce the influence of negative samples, the embodiment proposes a novel self-supervised learning framework, which actively selects samples in the pre-training stage instead of the downstream stage. Specifically, the embodiment extracts relevant data according to clinical prior knowledge and develops a dynamic threshold strategy for sample selection. This strategy aims to minimize the influence of irrelevant samples by evaluating the similarity between samples and the target domain, while dynamically setting the threshold to effectively handle the long-tail class imbalance problem. The key innovation of the embodiment method is to dynamically select training samples by introducing active learning in the self-supervised pre-training stage instead of the downstream, thereby shortening the training time and reducing the influence of negative samples. The further improvement of the embodiment is to dynamically set the distance threshold based on the sample density in the self-supervised framework, balance the tail data samples, and effectively alleviate the class imbalance problem. This enables the model to better recognize rare cell types, significantly improving the overall detection and classification accuracy. The embodiment method first eliminates image noise through Gaussian blur and K-means clustering, which helps to smooth the cell outline, highlight the core features of the cell, and eliminate background interference. Then, a mask is generated in the HSV color space based on clinical prior knowledge. High-dimensional features are extracted through ResNet, and the threshold is dynamically set according to the sample long tail to filter out long tail data and edge samples for focused training to improve the classification accuracy of the model. The entire process includes the following steps: data set processing, cell filtering, active sample selection, model training, and effect testing. The embodiment is compatible with existing self-supervised learning methods and can significantly improve classification performance. Experimental results show that on the clinical collected bone marrow megakaryocyte image data set containing a more comprehensive sub-class set, the invention shows significant performance improvement compared to traditional techniques, highlighting its effectiveness and practical application potential.

[0101] In addition, the embodiment also provides a bone marrow megakaryocyte automatic detection and counting system facing, which comprises a microprocessor and a memory connected to each other, and the microprocessor is programmed or configured to execute the bone marrow megakaryocyte automatic detection and counting method facing.

[0102] In addition, the embodiment also provides a computer readable storage medium, which stores a computer program or instructions programmed or configured to execute the bone marrow megakaryocyte automatic detection and counting method facing by a processor.

[0103] In addition, the embodiment also provides a computer program product, which comprises a computer program or instructions programmed or configured to execute the bone marrow megakaryocyte automatic detection and counting method facing by a processor.

[0104] Those skilled in the art will appreciate that embodiments of the present application can be readily used as a method, a system or a computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-readable storage media (including, but not limited to, disk memory, CD-ROMs, optical storage devices, etc.) embodying computer readable code. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense. The application is described with reference to flow diagrams and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and / or block of the flow diagrams and / or block diagrams, and combinations of flows and / or blocks in the flow diagrams and / or block diagrams, can be implemented by computer program instructions. Such computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 one or more flows and / or blocks Figure 1 one or more flows and / or blocks Figure 1 one or more flows and / or blocks Figure 1 one or more flows and / or blocks Figure 1 one or more flows and / or blocks Figure 1 one or more flows and / or blocks

[0105] The above description is only preferred embodiments of the application. The protection scope of the application is not limited to the above-mentioned embodiments. Any technical scheme falling within the concept of the application should be considered as falling within the protection scope of the application. It should be noted that some improvements and refinements made by those skilled in the art without departing from the principles of the application should also be considered as falling within the protection scope of the application.

Claims

1. A method for automatic detection and counting of bone marrow megakaryocytes, characterized by, The method comprises the following steps: S1, the local image obtained by segmenting the bone marrow smear panoramic image is subjected to cell filtering to extract a cell bounding box set; S2, the cell bounding box set is subjected to active sample selection to generate a guide mask; S3, the local image is subjected to sample active screening under the guidance of the guide mask; S4, a pre-trained masking autoencoder MAE is obtained based on the image samples obtained by active screening; S5, the encoder weight of the pre-trained masking autoencoder MAE is imported into a bone marrow megakaryocyte detection classification model based on the masking autoencoder MAE, and the bone marrow megakaryocyte detection classification model is trained end to end to realize bone marrow megakaryocyte detection and counting on the bone marrow megakaryocyte image data set of the target domain T; Step S2 comprises: S2.1, the bone marrow megakaryocyte images in the bone marrow megakaryocyte image data set of the target domain T are subjected to normalization processing; S2.2, the cell bounding box corresponding region of interest image in the cell bounding box set after normalization processing, and the bone marrow megakaryocyte image in the bone marrow megakaryocyte image data set of the target domain T are respectively extracted high-dimensional features using a pre-trained feature extraction model with shared parameters; S2.3, the high-dimensional features are clustered using a K-means clustering algorithm, and the number of samples contained in each cluster is counted; S2.4, the distance threshold is dynamically calculated based on the density of the samples: ， In the above formula, is a distance threshold, subscript denotes the th cluster, and denote the lower and upper bounds of the Euclidean distance of samples within the cluster, respectively, denotes the number of samples in the th cluster, denotes the maximum number of samples in all clusters; S2.5, for each cluster based on a distance threshold Filtering is performed including, within each cluster, computing the Euclidean distance of each sample within the cluster to the cluster center :​​ ， If the sample With cluster center Euclidean distance Exceeding the distance threshold Then the sample Considered an invalid sample from the current cluster After deleting the samples, we obtain the filtered cluster sample set: ， wherein, represents the original sample set of the th cluster, represents the filtered sample set of the th cluster; S2.6, the filtered sample set of each cluster is recombined into an optimized data set after filtering: wherein, is a filtered optimized dataset, is the total number of clusters obtained by clustering. S2.7, the region range of the mask generated by the masking autoencoder MAE under the guidance of the filtered and optimized data set is taken as the obtained guide mask.

2. The method for automated detection and counting of bone marrow megakaryocytes according to claim 1, wherein, Step S1 comprises: S1.1, the local image obtained by segmenting the bone marrow smear panoramic image is subjected to normalization processing to standardize the pixel value; S1.2, the local image after normalization processing is subjected to fuzzy processing to smooth the image and reduce noise; S1.3, the local image after fuzzy processing is subjected to preliminary pixel segmentation using a K-means clustering algorithm, and the pixels are grouped into a predetermined number of color clusters; S1.4, the local image after preliminary pixel segmentation is subjected to color cluster merging to reduce the number of color clusters; S1.5, the local image after color cluster merging is converted to HSV color space; S1.6, a cell mask for capturing cell color regions is generated based on a pre-defined HSV value range, and the cell masks are combined by logical OR operation to identify cell regions; S1.7, connected component analysis CCA is applied to label the cell regions in the mask, and a cell bounding box set is generated by filtering irrelevant regions according to the minimum size and filling rate.

3. The method for automated detection and counting of bone marrow megakaryocytes according to claim 2, wherein, In step S1.6, the mask for capturing cell color regions is generated based on a pre-defined HSV value range, which comprises: generating two single-color masks for capturing purple and dark blue regions based on a pre-defined HSV value range respectively, and then performing logical OR operation on the two single-color masks to obtain a composite mask as the cell mask for capturing cell color regions.

4. The method for automated detection and counting of bone marrow megakaryocytes according to claim 1, wherein, The sample active screening of the local image under the guidance of the guidance mask in step S3 includes: S3.1, based on the guidance mask generated in step S2, taking the image block in the local image that overlaps with the region of the guidance mask as a region of interest; S3.2, performing high-dimensional feature extraction on the image block of the region of interest, and the extraction method is consistent with the feature extraction method adopted in step S2.2, to ensure consistency of the feature space; S3.3, calculating the Euclidean distance between the feature vector of the image block of the region of interest and the high-confidence cluster center corresponding to the guidance mask, and screening out samples with smaller distance from the high-confidence cluster center; S3.4, retaining the screened samples to form a new high-confidence sample subset as the result of sample active screening.

5. The method for automated detection and counting of bone marrow megakaryocytes according to claim 1, wherein, In step S4, when the pre-trained masking autoencoder MAE is used to pre-train the masking autoencoder MAE based on the image samples obtained by active screening, the reconstruction error is used as a loss function to optimize the model parameters of the masking autoencoder MAE, and the function expression of the reconstruction error is: ， wherein, is the reconstruction error, denotes the original pixel value of the i-th occluded pixel in the input image, denotes the reconstructed pixel value of the i-th occluded pixel by the masking autoencoder MAE, denotes the set of occluded pixels, is the total number of the set of occluded pixels, denotes the mean squared error, MSE.

6. The method for automated detection and counting of bone marrow megakaryocytes according to claim 1, wherein, In step S5, when the encoder weight of the pre-trained masking autoencoder MAE is imported into the bone marrow megakaryocyte detection model for automatic detection and positioning of bone marrow megakaryocytes, and into the bone marrow megakaryocyte classification model for classification and counting of bone marrow megakaryocytes, the bone marrow megakaryocyte detection model used includes a bone marrow megakaryocyte detection model MAE-RT-DETR based on a real-time detection transformer RT-DETR and obtained by replacing the encoder part in the original RT-DETR model with the encoder weight of the masking autoencoder MAE, and a bone marrow megakaryocyte classification model MAE-ViT based on a visual transformer ViT and obtained by replacing the encoder part in the original ViT model with the encoder weight of the masking autoencoder MAE.

7. A system for automated detection and counting of bone marrow megakaryocytes comprising a microprocessor and memory interconnected, characterized in that, The microprocessor is programmed or configured to perform the bone marrow megakaryocyte automatic detection and counting method of any one of claims 1-6.

8. A computer-readable storage medium having stored therein a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to perform the bone marrow megakaryocyte automatic detection and counting method of any one of claims 1-6 by the processor.

9. A computer program product comprising computer programs or instructions, characterized in that, The computer program or instructions are programmed or configured to perform the bone marrow megakaryocyte automatic detection and counting method of any one of claims 1-6 by the processor.

Images