Automatic detection and counting method and system for bone marrow megakaryocyte and medium

Abstract

The invention discloses a bone marrow megakaryocyte-oriented automatic detection and counting method and system and a medium. The method comprises the following steps: carrying out cell filtering on a local image obtained by segmenting a bone marrow smear panoramic image to extract a cell bounding box set; performing active sample selection on the cell bounding box set to generate a guide mask; performing active sample screening under the guidance of the guidance mask; pre-training a masking auto-encoder (MAE) based on the image samples obtained by active screening; and importing the encoder weight of the pre-trained masking auto-encoder MAE into the marrow megakaryocyte detection and classification model based on the masking auto-encoder MAE, and carrying out end-to-end training on the marrow megakaryocyte detection and classification model so as to realize marrow megakaryocyte detection and counting of the marrow megakaryocyte image data set of the target domain T. The invention aims to realize automatic identification of the bone marrow megakaryocyte, solve the problem of inter-class imbalance of the bone marrow megakaryocyte, and improve the classification precision and identification capability of the bone marrow megakaryocyte.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method, system and medium for automatic detection and counting of bone marrow megakaryocytes. Background Art

[0002] Megakaryocytes are an important component of the bone marrow. They are classified into multiple subtypes due to their diverse morphology. Accurately distinguishing between different types of megakaryocytes is crucial for diagnosing blood diseases such as myelodysplastic syndrome (MDS). However, the automatic classification of megakaryocytes faces many challenges. Traditional megakaryocyte classification methods usually rely on manual annotation by experts, which is a cumbersome, time-consuming, and inconsistent process. Although deep learning technology has significantly improved the accuracy and efficiency of medical imaging tasks in recent years, these methods still have certain limitations, especially in the classification of megakaryocyte images, which mainly face the following problems: (1) Bone marrow smear images contain a large number of megakaryocytes. Accurately labeling each megakaryocyte requires a lot of manpower, especially when distinguishing between multiple megakaryocyte subtypes. The large number and variety of megakaryocytes limit the popularity of automatic classification systems in practical applications. (2) Megakaryocyte images often have a long-tail distribution phenomenon, that is, some megakaryocyte types appear less frequently in the image, causing the model to tend to process more common cell types during training and ignore rare cell types. This imbalance between classes makes the model perform poorly when facing a minority class, affecting the classification accuracy and diagnostic effect. (3) Megakaryocyte images are usually simple, with relatively single image information and a large background, making it difficult to extract effective features using traditional image processing methods. Therefore, how to effectively extract key features from images with low information content is the key to improving the accuracy of automatic megakaryocyte classification.

[0003] In recent years, self-supervised learning, as an emerging unsupervised learning method, has become an effective means to solve the above problems by learning the potential features in the data without labeled data. The Masked Autoencoder (MAE) model proposed by He Kaiming (He K, Chen X, Xie S, et al. Masked autoencoders are scalable vision learners[C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2022: 16000-16009.) and others achieves self-supervised learning by partially masking the input image and then training the model to reconstruct only the information of the unmasked part. This method has been shown to have good performance in computer vision tasks, especially in image representation learning and pre-training models. However, despite the success of self-supervised learning methods in many tasks, their application to the classification of megakaryocyte images still faces many challenges, mainly in the following aspects: (1) Megakaryocyte images with low information content: Compared with other medical images, megakaryocyte images have less information content. In particular, it is difficult to extract effective features from simple megakaryocyte images in traditional classification methods. (2) The impact of inter-class imbalance: Megakaryocyte image datasets usually have a significant inter-class imbalance phenomenon, with fewer image samples of certain megakaryocyte types and more samples of common megakaryocyte types. This class imbalance phenomenon exacerbates the bias in the model training process, causing the model to perform poorly when dealing with minority categories. Summary of the invention

[0004] Technical problem to be solved by the present invention: In view of the above-mentioned problems in the prior art, a method, system and medium for automatic detection and counting of bone marrow megakaryocytes are provided. The present invention aims to realize automatic identification of bone marrow megakaryocytes, solve the problem of imbalance between bone marrow megakaryocytes, and improve the classification accuracy and identification ability of bone marrow megakaryocytes.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is: A method for automatically detecting and counting bone marrow megakaryocytes comprises the following steps: S1, performing cell filtering on the local image obtained by segmenting the bone marrow smear panoramic image to extract a cell boundary box set; S2, active sample selection on the set of cell bounding boxes to generate guidance masks; S3, actively screening samples of the local image under the guidance of the guidance mask; S4, pre-trained masked autoencoder MAE based on image samples obtained by active screening; S5, import the encoder weights of the pre-trained masked autoencoder MAE into the bone marrow megakaryocyte detection and classification model based on the masked autoencoder MAE, and perform end-to-end training on the bone marrow megakaryocyte detection and classification model to realize the bone marrow megakaryocyte detection and counting of the bone marrow megakaryocyte image dataset of the target domain T.

[0006] Optionally, step S1 includes: S1.1, normalizing the local images obtained by segmenting the bone marrow smear panoramic image to standardize the pixel values; S1.2, blurring the normalized local image to smooth the image and reduce noise; S1.3, using K-means clustering algorithm to perform preliminary pixel segmentation on the blurred local image, and grouping the pixels into a predetermined number of color clusters; S1.4, merging color clusters of the local image after preliminary pixel segmentation to reduce the number of color clusters; S1.5, converting the local image after merging the color clusters into the HSV color space; S1.6, generating a cell mask for capturing the cell color region based on a predefined HSV value range, and merging the cell masks through a logical OR operation to identify the cell region; S1.7, connected component analysis (CCA) was applied to mark the cell regions in the mask, and irrelevant regions were filtered out according to the minimum size and filling rate to generate a set of cell bounding boxes.

[0007] Optionally, generating a mask for capturing cell color areas based on a predefined HSV value range in step S1.6 includes: generating two monochrome masks for capturing purple and dark blue areas based on a predefined HSV value range, respectively, and then performing a logical "OR" operation on the two monochrome masks to obtain a composite mask as a cell mask for capturing cell color areas.

[0008] Optionally, step S2 includes: S2.1, normalizing the bone marrow megakaryocyte images in the bone marrow megakaryocyte image dataset of the target domain T; S2.2, extracting high-dimensional features from the region of interest image corresponding to the cell bounding box in the normalized cell bounding box set and the bone marrow megakaryocyte image in the bone marrow megakaryocyte image dataset of the target domain T using a pre-trained feature extraction model with shared parameters; S2.3, use K-means clustering algorithm to cluster high-dimensional features and count the number of samples contained in each cluster; S2.4, dynamically calculate the distance threshold based on the density of samples: , In the above formula, is the distance threshold, subscript Indicates Clusters, and They represent the lower and upper limits of the Euclidean distance of samples within a cluster, respectively. Indicates The number of samples in a cluster, Indicates the maximum number of samples in all clusters; S2.5, for each cluster based on the distance threshold Filtering includes: within each cluster, calculating each sample in the cluster With cluster center The Euclidean distance : , If the sample With cluster center The Euclidean distance Exceeding distance threshold , then the sample Consider invalid samples from the current cluster , and obtain the filtered cluster sample set: , in, Indicates The original sample set of clusters, Indicates The sample set after the clusters are filtered; S2.6, merge the filtered sample sets of each cluster into a filtered and optimized data set:

[0009] in, is the data set after filtering and optimization. is the total number of clusters obtained by clustering; S2.7, the area range of the mask generated by the filtered and optimized dataset guiding the masked autoencoder MAE is used as the obtained guidance mask.

[0010] Optionally, in step S3, active sample screening of the local image under the guidance of the guidance mask includes: S3.1, based on the guidance mask generated in step S2, taking the image block in the local image overlapping with the guidance mask area as the region of interest; S3.2, performing high-dimensional feature extraction on the image block in the region of interest, and the extraction method is consistent with the feature extraction method adopted in step S2.2 to ensure the consistency of the feature space; S3.3, calculating the Euclidean distance between the feature vector of the image block in the region of interest and the center of the high-confidence cluster corresponding to the guidance mask, and screening out samples with a smaller distance to the center of the high-confidence cluster; S3.4, retaining the screened samples to form a new high-confidence sample subset as the result of active sample screening.

[0011] Optionally, in step S4, when pre-training the masked autoencoder MAE based on the image samples obtained by active screening, the reconstruction error is used as the loss function to optimize the model parameters of the masked autoencoder MAE, and the functional expression of the reconstruction error is: , in, is the reconstruction error, represents the original pixel value of the i-th masked pixel in the input image, represents the reconstructed pixel value of the i-th masked pixel by the masked autoencoder MAE, represents the set of obscured pixels, is the total number of shaded pixels, Represents mean square error MSE.

[0012] Optionally, in step S5, when the encoder weights of the pre-trained masked autoencoder MAE are imported into the bone marrow megakaryocyte detection model for automatic detection and positioning of bone marrow megakaryocytes and imported into the bone marrow megakaryocyte classification model for classification and counting of bone marrow megakaryocytes, the bone marrow megakaryocyte detection model adopted includes a bone marrow megakaryocyte detection model MAE-RT-DETR and a bone marrow megakaryocyte classification model MAE-ViT. The bone marrow megakaryocyte detection model MAE-RT-DETR is based on the real-time detection converter RT-DETR, and the encoder weights of the masked autoencoder MAE are used to replace the encoder part in the original RT-DETR model. The bone marrow megakaryocyte classification model MAE-ViT is based on the visual converter ViT, and the encoder weights of the masked autoencoder MAE are used to replace the encoder part in the original ViT model.

[0013] In addition, the present invention also provides a system for automatically detecting and counting bone marrow megakaryocytes, comprising a microprocessor and a memory connected to each other, wherein the microprocessor is programmed or configured to execute the method for automatically detecting and counting bone marrow megakaryocytes.

[0014] In addition, the present invention also provides a computer-readable storage medium, in which a computer program or instruction is stored. The computer program or instruction is programmed or configured to execute the automatic detection and counting method for bone marrow megakaryocytes through a processor.

[0015] In addition, the present invention also provides a computer program product, including a computer program or instructions, which are programmed or configured to execute the method for automatic detection and counting of bone marrow megakaryocytes through a processor.

[0016] Compared with the prior art, the present invention has the following advantages: (1) By introducing self-supervised learning and active sample selection strategies, the present invention greatly reduces the reliance on a large amount of manually annotated data. In particular, in the process of annotating megakaryocyte images, the workload of experts is reduced, the annotation cost and time are significantly reduced, and the data utilization efficiency is improved. (2) By dynamically adjusting the threshold and giving priority to the tail data samples for training, the present invention significantly improves the classification accuracy of the model when facing a minority of megakaryocytes. This strategy effectively addresses the long-tail distribution problem in megakaryocyte image datasets, improves the detection and classification capabilities of rare megakaryocyte types, and makes up for the neglect of rare megakaryocyte types by existing methods. (3) The present invention combines an efficient cell filtering method with an active sample selection strategy based on clinical prior knowledge to effectively improve the overall performance of the model. By processing the noise, extracting features and clustering analysis of the image, background interference is minimized, ensuring that the model can extract more representative features from complex and low-information megakaryocyte images, thereby improving the accuracy of detection and classification. (4) Compared with other cell detection and classification algorithms, the present invention has higher detection accuracy and better detection effect. (5) The present invention can not only identify common megakaryocyte types, but also provide efficient classification results for some uncommon or low-frequency megakaryocyte types. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 Schematic diagram of the basic flow of the method of the embodiment of the present invention.

[0018] Figure 2 Schematic diagram of the principle of the method of the embodiment of the present invention.

[0019] Figure 3 Schematic diagram of the principle of active sample selection in an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] Considering the low amount of information and the problem of class imbalance in bone marrow megakaryocyte images, the present invention proposes an innovative self-supervised learning framework, which further improves the classification accuracy of megakaryocyte images by introducing an active sample selection strategy in the pre-training stage. Specifically, the present invention uses clinical prior knowledge to guide data selection, and introduces a dynamic threshold strategy in the process of self-supervised learning to give priority to the selection of samples from a minority category for training, thereby effectively addressing the class imbalance problem and improving the model's ability to identify rare megakaryocyte types. Figure 1 and Figure 2 As shown, the method for automatic detection and counting of bone marrow megakaryocytes in this embodiment includes the following steps: S1, performing cell filtering on the local image obtained by segmenting the bone marrow smear panoramic image to extract a cell boundary box set; S2, active sample selection on the set of cell bounding boxes to generate guidance masks; S3, actively screening samples of the local image under the guidance of the guidance mask; S4, pre-trained masked autoencoder MAE based on image samples obtained by active screening; S5, import the encoder weights of the pre-trained masked autoencoder MAE into the bone marrow megakaryocyte detection and classification model based on the masked autoencoder MAE, and perform end-to-end training on the bone marrow megakaryocyte detection and classification model to realize the bone marrow megakaryocyte detection and counting of the bone marrow megakaryocyte image dataset of the target domain T.

[0022] The local images in this embodiment are obtained by segmenting the panoramic image of the bone marrow smear, obtaining the panoramic image of the clinical bone marrow smear, and segmenting each image into local images. After deleting the image of the large blank area, the bone marrow megakaryocytes are annotated by doctors with rich experience as a data set for full supervised learning. The remaining unlabeled images are used for data training of the self-supervised model. Specifically, a total of 35 original bone marrow smear panoramic images were collected in this embodiment, covering various subtypes of bone marrow megakaryocytes. Each panoramic image is segmented into local images of about 1280×768 pixels. Based on these local images, images with large blank areas or poor quality are eliminated, and 20,000 images are finally selected as data sets for training and verification. After the annotation by professional doctors, 7577 bone marrow megakaryocytes are annotated. After the annotation is completed, it is divided into training set, test set and verification set according to the ratio of 7:2:1, and the remaining more than 10,000 unlabeled images are used as data for self-supervised learning.

[0023] In this embodiment, step S1 includes: S1.1, normalizing the local images obtained by segmenting the bone marrow smear panoramic image to standardize the pixel values; S1.2, blurring the normalized local image to smooth the image and reduce noise; S1.3, using K-means clustering algorithm to perform preliminary pixel segmentation on the blurred local image, and grouping the pixels into a predetermined number of color clusters; S1.4, merging color clusters of the local image after preliminary pixel segmentation to reduce the number of color clusters; S1.5, converting the local image after merging the color clusters into the HSV color space; S1.6, generating a cell mask for capturing the cell color region based on a predefined HSV value range, and merging the cell masks through a logical OR operation to identify the cell region; S1.7, connected component analysis (CCA) was applied to mark the cell regions in the mask, and irrelevant regions were filtered out according to the minimum size and filling rate to generate a set of cell bounding boxes.

[0024] The Gaussian blur function can effectively reduce image noise and improve image quality. Therefore, the blur processing in step S1.2 of this embodiment refers to Gaussian blur processing, and the function expression of Gaussian blur processing is: , in, is the pixel in the local image The RGB value after Gaussian blur processing, is the standard deviation of the RGB values ​​of each pixel in the specified window area in the local image. and In this embodiment, the blur kernel size used in Gaussian blur processing is 3×3, that is, is the standard deviation of the RGB values ​​of each pixel in the 3×3 window area in the local image. Gaussian blur is used to smooth the image and reduce noise. This denoising process can enhance the accuracy of subsequent region extraction and segmentation, thereby improving the cell detection effect.

[0025] In this embodiment, the use of the K-means clustering algorithm in step S1.3 to perform preliminary pixel segmentation on the blurred local image refers to clustering based on the distance between each pixel of the local image and the cluster center of the color cluster, and the calculation function expression of the distance between each pixel of the local image and the cluster center of the color cluster is: , , in, for and The distance is the local image The RGB value of each pixel, For the The RGB value of the cluster center of each color cluster, Indicates The set of all pixels in a color cluster; K-means clustering is an unsupervised algorithm that divides pixels into different color clusters by minimizing the distance between the sample and its corresponding cluster center, thereby simplifying the image content and enhancing the separation of cells from the background. As an optional implementation, the first step of clustering in this embodiment is to preliminarily divide the image into 20 color clusters to represent the main color areas in the image. The second step is to merge these 20 clusters into 10 clusters to reduce the number of clusters and enhance the prominence of the cell area. Through the above steps, the processed image can effectively reduce background interference while retaining key cell information, thereby making the visual separation of cells from the background more obvious. This method helps to simplify the image content, facilitate subsequent feature extraction and classification, and improve the accuracy of the model.

[0026] In this embodiment, the processed image is converted to the HSV color space to more intuitively define the color and facilitate the detection of specific areas. In step S1.6 of this embodiment, the mask for capturing the cell color area based on the predefined HSV value range is generated, including: generating two monochrome masks for capturing the purple and dark blue areas based on the predefined HSV value range respectively, and then performing a logical "or" operation on the two monochrome masks to obtain a composite mask as a cell mask for capturing the cell color area, and these cell areas can be extracted as regions of interest for subsequent processing.

[0027] Next, step S1.7 applies connected component analysis CCA to mark the cell regions in the mask and generates a set of cell bounding boxes by filtering out irrelevant regions based on the minimum size and fill rate. Specifically, for each connected region, its bounding box is calculated and regions that do not meet the minimum size requirement or have a fill rate lower than 70% are filtered out. This step can remove the background region and retain only the relevant cell regions. For each image, a set of n bounding boxes is finally generated. ,These bounding boxes are used for further bone marrow megakaryocyte detection and classification.

[0028] like Figure 3 As shown, step S2 of this embodiment includes: S2.1, normalize the bone marrow megakaryocyte images in the bone marrow megakaryocyte image dataset of the target domain T to eliminate channel bias; in the active sample selection process, the target domain T represents the image dataset to be analyzed. In order to ensure the stability and consistency of clustering, the target domain data is first normalized. The normalization process is as follows: , In the above formula, for The normalized result is is the pixel in the i-th row and j-th column of the c-th channel, and Respectively represent channels This formula normalizes pixel values ​​to zero mean and unit standard deviation, effectively eliminating channel bias.

[0029] S2.2, extracting high-dimensional features from the region of interest image corresponding to the cell bounding box in the normalized cell bounding box set and the bone marrow megakaryocyte image in the bone marrow megakaryocyte image dataset of the target domain T using a pre-trained feature extraction model with shared parameters; wherein the feature extraction model can select the required feature extraction model according to actual needs, for example, as an optional implementation, in this embodiment, a pre-trained ResNet model is used to extract high-dimensional features of the normalized image; S2.3, K-means clustering algorithm is used to cluster high-dimensional features and count the number of samples contained in each cluster; K-means algorithm updates the cluster center by iterative update to minimize the square distance between the samples in the cluster and the cluster center. , cluster center The calculation method is: , In the above formula, The feature vector representing the image, Representation Cluster The objective function achieves clustering by assigning each data point to its nearest cluster center and recalculating the cluster center. This process continues until convergence or the stopping criterion is met.

[0030] S2.4, dynamically calculate the distance threshold based on the density of samples: , In the above formula, is the distance threshold, subscript Indicates Clusters, and They represent the lower and upper limits of the Euclidean distance of samples within a cluster, respectively. Indicates The number of samples in a cluster, Represents the maximum number of samples in all clusters; after clustering is completed, the density of each sample is calculated based on the Euclidean distance between the feature vector of each bone marrow megakaryocyte image and the center of its cluster. In order to deal with the long-tail distribution problem in the data set, the distance threshold is dynamically adjusted based on the density of the sample. Specifically, the threshold will be different based on the number of samples and the difference in distribution within the cluster. Through this method, clusters with higher density will set higher thresholds, while clusters with lower density (i.e., minority category cells) will set lower thresholds, so that the tail data is preferentially selected for training, thereby improving the model's classification and detection capabilities for rare cell types. Through the above-mentioned active sample selection process, the present invention effectively addresses the class imbalance problem in the data, and can give priority to minority category samples in the pre-training stage, optimize the learning process of the model, and improve the accuracy of blood cell detection and classification; S2.5, for each cluster based on the distance threshold Filtering includes: within each cluster, calculating each sample in the cluster With cluster center The Euclidean distance : , If the sample With cluster center The Euclidean distance Exceeding distance threshold , then the sample Consider invalid samples from the current cluster , and obtain the filtered cluster sample set: , in, Indicates The original sample set of clusters, Indicates The sample set after the clusters are filtered; S2.6, merge the filtered sample sets of each cluster into a filtered and optimized data set:

[0031] in, is the data set after filtering and optimization. is the total number of clusters obtained by clustering, so that the distance threshold can be dynamically set based on the sample density to identify and preferentially select samples belonging to the long-tail data to obtain the filtered samples; S2.7, the area range of the mask generated by the masked autoencoder MAE guided by the filtered and optimized data set is used as the obtained guidance mask.

[0032] In step S3 of this embodiment, active screening of samples of the local image under the guidance of the guidance mask includes: S3.1, based on the guidance mask generated in step S2, the image block in the local image that overlaps with the guidance mask area is taken as the region of interest; S3.2, high-dimensional feature extraction is performed on the image block in the region of interest, and the extraction method is consistent with the feature extraction method used in step S2.2 to ensure the consistency of the feature space; S3.3, the Euclidean distance between the feature vector of the image block in the region of interest and the center of the high-confidence cluster corresponding to the guidance mask is calculated, and samples with a smaller distance to the center of the high-confidence cluster are screened out; S3.4, the screened samples are retained to form a new high-confidence sample subset as the result of active sample screening.

[0033] In this embodiment, when the masked autoencoder MAE is pre-trained based on the image samples obtained by active screening in step S4, the masked autoencoder (MAE) self-supervised large model is used for pre-training, and the input of the model is the unlabeled bone marrow megakaryocyte image obtained after processing. In this embodiment, when the masked autoencoder MAE is pre-trained based on the image samples obtained by active screening in step S4, the reconstruction error is used as the loss function to optimize the model parameters of the masked autoencoder MAE, and the functional expression of the reconstruction error is: , in, is the reconstruction error, represents the original pixel value of the i-th masked pixel in the input image, represents the reconstructed pixel value of the i-th masked pixel by the masked autoencoder MAE, represents the set of obscured pixels, is the total number of shaded pixels, Represents mean square error MSE.

[0034] In the upstream stage of this embodiment, Masked Autoencoder (MAE) is used for self-supervised pre-training. By masking parts of the input image and reconstructing these areas, MAE learns the low-level feature representation of the input data and extracts the potential structural information in the megakaryocyte image. Specifically, the megakaryocyte image is input into the MAE model, and the reconstruction error is used as the loss function to optimize the model parameters. In the downstream stage, the encoder weights pre-trained by the above-mentioned MAE are imported into the bone marrow megakaryocyte detection model for supervised learning to achieve the megakaryocyte detection and classification tasks. In step S5 of this embodiment, the encoder weights of the pre-trained masked autoencoder MAE are imported into the bone marrow megakaryocyte detection model for automatic detection and positioning of bone marrow megakaryocytes, and imported into the bone marrow megakaryocyte classification model for classification and counting of bone marrow megakaryocytes. Figure 2As shown, the bone marrow megakaryocyte detection model used includes a bone marrow megakaryocyte detection model MAE-RT-DETR and a bone marrow megakaryocyte classification model MAE-ViT. The bone marrow megakaryocyte detection model MAE-RT-DETR is based on a real-time detection transformer RT-DETR (Real-Time Detection Transformer), and the encoder weights of the masked autoencoder MAE are used to replace the encoder part in the original RT-DETR model. The bone marrow megakaryocyte classification model MAE-ViT is based on a vision transformer ViT (Vision Transformer), and the encoder weights of the masked autoencoder MAE are used to replace the encoder part in the original ViT model. In this embodiment, based on the pre-trained masked autoencoder MAE, the bone marrow megakaryocyte detection model MAE-RT-DETR and the bone marrow megakaryocyte classification model MAE-ViT are used as downstream models to detect and classify megakaryocytes; the MAE-RT-DETR model is trained to realize automatic detection and positioning of megakaryocytes; and the MAE-ViT model is trained to realize classification and counting of megakaryocytes. For panoramic images of bone marrow smears, this method can automatically detect and count megakaryocytes, provide diagnostic references for doctors in clinics, and achieve the purpose of computer-aided diagnosis. MAE-ViT is used for image classification tasks, and the high-level semantic information of megakaryocyte categories is learned by fine-tuning the pre-trained features. MAE-RT-DETR is used for target detection in images, and the location and category of megakaryocytes are directly output by integrating the tasks of bone marrow megakaryocyte region localization and classification into an end-to-end optimization problem. By combining the self-supervised learning of MAE and the supervised fine-tuning of MAE-ViT and MAE-RT-DETR, bone marrow megakaryocytes can be efficiently extracted and identified from large-scale image data. During the training process, the cross entropy loss function is used for classification tasks, and the IoU (Intersection over Union) loss function is used for target detection tasks. The model is gradually optimized through the end-to-end training process, which significantly improves the accuracy of megakaryocyte detection and classification. The bone marrow megakaryocyte image to be detected is input into the trained bone marrow megakaryocyte detection model MAE-RT-DETR and bone marrow megakaryocyte classification model MAE-ViT to obtain the corresponding megakaryocyte image inference results.

[0035] In order to verify the automatic detection and counting method for bone marrow megakaryocytes in this embodiment, the test set is input into the trained data set in step 4 to obtain the prediction result, and the prediction result is compared with the true label to obtain the detection accuracy of the model. The experimental results are as follows: when only ViT is used for classification, the model accuracy is 72.5%; after introducing MAE self-supervised pre-training, the model accuracy of MAE-ViT is improved to 80.6%; on this basis, after adding the cell screening method based on clinical prior knowledge proposed in this embodiment, the accuracy is improved to 82.4%; finally, after adding the active sample selection strategy of this embodiment, the model accuracy is further improved to 83.9%. Compared with other methods, the method of this embodiment performs better in classification tasks: the accuracy of the ResNet (He K, Zhang X, Ren S, et al. Deep residual learning forimage recognition[C] / / Proceedings of the IEEE conference on computer visionand pattern recognition. 2016: 770-778.) method is 75.7%, the accuracy of Moco v3 (He K, FanH, Wu Y, et al. Momentum contrast for unsupervised visual representationlearning[C] / / Proceedings of the IEEE / CVF conference on computer vision andpattern recognition. 2020: 9729-9738.) is 75.1%, and the accuracy of MIXMIM (Chen K, Liu Z, Hong L,et al. Mixed autoencoder for self-supervised visual representation learning[C] / / Proceedings of the IEEE / CVF conference on computer vision and patternrecognition. 2023: 22742-22751.) is 78.6%. In terms of detection tasks, when only the self-supervised pre-trained MAE-RT-DETR model is used (with IoU 50 as the index), the performance is 65.0%; after adding the active learning method of this embodiment, the IoU 50 index rises to 66.2%, further proving the superiority of the method of this embodiment.

[0036] In summary, considering that the amount of information in bone marrow megakaryocyte images is low and there is a problem of class imbalance, in order to make full use of the effective information in the sample and reduce the impact of negative samples, this embodiment proposes a novel self-supervised learning framework, which novelly performs active sample selection in the pre-training stage rather than in the downstream stage. Specifically, this embodiment extracts relevant data based on clinical prior knowledge and develops a dynamic threshold strategy for sample selection. This strategy aims to minimize the impact of irrelevant samples by evaluating the similarity between the sample and the target field, while dynamically setting the threshold to effectively deal with the long-tail class imbalance problem. The key innovation of the method of this embodiment is that by introducing active learning in the self-supervised pre-training stage rather than the downstream, the training samples are dynamically selected, thereby shortening the training time and reducing the impact of negative samples. A further improvement of this embodiment is that the distance threshold is dynamically set based on the sample density in the self-supervised framework to balance the tail data samples, thereby effectively alleviating the class imbalance problem. This enables the model to better identify rare cell types and significantly improves the overall detection and classification accuracy. The method of this embodiment first eliminates image noise through Gaussian blur and K-means clustering, which helps to smooth cell contours, highlight the core features of cells, eliminate background interference, and preliminarily generate masks in the HSV color space through clinical prior knowledge. Then, high-dimensional features are extracted through ResNet, and thresholds are dynamically set according to the long tail of the samples to screen out those long-tail data and edge samples, and focus on training to improve the classification accuracy of the model. The whole process includes the following steps: data set processing, cell filtering, active sample selection, model training, and test results. This embodiment is compatible with existing self-supervised learning methods and can significantly improve classification performance. Experimental results show that on the clinically collected bone marrow megakaryocyte image data set, which contains a more comprehensive set of subcategories, the present invention shows significant performance improvement compared to traditional technologies, highlighting its effectiveness and practical application potential.

[0037] In addition, this embodiment also provides a system for automatic detection and counting of bone marrow megakaryocytes, including a microprocessor and a memory connected to each other, wherein the microprocessor is programmed or configured to execute the method for automatic detection and counting of bone marrow megakaryocytes.

[0038] In addition, this embodiment also provides a computer-readable storage medium, in which a computer program or instruction is stored. The computer program or instruction is programmed or configured to execute the automatic detection and counting method for bone marrow megakaryocytes through a processor.

[0039] In addition, this embodiment also provides a computer program product, including a computer program or instructions, which are programmed or configured to execute the method for automatic detection and counting of bone marrow megakaryocytes through a processor.

[0040] Those skilled in the art should understand that the technical solutions provided by the embodiments of the present application may be in the form of methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the process Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0041] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A method for automatic detection and counting of bone marrow megakaryocytes, characterized in that: The steps include: S1, performing cell filtering on the local image obtained by segmenting the bone marrow smear panoramic image to extract a cell boundary box set; S2, active sample selection on the set of cell bounding boxes to generate guidance masks; S3, actively screening samples of the local image under the guidance of the guidance mask; S4, pre-trained masked autoencoder MAE based on image samples obtained by active screening; S5, import the encoder weights of the pre-trained masked autoencoder MAE into the bone marrow megakaryocyte detection and classification model based on the masked autoencoder MAE, and perform end-to-end training on the bone marrow megakaryocyte detection and classification model to realize the bone marrow megakaryocyte detection and counting of the bone marrow megakaryocyte image dataset of the target domain T.

2. The method for automatic detection and counting of bone marrow megakaryocytes according to claim 1, characterized in that: Step S1 includes: S1.1, normalizing the local images obtained by segmenting the bone marrow smear panoramic image to standardize the pixel values; S1.2, blurring the normalized local image to smooth the image and reduce noise; S1.3, using K-means clustering algorithm to perform preliminary pixel segmentation on the blurred local image, and grouping the pixels into a predetermined number of color clusters; S1.4, merging color clusters of the local image after preliminary pixel segmentation to reduce the number of color clusters; S1.5, converting the local image after merging the color clusters into the HSV color space; S1.6, generating a cell mask for capturing the cell color region based on a predefined HSV value range, and merging the cell masks through a logical OR operation to identify the cell region; S1.7, connected component analysis (CCA) was applied to mark the cell regions in the mask, and irrelevant regions were filtered out according to the minimum size and filling rate to generate a set of cell bounding boxes.

3. The method for automatic detection and counting of bone marrow megakaryocytes according to claim 2, characterized in that: In step S1.6, generating a mask for capturing the cell color area based on the predefined HSV value range includes: generating two monochrome masks for capturing the purple and dark blue areas based on the predefined HSV value range respectively, and then performing a logical "OR" operation on the two monochrome masks to obtain a composite mask as a cell mask for capturing the cell color area.

4. The method for automatic detection and counting of bone marrow megakaryocytes according to claim 1, characterized in that: Step S2 includes: S2.1, normalizing the bone marrow megakaryocyte images in the bone marrow megakaryocyte image dataset of the target domain T; S2.2, extracting high-dimensional features from the region of interest image corresponding to the cell bounding box in the normalized cell bounding box set and the bone marrow megakaryocyte image in the bone marrow megakaryocyte image dataset of the target domain T using a pre-trained feature extraction model with shared parameters; S2.3, use K-means clustering algorithm to cluster high-dimensional features and count the number of samples contained in each cluster; S2.4, dynamically calculate the distance threshold based on the density of samples: ， In the above formula, is the distance threshold, subscript Indicates Clusters, and They represent the lower and upper limits of the Euclidean distance of samples within a cluster, respectively. Indicates The number of samples in a cluster, Indicates the maximum number of samples in all clusters; S2.5, for each cluster based on the distance threshold Filtering includes: within each cluster, calculating each sample in the cluster With cluster center The Euclidean distance : ， If the sample With cluster center The Euclidean distance Exceeding distance threshold , then the sample Consider invalid samples from the current cluster , and obtain the filtered cluster sample set: ， in, Indicates The original sample set of clusters, Indicates The sample set after the clusters are filtered; S2.6, merge the filtered sample sets of each cluster into a filtered and optimized data set: in, is the data set after filtering and optimization. is the total number of clusters obtained by clustering; S2.7, the area range of the mask generated by the filtered and optimized dataset guiding the masked autoencoder MAE is used as the obtained guidance mask.

5. The method for automatic detection and counting of bone marrow megakaryocytes according to claim 1, characterized in that: In step S3, active screening of samples of the local image under the guidance of the guidance mask includes: S3.1, based on the guidance mask generated in step S2, the image block in the local image that overlaps with the guidance mask area is taken as the region of interest; S3.2, high-dimensional feature extraction is performed on the image block in the region of interest, and the extraction method is consistent with the feature extraction method used in step S2.2 to ensure the consistency of the feature space; S3.3, the Euclidean distance between the feature vector of the image block in the region of interest and the center of the high-confidence cluster corresponding to the guidance mask is calculated, and samples with a smaller distance to the center of the high-confidence cluster are screened out; S3.4, the screened samples are retained to form a new high-confidence sample subset as the result of active sample screening.

6. The method for automatic detection and counting of bone marrow megakaryocytes according to claim 1, characterized in that: In step S4, when pre-training the masked autoencoder MAE based on the image samples obtained by active screening, the reconstruction error is used as the loss function to optimize the model parameters of the masked autoencoder MAE. The function expression of the reconstruction error is: ， in, is the reconstruction error, represents the original pixel value of the i-th masked pixel in the input image, represents the reconstructed pixel value of the i-th masked pixel by the masked autoencoder MAE, represents the set of obscured pixels, is the total number of shaded pixels, Represents mean square error MSE.

7. The method for automatic detection and counting of bone marrow megakaryocytes according to claim 1, characterized in that: In step S5, when the encoder weights of the pre-trained masked autoencoder MAE are imported into the bone marrow megakaryocyte detection model for realizing automatic detection and positioning of bone marrow megakaryocytes and imported into the bone marrow megakaryocyte classification model for realizing classification and counting of bone marrow megakaryocytes, the bone marrow megakaryocyte detection model adopted includes a bone marrow megakaryocyte detection model MAE-RT-DETR and a bone marrow megakaryocyte classification model MAE-ViT. The bone marrow megakaryocyte detection model MAE-RT-DETR is based on the real-time detection converter RT-DETR, and the encoder weights of the masked autoencoder MAE are used to replace the encoder part in the original RT-DETR model. The bone marrow megakaryocyte classification model MAE-ViT is based on the visual converter ViT, and the encoder weights of the masked autoencoder MAE are used to replace the encoder part in the original ViT model.

8. An automatic detection and counting system for bone marrow megakaryocytes, comprising a microprocessor and a memory connected to each other, characterized in that: The microprocessor is programmed or configured to execute the method for automatic detection and counting of bone marrow megakaryocytes according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program or instruction stored therein, characterized in that: The computer program or instruction is programmed or configured to execute the method for automatic detection and counting of bone marrow megakaryocytes according to any one of claims 1 to 7 through a processor.

10. A computer program product comprising a computer program or instructions, characterized in that The computer program or instruction is programmed or configured to execute the method for automatic detection and counting of bone marrow megakaryocytes according to any one of claims 1 to 7 through a processor.

Images