An improved single-stage bone marrow blood cell target recognition and counting method

Abstract

The application discloses a kind of based on improved single-stage marrow blood cell target identification and counting method, it is related to medical image processing technical field.The method steps are:S1, image acquisition and data preparation;S2, the construction and pre-processing of data set, and the data set after pre-processing is divided into training set, verification set and test set;S3, the initial model of improved single-stage target identification model is constructed;S4, the initial model is trained using training set and verification set, and marrow blood cell target identification and counting model is obtained;S5, marrow blood cell target identification and counting model is used to identify and locate marrow blood cell in test set image, output cell class and target frame coordinates, and counting is realized according to target frame number.The application solves the detection problem caused by the large scale variation of marrow cell, improves the classification precision, realizes the unification of high precision and high speed of marrow blood cell target identification and counting.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing, in particular to a bone marrow blood cell target recognition and counting method based on an improved single-stage. BACKGROUND

[0002] Bone marrow cell morphology examination is the gold standard for the diagnosis of blood system diseases such as leukemia and anemia. In the clinical diagnosis and treatment process, doctors need to observe bone marrow smears under a microscope and complete classification and counting of hundreds or even thousands of cells to provide core basis for disease diagnosis and condition assessment through accurate quantification of the proportion of each type of cell.

[0003] However, the current clinical bone marrow cell detection technology system still has many pain points, which seriously restricts the diagnosis and treatment efficiency and diagnosis consistency, which is specifically manifested as follows:

[0004] Manual microscopic examination is inefficient and highly subjective: traditional bone marrow cell classification and counting relies on manual microscopic examination by doctors, which is not only time-consuming and labor-intensive, but also highly dependent on the clinical experience and subjective judgment of the operator. Different doctors or even the same doctor at different times may have different judgments, making it difficult to ensure the standardization and consistency of diagnosis.

[0005] Bone marrow cell morphology is complex and difficult to identify: cells in bone marrow smears are densely distributed and often overlap and block each other. In addition, the morphology of granulocytes at different developmental stages (such as neutrophilic metamyelocytes and late myeloblasts) is highly similar, and the subtle structural differences are difficult to distinguish, which poses a great challenge to automatic cell recognition.

[0006] Limitations of algorithms: current mainstream cell detection algorithms cannot meet the accuracy and efficiency requirements. Traditional two-stage target detection algorithms (such as Faster R-CNN) have certain detection accuracy, but the inference speed is slow and cannot meet the real-time diagnosis requirements of clinical practice. Conventional single-stage algorithms (such as the original YOLO series) have insufficient feature extraction capability for small and dense targets, which easily leads to missed detection and false detection, and cannot meet the accuracy requirements of clinical diagnosis.

[0007] In summary, there is an urgent need for an automatic bone marrow blood cell recognition and counting method with high inference efficiency and high detection accuracy to break through the technical bottlenecks and promote the development of blood system disease diagnosis towards automation, standardization and precision. SUMMARY

[0008] The present application aims to provide a bone marrow blood cell target recognition and counting method based on an improved single-stage, which combines advanced data enhancement techniques, multi-scale feature fusion mechanisms and hybrid attention mechanisms to achieve efficient and accurate analysis of bone marrow cells.

[0009] To achieve the above object, the application provides a marrow blood cell target recognition and counting method based on an improved single-stage, and the specific steps are as follows:

[0010] Step S1, image acquisition and data preparation: using a high-resolution electron microscope, under 100x magnification, with the help of a marrow cell morphology examination system, collecting global image data of marrow granulocyte smears treated by standard Swiss staining method;

[0011] Step S2, construction and preprocessing of data set, and dividing the preprocessed data set into training set, validation set and test set;

[0012] Step S3, constructing an initial model of the improved single-stage target recognition model, wherein the initial model comprises a core network structure with a multi-scale feature fusion mechanism BiFPN and a mixed attention mechanism module CBAM;

[0013] Step S4, training the initial model using the training set and the validation set to obtain a marrow blood cell target recognition and counting model;

[0014] Step S5, using the marrow blood cell target recognition and counting model to recognize and locate marrow blood cells in the test set image, outputting cell categories and target frame coordinates, and counting according to the number of target frames.

[0015] Preferably, in step S2, the specific steps of constructing the data set are as follows:

[0016] Step S21, selecting an image with more than 200 cell units in the field of view as an effective sample, and retaining the cell samples distributed at the edges of the image;

[0017] Step S22, using three blood experts to cross-verify and vote-count the blood cells in the effective sample to generate an initial data set;

[0018] Step S23, preprocessing the initial data set, including image rotation, cutting, scaling, flipping, mixing, filling 0 pixels, splicing, and expanding the initial data set.

[0019] Preferably, in step S23, a new synthetic image sample is formed by data enhancement Mosaic, and the specific steps are as follows:

[0020] Step S231, randomly selecting four different images from the data set for random splicing;

[0021] Step S232, randomly selecting one target frame and adjusting the target frame from the original image position to the new coordinates in the new synthetic image;

[0022] Step S233, scaling and cropping operation is performed on the newly synthesized image to ensure that the target frame is fully visible in the new image and adjusted to the correct size;

[0023] Step S234, the new synthesized image is updated as a new sample, and the adjusted target frame position and size are updated as labels.

[0024] Preferably, in step S3, the Neck part of the improved single-stage target recognition model network introduces a multi-scale feature fusion mechanism BiFPN. The BiFPN mechanism realizes feature fusion through bidirectional and cross-layer connection, and the specific implementation is as follows:

[0025] A learnable weight is introduced at each fusion node Numerical stability is achieved through fast normalization, and the calculation formula is as follows:

[0026]

[0027] wherein, is the output feature map of the fusion node, is the input feature map, is a small value to prevent the denominator from being zero, i , j is the sum index.

[0028] Preferably, the CBAM module adopts a sequential application mechanism, including a channel attention module CAM and a spatial attention module SAM:

[0029] Channel attention module CAM: focus on "what" is meaningful in the feature channel;

[0030] Spatial attention module SAM: focus on "where" is meaningful in the feature space;

[0031] The CBAM module is implanted after multiple convolution blocks of the core network structure, which gives the network the ability to automatically select important features in the channel and spatial dimensions.

[0032] Preferably, in step S4, a combined loss function is used to guide model optimization during initial model training, and the combined loss function is composed of weighted sum of classification loss and positioning loss; the classification loss adopts binary cross entropy loss, which is used to judge whether each target frame contains blood cells and belongs to which category; the positioning loss adopts GIOU positioning loss, which is used to measure the difference between the predicted target frame and the real target frame, and improve the regression accuracy of the target frame;

[0033] The calculation formula of the combined loss function is as follows:

[0034] ;

[0035] wherein, is a combination loss, is a binary cross-entropy classification loss, is a GIOU positioning loss, is a weight coefficient.

[0036] Preferably, the initial model training process includes a frozen training phase and an unfreezing training phase:

[0037] In the frozen training phase, the weight parameters of the backbone network are frozen, and only the parameters of the neck network and the detection head are updated;

[0038] In the unfreezing training phase, the backbone network is unfreezed, and all parameters of the model are globally fine-tuned.

[0039] Therefore, the present application proposes an improved single-stage bone marrow blood cell target recognition and counting method, which has the following beneficial effects:

[0040] (1) The present application ensures the accuracy of labeling through expert cross-validation, and greatly enriches the background of the detection object through Mosaic data enhancement, significantly improving the recognition ability of the model for small targets and occluded targets.

[0041] (2) The present application introduces the BiFPN mechanism, realizes the effective fusion of deep semantic information and shallow detail information through weighted bidirectional feature pyramid, and solves the detection problem caused by large scale change of bone marrow cells.

[0042] (3) The present application makes the model focus on key identification features such as cell nucleus texture and cytoplasmic particles through the CBAM attention mechanism, suppresses background noise, and improves classification accuracy.

[0043] (4) The present application combines the training strategy of freezing and unfreezing and the application of GIOU loss function, accelerates the convergence speed of the model, and improves the regression accuracy of the prediction box, finally realizes the unification of high precision and high speed.

[0044] The technical solutions of the present application will be further described in detail below through the accompanying drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 is a flowchart of the improved single-stage bone marrow blood cell target recognition and counting method of the present application;

[0046] Figure 2 is a network architecture diagram of the improved single-stage target recognition model in the embodiment of the present application;

[0047] Figure 3 is a BiFPN feature fusion node structure diagram in the embodiment of the present application;

[0048] Figure 4 This is a schematic diagram of the CBAM hybrid attention mechanism module structure in an embodiment of the present invention. Detailed Implementation

[0049] To make the technical solutions, advantages, and objectives of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below. The described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention without creative effort are within the protection scope of the present invention.

[0050] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.

[0051] Example

[0052] like Figure 1 As shown, this invention provides an improved single-stage bone marrow blood cell target identification and counting method, the specific steps of which are as follows:

[0053] Step S1: Image acquisition and data preparation.

[0054] Due to the minute morphology of bone marrow cells, extremely high image clarity is required. This embodiment utilizes an Olympus BX43 optical microscope equipped with a high-resolution CCD camera. The acquisition process was performed at 100× oil immersion magnification to ensure clear visibility of the cell nuclear chromatin structure and cytoplasmic granules. Simultaneously, bone marrow smears treated with standard Swiss staining were acquired using the BEION V4.90 bone marrow cell morphology examination system. To ensure data comprehensiveness, full-area image data was acquired, covering regions with different degrees of bone marrow hyperplasia.

[0055] Step S2: Dataset construction and preprocessing, dividing the preprocessed dataset into training, validation, and test sets. The dataset construction steps are as follows:

[0056] Step S21, Sample Screening: Remove images with severely ruptured cells or extremely poor staining, and select images containing at least 200 cell units within the field of view; pay special attention to preserving cells at the image edges to train the model's ability to handle truncated targets.

[0057] Step S22, expert annotation (Ground Truth): three experienced blood experts cross-verify. First, two people independently annotate, and the third person arbitrates for inconsistent results, and finally determines the cell category (such as primitive granulocytes, promyelocytes, and neutrophilic myeloblasts) and position through a voting mechanism.

[0058] Step S23, pre-processing of the initial data set, including image rotation, cutting, scaling, flipping, mixing, filling 0 pixels, splicing, and expanding the initial data set.

[0059] A new synthetic image sample is formed by data enhancement Mosaic, and the specific steps are as follows:

[0060] Step S231, randomly select four different images from the data set for random splicing;

[0061] Step S232, randomly select one target box, and adjust the target box from the original image position to the new coordinates in the new synthetic image;

[0062] Step S233, scale and crop the new synthetic image to ensure that the target box is fully visible in the new image and adjust to the correct size;

[0063] Step S234, the new synthetic image is used as a new sample, the adjusted target box position and size are used as labels, and the data set is updated.

[0064] The updated data set is divided into a training set, a validation set and a test set according to a ratio of 7:2:1.

[0065] Step S3, constructing an initial model of an improved single-stage target recognition model.

[0066] This embodiment constructs a model based on a single-stage detection architecture (such as an improved version of the YOLO series architecture). As shown in Figure 2 The improvement points are mainly in the Neck (neck) and Attention (attention) parts.

[0067] Introduce multi-scale feature fusion BiFPN:

[0068] Traditional FPN (Feature Pyramid Network) usually only has a top-down one-way information flow, while PANet increases the bottom-up path but has a large amount of calculation; as Figure 3 shown, the present application adopts multi-scale feature fusion BiFPN, as follows:

[0069] Structural optimization: remove the node with only one input edge, add a path from the input node to the output node, and realize higher-level feature reuse.

[0070] Weighted fusion: considering that feature maps of different resolutions have different contributions to the output, the application introduces learnable weights at each fusion node , and numerical stability is achieved by fast normalization, and the calculation formula is as follows:

[0071]

[0072] wherein, is the output feature map of the fusion node, is the input feature map, is a minimum value to prevent the denominator from being zero, i , j is the summation index, , .

[0073] Embedded mixed attention mechanism CBAM:

[0074] As shown in Figure 4 , a CBAM module is inserted after the output feature layer of the backbone network; the CBAM module adopts a sequential application mechanism, including a channel attention module CAM and a spatial attention module SAM:

[0075] Channel attention (CAM): the spatial dimension is compressed through global average pooling and maximum pooling, and the channel weight is learned through MLP, which is used to pay attention to "what" is meaningful in the feature channel.

[0076] Spatial attention (SAM): based on the output of CAM, the channel dimension is pooled, and the spatial attention map is generated through the convolution layer, which is used to pay attention to "where" is meaningful in the feature space.

[0077] Step S4, using the training set and the validation set to train the initial model, and obtaining the bone marrow blood cell target recognition and counting model;

[0078] During the training process of the initial model, a combined loss function is used to guide the model optimization, and the combined loss function is composed of a weighted summation of a classification loss and a positioning loss; the classification loss adopts binary cross entropy loss, which is used to judge whether each target frame contains blood cells and belongs to which category; the positioning loss adopts GIOU positioning loss, which is used to measure the difference between the predicted target frame and the real target frame, and improve the regression accuracy of the target frame.

[0079] The calculation formula of the combined loss function is as follows:

[0080] ;

[0081] wherein, is the combined loss, is the binary cross entropy classification loss, GIOU loss is positioned for, a weight coefficient.

[0082] The initial model training process includes a frozen training phase and an unfrozen training phase:

[0083] In the frozen training phase (first 50 epochs), the weight parameters of the backbone network are frozen, and only the parameters of the neck network and the detection head are updated;

[0084] In the unfrozen training phase (last 100 epochs), the backbone network is unfrozen, and all parameters of the model are globally fine-tuned using a smaller learning rate.

[0085] Step S5, using the bone marrow blood cell target recognition and counting model to recognize and locate the bone marrow blood cells in the test set image, outputting the cell category and target frame coordinates, and realizing counting according to the number of target frames.

[0086] The experimental results verify that:

[0087] On a data set containing 1501 bone marrow smear images, the improved method of the application has achieved significant advantages: the mAP (average precision mean) reaches 94.5%, which is 6.2% higher than the 88.3% of the original single-stage model without improvement; at the same time, the single image inference time is only 0.04 seconds, which is equivalent to the speed of the original model, greatly improving the accuracy while ensuring the inference speed, fully meeting the real-time requirements of clinical auxiliary diagnosis.

[0088] It is worth noting that the contents not elaborated in the application are all prior art and are well known to those skilled in the art.

[0089] Therefore, the application provides a bone marrow blood cell target recognition and counting method based on an improved single stage, which solves the problem of cell size variation by introducing a BiFPN mechanism to fuse deep and shallow layer features, and focuses on key discriminative features by using a CBAM attention mechanism to improve classification accuracy. At the same time, the training strategy combining freezing and unfreezing and the GIOU loss function accelerate the convergence of the model and improve the regression accuracy of the prediction frame, and finally realize the unification of high precision and high speed of bone marrow blood cell recognition and counting.

[0090] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the application and not to limit them, although the application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that: it can still modify or equivalently replace the technical solutions of the application, and these modifications or equivalent replacements cannot make the modified technical solutions deviate from the spirit and scope of the technical solutions of the application.

Claims

1. A method for identifying and counting bone marrow hematopoietic cells based on an improved single-stage approach, characterized in that, The specific steps are as follows: Step S1, Image Acquisition and Data Preparation: Using a high-resolution electron microscope at 100× magnification, and with the aid of a bone marrow cell morphology examination system, acquire full-area image data of bone marrow granulocyte smears processed by the standard Swiss staining method. Step S2: Construct and preprocess the dataset, and divide the preprocessed dataset into training set, validation set and test set; Step S3: Construct the initial model of the improved single-stage target recognition model. The initial model includes a core network structure with a multi-scale feature fusion mechanism BiFPN and a hybrid attention mechanism module CBAM. Step S4: Train the initial model using the training set and validation set to obtain the bone marrow blood cell target recognition and counting model; Step S5: Use the bone marrow blood cell target recognition and counting model to identify and locate bone marrow blood cells in the test set images, output cell type and target box coordinates, and count according to the number of target boxes; In step S2, the specific steps for constructing the dataset are as follows: Step S21: Select images with more than 200 cell units in the field of view as valid samples, and retain cell samples distributed at the edge of the image; Step S22: Using a three-blood expert cross-validation and voting counting mechanism, blood cells in the valid samples are labeled with categories to generate the initial dataset; Step S23: Preprocess the initial dataset, including image rotation, cropping, scaling, flipping, blending, padding with 0 pixels, and stitching, to expand the initial dataset; In step S23, new synthetic image samples are generated through data augmentation using Mosaic. The specific steps are as follows: Step S231: Randomly select four different images from the dataset and randomly stitch them together; Step S232: Randomly select one of the target boxes and adjust the target box from its position in the original image to its new coordinates in the newly synthesized image; Step S233: Scale and crop the newly synthesized image to ensure that the target box is fully visible in the new image and adjusted to the correct size; Step S234: Use the new synthetic image as a new sample, and the adjusted target box position and size as labels to update the dataset.

2. The improved single-stage bone marrow hematopoietic cell target identification and counting method according to claim 1, characterized in that, In step S3, the improved single-stage target recognition model network introduces a multi-scale feature fusion mechanism, BiFPN, into its neck portion. The BiFPN mechanism achieves feature fusion through bidirectional and cross-layer connections, as detailed below: Introduce learnable weights at each fusion node. Numerical stabilization is achieved through fast normalization, and the calculation formula is as follows: in, This is the output feature map of the fusion node. For the input feature map, To prevent the minimum value where the denominator is zero, i , j For summation index.

3. The improved single-stage bone marrow hematopoietic cell target identification and counting method according to claim 2, characterized in that, The CBAM module adopts a sequential application mechanism, including the channel attention module CAM and the spatial attention module SAM.

4. The improved single-stage bone marrow hematopoietic cell target identification and counting method according to claim 3, characterized in that, In step S4, during the initial model training process, a combined loss function is used to guide model optimization. The combined loss function consists of a weighted sum of classification loss and localization loss. The classification loss uses binary cross-entropy loss to determine whether each target box contains blood cells and which category it belongs to. The localization loss uses GIOU localization loss to measure the difference between the predicted target box and the true target box, thereby improving the regression accuracy of the target box. The formula for calculating the combined loss function is as follows: ； in, For combined loss, For binary cross-entropy classification loss, Locate the loss for GIOU. These are the weighting coefficients.

5. The improved single-stage bone marrow hematopoietic cell target identification and counting method according to claim 4, characterized in that, The initial model training process includes a frozen training phase and a unfrozen training phase: During the frozen training phase, the weight parameters of the backbone network are frozen, and only the parameters of the neck network and the detection head are updated. During the unfreezing training phase, the backbone network is unfrozen, and all parameters of the model are globally fine-tuned.

Images