Myeloma minimal residual disease intelligent diagnosis system based on cellular morphology characteristics

Abstract

The invention discloses a myeloma minimal residual disease intelligent diagnosis system based on cell morphological characteristics, and the system comprises a bone marrow cell image input module which is used for receiving a nucleated cell segmentation image and then carrying out data enhancement processing on the nucleated cell segmentation image for the purpose of generating more diversified image samples; and the myeloma plasma cell intelligent classification module is internally provided with a nucleated cell and plasma cell classification model and is used for receiving the nucleated cell segmentation image subjected to data enhancement, identifying normal plasma cells and myeloma plasma cells from the nucleated cell segmentation image and calculating the percentage of the normal plasma cells and the myeloma plasma cells. The artificial intelligence technology is applied to myeloma minimal residual disease detection based on bone marrow blood cell morphology, the myeloma minimal residual disease diagnosis system which is low in cost, high in accuracy and high in detection speed can be achieved, additional blood sample collection does not need to be conducted on a patient, the acceptance degree of the patient is high, and the pain degree is low; wide medical scientificity and clinical application prospects are realized.

Description

Technical Field

[0001] The present invention relates to the field of computer technology-assisted medical detection, and in particular to an intelligent diagnosis system for minimal residual disease of myeloma based on cell morphological characteristics. Background Art

[0002] Multiple myeloma is a currently incurable malignant clonal disease of plasma cells. The course of the disease often enters the terminal stage after multiple treatments, remission, and relapse. Minimal residual disease in myeloma refers to the fact that a certain number of myeloma plasma cells may remain in the body of patients who have achieved remission after treatment. Its detection is of great significance for evaluating the treatment effect and judging the prognosis. The existing methods for evaluating minimal residual disease mainly include bone marrow morphology, immunohistochemistry, and flow cytometry. The first two methods are that morphologists use a microscope to classify and count the cells on the bone marrow blood smear after Wright staining, calculate the proportion of myeloma plasma cells in the patient's bone marrow specimen based on the cell morphological characteristics, or classify and count the cells of the bone marrow biopsy specimen after immunohistochemical staining; flow cytometry detection is to incubate a few milliliters of bone marrow blood with myeloma cell-specific antibodies and then perform flow cytometric analysis to count the proportion of myeloma plasma cells in the bone marrow blood cells.

[0003] However, the number of bone marrow blood cells read manually is very limited, and the accuracy of Wright's staining morphological classification of normal plasma cells and myeloma plasma cells under naked eye observation is low, which makes the sensitivity of detecting minimal residual disease through bone marrow morphology and immunohistochemistry poor. Although flow cytometry can make the detection sensitivity reach 10 –4 The results are greatly affected by the quality of specimen collection. In actual applications, the plasma cell percentage test results are often lower than the true value due to inevitable peripheral blood dilution and loss in the sample processing process, affecting the clinician's judgment on the patient's minimal residual myeloma status.

[0004] The existing patent document CN202110394842.X provides a device that can automatically collect medical cell images, which can observe and photograph samples for a long time with low damage; the existing patent document CN201811459147.1 provides an artificial intelligence method that can identify the location and boundaries of nucleated cells and segment cell images. At present, the cell morphology image acquisition and segmentation technology is mature, but there is no intelligent diagnosis system for minimal residual disease based on cell morphology images based on artificial intelligence technology. Summary of the invention

[0005] The present invention provides an intelligent diagnosis system for minimal residual disease in myeloma based on cell morphological characteristics, aiming to solve the problems existing in the manual detection method in the background technology.

[0006] The present invention provides an intelligent diagnosis system for minimal residual disease in myeloma based on cell morphological characteristics, comprising:

[0007] A bone marrow cell image input module, used for receiving a nucleated cell segmentation image, and then performing data enhancement processing on the nucleated cell segmentation image for the purpose of generating more diverse image samples;

[0008] Myeloma plasma cell intelligent classification module, with built-in nucleated cell and plasma cell classification models, is used to receive data-enhanced nucleated cell segmentation images, identify normal plasma cells and myeloma plasma cells, and calculate the percentage of the two;

[0009] The myeloma plasma cell intelligent classification module comprises:

[0010] A plasma cell identification submodule, for identifying and marking plasma cells from the received data-enhanced nucleated cell segmentation image;

[0011] The normal plasma cell and myeloma plasma cell classification submodule is used to identify normal plasma cells and myeloma plasma cells from the plasma cells identified and marked by the plasma cell identification submodule and calculate the percentage of the two.

[0012] The nucleated cell segmentation image contains nucleated cells with a number greater than 10 4 indivual.

[0013] In the myeloma minimal residual disease intelligent diagnosis system based on cell morphological characteristics, in the bone marrow cell image input module, one or more strategies of rotation, scaling, grayscale stretching, random cropping, horizontal flipping, and color jittering are selected to perform data enhancement processing on the data of the bone marrow blood Wright-stained smear nucleated cell segmentation image to generate more diverse image samples, thereby making the myeloma plasma cell intelligent classification module more accurate and robust in identifying and analyzing different samples.

[0014] The nucleated cell and plasma cell classification model is constructed based on the improved YOLOv8 target detection model; in the process of constructing the nucleated cell and plasma cell classification model, quaternion convolution and QGDM (quaternion gradient flow method) and SSSAM (separable sparse spatial attention method) modules are introduced to extract and fuse the features of the data-enhanced nucleated cell segmentation image;

[0015] The QGDM module adopts the quaternion gradient flow method to replace the convolution in the bottleneck layer with the quaternion convolution to form QBottleneck (quaternion bottleneck layer), and splits the input features after the quaternion convolution into two parts: X = [X 1 +X 2 ], where the first part X 1 Connect directly to the output, the second part X2 Enter the dense block and perform multiple QBottleneck operations before joining it with the first part. The specific formula is: [X 1 ,X 2 ]=QConv(X); Y=QConv(Concat(X 1 ,X 2 ,G(X 2 ),G 2 (X 2 ),…,G n (X 2 ))), where X 1 Connect directly to the end of the stage, X 2 Will enter a dense block; where X is the input, Y is the output, QConv is the quaternion convolution operation, Concat is the concatenation operation, G n (X) is to perform n QBottleneck operations;

[0016] In the SSSAM module, the input of this module is first divided into multiple blocks in the feature dimension and input into FSAM (Fast Spatial Attention Method) respectively. The self-attention mechanism is used to process the segmented block feature map, and the processed results are cross-added to aggregate the key area information contained in different feature blocks. The specific formula is: Where X is input, Y is output, M(X i ) is the FSAM operation, ⊙ is the pixel-by-pixel multiplication, × is the matrix multiplication, X T is the matrix transpose operation, i and j represent different feature block labels;

[0017] In the feature extraction process of constructing the nucleated cell and plasma cell classification model, the data-enhanced nucleated cell segmentation image undergoes 1 convolution, 3 quaternion convolutions, 4 QGDM processes, and 3 SSSAM processes.

[0018] The beneficial effects achieved by the present invention are:

[0019] Compared with the traditional manual counting method, the present invention applies artificial intelligence technology to the detection of minimal residual disease in multiple myeloma based on bone marrow blood Wright staining morphology. It can detect bone marrow nucleated cells and identify myeloma plasma cells in the myeloma cases to be tested with high throughput, thus achieving low-cost, high-accuracy and fast detection of minimal residual disease in myeloma. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a workflow diagram of an intelligent diagnosis system for minimal residual disease in myeloma based on cell morphology characteristics in an embodiment of the present invention;

[0021] Figure 2 It is a schematic diagram of the structure of the nucleated cell and plasma cell classification model constructed based on YOLOv8 in an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0023] The present invention is a myeloma minimal residual disease intelligent diagnosis system based on cell morphology characteristics (workflow as shown in Figure 1 shown) include:

[0024] The bone marrow cell image input module is used to receive the nucleated cell segmentation image, and then perform data enhancement processing on the nucleated cell segmentation image (the image acquisition, nucleated cell recognition and image segmentation methods refer to Chinese patent documents CN202110394842.X and CN201811459147.1) for the purpose of generating more diverse image samples; the nucleated cell segmentation image used by the bone marrow cell image input module contains nucleated cells with a number greater than 10 4 In the bone marrow cell image input module, one or more strategies of rotation, scaling, grayscale stretching, random cropping, horizontal flipping, and color jittering are selected to enhance the data of the bone marrow blood Wright-stained smear nucleated cell segmentation image to generate more diverse image samples, so that the myeloma plasma cell intelligent classification module is more accurate and robust in identifying and analyzing different samples;

[0025] The myeloma plasma cell intelligent classification module has a built-in nucleated cell and plasma cell classification model, which is used to receive the data-enhanced nucleated cell segmentation image, identify normal plasma cells and myeloma plasma cells from it and calculate the percentage of the two. The myeloma plasma cell intelligent classification module includes two submodules: a plasma cell recognition submodule and a normal plasma cell and myeloma plasma cell classification submodule; wherein the plasma cell recognition submodule is used to identify and mark plasma cells from the received data-enhanced nucleated cell segmentation image; the normal plasma cell and myeloma plasma cell classification submodule is used to identify normal plasma cells and myeloma plasma cells from the plasma cells identified and marked by the plasma cell recognition submodule and calculate the percentage of the two; when the proportion of myeloma plasma cells is greater than 0, it can be determined that the diagnosed object suffers from minimal residual disease of myeloma. The nucleated cell and plasma cell classification model is constructed using an improved YOLOv8 target detection model based on a convolutional neural network.

[0026] The method for constructing the nucleated cell and plasma cell classification model is as follows:

[0027] 1) Data collection and processing:

[0028] The nucleated cell segmentation image data of bone marrow blood Wright-stained smears of clinical myeloma cases were collected (the image data acquisition method refers to Chinese patent documents CN202110394842.X and CN201811459147.1); the collected data set was divided into training set, validation set and test set in a ratio of 3:1:1; one or more of the strategies of rotation, scaling, grayscale stretching, random cropping, horizontal flipping and color jittering were selected to perform data enhancement processing on the training data to generate more diverse image samples.

[0029] 2) Image feature extraction:

[0030] The nucleated cell segmentation image after data enhancement is used as the input of the feature extraction process; the features of the nucleated cell segmentation image are extracted and fused through quaternion convolution and QGDM (quaternion gradient flow method) and SSSAM (separable sparse spatial attention method) modules, and finally 256 deep learning features are extracted:

[0031] In the QGDM module, the quaternion gradient flow method is used. First, the convolution in the bottleneck layer is replaced by the quaternion convolution to form its quaternion version QBottleneck (quaternion bottleneck layer). Continuing the idea of ​​CSPNet (cross-stage partial network), the input features after quaternion convolution are split into X = [X 1 +X 2 ], the base layer feature map of this stage is divided into two parts. 1 Connect directly to the end of the stage, X 2 will enter a dense block. In the dense block, QBottleneck operations are performed n times on the input, and finally all intermediate results are combined with X 1 The concatenation is performed and then a quaternion convolution is performed. While promoting feature reuse, the redundancy of gradient information is reduced by truncating the gradient flow. The specific formula is as follows:

[0032] [X 1 ,X 2 ] = QConv(X)

[0033] Y = QConv(Concat(X 1 ,X 2 ,G(X 2 ),G 2 (X 2 ),…,G n (X 2 )))

[0034] Where X is the input, Y is the output, QConv is the quaternion convolution operation, Concat is the concatenation operation, G n (X) is to perform n QBottleneck operations;

[0035] In the SSSAM module, the input of this module is first divided into multiple blocks in the feature dimension and input into FSAM (Fast Spatial Attention Method) respectively. The self-attention mechanism is used to process these block feature maps, and the processed results are cross-added to aggregate the key area information contained in different feature blocks. The specific formula is as follows:

[0036]

[0037] Where X is input, Y is output, M(X i ) is the FSAM operation, ⊙ is the pixel-by-pixel multiplication, × is the matrix multiplication, X T is the matrix transpose operation, i and j represent different feature block labels;

[0038] In the feature extraction process of constructing the nucleated cell and plasma cell classification model, the data-enhanced nucleated cell segmentation image undergoes 1 convolution, 3 quaternion convolutions, 4 QGDM processes, and 3 SSSAM processes.

[0039] 3) Training of the nucleated cell and plasma cell classification model:

[0040] The extracted 256 deep learning features are sent to the classification head. In the classification head, the input features are first extracted and the number of channels is adjusted to 1280 feature channels through the convolution module. Then, the feature map is compressed into a vector of fixed length through adaptive average pooling, and then regularized through a random inactivation layer. Finally, the features are mapped to the classification space through a linear layer to construct a complete nucleated cell and plasma cell classification model based on YOLOv8.

[0041] The nucleated cell and plasma cell classification model includes multiple QGDM and SSSAM feature extraction processes.

[0042] When training the nucleated cell and plasma cell classification model, the batch size of the training set is set to 32, the epochs (training rounds) are set to 500 for 500 rounds of training, and the patience (number of early stops) is set to 100. The model parameters with the best performance on the validation set are saved as the optimal result of the one-fold training, and the model is cross-trained with five folds, and the model parameters with the best performance on the validation set are selected as the final result.

[0043] During the training of the nucleated cell and plasma cell classification model, the loss function uses the cross entropy loss function. The cross entropy uses a logical function to obtain the probability, adopts an inter-class competition mechanism, and is good at learning inter-class information. The loss function is expressed as:

[0044]

[0045] Where N represents the number of samples in a batch, M is the number of categories, and y ic is a sign function, which takes 1 when the sample matches the category, otherwise it takes 0. ic is the predicted probability that the observed sample i belongs to category c. The final structure of the nucleated cell and plasma cell classification model is as follows Figure 2 shown.

[0046] 4) Model performance evaluation:

[0047] In the nucleated cell and plasma cell classification model of the present invention, the final evaluation index uses the Top-1 accuracy and ROC-AUC value to comprehensively evaluate the performance of the model:

[0048] The final nucleated cell and plasma cell classification model performed well in the test set: in the first stage of distinguishing plasma cells, the top_1 was 97.16 and the ROC_AUC was 99.64; in the second stage of distinguishing myeloma plasma cells, the top_1 was 93.99 and the ROC_AUC was 98.31.

[0049] The input of the nucleated cell and plasma cell classification model constructed by the present invention is the single cell image of nucleated bone marrow cells after segmentation. First, it is processed by a series of convolutional layers and multiple modules of the backbone network to extract the multi-level features of the cell image, and then the feature map is classified by the head to output the category of each target image; wherein, a multi-feature collaborative prediction strategy combining fusion quaternion convolution and real-valued convolution is introduced, and the accuracy of cell classification of the nucleated cell and plasma cell classification model is improved by introducing color features and combining texture features; a lightweight sparse self-attention mechanism is introduced to strengthen the ability of the nucleated cell and plasma cell classification model to extract fine-grained spatial features of cells, so as to accurately distinguish the fine-grained discrimination information between different cells. Through the above-mentioned improvement measures, the feature mining and fusion capabilities of the nucleated cell and plasma cell classification model are further improved, while the model calculation amount and parameter amount are reduced, and the calculation efficiency of the model is improved.

[0050] After multiple verifications, the myeloma minimal residual disease intelligent diagnosis system based on cell morphology characteristics in this embodiment has been successfully applied in 10 -4 The recognition accuracy rate under sensitivity reached 96.67%, reaching the level of clinical application.

[0051] It can be found that the present invention applies artificial intelligence technology to the detection of minimal residual disease in multiple myeloma based on bone marrow Wright staining morphology, and can detect bone marrow nucleated cells and identify the percentage of myeloma plasma cells in high-throughput for the myeloma cases to be tested, thus realizing a low-cost and high-accuracy minimal residual disease detection system. The present invention innovatively performs high-sensitivity detection of minimal residual disease in myeloma from clinical bone marrow blood cell Wright staining morphological samples, avoiding the defect that the test results are lower than the actual proportion due to peripheral blood dilution and sample processing loss in myeloma plasma cell flow cytometry detection, and can achieve high-quality detection of minimal residual disease in myeloma at a lower cost of consumables, and does not require additional blood sample collection from patients. Patients have high acceptance and low pain, and have broad medical scientificity and clinical application prospects.

Claims

1. An intelligent diagnosis system for minimal residual disease in myeloma based on cell morphology, characterized in that: include: A bone marrow cell image input module, used for receiving a nucleated cell segmentation image, and then performing data enhancement processing on the nucleated cell segmentation image for the purpose of generating more diverse image samples; Myeloma plasma cell intelligent classification module, with built-in nucleated cell and plasma cell classification models, is used to receive data-enhanced nucleated cell segmentation images, identify normal plasma cells and myeloma plasma cells, and calculate the percentage of the two; The myeloma plasma cell intelligent classification module comprises: A plasma cell identification submodule, for identifying and marking plasma cells from the received data-enhanced nucleated cell segmentation image; The normal plasma cell and myeloma plasma cell classification submodule is used to identify normal plasma cells and myeloma plasma cells from the plasma cells identified and marked by the plasma cell identification submodule and calculate the percentage of the two.

2. The intelligent diagnosis system for minimal residual disease in myeloma based on cell morphology characteristics according to claim 1, characterized in that: The number of nucleated cells contained in the nucleated cell segmentation image is greater than 10 4 indivual.

3. The intelligent diagnosis system for minimal residual disease in myeloma based on cell morphology characteristics according to claim 1, characterized in that: In the bone marrow cell image input module, the data enhancement process is to select one or more strategies of rotation, scaling, grayscale stretching, random cropping, horizontal flipping, and color jittering to process the nucleated cell segmentation image, so that the myeloma plasma cell intelligent classification module is more accurate and robust in identifying and analyzing different samples.

4. The intelligent diagnosis system for minimal residual disease in myeloma based on cell morphology characteristics according to claim 1, characterized in that: The nucleated cell and plasma cell classification model is constructed based on the YOLOv8 target detection model; in the process of constructing the nucleated cell and plasma cell classification model, quaternion convolution, quaternion gradient flow method and separable sparse spatial attention method are introduced to extract and fuse the features of the data-enhanced nucleated cell segmentation image; In the quaternion gradient flow method, the convolution in the bottleneck layer is replaced by quaternion convolution to form a quaternion bottleneck layer, and the input features after quaternion convolution are split into two parts: X = [X1 + X2], where the first part X1 is directly connected to the output, and the second part X2 enters the dense block for multiple quaternion bottleneck layer operations and is spliced ​​with the first part. The specific formula is: [X1, X2] = QConv (X); Y = QConv (Concat (X1, X2, G (X2), G 2 (X2),...,G n (X2))), where X1 is directly connected to the end of the stage, and X2 will enter a dense block; where X is the input, Y is the output, QConv is the quaternion convolution operation, Concat is the concatenation operation, and G n (X) is n quaternion bottleneck layer operations; In the separable sparse spatial attention method, the input is first divided into multiple blocks in the feature dimension and respectively input into the fast spatial attention method; The self-attention mechanism is used to process the segmented block feature maps, and the processed results are cross-added to aggregate the key area information contained in different feature blocks; the specific formula is: Where X is input, Y is output, M(X i ) is the fast spatial attention method operation, ⊙ is the pixel-by-pixel multiplication, × is the matrix multiplication, X T is the matrix transpose operation, i and j represent different feature block labels; During the feature extraction process of constructing the nucleated cell and plasma cell classification model, the data-enhanced nucleated cell segmentation image undergoes 1 convolution, 3 quaternion convolutions, 4 quaternion gradient flow method processing, and 3 separable sparse spatial attention method processing.

Images