Intelligent typing method and system for m protein in immunofixation electrophoresis image

By using the CBAM-ResNet50 convolutional backbone network model and intelligent typing system, the problem of reliance on manual interpretation in traditional immunofixation electrophoresis diagnosis has been solved, achieving efficient and accurate M protein typing and improving the diagnostic and typing capabilities for diseases such as multiple myeloma.

CN120707502BActive Publication Date: 2026-03-03BEIJING GAOBO HOSPITAL CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-06-16
Publication Date
2026-03-03

AI Technical Summary

Technical Problem

Traditional immunofixation electrophoresis diagnosis relies on human interpretation, resulting in low diagnostic efficiency, inconsistent results, and a lack of systematic data management and in-depth analysis, which affects the optimization of disease diagnosis and treatment.

Method used

The CBAM-ResNet50 convolutional backbone network model was used for image standardization and deep learning. Combined with double-blind annotation, self-supervised learning and clinical rule constraints, an intelligent typing system was constructed, including image acquisition, preprocessing, neural network analysis and continuous learning modules.

Benefits of technology

It improves the objectivity and efficiency of diagnosis, reduces the influence of human factors, provides detailed diagnostic reports and analysis of difficult cases, and enhances the diagnostic level of primary healthcare institutions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120707502B_ABST
    Figure CN120707502B_ABST
Patent Text Reader

Abstract

This invention discloses an intelligent typing method and system for M protein in immunofixation electrophoresis images, belonging to the field of immunofixation electrophoresis image analysis technology. This invention establishes a double-blind annotation process and an expert knowledge-driven annotation standard with an embedded annotation verification algorithm based on physical constraints of electrophoretic mobility. Vahadane staining normalization and dynamic ROI extraction techniques are used to standardize the images, and intelligent noise suppression is applied to the image data to improve data quality. Through a hybrid neural network deep learning algorithm, the AI ​​model can perform standardized interpretation of electrophoresis images, reducing the influence of human factors and enabling the model to reproduce the specificity and accuracy of expert-level interpretation. This solves the problems of low accuracy due to strong subjectivity in traditional visual interpretation, such as overlapping bands and weak positives, and the low efficiency of seeking expert consultation for complex and difficult cases.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of immunofixation electrophoresis image analysis technology, specifically relating to an intelligent typing method and system for M protein in immunofixation electrophoresis images. Background Technology

[0002] Monoclonal immunoglobulins, also known as M proteins, are immunoglobulin molecules and their fragments (such as light chains and heavy chains) with the same structure and electrophoretic mobility, produced by the abnormal proliferation of monoclonal B lymphocytes or plasma cells. M proteins are important biomarkers for diseases such as multiple myeloma, macroglobulinemia, and malignant lymphoma. Currently, clinical diagnosis of monoclonal immunoglobulins mainly relies on immunofixation electrophoresis, and the typing results have significant clinical guiding value for the diagnosis and treatment of these diseases, especially multiple myeloma.

[0003] Traditional immunofixation electrophoresis typing diagnosis has the following main problems:

[0004] Low diagnostic efficiency: The diagnostic efficacy of traditional immunofixation electrophoresis is limited by two core issues. First, subjective dependence: Diagnostic results rely on the visual interpretation of electrophoresis films by clinical laboratory technicians, and the efficiency and quality of interpretation are positively correlated with the technician's experience level. Second, objective resource limitations: For complex and difficult cases (such as special band patterns or weakly positive samples), consultation with senior technicians, clinicians, or specialists is usually required. However, currently, China lacks a systematic training system for interpreting electrophoresis patterns, and there is a limited number and uneven regional distribution of senior technicians and experts proficient in clinical electrophoresis pattern diagnosis. Some difficult cases even require cross-hospital and cross-provincial assistance to complete the diagnosis. These multiple factors lead to prolonged diagnostic cycles and low work efficiency.

[0005] Significant interference from human factors: The interpretation of immunofixation electrophoresis patterns is highly dependent on the subjective experience of clinical laboratory technicians. Different technicians may interpret the same pattern differently due to differences in work experience, professional knowledge level, and reading habits. Such inconsistencies in interpretation not only affect the reliability of diagnosis, but may also delay the diagnosis, classification, and subsequent treatment of patients.

[0006] Insufficient data utilization: With the increasing volume of clinical immunofixation electrophoresis (IME) tests, massive amounts of electrophoresis data are stored in traditional image formats, lacking effective data management and in-depth analysis methods. This underutilized data potential limits its application value in optimizing disease diagnosis and treatment and in medical research, hindering further improvements in clinical diagnostic capabilities. Summary of the Invention

[0007] To address the shortcomings of the prior art, this application provides a method and system for intelligent typing of M protein in immunofixation electrophoresis images.

[0008] In its first aspect, this application proposes a smart typing method for M protein in immunofixation electrophoresis images, comprising the following steps:

[0009] Immunofixation electrophoresis film scan images were collected, and double-blind annotation and standardization were performed to obtain the scan images to be analyzed.

[0010] An initial neural network model is constructed, wherein the architecture of the initial neural network model is a CBAM-ResNet50 convolutional backbone network;

[0011] The scanned image to be analyzed is used to train the initial neural network model for subtyping. The subtyping training includes pre-training and fine-tuning training. The pre-training is performed through self-supervised tasks such as puzzle restoration and rotation prediction. In the fine-tuning training stage, labeled data is used to perform dynamic balancing of multi-task weights and clinical rule constraint reinforcement training to obtain the intelligent subtyping model to be tested.

[0012] Obtain the immunofixation electrophoresis film scan image that has only undergone standardized processing as the test image, input the test image into the intelligent typing model to be tested, calculate the performance index of the intelligent typing model to be tested, and select the model whose performance index exceeds the preset value as the final intelligent typing model;

[0013] The final intelligent typing model is deployed to the electrophoresis pattern intelligent expert analysis system. The system monitors and captures electrophoresis scan images generated by the electrophoresis instrument software system stored in the laboratory terminal through the file system. The electrophoresis scan images are standardized and preprocessed. The final intelligent typing model is then used to analyze the processed images and generate intelligent typing results for the M protein.

[0014] In some embodiments, the standardization process includes staining normalization, band ROI extraction, and grayscale normalization;

[0015] The staining normalization is: converting the immunofixation electrophoresis film scan image obtained by double-blind labeling into optical density space for quantitative analysis of staining intensity;

[0016] The Vahadane algorithm is used to separate the optical density vectors of various dyes using sparse nonnegative matrices, and the dyeing vector is extracted. and concentration matrix Maintain concentration matrix Keep it unchanged, apply the dye The generated coloring vector Replace with dye The generated coloring vector Then, the normalized image is reconstructed and inversely converted back to RGB space to ensure that images under different staining conditions have a consistent staining effect.

[0017] The strip ROI extraction is as follows: Gaussian filtering is applied to the stained and normalized image to remove noise, preserving the main structural information of the image; contrast-limited adaptive histogram equalization is used to enhance the contrast of the strips; the boundaries of the strips are located by edge detection; the strips are binarized to convert the image into a black and white binary image.

[0018] The grayscale normalization process involves matching a black-and-white binary image to a reference template, adjusting the image's mean, standard deviation, and nonlinear brightness, and then using grayscale normalization to ensure that the grayscale values ​​of different images have the same range and distribution, thus obtaining a standardized image.

[0019] In some embodiments, the initial neural network model includes an input layer, an initial convolutional layer, four residual stages, a multi-scale feature fusion module, and a multi-label classification head module. The initial convolutional layer, the four residual stages, the multi-scale feature fusion module, and the multi-label classification head module together constitute the CBAM-ResNet50 convolutional backbone network.

[0020] The initial convolutional layer includes a convolutional layer, a batch normalization layer, and a pooling layer;

[0021] The four residual stages include residual stage 1, residual stage 2, residual stage 3 and residual stage 4. Each stage consists of multiple residual blocks, with the number of residual blocks being [3, 4, 6, 3]. Each residual block has a CBAM module inserted after 3 basic convolutional layers. The CBAM module includes channel attention mechanism and spatial attention mechanism, with a compression ratio of 16, an input convolutional kernel of 7×7, a stride of 2, and a spatial attention convolutional kernel of 7×7.

[0022] The multi-scale feature fusion module includes upsampling, downsampling, and feature fusion.

[0023] The multi-label classification head module includes a global average pooling layer, a fully connected layer, and multiple independent binary classifiers.

[0024] In some embodiments, the pre-training employs a self-supervised learning task to enable the model to learn the basic features and structure of an image without labeled data. The pre-trained jigsaw puzzle reconstruction task includes:

[0025] The image received by the input layer is divided into... Grid, resulting in 9 The sub-block;

[0026] Randomly shuffle the order of the sub-blocks to generate a shuffled image;

[0027] Each sub-block is processed by the initial convolutional layer, and the output is... The feature map is formed by concatenating the feature maps of all sub-blocks along the channel dimension. The fused feature map outputs a probability distribution predicting the order of image sub-blocks through a global average pooling layer and a fully connected layer. ;

[0028] The calculation is performed using the first cross-entropy loss function:

[0029]

[0030] in As the first true label, The parameters of the initial convolutional layer and the classification head are optimized through backpropagation to obtain the first prediction probability.

[0031] In some embodiments, the pre-trained rotation prediction task includes:

[0032] The image received by the input layer is randomly rotated to generate a rotated image.

[0033] Through the initial convolutional layer and four residual stages, the output is... The deep features are obtained by passing them through a global average pooling layer and a fully connected layer, and outputting a 4-dimensional probability distribution. ;

[0034] The calculation is performed using the second cross-entropy loss function:

[0035]

[0036] in As a second real label, The parameters of the initial convolutional layer and the classification head are optimized through backpropagation to obtain the second prediction probability.

[0037] In some embodiments, the fine-tuning training uses labeled data to train the model, aiming to adapt it to a specific classification task. The specific steps are as follows:

[0038] The input layer receives the standardized image, which is then converted through an initial convolutional layer. Feature map;

[0039] The feature map passes through four residual stages in sequence. After each residual block in each residual stage outputs a feature map, it is immediately passed through the CBAM module for attention weighting.

[0040] After all residual stage processing is completed, the feature maps at different scales are adjusted through a multi-scale feature fusion module and stitched together along the channel dimension to form... fusion feature map;

[0041] The fused feature map is processed through a global average pooling layer, a fully connected layer, and independent binary classifiers to output the predicted probability of each label. ;

[0042] The calculation is performed using the third cross-entropy loss function:

[0043]

[0044] in, For the final true label, To ultimately predict the probability, the model parameters are adjusted through backpropagation, including weights, biases, scaling factors, offset factors, and attention mechanism parameters, so that the prediction results approximate the labeled results.

[0045] In some embodiments, the multi-task weight dynamic balancing training for fine-tuning includes:

[0046] The loss value is calculated independently for each puzzle reconstruction task or rotation prediction task. Calculate the gradient of the loss for each task with respect to the shared parameters. Adjust task weights according to gradient magnitude ratio This aligns the gradient magnitudes of each task.

[0047] Calculate the weighted total loss function: The parameters are updated during the backpropagation phase.

[0048] In some embodiments, the fine-tuning training of clinical rule constraints includes:

[0049] The diagnostic rules are transformed into differentiable mathematical expressions and directly used as regularization terms. Add to the model's loss function. During the backpropagation phase, the model parameters are updated using gradient descent.

[0050] Secondly, this application proposes an intelligent typing system for M protein in immunofixation electrophoresis images, including an image acquisition module, an image preprocessing module, a neural network model module, a diagnostic report generation module, a user interface module, and a continuous learning and online updating module;

[0051] The image acquisition module is used to capture the generated immunofixation electrophoresis film scan images from the software system of the electrophoresis instrument. This module uses a file system monitoring mechanism to detect new image files in a specified directory in real time and transmit them to the image preprocessing module.

[0052] The image preprocessing module is used to standardize the acquired images, including color normalization, strip ROI extraction, Gaussian filtering for noise reduction, contrast enhancement, edge detection, binarization, and grayscale normalization. This module ensures that the images input to the neural network model are consistent with the images in the training phase in terms of feature extraction and classification tasks.

[0053] The neural network model module is used to deploy the trained neural network model into the system and is responsible for analyzing and classifying the preprocessed images. The module includes an input layer, an initial convolutional layer, four residual stages, a multi-scale feature fusion module, and a multi-label classification head module, which outputs the predicted probability of each label.

[0054] The diagnostic report generation module is used to generate detailed diagnostic reports based on the prediction results of the neural network model. This module integrates the predicted immunoglobulin types and their probability values ​​into the report and provides visualized electrophoresis patterns and band analysis results for reference by clinical technicians and experts.

[0055] The user interface module provides a user-friendly interface, enabling technicians and experts to view diagnostic reports, mark error cases, and provide feedback. This module supports concurrent operation by multiple users, ensuring the system's efficiency and convenience.

[0056] The continuous learning and online update module is responsible for collecting error cases and newly labeled data from user feedback. It optimizes the neural network model through a continuous learning and online update mechanism. This module dynamically updates the model weights and clinical rule base to improve the model's classification accuracy and clinical applicability.

[0057] Thirdly, this application proposes an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0058] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described above.

[0059] The beneficial effects of this invention are:

[0060] This paper establishes an expert knowledge-driven annotation standard using a double-blind annotation process and an annotation verification algorithm that embeds physical constraints of electrophoretic mobility. Vahadane staining normalization and dynamic ROI extraction techniques are used to standardize images and intelligently suppress noise in image data, improving data quality and providing a reliable data foundation for subsequent AI model training and diagnosis. Through hybrid neural network deep learning algorithms, the AI ​​model can perform standardized interpretation of electrophoretic images, reducing the influence of human factors and enabling the model to reproduce the specificity and accuracy of expert-level analysis. In the analysis and judgment of difficult cases, the intelligent diagnostic system can also provide diagnostic criteria and treatment suggestions, helping to improve the diagnostic level and professional capabilities of technicians in primary healthcare institutions. It solves the problems of low accuracy due to strong subjectivity in traditional visual interpretation, such as overlapping bands and weak positives, and the low efficiency of seeking expert consultation for complex and difficult cases. The application of the AI ​​intelligent diagnostic system based on deep learning algorithms enables immunofixation electrophoresis detection to break through the traditional "experience-intensive" technical barriers, providing a universal solution for the early screening and diagnostic classification of diseases such as multiple myeloma. Attached Figure Description

[0061] Figure 1 This is the overall flowchart of the present invention.

[0062] Figure 2 This is a system principle block diagram of the present invention. Detailed Implementation

[0063] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein; rather, these embodiments are provided so that a more thorough understanding of the invention can be achieved and that the full scope of the invention can be conveyed to those skilled in the art.

[0064] In its first aspect, this application proposes a smart typing method for M protein in immunofixation electrophoresis images, comprising the following steps:

[0065] S100: Collect immunofixation electrophoresis film scan images, perform double-blind annotation and standardization to obtain the scan images to be analyzed;

[0066] Double-blind labeling process

[0067] Data allocation: 2000 electrophoresis film scans were randomly divided into two groups of 1000 each, and each group was independently labeled by two clinical electrophoresis testing experts (Expert A and Expert B). Labels included nine types, such as monoclonal IgGκ, IgGλ, and IgAκ, as well as biclonal combinations.

[0068] Annotation tools: Using a customized annotation system, experts generate structured labels by selecting the location and type of the spectral bands, and the system automatically records the annotation results.

[0069] Consistency check:

[0070] Calculate the Kappa coefficient for double-blind annotation: The system automatically compares the two sets of annotation results and calculates the Kappa coefficient to be 0.93 (>0.90 threshold), thus determining that the overall annotation consistency is qualified.

[0071] Handling of inconsistent samples: For the 150 samples with a Kappa coefficient < 0.90 (e.g., blurred bands, weak positive results), an arbitration mechanism will be initiated.

[0072] First arbitration: A third expert, C, independently interprets the results. Of the 120 cases, the majority of the results are adopted if they are consistent with the conclusions of experts A or B (e.g., expert C agrees with A in 90 cases and with B in 30 cases).

[0073] Second arbitration: Of the remaining 30 samples, expert C proposed 20 new conclusions (such as revising "monoclonal IgGκ" to "oligoclonal"), triggering additional expert discussions: a 5-person group consisting of experts A, B, C and two chief technicians discussed the disputed samples one by one according to the "Clinical Electrophoresis Diagnostic Guidelines", and finally reached a consensus through voting (such as a 4:1 determination of "biclonal IgGκ+IgAλ").

[0074] Annotated data processing:

[0075] Label structuring: Convert expert annotation results into multi-label vectors (such as the vector [1,0,0,…,0] corresponding to monoclonal IgGκ).

[0076] Data cleaning: Five samples that could not reach a consensus after arbitration (such as those with severe band distortion) were removed, leaving 1,995 valid data.

[0077] In some embodiments, the standardization process includes staining normalization, band ROI extraction, and grayscale normalization;

[0078] The staining normalization is: converting the immunofixation electrophoresis film scan image obtained by double-blind labeling into optical density space for quantitative analysis of staining intensity;

[0079] The Vahadane algorithm is used to separate the optical density vectors of various dyes using sparse nonnegative matrices, and the dyeing vector is extracted. and concentration matrix Maintain concentration matrix Keep it unchanged, apply the dye The generated coloring vector Replace with dye The generated coloring vector Then, the normalized image is reconstructed and inversely converted back to RGB space to ensure that images under different staining conditions have a consistent staining effect.

[0080] The strip ROI extraction is as follows: Gaussian filtering is applied to the stained and normalized image to remove noise, preserving the main structural information of the image; contrast-limited adaptive histogram equalization is used to enhance the contrast of the strips; the boundaries of the strips are located by edge detection; the strips are binarized to convert the image into a black and white binary image.

[0081] The grayscale normalization process involves matching a black-and-white binary image to a reference template, adjusting the image's mean, standard deviation, and nonlinear brightness, and then using grayscale normalization to ensure that the grayscale values ​​of different images have the same range and distribution, thus obtaining a standardized image.

[0082] The 1995 electrophoresis scan images were standardized and preprocessed to eliminate staining differences and noise interference. The RGB images were then converted to optical density space to extract staining vectors. and concentration matrix The image is then reconstructed and normalized. Next, the normalized image undergoes strip ROI extraction, employing Gaussian filtering for noise reduction, CLAHE contrast enhancement, edge detection, and binarization to accurately extract the strip regions. Finally, the ROI histogram is matched to a reference template, and the mean, standard deviation, and non-linear brightness are adjusted to achieve grayscale normalization. These steps effectively standardize the image, providing a high-quality data foundation for subsequent neural network model training and image analysis.

[0083] S200: Construct an initial neural network model, the architecture of which is a CBAM-ResNet50 convolutional backbone network;

[0084] In some embodiments, the initial neural network model includes an input layer, an initial convolutional layer, four residual stages, a multi-scale feature fusion module, and a multi-label classification head module. The initial convolutional layer, the four residual stages, the multi-scale feature fusion module, and the multi-label classification head module together constitute the CBAM-ResNet50 convolutional backbone network.

[0085] The initial convolutional layer includes a convolutional layer, a batch normalization layer, and a pooling layer;

[0086] The four residual stages include residual stage 1, residual stage 2, residual stage 3 and residual stage 4. Each stage consists of multiple residual blocks, with the number of residual blocks being [3, 4, 6, 3]. Each residual block has a CBAM module inserted after 3 basic convolutional layers. The CBAM module includes channel attention mechanism and spatial attention mechanism, with a compression ratio of 16, an input convolutional kernel of 7×7, a stride of 2, and a spatial attention convolutional kernel of 7×7.

[0087] The multi-scale feature fusion module includes upsampling, downsampling, and feature fusion.

[0088] The multi-label classification head module includes a global average pooling layer, a fully connected layer, and multiple independent binary classifiers.

[0089] 1. Input layer:

[0090] The received data consists of immunofixation electrophoresis film images, which, after standardization in step S100, are either grayscale images of a certain size or unstandardized RGB images. The input layer's function is to pass this image data to subsequent convolutional layers for processing.

[0091] 2. Initial convolutional layer:

[0092] Convolutional layer Conv1: The input image is passed through a... Convolution operations are performed using convolution kernels with strides of [missing information]. ,filling Output feature maps with 64 channels.

[0093] Batch normalization layer: Mean values ​​are applied to the feature maps of each of the 64 channels. and variance The normalization process learns the scaling and offset parameters.

[0094] Max pooling layer: using Pooling kernel, step size ,filling Downsampling of the feature map, output The feature map.

[0095] 3. Four residual stages:

[0096] Residual stage 1: Contains 3 residual blocks, each with a CBAM module inserted after 3 basic convolutional layers. The CBAM module has a compression ratio of 16, and the input convolutional kernel is... Step length Spatial attention convolution kernel is .

[0097] Residual stage 2: contains 4 residual blocks, each of which inserts a CBAM module after 3 basic convolutional layers.

[0098] Residual stage 3: Contains 6 residual blocks, each of which inserts a CBAM module after 3 basic convolutional layers.

[0099] Residual stage 4: Contains 3 residual blocks, each of which inserts a CBAM module after 3 basic convolutional layers.

[0100] The residual stages consist of four stages (stage1, stage2, stage3, stage4), each composed of multiple residual blocks with the number of blocks in the ranges [3, 4, 6, 3]. Each residual block is followed by a CBAM module after three basic convolutional layers. The CBAM module includes channel attention and spatial attention mechanisms, with a compression ratio of 16, an input convolutional kernel size of 7×7, and a stride of [missing value]. The spatial attention convolution kernel is 7×7. The CBAM module weights the feature map by calculating channel attention weights and spatial attention weights, thereby enhancing the expressive power of important features.

[0101] 4. Multi-scale feature fusion module:

[0102] Upsampling: Use bilinear interpolation or transposed convolution to upsample the low-resolution feature map to make it the same size as the high-resolution feature map.

[0103] Downsampling: Use pooling operations to downsample the high-resolution feature map to make it the same size as the low-resolution feature map.

[0104] Feature fusion: Feature maps of different scales are stitched together along the channel dimension to form a fused feature map of 28×28×256.

[0105] After all residual stages, upsampling (bilinear interpolation or transposed convolution) or downsampling (pooling operation) is used to adjust the feature maps at different scales to make their sizes consistent. Specifically, stage 1 outputs high-resolution detail features (56×56), stage 2 outputs medium-resolution structural features (28×28), stage 3 outputs low-resolution semantic features (14×14), and stage 4 outputs deep abstract features (7×7). For the feature maps output from each stage, the resolution is uniformly adjusted to 28×28, a 1×1 convolution is used to unify the number of channels to 256, and the convolution is performed along the channel dimension to output a 28×28×256 feature map.

[0106] 5. Multi-tag classification header module:

[0107] Global Average Pooling (GAP) layer: Performs global average pooling on the fused feature map, outputting a 256-dimensional feature vector.

[0108] Fully Connected (FC) Layer: Passes a 256-dimensional feature vector through a fully connected layer and outputs the predicted probability of each label.

[0109] Independent Binary Classifier: Each label corresponds to an independent binary classifier, and the output is a single clone. , , , , , , , and the probability of oligoclonalization .

[0110] The final decision-making module of the model includes a global average pooling layer (GAP), one shared fully connected layer (FC), and multiple independent binary classifiers (one for each label). The target label is set to single-clone. , , , , , , , Oligoclonal classification uses single-label classification, while biclonal classification uses multi-label classification, with each clone type treated as an independent binary classification task. Specifically, a global average pooling layer performs global averaging on the feature map, outputting a 256-dimensional feature vector. A fully connected layer maps the 256-dimensional feature vector to the label space, and multiple independent binary classifiers perform binary classification on each label, outputting the probability of each label.

[0111] Let's illustrate the construction process of a neural network model with a specific implementation example. Assume we have a standardized immunofixation electrophoresis film image with dimensions of 224*224*1. First, the image passes through an initial convolutional layer, followed by a 7×7 convolutional kernel, a batch normalization layer, and a max pooling layer, outputting a 56×56×64 feature map. Then, the feature map sequentially passes through four residual stages, each containing multiple residual blocks. Each residual block has a CBAM module inserted after the base convolutional layer for attention weighting. After all residual stages are completed, the feature map enters a multi-scale feature fusion module. This module adjusts the feature map size at different scales through upsampling or downsampling and concatenates them along the channel dimension to form a 28×28×256 fused feature map. Finally, the fused feature map passes through a global average pooling layer, a fully connected layer, and an independent binary classifier, outputting the predicted probability for each label. Through these steps, the neural network model is able to accurately extract image features and perform classification.

[0112] S300: The scanned image to be analyzed is used to train the initial neural network model for subtyping. The subtyping training includes pre-training and fine-tuning training. The pre-training is performed through self-supervised tasks such as puzzle restoration and rotation prediction. In the fine-tuning training stage, labeled data is used to perform dynamic balancing of multi-task weights and clinical rule constraint reinforcement training to obtain the intelligent subtyping model to be tested.

[0113] In some embodiments, the pre-training employs a self-supervised learning task to enable the model to learn the basic features and structure of an image without labeled data. The pre-trained jigsaw puzzle reconstruction task includes:

[0114] The image received by the input layer is divided into... Grid, resulting in 9 The sub-block;

[0115] Randomly shuffle the order of the sub-blocks to generate a shuffled image;

[0116] Each sub-block is processed by the initial convolutional layer, and the output is... The feature map is formed by concatenating the feature maps of all sub-blocks along the channel dimension. The fused feature map outputs a probability distribution predicting the order of image sub-blocks through a global average pooling layer and a fully connected layer. ;

[0117] The calculation is performed using the first cross-entropy loss function:

[0118]

[0119] in As the first true label, The parameters of the initial convolutional layer and the classification head are optimized through backpropagation to obtain the first prediction probability.

[0120] In some embodiments, the pre-trained rotation prediction task includes:

[0121] The image received by the input layer is randomly rotated to generate a rotated image.

[0122] Through the initial convolutional layer and four residual stages, the output is... The deep features are obtained by passing them through a global average pooling layer and a fully connected layer, and outputting a 4-dimensional probability distribution. ;

[0123] The calculation is performed using the second cross-entropy loss function:

[0124]

[0125] in As a second real label, The parameters of the initial convolutional layer and the classification head are optimized through backpropagation to obtain the second prediction probability.

[0126] In some embodiments, the fine-tuning training uses labeled data to train the model, aiming to adapt it to a specific classification task. The specific steps are as follows:

[0127] The input layer receives the standardized image, which is then converted through an initial convolutional layer. Feature map;

[0128] The feature map passes through four residual stages in sequence. After each residual block in each residual stage outputs a feature map, it is immediately passed through the CBAM module for attention weighting.

[0129] After all residual stage processing is completed, the feature maps at different scales are adjusted through a multi-scale feature fusion module and stitched together along the channel dimension to form... fusion feature map;

[0130] The fused feature map is processed through a global average pooling layer, a fully connected layer, and independent binary classifiers to output the predicted probability of each label. ;

[0131] The calculation is performed using the third cross-entropy loss function:

[0132]

[0133] in, For the final true label, To ultimately predict the probability, the model parameters are adjusted through backpropagation, including weights, biases, scaling factors, offset factors, and attention mechanism parameters, so that the prediction results approximate the labeled results.

[0134] In some embodiments, the multi-task weight dynamic balancing training for fine-tuning includes:

[0135] The loss value is calculated independently for each puzzle reconstruction task or rotation prediction task. Calculate the gradient of the loss for each task with respect to the shared parameters. Adjust task weights according to gradient magnitude ratio This aligns the gradient magnitudes of each task.

[0136] Calculate the weighted total loss function: The parameters are updated during the backpropagation phase.

[0137] In some embodiments, the fine-tuning training of clinical rule constraints includes:

[0138] The diagnostic rules are transformed into differentiable mathematical expressions and directly used as regularization terms. Add to the model's loss function. During the backpropagation phase, the model parameters are updated using gradient descent.

[0139] Suppose we have an unlabeled immunofixation electrophoresis film image with dimensions 224*224*1. First, in the pre-training phase, the model learns the basic features and structure of the image through jigsaw puzzle reconstruction and rotation prediction tasks. Then, in the fine-tuning phase, the model is trained using labeled data, passing through initial convolutional layers, four residual stages, a multi-scale feature fusion module, and a multi-label classification head module, outputting the predicted probability for each label. By dynamically balancing multi-task weights and strengthening clinical rule constraints, the model can accurately extract image features and perform classification, resulting in the intelligent typing model under test.

[0140] S400: Acquire a standardized immunofixation electrophoresis film scan image as a test image, input the test image into the intelligent typing model to be tested, calculate the performance index of the intelligent typing model to be tested, and select the model whose performance index exceeds the preset value as the final intelligent typing model.

[0141] During the model testing phase, double-blind labeled electrophoresis images, which were not used during training, were processed using the same standardized preprocessing methods as during training. The testing process includes the following steps:

[0142] Image preprocessing: The test images undergo the same standardization processing as the training phase, including color normalization, strip ROI extraction, Gaussian filtering for noise reduction, contrast enhancement, edge detection, binarization, and grayscale normalization, to ensure consistency between the test images and training images in feature extraction and classification tasks.

[0143] Model inference: The preprocessed image is input into the trained neural network model. The model performs inference through an initial convolutional layer, four residual stages, a multi-scale feature fusion module, and a multi-label classification head module, outputting the predicted probability for each label. .

[0144] Performance metrics calculation: Based on the model's prediction results and the true labels from the double-blind annotation, the following performance metrics are calculated:

[0145] Accuracy:

[0146]

[0147] in, For a real example, As a true counterexample, This is a false positive example. This is a false negative. Accuracy reflects the proportion of samples correctly classified by the model.

[0148] Precision:

[0149]

[0150] Precision reflects the proportion of samples that the model predicted as positive, but which were actually positive.

[0151] Recall:

[0152]

[0153] Recall reflects the proportion of samples that are actually positive, which the model predicts as positive.

[0154] F1-Score:

[0155]

[0156] A qualified model achieves a recall rate ≥90%, precision ≥88%, F1-Score ≥0.85, and accuracy ≥85%. An excellent model achieves a recall rate and precision ≥95%, F1-Score ≥0.92, accuracy ≥92%, and a clinical rule violation rate <1%.

[0157] To improve model performance and adapt to dynamically changing clinical needs, a continuous learning and online update mechanism is employed to optimize the model. Specific optimization methods include:

[0158] Continuous learning: By introducing new labeled data, the model is allowed to learn incrementally based on existing knowledge. The specific steps are as follows:

[0159] New data collection: New immunofixation electrophoresis film images were collected and subjected to the same standardization and double-blind annotation as in the training phase.

[0160] Model fine-tuning: Inputting new data into an existing model and updating the model parameters through fine-tuning training, so that the model can adapt to new data distributions and clinical needs.

[0161] Online Update: This feature allows the model to dynamically update its weights in real-world applications through an online learning mechanism. The specific steps are as follows:

[0162] Error case labeling: Technicians label error cases during system use and feed these cases back to the intelligent analysis system.

[0163] Online fine-tuning: The system performs online fine-tuning of the model based on feedback error cases, adjusting model parameters to improve classification accuracy.

[0164] Clinical rule base maintenance: Dynamically maintain the clinical rule base and domain knowledge to ensure that the model follows the latest clinical rules and diagnostic criteria during inference. Specific steps are as follows:

[0165] Rule updates: Update the rules and knowledge in the clinical rule base according to the latest clinical research and diagnostic guidelines.

[0166] Rule constraints: The updated clinical rules are transformed into differentiable mathematical expressions and added as regularization terms to the model's loss function, and the model parameters are updated through gradient descent.

[0167] A specific implementation case illustrates the model testing and optimization process: Preprocessed images are input into a trained neural network model. The model performs inference through an initial convolutional layer, four residual stages, a multi-scale feature fusion module, and a multi-label classification head module, outputting the predicted probability for each label. Based on the model's predictions and the true labels from double-blind annotation, performance metrics such as accuracy, precision, recall, and F1-Score are calculated to evaluate the model's performance. If the model's performance does not meet expectations, it is optimized through continuous learning and online updates. New labeled data and feedback from erroneous cases are introduced, and the model weights and clinical rule base are dynamically updated to improve the model's classification accuracy and clinical applicability. The final intelligent typing model is obtained when the performance metrics exceed preset values.

[0168] S500: Deploy the final intelligent typing model to the electrophoresis pattern intelligent expert analysis system, monitor and capture the electrophoresis scan images generated by the electrophoresis instrument software system stored in the laboratory terminal through the file system, perform standardized preprocessing on the electrophoresis scan images, analyze the processed images through the final intelligent typing model, and generate intelligent typing results for the M protein.

[0169] The specific implementation steps for system integration are as follows:

[0170] Environment Configuration: Configure the system runtime environment on a server or high-performance computing platform, including the operating system, deep learning framework (such as TensorFlow or PyTorch), database management system, etc. Ensure the system has sufficient computing resources and storage space to support large-scale image processing and high-concurrency user access.

[0171] Model deployment: Deploying the trained neural network model into the system. Specific steps include:

[0172] Model Export: Export the trained model to a deployable format (such as SavedModel or ONNX).

[0173] Model loading: Load the exported model into the system and perform necessary initialization operations.

[0174] Interface encapsulation: Provides a unified API interface for the model, making it convenient for other modules to call.

[0175] Module Integration: The image acquisition module, image preprocessing module, neural network model module, diagnostic report generation module, user interface module, and continuous learning and online update module are integrated into the system. Ensure that data transmission and calling logic between modules are correct and error-free.

[0176] System testing: Conduct comprehensive testing on the integrated system, including functional testing, performance testing, and compatibility testing. Ensure the system can operate stably in different scenarios and meet the expected performance and accuracy requirements.

[0177] User training: Provide system usage training to clinical technicians and experts to ensure they can operate the system proficiently, correctly interpret diagnostic reports, and effectively provide feedback on erroneous cases and opinions.

[0178] System Deployment: The system is formally deployed to the actual work environment to begin supporting clinical diagnosis. Regular system maintenance and updates are performed to ensure long-term stable operation and continuous optimization.

[0179] Secondly, this application proposes an intelligent typing system for M protein in immunofixation electrophoresis images, including an image acquisition module, an image preprocessing module, a neural network model module, a diagnostic report generation module, a user interface module, and a continuous learning and online updating module;

[0180] The image acquisition module is used to capture the generated immunofixation electrophoresis film scan images from the software system of the electrophoresis instrument. This module uses a file system monitoring mechanism to detect new image files in a specified directory in real time and transmit them to the image preprocessing module.

[0181] The image preprocessing module is used to standardize the acquired images, including color normalization, strip ROI extraction, Gaussian filtering for noise reduction, contrast enhancement, edge detection, binarization, and grayscale normalization. This module ensures that the images input to the neural network model are consistent with the images in the training phase in terms of feature extraction and classification tasks.

[0182] The collection of immunofixation electrophoresis film scans must include normal samples and monoclonal immunoglobulin typing samples from diseases such as multiple myeloma, and must be double-blinded and labeled by at least two clinical technicians and experts. Currently, differences in staining agents used by different electrophoresis instruments (Helena electrophoresis instruments: acid violet staining; Sebia electrophoresis instruments: crystal violet staining), different film specifications, and different scanning devices (Helena electrophoresis instruments: external EPSON scanner; Sebia electrophoresis instruments: built-in scanner) are addressed through staining normalization. Standardization of electrophoresis images involves Vahadane staining normalization to eliminate staining agent and equipment differences, band ROI extraction, and grayscale normalization. This ensures that film scans from different brands of electrophoresis instruments can be recognized, improving data quality and usability, and facilitating the widespread use of intelligent diagnostic analysis systems in various clinical laboratories.

[0183] The neural network model module is used to deploy the trained neural network model into the system and is responsible for analyzing and classifying the preprocessed images. The module includes an input layer, an initial convolutional layer, four residual stages, a multi-scale feature fusion module, and a multi-label classification head module, which outputs the predicted probability of each label.

[0184] A neural network model is constructed based on deep learning algorithms. The ResNet50 convolutional backbone network embeds a Convolutional Convolutional Attention Module (CBAM), a multi-scale feature fusion module, and a multi-label classification head. The ResNet50 convolutional backbone network is adapted to electrophoresis band features. The CBAM module uses a dual attention mechanism (channel + spatial) to focus the network on the electrophoretic band regions. The multi-scale feature fusion module aggregates multi-scale features processed by CBAM at each stage to generate fused global features, enabling the model to focus on both details and understand the overall structure. In electrophoresis images, shallow high-resolution features detect weak bands, thus addressing the problem of weak signals in low-concentration monoclonal protein bands. Deep features help identify wide polyclonal immunoglobulin bands. The fusion module integrates features from various scales to determine the type. The multi-label classification head performs multi-label prediction based on the multi-scale feature fusion feature map. By explicitly modeling label coexistence relationships, it significantly improves the detection capability of complex cases such as biclonal and oligoclonal antibodies.

[0185] This model enables automatic interpretation and diagnosis of monoclonal immunoglobulin typing by automatically learning the features and patterns in immunofixation electrophoresis data.

[0186] The diagnostic report generation module is used to generate detailed diagnostic reports based on the prediction results of the neural network model. This module integrates the predicted immunoglobulin types and their probability values ​​into the report and provides visualized electrophoresis patterns and band analysis results for reference by clinical technicians and experts.

[0187] The user interface module provides a user-friendly interface, enabling technicians and experts to view diagnostic reports, mark error cases, and provide feedback. This module supports concurrent operation by multiple users, ensuring the system's efficiency and convenience.

[0188] The continuous learning and online update module is responsible for collecting error cases and newly labeled data from user feedback. It optimizes the neural network model through a continuous learning and online update mechanism. This module dynamically updates the model weights and clinical rule base to improve the model's classification accuracy and clinical applicability.

[0189] A two-stage training strategy is employed, from pre-training to fine-tuning, while an adaptive model training framework is embedded throughout the training process. In the pre-training stage, 10%-15% of the original images are mixed with standardized images, forcing the model to learn device-independent features to enhance robustness. The electrophoresis images learned in the pre-training stage are unlabeled, and general features of the electrophoresis images are learned through self-supervised learning (SimCLR). In the fine-tuning stage, labeled standardized images are used for training. The labeled data provides clear clinical diagnostic targets, and combined with transfer learning and data augmentation, the accuracy and rationality of the model's output clinical diagnostic results are ensured. The optimal training path for immunofixation electrophoresis pattern analysis is achieved through self-supervised pre-training and supervised fine-tuning, enabling accurate identification of immunofixation electrophoresis images and classification of monoclonal immunoglobulins. The adaptive model training framework dynamically senses the training status during training and adjusts the training strategy in real time based on feedback signals. The adaptive fine-tuning training strategy employs dynamic balancing of multi-task weights, reinforcement of clinical rule constraints, and active learning of difficult samples. The multi-task weight dynamic balancing automatically adjusts the loss weights based on the convergence speed of multiple tasks such as immunoglobulin typing, band localization, image segmentation, and quality assessment; the clinical rule constraint reinforcement strategy is based on the identification rules of the monoclonal gamma globulin laboratory diagnostic guidelines to ensure that the model output diagnostic results conform to the identification rules; and active learning is performed on difficult samples with low model prediction confidence (<0.6).

[0190] Thirdly, this application proposes an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0191] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described above.

[0192] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0193] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0194] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0195] In the embodiments provided in this disclosure, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. Multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0196] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0197] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0198] If an integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in a computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0199] The above are merely preferred embodiments of the present invention. It should be noted that any modifications and improvements made by those skilled in the art without departing from the present technical solution should also be considered to fall within the scope of protection claimed by the present solution.

Claims

1. A smart typing method for M protein in immunofixation electrophoresis images, characterized in that: Includes the following steps: Immunofixation electrophoresis film scan images were collected, and double-blind annotation and standardization were performed to obtain the scan images to be analyzed. An initial neural network model is constructed, wherein the architecture of the initial neural network model is a CBAM-ResNet50 convolutional backbone network; The scanned image to be analyzed is used to train the initial neural network model for subtyping. The subtyping training includes pre-training and fine-tuning training. The pre-training is performed through self-supervised tasks such as puzzle restoration and rotation prediction. In the fine-tuning training stage, labeled data is used to perform dynamic balancing of multi-task weights and clinical rule constraint reinforcement training to obtain the intelligent subtyping model to be tested. Obtain the immunofixation electrophoresis film scan image that has only undergone standardized processing as the test image, input the test image into the intelligent typing model to be tested, calculate the performance index of the intelligent typing model to be tested, and select the model whose performance index exceeds the preset value as the final intelligent typing model; The final intelligent typing model is deployed to the electrophoresis pattern intelligent expert analysis system. The system monitors and captures electrophoresis scan images generated by the electrophoresis instrument software system stored in the laboratory terminal through the file system. The electrophoresis scan images are standardized and preprocessed. The final intelligent typing model is then used to analyze the processed images and generate intelligent typing results for the M protein.

2. The method according to claim 1, characterized in that: The standardization process includes staining normalization, band ROI extraction, and grayscale normalization. The staining normalization is: converting the immunofixation electrophoresis film scan image obtained by double-blind labeling into optical density space for quantitative analysis of staining intensity; The Vahadane algorithm is used to separate the optical density vectors of various dyes using sparse nonnegative matrices, and the dyeing vector is extracted. and concentration matrix Maintain concentration matrix Keep it unchanged, apply the dye The generated coloring vector Replace with dye The generated coloring vector Then, the normalized image is reconstructed and inversely converted back to RGB space to ensure that images under different staining conditions have a consistent staining effect. The strip ROI extraction is as follows: Gaussian filtering is applied to the stained and normalized image to remove noise, preserving the main structural information of the image; contrast-limited adaptive histogram equalization is used to enhance the contrast of the strips; the boundaries of the strips are located by edge detection; the strips are binarized to convert the image into a black and white binary image. The grayscale normalization process involves matching a black-and-white binary image to a reference template, adjusting the image's mean, standard deviation, and nonlinear brightness, and then using grayscale normalization to ensure that the grayscale values ​​of different images have the same range and distribution, thus obtaining a standardized image.

3. The method according to claim 2, characterized in that: The initial neural network model includes an input layer, an initial convolutional layer, four residual stages, a multi-scale feature fusion module, and a multi-label classification head module. The initial convolutional layer, the four residual stages, the multi-scale feature fusion module, and the multi-label classification head module together constitute the CBAM-ResNet50 convolutional backbone network. The initial convolutional layer includes a convolutional layer, a batch normalization layer, and a pooling layer; The four residual stages include residual stage 1, residual stage 2, residual stage 3 and residual stage 4. Each stage consists of multiple residual blocks, with the number of residual blocks being [3, 4, 6, 3]. Each residual block has a CBAM module inserted after 3 basic convolutional layers. The CBAM module includes channel attention mechanism and spatial attention mechanism, with a compression ratio of 16, an input convolutional kernel of 7×7, a stride of 2, and a spatial attention convolutional kernel of 7×7. The multi-scale feature fusion module includes upsampling, downsampling, and feature fusion. The multi-label classification head module includes a global average pooling layer, a fully connected layer, and multiple independent binary classifiers.

4. The method according to claim 3, characterized in that: The pre-training employs a self-supervised learning task, allowing the model to learn the basic features and structure of an image without labeled data. The pre-trained jigsaw puzzle reconstruction task includes: The image received by the input layer is divided into... Grid, resulting in 9 The sub-block; Randomly shuffle the order of the sub-blocks to generate a shuffled image; Each sub-block is processed by the initial convolutional layer, and the output is... The feature map is formed by concatenating the feature maps of all sub-blocks along the channel dimension. The fused feature map outputs a probability distribution predicting the order of image sub-blocks through a global average pooling layer and a fully connected layer. ; The calculation is performed using the first cross-entropy loss function: in As the first true label, The parameters of the initial convolutional layer and the classification head are optimized through backpropagation to obtain the first prediction probability.

5. The method according to claim 4, characterized in that: The pre-trained rotation prediction task includes: The image received by the input layer is randomly rotated to generate a rotated image. Through the initial convolutional layer and four residual stages, the output is... The deep features are obtained by passing them through a global average pooling layer and a fully connected layer, and outputting a 4-dimensional probability distribution. ; The calculation is performed using the second cross-entropy loss function: in As a second real label, The parameters of the initial convolutional layer and the classification head are optimized through backpropagation to obtain the second prediction probability.

6. The method according to claim 5, characterized in that: The fine-tuning training uses labeled data to train the model, aiming to adapt it to specific classification tasks. The specific steps are as follows: The input layer receives the standardized image, which is then converted through an initial convolutional layer. Feature map; The feature map passes through four residual stages in sequence. After each residual block in each residual stage outputs a feature map, it is immediately passed through the CBAM module for attention weighting. After all residual stage processing is completed, the feature maps at different scales are adjusted through a multi-scale feature fusion module and stitched together along the channel dimension to form... fusion feature map; The fused feature map is processed through a global average pooling layer, a fully connected layer, and independent binary classifiers to output the predicted probability of each label. ; The calculation is performed using the third cross-entropy loss function: in, For the final true label, To ultimately predict the probability, the model parameters, including weights, biases, scaling factors, offset factors, and attention mechanism parameters, are adjusted through backpropagation to make the prediction results approximate the labeled values.

7. The method according to claim 6, characterized in that: The multi-task weight dynamic balancing training for fine-tuning training includes: The loss value is calculated independently for each puzzle reconstruction task or rotation prediction task. Calculate the gradient of the loss for each task with respect to the shared parameters. Adjust task weights according to gradient magnitude ratio This aligns the gradient magnitudes of each task. Calculate the weighted total loss function: The parameters are updated during the backpropagation phase.

8. The method according to claim 7, characterized in that: The fine-tuning training includes clinical rule-based constraint reinforcement training, which includes: The diagnostic rules are transformed into differentiable mathematical expressions and directly used as regularization terms. Add to the model's loss function. During the backpropagation phase, the model parameters are updated using gradient descent.

9. A smart typing system for M protein in immunofixation electrophoresis images, characterized in that, It includes an image acquisition module, an image preprocessing module, a neural network model module, a diagnostic report generation module, a user interface module, and a continuous learning and online update module; The image acquisition module is used to capture the generated immunofixation electrophoresis film scan images from the software system of the electrophoresis instrument. This module uses a file system monitoring mechanism to detect new image files in a specified directory in real time and transmit them to the image preprocessing module. The image preprocessing module is used to standardize the acquired images, including color normalization, strip ROI extraction, Gaussian filtering for noise reduction, contrast enhancement, edge detection, binarization, and grayscale normalization. This module ensures that the images input to the neural network model are consistent with the images in the training phase in terms of feature extraction and classification tasks. The neural network model module is used to deploy the trained neural network model into the system and is responsible for analyzing and classifying the preprocessed images. The module includes an input layer, an initial convolutional layer, four residual stages, a multi-scale feature fusion module, and a multi-label classification head module, which outputs the predicted probability of each label. The diagnostic report generation module is used to generate detailed diagnostic reports based on the prediction results of the neural network model. This module integrates the predicted immunoglobulin types and their probability values ​​into the report and provides visualized electrophoresis patterns and band analysis results for reference by clinical technicians and experts. The user interface module provides a user-friendly interface, enabling technicians and experts to view diagnostic reports, mark error cases, and provide feedback. This module supports concurrent operation by multiple users, ensuring the system's efficiency and convenience. The continuous learning and online update module is responsible for collecting error cases and newly labeled data from user feedback. It optimizes the neural network model through a continuous learning and online update mechanism. This module dynamically updates the model weights and clinical rule base to improve the model's classification accuracy and clinical applicability.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method as described in any one of claims 1-8.

Citation Information

Patent Citations

  • Immune fixation electrophoresis image recognition method combining anti-deductive learning and deep learning

    CN117612163A

  • Histological stain pattern and artifacts classification using few-shot learning

    US20230196803A1