Interstitial tumor mitotic cell recognition and counting method based on deep learning
Through deep learning models and multiple optimization technologies, the problems of insufficient generalization ability of gastrointestinal stromal tumor mitotic cells recognition and counting and poor small target recognition effect in the prior art are solved, and efficient and accurate identification and counting are achieved , reduces the burden on doctors and improves the efficiency and quality of clinical diagnosis.
Patent Information
- Application Number
- CN202411119797.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-15
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-08-15
AI Technical Summary
The existing deep learning model lacks generalization ability in the recognition and count of mitotic cells of gastrointestinal stromal tumors, and it is difficult to cope with slices of different types and staining conditions. It has poor recognition effect on small targets, and lacks an automated error correction mechanism, resulting in The identification accuracy is low, the work burden on doctors is heavy, and clinical application is limited.
Using deep learning-based stromal tumor mitotic cell recognition and counting methods, combined with a variety of optimization technologies, including image preprocessing, data augmentation, multi-source domain adaptation technology and automated error detection and correction mechanisms, using YOLOv10 models and innovative modules such as C2fUIB, PSA, SCDown and SPPF improve the robustness and accuracy of the model through dual allocation strategies and transfer learning.
It significantly improves the recognition and counting efficiency of mitotic cells, reduces the possibility of errors, improves the stability and accuracy of the model, maintains high accuracy on different pathological sections, reduces the work burden of doctors, and realizes automated diagnosis. Assisted, it improves the efficiency of pathological analysis and diagnosis quality.
Smart Images

Figure CN118918096A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of bioinformatics, and in particular to a method for identifying and counting mitotic cells of mesenchymal tumors based on deep learning. Background Art
[0002] In the medical field, gastrointestinal stromal tumor (GIST) is a relatively common soft tissue tumor with a high degree of malignancy. Early diagnosis and treatment are crucial to improving the survival rate of patients. When performing pathological analysis of gastrointestinal stromal tumor, identifying and counting mitotic cells in tumor tissue is an important basis for evaluating tumor proliferation activity and judging the degree of tumor malignancy.
[0003] At present, the identification and counting of mitotic cells in clinical practice mainly rely on manual operations by pathologists. Doctors observe stained tissue sections under a microscope and manually mark and count mitotic cells. This method requires extremely high professional experience from doctors, and the operation process is cumbersome, time-consuming and laborious. In addition, the manual identification process is limited by the doctor's energy and attention. Long-term repetitive work can easily lead to fatigue, which in turn affects the accuracy and efficiency of identification.
[0004] Although with the development of computer vision technology, deep learning models have shown excellent performance in the field of image recognition and have gradually been introduced into medical image analysis, some studies have tried to use deep learning models to automatically identify mitotic cells in tissue sections. However, the existing automated recognition systems still have some problems:
[0005] First, existing deep learning models are usually trained for specific types of pathological sections, and they lack generalization capabilities when dealing with sections of different types and with different staining conditions. This makes the model unable to cope with complex and diverse pathological images in actual clinical applications, and is prone to missed detection or false detection.
[0006] Secondly, because mitotic cells account for a very small proportion of pathological sections and are often mixed with surrounding normal cells, existing models are less effective in identifying these small targets. Convolutional neural networks (CNNs) are easily limited by convolution operations when processing small targets and cannot accurately detect all mitotic cells.
[0007] In addition, the existing system lacks an automated error correction mechanism. Even in the model prediction results, doctors still need to manually verify and correct them. However, the existing system lacks an effective error detection and correction mechanism, and doctors need to spend a lot of time checking the model's prediction results one by one, which is almost the same as manual operation, resulting in limited overall efficiency improvement.
[0008] These problems have resulted in low recognition accuracy, heavy workload for doctors, and limited clinical applications. Due to the lack of generalization ability of the model and poor recognition of small targets, the existing systems often cannot achieve the accuracy of manual operation by doctors in actual clinical applications. Missed detection or false detection occurs from time to time, which may directly affect the judgment of tumor proliferation activity, leading to misdiagnosis or delayed treatment. Although the existing system can preliminarily screen out some mitotic cells, it still requires doctors to perform a lot of manual verification and correction, resulting in the failure to significantly reduce the workload of doctors, and repetitive operations are prone to fatigue, further increasing the risk of errors. Due to problems with recognition accuracy and efficiency, the promotion of existing automated systems in actual clinical applications is limited, doctors and hospitals do not have high trust in them, the system cannot fully play its due role, and fails to achieve true automated diagnostic assistance, affecting the overall efficiency and diagnostic quality of pathological analysis. Summary of the invention
[0009] In order to overcome the shortcomings of the existing technology, the present invention proposes a method for identifying and counting mitotic cells of gastrointestinal stromal tumors based on deep learning. By combining the deep learning model and a variety of optimization techniques, efficient and accurate identification of mitotic cells of gastrointestinal stromal tumors is achieved, which not only greatly improves work efficiency and reduces the possibility of mislabeling, but also provides solid technical support for clinical diagnosis and treatment.
[0010] To achieve the above object, the present invention provides a method for identifying and counting mitotic cells of mesenchymal tumors based on deep learning, comprising the following steps:
[0011] Step 1: Obtain and preprocess the stained gastrointestinal stromal tumor tissue slice image, the preprocessing includes color normalization, noise removal, contrast enhancement and other operations on the slice image to ensure the quality and consistency of the slice image and eliminate the human differences in the staining process. The preprocessing step also includes converting the image into a grayscale image to simplify the input form of the model.
[0012] Step 2: Segment the preprocessed slice image to generate image patches suitable for deep learning model processing. The segmentation operation uses a step size of 256 pixels to divide the 512×512 pixel slice into multiple smaller sub-image patches to ensure that the target cells are completely retained in different image patches and reduce the impact of edge effects on model recognition accuracy. For image patches that do not contain target cells or contain only a few cells, they are further eliminated through RGB pixel value threshold screening.
[0013] Step 3: Use data augmentation techniques to expand the generated image blocks, including but not limited to color adjustment (such as hue, saturation, sharpness, and contrast), random rotation, translation, scaling, cropping, affine transformation, and random copying of target cells to other image locations through copy-paste technology. This data augmentation step is designed to simulate various image variations that may occur in actual operations, thereby improving the robustness and generalization ability of the deep learning model.
[0014] Step 4: Build a target detection model based on YOLOv10 to identify and locate mitotic cells in images. The YOLOv10 model contains multiple innovative modules, including C2fUIB module, Partial self-attention (PSA) module, Spatial-channel decoupled downsampling (SCDown) module and improved Spatial Pyramid Pooling for Fast interface (SPPF) module. The model implements NMS (Non-Maximum Suppression)-free training through a dual allocation strategy, which significantly improves the inference efficiency and accuracy of the model.
[0015] Step 5: Apply multi-source domain adaptation technology to enhance the generalization ability of the model by introducing data from multiple source domains. The multi-source domain adaptation technology includes using multiple target detection heads in the source domain training process, each detection head corresponds to specific source domain data, and using the gradient reversal module to generate domain adversarial features to reduce mutual interference between source domains. Subsequently, the student model is trained on the target domain, and pseudo labels are generated by the teacher model to further improve the detection accuracy in the target domain.
[0016] Step 6: Count the identified mitotic cells and generate corresponding statistical data. The statistical data include the total number of mitotic cells in each slice, cell density distribution, confidence ranking results, and the distribution pattern of mitotic cells in different pathological slices. The statistical results can be directly used to assist doctors in disease assessment and treatment plan formulation.
[0017] Step 7: Display the model's prediction results on the functional platform, and doctors verify and correct the annotation results. The functional platform includes an intuitive user interface that allows doctors to view the specific location and annotation results of each cell identified by the model. Doctors can make annotation corrections by clicking and dragging. The platform also supports batch operations by doctors to improve work efficiency.
[0018] Step 8: Retrain the model using the doctor's corrected data to continuously optimize the model's recognition and counting performance. The retraining step includes dividing the corrected data set into multiple cross-validation sets, adjusting model hyperparameters (such as learning rate, batch size, number of training rounds, etc.) and introducing new data enhancement strategies to further improve the stability and accuracy of the model in practical applications.
[0019] Furthermore, the data enhancement technique applied in step 3 includes the following operations:
[0020] Step 3.1: Color adjustment, including adjusting the hue, saturation, sharpness and contrast of the image to simulate the image changes under different staining conditions to ensure that the model can effectively identify target cells under different staining effects;
[0021] Step 3.2: Affine transformation, including random rotation (0°, 90°, 180°, 270°), translation, scaling, and cropping operations to simulate the possible changes in slicing and shooting angles in actual operations;
[0022] Step 3.3: copy-paste operation, randomly copying the identified target cell area to other locations in the image, thereby increasing the diversity of target cells in the training data and enhancing the robustness of the model;
[0023] Step 3.4: Data denoising and background removal, remove pure white areas or areas with almost no cells, and reduce the interference of irrelevant information on the model.
[0024] Furthermore, the innovative modules of the YOLOv10 model used in step 4 include:
[0025] Step 4.1: C2fUIB module, by adopting reparameterization technology, increases the receptive field of the convolution kernel and reduces the computational consumption, thereby improving the detection accuracy and inference speed of the model;
[0026] Step 4.2: Partial Self-Attention Module (PSA), by dividing the feature map into two parts, one part is processed by the Multi-Head Self-Attention Module (MHSA), and the other part keeps the original features, and then the two parts of the features are fused to enhance the model's detection ability for small objects;
[0027] Step 4.3: Spatial channel decoupling downsampling module (SCDown), which reduces the computational cost and number of parameters by decoupling spatial reduction and channel increase operations, while maximizing information retention and enhancing the recognition ability of small objects during the downsampling process;
[0028] Step 4.4: The improved spatial pyramid pooling fast interface (SPPF) reduces the number of parameters and maintains the multi-scale feature aggregation capability by changing the parallel structure to a serial structure, thereby improving the model's ability to detect objects of different scales.
[0029] Furthermore, the multi-source domain adaptation technology in step 5 includes:
[0030] Step 5.1: In the source domain training process, multiple source domains correspond to multiple target detection heads one by one, and the feature encoders are shared. During back propagation, domain adversarial features are generated through the gradient reversal module, thereby reducing the mutual interference between source domains and optimizing the adaptation performance of the target domain.
[0031] Step 5.2: Target domain training process: Use the pseudo labels generated by the teacher model to train the student model on the target domain, and gradually update the weights of the student model to enhance its detection accuracy in the target domain;
[0032] Step 5.3: Domain adaptation effect evaluation: t-SNE dimensionality reduction technology is used to visualize the features of the source domain and the target domain, verify the adaptability of the model to the feature distribution between different domains, and evaluate the actual effect of the multi-source domain adaptation technology by comparing with the baseline model.
[0033] Furthermore, the statistical data in step 6 further includes:
[0034] Step 6.1: A detailed location distribution map of mitotic cells in each pathological section, wherein the location of the detected mitotic cells in the original section image is marked to form a two-dimensional coordinate system to display the precise coordinate location of each cell. This distribution map can intuitively display the spatial distribution of mitotic cells in tumor tissues, helping doctors better understand the proliferation pattern and regional activity of tumors.
[0035] Step 6.2: Generate a heat map based on cell density. The heat map calculates the density of mitotic cells in the slice and displays the cell density in different areas in the form of color gradient. The depth of color corresponds to the density of cells. The heat map can help doctors quickly identify the most active proliferation areas in the tumor, providing an important basis for pathological diagnosis and the formulation of treatment plans.
[0036] Step 6.3: A list of mitotic cells sorted by confidence, wherein the list is sorted based on the confidence value of each cell detected by the model. The confidence value indicates the degree of certainty of the model that the cell is a mitotic cell, and the doctor can prioritize and correct the recognition results with lower confidence. The list should include the confidence score, location information, and related image blocks of each cell so that the doctor can quickly locate and judge.
[0037] Step 6.4: Count the total number and average density of mitotic cells in different pathological sections. The statistical data should include the total number of cells in each section, the cell density per square millimeter, and the local density calculated based on different tumor areas. These statistical results can be used to compare the tumor proliferation characteristics of different patients or different sections, and provide reference data for personalized treatment.
[0038] Step 6.5: Generate a summary report containing multiple patient slices, which integrates all test results, including the total cell number, average density, active area distribution, and confidence analysis of each patient, to form a complete test report to help doctors make an overall evaluation and comparison.
[0039] Furthermore, the retraining process in step 8 further includes:
[0040] Step 7.1: Collection and organization of revised data. The data is generated by doctors after they revise the model prediction results on the functional platform. The data includes revised cell annotation, confidence adjustment, and removal of false detection information. The organization process includes labeling and error analysis of the revised data, and merging the revised data into a new training set.
[0041] Step 7.2: Division and preparation of training sets: divide the collected corrected data into multiple cross-validation sets according to a certain ratio (e.g. 70% training set, 15% validation set, 15% test set) to ensure the generalization ability of the model on different data sets. The preparation of training sets also includes secondary enhancement of the data to ensure the diversity and randomness of the data.
[0042] Step 7.3: Optimize and adjust the model hyperparameters. Before each retraining, optimize and adjust the model hyperparameters (such as learning rate, batch size, number of training rounds, etc.) according to the scale and characteristics of the correction data. Use hyperparameter tuning tools such as Optuna to automatically search within the set range, and determine the best hyperparameter combination through multiple rounds of experiments.
[0043] Step 7.4: Execution of the retraining process, which includes inputting the corrected training data into the model for retraining, monitoring the accuracy and loss value of the model through the validation set, and adopting an early stopping strategy (stopping training when the accuracy of the validation set no longer improves) to avoid overfitting. After the training is completed, the final weights of the model will be saved and used for the next detection task.
[0044] Step 7.5: Model evaluation after retraining, which includes a comprehensive analysis of the model's performance on the test set, with a special focus on the model's performance improvement on the revised data, including indicators such as recall, precision, and F1 score. The effect of retraining is evaluated by comparing with the initial model and other baseline models, and the evaluation results are fed back to the development and application teams to decide whether to further optimize.
[0045] Step 7.6: Model deployment and update: Deploy the retrained and evaluated model to the functional platform and update it to the system used by doctors in real time. Ensure that the new model can automatically replace the old model and ensure its stability and accuracy in actual use. At the same time, record the feedback on the use of the new model for reference in the next retraining.
[0046] Furthermore, the recognition accuracy and robustness of the model are further optimized through multi-scale feature aggregation and model enhancement training, the steps of which include:
[0047] Step 8.1: Extract multi-scale features. Input the preprocessed image blocks into the deep learning model and extract feature maps of different scales through specific network layers to ensure that the model can capture multiple morphological features of cells.
[0048] Step 8.2: Perform feature fusion, fuse the extracted features of different scales, and use the spatial pyramid pooling module or its improved version to aggregate multi-scale information to enhance the model's ability to recognize small targets;
[0049] Step 8.3: Introduce model enhancement training technology, through the hybrid loss function and data enhancement strategy, enhance the model's ability to resist complex samples and noise, and improve recognition accuracy;
[0050] Step 8.4: Use transfer learning technology to take the pre-trained model trained in the related field as the basis, and improve the adaptability of the model to the target task by fine-tuning it in the target field.
[0051] Furthermore, an automated error detection and correction mechanism is included, the steps of which include:
[0052] Step 9.1: Automatic verification of model prediction results. The system performs preliminary verification on the mitotic cells detected by the model and identifies annotations with low confidence or possible errors.
[0053] Step 9.2: Generate an error detection report. The system will generate an error report for the low-confidence or suspected erroneous annotations detected, listing in detail the cell locations that may have problems and the corresponding confidence values;
[0054] Step 9.3: Manual correction and feedback: The doctor manually corrects the annotations in the error report, confirms or corrects the detection results of the model, and feeds the corrected data back to the system;
[0055] Step 9.4: Dynamically adjust the model weights. The system automatically adjusts the model weights and parameters based on the doctor's correction results to improve the accuracy of the model in subsequent tasks.
[0056] Compared with the prior art, the present invention has the following beneficial effects:
[0057] 1. The present invention provides a method for identifying and counting mitotic cells of mesenchymal tumors based on deep learning. By transferring the doctor's work from tedious manual identification to the correction of model prediction results, the doctor no longer needs to look for mitotic cells one by one on the complete pathological sections. This greatly saves the doctor's working time and energy and improves the overall work efficiency; in the process of retraining the model, the doctor's corrected data is used for optimization to continuously improve the recognition accuracy of the model. With the enhancement of the model's recognition ability, its stability and robustness on different pathological sections are effectively guaranteed. The model can adapt to a variety of pathological section data and maintain a high accuracy rate.
[0058] 2. The present invention provides a method for identifying and counting mitotic cells in stromal tumors based on deep learning. Since the model can accurately identify the vast majority of mitotic cells during the detection process, doctors only need to correct the results predicted by the model. This correction process only targets the parts of the model annotation that have low confidence or may have errors, significantly reducing the possibility of mislabeling. In this way, the performance of the model can be continuously optimized and the error rate can be gradually reduced.
[0059] 3. The present invention provides a method for identifying and counting mitotic cells in mesenchymal tumors based on deep learning. In actual tests, the model can predict hundreds of thousands of image patches of multiple patients in a very short time. The model summarizes the detection results by confidence through automated calculation and processing, and generates accurate mitotic cell statistics. Compared with traditional manual operations, the model shows significant advantages in processing large-scale pathological data, greatly shortening the detection time.
[0060] 4. The present invention provides a method for identifying and counting mitotic cells in stromal tumors based on deep learning. The detection results of the model are not much different from those of doctors' manual operations, and can better complete the identification of mitotic cells. The prediction accuracy of the model is highly close to that of manual annotation, providing reliable technical support for pathological analysis and clinical diagnosis; through automated detection and statistics, the model can generate detailed cell density distribution maps and heat maps to help doctors better understand the proliferation pattern and regional activity of tumors. These data provide a scientific basis for doctors' diagnosis and treatment plan formulation, and promote the implementation of personalized treatment. BRIEF DESCRIPTION OF THE DRAWINGS
[0061] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0062] Figure 1 This is the YOLOv10 structure diagram of the present invention
[0063] Figure 2 This is a schematic diagram of the SPPF structure of the present invention.
[0064] Figure 3 The multi-source domain adaptation method of the present invention
[0065] Figure 4-5 Schematic diagram of the migration of characteristic domains of the present invention
[0066] Figure 6 Schematic diagram of data enhancement of the present invention
[0067] Figure 7 Schematic diagram of file upload of the present invention
[0068] Figure 8 A schematic diagram of the life cycle of the present invention
[0069] Fig. 9 Schematic diagram of file downloading of the present invention
[0070] Fig.10 Schematic diagram of an embodiment of the present invention DETAILED DESCRIPTION
[0071] The technical solution of the present invention will be more clearly and completely explained below through description of preferred embodiments of the present invention in combination with the accompanying drawings.
[0072] Terminology explanation:
[0073] Explanation of the abbreviations:
[0074] GIST (Gastrointestinal Stromal Tumor): Gastrointestinal stromal tumor, a common soft tissue tumor.
[0075] CNN (Convolutional Neural Network): Convolutional neural network, a deep learning model structure used for image recognition and classification.
[0076] YOLOv10 (You Only Look Once, version 10): An advanced object detection model with high accuracy and high speed.
[0077] C2fUIB: A module in the YOLOv10 model that replaces the Bottleneck structure in YOLOv8 and uses a complex convolution kernel design to increase the receptive field and reduce computational consumption.
[0078] PSA (Partial Self-Attention): Partial self-attention module, which enhances the small target detection capability by splitting the feature map and processing part of it using a multi-head self-attention module.
[0079] SCDown (Spatial: Channel Decoupled Downsampling): A spatial channel decoupled downsampling module that reduces computational cost and number of parameters while preserving information by decoupling spatial reduction and channel increase operations.
[0080] SPP (Spatial Pyramid Pooling): Spatial pyramid pooling, a technology that aggregates multi-scale feature information, is used to enhance the model's ability to detect targets of different scales.
[0081] SPPF (Spatial Pyramid Pooling for Fast interface): An improved spatial pyramid pooling module that changes the parallel structure to a serial structure, reducing the number of parameters while maintaining multi-scale feature aggregation capabilities.
[0082] NMS (Non: Maximum Suppression): Non-maximum suppression, a post-processing step used in target detection algorithms to reduce repeated detection boxes.
[0083] MHSA (Multi: Head Self: Attention): Multi-head self-attention, often used to enhance the contextual information processing capabilities of neural network models.
[0084] EMA (Exponential Moving Average): Exponential moving average, used to smooth time series data or update model weights.
[0085] T-SNE (t-Distributed Stochastic Neighbor Embedding): t-Distributed Stochastic Neighbor Embedding, a technique for dimensionality reduction and visualization of high-dimensional data.
[0086] TPU (Tensor Processing Unit): Tensor processing unit, a hardware device developed by Google to accelerate machine learning model training and reasoning.
[0087] GPU (Graphics Processing Unit): Graphics processing unit, commonly used to accelerate image processing and training and reasoning of deep learning models.
[0088] Optuna: A Python library for hyperparameter optimization that improves model performance by automatically searching for the optimal hyperparameter combination.
[0089] FLOPs (Floating Point Operations per Second): The number of floating point operations per second, an indicator used to measure the computational complexity of the model.
[0090] KL Divergence (Kullback: Leibler Divergence): KL divergence, a statistic that measures the difference between two probability distributions.
[0091] MMD (Maximum Mean Discrepancy): Maximum mean difference, a method to measure the difference between different distributions, often used in domain adaptation and transfer learning.
[0092] TF: IDF (Term Frequency: Inverse Document Frequency): Term frequency: Inverse document frequency, a statistical method used in information retrieval and text mining to evaluate the importance of words in a document.
[0093] JSON (JavaScript Object Notation): A lightweight data exchange format commonly used for network data transmission and storage.
[0094] Labelme: An open source tool for image annotation, commonly used in the production of deep learning datasets.
[0095] SVS (Scalable Vector Graphics): Scalable vector graphics, an XML-based vector graphics format, usually used for the storage and display of high-resolution images.
[0096] The present invention is specifically:
[0097] 1. Image preprocessing
[0098] 1.1 Slice acquisition and image preprocessing
[0099] Starting with a patient's GIST tissue section, the sections are first stained to ensure that mitotic cells can be clearly identified in the microscopic image. The stained tissue sections are image scanned to generate high-resolution digital images. The images are usually oversized, the processing efficiency of the original images is low, and the noise and uneven staining in the images may interfere with the recognition accuracy of the model.
[0100] 1.2 Image segmentation
[0101] The large-size image is divided into 512×512 pixel patches with a step size of 256 pixels. This segmentation method helps to maintain the integrity of cells and facilitates the model to identify the morphological characteristics of cells. For the segmented image patches, preliminary screening is performed to eliminate those image patches with almost no cells or with a large edge ratio, reducing the computational burden of the model processing invalid data.
[0102] 1.3 Data Augmentation
[0103] Perform data augmentation on the filtered image blocks, such as Figure 6 As shown, to improve the robustness and generalization ability of the model. Data enhancement methods include:
[0104] Color adjustment: Randomly adjust the hue, saturation, sharpness and contrast of the image to simulate image changes under different dyeing conditions.
[0105] Affine transformation: Randomly rotate (0°, 90°, 180°, 270°), translate, scale, and crop the image to enhance the model’s ability to cope with image geometric transformations.
[0106] Copy-paste technology: randomly copy the identified target cells to other locations in the image to increase the diversity of target cells in the training data.
[0107] 2. Model construction and training
[0108] 2.1 Backbone Model Selection
[0109] YOLOv10 is selected as the basic detection model, which has high detection speed and accuracy. The structure of the YOLOv10 model has been optimized and contains the following key modules:
[0110] C2fUIB module: replaces the Bottleneck structure in YOLOv8 and uses a more complex convolution kernel design to increase the receptive field and reduce computational consumption.
[0111] Partial Self-Attention Module (PSA): The feature map is divided into two parts, one of which is processed by the Multi-Head Self-Attention Module (MHSA) and the other keeps the original features. The two parts are then fused to enhance the detection capability of small objects.
[0112] Spatial Channel Decoupled Downsampling Module (SCDown): By decoupling spatial reduction and channel addition operations, it reduces the computational cost and the number of parameters while preserving information and improving the small object detection performance during the downsampling process.
[0113] 2.2 Model Training
[0114] During the training process, multi-source domain adaptation technology is used to enhance the generalization ability of the model. Specifically:
[0115] Source domain training: The model is trained on multiple source domains, and each source domain data corresponds to an independent object detection head to reduce mutual interference between source domains.
[0116] Target domain training: Generate pseudo labels through the teacher model, train the student model on the target domain, and gradually adjust the model weights to improve the detection accuracy on the target domain.
[0117] 2.3 Model Optimization
[0118] Model enhancement training techniques were used, including the use of mixed loss functions and data enhancement strategies, to improve the model's ability to resist complex samples and noise. On this basis, transfer learning techniques were used to fine-tune the pre-trained model in related fields to improve the model's performance in the task of identifying mitotic cells in gastrointestinal stromal tumors.
[0119] 3. Identification and counting of mitotic cells
[0120] 3.1 Image block input and recognition
[0121] The preprocessed image blocks are input into the trained YOLOv10 model. The model identifies mitotic cells in the image through multi-scale feature aggregation and generates corresponding detection results, including the position coordinates and confidence scores of each cell.
[0122] 3.2 Counting and Statistics
[0123] Once the model has identified all mitotic cells, the system automatically counts these cells and generates detailed statistics. The statistics include:
[0124] The total number of cells in each section
[0125] Cell density distribution in each area
[0126] Test results sorted by confidence
[0127] Generate a heat map of each slice to show the density and distribution of cells
[0128] 4. Functional platform and doctor feedback
[0129] 4.1 Functional platform display and doctor correction
[0130] The prediction results of the model are displayed on the functional platform, and doctors can view the test results of each slice and manually verify and correct the annotations. The functional platform provides an intuitive user interface that allows doctors to click and drag to correct annotations.
[0131] 4.2 Feedback Data Collection and Retraining
[0132] After the doctor completes the annotation correction, the system automatically collects and summarizes the corrected data to generate a new training data set. The system automatically starts the retraining process, adjusts the model's weights and parameters based on the doctor's feedback, and improves the model's accuracy and stability.
[0133] 4.3 Model Update and Deployment
[0134] After retraining is completed, the system generates a new version of the model and updates the model to the doctor through an automated push mechanism, replacing the old version of the model. The system also backs up the old version of the model so that it can quickly restore to the previous version when the new version of the model performs poorly.
[0135] 5. Automated error detection and correction
[0136] 5.1 Automatic verification and error report generation
[0137] The system automatically verifies each model prediction result, identifies annotations with low confidence or possible errors, and generates a detailed error detection report, listing the suspected erroneous cell locations and corresponding confidence values.
[0138] 5.2 Manual Correction and Dynamic Adjustment
[0139] Doctors manually correct the annotations based on the error reports and feed the corrected data back to the system. The system automatically adjusts the model’s weights and parameters based on the correction results to further improve the model’s accuracy.
[0140] The present invention can effectively realize the automated identification and counting of mitotic cells in gastrointestinal stromal tumors, significantly improve the efficiency and accuracy of detection, and provide important technical support for pathological diagnosis and treatment.
[0141] As a specific implementation method,
[0142] 1. Pre-image preprocessing
[0143] There are two problems with stained biological tissue sections of patients:
[0144] Compared to the input size of general deep models, the slice size is too large.
[0145] In order to make the image suitable for deep model analysis, the image is segmented here. In order to reduce the loss of target cells (being cut off) caused by segmentation, it is segmented into patches with a size of 512×512 with a step length of 256. The larger image is convenient for doctors to observe and mark the target cells. The marking tool is Labelme, and the method is to frame the target cells with a rectangular frame. At the same time, in order to improve the recall rate of target recognition, the 512x512 size slice is further segmented into 256x256 sub-slices (still with a step length), and only the sub-slices containing the target annotation are retained.
[0146] The patches with very few or even no cells are removed because they are the edge of the tissue slice image and often appear pure white in the final svs file. Take a patch with very few cells and convert it into a grayscale image. Then calculate the pixel value divided by the number of pixels to get the threshold. The other patches are processed in the same way. Therefore, the mean of the RGB pixel value of the slice can be thresholded to remove the part with no cells or very few cells. According to the doctor's examination, the cells separated by the cutting boundary are rarely the required mitotic cells, so it can be considered that such processing will not affect the final result.
[0147] 2) Different degrees of dyeing make the background color appear different.
[0148] In order to avoid the possible interference of different backgrounds on cell recognition, other image preprocessing operations besides segmentation are adopted here, including copy-paste (randomly copying the target to other parts of the image to form more data), color adjustment (hue, saturation, sharpness, contrast), and small-scale affine transformation (random cropping, rotation, etc., where rotation mainly includes clockwise 90°, 180° and 270°). The effect can be seen above. Figure 2 As shown, the goal of reducing the background gap is achieved as much as possible.
[0149] The overall style (size, color) of the cell slice image is relatively simple, but different imaging devices and operational differences in the staining process will cause slight disturbances (such as changes in the cell size scale, changes in the degree of staining, and changes in imaging clarity). Therefore, the above data enhancement method is a good data expansion for these changes, and at the same time, it does not destroy the morphological characteristics of the cells, nor does it destroy the meaning of the image.
[0150] Dividing cell detection
[0151] Dataset: After the above processing, we get the interstitial tumor mitotic cell dataset (hereinafter referred to as the interstitial tumor dataset), which consists of tissue slices from 14 patients, divided into 4252 slices containing target regions, with a total of 5409 target regions. The training, validation, and test sets are divided in a ratio of 3:1:1.
[0152] Cell detection:
[0153] Backbone model: The model uses the most mature industrial-grade detection network YOLOv10 at this stage, and its performance in accuracy and speed has reached a new level. In this work, the author first proposed continuous dual allocation for YOLOs NMS-free training, which brought competitive performance and low inference latency. In addition, the overall efficiency-accuracy driven model design strategy of YOLOS is introduced. The various components of YOLOS are comprehensively optimized from the perspectives of efficiency and accuracy, which greatly reduces the computational overhead and enhances performance. Specific models such as Figure 1 As shown:
[0154] Among them, C2fUIB just replaces the Bottleneck structure of C2f in YOLOv8 with CIB structure. Bottleneck structure is a very common convolutional network structure since ResNet, which will not be introduced here. CIB structure is similarly a 5-layer network of 3×3, 1×1, 3×3 (if RepVGGDW convolutional structure is used, it is 7×7. This structure uses re-parameterization to increase the convolution kernel to increase the receptive field of the convolutional network while reducing computational consumption and optimization problems), 1×1, 3×3.
[0155] Partial self-attention (PSA): The features are evenly divided into two parts after 1×1 convolution. Only one part is input into the NPSA block consisting of a multi-head self-attention module (MHSA) and a feed-forward network (FFN). Then, the two parts are connected and fused through 1×1 convolution. In addition, it follows that the dimensions of query and key are assigned to half of the value, and LayerNorm is replaced with BatchNorm for fast inference.
[0156] Spatial-channel decoupled downsampling (SCDown): Conventional downsampling uses 3×3 standard convolution with a stride of 2, and simultaneously implements spatial downsampling (from H×W to H / 2×W / 2) and channel transformation (from C to 2C). This introduces a non-negligible computational cost of O(9HWC^2) and the number of parameters of O(18C^2). V10 decouples the spatial reduction and channel increase operations to achieve more efficient downsampling. Specifically, point-to-point convolution is first used to adjust the channel dimension, and then depth-separable convolution is used for spatial downsampling. This reduces the computational cost to O(2HWC^2+9HWC) and the number of parameters to O(2C^2+18C). At the same time, it maximizes the retention of information in the downsampling process, thereby maintaining competitive performance while reducing latency.
[0157] Spatial Pyramid Pooling for Fast interface (SPPF): An improved version of Spatial Pyramid Pooling (SPP), which changes from a parallel structure to a serial structure. Here, the receptive field of the second 5x5 maxpool in the serial structure is 9x9, which is equivalent to a parallel 9x9 maxpool. While reducing the number of parameters, it maintains the ability of SPP to aggregate multi-scale features, such as Figure 2 As shown;
[0158] Its innovations compared with previous work:
[0159] 1. NMS-free training
[0160] A consistent dual assignment strategy is proposed to achieve NMS-free training while maintaining competitive performance and low inference latency. This step essentially saves the part of inference time that screens out duplicate prediction boxes. Specifically, during training, YOLO usually utilizes TAL to assign multiple positive samples to each instance. The one-to-many assignment method produces rich supervisory signals, promotes optimization and enables the model to achieve excellent performance. However, this requires YOLO to rely on NMS post-processing, which leads to suboptimal inference efficiency when deployed. Unlike one-to-many assignment, one-to-one matching assigns only one prediction to each ground truth, avoiding NMS post-processing. However, this results in weak supervision, so that the accuracy and convergence speed are not ideal.
[0161] The dual assignment strategy combines the advantages of the above two strategies. The study introduced another one-to-one head for YOLO. It retains the same structure as the original one-to-many branch and adopts the same optimization objective, but uses one-to-one matching to obtain label assignments. During training, the two heads are jointly optimized to provide rich supervision; during inference, YOLOv10 discards the one-to-many head and uses the one-to-one head to make predictions. This enables YOLO to be deployed end-to-end without incurring any additional inference costs.
[0162] As a specific implementation method,
[0163] 2. Model design strategy
[0164] A comprehensive efficiency-accuracy driven model design strategy is introduced, which comprehensively optimizes various components of YOLOs, significantly reduces computational overhead and enhances the model's capabilities from the perspectives of efficiency and accuracy. Each design component, including lightweight classification head, SCDown, and sorting-guided module design, helps reduce the number of parameters, FLOPs, and latency. Importantly, these improvements are achieved while maintaining excellent performance. Figure 3 As shown:
[0165] As a specific implementation method,
[0166] 3. Multi-source domain adaptation:
[0167] Multi-source domain adaptation is a machine learning method that uses data from multiple source domains to improve the performance of a task (such as classification or regression) in the target domain. This method is particularly suitable for situations where there are distribution differences between the source and target domains, but these source domains share the same type of task objectives. The key to multi-source domain adaptation is to mine and utilize shared information between different source domains and the target domain while dealing with differences between domains. A major challenge of multi-source domain adaptation is how to effectively integrate information from multiple source domains while taking into account the heterogeneity between domains. A significant advantage of this method is that it can use multiple related source domains to improve the performance of the target domain, which is usually more effective than single source domain adaptation or traditional machine learning methods. For this reason, the MIDOG2022 dataset is chosen as the source domain here, because it also records mitotic cells, but the corresponding diseases are different, which is a typical case of similar types but different distributions.
[0168] MIDOG2022 dataset: A dataset provided by the MItosis DOmain Generalization Challenge 2022. It comes from images of 6 different tumor types, 5 of which are labeled. The set contains a total of 405 cases and includes 9501 mitotic figure annotations in the training set. The task is to detect mitotic figures from histopathological images, and the performance of the model will be evaluated on 10 different tumor types, with F1 score as the main indicator.
[0169] Method: First, a teacher model is trained on the source domain, then the teacher model is used to generate pseudo labels on the target domain, and finally the pseudo labels are used to train the final student model on the target domain.
[0170] 1. Source domain training
[0171] Multiple source domains correspond to multiple target detection heads one by one and share the same feature encoder. During back propagation, sample features that do not correspond to the source domain of the target detection head first pass through a layer of gradient reversal module before being input into the target detection head to generate domain adversarial features. That is, the target detection head of a specific source domain will encode the features of the source domain, and at the same time confuse other target detection heads through gradient reversal. Each target detection head is separated from other source domains to encode the knowledge of a specific source domain. In this way, mutual interference between source domains can be reduced, which is conducive to reducing knowledge degradation.
[0172] Figure 4 There are three different ways of multi-source domain adaptation: (a) Simple combination: Due to the domain shift between sources, the simply combined sources may interfere with each other. (b) Alignment: Multi-source alignment reduces domain shift, but it reduces target-related knowledge when alignment is performed without target domain guidance. (c) Decoupling: Decouple multiple source domains and prevent their mutual interference to retain target-related knowledge in a specific domain. Naively, the target detection heads of different source domains can use the average method to obtain the final result. However, the correlation between different source domains and the target domain is different, and the source domain with higher correlation should have a higher weight. Specifically, the similarity between two domains can be considered from two granularities: "slice level" and "target level". The mathematical formulas for measuring the distance between source and target domain data include KL Divergence, MMD, H-divergence, and Wassersteindistance.
[0173] 2. Target Domain Training
[0174] In order to alleviate the overfitting of the pseudo labels of the teacher model on the target domain, EMA is used to average the weights of the student model on the target domain to the teacher model on the source domain.
[0175] Training settings:
[0176] First, set a common range for hyperparameters such as learning rate and batch size, and then use the Optuna tool (a python library for hyperparameter optimization) to search for hyperparameters within the range. Then, fine-tune the obtained hyperparameters and finally determine: use Adam optimizer for optimization, with an initial learning rate of 0.01, a final learning rate cosine drop to 0.001, a batch size of 128, and a total of 200 epochs for training.
[0177] Evaluation method:
[0178] The effect of multi-source domain adaptation can be evaluated from the following perspectives:
[0179] 1. The degree of alignment between features of different domains for domain-adaptive target detectors. Can multiple source domains better adapt to target domain features?
[0180] 2. Final object detection performance. Is there a significant performance improvement compared to the baseline method (such as the method without domain adaptation)?
[0181] Experimental results:
[0182] Feature domain migration, use t-sne to reduce the dimension of features and visualize them, such as Figure 4-5 As shown:
[0183] Among them, GIST is a mesenchymal tumor, and the other five categories are the five categories of MIDOG. Compared with the baseline method without domain adaptation, the feature encoder of the domain-adapted target detector enables the source domain features to be better adapted to the target domain.
[0184] 2) Target detection results
[0185] Precision Recall F1 Baseline 0.811 0.239 0.369 MDA 0.683 0.489 0.570
[0186] It can be found that compared with the baseline method without domain adaptation, the target detection performance of the domain-adapted target detector is significantly better, especially in terms of recall rate, which can significantly reduce the false negative phenomenon in clinical diagnosis.
[0187] 4. Re-calibration of the model
[0188] In order to improve the performance of the model, we continuously optimize the model and establish a functional platform for communicating with doctors. The basic process is: after the pre-trained model predicts the test set, it is placed on a dedicated website to display the prediction results. After the doctor logs in to the platform, he corrects the prediction results (i.e., annotations) (the correct ones are retained, and the wrong ones are deleted). After that, the developers download the corrected data from the website for model training, and train it in a 50-fold crossover (the data is randomly divided into 5 parts. Each time, one part is used as a test machine and the others are used as training sets). Each training hyperparameter is adjusted according to the linear proportion of the above basic hyperparameters based on the scale of the data to be trained. The training process stops early when the accuracy of the test set no longer increases.
[0189] For the functional platform, the main functions are as follows:
[0190] 3.1 File Upload
[0191] like Figure 7 As shown in the figure, this step receives the image, patient name, image type and storage location information through the HTTP interface. The system performs multiple processing steps: verifying the integrity and compliance of the uploaded file; verifying patient information to ensure accurate data association; automatically identifying and processing file formats; optimizing storage strategies to avoid duplication and record file metadata in detail; parsing tag information in JSON to deepen data value; implementing full encryption to strengthen data security and privacy protection. This solution integrates upload, verification, processing, storage and security management functions, improving the intelligence and security of medical image data processing.
[0192] 3.2 Annotation Lifecycle
[0193] like Figure 8 As shown in the figure, this step demonstrates a medical image label update technology that receives label update instructions through API to optimize the review process. The main functions include: receiving API requests, verifying the existence of image files, ensuring data consistency through anti-review protection mechanisms, updating image review status, and managing the review and truth information of each label.
[0194] 3.3 File Download
[0195] like Fig. 9 As shown in the figure, medical image download can provide image files and JSON download services with labels on demand. The process includes: receiving download requests and extracting images, encoding image data; collecting and filtering correct label information to form a list of LabelShape objects; building an ImageJson object containing image and label metadata; finally, converting this object into a downloadable JSON file attachment and sending it through HTTP response.
[0196] As a specific embodiment, Fig.10 As shown,
[0197] 1. Detection model:
[0198] After making predictions on existing data, developers obtain the prediction results Pred_data and send them to the functional platform for storage.
[0199] When the corrected data set Recised_data is returned, a new round of training is automatically started to update the self-weight parameters.
[0200] 2. Functional platform:
[0201] The model's prediction results are obtained, displayed, and the logged-in doctor completes the inspection and correction of the annotations.
[0202] Accept the new round of patient data new_data uploaded by hospital doctors and send it back to the model for prediction.
[0203] The predicted results are analyzed and the resulting information is delivered to the doctors in need to adjust the treatment plan.
[0204] 3. Hospital:
[0205] The doctor logs into the functional platform to verify the model's prediction results.
[0206] Upload new data for model prediction and get the results analysis given by the platform.
[0207] The above specific implementations are only descriptions of the preferred implementations of the present invention, and do not limit the protection scope of the present invention. Without departing from the design concept and spirit of the present invention, various modifications, substitutions and improvements made by ordinary technicians in this field to the technical solution of the present invention based on the text description and drawings provided by the present invention should all fall within the protection scope of the present invention. The protection scope of the present invention is determined by the claims.
Claims
1. A method for identifying and counting mitotic cells in mesenchymal tumors based on deep learning, characterized in that: The following steps are involved: Step 1: obtaining and preprocessing the stained gastrointestinal stromal tumor tissue slice image, the preprocessing includes color normalization, denoising and contrast enhancement operations on the slice image, and eliminating the artificial differences in the staining process; the preprocessing step also includes converting the image into a grayscale image to simplify the input form of the model; Step 2: Segment the preprocessed slice image to generate image blocks suitable for deep learning model processing. The segmentation operation uses a step size of 256 pixels to divide the 512×512 pixel slice into multiple sub-image blocks, and reduce the impact of edge effects on model recognition accuracy; for image blocks that do not contain target cells or contain only a few cells, they are eliminated through RGB pixel value threshold screening; Step 3: Use data augmentation techniques to expand the generated image blocks, including but not limited to color adjustment, random rotation, translation, scaling, cropping, affine transformation, and randomly copying the target cells to other image locations through copy-paste technology; Step 4: construct a target detection model based on YOLOv10 to identify and locate mitotic cells in the image; the YOLOv10 model includes innovative modules, including a C2fUIB module, a partial self-attention module, a spatial channel decoupling downsampling module, and an improved spatial pyramid pooling fast interface module; Step 5: Apply multi-source domain adaptation technology to enhance the generalization ability of the model by introducing data from multiple source domains; the multi-source domain adaptation technology includes using multiple target detection heads in the source domain training process, each detection head corresponds to specific source domain data, and using a gradient reversal module to generate domain adversarial features, thereby reducing mutual interference between source domains; then, train the student model on the target domain, and generate pseudo labels through the teacher model to further improve the detection accuracy in the target domain; Step 6: Count the identified mitotic cells and generate corresponding statistical data; the statistical data include the total number of mitotic cells in each slice, cell density distribution, confidence ranking results, and the distribution pattern of mitotic cells in different pathological slices; the statistical results are used to assist doctors in disease assessment and formulation of treatment plans; Step 7: Display the model's prediction results on the functional platform, and let the doctor verify and correct the annotation results; the functional platform includes an intuitive user interface that allows doctors to view the specific location and annotation results of each cell identified by the model. Doctors can modify the annotation by clicking and dragging; Step 8: Retrain the model using the doctor's corrected data to continuously optimize the model's recognition and counting performance; the retraining step includes dividing the corrected data set into multiple cross-validation sets, adjusting the model hyperparameters and introducing new data enhancement strategies to further improve the stability and accuracy of the model in practical applications.
2. The method for identifying and counting mitotic cells of mesenchymal tumors based on deep learning according to claim 1, characterized in that: The data augmentation techniques applied in step 3 include the following operations: Step 3.1: Color adjustment, including adjusting the hue, saturation, sharpness and contrast of the image to simulate the image changes under different staining conditions to ensure that the model can effectively identify target cells under different staining effects; Step 3.2: Affine transformation, including random rotation, translation, scaling, and cropping operations to simulate the possible changes in slicing and shooting angles in actual operations; Step 3.3: copy-paste operation, randomly copying the identified target cell area to other locations in the image, thereby increasing the diversity of target cells in the training data and enhancing the robustness of the model; Step 3.4: Data denoising and background removal, remove pure white areas or areas with almost no cells, and reduce the interference of irrelevant information on the model.
3. The method for identifying and counting mitotic cells of mesenchymal tumors based on deep learning according to claim 1, characterized in that: The innovative modules of the YOLOv10 model used in step 4 include: Step 4.1: C2fUIB module, by adopting reparameterization technology, increases the receptive field of the convolution kernel and reduces the computational consumption, thereby improving the detection accuracy and inference speed of the model; Step 4.2: Partial self-attention module, by dividing the feature map into two parts, one part is processed by the multi-head self-attention module, and the other part keeps the original features, and then the two parts of the features are fused to enhance the model's detection ability for small targets; Step 4.3: The spatial channel decoupling downsampling module reduces the computational cost and number of parameters by decoupling the spatial reduction and channel increase operations, while retaining information to the maximum extent and enhancing the recognition ability of small targets during the downsampling process; Step 4.4: Improved spatial pyramid pooling fast interface, by changing the parallel structure to a serial structure, reduces the number of parameters and maintains the multi-scale feature aggregation capability, thereby improving the model's ability to detect objects of different scales.
4. The method for identifying and counting mitotic cells of mesenchymal tumors based on deep learning according to claim 1, characterized in that: The multi-source domain adaptation techniques in step 5 include: Step 5.1: In the source domain training process, multiple source domains correspond to multiple target detection heads one by one, and the feature encoders are shared. During back propagation, domain adversarial features are generated through the gradient reversal module, thereby reducing the mutual interference between source domains and optimizing the adaptation performance of the target domain. Step 5.2: Target domain training process: Use the pseudo labels generated by the teacher model to train the student model on the target domain, and gradually update the weights of the student model to enhance its detection accuracy in the target domain; Step 5.3: Domain adaptation effect evaluation: t-SNE dimensionality reduction technology is used to visualize the features of the source domain and the target domain, verify the adaptability of the model to the feature distribution between different domains, and evaluate the actual effect of the multi-source domain adaptation technology by comparing with the baseline model.
5. The method for identifying and counting mitotic cells of mesenchymal tumors based on deep learning according to claim 1, characterized in that: The statistics in step 6 further include: Step 6.1: A detailed position distribution map of mitotic cells in each pathological section, wherein the position of the detected mitotic cells in the original section image is marked to form a two-dimensional coordinate system to display the precise coordinate position of each cell; such a distribution map can intuitively display the spatial distribution of mitotic cells in tumor tissues, helping doctors better understand the proliferation pattern and regional activity of tumors; Step 6.2: Generate a heat map based on cell density. The heat map calculates the density of mitotic cells in the slice and displays the cell density in different areas in the form of color gradients. The depth of the color corresponds to the density of the cells. The heat map can help doctors quickly identify the most active proliferation areas in the tumor, providing an important basis for pathological diagnosis and formulation of treatment plans; Step 6.3: A list of mitotic cells sorted by confidence, wherein the list of mitotic cells is sorted based on the confidence value of each cell detected by the model; the confidence value indicates the degree of certainty of the model that the cell is a mitotic cell, and the doctor will first verify and correct the recognition results with low confidence; Step 6.4: Count the total number and average density of mitotic cells in different pathological sections. The statistical data include the total number of cells in each section, the cell density per square millimeter, and the local density calculated according to different tumor areas; Step 6.5: Generate a summary report containing multiple patient slices, which integrates all test results, including the total cell number, average density, active area distribution, and confidence analysis of each patient, to form a complete test report to help doctors make an overall evaluation and comparison.
6. The method for identifying and counting mitotic cells of mesenchymal tumors based on deep learning according to claim 1, characterized in that: The retraining process in step 8 further includes: Step 7.1: Collection and organization of revised data. The revised data is generated by doctors after they revise the model prediction results on the functional platform. The data includes revised cell annotation, confidence adjustment, and removal of error detection information. The organization process includes labeling and error analysis of the revised data, and merging the revised data into a new training set. Step 7.2: Divide and prepare the training set. Divide the collected corrected data into multiple cross-validation sets in proportion to ensure the generalization ability of the model on different data sets. The preparation of the training set also includes secondary enhancement of the data to ensure the diversity and randomness of the data. Step 7.3: Optimize and adjust the model hyperparameters. Before each retraining, optimize and adjust the model hyperparameters according to the scale and characteristics of the correction data. Use the hyperparameter tuning tool to automatically search within the set range, and determine the best hyperparameter combination through multiple rounds of experiments. Step 7.4: Execution of the retraining process, including inputting the corrected training data into the model for retraining, monitoring the accuracy and loss value of the model through the validation set, and adopting an early stopping strategy, that is, stopping training when the accuracy of the validation set no longer improves to avoid overfitting; after the training is completed, the final weights of the model will be saved and used for the next detection task; Step 7.5: Model evaluation after retraining, including a comprehensive analysis of the model's performance on the test set and the model's performance improvement on the revised data; evaluate the effect of retraining by comparing it with the initial model and other baseline models, and feedback the evaluation results to the development and application teams to decide whether to further optimize; Step 7.6: Model deployment and update: deploy the retrained and evaluated model to the functional platform and update it to the system used by doctors in real time to ensure that the new model can automatically replace the old model. At the same time, record the usage feedback of the new model for reference for the next retraining.
7. The method for identifying and counting mitotic cells of mesenchymal tumors based on deep learning according to claim 1, characterized in that: The recognition accuracy and robustness of the model are further optimized through multi-scale feature aggregation and model enhancement training, the steps of which include: Step 8.1: Extract multi-scale features. Input the preprocessed image blocks into the deep learning model and extract feature maps of different scales through specific network layers to ensure that the model can capture multiple morphological features of cells. Step 8.2: Perform feature fusion, fuse the extracted features of different scales, and use the spatial pyramid pooling module or its improved version to aggregate multi-scale information to enhance the model's ability to recognize small targets; Step 8.3: Introduce model enhancement training technology, through the hybrid loss function and data enhancement strategy, enhance the model's ability to resist complex samples and noise, and improve recognition accuracy; Step 8.4: Use transfer learning technology to take the pre-trained model trained in the related field as the basis, and improve the adaptability of the model to the target task by fine-tuning it in the target field.
8. The method for identifying and counting mitotic cells of mesenchymal tumors based on deep learning according to claim 1, characterized in that: Also included is an automated error detection and correction mechanism, the steps of which include: Step 9.1: Automatic verification of model prediction results. The system performs preliminary verification on the mitotic cells detected by the model and identifies annotations with low confidence or possible errors. Step 9.2: Generate an error detection report. The system will generate an error report for the low-confidence or suspected erroneous annotations detected, listing in detail the cell locations that may have problems and the corresponding confidence values; Step 9.3: Manual correction and feedback: The doctor manually corrects the annotations in the error report, confirms or corrects the detection results of the model, and feeds the corrected data back to the system; Step 9.4: Dynamically adjust the model weights. The system automatically adjusts the model weights and parameters based on the doctor's correction results to improve the accuracy of the model in subsequent tasks.
Citation Information
Patent Citations
Cell mitosis detection method of breast cancer scoring system
CN107103601A
Method and Apparatus for Diagnosing Malignant Tumor
KR101889722B1
System and method for automatically identifying mitosis in h&e stained breast cancer pathological images
US20230401707A1
Weakly-Supervised System, Method and Workflow for Processing Whole Slide Image for Disease Detection
US20240193765A1
Cited By
Cell image recognition method and system based on model fine tuning
CN119478940A
Cell image recognition method and system based on model fine-tuning
CN119478940B
Deep learning-based woody plant vascular tissue cell attribute statistical method
CN120544192A
Dynamic model optimization method and system based on attribute driving
CN120976684A
An attribute-driven dynamic model optimization method and system
CN120976684B
