A medical image assisted diagnosis system based on deep learning

Abstract

This invention discloses a deep learning-based medical image-assisted diagnostic system, specifically relating to the field of medical imaging artificial intelligence. It includes an image preprocessing and feature decoupling module, a multi-task collaborative diagnosis and interpretability generation module, a cross-modal temporal tracking module, and a human-machine collaborative decision optimization module. By decoupling the network, it separates device-independent pathological semantic features. The multi-task network synchronously completes lesion localization, classification, and grading, while dynamically generating interpretable reports that integrate visual highlights and textual evidence. For follow-up patients, the system models the temporal evolution of lesion features, outputs a quantitative evolution map, receives doctor feedback through an interactive interface, and employs an anti-forgetting incremental learning strategy to optimize the model in a targeted manner. This invention improves diagnostic accuracy, interpretability, and the ability to dynamically track individual disease progression, and can continuously self-optimize through human-machine collaboration.

Description

Technical Field

[0001] This invention relates to the field of medical imaging artificial intelligence technology, and more specifically, to a medical imaging-assisted diagnostic system based on deep learning. Background Technology

[0002] With the rapid development of artificial intelligence technology, deep learning-based medical image-assisted diagnostic systems have become a research hotspot in the field of medical image analysis. These systems can automatically analyze medical images such as CT, MRI, and X-rays to assist doctors in lesion detection, localization, and qualitative analysis. They have shown great potential in improving diagnostic efficiency and reducing doctors' workload, and are an important component of smart healthcare.

[0003] Currently, typical existing technical solutions employ end-to-end deep learning models, such as those based on convolutional neural networks (CNN) or Transformer architectures, to directly extract and classify features from raw medical images. These typically include a unified feature encoder, followed by multiple parallel task heads, each used to complete tasks such as lesion localization and nature classification. The implementation process is as follows: first, the images undergo simple normalization preprocessing, then they are input into a pre-trained backbone network to extract general feature maps, and finally, the task heads output their respective diagnostic results.

[0004] However, in practical use, it still has some shortcomings. For example, the extracted features are mixed with real pathological information and non-pathological interference such as imaging equipment and scanning parameters, resulting in poor model generalization ability and large performance fluctuations in different hospitals or equipment. Secondly, the diagnostic process lacks intuitive and interpretable evidence, making it difficult for doctors to understand and trust its output results. The system usually only performs static analysis on a single examination and cannot effectively utilize the patient's historical imaging data for longitudinal comparison and disease progression tracking. Finally, the system lacks an effective continuous learning mechanism and cannot safely and purposefully optimize the model based on feedback from doctors in clinical practice. It is easy to forget old diagnostic capabilities when learning new knowledge, making it difficult to iterate and upgrade in actual workflows. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a deep learning-based medical image-assisted diagnostic system, which addresses the problems raised in the background art through the following solutions.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a medical image-assisted diagnosis system based on deep learning, comprising an image preprocessing and feature decoupling module: receiving and standardizing raw medical images, and having a built-in feature decoupling network jointly trained based on mutual information minimization constraints, separating the standardized images into pathological semantic feature vectors and imaging domain feature vectors with low statistical correlation; Multi-task collaborative diagnosis and interpretability generation module: The input end connects to the aforementioned module to obtain semantic feature vectors. It has a built-in shared encoder and parallel sub-networks for lesion localization, nature classification and severity grading. It integrates an interpretability generation unit and dynamically generates a visual diagnostic report that integrates pixel-level highlighted areas and structured text decision basis based on the encoder activation map and sub-network decision weights. Cross-modal temporal tracking module: When the same patient has multiple historical images, it calls the semantic feature vectors of the current and the past, establishes the dynamic evolution relationship between feature sequences through the built-in temporal relationship network and adopts an adaptive attention mechanism, and outputs a temporal evolution map and indicators that quantitatively describe the trend of lesion changes. Human-machine collaborative decision optimization module: Receives the visualized diagnostic report and evolution map, obtains feedback from doctors on the correction or confirmation of the system conclusions through the interactive interface, integrates incremental learning units, and applies elastic weight consolidation algorithm constraints based on the feedback data to perform targeted fine-tuning of specific parameters in the feature decoupling network and multi-task diagnostic module.

[0007] The technical effects and advantages of this invention are as follows: This invention introduces a feature decoupling network with mutual information minimization constraints to actively separate and statistically independent the pathological semantic features and imaging domain features in medical images. This significantly improves the model's generalization ability and robustness to image data across devices and institutions from the source, effectively overcoming the core pain point of unstable diagnostic performance caused by device differences in existing solutions. This invention deeply integrates multi-task collaborative diagnosis with interpretability generation, simultaneously completing lesion localization, classification, and grading, while dynamically generating reports with associated pixel-level highlighted areas and structured text logic. This improves the overall efficiency of diagnosis and provides doctors with intuitive and reliable decision-making basis, greatly enhancing the acceptability and practical value of artificial intelligence systems in clinical practice. This invention constructs a closed-loop system that integrates time-series dynamic tracking and human-machine collaborative optimization. By modeling the time-series evolution of patients' historical data, the system can provide quantitative analysis of disease progression. By receiving feedback from doctors and adopting an incremental learning strategy to prevent forgetting, it achieves safe and continuous self-iteration and optimization in real clinical environments, thus possessing the evolutionary capability to serve for a long time and adapt to complex medical scenarios. Attached Figure Description

[0008] Figure 1 This is a schematic diagram of the overall structure of the present invention; Figure 2 This is a schematic diagram illustrating the detailed process of feature decoupling in this invention; Figure 3 This is a schematic diagram illustrating the details of the multi-task diagnostics method of the present invention; Figure 4This is a schematic diagram illustrating the detailed timing analysis process of the present invention; Figure 5 This is a schematic diagram illustrating the human-machine collaboration details of the present invention. Detailed Implementation

[0009] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0010] As attached Figure 1 Appendix Figure 2 The system shown is a deep learning-based medical image-assisted diagnosis system, which includes an image preprocessing and feature decoupling module: receiving and standardizing raw medical images, and building a feature decoupling network jointly trained based on mutual information minimization constraints to separate the standardized images into pathological semantic feature vectors and imaging domain feature vectors with low statistical correlation.

[0011] It should be further explained that the data selection is as follows: Multimodal raw medical imaging data: covering CT (low dose, conventional dose), MRI (T1-weighted, T2-weighted, DWI sequence), and ultrasound images. The data comes from the clinical data of 3 tertiary hospitals (2 general hospitals and 1 specialized oncology hospital) over the past 5 years, and includes a total of 12,000 cases (4,000 cases for each type of imaging). These include 8 common diseases and 20 subtypes (specifically divided into 3 types of lung cancer, 2 types of liver cancer, 2 types of cerebral hemorrhage, 2 types of breast nodules, 3 types of cerebral infarction, 3 types of pneumonia, 2 types of cirrhosis, and 3 types of thyroid nodules).

[0012] Imaging equipment metadata: synchronously acquires the corresponding image equipment model, scanning parameters (including tube voltage, tube current, and scan slice thickness for CT, TR / TE value and flip angle for MRI, and probe frequency and gain for ultrasound), equipment calibration records, scan time, and metadata labeled with the examination site.

[0013] The gold standard pathological data is annotated by three physicians with the title of associate chief physician or above. It includes lesion location (including pixel-level bounding box coordinates), pathological type, pathological grade, lesion morphological characteristics, and lesion size (major axis / minor axis), which serve as the supervision basis for pathological semantic features.

[0014] Data processing employs a "layered preprocessing + cross-modal alignment" approach, with the following specific steps: First, pixel value calibration is performed. Based on calibration records in the device metadata, pixel values ​​are normalized for images acquired by different devices and with different parameters to eliminate inherent device errors. Then, an improved Z-score normalization algorithm is used, incorporating local region statistical information. Local mean and standard deviation are calculated in 3×3 pixel blocks to standardize the image by region, addressing the edge feature distortion problem caused by traditional global normalization. Specifically, a 3×3 pixel window is slid along rows / columns, covering the entire image block by block to ensure no edge areas are missed. If an edge block is less than 3×3, mirror filling is used to complete it. For each 3×3 pixel block, the grayscale mean of all pixels within the block is calculated. and standard deviation For each pixel within the current pixel block, substitute it into the formula. Complete standardization, When the value is 0, add a minimum value to avoid the denominator being 0. After traversal, stitch together all the standardized pixel blocks to obtain the entire region-standardized image.

[0015] An adaptive noise removal network based on U-Net is used to specifically remove inherent noise from images of different modalities. Pre-training is performed using noise type labels to automatically identify image modalities and match denoising strategies. Specifically, the process involves: collecting multimodal images and corresponding noise type labels to construct a training dataset; using U-Net as the basic architecture, inputting noisy images and outputting denoised images, with the noise type labels supervising the network to learn the mapping relationship between "modality-noise type-denoising parameters"; inputting the image to be processed into the pre-trained network, which automatically identifies the image modality and corresponding inherent noise type through the network's front-end feature extraction branch; and based on the identification results, the network calls the matched denoising strategy to perform targeted denoising on the image and output the denoised image.

[0016] To address the issue of limited data on rare diseases, a generative data augmentation method based on pathological semantics is adopted. Using the StyleGAN3 model, known pathological features of rare diseases are injected into normal images to generate high-quality synthetic images, ensuring that the data volume of each disease category accounts for no less than 5%. At the same time, geometric augmentation techniques, including random flipping, rotation, and scaling, are used to improve the model's generalization ability. During the augmentation process, constraints are imposed based on the anatomical structure markers of the images to avoid compromising the anatomical rationality of the images.

[0017] Based on the anatomical key point detection network, common anatomical key points are extracted from different modal images. The thin plate spline interpolation (TPS) algorithm is used to achieve spatial alignment of multimodal images, ensuring that the features of the same pathological region in different modal images can be effectively associated during the subsequent feature decoupling process.

[0018] The architecture of "mutual information minimization constraint + dual-branch joint training" is adopted, and the specific implementation is as follows: VisionTransformer (ViT) is used as the basic encoder, and a pathological semantic feature branch and an imaging domain feature branch are built after the encoder. The semantic feature branch uses a convolutional attention module (CBAM) to enhance the feature response of the pathological region, and the domain feature branch introduces a device parameter perception layer (encodes the device metadata into a vector and then fuses it with the image features).

[0019] An improved mutual information computation unit is introduced, and the pathological semantic feature vector output by the feature decoupling network is extracted through KL divergence regularization constraints. and imaging domain feature vector Solving the problem using dual-scale computational units. and The global mutual information (overall feature distribution correlation) and local mutual information (corresponding pathological region feature correlation) are calculated, and the average of the two is taken as the final mutual information value MI. With "MI < 0.1" as the target, a KL divergence regularization term is constructed to quantify the distribution difference between the current MI and the preset threshold (0.1). The KL divergence regularization term is incorporated into the total loss function, and the feature decoupling network parameters are updated through backpropagation. Iterative optimization is performed until MI is stably lower than 0.1, so as to achieve statistical independence of the two feature vectors.

[0020] Two branch losses are defined: pathological semantic classification loss: based on the gold standard annotation of pathology, the difference between the semantic feature branch output and the true label is calculated by cross-entropy loss; imaging domain classification loss: based on the device model and scanning parameter labels, the classification error of the domain feature branch output is calculated by cross-entropy loss. The two are weighted and summed to obtain the joint loss function (the initial weight ratio is set to semantic loss: domain loss = 7:3, to prioritize semantic feature learning).

[0021] Set independent learning rates for the semantic feature branch and the domain feature branch respectively (initial semantic branch learning rate = 1e-3, domain branch learning rate = 5e-4) to ensure that the semantic branch learning has a higher priority in the early stage of training.

[0022] In the early training phase (first 30% of iterations): the weight ratio is fixed, and the network parameters are updated through backpropagation, with a focus on optimizing the semantic feature extraction capability; in the later training phase (last 70% of iterations): an adaptive adjustment mechanism is activated, gradually adjusting the weight ratio to 5:5, while simultaneously increasing the learning rate of the domain branch to be consistent with that of the semantic branch (1e-3), and strengthening the mutual information minimization constraint.

[0023] Training terminates when the joint loss function value stabilizes (fluctuation < 1e-5 for 10 consecutive iterations). The semantic feature branch outputs a 1024-dimensional pathological semantic feature vector, and the domain feature branch outputs a 512-dimensional imaging domain feature vector.

[0024] As attached Figure 3 As shown, the multi-task collaborative diagnosis and interpretability generation module: the input end is connected to the aforementioned module to obtain semantic feature vectors, and has a built-in shared encoder and parallel sub-networks for lesion localization, nature classification and severity grading. It integrates an interpretability generation unit and dynamically generates a visual diagnostic report that integrates pixel-level highlighted areas and structured text decision basis based on the encoder activation map and sub-network decision weights.

[0025] It should be further explained that the data selection is as follows: Pathological semantic feature vector data: 1024-dimensional vectors output by the preceding module, corresponding to 12,000 labeled cases.

[0026] Multi-task labeled data: Based on the original pathological gold standard labeling, supplemented lesion location bounding boxes (pixel-level annotation), lesion nature classification labels (benign / malignant / borderline), and severity grading labels (such as TNM staging for lung cancer and Child-Pugh classification for cirrhosis) are added, and clinical diagnostic reports (structured text) of corresponding cases are collected at the same time.

[0027] Physician Diagnostic Thought Log Data: Five senior radiologists were invited to record their thought processes during the image review and diagnosis process (e.g., "A nodule was found in the upper lobe of the right lung with spiculated edges, suggesting possible malignancy"), and diagnostic thought logs for 800 cases were collected.

[0028] The pathological semantic feature vectors output by the preceding module are adaptively adjusted in dimension. Principal component analysis (PCA) combined with prior pathological knowledge is used to select feature dimensions that are strongly related to the current diagnostic task, thereby reducing the interference of redundant features on the diagnostic results. At the same time, a feature normalization layer is introduced to map the feature vectors to a unified numerical range [0,1], ensuring the consistency of the input features of the multi-task sub-network.

[0029] The method of "dual physician cross-validation + AI-assisted validation" is adopted to clean the multi-task labeled data and remove inconsistent data. A one-to-one correspondence between labeled data and pathological semantic feature vectors is established. By using the spatial location information of feature vectors, the spatial alignment of the lesion localization bounding box with the feature vector is achieved, ensuring that the localization sub-network can accurately associate features with lesion areas.

[0030] The diagnostic thinking log is segmented into words, tagged with parts of speech, and information is extracted to construct a diagnostic logic knowledge base. The log text is associated with the corresponding pathological semantic feature vectors and labeled data to form a "feature-label-thinking" triple data, which provides training data for the interpretable generation unit.

[0031] A Transformer-based shared encoder is designed to perform deep encoding on the optimized pathological semantic feature vectors, extract common features across multiple tasks, and build three sub-networks in parallel: The lesion localization sub-network is based on the YOLOv8 architecture, embedding a feature pyramid fusion module (FPN+PAN), while reserving an input interface for pathological semantic features to construct an improved YOLOv8 localization sub-network. This sub-network, combined with the spatial information of pathological semantic features, improves the localization accuracy of small lesions (diameter <5mm); the nature classification sub-network uses a fully connected network based on an attention mechanism, introducing a multi-scale feature fusion layer to capture differences in pathological features at different levels; and the severity grading sub-network uses an LSTM+attention structure, combining multi-dimensional features of the lesion (size, shape, and extent of invasion) for grading, with the specific criteria as follows: Grade I (mild / early stage): The maximum diameter of the lesion is <5mm; the shape is regular (smooth edges, no lobulation / spiculation); there is no infiltration or it is limited to the superficial layer of the primary tissue, and it does not invade key structures such as surrounding blood vessels and nerves; Grade II (Moderate / Progressive): Lesions with a maximum diameter of 5-20 mm; slightly irregular shape (slightly lobulated edges, no obvious spiculation); local infiltration into the deep layers of the primary tissue, without breaking through the tissue capsule, and no signs of distant metastasis; Grade III (Severe / Late Stage): Lesions with a maximum diameter of 20-50 mm; irregular shape (obvious lobulation, spiculation, or accompanied by calcification / necrosis); break through the tissue capsule and invade 1-2 adjacent organs / structures; Grade IV (Very Severe / Terminal): Lesion diameter > 50 mm; extremely irregular morphology (extensive spiculation, confluent growth); multi-organ infiltration or distant metastasis, or accompanied by serious complications.

[0032] A dual-path fusion architecture of "pixel-level highlight generation + structured text generation" is adopted: Based on the activation map of the shared encoder, combined with the decision weights of each sub-network, an improved algorithm of gradient-weighted class activation mapping (Grad-CAM) is used to input the image to be processed into the diagnostic model to obtain convolutional feature maps and target output layer gradients. After gradient-weighted feature fusion with pathological prior constraints, interpolation upsampling, and threshold denoising, it is superimposed with the original image to generate a pixel-level highlight heatmap of the lesion. The introduction of pathological prior knowledge constraints ensures that the highlighted area accurately covers the core area of ​​the lesion and avoids false highlighting of the background area. A fine-tuning architecture based on a pre-trained model is adopted, using "feature-annotation" Using the "Thinking" triplet data as the training set, a diagnostic logic attention mechanism is introduced to generate key information in the text-related feature vector. At the same time, a structured template (including "lesion location, morphological features, diagnostic basis, and conclusion") is adopted to ensure the readability and standardization of the text. Finally, through the fusion module, the pixel-level lesion highlight heatmap is associated and bound with the structured text diagnostic report containing lesion location, morphological features, and diagnostic basis, realizing the linkage and interaction between the highlighted area of ​​the heatmap and the key information in the text. In the end, an integrated visual diagnostic report is generated, which supports the linkage viewing of the highlighted area and the text. Clicking on the lesion location in the text can locate the corresponding area in the heatmap.

[0033] A "staged joint training" strategy is adopted. In the first stage, the basic parameters of the shared encoder and each sub-network are trained, with the independent loss function of each sub-network (CIoU loss for localization loss, cross-entropy loss for classification loss, and Focal loss for hierarchical loss) as the optimization objective. In the second stage, a multi-task collaborative loss function is introduced to calculate the correlation of the output results of each sub-network and constrain the collaboration. At the same time, the loss of the interpretability generation unit is added (BLEU value for text generation loss and IoU loss for highlight region matching loss) to achieve joint optimization of multi-task and interpretability generation.

[0034] As attached Figure 4 As shown, the cross-modal temporal tracking module: when the same patient has multiple historical images, it calls the semantic feature vectors of the current and the past, establishes the dynamic evolution relationship between feature sequences through the built-in temporal relationship network and adopts an adaptive attention mechanism, and outputs a temporal evolution map and indicators that quantitatively describe the trend of lesion changes.

[0035] It should be further explained that the data selection is as follows: temporal pathological semantic feature vector data of the same patient: patient cases with 2-5 follow-up images were selected, and a total of 3,000 cases were included. Each case includes pathological semantic feature vectors at different time points (interval of 1-12 months), and the examination time and clinical treatment plan at each time point are recorded.

[0036] Pathological annotation data corresponding to time-series images: Collect lesion size (long diameter, short diameter, volume), morphology (margin smoothness, lobulation sign, spiculation sign), number (single lesion / multiple lesions and specific number), location (anatomical division, relationship with surrounding tissues), and infiltration range annotation data of images at each time point, as a monitoring basis for the quantification of time-series changes.

[0037] Clinical efficacy evaluation data includes physicians' evaluation of patients' treatment effects (effective, stable, or progressive), patients' quality of life scores, lesion remission rate, progression-free survival, and the type and severity of adverse reactions.

[0038] Based on patient ID and examination time, time-series alignment is performed on data from different time points of the same patient to construct a patient-level time-series data sequence. For cases where feature vectors are missing or incompletely labeled at some time points, a time-series interpolation method based on attention mechanism is adopted to supplement missing data by combining feature vectors from adjacent time points and clinical treatment information, ensuring the integrity of the time-series sequence. At the same time, data with examination intervals that are too short (<1 month) or too long (>12 months) are removed to avoid data noise affecting the time-series relationship modeling.

[0039] The pathological semantic feature vectors output at adjacent time points for the same patient are calculated by dimension-wise difference to obtain the difference value. The difference values ​​of all dimensions are integrated in sequence to construct the difference feature vector, which serves as an auxiliary feature to characterize the temporal changes of lesions. At the same time, the treatment plan is encoded into a vector using one-hot encoding. In the vector, "1" indicates that the patient has received the treatment plan of the corresponding dimension, and "0" indicates that the patient has not received it. For example, surgery is encoded as [1,0,0], chemotherapy is encoded as [0,1,0], and combined radiotherapy and chemotherapy is encoded as [0,0,1]. This vector is fused with the difference feature vector to enhance the model's ability to perceive treatment-related changes.

[0040] The temporal feature vector sequence of the same patient is standardized to eliminate the influence of individual differences between different patients on the quantification of temporal changes. The temporal normalization method is adopted to convert the feature vector at the time of the first visit into the relative change value, which makes it easier to intuitively reflect the changing trend of the lesion.

[0041] The "Adaptive Attention Temporal Transformer" architecture is adopted. The temporal pathological semantic feature vector sequence, differential feature vector, and treatment plan encoding vector of the same patient are concatenated into a temporal input sequence of a unified dimension and sorted by examination time. The input sequence is fed into the encoder of the Adaptive Attention Temporal Transformer. The model dynamically adjusts the attention weights according to the contribution of feature vectors at each time point to lesion changes, strengthening features at key time points and weakening features at time points with no significant changes. A cross-modal attention layer is introduced to calculate intermodal attention weights for temporal features of different modalities, realizing the association and fusion of features of different modalities. The output is a temporal feature sequence that integrates adaptive attention and cross-modal information, which is used for subsequent modeling of dynamic evolution relationships of lesions and improving cross-modal temporal tracking capabilities.

[0042] The temporal relational network outputs the association weight matrix between feature vectors at each time point. Specifically, the fused temporal feature sequence (including pathological semantics, differential features, and treatment plan coding features) processed by the adaptive attention temporal Transformer is input into the temporal relational network in chronological order. The network calculates the similarity between feature vectors at any two time points through an attention scoring mechanism, and converts the similarity into association weights to reflect the association strength of lesion features at different time points. Using time points as the row / column dimension, the association weights of all pairs of time points are filled into the corresponding positions in the matrix to generate an association weight matrix with the dimension of the number of time points × the number of time points, and then outputs it.

[0043] A dynamic evolution graph of lesions is constructed based on a matrix. The nodes in the graph represent the lesion features at each time point, and the edges represent the evolutionary relationships and weights between the features. A graph neural network (GNN) is used to optimize the evolution graph, eliminating redundant correlations and strengthening key evolution paths. Through the graph analysis module, key parameters in the evolution graph are extracted to quantitatively describe the trend of lesion changes.

[0044] Generates a time-series feature change line chart to visually display the changing trends of each key pathological feature over time; a lesion area evolution heat map, combined with images at each time point, shows the dynamic changes in lesion size and shape; output quantitative indicators include: lesion volume change rate, feature similarity change coefficient, treatment response index (calculated by combining the correlation between treatment plan and lesion changes), and disease progression risk prediction value (predicting the risk of disease changes in the next 3-6 months based on the time-series evolution pattern).

[0045] As attached Figure 5 As shown, the human-machine collaborative decision optimization module receives the visualized diagnostic report and evolution map, obtains feedback from doctors on the correction or confirmation of the system conclusions through the interactive interface, integrates incremental learning units, and applies elastic weight consolidation algorithm constraints based on the feedback data to perform targeted fine-tuning of specific parameters in the feature decoupling network and multi-task diagnostic module.

[0046] It should be further explained that the specific data selection is as follows: Doctor feedback data: Through the interactive interface, doctors' opinions on the system's output of visual diagnostic reports, time-series evolution graphs and quantitative indicators are collected, along with their confirmation results. A total of 500 doctors with different professional titles (resident physicians, attending physicians, associate chief physicians and chief physicians) are planned to collect feedback data on 10,000 cases.

[0047] Historical diagnostic data and model parameter data: Collect the system's historical diagnostic results (including correct and incorrect diagnostic cases), corresponding pathological semantic feature vectors, imaging domain feature vectors, and output results of multi-task sub-networks; at the same time, record the historical parameters of the feature decoupling network and the multi-task diagnostic module.

[0048] Clinical case follow-up data: Follow-up data of patients diagnosed by doctors were collected (such as subsequent examination results, treatment effects, and disease progression). A total of 2,000 cases with complete follow-up records were included.

[0049] Structural processing of doctor feedback data: Natural language correction opinions from doctors are structurally parsed. Core entities are accurately extracted from the text using Named Entity Recognition (NER), and relationships between entities are mined through relation extraction. This achieves structured semantic parsing of the text, extracting correction types (location correction, classification correction, hierarchical correction), error locations (image anatomical partitions, pixel coordinate ranges), error content (original error judgment result), correct conclusion (corrected judgment result), and correction basis (pathological gold standard / clinical treatment guidelines / multimodal imaging evidence). Correction opinions are associated with corresponding case data (feature vectors, diagnostic reports, evolution maps) to construct a "feedback-case-model output" triplet. Weights are assigned to the feedback data based on doctor's title, diagnostic experience, and consistency of feedback opinions (overlap with feedback from multiple doctors), assigning each feedback data point a weight of 0.5-1.0 to ensure that important feedback information has a greater impact on model optimization.

[0050] Historical diagnostic data are categorized according to error type (missed diagnosis, misdiagnosis, insufficient diagnostic evidence), disease type, and imaging modality. The model parameter characteristics corresponding to each type of error are analyzed. Historical parameters with stable model performance are selected as the basis for consolidating elastic weights. At the same time, outlier detection is performed on the feature vector data to remove abnormal feature vectors caused by data acquisition or preprocessing errors, so as to avoid them interfering with incremental learning.

[0051] The follow-up results are correlated with the system's historical diagnostic results and doctor feedback data to calculate the consistency between the system's initial diagnostic results, the doctor's revised diagnostic results, and the follow-up results. The contribution of different types of feedback to the improvement of diagnostic accuracy is analyzed to provide a basis for adjusting the optimization strategy. For feedback types with high contribution, their weight in incremental learning is increased.

[0052] The interface is designed to be lightweight and professional, allowing doctors to accurately correct visual diagnostic reports, annotate time-series evolution graphs, and adjust quantitative indicators. It incorporates a built-in feedback guidance mechanism, automatically prompting doctors to provide supplementary evidence when they offer corrections, ensuring the completeness of the feedback data. Simultaneously, it records doctors' time spent reviewing images, initial feedback time, correction submission time, cumulative number of corrections, and frequency of various correction types, enabling analysis of doctors' usage habits and optimization of interface design and model output formats.

[0053] Based on structured doctor feedback data, the error patterns of the model are analyzed to identify key modules causing errors and determine the range of parameters that need fine-tuning, avoiding "catastrophic forgetting" caused by fine-tuning all network parameters. An improved Elastic Weight Consolidation (EWC) algorithm is introduced to statistically analyze the gradient contribution of each model parameter to historical correct diagnostic cases and quantify the importance weight of each parameter (the higher the contribution, the greater the weight). Strong regularization constraints are applied to core parameters with high importance weights (limiting large parameter changes and preserving historical correct diagnostic capabilities), while weak regularization constraints are applied to low-weight parameters that need correction (allowing flexible parameter adjustments to adapt to correction needs). Differentiated regularization terms are incorporated into the model loss function, and parameters are iteratively updated through backpropagation to accurately correct erroneous parameters while preserving historical effective diagnostic capabilities.

[0054] Based on the output of the incremental learning unit, specific parameters in the feature decoupling network and the multi-task diagnosis module are fine-tuned: for the feature decoupling network, the parameters of the pathological semantic feature branches related to the misdiagnosed cases are fine-tuned to improve the extraction capability of specific pathological features; for the multi-task diagnosis module, the attention weights and fully connected layer parameters of the corresponding error sub-network are fine-tuned to optimize the diagnostic decision logic.

[0055] The fine-tuning process employs a mini-batch gradient descent algorithm, with the weighted loss of physician feedback data as the optimization target. After fine-tuning, the results are validated using a test set (2000 cases not involved in training) and follow-up data. Diagnostic accuracy, sensitivity, specificity, precision, area under the receiver operating characteristic (AUC), positive predictive value, and negative predictive value are calculated. If any indicator fails to reach a preset threshold, the feedback data and error patterns are re-analyzed, and the fine-tuning strategy is adjusted to meet the target performance.

[0056] The optimized model output results are fed back to the corresponding doctors, and their evaluation opinions on the optimized results are collected. At the same time, a model performance monitoring mechanism is established to track the diagnostic effect of the optimized model in clinical applications in real time. Every 3 months, based on newly collected doctor feedback data and follow-up data, the above optimization process is repeated to achieve continuous iterative optimization of the model. Key data in the optimization process (such as feedback data, parameter changes, and performance improvement indicators) are recorded and visualized to provide a basis for clinical application and model improvement.

[0057] Secondly: The accompanying drawings of the embodiments disclosed in this invention only involve the structures involved in the embodiments disclosed in this invention. Other structures can refer to the general design. In the absence of conflict, the same embodiment and different embodiments of this invention can be combined with each other. In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A deep learning-based medical image-assisted diagnostic system, characterized in that, include: Image preprocessing and feature decoupling module: Receives and standardizes raw medical images, and has a built-in feature decoupling network jointly trained based on mutual information minimization constraints to separate the standardized images into pathological semantic feature vectors and imaging domain feature vectors with low statistical correlation; Multi-task collaborative diagnosis and interpretability generation module: The input end connects to the aforementioned module to obtain semantic feature vectors. It has a built-in shared encoder and parallel sub-networks for lesion localization, nature classification and severity grading. It integrates an interpretability generation unit and dynamically generates a visual diagnostic report that integrates pixel-level highlighted areas and structured text decision basis based on the encoder activation map and sub-network decision weights. Cross-modal temporal tracking module: When the same patient has multiple historical images, it calls the semantic feature vectors of the current and the past, establishes the dynamic evolution relationship between feature sequences through the built-in temporal relationship network and adopts an adaptive attention mechanism, and outputs a temporal evolution map and indicators that quantitatively describe the trend of lesion changes. Human-machine collaborative decision optimization module: Receives the visualized diagnostic report and evolution map, obtains feedback from doctors on the correction or confirmation of the system conclusions through the interactive interface, integrates incremental learning units, and applies elastic weight consolidation algorithm constraints based on the feedback data to perform targeted fine-tuning of specific parameters in the feature decoupling network and multi-task diagnostic module.

2. The medical image-assisted diagnostic system based on deep learning according to claim 1, characterized in that: The feature decoupling network employs a basic encoder that connects the pathological semantic feature branch and the imaging domain feature branch in parallel. The imaging domain feature branch includes a device parameter perception layer, which encodes imaging device metadata into vectors and fuses them with image features.

3. The medical image-assisted diagnostic system based on deep learning according to claim 1, characterized in that: The interpretability generation unit adopts a dual-path fusion architecture, including a pixel-level highlight generator based on improved gradient-weighted class activation mapping and a structured text generator based on a pre-trained language model. The structured text generator introduces a diagnostic logic attention mechanism and is trained using triple data consisting of "feature-label-thought" to generate text descriptions.

4. The medical image-assisted diagnostic system based on deep learning according to claim 1, characterized in that: The severity grading subnetwork adopts an LSTM structure combined with an attention mechanism. The grading criteria are based on a comprehensive assessment of the multi-dimensional features of the lesion, including the maximum diameter of the lesion, the regularity of its shape, and the relationship between the infiltration range and key anatomical structures, classifying the severity into grades I to IV.

5. A deep learning-based medical image-assisted diagnostic system according to claim 1, characterized in that: The temporal relationship network is an adaptive attention temporal Transformer architecture. The input is a temporal sequence composed of pathological semantic feature vectors of the same patient at different time points, feature difference vectors of adjacent time points, and treatment plan encoding vectors. The adaptive attention mechanism assigns dynamic weights to the features of each time point in the temporal sequence, and the temporal features of different modalities are fused through a cross-modal attention layer.

6. The medical image-assisted diagnostic system based on deep learning according to claim 1, characterized in that: The time-series evolution map includes a dynamic evolution map of lesions, a time-series feature change line graph, and a heat map of lesion region evolution; the indicators include lesion volume change rate, feature similarity change coefficient, treatment response index, and disease progression risk prediction value.

7. A deep learning-based medical image-assisted diagnostic system according to claim 1, characterized in that: The interactive interface is equipped with a feedback guidance mechanism. When it receives a doctor's correction opinion, it automatically prompts for supplementary correction evidence and performs structured analysis on the doctor's natural language correction opinion to extract the correction type, error location, correct conclusion and correction evidence, forming structured feedback data.

8. The medical image-assisted diagnostic system based on deep learning according to claim 1, characterized in that: The elastic weight consolidation algorithm includes analyzing historical diagnostic data, statistically analyzing the gradient contribution of each model parameter to historical correct diagnostic cases, and quantifying the importance weight of each parameter. During the fine-tuning process, strong regularization constraints are applied to parameters with high importance weights, while weak regularization constraints are applied to parameters with low importance weights. The fine-tuning targets are limited to the pathological semantic feature branch parameters of the feature decoupling network located by high-quality feedback data from doctors, and the specific layer parameters of the corresponding error subnetwork in the multi-task diagnosis module.

Images