A medical image automatic labeling and enhancement method and system based on a generative adversarial network

Abstract

The application discloses a kind of medical image automatic labeling and enhancement method and system based on generative adversarial network;Belong to medical image processing and artificial intelligence technical field;Method includes: through federated learning security training fusion anatomical prior generative adversarial network model;Image is enhanced and synthesized using the generative adversarial network model, and automatic segmentation and detection are simultaneously executed;Based on uncertainty measure, sample is screened and is marked by doctor, and feedback optimization model;Result is evaluated and visualized output in multidimension;System includes data preparation and federal training module, image generation and intelligent labeling module, active learning optimization module and evaluation and visualized output module, supports distributed deployment and security collaborative training;The present application realizes high-quality enhancement and accurate automatic labeling of medical image under the premise of protecting data privacy, improves image diagnosis efficiency and model generalization ability.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing and artificial intelligence technology, specifically relating to a method and system for automatic annotation and enhancement of medical images based on generative adversarial networks, which is applicable to intelligent analysis, assisted diagnosis and data enhancement of medical images. Background Technology

[0002] Accurate annotation and quality enhancement of medical images are crucial foundations for assisted diagnosis, treatment planning, and medical research. Currently, this field faces several major technical bottlenecks: First, high-quality medical image annotation heavily relies on manual sketching by radiologists, a time-consuming and labor-intensive process prone to subjective variations. Second, the limited data samples from single medical institutions make it difficult to train AI models with strong generalization capabilities, while centralized collection of data from multiple institutions presents significant challenges related to patient privacy, data security, and regulatory compliance. Third, existing deep learning-based automatic annotation methods often lack accuracy and robustness when data is scarce or lesion morphology is complex. Furthermore, existing image enhancement methods may introduce anatomical distortions while improving image quality, affecting the reliability of subsequent diagnoses. Finally, the evaluation of model output results is often limited to single image quality indicators, lacking a comprehensive evaluation system directly linked to clinical diagnostic performance. Therefore, there is an urgent need for a technical solution that can effectively utilize multi-center data without disclosing the original data, collaboratively achieve high-quality automatic image generation and accurate and reliable annotation, and continuously optimize itself. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method and system for automatic annotation and enhancement of medical images based on generative adversarial networks, so as to achieve automated quality enhancement and high-precision annotation of medical images.

[0004] The objective of this invention can be achieved through the following technical solutions: This application provides a method and system for automatic annotation and enhancement of medical images based on generative adversarial networks. The specific methods are as follows; Step 1: Data preparation and federated model training. Obtain medical image datasets from multiple data sources, perform anonymization and standardization preprocessing on the datasets, and train a generative adversarial network model that integrates anatomical prior knowledge using a federated learning paradigm. Step 2: Image generation and automatic annotation. Using the trained generative adversarial network model, the input medical images are enhanced and synthetic images are generated. At the same time, the annotation network trained in conjunction with the generator is activated to perform automatic segmentation and target detection on the input and synthetic images, generating structured annotation information. Step 3: Active learning optimization. Based on the automatic annotation results output by the annotation network, calculate its uncertainty measure, filter out samples with uncertainty higher than a preset threshold, submit them to radiologists and professionals for manual annotation, and add the manually annotated new samples to the training dataset to update the generative adversarial network model and the annotation network. Step 4: Results evaluation and output. Perform multi-dimensional quantitative and qualitative evaluation based on clinical task efficacy on the enhanced and generated images and automatically labeled results. Visualize and output the final results that meet the preset quality standards.

[0005] An automatic medical image annotation and enhancement system based on generative adversarial networks; specifically including: The data preparation and federated training module is distributed and deployed in local servers within each medical institution to protect the raw data, as well as an external central server that runs independently to coordinate model training tasks. The data preparation and federated training module is used to obtain medical image data from each institution, perform anonymization and standardization preprocessing on each local server, and, under the coordination of the central server, securely aggregate the computational results of all parties using a federated learning approach to jointly train a generative adversarial network model that incorporates anatomical prior knowledge constraints. The image generation and intelligent annotation module is deployed on an external central server where the model training is completed. To improve the elasticity, scalability and reliability of the service, the image generation and intelligent annotation module can also be deployed on a cloud-based high-performance computing platform interconnected with the central server. The image generation and intelligent annotation module is used to load and run the aforementioned trained generative adversarial network model, to perform quality enhancement, modality conversion and pathological feature synthesis on the received medical images, and to automatically generate corresponding lesion segmentation maps and target detection boxes using the built-in annotation network that works in collaboration with the adversarial network model generator. The active learning optimization module is deployed on the clinical doctor's workstation where radiologists perform daily image interpretation and annotation. The active learning optimization module is used to receive automatic annotation results, quantify their uncertainty by calculating entropy and prediction variance, and automatically select the images and regions that most need to be reviewed based on this uncertainty and push them to the doctor for manual fine annotation. Then, the high-quality annotation data confirmed and corrected by the doctor is used as new samples and fed back to the training process to continuously optimize the model performance. The assessment and visualization output module is also deployed on the clinician workstation. This module is used to perform a comprehensive assessment of the images and annotation results generated by the system, combining automated index calculation with subjective evaluation by radiologists and professionals. The final results that pass the assessment are provided to doctors in the form of image and annotation information overlay rendering, 3D visualization, and interactive operation interface for auxiliary diagnosis and analysis.

[0006] The beneficial effects of this invention are as follows: To address the issues of limited data from a single institution leading to weak model generalization ability, and the severe privacy and compliance challenges posed by centralized collection of multi-center data, this invention adopts a federated learning paradigm and integrates homomorphic encryption and differential privacy technologies to construct a distributed secure training architecture. Each participant completes data anonymization and preprocessing locally, and only uploads encrypted and noise-protected model gradient updates, which are then securely aggregated by a central server. This ensures that the original medical data never leaves the local machine, completely breaking down data barriers, effectively aggregating the value of multi-center data, and collaboratively training a high-performance, highly generalizable generative adversarial network basic model. To address the shortcomings of automatic annotation methods in complex lesions and the potential for anatomical distortion introduced by traditional image enhancement, this invention utilizes a generative adversarial network that integrates prior anatomical knowledge. The generator's optimization objective innovatively incorporates an anatomical constraint loss based on deformable registration, forcing the generated or enhanced images to maintain consistency with standard anatomical atlases in key structures. Simultaneously, a lightweight annotation network sharing features with the generator is employed for collaborative training and parallel inference, fundamentally ensuring the anatomical rationality of the output images, avoiding structural distortion, and enabling the simultaneous output of high-quality enhanced images and high-precision, structured automatic segmentation and detection results in a single forward pass, thus improving annotation accuracy and clinical usability. To address the challenges of continuous model evolution and the disconnect between existing evaluation systems and clinical diagnostic efficacy, this invention constructs an active learning mechanism based on uncertainty quantification and a multi-dimensional clinical task efficacy evaluation system. By calculating the entropy and prediction variance of automatically labeled results, high-uncertainty samples are precisely selected for physician review, and the corrected, high-quality samples are fed back into the model for incremental learning. The evaluation process not only calculates traditional image quality indicators but also introduces independent downstream diagnostic model quantification to improve performance. Furthermore, double-blind expert subjective scoring is conducted, forming an efficient self-evolutionary closed loop of model labeling, expert correction, and model optimization. This ensures that human effort is focused on the model's weaknesses, and by anchoring the evaluation to downstream diagnostic task performance and clinician subjective experience, a close link between system output and ultimate clinical application value is guaranteed. Attached Figure Description

[0007] To better understand and implement this application, the technical solution is described in detail below with reference to the accompanying drawings.

[0008] Figure 1 This is a flowchart illustrating the automatic medical image annotation and enhancement method based on generative adversarial networks provided in Embodiment 1 of this application. Figure 2 This is a schematic diagram of the components of the automatic medical image annotation and enhancement system based on generative adversarial networks provided in Embodiment 2 of this application. Detailed Implementation

[0009] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, exemplary embodiments will be described in detail below, examples of which are illustrated in the accompanying drawings. In the following description, when referring to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0010] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used herein are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0011] The following detailed description of the specific implementation methods, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided in detail.

[0012] Example 1 Please see Figure 1 This embodiment provides an automatic annotation and enhancement method for medical images based on generative adversarial networks; including the following steps: Step 1: Data Preparation and Federated Model Training. Medical image datasets from multiple data sources are acquired. The datasets undergo anonymization and standardization preprocessing, which is automatically executed on the local servers deployed on each participating node. Anonymization and deidentification are achieved by stripping all direct and indirect identifying fields, such as patient name, ID number, examination number, and hospital identifier, from the medical digital imaging and communication standard header file. Spatial resampling is based on internationally recognized medical image templates, such as the Montreal Neurological Institute template or atlases specific to various anatomical locations. Linear or spline interpolation algorithms are used to align the image voxel space to the standard coordinate space and resolution. Pixel intensity value normalization is based on image modal characteristics. For example, for computed tomography images, Henle unit window level adjustment is used to linearly map to the zero-to-one interval; for magnetic resonance imaging, quantile clipping or histogram matching is used to eliminate signal intensity inconsistencies caused by differences in equipment and protocols. Finally, a federated learning paradigm is used to train a generative adversarial network model that integrates prior anatomical knowledge. Further, in step one, data preprocessing specifically involves anonymizing and de-identifying medical images, resampling spatial orientations to standard templates, and normalizing pixel intensity values ​​to a unified range. Federated model training specifically involves the following process: each participating node, i.e., each medical institution's local server, uses local data to train the generative adversarial network (GAN) model. The GAN model includes a generator based on an encoder-decoder structure and a discriminator based on a convolutional neural network. During training, the generator uses low-quality or noisy images as input. Its optimization objective, in addition to adversarial loss, also integrates consistency constraint loss based on deformable registration networks and standard anatomical atlas segmentation maps of key organs. The update gradient of the model parameters is calculated. This gradient is the partial derivative vector of the model weights calculated using the backpropagation algorithm on local data batches. The gradient vector is encrypted using a partially homomorphic encryption scheme, such as the Paillier algorithm, or Gaussian noise or Laplacian noise that meets differential privacy requirements is added to protect the gradient. The addition of differential privacy noise follows strict privacy budget constraints, namely, setting specific privacy budget parameters ε and relaxation parameters δ. ε is typically set in the range of 0.1 to 5.0, and δ is set to a very small value, such as less than 10 to the power of -5. The noise scaling parameter is calculated based on the set ε and δ values, as well as the gradient sensitivity determined by gradient clipping. The protected gradients are then uploaded to the central server, which performs secure aggregation of the aggregated gradients to update the global model parameters. For encrypted gradients, the secure aggregation operation first decrypts them or performs a weighted average directly in the homomorphic ciphertext domain; for gradients with added noise, a direct average is performed. The aggregation weights can be dynamically allocated based on the data volume or contribution of each node, and the updated parameters are distributed to each participating node. This process iterates until the model converges. The criteria for model convergence are that the decrease in the global loss function value over multiple consecutive training rounds is less than a preset threshold, or the quality evaluation index of the generated image on the reserved validation set tends to stabilize.

[0013] During the training of the federated model, after calculating the update gradient of the model parameters and before encryption and noise protection, a gradient pruning step is also included. This step calculates the global norm or layer-by-layer norm of the update gradient and compares it with a preset pruning threshold. The pruning threshold is determined by empirical formulas or dynamic adjustment strategies based on the model parameter size, training stage, and preset privacy budget. When the norm exceeds the pruning threshold, the gradient vector is scaled proportionally to within the threshold, thereby controlling the magnitude of the gradient to ensure training stability and providing a deterministic sensitivity boundary for adding differential privacy noise later. The norm of the updated gradient is pruned to ensure that it does not exceed a preset pruning threshold; during the secure aggregation process, the central server dynamically allocates aggregation weights to the protected updates from each node based on the effective data volume or historical contribution of each participating node in this round of training. The effective data volume is measured by the number of medical image samples that the statistical nodes actually participate in forward and backward propagation in this round of training. The historical contribution is calculated by comprehensively evaluating multiple dimensions such as the quality of gradient updates provided by the nodes in previous training rounds, the continuity and stability of their participation in training. The specific method of dynamically allocating aggregation weights is as follows: the central server first decrypts or directly processes the protected updates reported by each node, and then performs a weighted average calculation on the decrypted or processed gradient updates based on the calculated weight coefficients of each node. The weight coefficients are positively correlated with the effective data volume or historical contribution of each node, and the sum of all weight coefficients is one, thereby realizing the global model update that integrates the knowledge contributions of all parties.

[0014] Specifically, the data preparation and federated model training steps address the challenges of privacy, security, and standardization in multi-center medical image data collaboration. This method first automates data anonymization and spatial intensity normalization locally, eliminating data heterogeneity caused by differences in devices and protocols, providing a high-quality and consistent data foundation for model training. Then, by introducing a federated learning framework that integrates homomorphic encryption and differential privacy technologies, participating nodes collaboratively train a generative adversarial network model without sharing original data. This breaks down data barriers and leverages the value of dispersed multi-source data while strictly protecting patient privacy and data security. During training, the generator integrates a consistency constraint loss based on anatomical atlases, ensuring the rationality of the anatomical structure in the generated or enhanced images. This process ultimately establishes a convergent distributed training paradigm with prior anatomical knowledge, laying a safe, reliable, and high-performance model foundation for subsequent image enhancement and automatic annotation tasks.

[0015] Step 2: Image generation and automatic annotation. Using the trained generative adversarial network model, the input medical image is enhanced or a synthetic image is generated. The enhancement process includes at least one of denoising, super-resolution reconstruction, artifact suppression, or modality conversion. The synthetic image is a clinically significant simulated image generated based on the distribution of specific pathological features or anatomical structural variability. At the same time, the annotation network trained in conjunction with the generator is activated to perform automatic segmentation or target detection on the input image or synthetic image, generating structured annotation information. The structured annotation information includes pixel-level semantic category labels, instance numbers, and the coordinates and confidence scores of the target bounding box. Furthermore, in step two, the generator of the generative adversarial network model receives the input image and outputs the processed image. Its training loss function includes an anatomical structure constraint loss term based on deformable registration. The loss term is calculated by the deformation field output by the deformable registration module and the segmentation map generated by the pre-trained segmentation network. Specifically, the generator output image is first non-rigidly aligned with the standard anatomical atlas using the deformable registration network to obtain a dense displacement field. Then, the displacement field is applied to the key anatomical structure segmentation map generated by the pre-trained segmentation network from the real image to obtain a deformation-corrected reference segmentation map. Finally, the cross-entropy loss or Dessian similarity coefficient loss between the segmentation map obtained by the generator output image through the same pre-trained segmentation network and the reference segmentation map is calculated. The deformable registration is performed by an independent, encoder-decoder-based convolutional neural network. The convolutional neural network takes the output image of the generator and the standard anatomical atlas as input and outputs a dense displacement field. The encoder part of the encoder-decoder convolutional neural network progressively extracts multi-scale deep semantic features of the input image pairs through multiple downsampling stages, while the decoder part restores the spatial resolution through continuous upsampling operations and finally regresses the displacement components of each voxel in two or three spatial directions, thus forming a dense displacement field. During training, the network aims to minimize the similarity metric loss between the output image and the standard atlas, as well as the smoothing regularization loss of the displacement field. The calculation process of the anatomical structure constraint loss includes: using the deformable registration network, registering the generator's output image with a standard anatomical atlas to obtain a corresponding dense displacement field; the dense displacement field is a multidimensional array with the same spatial size as the input image, where the vector value at each spatial location represents the displacement required for the voxel to move to the corresponding position in the standard anatomical atlas; applying this dense displacement field to the key anatomical structure segmentation map generated by the pre-trained segmentation network from the real image or standard atlas, and obtaining a deformed reference segmentation map through spatial transformation; the spatial transformation operation uses forward or backward mapping methods based on linear interpolation or spline interpolation to relocate each voxel label in the original segmentation map according to the displacement field; finally, inputting the generator's output image into the same pre-trained segmentation network to obtain its predicted segmentation map, and obtaining the anatomical structure constraint loss by calculating the consistency difference between the predicted segmentation map and the reference segmentation map; the consistency difference is usually calculated and quantified using a pixel-level cross-entropy loss function or a region-level Dessian similarity coefficient loss function. The annotation network shares some initial convolutional layers with the generator, receives intermediate feature maps extracted by the generator, and upsamples them to the original resolution through transposed convolutional layers to output pixel-level segmentation masks and target bounding boxes. The shared initial convolutional layers refer to the annotation network directly reusing the feature maps output by one or more downsampled convolutional modules at the front end of the generator encoder as its own input, thereby reducing parameter redundancy and computational overhead. The annotation network's own independent decoding branch consists of multiple cascaded upsampling modules. Each upsampling module includes transposed convolution operations, feature concatenation operations, and regular convolution operations, used to gradually fuse deep semantic features from the shared layers with features from the annotation network's own intermediate layers to recover detailed information. Finally, the network generates segmentation masks and target bounding boxes through two independent output heads. The segmentation output head outputs the class probability of each pixel through convolutional layers and normalized exponential function activation, while the detection output head outputs the class confidence and coordinate offset of preset anchor boxes through convolutional layers and linear activation.

[0016] The generator employs an encoder and decoder structure. The encoder contains consecutive downsampled convolutional blocks for extracting multi-scale features. Each downsampled convolutional block typically consists of a convolutional layer, an instance normalization layer, and a leaky linear rectified activation function stacked sequentially. The feature map space size is halved at each level while the number of channels is multiplied. The decoder contains upsampled transposed convolutional blocks symmetrical to the encoder and skip connections for reconstructing high-resolution images. Each upsampled transposed convolutional block receives features from the corresponding encoder layer through skip connections and performs channel concatenation and feature fusion. Finally, the features are mapped back to the image space through convolutional layers and hyperbolic tangent activation functions.

[0017] Specifically, this method effectively solves the problem of synergistically achieving medical image quality improvement and automated structured annotation through image generation and automatic annotation steps. It utilizes a well-trained generative adversarial network model to perform various enhancement processes on input images, including denoising, resolution enhancement, and artifact suppression. It can also generate synthetic images with clear pathological features based on clinical priors, thereby expanding the diversity of high-quality training data. Crucially, the generator's optimization process introduces anatomical constraints based on non-rigid registration. By aligning the output image with standard atlases and comparing the segmentation consistency of key anatomical regions, it ensures that all enhancement or synthetic results conform to the real anatomical structure, avoiding morphological distortion caused by image processing and ensuring the reliability of subsequent analysis. Simultaneously, the annotation network, sharing features with the generator, is activated synchronously, enabling efficient pixel-level semantic segmentation and target detection of the original or generated images, directly outputting structured annotations containing category, instance, and location information. This design achieves parallel processing and mutual promotion of image quality improvement and automatic annotation tasks within a unified model framework, improving the overall processing efficiency and output quality from original images to enhanced images and annotation results that can be directly used for assisted diagnosis.

[0018] Step 3: Active learning optimization. Based on the automatic annotation results output by the annotation network, calculate its uncertainty measure. The uncertainty measure is used to quantify the model's confidence in its own prediction results and is the core basis for screening samples that need to be manually reviewed. Samples with uncertainty higher than a preset threshold are selected and submitted to radiologists and professionals for manual annotation. The newly annotated samples are added to the training dataset to update the generative adversarial network model and the annotation network. Furthermore, in step three, the calculation of uncertainty measurement specifically includes obtaining the pixel-level semantic category probability distribution output by the annotation network after inference on the input image, as the prediction result; the pixel-level semantic category probability distribution is a three-dimensional tensor, the first two dimensions of which correspond to the spatial height and width of the input image, and the third dimension is equal to the total number of anatomical structures or lesion categories to be segmented, and the vector at each spatial location represents the normalized probability of the pixel belonging to each category; Based on the prediction results, a first uncertainty measure is obtained by calculating the entropy of its probability distribution. The entropy is calculated by performing a negative operation on the dot product of the class probability vector at each pixel and its corresponding logarithmic probability vector, and then summing the results. Mathematically, the negative operation is represented as follows: for each pixel, first calculate the logarithmic value of each element in its class probability vector, then multiply the original probability vector element-wise with the corresponding logarithmic value vector and sum the results, and finally take the negative value of the sum. The final first uncertainty measure is obtained by taking the arithmetic mean or summing the entropy values ​​of all valid pixels in the image. Simultaneously, multiple random dropout inferences are performed in the convolutional layers of the decoding branch of the annotation network to obtain multiple sets of prediction results. A second uncertainty measure is obtained by calculating the variance between these prediction results. The random dropout inference is performed by temporarily setting the output neurons of one or more specified convolutional layers in the decoding branch of the annotation network to zero with a preset dropout probability during the forward propagation process. Each forward propagation generates an independent set of random dropout patterns, thereby repeatedly performing forward inference a preset number of times on the same input image to obtain multiple sets of prediction results. The variance is calculated by statistically analyzing the dispersion of the probability value of each pixel in each category across different forward inference results, and quantifying the final model cognitive uncertainty by calculating the standard deviation or variance of these probability values ​​and aggregating them across categories and spaces. The screening process, based on the first and second uncertainty measures, selects a set of samples with the highest uncertainty measure values ​​to form a labeling set. Specifically, the screening strategy involves first assigning normalization coefficients or fusion weights to the first and second uncertainty measures, calculating the overall uncertainty score for each sample, and then selecting a fixed number of samples with the highest overall scores, or selecting all samples whose overall scores exceed a dynamic threshold, according to a preset budget or percentage. Radiologists and professionals manually label the samples in the set using the drawing tools provided in the graphical interface, and the corrected labeling data is stored as high-quality labels. The graphical interface provides multi-scale brushes, intelligent contour capture, and region filling. The system includes interactive tools for filling, erasing, and dragging / adjusting bounding boxes, and supports overlay modifications on the automatically annotated draft layer. Corrected annotation data is stored as a binary mask image file or a structured coordinate list aligned with the original image space, along with the revision time and operator information. The model update process uses an expanded training set containing new annotated samples to jointly fine-tune the parameters of the generative adversarial network (GAN) model and the annotation network. This joint fine-tuning is initiated with a small learning rate, and the optimization objectives include the generator's adversarial loss and anatomical constraint loss, and the annotation network's supervised segmentation and detection losses. During training, some pre-trained layer parameters are frozen, and the process iterates until performance convergence on an independent validation set.

[0019] During the screening process, samples are selected based on a comprehensive uncertainty metric. The comprehensive uncertainty metric is the result of linearly weighting the first uncertainty metric and the second uncertainty metric according to preset weights. The preset weights are determined based on grid search of the performance gains of the model after fine-tuning the selected samples under different weight combinations on the historical validation set or based on prior experience, aiming to balance the differences in the contributions of the two types of uncertainty to model optimization. Before performing the weighted summation, the first uncertainty metric and the second uncertainty metric need to be normalized to eliminate the influence of their different dimensions and numerical ranges on the weighting result. During the screening process, while prioritizing the selection of samples with the highest comprehensive uncertainty metric, the similarity between samples in the feature space is also calculated to remove duplicates of overly similar samples, thus ensuring the case diversity of the set to be labeled. The calculation of feature space similarity involves first extracting the feature map output from the penultimate convolutional layer of the encoder in the annotation network, and then performing global average pooling to obtain a fixed-dimensional feature vector for each sample. Next, the cosine similarity or Euclidean distance between the candidate sample feature vector and the feature vectors of all samples currently selected for annotation is calculated as a similarity metric. Deduplication is performed in each round of filtering iterations. The specific process is as follows: the annotation set is initialized to empty; all candidate samples are sorted in descending order of their comprehensive uncertainty metric and evaluated sequentially. For the currently evaluated sample, if its feature similarity with any existing sample in the annotation set exceeds a preset similarity threshold, the sample is skipped; otherwise, it is added to the annotation set. This iterative process continues until the annotation set reaches a preset sample capacity or all candidate samples have been evaluated. This effectively avoids redundancy in deep feature representation of selected samples while prioritizing high uncertainty, ensuring diverse coverage of pathological morphology and imaging features.

[0020] Specifically, by introducing an active learning optimization mechanism, this method effectively addresses key issues in automated annotation processes, such as low confidence in model predictions, insufficient identification of difficult samples, and a lack of high-quality feedback data for subsequent model iterations. Its core lies in scientifically quantifying the uncertainty of each prediction by calculating the entropy of the automated annotation results and the variance of multiple random inferences. This automatically and accurately identifies image regions or complete cases where the model lacks confidence due to blurred lesion boundaries, morphological variations, or insufficient training. Based on the quantified uncertainty measure, this method intelligently filters and prioritizes the samples most in need of review, forming a queue of annotation tasks. This allows radiologists to efficiently focus their manual review efforts on the truly weak points of the model, greatly improving the relevance and value of manual review. The refined annotation data generated after doctors correct high-uncertainty areas using interactive tools is used as high-quality, high-information new samples. Subsequently, these samples are used to selectively fine-tune the existing model, enabling it to continuously learn from its own predictive uncertainty and expert corrections. This closed-loop process effectively integrates and progressively enhances the knowledge of human experts with the model's capabilities, continuously optimizing the model's annotation accuracy and generalization robustness in complex scenarios.

[0021] Step 4: Result Evaluation and Output. Perform multi-dimensional quantitative and qualitative evaluation based on clinical task efficacy on the enhanced and generated images and automatically labeled results. Clinical task efficacy specifically refers to the effectiveness and reliability of medical images in actual clinical application scenarios such as assisting disease diagnosis, treatment planning, or prognosis assessment. Visualize and output the final results that meet the preset quality standards. The quality standards include the threshold requirements of quantitative indicators and the average level of qualitative scores.

[0022] Further, in step four, the multi-dimensional evaluation specifically includes: calculating the peak signal-to-noise ratio (PSNR) and structural similarity index of the generated image and the real image; the calculation process is performed on the registered image pair, the PSNR reflects the pixel-level error between the reconstructed image and the reference image, and the structural similarity index comprehensively evaluates the overall similarity of the two images from three dimensions: brightness, contrast, and structure; the generated and enhanced images are input into an independent downstream diagnostic model, which is a deep neural network pre-trained on a high-quality labeled dataset for specific disease diagnosis or anatomical structure segmentation, such as a convolutional neural network for lung nodule detection or a fully convolutional network for brain tumor segmentation, to evaluate the improvement of the downstream diagnostic model's performance indicators in classification and segmentation tasks; the performance indicators include, but are not limited to, accuracy, sensitivity, and specificity in classification tasks, and Dessian similarity coefficient, intersection-over-union ratio, and Hausdorff distance in segmentation tasks, and the improvement is measured by comparing the differences in indicators when the model uses the original image and the enhanced image as input; organization Radiologists and professionals subjectively rate the enhanced, generated images and their corresponding automatic annotations without the radiologists' knowledge. This process employs a double-blind design; evaluators are unaware whether the evaluated images are real or system-generated, nor are they aware of the source of the annotations. Scoring is based on a pre-defined clinical diagnostic applicability scale, which covers dimensions such as anatomical clarity, noise level, artifact severity, and overall diagnostic confidence. Each dimension uses an ordinal rating system. The visualization output displays the evaluated images and corresponding annotations as overlays. Annotations, such as segmentation contours or detection boxes, are overlaid on the base image in semi-transparent or high-contrast colors and can be shown or hidden. Interactive functions such as window width and level adjustment, contour delineation, and modification are provided. Window width and level adjustment allows users to dynamically adjust the image's contrast and brightness to optimize the visualization of different tissues. The contour delineation and modification functions allow physicians to directly fine-tune the boundaries of the automatic annotations on the final output interface. All interactive operations are recorded and can be exported as part of the report.

[0023] Specifically, by establishing a multi-dimensional evaluation and output process based on clinical task effectiveness, this approach effectively addresses the lack of systematic, reliable, and clinically relevant validation of generative model outputs. Its core lies in constructing a comprehensive evaluation system that integrates objective quantitative indicators, downstream task effectiveness gains, and double-blind expert subjective evaluation. This system not only verifies image fidelity at the pixel and structural levels by calculating peak signal-to-noise ratio and structural similarity index, but more importantly, it introduces an independent downstream diagnostic model to directly quantify the improvement in performance indicators of enhanced or synthesized images for actual diagnostic tasks, thus anchoring the evaluation to ultimate clinical application value. Simultaneously, a rigorous double-blind subjective scoring design is employed to evaluate the diagnostic applicability of images from a clinician's perspective, ensuring that the results align with real-world image reading habits and needs. The images and annotation results evaluated above are ultimately visualized and output through an interactive interface supporting dynamic window width and level adjustment, annotation layer overlay display, and online contour editing. The final output meets technical standards, withstands clinical scrutiny, and can directly serve auxiliary diagnostic and analytical work in a familiar and flexible manner for physicians.

[0024] This embodiment describes a closed-loop method for intelligent medical image processing based on generative adversarial networks (GANs). The method establishes a compliant multi-center model training paradigm at the source by combining federated learning and privacy protection technologies. Based on this, a GAN integrating anatomical structural constraints is designed, enabling it to simultaneously output anatomically reasonable enhanced images and preliminary structured annotations. Furthermore, by quantifying the uncertainty of model predictions, the method constructs an efficient active learning loop, continuously incorporating expert knowledge to precisely optimize the model. The method innovatively adopts downstream diagnostic task performance improvement as a key evaluation criterion, combined with double-blind subjective evaluation, forming a comprehensive evaluation system oriented towards clinical utility. The entire process is interconnected, achieving automated and intelligent output from distributed raw data to high-quality, directly diagnostically assisted enhanced images and precise annotations, and providing a mechanism for sustainable model evolution.

[0025] Example 2 Please see Figure 2 This embodiment provides an automatic medical image annotation and enhancement system based on generative adversarial networks; including: The data preparation and federated training module is distributed across local servers within each medical institution to protect the raw data, and an external central server that runs independently to coordinate model training tasks. The local servers are physical servers deployed within the medical institution's firewall or certified private cloud virtual machines, while the central server is a physical server located in a trusted third party or a dedicated tenant area in a public cloud. The data preparation and federated training module is used to obtain medical image data from each institution, perform anonymization and standardization preprocessing on each local server, and drive the preprocessing process with automated scripts deployed on the local servers. It also interfaces with the hospital's image archiving and communication system through standardized interfaces to obtain the raw data. Under the coordination of the central server, the module securely aggregates the computational results of all parties using a federated learning approach to jointly train a generative adversarial network model that incorporates anatomical prior knowledge constraints. Furthermore, the data preparation and federated training module also includes a local model training unit and a secure communication unit deployed on each of the local servers, and a model coordination unit and a secure aggregation unit deployed on the external central server. The local model training unit is used to perform forward and backward propagation calculations on the received generative adversarial network model using local data that has been anonymized and normalized. The specific process includes downloading the global model weight file for the current round from the central server, loading it into the local graphics processor memory, using local data to perform forward propagation calculations in batches to calculate the loss, and then calculating the gradient tensors of the parameters of each layer of the model through the backpropagation algorithm to generate local updates of the model parameters. The secure communication unit processes the local update using homomorphic encryption and differential privacy noise. Homomorphic encryption is implemented using a partial homomorphic encryption library of support vector addition and scalar multiplication, encrypting the gradient tensor element by element. Adding differential privacy noise involves superimposing a noise matrix sampled from a zero-mean Gaussian or Laplace distribution onto the gradient tensor. The noise scale is dynamically calculated based on the gradient norm boundary and a preset privacy budget parameter, generating a protected update, which is then transmitted to an external central server via a secure channel. The secure channel is a point-to-point encrypted network connection built based on transport layer security protocols or national cryptographic algorithms. The model coordination unit is used to distribute global generative adversarial network model parameters during initialization and iteration to each local server. The model coordination unit maintains a global model version number. At the start of each training round, all registered local servers broadcast training instructions and model parameter files containing the latest version number and schedule training rounds. The scheduling strategy includes waiting for a fixed number of nodes to participate, setting the maximum waiting time for each training round, and handling dynamic changes such as node disconnection or joining. The secure aggregation unit is used to perform secure aggregation operations on the protected updates uploaded by each local server. For encrypted updates, the secure aggregation unit performs a weighted average operation in the encrypted state, and the weights are usually determined according to the proportion of data that each node participates in training in this round. For updates with added noise, the unit first decrypts or directly performs a weighted average of the noise gradient to calculate the next generation of global model parameters, thereby completing one round of federated training iteration. After the aggregation is completed, the secure aggregation unit passes the generated new global parameters to the model coordination unit to initiate the next round of training or terminate the training process after determining that the model has converged.

[0026] Specifically, the data preparation and federated training modules construct a secure and efficient distributed training architecture, successfully resolving the core contradiction between data privacy protection and collaborative model performance improvement in multi-center collaborations in the field of medical imaging. By completing data anonymization, standardization, and initial model training locally at each medical institution, it ensures that patients' original sensitive data does not leave their trust boundaries, fundamentally meeting the mandatory requirements of data security and privacy regulations. Simultaneously, by leveraging homomorphic encryption and differential privacy technologies to protect model updates during transmission, and combined with the coordination and secure aggregation mechanism of the central server, it effectively integrates knowledge updates from various parties without aggregating the original data, jointly training a high-performance global generative adversarial network model that incorporates anatomical priors. This architecture breaks down data barriers, achieves compliant multi-center collaborative computing, and provides a unified, reliable, and high-quality model foundation for subsequent image enhancement and automatic annotation tasks, ensuring the system's compliance and effectiveness from the outset.

[0027] The image generation and intelligent annotation module is deployed on an external central server where the model training is completed. To improve the elasticity, scalability and reliability of the service, the image generation and intelligent annotation module can also be deployed on a cloud high-performance computing platform interconnected with the central server. The cloud platform can provide a containerized deployment environment and on-demand allocated graphics processor resources, and support load balancing and automatic scaling. The image generation and intelligent annotation module is used to load and run the aforementioned trained generative adversarial network model, and to perform quality enhancement, modality conversion and pathological feature synthesis on the received medical images. At the same time, it uses the built-in annotation network that works in conjunction with the adversarial network model generator to automatically generate corresponding lesion segmentation maps and target detection boxes. Furthermore, the image generation and intelligent annotation module also includes a model loading and service interface unit, an anatomical structure constraint generation unit, and a lightweight parallel annotation unit; The model loading and service interface unit is used to load the parameters of the trained generative adversarial network model from storage to the memory of the computing device. The storage is a distributed file system or object storage service of a central server or cloud platform. The loading process includes reading the model architecture configuration file and binary weight file, and instantiating the corresponding neural network computation graph in memory according to the file content. It also provides standardized application programming interfaces and network service interfaces to receive input image data and return processing results. The application programming interface supports calls from common programming languages. The network service interface is a representational state transition architecture interface or a remote procedure call interface based on the Hypertext Transfer Protocol. It receives data encapsulated in medical digital imaging and communication standard formats or general image formats, and the returned results include both processed image files and structured annotation data files. The anatomical structure constraint generation unit is integrated into the generator of the generative adversarial network model. This generator employs an encoder-decoder structure. The encoder extracts multi-scale features of the input image through successive downsampling convolutional layers. The encoder typically contains four to five downsampling stages, each consisting of two to three convolutional layers, an instance normalization layer, and an activation function. The feature map spatial size is halved successively, while the number of channels is doubled successively. The decoder reconstructs the high-resolution output image through upsampling transposed convolutional layers containing skip connections. The decoder stage is symmetrical to the encoder; each upsampling stage doubles the feature map size through transposed convolution and... The feature maps of the over-skip connection and the corresponding stage of the encoder are concatenated in the channel dimension to fuse low-level details and high-level semantic information. The anatomical structure constraint generation unit introduces anatomical structure constraint loss based on deformable registration during the training and inference process of the generator to ensure the anatomical rationality of the output image. In the inference stage, the anatomical structure constraint generation unit calls the pre-trained deformable registration network and segmentation network to calculate the consistency between the generator output image and the anatomical atlas on the key structure segmentation map, which serves as a feedback signal but is not used to update the weights, so as to ensure that the anatomical structure of each generated result is consistent with the prior knowledge. The lightweight parallel annotation unit includes an annotation network that shares the underlying convolutional layers with the generator encoder. The shared underlying convolutional layers typically refer to the first two to three stages of the encoder. This design eliminates the need for the annotation network to repeatedly compute basic visual features, significantly reducing computational overhead and memory usage. The annotation network receives intermediate feature maps extracted by the generator encoder and performs upsampling and feature fusion through an independent decoding branch. The independent decoding branch consists of a series of upsampling convolutional modules. Each module upsamples the input features and fuses them with features of the same scale from the shared encoder, gradually restoring spatial resolution and outputting pixel-level semantic segmentation probability maps and object detection bounding box coordinates in parallel. The semantic segmentation probability map obtains the probability of each pixel belonging to a category through a 1-to-1 convolutional layer and a normalized exponential function activation. The object detection branch connects the region proposal network and the bounding box regression head after a series of convolutional layers, outputting the category confidence and coordinate offset of preset anchor boxes.

[0028] Specifically, the image generation and intelligent annotation module constructs a core of efficient and reliable medical image processing and annotation services. Through the model loading and service interface unit, the trained complex generative adversarial network model is transformed into a standardized service that can be remotely invoked, solving the problems of difficult model deployment and inconsistent interfaces, enabling clinicians to conveniently submit images and obtain processing results. Its core anatomical structure constraint generation unit enforces anatomical rationality checks at every stage of image enhancement and synthesis, fundamentally solving the structural distortion problem that may arise from generative models, ensuring the clinical usability and diagnostic reliability of the output images. Simultaneously, the lightweight parallel annotation unit, through a feature sharing mechanism, achieves high-precision lesion segmentation and target detection synchronously with the image enhancement process with almost no additional computational burden, solving the inefficiency and inconsistent results caused by the separation of enhancement and annotation tasks. Ultimately, the image generation and intelligent annotation module achieves the effect of collaboratively producing high-quality enhanced images and accurate structured annotations in a single process, providing real-time and integrated data support for assisted diagnosis.

[0029] An active learning optimization module is deployed on the clinical doctor's workstation where radiologists perform daily image interpretation and annotation. The workstation is a computer terminal equipped with a professional medical monitor and integrated into the operation interface of the hospital's image archiving and communication system or radiology information system. The active learning optimization module receives automatic annotation results, quantifies their uncertainty by calculating entropy and prediction variance, and automatically selects the images and regions that most need to be reviewed based on this uncertainty and pushes them to doctors for manual fine annotation. The high-quality annotation data confirmed and corrected by the doctors is then used as new samples and fed back into the training process to continuously optimize model performance. Furthermore, the active learning optimization module also includes an uncertainty quantification and screening unit, a human-computer interaction and data management unit, and an incremental learning triggering unit; The uncertainty quantification and filtering unit is used to process the received automatic annotation results. Specifically, it calculates the entropy of the probability distribution by performing a dot product operation between the negative of the class probability vector and its own logarithmic vector on each pixel output by the annotation network, summing the results, and performing multiple random dropout inferences to calculate the prediction variance. Random dropout inference is activated during the forward propagation of the annotation network. A specified proportion of the output of the convolutional layer in its decoding branch is randomly set to zero. This process is repeated a specified number of times to obtain a set of predictions. Then, the variance matrix of this set of predictions in the pixel class or bounding box coordinates is calculated to generate a quantified uncertainty metric. Based on this metric, image regions exceeding a preset threshold and complete cases are automatically filtered out. The threshold can be dynamically adjusted based on historical annotation experience or the current performance of the model. During the filtering process, regional connectivity analysis is combined to gather high uncertainty pixels into candidate regions and organize and generate a queue of tasks to be manually annotated, which includes priority ranking. The ranking algorithm comprehensively considers the magnitude of the uncertainty metric value, the area of ​​the region, and the historical annotation status of the case. The human-computer interaction and data management unit is used to display the target images in the task queue, the corresponding initial automatic annotation results, and the highlighted uncertain areas to the doctor. The display interface adopts a side-by-side or layer overlay view. The highlighted areas are usually covered with a semi-transparent warm color on the image. It provides graphical drawing, erasing, and correction tools to receive and save high-quality finely annotated data generated by the doctor's interaction. The tools include variable-size circular or square brushes, intelligent edge-out polygon tools, region filling tools, and bounding box drag adjustment handles. All interactive operations are responded to in real time and are visible. The data and its corresponding original images are encapsulated into new training samples. The encapsulation format is a pair of original image files and annotation mask image files. At the same time, metadata files containing case identifiers, annotation time, correction areas, and annotator identities are generated. The incremental learning triggering unit is used to send the encapsulated new training samples to the data preparation and federated training module. The sending process is carried out through a secure internal network or virtual private network, using an asynchronous message queue or file transfer protocol to trigger the incremental learning process of the generative adversarial network model and the labeled network. The triggering condition can be that the number of new samples accumulates to a set batch, or it can be started at a fixed time period. The incremental learning process mixes the new samples with historical data and fine-tunes the model in a limited number of rounds with a small learning rate.

[0030] Specifically, the active learning optimization module constructs an efficient human-machine collaborative optimization closed loop, effectively solving three core challenges faced by automatic annotation systems in clinical application: model confidence assessment, expert effort allocation, and continuous model evolution. Through uncertainty quantification and screening units, using indicators such as entropy and prediction variance, it scientifically and automatically identifies high-risk areas and cases with ambiguous boundaries, complex morphologies, or insufficient model understanding in the automatic annotation results, changing the blindness and inefficiency of traditional manual review. The human-machine interaction and data management unit accurately pushes the selected targets to doctors and guides experts to efficiently review and accurately annotate high-uncertainty areas through graphical tools integrated into their daily workflows. This allows radiologists to focus their expertise and time on the aspects where the machine needs the most help, greatly improving the value and efficiency of human feedback. Finally, the incremental learning triggering unit automatically completes the encapsulation and feedback of high-quality new samples, triggering targeted fine-tuning of the model. This mechanism transforms expert correction behavior into the driving force for continuous system learning, forming a self-reinforcing cycle of model annotation-expert correction-model optimization, achieving a steady and autonomous improvement in the system's annotation accuracy and robustness in real clinical use.

[0031] The evaluation and visualization output module is also deployed on the clinician workstation, which is equipped with a medical-grade grayscale or color display that supports high dynamic range display and has passed the medical digital imaging and communication standard consistency test. The evaluation and visualization output module is used to perform a comprehensive evaluation of the images and annotation results generated by the system by combining automated index calculation with subjective evaluation by radiologists and professionals. The final evaluation results are provided to doctors in the form of image and annotation information overlay rendering, three-dimensional visualization and interactive operation interface for auxiliary diagnosis and analysis. Furthermore, the assessment and visualization output module also includes an automatic assessment engine, a clinical staff scoring management unit, and a fusion visualization and report generation unit; The automatic evaluation engine is used to automatically calculate the peak signal-to-noise ratio and structural similarity index of the images generated by the system. Before the calculation, the generated image is rigidly registered with the reference real image to ensure complete spatial alignment. Then, the calculation is performed on the entire image or the specified region of interest, and the annotation results are used to call the preset downstream diagnostic model. The downstream diagnostic model is built on a fifty-layer residual neural network. The downstream diagnostic model is pre-trained on a large-scale, high-quality labeled independent dataset, and the weights are frozen as fixed feature extractors and classifiers to quantify the improvement of its performance indicators for classification and segmentation tasks. The specific process is to use the original image and the system-enhanced image as inputs to the downstream diagnostic model, record and compare the various performance indicators it obtains on the same test set, and calculate the relative improvement percentage. The clinical staff scoring management unit is used to organize and manage the double-blind review process. The clinical staff scoring management unit provides an independent online review portal. Review experts log in through their personal encrypted accounts and distribute the images and annotation results to be evaluated to the radiologists and professionals participating in the review. The distribution process is randomized, and the same expert will not review different versions of the same case repeatedly. It also ensures that the radiologists and professionals participating in the review do not know whether the images are real or synthetic, or whether the annotations are automatic or manual. The system collects and summarizes their subjective quality scores based on clinical diagnostic value. The scoring uses a five- or seven-point Likert scale, and the dimensions cover diagnostic confidence, image clarity, noise and artifact levels, and annotation accuracy. The system automatically calculates the mean score and standard deviation of each dimension. The integrated visualization and report generation unit is used to spatially register and fused render the final evaluated images with corresponding annotation information. Registration ensures that the annotation contours and image anatomical structures are aligned at the sub-pixel level. Rendering uses anti-aliasing technology to smooth edges and displays the images using layer transparency overlay, pseudo-color highlighting, 3D volume rendering, and multi-plane reconstruction. Multi-plane reconstruction supports simultaneous browsing and annotation projection of axial, coronal, and sagittal planes. It also provides doctors with window width and level adjustment, annotation contour editing, and quantitative measurement tools. Window width and level adjustment provides preset quick options for organ or tissue types. Contour editing allows doctors to fine-tune automatically generated boundaries by dragging control points. Quantitative measurement tools include distance, angle, area, and volume measurements. Measurement results are displayed in real time and linked to the report. Based on user operations, it automatically generates a structured diagnostic report containing key images and quantitative analysis results. The report template conforms to clinical standards and automatically embeds image thumbnails of key slices, measurement values, annotation diagrams, and quality indicator summaries obtained from the evaluation engine. It supports exporting to a portable document format or direct uploading to the hospital information system.

[0032] Specifically, the evaluation and visualization output module establishes a multi-dimensional, verifiable, and clinically relevant delivery terminal, effectively addressing key issues such as the lack of authoritative evaluation standards for intelligent system outputs, their disconnect from actual clinical needs, and their difficulty in seamlessly integrating into diagnostic workflows. It constructs a comprehensive evaluation system combining objective quantification and subjective evaluation. The automatic evaluation engine not only calculates traditional image quality indicators but also directly verifies and quantifies the performance improvement of enhanced images and annotation results on actual diagnostic tasks by calling independent downstream diagnostic models, thus closely linking technical indicators with ultimate clinical value. Simultaneously, a clinical staff scoring management unit... Through a rigorous double-blind review process, subjective scores based on diagnostic confidence are collected from the perspective of real users, ensuring the usability and reliability of the system's output under the professional judgment of physicians. Finally, all evaluation results are transformed into an interactive image reading environment familiar to physicians through the integrated visualization and report generation unit. This environment provides a complete set of tools, from multi-plane browsing and window width and level adjustment to annotation editing and quantitative measurement, and can automatically generate structured diagnostic reports. This directly transforms the processing results of artificial intelligence into auxiliary diagnostic materials that conform to clinical habits, are operable, and can be archived, truly realizing an efficient and reliable transformation from algorithm output to clinical practice.

[0033] This embodiment illustrates a modular, distributed system architecture for implementing the aforementioned method; the system, through the collaboration of four core functional modules, transforms the method process into specific deployable technical components and ensures end-to-end implementation from data security collaborative processing to clinical outcome delivery; The system constructs a data preparation and federated training architecture consisting of local servers distributed across various medical institutions and a central coordination server. This architecture not only achieves privacy protection by ensuring that the original data does not leave the local area, but also standardizes and automates the federated learning training process through modular secure communication and aggregation units, providing a compliant and high-performance model foundation for the entire system. The image generation and intelligent annotation module, as the core computing service, achieves efficient unification and parallel execution of augmentation processing and automatic annotation tasks at the service level through the integrated design of model loading interface, anatomical constraint unit and parallel annotation unit, ensuring the real-time performance and anatomical rationality of the processing results. The active learning optimization module is deeply integrated into the doctor's diagnostic workstation. Through the embedded uncertainty quantification, human-computer interaction and incremental learning triggering units, it establishes a seamless, data-driven model optimization closed loop in the doctor's daily workflow, enabling the system to continuously and autonomously evolve during use. The assessment and visualization output module serves as the final interface between the system and clinical diagnostic work. It not only provides automated tools for comprehensive objective and subjective evaluation, but more importantly, by integrating visualization and structured report generation functions, it directly transforms the output results of artificial intelligence into clinically standardized and interactive diagnostic auxiliary materials.

[0034] Example 3 This embodiment provides a specific application scenario, detailing how the automatic medical image annotation and enhancement method and system based on generative adversarial networks described in this invention can be deployed and run in a real clinical environment to solve specific clinical problems.

[0035] Application scenario: Precise segmentation and enhanced diagnosis of multicenter gliomas; Gliomas are common intracranial malignant tumors, and their accurate segmentation is crucial for surgical planning, efficacy evaluation, and prognosis. However, gliomas are morphologically variable and have indistinct boundaries, and multi-center studies are limited by patient privacy, data heterogeneity, and a lack of labeled resources. A regional medical alliance plans to collaborate with the neuroimaging centers of five of its affiliated tertiary hospitals to jointly build an intelligent auxiliary diagnostic platform for gliomas, requiring: 1. Data from each hospital to remain within the hospital premises, complying with the Personal Information Protection Law and the Medical Data Security Management Measures; 2. The ability to collaboratively train high-performance automatic segmentation and image enhancement models using multi-center data; 3. The ability to provide doctors with high-quality enhanced images (such as denoising and resolution enhancement) and high-precision automatic tumor segmentation results; 4. The ability for the system to continuously learn from doctor feedback during use and continuously improve its performance in rare or complex cases.

[0036] Based on the system described in this invention, a local server (configured with a high-end GPU) is deployed within each of the five hospitals to store and process brain MRI (T1, T1c, T2, FLAIR sequences) data. Each hospital's local server runs a local model training unit and a secure communication unit. The consortium commissioned a third-party independent technical service provider to deploy an external central server in a cloud environment that meets Level 3 cybersecurity standards, running a model coordination unit and a secure aggregation unit; deployed on the high-performance computing cluster of the aforementioned third-party cloud platform, providing containerized model inference services; integrated into the existing PACS / RIS systems of the neuroradiologists' workstations in each hospital as a plug-in; and also integrated into the doctor's workstations, deeply integrated with PACS Viewer.

[0037] Based on the method described in this invention, each hospital's local server automatically obtains the desensitized (all patient identification information removed) MRI image dataset of gliomas from the hospital's PACS and performs standardized preprocessing (spatial registration to the MNI152 standard brain template, intensity normalization to the [0,1] interval). The central server initializes a GAN model that incorporates anatomical priors (the generator is a variant of U-Net, and the discriminator is CNN). The generator's loss function is pre-set with an anatomical constraint loss calculated based on the BrainNetCNN pre-trained segmentation network and deformable registration network, aiming to ensure that the generated brain images maintain normal anatomical structures. The central server distributes the initial model parameters to the local servers of five hospitals. Each hospital trains its local model using local data (e.g., 500 cases from Hospital A, 300 cases from Hospital B, etc.) and calculates the gradient. The secure communication unit encrypts the gradient using the Paillier homomorphic encryption algorithm and adds Gaussian noise that conforms to (ε, δ)-differential privacy. The encrypted and noisy gradient is securely transmitted to the central server. The secure aggregation unit performs a weighted average of the gradient in the encrypted domain (the weights are proportional to the amount of data from each hospital), decrypts it to obtain the aggregated gradient, and updates the global model. After 100 iterations, the model achieves a Dice coefficient of 0.88 for tumor segmentation on the independent validation set (from another hospital that did not participate in the training), and the PSNR of the generated image is greater than 32dB, indicating that the model has converged. During the training process, the original data from each hospital never leaves the hospital.

[0038] The doctor opens a FLAIR sequence image of a newly admitted patient suspected of having a glioma at the workstation and finds the image noisy and the tumor boundaries unclear. The doctor uploads the case image to the cloud image generation and intelligent annotation module through the PACS plugin interface. The model loading and service interface unit of this module calls the pre-trained global model. The anatomical structure constraint generation unit drives the generator to denoise and super-resolution enhance the input image, while ensuring that the generated key structures such as the ventricles and corpus callosum have reasonable morphology. The lightweight parallel annotation unit runs synchronously, using shared features to output the glioma segmentation probability map and the bounding boxes of sub-regions such as the tumor core and edema zone. Within seconds, the processing results (enhanced FLAIR image and tumor segmentation mask) are returned to the doctor's workstation interface.

[0039] The uncertainty quantification and screening unit of the active learning optimization module automatically analyzes the automatic segmentation results. It calculates that the segmentation boundary entropy is high and performs five Monte Carlo Dropout inferences on the decoding layer, resulting in a large predicted variance for the tumor region, indicating that the model is uncertain about the tumor boundary for this case. This case is automatically added to the doctor's workstation's list of tasks requiring fine annotation, and areas with high uncertainty are highlighted. During senior physician review, the system prompts that this case requires close attention. The physician uses the smart brush and polygon tools provided by the plugin to correct and confirm the automatically segmented boundary by comparing the original and enhanced images, taking approximately 3 minutes (far less than the 20 minutes required for completely manual segmentation). The human-computer interaction and data management unit encapsulates the physician's corrected fine-annotation mask with the original image into a new sample. This week, 50 similar high-value fine-annotation samples were generated from 5 hospitals. The incremental learning trigger unit securely sends these samples to the central server over the weekend. The central server initiates a round of incremental learning, using these 50 new samples and some historical data to fine-tune the global model. After fine-tuning, the model's segmentation Dice coefficient for similar cases with blurred boundaries improves by 5%.

[0040] For a batch of generated images used for research evaluation, the automatic evaluation engine of the evaluation and visualization output module automatically calculates their SSIM values ​​and calls an independent, published glioma survival prediction model (based on ResNet-50). It was found that using the system-enhanced images as input improved the prediction model's C-index by 0.04. Simultaneously, the module organized a double-blind review by six senior radiologists from three hospitals. Doctors, unaware of the difference, scored their diagnostic confidence (1-5 points) on the mixed real and system-generated images. Statistics showed that the average score for the system-generated images was 4.2, with no statistically significant difference from the real images. For routine diagnostic cases, doctors view the enhanced images and the overlaid tumor segmentation contours in the fusion visualization interface. They can optimize the display using window width and level adjustments, fine-tune the automatic segmentation using the contour editing tool, and calculate the tumor volume with a single click using the volume measurement tool. After diagnosis, the system automatically generates a structured report, including key layer image screenshots, a tumor segmentation diagram, volume measurements, and image quality indices (PSNR, SSIM). The report can be directly imported into the hospital's electronic medical record system.

[0041] Under the premise of absolutely protecting the data sovereignty of each hospital and the privacy of patients, a cross-institutional collaborative intelligent model has been established, whose performance is significantly better than any single-center trained model. Doctors obtain higher quality images and more accurate automatic segmentation drafts, reducing the annotation workload by an average of 70% and shortening the time for issuing diagnostic reports by 30%. Through active learning loop, the system is continuously optimized for weaknesses in real-world use, especially in handling glioma subtypes with unclear boundaries and rare morphologies, where accuracy continues to improve. An evaluation system that combines objective and subjective approaches and is closely linked to clinical efficacy has been established, and the output results can be directly and reliably used for auxiliary diagnosis, surgical planning, and clinical research.

[0042] This embodiment demonstrates that the method and system described in this invention can effectively solve a series of problems such as data privacy, annotation bottlenecks, model generalization, and clinical integration in specific and complex clinical scenarios (precision diagnosis of multicenter gliomas), and achieve safe, efficient, and continuously evolving intelligent analysis of medical images.

[0043] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for automatic annotation and enhancement of medical images based on generative adversarial networks, characterized in that, Includes the following steps: Step 1: Data preparation and federated model training. Obtain medical image datasets from multiple data sources, perform anonymization and standardization preprocessing on the datasets, and train a generative adversarial network model that integrates anatomical prior knowledge using a federated learning paradigm. Step 2: Image generation and automatic annotation. Using the trained generative adversarial network model, the input medical images are enhanced and synthetic images are generated. At the same time, the annotation network trained in conjunction with the generator is activated to perform automatic segmentation and target detection on the input and synthetic images, generating structured annotation information. Step 3: Active learning optimization. Based on the automatic annotation results output by the annotation network, calculate its uncertainty measure, filter out samples with uncertainty higher than a preset threshold, submit them to radiologists and professionals for manual annotation, and add the manually annotated new samples to the training dataset to update the generative adversarial network model and the annotation network. Step 4: Results evaluation and output. Perform multi-dimensional quantitative and qualitative evaluation based on clinical task efficacy on the enhanced and generated images and automatically labeled results. Visualize and output the final results that meet the preset quality standards.

2. The method according to claim 1, characterized in that, In step one, data preprocessing specifically involves anonymizing and de-identifying medical images, resampling spatial orientations to a standard template, and normalizing pixel intensity values ​​to a unified range. Federated model training specifically involves the following process: each participating node trains the generative adversarial network model using local data, calculates the update gradient of the model parameters, protects the gradient using partially homomorphic encryption that supports addition and multiplication operations, and adds differential privacy noise. The protected gradient is then uploaded to the central server, which securely aggregates the converged gradients to update the global model parameters and distributes the updated parameters to each participating node. This process iterates until the model converges.

3. The method according to claim 2, characterized in that, During the training process of the federated model, after calculating the gradient update of the model parameters and before encryption and noise protection, a gradient pruning step is also included: The norm of the updated gradient is pruned to ensure it does not exceed a preset pruning threshold. During the secure aggregation process, the central server dynamically allocates aggregation weights to the protected updates from each node based on the effective data volume or historical contribution of each participating node in this round of training.

4. The method according to claim 1, characterized in that, In step two, the generator of the generative adversarial network model receives the input image and outputs the processed image. The optimization objective of the generator includes an anatomical structure constraint loss term based on deformable registration. The deformable registration is performed by an independent, encoder-decoder-based convolutional neural network that takes the generator's output image and a standard anatomical atlas as input and outputs a dense displacement field. The calculation process of the anatomical structure constraint loss includes: using the deformable registration network, registering the generator's output image with a standard anatomical atlas to obtain a corresponding dense displacement field; applying this dense displacement field to the key anatomical structure segmentation map generated by the pre-trained segmentation network from the real image or standard atlas, and obtaining a deformed reference segmentation map through spatial transformation; finally, inputting the generator's output image into the same pre-trained segmentation network to obtain its predicted segmentation map, and obtaining the anatomical structure constraint loss by calculating the consistency difference between the predicted segmentation map and the reference segmentation map. The annotation network shares some initial convolutional layers with the generator, receives intermediate feature maps extracted by the generator, and upsamples them to the original resolution through a transposed convolutional layer to output pixel-level segmentation masks and target bounding boxes. The generator employs an encoder and decoder structure. The encoder contains consecutive downsampled convolutional blocks for extracting multi-scale features, and the decoder contains upsampled transposed convolutional blocks symmetrical to the encoder and skip connections for reconstructing high-resolution images.

5. The method according to claim 1, characterized in that, In step three, the calculation of the uncertainty measure includes: The pixel-level semantic category probability distribution output by the annotation network after inference on the input image is obtained as the prediction result; Based on the prediction results, a first uncertainty measure is obtained by calculating the entropy of its probability distribution. The entropy is calculated by performing the negative operation of the dot product of the class probability vector at each pixel with its corresponding probability vector and summing the results. Meanwhile, multiple random dropout inferences are performed in the convolutional layer of the decoding branch of the labeled network to obtain multiple sets of prediction results. The second uncertainty measure is obtained by calculating the variance between these prediction results. The screening process selects several samples with the highest uncertainty values ​​based on the first and second uncertainty measures to form a set to be labeled; radiologists and professionals use the drawing tools provided by the graphical interface to manually label the samples in the set, and the corrected labeled data is stored as high-quality labels; the model update process uses an expanded training set containing the new labeled samples to jointly fine-tune the parameters of the generative adversarial network model and the labeling network.

6. The method according to claim 5, characterized in that, During the screening process, samples are selected based on a comprehensive uncertainty metric. The comprehensive uncertainty measure is the result of linearly weighted summation of the first uncertainty measure and the second uncertainty measure according to preset weights; During the screening process, while prioritizing the selection of samples with the highest comprehensive uncertainty metric, the similarity between samples in the feature space is also calculated to remove duplicates of overly similar samples, thus ensuring the case diversity of the set to be labeled.

7. The method according to claim 1, characterized in that, Step four, the multi-dimensional evaluation specifically includes: Calculate the peak signal-to-noise ratio and structural similarity index between the generated image and the real image; The generated and enhanced images are input into an independent downstream diagnostic model, which is built on a fifty-layer residual neural network. The performance improvement of the downstream diagnostic model in classification and segmentation tasks is evaluated by comparing the performance indicators obtained by the downstream diagnostic model on the images before and after processing. Radiologists and professionals were instructed to subjectively rate the quality of the enhanced, generated images and their corresponding automatic annotations without the radiologists' knowledge. The visualization output displays the evaluated images and corresponding annotation information in an overlay manner, and provides interactive functions such as window width and level adjustment, outline drawing, and modification.

8. A medical image automatic annotation and enhancement system based on generative adversarial networks for implementing the method of any one of claims 1 to 7, characterized in that, include: The data preparation and federated training module is distributed and deployed in local servers within each medical institution to protect the raw data, as well as an external central server that runs independently to coordinate model training tasks. The data preparation and federated training module is used to obtain medical image data from each institution, perform anonymization and standardization preprocessing on each local server, and, under the coordination of the central server, securely aggregate the computational results of all parties using a federated learning approach to jointly train a generative adversarial network model that incorporates anatomical prior knowledge constraints. The image generation and intelligent annotation module is deployed on an external central server where the model training is completed. To improve the elasticity, scalability and reliability of the service, the image generation and intelligent annotation module can also be deployed on a cloud-based high-performance computing platform interconnected with the central server. The image generation and intelligent annotation module is used to load and run the aforementioned trained generative adversarial network model, to perform quality enhancement, modality conversion and pathological feature synthesis on the received medical images, and to automatically generate corresponding lesion segmentation maps and target detection boxes using the built-in annotation network that works in collaboration with the adversarial network model generator. The active learning optimization module is deployed on the clinical doctor's workstation where radiologists perform daily image interpretation and annotation. The active learning optimization module is used to receive automatic annotation results, quantify their uncertainty by calculating entropy and prediction variance, and automatically select the images and regions that most need to be reviewed based on this uncertainty and push them to the doctor for manual fine annotation. Then, the high-quality annotation data confirmed and corrected by the doctor is used as new samples and fed back to the training process to continuously optimize the model performance. The assessment and visualization output module is also deployed on the clinician workstation. This module is used to perform a comprehensive assessment of the images and annotation results generated by the system, combining automated index calculation with subjective evaluation by radiologists and professionals. The final results that pass the assessment are provided to doctors in the form of image and annotation information overlay rendering, 3D visualization, and interactive operation interface for auxiliary diagnosis and analysis.

9. The system according to claim 8, characterized in that, The data preparation and federated training module also includes a local model training unit and a secure communication unit deployed on each of the local servers, and a model coordination unit and a secure aggregation unit deployed on the external central server. The local model training unit is used to perform forward and backward propagation calculations on the received generative adversarial network model using local data that has been anonymized and normalized before processing, and to generate local updates of the model parameters. The secure communication unit is used to process the local update using homomorphic encryption and differential privacy noise, generate a protected update, and transmit it to an external central server through a secure channel. The model coordination unit is used to distribute global generative adversarial network model parameters during initialization and iteration to each local server, and to schedule training rounds. The secure aggregation unit is used to perform secure aggregation operations on the protected updates uploaded by each local server, calculate the next generation of global model parameters, and thus complete one round of federated training iteration. The image generation and intelligent annotation module also includes a model loading and service interface unit, an anatomical structure constraint generation unit, and a lightweight parallel annotation unit; The model loading and service interface unit is used to load the parameters of the trained generative adversarial network model from storage into the memory of the computing device, and to provide a standardized application programming interface and network service interface to receive input image data and return processing results. The anatomical structure constraint generation unit is integrated into the generator of the generative adversarial network model; The generator employs an encoder-decoder structure. The encoder extracts multi-scale features of the input image through continuously downsampled convolutional layers, while the decoder reconstructs a high-resolution output image through upsampled transposed convolutional layers containing skip connections. The anatomical constraint generation unit introduces anatomical constraint loss based on deformable registration during the training and inference process of the generator to ensure the anatomical rationality of the output image. The lightweight parallel annotation unit includes an annotation network that shares the underlying convolutional layer with the generator encoder; The annotation network receives the intermediate feature map extracted by the generator encoder, performs upsampling and feature fusion through an independent decoding branch, and outputs pixel-level semantic segmentation probability maps and object detection bounding box coordinates in parallel.

10. The system according to claim 8, characterized in that, The active learning optimization module also includes an uncertainty quantification and screening unit, a human-computer interaction and data management unit, and an incremental learning triggering unit; The uncertainty quantification and screening unit is used to process the received automatic annotation results, calculate the prediction variance by calculating the entropy of its probability distribution and performing multiple random discard inferences, generate a quantified uncertainty metric, and automatically screen out image regions and complete cases that exceed a preset threshold based on the metric, and organize and generate a queue of tasks to be manually annotated with priority sorting. The human-computer interaction and data management unit is used to show doctors the target images in the task queue, the corresponding initial automatic annotation results, and the highlighted uncertain areas. It provides graphical drawing, erasing, and correction tools to receive and save high-quality finely annotated data generated by doctor interaction, and encapsulates the data and its corresponding original images into new training samples. The incremental learning triggering unit is used to send the encapsulated new training samples to the data preparation and federated training module to trigger the incremental learning process of the generative adversarial network model and the annotation network. The assessment and visualization output module also includes an automatic assessment engine, a clinical staff scoring management unit, and a fusion visualization and report generation unit; The automatic evaluation engine is used to automatically calculate the peak signal-to-noise ratio and structural similarity index of the images generated by the system, and to call a preset downstream diagnostic model for the annotation results. The downstream diagnostic model is built on a fifty-layer residual neural network and quantifies its improvement on the performance indicators of classification and segmentation tasks. The clinical staff scoring management unit is used to organize and manage the double-blind review process, distribute the images and annotation results to be evaluated to the radiologists and professionals participating in the review, and ensure that the radiologists and professionals participating in the review do not know whether the images are real or synthetic, or whether the annotations are automatic or manual, and collect and summarize their subjective quality scores based on clinical diagnostic value. The fusion visualization and report generation unit is used to spatially register and fusion render the final image that has passed the evaluation with the corresponding annotation information, and display it in the form of layer transparency overlay, pseudo-color highlighting, three-dimensional volume drawing and multi-plane reconstruction. At the same time, it provides doctors with window width and window level adjustment, annotation contour editing, and quantitative measurement tools, and automatically generates a structured diagnostic report containing key images and quantitative analysis results based on user operations.

Images