Endoscope image data enhancement method and system based on adversarial network

Abstract

The invention provides an endoscopic image data enhancement method and system based on an adversarial network, and the method comprises the steps: obtaining a multi-domain medical data set, importing the multi-domain medical data set into a medical feature extraction model, obtaining a sample feature vector, obtaining a high-value rare sample based on the sample feature vector, and carrying out the feature space enhancement, the method comprises the steps of packaging conventional samples and high-value rare samples into a training data set, constructing and training an anatomical domain encoder and a pathological instance encoder based on the training data set, constructing a meta-domain image enhancement model, obtaining meta-tasks by adopting the anatomical domain encoder and the pathological instance encoder based on the training data set, and carrying out image enhancement on the meta-tasks. And training a meta-domain image enhancement model based on the meta-task, obtaining a new-domain medical data set, and obtaining a new-domain synthetic data set by adopting an anatomical domain encoder, a pathological instance encoder and the meta-domain image enhancement model. And an endoscope image data enhancement method of a new domain style is efficiently adapted.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to a method and system for enhancing endoscopic image data based on adversarial networks. Background Technology

[0002] Acquisition and annotation of endoscopic image data is a crucial foundational step in the field of medical image analysis. However, constructing high-quality, large-scale datasets faces significant technical challenges. First, endoscopic image data inherently suffers from severe class imbalance. During clinical acquisition, the number of normal or common lesion samples is enormous, while the number of rare case images with high medical research value is relatively small. This imbalance in data distribution makes it difficult for models trained on such data to fully learn and represent rare patterns. Traditional data augmentation methods only perform low-level expansion in the image space and cannot create samples with new semantic features, thus failing to improve the data distribution imbalance problem.

[0003] On the other hand, medical imaging data has significant multi-domain differences. Endoscopic images from different sources have significant differences in color style, texture and resolution. When a dataset built on one data source is directly applied to another data source, its effectiveness will be significantly reduced due to inter-domain differences. Existing technologies usually require collecting and manually labeling a large amount of data in the new domain for adaptation, which is costly, inefficient and difficult to implement on a large scale.

[0004] In summary, existing endoscopic image data augmentation techniques are limited in their ability to construct high-quality, highly generalizable datasets due to their inability to address data distribution imbalances and adapt to differences across multiple domains. There is an urgent need in this field for a data augmentation method that can improve data distribution and efficiently adapt to new domain styles. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a method and system for enhancing endoscopic image data based on adversarial networks.

[0006] The first aspect of this invention provides a method for enhancing endoscopic image data based on adversarial networks, comprising: Obtain a multi-domain medical dataset and import it into a medical feature extraction model to obtain sample feature vectors. Based on the sample feature vector, the isolated forest model is used to obtain regular samples and candidate anomaly samples. The candidate anomaly samples are imported into the rare validation model, and high-value rare samples are obtained. Feature space enhancement is performed on the high-value rare samples, and regular samples and high-value rare samples are encapsulated into a training dataset. Based on the training dataset, construct and train the anatomical domain encoder and the pathological instance encoder; A meta-domain image enhancement model is constructed. Based on the training dataset, the task meta-loss is obtained by using an anatomical domain encoder and a pathological instance encoder. The meta-domain image enhancement model is then trained based on the task meta-loss. Obtain a new domain medical dataset. Based on the new domain medical dataset, use an anatomical domain encoder to obtain the new domain anatomical domain encoding and use a pathological instance encoder to obtain the target instance encoding. Import the new domain anatomical domain encoding and the target instance encoding into the meta-domain image enhancement model to obtain the new domain synthetic dataset.

[0007] According to a preferred embodiment, a multi-domain medical dataset is obtained, and the multi-domain medical dataset is imported into a medical feature extraction model to obtain sample feature vectors, including: Acquire multi-domain endoscopic images and their corresponding image annotation information, and perform standardization processing on the multi-domain endoscopic images and their corresponding image annotation information; The image annotation information includes at least bounding boxes, segmentation masks, and pathological category labels; Multi-domain endoscopic images and their corresponding image annotation information are encapsulated into a multi-domain medical dataset; The pre-trained feature extraction network is used as the medical feature extraction model; Multi-domain endoscopic images from a multi-domain medical dataset are imported into a medical feature extraction model to obtain the sample feature vector corresponding to each multi-domain endoscopic image.

[0008] According to a preferred embodiment, a regular sample and candidate anomaly sample are obtained using an isolated forest model based on sample feature vectors. The candidate anomaly sample is then imported into a rare validation model to obtain high-value rare samples. Feature space augmentation is performed on the high-value rare samples. The regular sample and the high-value rare sample are then encapsulated into a training dataset, including: The sample feature vector is input into the isolated forest model to obtain the anomaly score corresponding to the sample feature vector, and then the normal samples and candidate anomaly samples are obtained based on the anomaly score; The sample feature vector of the candidate abnormal sample is input into the rare verification model to obtain the rareness confidence score, and corresponding sample labels are generated for the candidate abnormal sample according to the preset rareness threshold. The rare validation model includes a reconstructed quality validation layer, a generated consistency validation layer, and a clinically relevant proxy validation layer. The sample labels include high-value rare samples, samples to be reviewed, and noisy samples; We employ latent spatial augmentation techniques to process high-value rare samples, thereby obtaining augmented high-value rare samples. The regular samples and enhanced high-value rare samples are packaged into a training dataset.

[0009] According to a preferred embodiment, the sample feature vector of the candidate anomaly sample is input into a rareness verification model to obtain a rareness confidence score, and corresponding sample labels are generated for the candidate anomaly sample based on a preset rareness threshold, including: Candidate anomaly samples are imported into the reconstruction quality verification layer, and a pre-trained reconstruction autoencoder is used to decode the anomaly sample to obtain the reconstructed image of the anomaly sample. The reconstructed lesion region is obtained from the image reconstructed based on the abnormal sample. The corresponding original lesion region is obtained from the multi-domain endoscopic image in the multi-domain medical dataset. The structural similarity index is obtained from the reconstructed lesion region and the original lesion region as the first verification score. The candidate anomaly samples are imported into the consistency verification layer. Based on the candidate anomaly samples, multiple verification sample feature vectors are generated by feature interpolation or adding noise. The verification sample feature vectors are then decoded into verification sample images. A pre-trained lesion morphology classifier is used to classify the verification sample images, and the statistical uncertainty of the classification results is obtained. A second verification score is generated based on the statistical uncertainty. Candidate abnormal samples are imported into the clinical relevance proxy verification layer. Morphological indicators of abnormal samples are obtained based on candidate abnormal samples, and the distribution of baseline morphological indicators is obtained based on regular samples. Obtain the Mahalanobis distance between the distributions of morphological indicators of abnormal samples and baseline morphological indicators, and generate a third verification score based on the Mahalanobis distance; Based on the first verification score, the second verification score, and the third verification score, a rareness confidence score is generated for the candidate abnormal sample.

[0010] According to a preferred embodiment, an anatomical domain encoder and a pathological instance encoder are constructed and trained based on a training dataset, including: The anatomical domain encoder is trained based on a training dataset using contrastive loss and adversarial loss. The anatomical domain encoder takes endoscopic images as input and outputs a low-dimensional vector representing the anatomical domain data, which serves as the anatomical domain feature vector. The pathological instance encoder is trained based on the training dataset using focus loss and correlation constraints. The pathological instance encoder takes the lesion area of ​​the endoscopic image as input and outputs a low-dimensional vector representing the pathological instance data, which serves as the feature vector of the pathological instance.

[0011] According to a preferred embodiment, a meta-domain image enhancement model is constructed. Based on a training dataset, a task-specific meta-loss is obtained using an anatomical domain encoder and a pathological instance encoder. The meta-domain image enhancement model is then trained based on the task-specific meta-loss, including: The training dataset is sampled according to the meta-learning task paradigm to obtain multiple meta-tasks, each of which contains a support set and a query set. For the current meta-task, the target domain encoding is obtained using a dissecting domain encoder based on the support set; Based on the query set and the corresponding target domain encoding, the image generation adversarial training process is executed to obtain the meta-image generation adversarial network adapted to the current meta-task, and the task meta-loss of the meta-image generation adversarial network on the query set is obtained. Obtain the meta-image generative adversarial networks adapted to all meta-tasks and their corresponding task meta-losses. Based on all task meta-losses, obtain the training meta-loss. Based on the training meta-loss, a meta-optimization algorithm is used to update the meta-parameters of the meta-domain image enhancement model, and the process is iteratively executed until the preset convergence criterion is met.

[0012] According to a preferred embodiment, based on the query set and the corresponding target domain encoding, an image generation adversarial training process is executed to obtain the meta-image generation adversarial network adapted to the current meta-task, and to obtain the task meta-loss of the meta-image generation adversarial network on the query set, including: The meta-image generative adversarial network includes a meta-image generator and a meta-image discriminator; Based on the query set, a pathological instance encoder is used to obtain the feature vector of the pathological instance; The target domain encoding and pathological instance feature vectors are used as conditions and input into the meta-image generator to generate an adversarial reconstruction image. The adversarial reconstructed image and the endoscopic image corresponding to the query set are input into the meta-image discriminator to obtain the adversarial loss; The reconstructed image loss is obtained based on the adversarial reconstructed image and the corresponding endoscopic image in the query set; The meta-image generator and meta-image discriminator are updated based on adversarial loss and reconstructed image loss, and used as a meta-image generative adversarial network adapted to the current meta-task.

[0013] According to a preferred embodiment, the parameters of the meta-domain image enhancement model are updated using a meta-optimization algorithm based on meta-loss, and the process is iteratively executed until a preset convergence criterion is met, including: For each meta-task, obtain the meta-loss gradient of the training meta-loss on the parameters of the meta-domain image augmentation model, and update the parameters of the meta-domain image augmentation model based on the meta-loss gradient. When the training meta-loss triggers any one of the threshold convergence condition, iterative convergence condition, or performance convergence condition in multiple consecutive iterations, it is determined that the training of the meta-domain image enhancement model has converged.

[0014] According to a preferred embodiment, a new domain medical dataset is obtained; based on the new domain medical dataset, an anatomical domain encoder is used to obtain a new domain anatomical domain code; a target instance code is obtained based on a pathological instance encoder; the new domain anatomical domain code and the target instance code are imported into a meta-domain image enhancement model to obtain a new domain synthetic dataset, including: Import the new domain endoscope images from the new domain medical dataset into the medical feature extraction model to obtain the new domain sample feature vector corresponding to each new domain endoscope image; The average new domain code is obtained by using an anatomical domain encoder based on the feature vector of the new domain sample. Obtain the expected pathological parameters, and use a pre-trained semantic mapping model to obtain the expected pathological code based on the expected pathological parameters; The average new domain code and the expected pathological code are imported as conditional data into the meta-image generator within the meta-domain image enhancement model to generate a new domain synthetic dataset.

[0015] A second aspect of the present invention also provides an endoscopic image data enhancement system based on adversarial networks, comprising: The data processing and enhancement module is used to acquire multi-domain medical datasets and import them into a medical feature extraction model. It acquires sample feature vectors, uses an isolated forest model based on the sample feature vectors to acquire regular samples and candidate abnormal samples, imports the candidate abnormal samples into a rare validation model to acquire high-value rare samples, performs feature space enhancement, and encapsulates regular samples and high-value rare samples into a training dataset. An encoder training module is used to construct and train an anatomical domain encoder and a pathological instance encoder based on the training dataset. Metadomain image enhancement model module, which is used to construct a metadomain image enhancement model, obtains task meta-loss using the anatomical encoder and the pathological instance encoder based on the training dataset, and trains the metadomain image enhancement model based on the task meta-loss. The new domain synthetic data module is used to acquire a new domain medical dataset, and uses the anatomical domain encoder to acquire the new domain anatomical domain code, and uses the pathological instance encoder to acquire the target instance code, and imports the new domain anatomical domain code and the target instance code into the meta-domain image enhancement model to acquire the new domain synthetic dataset.

[0016] Based on the above, this application embodiment obtains a multi-domain medical dataset and uses an isolated forest model and a rare validation model to screen out high-value rare samples and perform feature space enhancement to construct a balanced training dataset. By intelligently mining and enhancing rare samples, it overcomes the limitation of traditional enhancement methods in generating new semantic content, effectively solves the problem of data distribution imbalance in endoscopic data, and provides a more balanced data foundation for model training.

[0017] On the other hand, by constructing an anatomical domain encoder and a pathological instance encoder to decouple image features, and using a meta-learning framework to train a meta-domain image enhancement model, the model gains the ability to quickly adapt to the features of new domain data. This method only requires a small number of new domain samples to generate synthetic images that are highly adapted in anatomical style and clear in pathological features, reducing the dependence on large-scale annotation of new domain data and effectively overcoming the problem of poor data availability caused by domain offset. Attached Figure Description

[0018] Figure 1 The execution flowchart of the endoscopic image data augmentation method based on adversarial networks of the present invention is presented.

[0019] Figure 2 A flowchart of the meta-learning training paradigm in the endoscopic image data augmentation method based on adversarial networks of the present invention is presented.

[0020] Figure 3 The flowchart of the operation of the meta-domain image enhancement model in the endoscopic image data enhancement method based on adversarial networks of the present invention is presented.

[0021] Figure 4 A schematic diagram of the endoscopic image data enhancement system based on adversarial networks of the present invention is shown. Detailed Implementation

[0022] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0023] It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of this application can be combined with each other.

[0024] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0025] like Figure 1 , Figure 2 , Figure 3 , Figure 4 As shown: The first aspect of this invention provides a method for enhancing endoscopic image data based on adversarial networks, comprising: Step S1: Obtain a multi-domain medical dataset and import it into the medical feature extraction model to obtain sample feature vectors.

[0026] Step S1 includes: Step S11: Obtain multi-domain endoscopic images and corresponding image annotation information, and perform standardization processing on the multi-domain endoscopic images and corresponding image annotation information.

[0027] The image annotation information includes at least bounding boxes, segmentation masks, and pathology category labels.

[0028] Specifically, multi-domain endoscopic images refer to a collection of endoscopic images acquired from multiple different sources, covering different medical institutions, different models of endoscopic equipment, and different anatomical locations. This is to capture inter-domain differences that exist in clinical practice, such as differences in lighting conditions, resolution, and the diversity of lesion manifestations. The corresponding image annotation information can be obtained through professional medical image annotation databases and platforms. The image annotation information includes at least bounding boxes for locating lesion areas, segmentation masks for pixel-level segmentation of lesions, and pathological category labels for identifying pathological properties. Pathological category labels include normal, inflammation, polyps, etc. The pathological category label is determined by the specific lesion area. For example, assuming a colonoscopy, multi-domain endoscopic images may include colonoscopy images and corresponding image annotation information acquired from a type A colonoscopy system in hospital 1, colonoscopy images and corresponding image annotation information acquired from a type B colonoscopy system in hospital 2, and colonoscopy images and corresponding image annotation information from the public dataset Kvasir-SEG.

[0029] Furthermore, bilinear interpolation can be used to scale multi-domain endoscopic images to a fixed resolution, such as 640×480 pixels, to eliminate image scale differences caused by different devices. White balance algorithms can be used to correct the color distribution of multi-domain endoscopic images, making the color distribution of multi-domain endoscopic images more consistent and reducing the impact of light source differences. For example, for each multi-domain endoscopic image, the mean of the RGB channels is calculated and adjusted to the equalization value, so that the brightness range of the corrected image is standardized to [0, 255]. The annotation information is converted into a unified format. For example, the bounding box is stored as a coordinate list in COCO format, the segmentation mask is saved as a PNG binary image, and the pathological category label is mapped to an integer code, such as normal=0, inflammation=1, polyp=2. The standardized multi-domain endoscopic images and corresponding image annotation information are obtained, providing a consistent and standardized data foundation for subsequent steps.

[0030] In some possible embodiments, an endoscopic image from a type A colonoscopy system at Hospital 1, with a resolution of 1280×1024 pixels, is standardized to a resolution of 640×480 pixels. Color deviation is corrected to eliminate color cast caused by differences in the color temperature of the endoscopic light source, making the tissue color closer to the real representation. The corresponding polyp bounding box is converted from the original annotation to the COCO standard normalized coordinate format, such as [0.35, 0.42, 0.15, 0.12], indicating that the polyp bounding box is located at 35% of the image width and 42% of the image height, with a width of 15% of the image width and a height of 12% of the image height. The corresponding segmentation mask is saved as a single-channel PNG binary image, where the pixel value of the polyp region is 255 and the pixel value of the background region is 0.

[0031] Step S12: Encapsulate the multi-domain endoscopic images and their corresponding image annotation information into a multi-domain medical dataset.

[0032] Specifically, the standardized multi-domain endoscopic images and image annotation information are organized into a structured dataset according to a one-to-one mapping relationship, and this dataset serves as a multi-domain medical dataset. The dataset can be stored in standard formats such as JSON or HDF5. Each data entry in the multi-domain medical dataset should include the storage path of the multi-domain endoscopic image, the corresponding image annotation information, and the domain identifier. The domain identifier includes at least the device ID or hospital code. The domain identifier is used to provide clear domain information for the subsequent training process of the anatomical domain encoder and the meta-domain image enhancement model.

[0033] Step S13: Use the pre-trained feature extraction network as the medical feature extraction model.

[0034] Specifically, medical feature extraction models can employ convolutional neural networks pre-trained on large general computer vision datasets such as ImageNet, including ResNet-50 and EfficientNet-B4. Through pre-training, these models possess the ability to recognize general visual features such as edges, textures, and shapes. Utilizing such pre-trained convolutional neural networks as medical feature extraction models and performing transfer learning can improve the initialization performance of medical feature extraction models on medical image tasks, as well as their generalization performance on limited endoscopic images, and alleviate the overfitting problem that may be caused by limited image annotation information.

[0035] Furthermore, the medical feature extraction model is fine-tuned based on professional endoscopic datasets such as Kvasir and HyperKvasir. A small learning rate, such as 1e-4, is used during the fine-tuning process. The medical feature extraction model is then trained on the endoscopic dataset to better adapt its weight parameters to the feature distribution of endoscopic images. For example, a ResNet-50 model pre-trained on ImageNet is fine-tuned using the Kvasir dataset. The Kvasir dataset is a medical image segmentation dataset containing polyp images and corresponding pixel-level annotations, commonly used in research on automatic segmentation of colorectal polyps. After fine-tuning with the Kvasir dataset, the model's weight parameters are adjusted, enabling the medical feature extraction model to have a higher responsiveness to medical features such as the structure and texture of lesions, thereby extracting more targeted sample feature vectors.

[0036] Step S14: Import the multi-domain endoscopic images from the multi-domain medical dataset into the medical feature extraction model to obtain the sample feature vector corresponding to each multi-domain endoscopic image.

[0037] Specifically, multi-domain endoscopic images are input into a pre-trained and fine-tuned medical feature extraction model for forward propagation. High-level semantic features can be extracted from the last hidden layer or global pooling layer of the medical feature extraction model. These high-level semantic features are represented as a low-dimensional dense vector, i.e., a sample feature vector. The sample feature vector is specifically an encoding of the multi-domain endoscopic image. For example, for a medical feature extraction model based on ResNet-50, inputting a multi-domain endoscopic image yields a 2048-dimensional sample feature vector. This sample feature vector represents the deep features in the multi-domain endoscopic image. For instance, in a gastroscopy image labeled as a polyp, the sample feature vector extracted by the medical feature extraction model will highlight the medical features corresponding to polyps. In a gastroscopy image labeled as normal, the corresponding sample feature vector will reflect the medical features corresponding to normal. By calculating the distance between these high-dimensional feature vectors using methods such as cosine similarity, the similarity of different multi-domain endoscopic images in the feature space can be quantified.

[0038] Step S2: Based on the sample feature vectors, use the isolated forest model to obtain regular samples and candidate abnormal samples. Import the candidate abnormal samples into the rare validation model and obtain high-value rare samples. Perform feature space enhancement on the high-value rare samples and encapsulate the regular samples and high-value rare samples into a training dataset.

[0039] Step S2 includes: Step S21: Input the sample feature vector into the isolated forest model, obtain the anomaly score corresponding to the sample feature vector, and obtain regular samples and candidate anomaly samples based on the anomaly score; Specifically, the sample feature vector is input into the Isolation Forest model. The Isolation Forest model constructs multiple isolated trees by randomly selecting the dimension of the sample feature vector and a split value in that dimension, thereby calculating the anomaly score of each sample feature vector. The anomaly score reflects the degree to which the sample deviates from the normal distribution. The score ranges from 0 to 1. The closer the score is to 1, the more likely the sample is to be an anomaly. Based on a preset anomaly score threshold, the sample feature vector is divided into normal samples and candidate anomaly samples. For example, if the anomaly score threshold is set to 0.7, samples with scores below 0.7 are marked as normal samples, indicating that they conform to the normal distribution, while samples with scores above or equal to 0.7 are marked as candidate anomaly samples.

[0040] Understandably, the Isolation Forest model is based on the principle that outlier data is more easily randomly partitioned and isolated, enabling the initial screening of feature vectors of large-scale samples. This method avoids the computational bottleneck caused by traditional anomaly detection algorithms based on distance or density, which blindly mine high-value rare samples under large data volumes. It also reduces the amount of data that the subsequent rare validation model needs to process, thereby improving the overall efficiency of the process of mining high-value rare samples from large amounts of medical data.

[0041] In some possible implementations, assuming there are 10,000 sample feature vectors, after being imported into the isolated forest model for calculation, approximately 9,500 of the samples have anomaly scores below 0.7 and are classified as normal samples, while 500 samples have anomaly scores above 0.7 and are classified as candidate anomaly samples.

[0042] Step S22: Input the sample feature vector of the candidate abnormal sample into the rare verification model, obtain the rareness confidence score, and generate the corresponding sample label for the candidate abnormal sample according to the preset rareness threshold.

[0043] It should be noted that the rare validation model includes a reconstruction quality validation layer, a generation consistency validation layer, and a clinical relevance proxy validation layer, and the sample labels include high-value rare samples, samples to be reviewed, and noisy samples.

[0044] Step S22 includes: Step S22-1: Import the candidate abnormal samples into the reconstruction quality verification layer, and use a pre-trained reconstruction autoencoder to decode based on the candidate abnormal samples to obtain the reconstructed image of the abnormal samples.

[0045] Specifically, a reconstruction autoencoder is trained based on the regular samples obtained in step S21. The reconstruction autoencoder can adopt an encoder-decoder structure. The encoder is a neural network containing three fully connected layers. Its input dimension is the same as the dimension of the sample feature vector, and its output is a latent space vector, such as 128-dimensional. The decoder is a symmetric structure that reconstructs the latent space vector into an image with the same size as the corresponding multi-domain endoscope image, and uses it as an image to reconstruct abnormal samples. The training data consists of the sample feature vectors of all regular samples obtained in step S21 and their corresponding original multi-domain endoscope images. The training objective is to minimize the error between the regular reconstructed image and the corresponding multi-domain endoscope image, such as MSE loss. The reconstruction autoencoder learns the mapping relationship from the sample feature space to the image space for regular samples.

[0046] Furthermore, the sample feature vectors of the candidate anomalous samples are input into the reconstruction autoencoder to generate anomalous sample reconstruction images. Since the reconstruction autoencoder is trained on regular samples, the reconstruction images may not accurately restore the corresponding multi-domain endoscope images for anomalous samples, thus revealing anomalous characteristics.

[0047] Step S22-2: Obtain the reconstructed lesion region based on the image reconstructed from the abnormal sample, obtain the corresponding original lesion region based on the multi-domain endoscopic images in the multi-domain medical dataset, and obtain the structural similarity index based on the reconstructed lesion region and the original lesion region as the first verification score.

[0048] Specifically, for each candidate abnormal sample, the original lesion region is extracted based on the segmentation mask corresponding to its multi-domain endoscopic image, and the corresponding reconstructed lesion region is extracted from the reconstructed image. The structural similarity index between the original lesion region and the reconstructed lesion region is calculated. The structural similarity index can measure the similarity of images in terms of brightness, contrast and structure. The value of the structural similarity index ranges from 0 to 1. The structural similarity index is used as the first verification score. A low structural similarity index indicates poor reconstruction quality and suggests that the sample may have rare characteristics.

[0049] Step S22-3: Import the candidate anomaly samples into the consistency verification layer, generate multiple verification sample feature vectors based on the candidate anomaly samples by feature interpolation or adding noise, and decode the verification sample feature vectors into verification sample images.

[0050] Specifically, the feature vectors of candidate abnormal samples are slightly transformed, such as by adding weak Gaussian noise to each dimension of the feature vectors, or by linear interpolating with the feature vectors of the k nearest neighbors, to generate multiple verification sample feature vectors. A consistency decoder is then trained. This consistency decoder can employ an image decoder structure similar to or the same as the reconstruction autoencoder in step S22-1, and is trained using the feature vectors corresponding to regular samples and the mean square error of multi-domain endoscopic images as reconstruction losses. It learns a stable and smooth mapping relationship from the feature space to the image space, enabling the reconstruction autoencoder to decode sample feature vectors with the same semantics into images with consistent visual content. Its input is the verification sample feature vector, and its output is the verification sample image. In generating consistency verification, a... The consistency decoder decodes the feature vector of the verification sample into a verification sample image to evaluate the stability of the sample in the feature space. If the verification sample image and the multi-domain endoscopic image corresponding to the candidate abnormal sample maintain semantic consistency, it indicates that the candidate abnormal sample has stable pathological features. If the verification sample image shows a large difference, it suggests that it may be a noise sample. The structural similarity index or peak signal-to-noise ratio between the verification sample image and the corresponding multi-domain endoscopic image can be calculated to evaluate whether the two maintain semantic consistency or have a large difference. For example, assuming the structural similarity index threshold is 0.8, if the structural similarity index between the verification sample image and the corresponding multi-domain endoscopic image is 0.58, which is lower than the structural similarity index threshold, it is considered that the two have a large difference, and the candidate abnormal sample may be a noise sample.

[0051] It is understandable that candidate abnormal samples generated due to their actual pathological significance should be able to maintain semantic consistency in the local domain of the feature space. If a slight perturbation can lead to a change in the essential features of the decoded image, such as a large number of variations in the lesion morphology, then it is inferred that the current candidate abnormal sample may be noise.

[0052] Step S22-4: Use a pre-trained lesion morphology classifier to classify the verification sample images, obtain the statistical uncertainty of the classification results, and generate a second verification score based on the statistical uncertainty.

[0053] Specifically, a pre-trained lesion morphology classifier is used. This classifier can be a ResNet-18 model pre-trained on public datasets such as ImageNet, with its last fully connected layer replaced by a fully connected layer adapted to the number of categories in this task, such as normal, inflammation, and polyps. Subsequently, the lesion morphology classifier is fine-tuned on public medical image datasets such as the Kvasir dataset using image patches of lesion regions and corresponding pathological category labels. The training objective is to minimize the cross-entropy loss. The fine-tuned lesion morphology classifier is used to classify validation sample images, obtaining the probability distribution of each validation sample image. Based on the probability distribution, the entropy or variance of the predicted probability is obtained and used to quantify statistical uncertainty. The higher the entropy value, the more uncertain the lesion morphology classifier is about the category of the validation sample image. This high uncertainty may be associated with high-value rare samples. The mean of the statistical uncertainty of all validation sample images is calculated as the second validation score. For example, assuming that the statistical uncertainty is quantified by the entropy of the predicted probability, if the average entropy of the validation sample images generated by a candidate anomalous sample is 0.8, then the second validation score of the candidate anomalous sample is 0.8.

[0054] Step S22-5: Import candidate abnormal samples into the clinical relevance proxy verification layer, obtain abnormal sample morphological indicators based on candidate abnormal samples, and obtain the distribution of benchmark morphological indicators based on regular samples.

[0055] Specifically, based on the segmentation mask of the multi-domain endoscopic image corresponding to the candidate abnormal sample, morphological indicators are obtained and used as morphological indicators of the abnormal sample. The morphological indicators may include the area and roundness of the lesion region. Based on the segmentation mask corresponding to the regular sample, morphological indicators are obtained, and the covariance of the morphological indicators can be calculated as the baseline morphological indicator distribution.

[0056] Step S22-6: Obtain the Mahalanobis distance between the distributions of the morphological indices of the abnormal samples and the baseline morphological indices, and generate a third verification score based on the Mahalanobis distance.

[0057] Specifically, the Mahalanobis distance between the distribution of morphological indicators of abnormal samples and the distribution of baseline morphological indicators is calculated. The Mahalanobis distance takes into account the covariance structure of the distribution and can measure the degree to which the sample deviates from the normal morphology. The larger the Mahalanobis distance, the more likely the candidate abnormal sample is to be rare. The Mahalanobis distance is normalized and used as the third verification score.

[0058] Step S22-7: Based on the first verification score, the second verification score and the third verification score, generate the corresponding rareness confidence score for the candidate abnormal sample.

[0059] Specifically, the first, second, and third verification scores are normalized to the same dimension, for example, the interval [0, 1]. To ensure that the first, second, and third verification scores are consistent in their direction when representing rarity, the first verification score is converted to 1 - the first verification score. The larger the value, the higher the rarity. Then, through weighted fusion and other methods, the rarity confidence score is obtained. For example, the first, second, and third verification scores are normalized to the range [1, 10]. The weights can be set as 0.4 for the first verification score, 0.3 for the second verification score, and 0.3 for the third verification score. This weight allocation aims to emphasize the fundamental role of reconstruction quality in rarity assessment. Then, the rarity confidence score = 0.4 × first verification score + 0.3 × second verification score + 0.3 × third verification score.

[0060] Furthermore, after obtaining the rarity confidence score, corresponding sample labels are generated for candidate abnormal samples based on a preset rarity threshold. For example, if the threshold for high-value rare samples is set to 0.7 and the threshold for noise samples is set to 0.3, then if the rarity confidence score is greater than or equal to 0.7, it is marked as a high-value rare sample, indicating that the sample has highly credible rare features. If the rarity confidence score is less than or equal to 0.3, it is marked as a noise sample, which can be removed to purify the dataset and improve the stability and performance of subsequent model training. If the rarity confidence score is between 0.3 and 0.7, it is marked as a sample to be reviewed, which can be retained for further review by domain experts to balance the efficiency of automated processing and the accuracy of decision-making.

[0061] Step S23: Use latent space augmentation techniques to process high-value rare samples to obtain augmented high-value rare samples.

[0062] Specifically, latent space augmentation techniques increase the diversity of each high-value rare sample by transforming it in the feature space. For example, generative adversarial networks or variational autoencoders can be used to interpolate or add noise near the sample feature vectors of high-value rare samples to generate augmented feature vectors, which are then decoded into augmented images. The augmentation process preserves the rare characteristics of high-value rare samples and generates more variants to balance the distribution of the dataset. This step is used to expand the number of rare samples. For example, for the sample feature vector of a high-value rare sample, five augmented feature vectors are generated through linear interpolation and decoded into five augmented images. These augmented images are similar to the high-value rare sample in the feature space but have subtle variations.

[0063] Step S24: Package the regular samples and augmented high-value rare samples into a training dataset.

[0064] Specifically, the regular samples obtained in step S21 are merged with the enhanced high-value rare samples obtained in step S23 to form a training dataset. The training dataset is packaged in a standard format, such as HDF5 or JSON. The training dataset includes the storage path of multi-domain endoscopic images, the corresponding image annotation information, and the domain identifier. The training dataset is used for model training in subsequent steps to ensure that the data has both regularity and rarity. By keeping the overall data distribution relatively close to the real world, the basic performance of the model is maintained. At the same time, by introducing an appropriate amount of enhanced high-value rare samples, the class distribution of the data is balanced, thereby improving the model's ability to identify rare pathological patterns and its overall generalization ability. For example, regular samples account for 90% of the training dataset, and enhanced high-value rare samples account for 10%. This ratio is set to effectively alleviate the scarcity of rare samples in the dataset, while avoiding the distortion of the model's learning of regular pathological patterns due to an excessively high proportion of rare samples, rather than accurately simulating the actual clinical incidence distribution.

[0065] It should be noted that during the encapsulation process, the regular samples retain the corresponding multi-domain endoscope image path, image annotation information and domain identifier. For the enhanced high-value rare samples generated in step S23, their image annotation information and domain identifier are inherited from the corresponding high-value rare samples to ensure that the enhanced samples are consistent with the enhanced high-value rare samples in terms of semantics and domain attributes.

[0066] Step S3: Based on the training dataset, construct and train the anatomical domain encoder and the pathological instance encoder.

[0067] Step S3 includes: Step S31: The anatomical domain encoder is trained based on the training dataset using contrastive loss and adversarial loss. The anatomical domain encoder takes endoscopic images as input and outputs a low-dimensional vector representing the anatomical domain data, which serves as the anatomical domain feature vector.

[0068] It should be noted that the anatomical domain encoder adopts a convolutional neural network structure. Its input is a standardized multi-domain endoscopic image and its corresponding domain identifier, such as an RGB image with a size of 224×224 pixels. The output is a low-dimensional vector used to represent the anatomical domain features of the image, such as scene lighting, imaging device characteristics, and the overall anatomical structure of the imaged area. The anatomical domain encoder is jointly trained with contrast loss and adversarial loss to ensure that the output low-dimensional vector can distinguish different domains while maintaining the compactness of the features within the domain.

[0069] Specifically, for each image sample in the training dataset, it is compared with other image samples in the same domain and image samples in different domains. The optimization goal is to minimize the distance between the feature vectors of samples with the same domain identifier in the feature space, such as the cosine distance, while maximizing the distance between the feature vectors of samples with different domain identifiers, so that the anatomy domain encoder learns feature representations that are sensitive to domain changes.

[0070] Furthermore, adversarial loss is achieved by introducing a domain discriminator. The domain discriminator takes the low-dimensional vector output by the anatomical domain encoder as input and determines which domain the vector originates from. The training objective of the anatomical domain encoder is to adversarially challenge the domain discriminator and generate low-dimensional vectors whose domain origin is difficult for the domain discriminator to determine. This adversarial training prompts the anatomical domain encoder to learn more generalized anatomical structural features that are independent of specific domain information and suppress noise information that is strongly correlated with specific equipment or hospitals.

[0071] By combining contrastive loss and adversarial loss for end-to-end training, the anatomical domain encoder can ultimately extract a low-dimensional vector that has a certain domain discrimination capability without overly relying on domain-specific details. The dimension of this vector can be set according to the requirements, such as 128-dimensional or 256-dimensional.

[0072] Step S32: The pathological instance encoder is trained based on the training dataset using focus loss and correlation constraints. The pathological instance encoder takes the lesion area of ​​the endoscopic image as input and outputs a low-dimensional vector representing the pathological instance data, which is then used as the pathological instance feature vector.

[0073] It should be noted that the pathological instance encoder adopts a convolutional neural network structure. Its input is the lesion region obtained based on the segmentation mask or bounding box in the image annotation information, and the output is a low-dimensional vector used to represent the pathological instance data, that is, the features of the lesion region itself, such as its shape, texture, boundary and other local attributes closely related to the pathological nature.

[0074] Specifically, the training of the pathological instance encoder is mainly based on the pathological category labels corresponding to the lesion areas, and the focus loss is used as the main supervision signal. The focus loss is an improved form of cross-entropy loss. By introducing an adjustable focus parameter, it effectively solves the problem of the imbalance of the number of samples of different categories in the training data. For example, there are far more normal tissue areas than rare lesion areas. By reducing the loss contribution of correctly classified and easily classified samples, the focus loss makes the model pay more attention to the difficult-to-classify samples during the training process. The difficult-to-classify samples may be minority classes or lesions with atypical morphology, thereby improving the feature extraction ability of the pathological instance encoder for rare or complex pathological instances.

[0075] Furthermore, a correlation constraint is introduced during the training process. Specifically, in the feature space, the feature vectors of training data with the same pathological category label should have high similarity, i.e., small intra-class distance, while the feature vectors of training data with different pathological category labels should have large differences, i.e., large inter-class distance. The correlation constraint can be implemented by adding a term based on the distance between sample feature vectors to the loss function, such as triplet loss or center loss. By introducing the correlation constraint, the ability of the output low-dimensional vector to represent pathological instance data is improved.

[0076] Understandably, through the combined effect of focus loss and correlation constraints, the pathological instance encoder can learn a feature representation that is highly discriminative of the lesion area. This feature can highlight the key characteristics of the pathological instance and remain sensitive to changes in its category. The dimension of the output low-dimensional vector can be set, for example, 64 or 128 dimensions.

[0077] In summary, step S3 trained two complementary encoders: the anatomical encoder focuses on global, domain-related anatomical background features of the image, while the pathological instance encoder focuses on local, pathological features related to the lesion area of ​​the image. The low-dimensional vectors output by the anatomical encoder and the pathological instance encoder will provide key conditional inputs for the construction of the meta-domain image enhancement model in the subsequent step S4.

[0078] Step S4: Construct a meta-domain image enhancement model. Based on the training dataset, use an anatomical domain encoder and a pathological instance encoder to obtain the task meta-loss, and train the meta-domain image enhancement model based on the task meta-loss.

[0079] Step S4 includes: Step S41: Sample the training dataset according to the meta-learning task paradigm to obtain multiple meta-tasks. Each meta-task contains a support set and a query set.

[0080] It should be noted that before sampling begins, the training dataset is randomly divided into a meta-training set and a meta-validation set. The meta-training set is used for meta-task sampling and meta-parameter updates during the training process of the meta-domain image enhancement model. The meta-validation set does not participate in training and is used to determine whether the meta-domain image enhancement model is overfitting and to determine when the training converges.

[0081] Specifically, the meta-learning task paradigm employs an optimized meta-learning framework, such as model-independent meta-learning. Its core objective is to enable the meta-domain image enhancement model, through training, to quickly adapt to image enhancement tasks in a target domain using only a small number of samples and a few iterations. When sampling the meta-training set, a domain identifier-based sampling strategy is used. Specifically, all samples in the support set and query set of the meta-task must originate from the same domain, i.e., possess the same domain identifier, such as the same device ID or hospital code. Each meta-task simulates an image enhancement scenario for a specific domain. The support set is used within the meta-task to enhance the meta-domain image. The enhancement model undergoes a small number of gradient updates to quickly adapt it to the specific domain represented by the current meta-task. The query set is then used to evaluate the performance of the adapted meta-domain image enhancement model on the same specific domain, obtaining the evaluation result, i.e., the task loss, which serves as a feedback signal to update the meta-parameters of the meta-domain image enhancement model outside the meta-task. This allows the model to be trained to achieve better and faster adaptation capabilities. If the support set and query set come from different domains, the adaptation effect of the meta-domain image enhancement model to a specific domain cannot be accurately evaluated. The samples in the support set and query set are randomly and disjointly sampled from the data in the same domain to ensure the fairness of the evaluation.

[0082] Understandably, a domain is randomly selected from the training dataset, and samples of multiple pathological category labels, such as normal, inflammation, and polyps, are randomly selected from the corresponding multi-domain medical dataset. The support set extracts a small number of samples from these categories, such as 5 samples from each category, to simulate the actual situation where image annotation information is lacking in the new domain. The query set can extract 15 samples from each category from the remaining data for performance evaluation. This constitutes an N-way K-shot meta-task. The total number of meta-tasks can be set according to the size of the training dataset and the number of domains. For example, 50 meta-tasks can be generated for each of the 20 available different domains, for a total of 1000 meta-tasks, to ensure that the meta-domain image enhancement model can be exposed to a sufficiently diverse range of domain characteristics during training, thereby learning a universal and rapid adaptive capability.

[0083] Furthermore, the same process of obtaining the corresponding meta-tasks based on the meta-training set is adopted to obtain the corresponding meta-tasks based on the meta-validation set, and then form a validation meta-task set.

[0084] Step S42: For the current meta-task, obtain the target domain encoding using the dissecting domain encoder based on the support set.

[0085] Specifically, the multi-domain endoscopic images corresponding to the support set in the meta-task are obtained and input into the anatomical domain encoder trained in step S3. The anatomical domain feature vector corresponding to each multi-domain endoscopic image is obtained. The anatomical domain feature vector represents the global domain-related attributes of the multi-domain endoscopic image, such as imaging device characteristics, scene lighting, anatomical structure background, etc. The anatomical domain feature vectors corresponding to all the support sets are aggregated, such as by taking the mean or weighted average, to generate a comprehensive, low-dimensional target domain code. This target domain code represents the overall style features of the specific domain represented by the current meta-task. The target domain code is input into the meta-image generator to guide it to generate an image that conforms to the style features of the domain. For example, for the meta-task with the support set from the A-type colonoscopy system of Hospital 1, a 128-dimensional target domain code is obtained through the anatomical domain encoder. The target domain code represents the overall style features of the multi-domain endoscopic images output by the A-type colonoscopy system of Hospital 1, such as color bias, brightness, and texture style.

[0086] Step S43: Based on the query set and the corresponding target domain encoding, execute the image generation adversarial training process, obtain the meta-image generation adversarial network adapted to the current meta-task, and obtain the task meta-loss of the meta-image generation adversarial network on the query set.

[0087] The meta-image generative adversarial network includes a meta-image generator and a meta-image discriminator.

[0088] It should be noted that the meta-image generative adversarial network can be built based on the conditional generative adversarial network framework. The meta-image generator can adopt a variant of U-Net, etc. The input of the meta-image generator is conditional information and optional noise vector. The conditional information is specifically the target domain encoding and the pathological instance feature vector. The noise vector is used to generate diverse output images with subtle variations under the same conditions, thereby enhancing the richness of the synthetic dataset. The output of the meta-image generator is the adversarial reconstructed image. The meta-image discriminator can be a convolutional neural network. Its input is a multi-domain endoscopic image, an adversarial reconstructed image, and the corresponding conditional information. The multi-domain endoscopic image, the adversarial reconstructed image, and the corresponding conditional information are input into the meta-image discriminator for discrimination. The output is the probability that the input image is a multi-domain endoscopic image.

[0089] Step S43 includes: Step S43-2: Obtain the feature vector of the pathological instance using a pathological instance encoder based on the query set.

[0090] Specifically, each multi-domain endoscopic image in the query set of the current meta-task is input into the pathological instance encoder to extract the pathological instance feature vector corresponding to its lesion area. The pathological instance feature vector encodes the local pathological features of the multi-domain endoscopic image, such as the morphology, texture, and boundary of the lesion, to ensure that the pathological features of the generated adversarial reconstruction image are consistent with those of each multi-domain endoscopic image in the query set.

[0091] Step S43-3: Input the target domain encoding and pathological instance feature vector as conditions into the meta-image generator to generate an adversarial reconstruction image.

[0092] Specifically, the target domain encoding is fused with the corresponding pathological instance feature vector. This fusion can be achieved through concatenation or cross-attention mechanisms, and the result is input into the meta-image generator to generate the corresponding synthetic image, i.e., the adversarial reconstruction image. For example, the meta-image generator combines the target domain encoding representing the Hospital 1 style with a pathological instance feature vector representing the pathological features of polyps to generate an adversarial reconstruction image that has the Hospital 1 style and contains the pathological features of polyps.

[0093] Step S43-4: Input the adversarial reconstructed image and the endoscopic image corresponding to the query set into the meta-image discriminator to obtain the adversarial loss.

[0094] Specifically, the meta-image discriminator simultaneously receives multi-domain endoscopic images and adversarial reconstructed images from the query set and determines their authenticity. The multi-domain endoscopic images are real images, while the adversarial reconstructed images are generated images. The adversarial loss can adopt standard GAN loss functions, such as least squares loss or hinge loss, to drive the meta-image generator to generate realistic images that can deceive the meta-image discriminator. The loss of the meta-image discriminator specifically includes classifying real images as real and generating images as fake, while the loss of the meta-image generator is to make the adversarial reconstructed images be classified as real by the meta-image discriminator.

[0095] Step S43-5: Obtain the reconstructed image loss based on the adversarial reconstructed image and the endoscopic image corresponding to the query set.

[0096] Specifically, the reconstructed image loss is used to constrain the similarity between the generated image and the real image at the pixel level and perceptual level to ensure content fidelity. The reconstructed image loss includes at least pixel-level loss and perceptual loss. The pixel-level loss can be L1 loss or L2 loss. The pixel-level loss directly measures the difference in pixel intensity between the adversarial reconstructed image and the multi-domain endoscopic image. The perceptual loss can be obtained based on a pre-trained deep network, such as VGG. The adversarial reconstructed image and the multi-domain endoscopic image are respectively input into the pre-trained deep network, feature maps are extracted at an intermediate layer, and the L1 loss or L2 loss between the feature maps is calculated and used as the perceptual loss. The perceptual loss measures the similarity between the adversarial reconstructed image and the multi-domain endoscopic image at the semantic feature level, and is more effective in preserving details and textures.

[0097] Furthermore, the weighted sum of pixel-level loss and perceptual loss is calculated and used as the loss for reconstructing the image. The weighting weights can be empirically tuned based on specific task requirements or through performance feedback on the meta-task.

[0098] Step S43-6: Update the meta-image generator and meta-image discriminator based on adversarial loss and reconstructed image loss, and use them as the meta-image generative adversarial network adapted to the current meta-task.

[0099] Specifically, within the current meta-task, gradient descent algorithms, such as the Adam optimizer, can be used to minimize the weighted sum of adversarial loss and reconstructed image loss. This allows for a small number of iterations (e.g., 5-10) to update the parameters of the meta-image generator and meta-image discriminator. The updated network parameters constitute the meta-image generative adversarial network adapted for the current meta-task. The total loss of the updated image generative adversarial network, such as the weighted sum of adversarial loss and reconstructed loss, is calculated on the query set and used as the task meta-loss for the current meta-task. This task meta-loss reflects the performance of the adapted meta-image generative adversarial network in generating realistic images that conform to the style and content of the current domain.

[0100] Step S44: Obtain the meta-image generative adversarial networks adapted to all meta-tasks and their corresponding task meta-losses. Based on all task meta-losses, obtain the training meta-loss.

[0101] Specifically, for all meta-tasks generated from samples in the meta-training set, steps S42 to S43 are repeated to obtain the meta-image generative adversarial network adapted to each meta-task and its corresponding task meta-loss. The average value of all task meta-losses is calculated as the training meta-loss. The training meta-loss characterizes the average performance of the current parameters of the meta-model on all training tasks and is the basis for meta-optimization.

[0102] Step S45: Based on the training meta-loss, the meta-optimization algorithm is used to update the meta-parameters of the meta-domain image enhancement model, and the process is iterated until the preset convergence criterion is met.

[0103] Step S45 includes: Step S45-1: For each meta-task, obtain the meta-loss gradient of the training meta-loss on the parameters of the meta-domain image augmentation model, and update the parameters of the meta-domain image augmentation model based on the meta-loss gradient.

[0104] Specifically, a model-independent meta-learning algorithm can be used to calculate the gradient of the training meta-loss with respect to the initial parameters of the meta-image generator. This gradient is obtained by backpropagating the task meta-loss of all meta-tasks, indicating how to adjust the meta-parameters to make the meta-domain image enhancement model perform better when facing new tasks. Subsequently, the gradient descent algorithm is used to update the initial parameters of the meta-image generator.

[0105] Step S45-2: When the training meta-loss triggers any one of the threshold convergence condition, iterative convergence condition, and performance convergence condition in multiple consecutive iterations, it is determined that the training of the meta-domain image enhancement model has converged.

[0106] Specifically, the threshold convergence condition is that the decrease in the meta-loss in P consecutive iterations is less than the threshold T. The specific values ​​of P and T can be set according to the training stability requirements of the training process, such as P=10 and T=0.001. The iterative convergence condition is that the number of training iterations reaches the maximum preset value, such as 10,000 times. The performance convergence condition is that the average task loss is calculated on the validation meta-task set constructed based on the meta-validation set and used as the validation meta-loss. When the validation meta-loss no longer decreases significantly or begins to increase, this indicates overfitting. Training is stopped when any condition is met, the meta-parameters at this time are saved, and the training of the meta-domain image enhancement model is completed.

[0107] Step S5: Obtain the new domain medical dataset. Based on the new domain medical dataset, use the anatomical domain encoder to obtain the new domain anatomical domain encoding, and use the pathological instance encoder to obtain the target instance encoding. Import the new domain anatomical domain encoding and the target instance encoding into the meta-domain image enhancement model to obtain the new domain synthetic dataset.

[0108] Step S5 includes: Step S51: Import the new domain endoscope images from the new domain medical dataset into the medical feature extraction model to obtain the new domain sample feature vector corresponding to each new domain endoscope image.

[0109] Understandably, the New Domain Medical Dataset refers to a set of endoscope images collected by a new partner hospital or a new model of endoscope equipment that did not participate in the training process of steps S1 to S4, for the target application scenario.

[0110] Specifically, the new domain endoscopic image undergoes the same standardization preprocessing as in step S11, such as resolution unification and color correction. The standardized new domain endoscopic image is then input into the medical feature extraction model, which has been pre-trained and fine-tuned in step S1, for forward propagation to extract high-level semantic features. This yields a new domain sample feature vector corresponding to each new domain endoscopic image. The new domain sample feature vector is a high-level encoding of the content of the new domain endoscopic image. For example, a colonoscopy image acquired by a C-type colonoscopy system from hospital 3 is input into the medical feature extraction model, resulting in a 2048-dimensional new domain sample feature vector.

[0111] Step S52: Obtain the average new domain code using the anatomical domain encoder based on the feature vector of the new domain sample.

[0112] Specifically, the feature vectors of the new domain samples are input into the anatomical domain encoder trained in step S3. The anatomical domain encoder obtains a corresponding low-dimensional anatomical domain feature vector for each new domain sample feature vector. The anatomical domain feature vector represents the global domain attributes of each new domain endoscopic image, such as equipment characteristics and overall illumination. An aggregation operation is performed on the anatomical domain feature vectors corresponding to all new domain sample feature vectors, such as calculating the mean, to generate a comprehensive average new domain code. This code condenses the domain-shared features of the entire new domain dataset and serves as a condition for controlling the consistency of the domain attributes generated by the image. For example, based on 100 colonoscopy images acquired from the C-type colonoscopy system of Hospital 3, and imported into the medical feature extraction model, 100 corresponding new domain sample feature vectors are obtained. Based on the 100 new domain sample feature vectors, the anatomical domain encoder obtains the corresponding low-dimensional anatomical domain feature vectors. The average of all anatomical domain feature vectors is calculated to obtain the average new domain code. The average new domain code represents the typical color and texture style of the C-type colonoscopy system of Hospital 3.

[0113] Step S53: Obtain the expected pathological parameters, and use a pre-trained semantic mapping model to obtain the expected pathological code based on the expected pathological parameters.

[0114] Specifically, the expected pathological parameters are specified by the user based on downstream tasks, such as training a polyp classifier. These parameters can take the form of text descriptions, pathological category labels, or morphological indicators. Based on a pre-trained semantic mapping model, these high-level semantic expected pathological parameters are mapped to expected pathological codes. The semantic mapping model can be a fully connected neural network, with its input being the code of the expected pathological parameters, such as one-hot encoding, and its output being a low-dimensional vector with the same dimension as the pathological instance feature vector output by the pathological instance encoder. The training data for the semantic mapping model consists of the pathological category labels of all samples in the training dataset and their corresponding pathological instance feature vectors extracted by the pathological instance encoder. The training objective is to minimize the mean squared error loss between the expected pathological code output by the semantic mapping model and the pathological instance feature vector. Through training, the model learns to map high-level semantic pathological parameters to corresponding coordinates in the feature space. For example, the pathological category label is mapped to an integer code, specifically normal = 0, inflammation = 1, polyp = 2. Inputting two labels into the semantic mapping model outputs a 64-dimensional expected pathological code, which is similar in pathological features to the sample feature vector of a real polyp image.

[0115] Step S54: Import the average new domain code and the expected pathological code as conditional data into the meta-image generator in the meta-domain image enhancement model to generate the new domain synthetic dataset.

[0116] Specifically, the average new domain encoding can be used to control the domain style of the generated images, and the expected pathology encoding can be used to control the pathological content of the generated images. The average new domain encoding and the expected pathology encoding are used as conditional inputs and imported into the meta-domain image enhancement model. The meta-image generator in the meta-domain image enhancement model generates a large number of synthetic images based on the conditional inputs. These synthetic images are visually consistent with the new domain and meet the specified requirements in terms of pathological features, thus forming a new domain synthetic dataset.

[0117] Understandably, the new domain synthetic dataset addresses data distribution imbalance by generating rare pathological features and adapts to multi-domain differences by injecting new domain styles. It can be used to enhance the training data of downstream models in the new domain, such as medical image segmentation models, thereby improving the performance and generalization ability of downstream models.

[0118] A second aspect of the present invention provides an endoscopic image data enhancement system based on adversarial networks, comprising: The data processing and enhancement module is used to acquire multi-domain medical datasets and import them into a medical feature extraction model. It acquires sample feature vectors, uses an isolated forest model based on the sample feature vectors to acquire regular samples and candidate abnormal samples, imports the candidate abnormal samples into a rare validation model to acquire high-value rare samples, performs feature space enhancement, and encapsulates regular samples and high-value rare samples into a training dataset. An encoder training module is used to construct and train an anatomical domain encoder and a pathological instance encoder based on the training dataset. Metadomain image enhancement model module, which is used to construct a metadomain image enhancement model, obtains task meta-loss using the anatomical encoder and the pathological instance encoder based on the training dataset, and trains the metadomain image enhancement model based on the task meta-loss. The new domain synthetic data module is used to acquire a new domain medical dataset, and uses the anatomical domain encoder to acquire the new domain anatomical domain code, and uses the pathological instance encoder to acquire the target instance code, and imports the new domain anatomical domain code and the target instance code into the meta-domain image enhancement model to acquire the new domain synthetic dataset.

[0119] The specific usage and function of this invention are described below: Based on the above aspects, this application embodiment obtains a multi-domain medical dataset and uses an isolated forest model and a rare validation model to screen out high-value rare samples and perform feature space enhancement to construct a balanced training dataset. By intelligently mining and enhancing rare samples, it overcomes the limitation of traditional enhancement methods in generating new semantic content, effectively solves the problem of data distribution imbalance in endoscopic data, and provides a more balanced data foundation for model training.

[0120] On the other hand, by constructing an anatomical domain encoder and a pathological instance encoder to decouple image features, and using a meta-learning framework to train a meta-domain image enhancement model, the model gains the ability to quickly adapt to the features of new domain data. This method requires only a small number of new domain samples to generate synthetic images that are highly adapted in anatomical style and clear in pathological features, reducing the dependence on large-scale annotation of new domain data and effectively overcoming the problem of poor data usability caused by domain offset.

Claims

1. A method for enhancing endoscopic image data based on adversarial networks, characterized in that, The method includes: Obtain a multi-domain medical dataset and import it into a medical feature extraction model to obtain sample feature vectors. Based on the sample feature vector, the isolated forest model is used to obtain regular samples and candidate anomaly samples. The candidate anomaly samples are imported into the rare validation model, and high-value rare samples are obtained. Feature space enhancement is performed on the high-value rare samples, and regular samples and high-value rare samples are encapsulated into a training dataset. Based on the training dataset, construct and train the anatomical domain encoder and the pathological instance encoder; A meta-domain image enhancement model is constructed. Based on the training dataset, the task meta-loss is obtained by using an anatomical domain encoder and a pathological instance encoder. The meta-domain image enhancement model is then trained based on the task meta-loss. Obtain a new domain medical dataset. Based on the new domain medical dataset, use an anatomical domain encoder to obtain the new domain anatomical domain encoding and use a pathological instance encoder to obtain the target instance encoding. Import the new domain anatomical domain encoding and the target instance encoding into the meta-domain image enhancement model to obtain the new domain synthetic dataset.

2. The endoscopic image data enhancement method based on adversarial networks according to claim 1, characterized in that, Obtain a multi-domain medical dataset, import it into a medical feature extraction model to obtain sample feature vectors, including: Acquire multi-domain endoscopic images and their corresponding image annotation information, and perform standardization processing on the multi-domain endoscopic images and their corresponding image annotation information; The image annotation information includes at least bounding boxes, segmentation masks, and pathological category labels; Multi-domain endoscopic images and their corresponding image annotation information are encapsulated into a multi-domain medical dataset; The pre-trained feature extraction network is used as the medical feature extraction model; Multi-domain endoscopic images from a multi-domain medical dataset are imported into a medical feature extraction model to obtain the sample feature vector corresponding to each multi-domain endoscopic image.

3. The endoscopic image data enhancement method based on adversarial networks according to claim 1, characterized in that, Based on sample feature vectors, an isolated forest model is used to obtain regular samples and candidate anomaly samples. The candidate anomaly samples are then imported into a rare validation model, and high-value rare samples are obtained. Feature space augmentation is performed on the high-value rare samples. The regular samples and high-value rare samples are then packaged into a training dataset, including: The sample feature vector is input into the isolated forest model to obtain the anomaly score corresponding to the sample feature vector, and then the normal samples and candidate anomaly samples are obtained based on the anomaly score; The sample feature vector of the candidate abnormal sample is input into the rare verification model to obtain the rareness confidence score, and corresponding sample labels are generated for the candidate abnormal sample according to the preset rareness threshold. The rare validation model includes a reconstructed quality validation layer, a generated consistency validation layer, and a clinically relevant proxy validation layer. The sample labels include high-value rare samples, samples to be reviewed, and noisy samples; We employ latent spatial augmentation techniques to process high-value rare samples, thereby obtaining augmented high-value rare samples. The regular samples and enhanced high-value rare samples are packaged into a training dataset.

4. The endoscopic image data enhancement method based on adversarial networks according to claim 3, characterized in that, The sample feature vectors of candidate anomalies are input into the rareness verification model to obtain rareness confidence scores, and corresponding sample labels are generated for the candidate anomalies based on a preset rareness threshold, including: Candidate anomaly samples are imported into the reconstruction quality verification layer, and a pre-trained reconstruction autoencoder is used to decode the anomaly sample to obtain the reconstructed image of the anomaly sample. The reconstructed lesion region is obtained from the image reconstructed based on the abnormal sample, the corresponding original lesion region is obtained from the multi-domain endoscopic image, and the structural similarity index is obtained from the reconstructed lesion region and the original lesion region as the first verification score. The candidate anomaly samples are imported into the consistency verification layer. Based on the candidate anomaly samples, multiple verification sample feature vectors are generated by feature interpolation or adding noise. The verification sample feature vectors are then decoded into verification sample images. A pre-trained lesion morphology classifier is used to classify the verification sample images, and the statistical uncertainty of the classification results is obtained. A second verification score is generated based on the statistical uncertainty. Candidate abnormal samples are imported into the clinical relevance proxy verification layer. Morphological indicators of abnormal samples are obtained based on candidate abnormal samples, and the distribution of baseline morphological indicators is obtained based on regular samples. Obtain the Mahalanobis distance between the distributions of morphological indicators of abnormal samples and baseline morphological indicators, and generate a third verification score based on the Mahalanobis distance; Based on the first verification score, the second verification score, and the third verification score, a rareness confidence score is generated for the candidate abnormal sample.

5. The endoscopic image data enhancement method based on adversarial networks according to claim 1, characterized in that, Based on the training dataset, an anatomical domain encoder and a pathological instance encoder were constructed and trained, including: The anatomical domain encoder is trained based on a training dataset using contrastive loss and adversarial loss. The anatomical domain encoder takes endoscopic images as input and outputs a low-dimensional vector representing the anatomical domain data, which serves as the anatomical domain feature vector. The pathological instance encoder is trained based on the training dataset using focus loss and correlation constraints. The pathological instance encoder takes the lesion area of ​​the endoscopic image as input and outputs a low-dimensional vector representing the pathological instance data, which serves as the feature vector of the pathological instance.

6. The endoscopic image data enhancement method based on adversarial networks according to claim 1, characterized in that, A meta-domain image enhancement model is constructed. Based on the training dataset, the task-specific meta-loss is obtained using an anatomical domain encoder and a pathological instance encoder. The meta-domain image enhancement model is then trained based on the task-specific meta-loss, including: The training dataset is sampled according to the meta-learning task paradigm to obtain multiple meta-tasks, each of which contains a support set and a query set. For the current meta-task, the target domain encoding is obtained using a dissecting domain encoder based on the support set; Based on the query set and the corresponding target domain encoding, the image generation adversarial training process is executed to obtain the meta-image generation adversarial network adapted to the current meta-task, and the task meta-loss of the meta-image generation adversarial network on the query set is obtained. Obtain the meta-image generative adversarial networks adapted to all meta-tasks and their corresponding task meta-losses. Based on all task meta-losses, obtain the training meta-loss. Based on the training meta-loss, a meta-optimization algorithm is used to update the meta-parameters of the meta-domain image enhancement model, and the process is iteratively executed until the preset convergence criterion is met.

7. The endoscopic image data enhancement method based on adversarial networks according to claim 6, characterized in that, Based on the query set and the corresponding target domain encoding, an image generative adversarial training process is executed to obtain the meta-image generative adversarial network adapted to the current meta-task, and to obtain the task meta-loss of the meta-image generative adversarial network on the query set, including: The meta-image generative adversarial network includes a meta-image generator and a meta-image discriminator; Based on the query set, a pathological instance encoder is used to obtain the feature vector of the pathological instance; The target domain encoding and pathological instance feature vectors are used as conditions and input into the meta-image generator to generate an adversarial reconstruction image. The adversarial reconstructed image and the endoscopic image corresponding to the query set are input into the meta-image discriminator to obtain the adversarial loss; The reconstructed image loss is obtained based on the adversarial reconstructed image and the corresponding endoscopic image in the query set; The meta-image generator and meta-image discriminator are updated based on adversarial loss and reconstructed image loss, and used as a meta-image generative adversarial network adapted to the current meta-task.

8. The endoscopic image data enhancement method based on adversarial networks according to claim 6, characterized in that, The parameters of the meta-domain image enhancement model are updated using a meta-optimization algorithm based on meta-loss, and the process is iteratively executed until a preset convergence criterion is met, including: For each meta-task, obtain the meta-loss gradient of the training meta-loss on the parameters of the meta-domain image augmentation model, and update the parameters of the meta-domain image augmentation model based on the meta-loss gradient. When the training meta-loss triggers any one of the threshold convergence condition, iterative convergence condition, or performance convergence condition in multiple consecutive iterations, it is determined that the training of the meta-domain image enhancement model has converged.

9. The endoscopic image data enhancement method based on adversarial networks according to claim 1, characterized in that, A new domain medical dataset is obtained. Based on this dataset, an anatomical domain encoder is used to obtain the new domain anatomical domain encoding, and a pathological instance encoder is used to obtain the target instance encoding. The new domain anatomical domain encoding and the target instance encoding are then imported into a meta-domain image enhancement model to obtain a new domain synthetic dataset, including: Import the new domain endoscope images from the new domain medical dataset into the medical feature extraction model to obtain the new domain sample feature vector corresponding to each new domain endoscope image; The average new domain code is obtained by using an anatomical domain encoder based on the feature vector of the new domain sample. Obtain the expected pathological parameters, and use a pre-trained semantic mapping model to obtain the expected pathological code based on the expected pathological parameters; The average new domain code and the expected pathological code are imported as conditional data into the meta-image generator within the meta-domain image enhancement model to generate a new domain synthetic dataset.

10. An endoscopic image data enhancement system based on adversarial networks, applied to the method described in any one of claims 1 to 9, characterized in that, include: The data processing and enhancement module is used to acquire multi-domain medical datasets and import them into a medical feature extraction model. It acquires sample feature vectors, uses an isolated forest model based on the sample feature vectors to acquire regular samples and candidate abnormal samples, imports the candidate abnormal samples into a rare validation model to acquire high-value rare samples, performs feature space enhancement, and encapsulates regular samples and high-value rare samples into a training dataset. An encoder training module is used to construct and train an anatomical domain encoder and a pathological instance encoder based on the training dataset. Metadomain image enhancement model module, which is used to construct a metadomain image enhancement model, obtains task meta-loss using the anatomical encoder and the pathological instance encoder based on the training dataset, and trains the metadomain image enhancement model based on the task meta-loss. The new domain synthetic data module is used to acquire a new domain medical dataset, and uses the anatomical domain encoder to acquire the new domain anatomical domain code, and uses the pathological instance encoder to acquire the target instance code, and imports the new domain anatomical domain code and the target instance code into the meta-domain image enhancement model to acquire the new domain synthetic dataset.

Images