Visual image data enhancement and synthesis system based on generative model

Abstract

The invention discloses a visual image data enhancement and synthesis system and method based on a generative model. Relates to the technical field of visual images. The task utility evaluation module is used for calculating a dynamic task utility loss directly reflecting downstream task performance; and the style consistency evaluation module is used for calculating a label style consistency loss reflecting the difference between the generated label and the standard style. The generation module carries out training according to a composite loss function formed by the two losses. The invention further discloses a dynamic guidance mechanism based on course learning, and the training strategy can be dynamically adjusted according to the learning state of the downstream model. According to the method, the synthetic visual data which is more effective for downstream tasks, more stable in labeling quality and more reliable can be generated.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of visual image, in particular to a visual image data enhancement and synthesis system based on a generative model. BACKGROUND

[0002] In the fields of medical images, autonomous driving, etc., the performance of deep learning models is highly dependent on large-scale and high-quality labeled data, which is costly and limited to obtain. Therefore, the field has extensively studied the use of generative models such as generative adversarial networks to synthesize new data to expand the training set. This conventional method strives to improve the visual fidelity of generated images through the adversarial game between the generator and the discriminator. However, the existing technology exposes several deep technical limitations in practice. First, the core of its training objective is "visual fidelity", but this is fundamentally different from the "task utility" pursued by downstream specific tasks such as lesion segmentation or object detection, resulting in low efficiency of synthesized data enhancement. Second, existing methods indiscriminately learn and amplify the "style inconsistency" in the dataset due to subjective differences in manual labeling, resulting in inconsistent and blurred generated labels, introducing noise, and limiting the upper limit of the accuracy of downstream models. Therefore, how to make synthesized data shift from pursuing "good-looking" to pursuing "good-use", while ensuring the "high quality" of labeling and achieving "dynamic" in the training process, is a technical problem to be solved in the field. SUMMARY

[0003] The present application provides a visual image data enhancement and synthesis system based on a generative model to solve the technical problems of low task utility of synthesized data and unstable quality of generated labels in the prior art.

[0004] In view of the above problems, the present application provides a visual image data enhancement and synthesis system based on a generative model, comprising: a generation module for generating a synthesized data pair comprising a synthesized image and a synthesized label; a task utility evaluation module for inputting the synthesized image into a downstream task agent model to obtain a predicted label, and calculating a dynamic task utility loss based on the predicted label and the synthesized label; a style consistency evaluation module for calculating a label style consistency loss based on the synthesized label and a pre-calculated label style prototype; wherein the generation module is configured to update its own network parameters according to a composite loss function composed of the dynamic task utility loss and the label style consistency loss.

[0005] The technical scheme provided by the present application has at least the following technical effects or advantages: The present application turns the training target of the generator from the traditional visual fidelity to the task utility directly related to the final application by introducing a task utility loss directly driven by the downstream task agent model. This enables the system to tend to generate the data most valuable to the performance improvement of the downstream model, thereby reducing the amount of synthetic data required to enhance the performance of the downstream model of the same size and improving the overall efficiency of data enhancement.

[0006] The present application explicitly constrains the annotations output by the generator by introducing a style consistency loss based on an annotated style prototype. This mechanism effectively suppresses the learning of inconsistent annotation styles in the source data set due to human subjective differences, making the generated synthetic annotations more stable and clear in key details such as edge morphology, reducing noise introduced by annotation quality problems, and helping to improve the precision upper limit of downstream segmentation tasks. BRIEF DESCRIPTION OF DRAWINGS

[0007] Figure 1 A visual image data enhancement and synthesis system architecture based on a generative model. DETAILED DESCRIPTION

[0008] The above technical solutions will be described in detail below in conjunction with the drawings and specific embodiments to better understand the above technical solutions. Obviously, the described embodiments are only part of the embodiments of the present application, not all embodiments of the present application, and it should be understood that the present application is not limited to the example embodiments for explaining the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application. In addition, it should be noted that, for convenience of description, only the parts related to the present application are shown in the drawings, not all.

[0009] Please refer to Figure 1 A visual image data enhancement and synthesis system based on a generative model, comprising: a generation module for generating a synthetic data pair comprising a synthetic image and a synthetic annotation; a task utility evaluation module for inputting the synthetic image into a downstream task agent model to obtain a predicted annotation, and calculating a dynamic task utility loss based on the predicted annotation and the synthetic annotation; a style consistency evaluation module for calculating an annotation style consistency loss based on the synthetic annotation and a pre-calculated annotation style prototype; wherein the generation module is configured to update its own network parameters according to a composite loss function composed of the dynamic task utility loss and the annotation style consistency loss.

[0010] Embodiment: Efficient synthetic system applied to high-resolution colorectal polyp endoscopic image segmentation data This embodiment is deployed in a specific clinical auxiliary diagnosis engineering scenario: using artificial intelligence models to analyze high-definition colonoscopy video streams in real time, automatically detect and accurately segment precancerous lesions of colorectal cancer, polyps, to assist doctors in diagnosis and reduce the rate of missed diagnosis.

[0011] However, when building this system, there are multiple challenges from the data level that are technically related, which existing data enhancement techniques cannot effectively solve: 1. Heterogeneity problem of multi-source devices: Medical centers usually use multiple types of endoscopic devices from different manufacturers (e.g., Olympus, Fuji, Pentax). Directly using models trained on these mixed data, the performance will drop sharply when facing images generated by specific devices that the model has not "seen" before.

[0012] 2. High variability of lesion morphology and scarcity of edge cases: Colorectal polyps have various shapes, from easily identifiable protruding types (Yamada I-III types) to flat and depressed types (Yamada IIb type) with blurred boundaries with the surrounding mucosa. The latter is a type that is easily missed in clinical practice. In existing data sets, the number of samples of such edge cases is small, leading to insufficient recognition ability of traditional models. According to the baseline test, a model trained on a regular data set has a 20% miss rate for flat polyps, which constitutes a performance limitation in clinical application scenarios.

[0013] 3. Subjective inconsistency of manual annotation: Even experienced gastroenterologists have unavoidable minor deviations when pixel-level outlining the boundaries of the same polyp image. This "annotation style" difference between experts is indiscriminately learned by traditional generative models, resulting in unstable and artifact-laden annotation edges, introducing harmful noise for the training of downstream segmentation models and limiting the upper limit of their accuracy.

[0014] A visual image data enhancement and synthesis system based on a generative model according to the present invention, in a complete engineering application, its life cycle can be divided into the following three logically continuous stages: Stage 1: One-time offline preparation process, mainly including: Data preparation and preprocessing: Collect original endoscopic images from multi-source devices and their corresponding expert annotations, perform uniform size normalization (e.g., scale to 512x512 pixels), data cleaning, and format conversion.

[0015] Annotation style prototype computation: Run the "annotation style prototype computation unit" to statistically analyze all real annotations and generate and persist a standard annotation style prototype P.

[0016] Model loading and configuration: Load the pre-trained and frozen "downstream task agent model B'"; initialize the network parameters of the "generator A'"; load and configure the "curriculum learning scheduling module", set its initial state and related thresholds.

[0017] Phase two: This phase is an automated, closed-loop iterative process: The system starts an unsupervised training loop. In this loop, the "generator A'" continuously generates synthetic "image-annotation pairs" under the dynamic guidance of the "curriculum learning scheduling module".

[0018] Each batch of generated synthetic data is used to calculate "dynamic task utility loss L_task" and "annotation style consistency loss L_style", and the generator's parameters are updated accordingly.

[0019] This process continues until the curriculum learning scheduling module determines that the learning of the downstream task agent model B' has reached saturation, or the pre-set amount of synthetic data is reached.

[0020] Phase three: This phase focuses on the management and utilization of synthetic data assets: All generated "image-annotation pairs" are not discarded at will, but are structuredly stored in a database or file system along with a set of rich metadata.

[0021] Metadata at least includes: a unique sample ID, the timestamp when the sample is generated, the "curriculum learning phase" it belongs to (basic construction, hard core mining or leak repair), the calculated L_task and L_style loss value, and the version number of the generator model that generates it.

[0022] This storage mechanism with rich metadata allows subsequent researchers to conduct fine-grained data tracing and analysis. For example, all samples generated in the "hard core mining period" and with the highest L_task value can be filtered out to be used for intensive training or testing of the final deployed segmentation model, thereby maximizing data value.

[0023] So far, a macro framework from problem definition, environment configuration to complete life cycle planning has been constructed. The next part goes deep into the internal of each technical component of the framework and describes it in detail.

[0024] In a specific embodiment of the present invention, the generator A' is a deep neural network module whose function is defined as: receiving a random noise vector of a predetermined dimension as input, and transforming it through a series of neural network layers, and finally outputting a multi-channel tensor containing paired, size-matched synthetic images and their pixel-level annotations.

[0025] In a preferred embodiment, the internal network structure of the generator A' adopts a U-Net architecture, which is selected for its superior ability to preserve spatial details in medical image segmentation tasks.

[0026] The data structure processed by generator A' is defined as follows: its input, the random noise vector z, is a two-dimensional tensor sampled from a standard normal distribution with a data shape of (B, 128), where B represents the batch size and 128 is the preset noise dimension. Its output, the synthetic data pair G_pair, is a four-dimensional tensor with a data shape of (B, 2, 512, 512). In the output tensor, the slice with index 0 along the second dimension (channel dimension), G_pair[:, 0, :, :], is defined as the synthesized endoscopic image G_img, whose pixel values ​​are normalized to the interval [0, 1]; the slice with index 1, G_pair[:, 1, :, :], is defined as the synthesized polyp segmentation annotation G_mask, which passes through a Sigmoid activation function at the end of the network so that each pixel value in the output tensor also falls in the interval [0, 1]. This value represents the probability that the pixel point belongs to the polyp area in a physical sense.

[0027] The downstream task proxy model B' is another deep neural network module. Its role is defined as a stable performance evaluation proxy for simulating the real downstream segmentation task.

[0028] In a preferred embodiment, the internal network structure of the downstream task proxy model B' also adopts the U-Net architecture, but its network depth or number of channels is reduced compared to the final deployed segmentation model to achieve higher computational efficiency.

[0029] The data structure processed by the downstream task agent model B' is defined as follows: its input is the composite image G_img separated from G_pair. Its output is a three-dimensional tensor of data form (B, 1, 512, 512), defined as the predicted segmentation mask Pred_mask. This tensor is also processed by the Sigmoid activation function and represents the segmentation prediction result of model B' for the input image G_img.

[0030] The annotation style prototype module is a functional module responsible for defining and measuring the standard annotation style in a statistical sense. The module logically consists of an annotation style prototype calculation unit and an online constraint unit.

[0031] The core data structure of the annotation style prototype module, namely the annotation style prototype P, is defined as a three-dimensional binary tensor with a data form of (1, 1, 512, 512), whose internal element values are only 0 or 1, representing the "consensus" or "average morphology" of all real expert annotations in the data set.

[0032] The course learning scheduling module is an algorithm process module, and its function is defined as: by periodically monitoring the learning state of the model B', and according to the preset rules, dynamically selecting and applying different training strategies to guide the generator A'.

[0033] The course learning scheduling module manages a key data structure, namely the feature history buffer. This data structure is defined as a first-in-first-out (FIFO) queue, and its capacity is set to store the feature vectors extracted by the synthetic image G_img at the end of the encoder path of the model B' in the last K training batches.

[0034] For example, if K=100, the feature vector dimension output by B' encoder is 512, and the batch size is 4, then the logical form of this buffer is a queue that can accommodate 400 feature vectors with a dimension of 512.

[0035] Further, the offline calculation unit: the generation algorithm of the annotation style prototype P is executed offline once before the system starts training, and its step-by-step logical process is as follows: Step 1: initialization. The system first creates a two-dimensional matrix in memory with the same size as the input annotation image and a data type of float, and initializes all its elements to zero. This matrix is named prob_map here.

[0036] Step 2: accumulation. The system will iterate through a set of N real expert annotations C_mask. For each annotation image in the set, the system will perform a pixel-by-pixel addition operation between it and prob_map, and store the result back to prob_map. After this step, each pixel value in prob_map is equal to the sum of all real annotations at that pixel position.

[0037] Step 3: averaging. The system divides each pixel value in prob_map by the total number of real annotations N. After this step, each pixel value in prob_map represents the statistical probability of the pixel position being marked as "polyp" (i.e. value 1) in all real annotations.

[0038] Fourth step: thresholding and generation. The system creates a zero matrix of the same size as the prototype P for storing the final result. Then, the system iterates through each pixel of the prob_map. If the probability value of a pixel is greater than a pre-defined threshold (in a preferred embodiment, the threshold is 0.5), the corresponding position in the prototype P is assigned a value of 1; otherwise, it remains 0. After this step, the binary matrix is the desired annotation style prototype P.

[0039] Further, the internal logic of a single training iteration: the following is the step-by-step logic of the interaction between the various algorithmic modules within the system during a training iteration.

[0040] Course learning scheduling module: learning progress monitoring and phase determination, triggered at the beginning of each training round (Epoch).

[0041] First step: the course learning scheduling module first loads the downstream task agent model B' and performs a complete evaluation on an independent, non-training pure real validation set D_val. Through this evaluation, the Dice Similarity Coefficient of the model B' on the validation set is calculated, and this value is named the current performance indicator P_current. Then, by subtracting P_current from the historical performance indicator P_prev recorded in the last round, the performance growth rate P_growth_rate is obtained. The current performance indicator P_current and the performance growth rate P_growth_rate are equivalent quantitative indicators, which together constitute the comprehensive judgment basis for the learning state of the downstream task agent model.

[0042] Second step: the course learning scheduling module determines the course phase to be used in the current round according to a set of pre-defined rules with clear logical order. The judgment logic is as follows: Rule (a): if it is the first training round, or if the value of P_current is less than a pre-defined basic performance threshold (for example, 0.70), the course learning scheduling module sets the current course phase current_phase to "1", i.e. "basic construction period strategy".

[0043] Rule (b): if rule (a) is not met, and the value of P_current is greater than or equal to the basic performance threshold, and the value of P_growth_rate is greater than a pre-defined stagnation judgment threshold (for example, 0.001), the course learning scheduling module sets current_phase to "2", i.e. "hard core mining period strategy".

[0044] Rule (c): If the preceding rules are not satisfied, and the value of P_growth_rate is less than or equal to a stagnation judgment threshold, or, if the loss distribution entropy H_loss_prev recorded in the last round is less than a preset diversity warning threshold (for example, 0.2), the curriculum learning scheduling module sets current_phase to "3", i.e., the "leakage checking and filling period".

[0045] In all other cases, the curriculum learning scheduling module sets current_phase to "2" by default.

[0046] Further, the dynamic task utility loss L_task is calculated, and the algorithm is executed in each training batch (Batch).

[0047] First step: The system first calculates the Davies similarity coefficient between the predicted segmentation map Pred_mask of model B' and the "pseudo ground truth" label G_mask provided by generator A', and then subtracts 1 from the coefficient value to obtain the basic Davies loss base_dice_loss.

[0048] Second step: The system selects one of the following three calculation paths according to the current current_phase: Path (a) - Stage 1: If the current is "basic construction period", the calculation method of L_task is: first calculate the Davies similarity coefficient (i.e. 1 - base_dice_loss), then take its natural logarithm, and finally take the reciprocal. The calculation method aims to guide the generator to generate samples that enable B' to obtain a high Davies coefficient.

[0049] Path (b) - Stage 2: If the current is "hard core mining period", the calculation method of L_task is: take the base_dice_loss value to the power of a preset focus parameter γ (for example, 2.0), and then multiply it by base_dice_loss itself. The calculation method aims to weight the high-loss, difficult samples.

[0050] Path (c) - Stage 3: If the current is the "Leakage Filling Period", the calculation of L_task contains two parts. The first part is the same as the calculation of path (b). The second part is the calculation of the diversity regularization term, whose flow is as follows: first, extract the feature vector of the current synthesized image G_img from the end of the encoder path of model B'; then, calculate the average cosine similarity between the feature vector and all the historical feature vectors stored in Feat_History_Buffer, to get a similarity penalty value; finally, weight the penalty value with a pre-set regularization coefficient β (for example, 0.1). The final value of L_task is the sum of the two parts. At the same time, the system stores the feature vector of the current batch in Feat_History_Buffer.

[0051] Further, the annotation style consistency loss L_style is calculated, which is also executed in each training batch.

[0052] First step: the system respectively applies a standard Sobel edge detection operator to the segmentation annotation G_mask output by generator A' and the pre-calculated annotation style prototype P. The output of this operation is two images representing the edge gradients of G_mask and P respectively.

[0053] Second step: the system calculates the L1 norm distance between the two edge gradient images, that is, the sum of the absolute values of the differences between the corresponding pixels of the two images. The sum is the annotation style consistency loss L_style.

[0054] Based on the detailed description of the system's core components and algorithms above, this part will further elaborate the calibration method of key parameters, design considerations, and technical discussions constructed to support dependent claims in the present invention.

[0055] The calibration method of these key parameters will be described in detail below.

[0056] Calibration method of composite loss function weight coefficients λ1 and λ2: Parameter definition: λ1 is the weight coefficient of "dynamic task utility loss L_task", and λ2 is the weight coefficient of "annotation style consistency loss L_style". These two coefficients jointly determine the balance between "pursuing task utility" and "maintaining annotation quality" for generator A'.

[0057] Calibration methodology: Magnitude alignment: in the early stage of training, first run the system for several iterations independently, but do not update the parameters, only observe and record the average magnitude of the two loss values L_task and L_style. For example, if the magnitude of L_task is in the order of 10 -1left and right, while the magnitude of L_style is in the order of 10 -3 left and right, the weights should be set so that the expected magnitude of λ1 * L_task and λ2 * L_style are roughly equal. This is an initial step to guarantee the stability of the gradient.

[0058] Grid search and agent evaluation: Based on the magnitude alignment, a two-dimensional grid search space is set. For example, the search range of λ1 is [0.5, 1.0, 2.0], and the search range of λ2 is [0.01, 0.05, 0.1]. For each set of (λ1, λ2) values in the grid, a shortened version (e.g., 20% of the total training time) of training is performed.

[0059] Optimal value determination: After each shortened version of training is completed, an independent evaluation model is trained using the generated synthetic data, and the performance (such as the Dice coefficient) of the evaluation model on the final test set is recorded. Select the (λ1, λ2) combination that can achieve the highest performance of the evaluation model as the optimal parameter configuration for this application scenario.

[0060] Through the above calibration method, in a preferred embodiment, the weight coefficients suitable for the colorectal polyp segmentation scenario of the present application can be set as λ1 = 1.0 and λ2 = 0.05.

[0061] Calibration method of internal parameters of curriculum learning scheduling module: Parameter definition: including the focus parameter γ in stage two L_task; the regularization coefficient β and feature history buffer size K in stage three L_task; and the performance threshold and entropy threshold for stage judgment.

[0062] Calibration methodology (taking the focus parameter γ as an example): Parameter understanding: First, it should be understood that the larger the value of γ, the higher the attention of the system to "hard core" samples, but too large γ may lead to unstable training.

[0063] Single-variable sensitivity analysis: Fix all other parameters and select a set of candidate values for γ, such as [1.0, 1.5, 2.0, 2.5, 3.0].

[0064] Performance curve observation: For each candidate value, run a complete training process. During the training process, not only the final downstream task performance should be focused on, but also the smoothness of the training loss curve and the diversity of the generated samples should be observed.

[0065] Trade-off selection: Select a gamma value that can improve the downstream performance, without causing dramatic fluctuations in training loss, and without leading to a significant decrease in the diversity of generated samples. This value is the optimal trade-off point. For beta, K and other threshold parameters, similar single-variable sensitivity analysis methods can be used for calibration.

[0066] According to the above methodology, a suitable focus parameter gamma value is about 2.0; an exemplary regularization coefficient beta can be set to 0.1; and the feature history buffer size K can be set to store the feature data of the last 100 batches.

[0067] To further illustrate the design motivation of the specific technical features in the present application and the technical advantages brought by them, the following two key technical points are described in detail.

[0068] In a preferred embodiment of the present application, the generation method of the labeled style prototype P, that is, the method of pixel-level probability statistics on all real labels and thresholding of the generated probability map, has the following technical advantages compared to other simplified methods (such as direct arithmetic averaging on binary labeled images). First, the present method has higher robustness to noise and abnormal labels that may exist in the data set. Specifically, the simple arithmetic average method is extremely sensitive to outliers. For example, if there are a few bad labels in the data set due to operation errors or inaccurate expert judgment, which are morphologically abnormal or positionally offset, these bad labels will have an excessive and disproportionate impact on the average result, resulting in the final prototype P being contaminated. Second, the probability map method used in the present application is essentially a voting mechanism. Each pixel value in prob_map represents the "confidence" that the point is identified as a "polyp" by all experts. The pixel points in an isolated and accidental "bad label" have very low weight in the voting process, and their influence will be "overwhelmed" by a large number of normal labels. The thresholding step (e.g., threshold 0.5) that follows is equivalent to a "majority over minority" decision-making process, which can effectively filter out the influence of these noise and outliers. Therefore, this generation method can ensure that the final generated prototype P has higher statistical robustness and credibility.

[0069] In another preferred embodiment of the present application, the curriculum learning scheduling module adopts a "reverse target guiding" strategy in the initial training stage, i.e., in the basic construction period, the calculation method of the dynamic task utility loss is set to guide the generator to generate samples that can improve the performance of the downstream task agent model. The design consideration of this strategy is to avoid instability in the initial training stage. In the initial stage of training, the output of the generator A' is random in structure and content. If it is forced to generate difficult samples at this time, it may cause gradient information confusion, even cause gradient vanishing or explosion, so that the training cannot converge. The "reverse target guiding" strategy encourages the generator A' to generate simple samples that can be easily and correctly segmented by the model B' by establishing a stable positive feedback loop in the initial stage, which provides a stable and clear gradient signal for the parameter update of A', so that it can quickly learn to generate images with reasonable macrostructure. This strategy reduces the task difficulty in the initial stage, builds a reliable training starting point for the entire system, and lays a solid foundation for challenging difficult samples in the subsequent "hard core mining period".

[0070] After completing the detailed description of the core technical solutions, parameter calibration methods, and key design considerations of the present application, this part will further describe the optional implementation paths and application scope of the solutions.

[0071] Several core functional modules of the present application can be implemented through various specific algorithm paths, and these paths all embody the core technical concept of the present application.

[0072] For example, for the calculation of the "annotation style consistency loss L_style", in addition to the method based on the spatial domain edge L1 distance adopted in the foregoing embodiments, there are other feasible calculation methods. One alternative implementation is to measure the annotation style in the frequency domain. This method can include the following steps: performing two-dimensional fast Fourier transform (FFT) on the generated annotation G_mask and the style prototype P, and calculating the distance (e.g., L2 norm distance) between the amplitude spectra of the two to serve as L_style. Another alternative implementation is to calculate based on the structural similarity index (SSIM) of the perception model, in which case L_style can be defined as 1 - SSIM(G_mask, P).

[0073] For example, for the "learning progress measurement of the curriculum learning scheduling module", in addition to the method of evaluating performance indicators on an independent validation set adopted in the foregoing embodiments, there is also an alternative measurement method based on the internal state of the model. One alternative implementation is to monitor the average gradient norm of the network parameters of the downstream task agent model B' during training. When the moving average of this gradient norm continuously falls below a preset stable state threshold for multiple consecutive training rounds, it can also be used as a basis for triggering the curriculum stage conversion.

[0074] Although the medical image segmentation scene of the colorectal cancer polyp is described in detail in the embodiments of the present application, the core technical solution, i.e., the method of dynamically guiding data synthesis by combining task utility, style consistency and curriculum learning, is not limited to this specific application. The underlying technical logic of the solution has good universality.

[0075] The above is only a preferred embodiment of the present application, and does not limit the present application in any form. Although the present application has been disclosed in a preferred embodiment, it is not intended to limit the present application, and any person skilled in the art can make some changes or modifications to the above-mentioned methods and technical contents within the scope of the technical solution of the present application to make equivalent embodiments with equivalent changes. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present application without departing from the content of the technical solution of the present application still belongs to the protection scope of the technical solution of the present application. The present application is limited by the claims and the entire scope and equivalents thereof.

Claims

1. A visual image data enhancement and synthesis system based on a generative model, characterized by: include: a generation module for generating a synthetic data pair comprising a synthetic image and a synthetic annotation; a task utility evaluation module configured to input the synthesized image into a downstream task proxy model to obtain a predicted annotation, and calculate a dynamic task utility loss based on the predicted annotation and the synthesized annotation; a style consistency evaluation module for calculating a labeling style consistency loss based on the synthesized labeling and a pre-computed labeling style prototype; The generation module is configured to update its own network parameters according to a composite loss function consisting of the dynamic task utility loss and the annotation style consistency loss.

2. A visual image data enhancement and synthesis system based on a generative model as claimed in claim 1, characterized in that: The system also includes: a course learning scheduling module, which is used to monitor the learning status of the downstream task agent model and, based on the learning status, control the task utility evaluation module to select one of at least two preset training strategies to determine the specific calculation method of the dynamic task utility loss.

3. The visual image data enhancement and synthesis system based on a generative model according to claim 2, characterized in that: The course learning scheduling module is configured to: when the learning state of the downstream task agent model is lower than a basic performance threshold, adopt a basic construction period strategy, wherein the calculation method of the dynamic task utility loss is set to encourage reducing the task loss of the downstream task agent model; when the learning state of the downstream task agent model is higher than the basic performance threshold, adopt a hard core mining period strategy, wherein the calculation method of the dynamic task utility loss is set to weight samples that cause high task loss to the downstream task agent model.

4. The visual image data enhancement and synthesis system based on a generative model according to claim 1, characterized in that: The system further includes: a labeling style prototype calculation unit, configured to perform pixel-level probability statistics on multiple real labels in a real label set to generate a probability map, and perform thresholding on the probability map to generate a labeling style prototype for use by the style consistency evaluation module.

5. The visual image data enhancement and synthesis system based on a generative model according to claim 1 or 4, characterized in that: The style consistency evaluation module is configured to: extract edges of the synthetic annotation and the annotation style prototype respectively; and calculate a distance between the edge of the synthetic annotation and the edge of the annotation style prototype as the annotation style consistency loss.

6. The system for enhancing and synthesizing visual image data based on a generative model according to claim 1, wherein: The downstream task proxy model included in the task utility evaluation module is a pre-trained segmentation model whose network parameters are frozen during the training process of the generation module.

Images