An AI model performance-oriented training data intelligent synthesis and enhancement system
By generating structured failure fingerprints through failure case diagnosis of AI models, generating targeted data using a prompt word injection module, and filtering high-value data through an uncertainty assessment and screening module, an automated closed-loop process is constructed. This solves the blindness problem of data synthesis systems in existing technologies and achieves rapid improvement in model performance and efficient utilization of resources.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG ANYIXIN TECH CO LTD
- Filing Date
- 2025-08-20
- Publication Date
- 2026-05-15
AI Technical Summary
Existing data synthesis systems lack feedback on the current performance status of a specific AI model to be trained, resulting in a mismatch between the generated data content and the capabilities that the model needs to compensate for, inefficient use of computing resources, and slow model performance convergence.
The failure case diagnosis module generates structured failure fingerprints, the prompt word injection module generates targeted data, and the uncertainty assessment and screening module filters high uncertainty data to build an automated closed-loop process and dynamically adjust data augmentation strategies.
Ensure the data generation is targeted, make efficient use of computing resources, rapidly improve model performance, and achieve continuous optimization of model capabilities.
Smart Images
Figure CN121071479B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence technology, and in particular to an intelligent synthesis and enhancement system for training data aimed at improving the performance of AI models. Background Technology
[0002] With the rapid development of deep learning technology, artificial intelligence (AI) models are increasingly widely used in many fields such as computer vision and natural language processing. The performance of a model largely depends on the quantity and quality of training data. However, in practical applications, obtaining large-scale, diverse, and high-quality training data that covers all corner cases is costly, and in some cases, an impossible task.
[0003] To address the problem of data scarcity, data augmentation and data synthesis techniques have emerged. Traditional data augmentation methods, such as geometric transformations (rotation, cropping) and color dithering, can expand datasets to some extent, but the diversity of the generated data is limited, making it difficult to create truly novel and challenging samples for models. In recent years, generative AI techniques, represented by Generative Adversarial Networks (GANs) and diffusion models, have been used for data synthesis, capable of generating high-quality, realistic data.
[0004] However, existing data synthesis systems typically lack feedback on the current performance state of a specific AI model being trained when generating training samples. These systems generally employ random or fixed data augmentation strategies to expand the dataset, and the generated data content is not directly related to the specific performance deficiencies exhibited by the AI model under specific data distributions or challenging scenarios. This undirected data augmentation method results in a large amount of synthetic data that does not match the capabilities that the model needs to compensate for. Therefore, when used for model retraining, it has limited effect on improving the model's performance metrics (such as accuracy and recall) in these specific challenging scenarios, and also causes inefficient use of computing resources and slow model performance convergence. Summary of the Invention
[0005] This invention provides an intelligent synthesis and enhancement system for training data aimed at improving the performance of AI models, in order to solve the problems of blind data enhancement and inability to specifically compensate for the shortcomings of AI model capabilities in existing technologies.
[0006] In view of the above problems, the present invention provides an intelligent synthesis and enhancement system for training data aimed at improving AI model performance, characterized in that the system comprises:
[0007] The failure case diagnosis module is used to analyze historical prediction failure cases of the target AI model to generate a structured failure fingerprint; the failure fingerprint represents one or more data features that caused the target AI model to fail to predict.
[0008] The prompt word injection module, connected to the failure case diagnosis module, is used to convert the structured failure fingerprint into enhanced prompt words;
[0009] A data generation engine, connected to the prompt injection module, is used to generate a data pool containing candidate data related to the data features represented by the failed fingerprint, based on the enhanced prompts.
[0010] An uncertainty assessment and screening module, connected to the data generation engine and the target AI model, is used to assess the uncertainty of each candidate data in the data pool using the target AI model, and to screen out data with high uncertainty from the data pool as enhanced training data based on a preset screening strategy.
[0011] The technical solution provided in this application has at least the following technical effects or advantages:
[0012] This invention ensures the targeted nature of data generation from the source by diagnosing model failure cases, completely eliminating the blind approach of traditional data augmentation. All generated samples directly serve to compensate for known performance defects in the model, ensuring efficient use of computing resources.
[0013] By employing "source guidance" in the prompt injection module and targeted filtering in the uncertainty assessment and screening module, dual safeguards are achieved, ensuring that every piece of data ultimately used for training has extremely high marginal benefits for the current state of the model, thereby accelerating model convergence and improving performance.
[0014] The entire "diagnosis-generation-screening" process can be executed automatically and cyclically. The system can adaptively adjust the data augmentation strategy according to the dynamic changes in model performance, thereby achieving continuous optimization of model capabilities. Attached Figure Description
[0015] Figure 1 This is a schematic diagram of the architecture of an intelligent synthesis and enhancement system for training data aimed at improving the performance of AI models, according to the present invention. Detailed Implementation
[0016] This invention provides an intelligent training data synthesis and enhancement system for AI model performance, aiming to solve the technical problem that training data cannot accurately and efficiently compensate for the shortcomings of existing AI models during training. This system constructs an automated closed-loop workflow of "diagnosis-generation-screening-retraining," which can dynamically discover performance defects of the model in specific data patterns and synthesize high-value training data in a targeted manner, thereby achieving continuous and targeted optimization of model capabilities.
[0017] The above technical solutions will now be described in detail with reference to the accompanying drawings and specific embodiments to provide a better understanding of them. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. It should be understood that the present invention is not limited to the exemplary embodiments used only to explain the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention. Furthermore, it should be noted that, for ease of description, only the parts related to the present invention are shown in the drawings, not all of them.
[0018] Please see Figure 1 A training data intelligent synthesis and enhancement system for improving AI model performance, the system comprising:
[0019] The failure case diagnosis module is used to analyze historical prediction failure cases of the target AI model to generate a structured failure fingerprint; the failure fingerprint represents one or more data features that caused the target AI model to fail to predict.
[0020] The prompt word injection module, connected to the failure case diagnosis module, is used to convert the structured failure fingerprint into enhanced prompt words;
[0021] A data generation engine, connected to the prompt injection module, is used to generate a data pool containing candidate data related to the data features represented by the failed fingerprint, based on the enhanced prompts.
[0022] An uncertainty assessment and screening module, connected to the data generation engine and the target AI model, is used to assess the uncertainty of each candidate data in the data pool using the target AI model, and to screen out data with high uncertainty from the data pool as enhanced training data based on a preset screening strategy.
[0023] Failure case diagnosis stage: Quantify and summarize the weak areas of the model's predictive performance, automatically and systematically discover common data patterns that lead to prediction failures from a large number of historical behaviors of the model, and transform these abstract patterns into machine-readable, structured failure fingerprints.
[0024] In a specific application scenario, the operation process for this step is as follows:
[0025] Step 1: The system first needs a clearly defined evaluation target. Specifically, it loads the target AI model to be optimized and specifies a validation dataset for evaluating its performance. The system automatically invokes the model to make predictions for each image in the validation set and compares the model's predictions with the ground truth labels provided in the dataset. All cases where the predictions do not match the ground truth labels are collected by the system, forming an initial failure case set. This set forms the data foundation for all subsequent analyses.
[0026] The second step: In order for the machine to understand the commonalities among these failure cases, each failed image sample must be transformed into a quantifiable data representation that can be analyzed. To this end, the system generates a multi-dimensional feature vector for each failure case, which comprehensively describes the image from both low-order physical properties and high-order semantic content levels.
[0027] The specific method is as follows: the system performs low-order visual feature extraction and high-order semantic feature extraction in parallel, and then concatenates the two feature vectors to form a composite feature vector with more information content.
[0028] Extracting low-level visual features: This step aims to capture the physical and statistical properties of the image, which are often related to factors that directly affect the model's perception, such as lighting, weather, and shooting quality.
[0029] The specific method is as follows: the system calls a mature image processing function library (such as OpenCV) to perform calculations on the image samples of each failed case. For example:
[0030] Brightness value: The image is converted to grayscale, and then the arithmetic mean of the intensities of all pixels is calculated. This value visually reflects the overall brightness of the image.
[0031] Contrast ratio: Calculated by the standard deviation of pixel intensity in a grayscale image. A larger standard deviation means a more significant difference between bright and dark areas in the image, resulting in higher contrast.
[0032] Sharpness value: The Laplacian operator is applied to the image to enhance the edges, and then the variance of the resulting image is calculated. The larger the variance, the richer the edge details of the image, and the clearer it appears subjectively.
[0033] These calculated scalar values together form a low-dimensional feature vector that describes the physical state of the image.
[0034] Extracting high-order semantic features: This step aims to capture the abstract content of the image, scene context, and category information, which are the basis for the model to perform advanced cognition and judgment.
[0035] Specifically, the system inputs image samples into a powerful, pre-trained image encoder and obtains a high-dimensional vector as its output. To ensure the consistency and reproducibility of feature extraction, the encoder keeps its internal parameters fixed during the feature extraction process and does not perform any updates.
[0036] For example, in one embodiment, the encoder uses the ViT-L / 14 image encoder of the CLIP model. It has been pre-trained using contrastive learning on large-scale image-text pairs, and its output feature vectors contain rich visual semantic information and have a high degree of intrinsic correlation with natural language, laying the foundation for seamlessly converting diagnostic results into text prompts.
[0037] The third step: After transforming each failure case into a composite feature vector, the system needs to discover potential, clustered patterns in these high-dimensional data points.
[0038] Specifically, the system applies an unsupervised clustering algorithm to the set of composite feature vectors of all failed cases. Before clustering, all feature vectors are usually normalized (e.g., L2 normalization) to eliminate the influence of different feature dimensions.
[0039] For example, in one embodiment, the system employs DBSCAN (a density-based spatial clustering algorithm). This algorithm can discover clusters of arbitrary shapes and is insensitive to noise points. Its key parameter configurations are as follows:
[0040] Neighborhood radius eps: This parameter is determined using a k-distance graph method. Specifically, the distance from each feature point to its k-th nearest neighbor is calculated (where k can be set to 4 empirically), then all these distances are sorted and plotted as a curve. The distance value corresponding to the point where the slope of the curve changes most drastically (i.e., the threshold change point) is selected as the value of eps.
[0041] Minimum number of samples min_samples: This parameter can be set to a small value based on experience, such as 4, which means that a core object needs at least 4 neighbors to form a cluster.
[0042] Handling special cases: If the total number of failed cases to be processed is too small, for example, below a preset threshold (such as 20), clustering may not be statistically meaningful. In this case, the system can skip the clustering step and treat all cases as an independent, unified cluster for subsequent processing.
[0043] Step 4: Generate structured failure fingerprints. For each cluster formed by the clustering operation, the system needs to transform it into a multi-dimensional, human-interpretable, and machine-processable information carrier, namely, a "structured failure fingerprint".
[0044] Specifically, the system performs in-depth statistical analysis on the features of all samples within the cluster and encapsulates the analysis results into a JSON data object for subsequent modules to parse and use.
[0045] For example, a JSON object representing a failed fingerprint might contain the following key fields:
[0046] Low-order visual feature range: Summarize the common physical characteristics of this failure cluster.
[0047] The specific method is as follows: traverse the low-order visual feature vector of each sample in the cluster, and for each dimension in the vector (such as brightness, contrast, and sharpness), calculate the minimum and maximum values of all samples in the cluster in that dimension.
[0048] For example, the analysis result for a cluster might be {"brightness":{"min":0.1,"max":0.3},"contrast":{"min":0.2,"max":0.4}}. This numerical range reveals a strong correlation between this failure mode and a "low brightness, low contrast" environment.
[0049] Higher-order semantic feature centroid: Find the most representative semantic core for this failure cluster.
[0050] The specific method is as follows: calculate the mathematical centroid of all higher-order semantic feature vectors within the cluster, that is, the average vector of all vectors. This centroid vector is located at the center of the cluster in the feature space and can be regarded as the "average paradigm" of this type of failure mode at the semantic level.
[0051] Keyword list: Labels the failure cluster with human-understandable, descriptive semantic tags.
[0052] Specifically, the system adopts a primary / backup strategy to ensure its versatility.
[0053] The main strategy is as follows: When the sample itself contains metadata in text form (such as file name, tag, description, etc.), the system summarizes the metadata of all samples in the cluster and extracts the most frequent and representative nouns and adjectives through techniques such as word frequency statistics.
[0054] Alternative strategy: When the sample does not contain metadata, the system invokes a pre-trained, independent image annotation model, taking the high-order semantic feature centroid vector calculated in the previous step, or several representative sample images closest to the centroid in the cluster, as input to generate a descriptive natural language text. Subsequently, the system extracts high-frequency nouns and adjectives from the generated text.
[0055] For example, through this step, a failed cluster can be labeled with a list of keywords such as ["night", "rain", "pedestrian"].
[0056] This completes the failure case diagnosis phase. The one or more structured failure fingerprints it outputs provide input for subsequent phases.
[0057] II. Prompt Word Construction Stage: Translating and Instructing Failure Patterns. This stage automatically and unambiguously "translates" the machine-readable failure patterns extracted in the previous step into highly targeted natural language instructions that can be directly understood and executed by large-scale generative AI models, i.e., "enhanced prompt words".
[0058] This step is completed independently by the prompt word injection module, and its internal processing flow is as follows:
[0059] Step 1: Parse Failure Fingerprints. The module first receives one or more JSON-formatted failure fingerprint objects from the failure case diagnosis module. Then, the module performs a parsing operation on each fingerprint object to extract its key fields.
[0060] For example, the main extraction is a "keyword list" field containing multiple strings, and a "low-order feature parameter" object containing multiple low-order visual features and their corresponding statistical intervals.
[0061] Step 2: Rule-based feature mapping to natural language. This transforms structured data into semantic descriptions. The core component of this module is a pre-defined, extensible mapping rule library that defines the mapping relationships from specific structured features to concrete descriptive natural language phrases.
[0062] The specific method is as follows: the module iterates through each piece of feature information parsed and searches for matching rules in the rule base. This process is based on logical judgments of conditions.
[0063] For example, the rule base can be implemented as a data structure containing multiple rule entries, where each rule defines a trigger condition and an output phrase:
[0064] Brightness rules:
[0065] Triggering condition: Check if the maximum value of the "brightness" feature range in the failed fingerprint is lower than a preset threshold (e.g., 0.35).
[0066] Output phrase: If the condition is met, output "darkscene,dimlighting,nightvisionphotostyle".
[0067] Contrast rules:
[0068] Trigger condition: Check if the maximum value of the "contrast" feature range is lower than a preset threshold (e.g., 0.4).
[0069] Output phrase: If satisfied, output "lowcontrast,foggy,hazyenvironment".
[0070] Keyword rules:
[0071] Trigger condition: Check if the "keyword list" contains a specific word (e.g., "rain").
[0072] Output phrase: If included, output "rainyweather,wetsurfaces,waterreflections".
[0073] The system will perform all these checks and collect the output phrases generated by all matching rules to form a descriptive phrase set.
[0074] Step 3: Dynamically construct enhanced prompts. The parsed keywords and rule-generated descriptive phrases are injected into a base template to form the final enhanced prompts.
[0075] Specifically, the module selects one from one or more preset basic prompt word templates. These templates are text strings containing placeholders used to standardize the structure of the final output.
[0076] For example, a template could be: "ahigh-quality,photorealisticimageofa{object},{scene_description},{style_and_lighting}".
[0077] Subsequently, the module performs an injection operation, filling the corresponding placeholders in the template with all the keywords from the "keyword list" parsed in the first step, as well as all the descriptive phrases collected in the second step, through string formatting or concatenation.
[0078] Through this process, the module dynamically constructs and outputs one or more rich and highly targeted enhanced prompts, which are then passed to the data generation engine.
[0079] III. Data Generation Stage: Visualizing and Creating Difficult Samples. This stage utilizes the content creation capabilities of large-scale generative AI models to visualize the natural language instructions generated in the previous step, which contain information about specific failure scenarios, into a batch of high-quality image data that is highly relevant to the failure scenarios.
[0080] The specific processing flow for this step is as follows:
[0081] Step 1: Receive and configure the generation task. The data generation engine interacts with other parts of the system through a well-defined API interface. It receives requests from the prompt injection module, the core payload of which is one or more enhanced prompt strings.
[0082] Specifically, in addition to the prompt, the request may also include a series of parameters for fine-grained control of the generation process, which together constitute a complete generation task configuration.
[0083] Step 2: Invoke the underlying model to perform generation. Based on the received generation task configuration, the engine initiates one or more generation calls to the underlying generative AI model service. This model can be a Transformer-based model or a diffusion model (e.g., StableDiffusion).
[0084] For example, when using the StableDiffusion model, to ensure the quality and reproducibility of the generated results, the configuration parameters for the generation call can include:
[0085] Generate Quantity Parameter: An integer specifying the total number of images to be generated in this round of the task. For example, it can be set to 16 to obtain sufficiently diverse samples in a single call while controlling computational overhead.
[0086] Image resolution parameter: Defines the size of the output image. For example, it can be configured to generate an image of 1024x768 pixels to meet the input resolution requirements of most computer vision tasks.
[0087] Guidance scale parameter: Controls the degree to which the generated image matches the enhanced cue description. For example, it can be set to 7.5. This value is a commonly used balance between the creativity of the generated image and the accuracy of following instructions.
[0088] In addition, other parameters, such as sampling steps and random seed, can be configured to further control the generation process.
[0089] Step 3: Collecting and Organizing the Generation Results. After the underlying generative AI model completes its calculations and returns the generated image data, the data generation engine is responsible for collecting and organizing these results. This step includes checking the execution status of each generation task, handling possible anomalies such as generation failures or timeouts, and extracting all successfully generated image data.
[0090] Finally, the engine organizes all successfully collected image data into a structured dataset, known as the "candidate data pool," and outputs it as the processing result to the next module.
[0091] IV. Data Value Assessment and Screening: Refining and filtering high-value samples. The principle is that not all synthetic data has equal training value for the model; the most valuable data are those samples that the current model finds most difficult to accurately judge.
[0092] This step is completed by the uncertainty assessment and screening module, and its processing flow is as follows:
[0093] Step 1: Sample-by-sample inference. The module first receives the candidate data pool from the data generation engine and loads the latest version of the target AI model. Then, the module performs a traversal operation, taking each synthetic data sample in the data pool as input and calling the target AI model to perform a forward propagation inference once.
[0094] Step 2: Quantifying Uncertainty. For each inference, the module obtains the model's predicted output, typically a probability distribution vector representing the sample's belonging to various preset categories. The core task of the module is to transform this probability vector into a scalar value that measures the highest degree of uncertainty the model possesses, i.e., an uncertainty index.
[0095] For example, calculating information entropy:
[0096] The specific calculation process is as follows: For each synthetic sample that passes the initial screening, input it into the current model to be trained for forward inference to obtain the prediction probability vector p = (p1, p2, ..., pC);
[0097] Calculate information entropy as a measure of uncertainty:
[0098]
[0099] The higher the entropy value, the flatter the probability distribution (i.e., the closer the predicted probabilities of each category are), indicating that the model's classification result for the sample is more uncertain.
[0100] For example, other optional indicators:
[0101] Variance of prediction results: The variance of the prediction results is calculated by enabling the Dropout layer in the model and performing multiple forward propagations. Higher variance indicates that the prediction results are unstable, i.e., there is higher uncertainty.
[0102] KL divergence with a baseline distribution: This calculates the KL divergence between the model's predicted probability distribution and a baseline distribution representing complete uncertainty (e.g., a uniform distribution). The smaller the KL divergence, the closer the model's predictions are to complete uncertainty.
[0103] The module uses the calculated scalar value as the final metric for measuring the uncertainty of the candidate data.
[0104] Step 3: Strategy-based sorting and filtering. After calculating the corresponding uncertainty index for all candidate data in the data pool, the module sorts all samples in descending order according to the index value and selects the final sample subset based on a preset filtering strategy.
[0105] For example, a fixed threshold strategy can be adopted: when the uncertainty index of a candidate data is higher than a preset threshold, it is determined to be data with high uncertainty.
[0106] In another embodiment, an adaptive strategy is employed: to avoid the difficulty of manually setting thresholds, the strategy can be configured to select the top K percent of samples based on their uncertainty index. Here, K is a configurable percentage, for example, it can be set to 20, meaning that the 20% of samples with the highest uncertainty for the current model are selected.
[0107] The subset of samples selected by this strategy ultimately constitutes high-value augmented training data and is passed to the retraining module.
[0108] V. Model Retraining and Iteration: Absorbing New Knowledge and Achieving a Closed Loop. This stage injects the high-value data extracted from the previous stages into the model's training process, supplementing the model's specific knowledge gaps, and ultimately building the entire process into an intelligent loop that can continuously improve itself.
[0109] This step is executed and coordinated by the retraining module, and its processing flow is as follows:
[0110] Step 1: Constructing an enhanced training set. To ensure that the model does not forget the general knowledge it has already learned on the original data distribution while learning new knowledge, the module first performs a dataset merging operation.
[0111] The specific method involves merging the enhanced training data selected in the previous step with a larger subset of basic data randomly sampled from the original training dataset. The ratio of the merged dataset to the original dataset is a configurable key parameter.
[0112] For example, to ensure the stability of the training process, the ratio of the synthetic data to the original data can be set to 1:4.
[0113] Step 2: Update model parameters. The module feeds the constructed augmented training set into the standard training process of the target AI model, starts one or more training epochs, calculates the loss, backpropagates gradients, and updates the model's internal parameter weights on the augmented training set. Through this process, the model is forced to adapt to samples that it previously struggled to identify, thereby compensating for its knowledge gaps.
[0114] Step 3: Achieve closed-loop iteration and process traceability. To achieve continuous system optimization, this system is designed as a closed loop capable of continuous operation.
[0115] Specifically, after a training cycle is completed, the retraining module triggers a signal to the system's top-level controller. This controller then calls the failure case diagnosis module and uses the newly updated, more powerful target AI model to begin a new round of failure case diagnosis. This new round of diagnosis will discover deeper or entirely new failure patterns currently existing in the model, thereby driving the entire process to restart.
[0116] Termination condition: This loop process can be stopped by a preset termination condition.
[0117] For example, the termination condition may be one of the following: the total number of iterations of the system reaches a preset maximum value (e.g., 10 rounds), or the key performance indicators of the target AI model on the validation set do not show significant improvement for several consecutive iteration cycles (e.g., 3 cycles).
[0118] Process traceability: Throughout the entire process, the source information of each synthetic sample used for training (including the failure fingerprint ID on which it was generated and the augmentation prompt words used) can be recorded and stored in a log database or file to achieve auditing and traceability of the entire data augmentation process.
[0119] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, etc.) containing computer-usable program code.
[0120] It should be understood that the embodiments disclosed in this invention and the above description enable those skilled in the art to implement this invention. However, this invention is not limited to the embodiments mentioned above. It should be understood that those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this invention, and should all be included within the protection scope of this invention.
Claims
1. A training data intelligent synthesis and enhancement system for improving AI model performance, characterized in that, The system includes: The failure case diagnosis module is used to analyze the historical prediction failure cases of the target AI model to generate a structured failure fingerprint. The failure fingerprint is a data object in JSON format, which includes low-order visual feature intervals obtained by statistical analysis of the low-order visual features of the historical prediction failure cases, and semantic feature centroids obtained by unsupervised clustering of the high-order semantic feature vectors of the historical prediction failure cases to obtain failure pattern clusters and calculating the mathematical centroids of the high-order semantic feature vectors within the failure pattern clusters. The low-order visual feature intervals characterize the minimum and maximum values of the image samples that cause the target AI model to fail to predict in the dimensions of brightness, contrast, or sharpness. The prompt word injection module, connected to the failure case diagnosis module, is used to map the low-order visual feature regions and semantic feature centroids in the failure fingerprint into natural language description phrases, and embed the natural language description phrases into the basic prompt word template to generate enhanced prompt words; A data generation engine, connected to the prompt injection module, is used to generate a data pool containing candidate data related to the data features represented by the failed fingerprint, based on the enhanced prompts. An uncertainty assessment and screening module, connected to the data generation engine and the target AI model, is used to input each candidate data in the data pool into the target AI model for forward inference, obtain the prediction probability distribution of the target AI model for each candidate data, calculate the uncertainty index of the prediction probability distribution, and screen out data with high uncertainty from the data pool as augmented training data based on a preset screening strategy.
2. The intelligent synthesis and enhancement system for training data aimed at AI model performance as described in claim 1, characterized in that, The failure fingerprint is structured data and includes at least one of the following types of information: low-order visual features, high-order semantic features, target object attributes, or scene environment parameters related to the historical prediction failure cases.
3. The intelligent synthesis and enhancement system for training data aimed at AI model performance as described in claim 1, characterized in that, The prompt word injection module is used to map the structured feature information contained in the failed fingerprint into a natural language description, and embed or integrate the natural language description into the basic prompt words to generate the enhanced prompt words.
4. The intelligent synthesis and enhancement system for training data aimed at AI model performance as described in claim 1, characterized in that, The uncertainty assessment and screening module is used to calculate the entropy of the predicted probability distribution of the target AI model for each candidate data, the variance of the prediction result, or the KL divergence with the preset benchmark distribution, and uses the calculation results as an indicator to measure the uncertainty.
5. The intelligent synthesis and enhancement system for training data aimed at AI model performance as described in claim 1, characterized in that, The preset screening strategy is as follows: when the uncertainty index of a candidate data is higher than a preset threshold, the candidate data is determined to be data with high uncertainty.
6. The intelligent synthesis and enhancement system for training data aimed at AI model performance as described in claim 1, characterized in that, The selected enhanced training data is used to retrain or fine-tune the target AI model.
7. The intelligent synthesis and enhancement system for training data aimed at AI model performance as described in claim 6, characterized in that, The failure case diagnosis module is used to analyze the predicted failure cases of the target AI model after retraining or fine-tuning, so as to iteratively generate new failure fingerprints.
8. The intelligent synthesis and enhancement system for training data aimed at AI model performance as described in claim 1, characterized in that, The data generation engine is either a Transformer-based model or a diffusion model.