Large model-energy collaborative structured data enhancement method based on cross-domain knowledge guidance
By using a cross-domain knowledge-guided large model and energy synergy method, the problems of cross-domain knowledge transfer and multi-dimensional quality control in data augmentation under small sample scenarios are solved, generating high-quality datasets and improving the model training effect in fields such as financial risk control.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHONGQING UNIV OF POSTS & TELECOMM
- Filing Date
- 2026-01-28
- Publication Date
- 2026-05-15
AI Technical Summary
Existing technologies lack cross-domain knowledge transfer capabilities in small-sample scenarios, and the quality control of generated data is limited, making it difficult to achieve efficient and multi-dimensional data augmentation in fields such as financial risk control, resulting in unstable model training effects.
By using a cross-domain knowledge-guided approach to coordinate large models and energy, we can identify the main domain and related domains of small sample data, generate multiple sets of candidate samples, perform statistical filtering, energy assessment and cross-domain consistency verification, and finally perform format standardization to form an enhanced dataset.
It improves the data augmentation effect in small sample scenarios, and the generated data is more reliable in terms of statistical distribution and semantic rationality. It is suitable for data-sensitive and scarce fields such as finance, and supports model training and business decision-making.
Smart Images
Figure CN122045816A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data augmentation technology, and in particular to a cross-domain knowledge-guided large model-energy collaborative structured data augmentation method. Background Technology
[0002] In fields such as financial risk control that rely on structured data, data quality and scale directly determine the effectiveness of model training and the accuracy of business decisions. However, practical applications generally face three core problems: extremely small sample sizes, lack of domain knowledge, and low distribution fidelity. These problems make it difficult for traditional machine learning models to fully learn data patterns, and they generally suffer from overfitting, weak generalization ability, and insufficient sample distribution coverage. Therefore, there is an urgent need for efficient data augmentation techniques suitable for extremely small sample scenarios to overcome these bottlenecks.
[0003] Early structured data augmentation techniques primarily relied on "distribution interpolation," such as SMOTE, which generates synthetic samples through linear interpolation, and ADASYN, which balances class distributions through adaptive sampling. While these methods are simple to operate, they have significant limitations: linear interpolation assumes a smooth feature space and cannot adapt to complex nonlinear distributions; synthetic samples are limited to convex combinations of existing samples and cannot explore uncovered regions of the feature space; furthermore, when faced with extremely small sample scenarios, the information content of the original samples is severely insufficient, and interpolation methods cannot add effective statistical information, only alleviating the superficial problem of insufficient samples and failing to fundamentally improve the model's generalization ability.
[0004] With the development of deep learning technology, generative model-based augmentation methods have gradually become mainstream. Among them, methods such as Conditional Generative Adversarial Networks (CTGAN) and TabDDPM have made progress in the field of tabular data generation. CTGAN processes mixed-type data through pattern-specific normalization, but it suffers from unstable training and pattern collapse in small sample scenarios, and the generated data often exhibits defects such as concentrated feature distribution and insufficient diversity. Although TabDDPM can generate high-quality samples through progressive denoising, it requires a large number of training samples, is difficult to converge in small sample scenarios, and lacks the ability to explicitly model domain knowledge, so the generated samples may violate business logic constraints.
[0005] In recent years, energy-based models have demonstrated unique advantages in tabular data modeling. Among them, TabEBM constructs class-specific energy functions through a pre-trained Transformer model (TabPFN), enabling it to accurately capture the distribution characteristics of small sample data. TabEBM generates new samples in the energy landscape through SGLD sampling, maintaining high statistical fidelity even in small sample scenarios. However, existing energy model methods still have key drawbacks—limited generation space: constrained by the coverage of the training set, energy models struggle to reasonably extrapolate to feature regions not covered by the training samples, resulting in overly concentrated samples near existing samples, failing to fully expand data diversity; and a lack of domain knowledge guidance means that while the generated samples may have a reasonable statistical distribution, they may lack semantic rationality relevant to the business scenario. For example, in medical data, samples such as "elderly children" might be generated, creating logical contradictions.
[0006] Meanwhile, Large Language Models (LLMs) have demonstrated new potential in the field of structured data understanding and generation due to their powerful cross-domain knowledge transfer capabilities. By learning massive amounts of domain knowledge (medical literature, financial reports, sociological research, etc.) during the pre-training stage, LLMs can understand the semantic meaning of features from different domains and their interrelationships, generating reasonable samples that conform to domain common sense. However, when LLMs are used alone to generate tabular data, they suffer from statistical distribution distortion. Limited by the expressive power of prompt words and the probabilistic modeling mechanism of the model, LLMs struggle to accurately match the local statistical distribution of a specific dataset (such as the mean, variance, and correlation of features). The generated samples are prone to problems such as feature values exceeding reasonable ranges, blurred category boundaries, and distribution shifts. This results in synthetic data failing to effectively supplement the statistical information of real data and may even introduce noise that interferes with downstream model training.
[0007] More critically, existing methods lack a systematic mechanism for utilizing cross-domain knowledge. In small-sample scenarios, the training set contains extremely limited information. Traditional methods rely solely on the statistical features of the existing small sample for enhancement, failing to overcome the information bottleneck. Human experts, however, naturally draw upon knowledge from related fields when faced with small-sample problems—for example, credit scoring experts analyzing microloan samples in emerging consumption scenarios combine knowledge from consumer behavior psychology, payment security, and macroeconomics. Current technologies lack effective methods to embed this "cross-domain knowledge transfer" capability into the data augmentation process, resulting in the breadth and depth of knowledge in the generated data being limited by the training set itself, making it difficult to achieve true "knowledge augmentation."
[0008] Furthermore, existing methods generally lack multi-dimensional verification mechanisms for quality control: traditional generative models primarily focus on the matching degree of statistical distributions, evaluating generation quality through metrics such as KL divergence and MMD distance, but neglect the semantic rationality and cross-domain logical consistency of samples; LLM generation methods, while ensuring semantic fluency, lack precise control over statistical fidelity; energy model methods can ensure the accuracy of class boundaries, but cannot verify whether generated samples conform to business logic from multiple domain perspectives. This single-dimensional quality control results in generated data that, while satisfactory in some aspects, exhibits inconsistent overall quality, leading to unstable performance when applied to downstream tasks. Summary of the Invention
[0009] This application provides a cross-domain knowledge-guided large model-energy collaborative structured data augmentation method to address the problems of poor adaptability to small samples, lack of cross-domain knowledge transfer capability, and limited quality control in existing technologies. The method includes:
[0010] Obtain a small sample dataset, input the small sample dataset into a pre-trained large language model for analysis and processing, and identify the main domain and related domains corresponding to the small sample data;
[0011] A pre-trained large language model is used to generate multiple sets of candidate samples in parallel for different domains, and the generated multiple sets of candidate samples are combined into a candidate sample set.
[0012] Statistical filtering is performed on the candidate sample set to obtain the first sample set;
[0013] The first sample set is subjected to fusion statistics and energy assessment to obtain the second sample set;
[0014] Perform cross-domain consistency verification on the second sample set to obtain the third sample set;
[0015] The third sample set is format-normalized to obtain the augmented dataset.
[0016] Optionally, the step of inputting the small sample dataset into a pre-trained large language model for analysis and processing to identify the main domain and related domains corresponding to the small sample data includes:
[0017] Extract feature metadata from a small sample dataset, the feature metadata including: feature type, numerical range, and statistical distribution;
[0018] A pre-trained large language model is used to perform feature semantic understanding and task target mapping on the extracted feature metadata, and output the main domain and related domains corresponding to small sample data.
[0019] A mapping relationship between features and domain knowledge is constructed, and transferable knowledge points from each domain are extracted to form a structured domain information database. The structured domain information database includes domain relevance, transferable knowledge points, and feature-domain mapping.
[0020] Optionally, the domain relevance satisfies the following formula:
[0021] ;
[0022] in, For semantic weights, Cosine similarity of the domain text. Main field, Candidate fields, For feature weights, For the feature coverage of domain knowledge, Main domain characteristics.
[0023] Optionally, the step of generating multiple sets of candidate samples in parallel for different domains using a pre-trained large language model includes:
[0024] Calculate the domain feature coverage based on the feature coverage of the main domain features in the candidate domain and the total number of main domain features;
[0025] The domain contribution weight is determined based on the domain feature coverage and domain relevance;
[0026] The number of candidate samples for each domain is determined based on the domain contribution weight, the preset target sample size, and the redundancy coefficient.
[0027] Optionally, the step of statistically filtering the candidate sample set to obtain the first sample set includes:
[0028] Calculate the feature statistics of the real data, and then calculate the feature weights based on the feature statistics of the real data.
[0029] The statistical bias penalty value of the candidate sample is calculated based on the feature weights, the feature values of the candidate sample, and the feature statistics of the real data.
[0030] Candidate samples whose statistical bias penalty value exceeds the preset statistical bias threshold are filtered to obtain the first sample set.
[0031] Optionally, after filtering candidate samples whose statistical bias penalty values exceed a preset statistical bias threshold to obtain a first sample set, the method further includes:
[0032] Calculate the class deviation between the candidate samples in the first sample set and the true data;
[0033] If the class deviation value is greater than the preset tolerance threshold, the candidate samples of the first sample set are adjusted until the class deviation value is no greater than the preset tolerance threshold, thus obtaining the adjusted first sample set.
[0034] Optionally, the step of performing fusion statistics and energy assessment on the first sample set to obtain the second sample set includes:
[0035] Calculate the statistical fidelity of the candidate samples;
[0036] For any sample category, calculate the average energy of the real samples in that category, and calculate the energy value of the candidate samples in the first sample set in that category based on the trained energy model;
[0037] The normalized energy of the candidate sample is determined based on the average energy of the real samples in the category and the energy value of the candidate sample in the first sample set in the category.
[0038] The statistical fidelity and normalized energy values of the candidate samples in the first sample set are weighted and fused to obtain the joint score of the candidate samples.
[0039] Candidate samples in the first sample set whose joint scores are lower than the preset joint screening threshold are filtered to obtain the second sample set.
[0040] Optionally, the step of performing cross-domain consistency verification on the second sample set to obtain the third sample set includes:
[0041] The candidate samples in the second sample set are evaluated using a pre-trained large language model to obtain expert scores;
[0042] The expert scores are standardized to obtain standardized scores.
[0043] The cross-domain fusion score of the candidate samples is calculated based on the domain contribution weight and the standardized score;
[0044] Candidate samples in the second sample set whose cross-domain fusion scores are lower than the cross-domain verification threshold are filtered to obtain the third sample set.
[0045] This application provides a cross-domain knowledge-guided large model-energy collaborative structured data augmentation method. This method acquires a small sample dataset, inputs it into a pre-trained large language model for analysis, and identifies the main domain and related domains corresponding to the small sample data. The pre-trained large language model then generates multiple sets of candidate samples in parallel for different domains, forming a candidate sample set. Statistical filtering is performed on the candidate sample set to obtain a first sample set. Fusion statistics and energy assessment are then performed on the first sample set to obtain a second sample set. Cross-domain consistency verification is performed on the second sample set to obtain a third sample set. Finally, the third sample set is standardized to obtain the augmented dataset, thus improving the data augmentation effect in small sample scenarios. Attached Figure Description
[0046] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0047] Figure 1 This is a flowchart illustrating the cross-domain knowledge-guided large model-energy collaborative structured data augmentation method provided in this application embodiment.
[0048] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation
[0049] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0050] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein.
[0051] In this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.
[0052] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.
[0053] Figure 1 This is a flowchart illustrating the cross-domain knowledge-guided large model-energy collaborative structured data augmentation method provided in this application embodiment. Figure 1 As shown, the cross-domain knowledge-guided large model-energy collaborative structured data augmentation method provided in this embodiment includes:
[0054] S1: Obtain a small sample dataset, input the small sample dataset into a pre-trained large language model for analysis and processing, and identify the main domain and related domains corresponding to the small sample data.
[0055] Understandably, this method can be applied in scenarios such as financial credit scoring, and small sample datasets can be used as samples for credit default prediction.
[0056] Specifically, the small sample dataset is input into a pre-trained large language model for analysis and processing. Identifying the main domain and related domains corresponding to the small sample data involves: extracting feature metadata from the small sample dataset, including feature type, numerical range, and statistical distribution; using the pre-trained large language model to perform feature semantic understanding and task target mapping on the extracted feature metadata, outputting the main domain and related domains corresponding to the small sample data; constructing a mapping relationship from features to domain knowledge, extracting transferable knowledge points from each domain, and forming a structured domain information database, which includes domain relevance, transferable knowledge points, and feature-domain mappings.
[0057] For example, the system automatically parses the input data, extracts metadata such as feature type, numerical range, and statistical distribution, and constructs a feature metadata database.
[0058] Furthermore, based on the feature analysis results, prompt words are constructed. LLM identifies the main domain "consumer finance" (credit risk assessment sub-task) and related domains "behavioral economics," "socioeconomics," and "employment market analysis" through feature semantic understanding, task goal mapping, and cross-domain knowledge bridging. It outputs a JSON structure containing domain relevance, transferable knowledge points, and feature-domain mappings, which is then parsed by the system to build a domain knowledge base.
[0059] More specifically, domain relevance satisfies the following formula:
[0060]
[0061] in, This is the semantic weight, usually set to 0.6. Cosine similarity of the domain text. Main field, Candidate fields, This is the feature weight, usually set to 0.4. For the feature coverage of domain knowledge, Main domain characteristics.
[0062] Understandably, by ranking the relevance of different domains, the top K domains with the highest relevance are considered as relevant domains and used for generating subsequent candidate samples.
[0063] S2: Use a pre-trained large language model to generate multiple sets of candidate samples in parallel for different domains, and then combine the generated sets of candidate samples into a candidate sample set.
[0064] Specifically, prompts are built for each domain, comprising five modules: domain knowledge guidance, feature domain perspective, parameter constraints, domain logic constraints, and task generation instructions.
[0065] Furthermore, the domain feature coverage is calculated based on the number of features covered by the main domain features in the candidate domains and the total number of main domain features; the domain contribution weight is determined based on the domain feature coverage and domain relevance; and the number of candidate samples in each domain is determined based on the domain contribution weight, the preset target sample size, and the redundancy coefficient.
[0066] More specifically, the domain feature coverage satisfies the following formula:
[0067]
[0068] in, Total number of features in the main domain Representative main domain characteristics In the middle, it is possible to select candidate fields Features for obtaining effective insights.
[0069] Domain contribution weights satisfy the following formula:
[0070]
[0071] Among them, normalization ensures ;
[0072] The number of items generated in each field is allocated according to the following formula:
[0073]
[0074] in, This is the redundancy coefficient. For the target sample size, For the field Number of generated; Total number of generated , usually set This leaves ample room for subsequent screening.
[0075] In an optional embodiment, after generating candidate samples, the method further includes: removing redundant text, verifying the consistency of the table header, the correctness of the number and type of fields, filtering out samples with abnormal labels, and finally obtaining valid candidate samples.
[0076] S3: Perform statistical filtering on the candidate sample set to obtain the first sample set.
[0077] Specifically, it includes the following steps:
[0078] S31: Calculate the feature statistics of the real data, and calculate the feature weights based on the feature statistics of the real data.
[0079] The real data is a small sample dataset.
[0080] The feature weights satisfy the following formula:
[0081]
[0082] in, Features in real data standard deviation Features in real data standard deviation The total number of features.
[0083] S32: Calculate the statistical bias penalty value of the candidate sample based on the feature weights, the feature values of the candidate sample, and the feature statistics of the real data.
[0084] The statistical bias penalty satisfies the following formula:
[0085]
[0086] in, For the sample The 1 eigenvalue, Features in real data The mean, Features in real data The mean and standard deviation.
[0087] S33: Filter candidate samples whose statistical bias penalty value exceeds the preset statistical bias threshold in the candidate sample set to obtain the first sample set.
[0088] Specifically, the sample retention conditions are as follows:
[0089]
[0090] in, This is the statistical deviation threshold, which is usually set to 2.0.
[0091] In an optional embodiment, after filtering candidate samples whose statistical deviation penalty value exceeds a preset statistical deviation threshold to obtain a first sample set, the method further includes: calculating the class deviation value between the candidate samples in the first sample set and the real data; and adjusting the candidate samples in the first sample set until the class deviation value is not greater than the preset tolerance threshold when the class deviation value is greater than a preset tolerance threshold, thereby obtaining an adjusted first sample set.
[0092] For example, verify that the category distribution constraint is satisfied:
[0093]
[0094] in, For the categories in the candidate data proportion, Categories in real data proportion, This is the tolerance threshold, typically set to 0.1. It is a set of categories.
[0095] Furthermore, if the proportion deviation of a certain category is greater than 10%, the first sample set is randomly sampled or supplemented to balance the categories.
[0096] S4: Perform fusion statistics and energy assessment on the first sample set to obtain the second sample set.
[0097] Specifically, it includes the following steps:
[0098] S41: Calculate the statistical fidelity of the candidate samples.
[0099] The statistical fidelity of the candidate samples satisfies the following formula:
[0100] ;
[0101] Understandably, statistical fidelity is a simplified approximation of KL divergence; a higher value indicates that the sample is more similar to the true distribution.
[0102] S42: For any sample category, calculate the average energy of the real samples in that category, and calculate the energy value of the candidate samples in the first sample set in that category based on the trained energy model.
[0103] Specifically, samples are calculated using the pre-trained classifier TabPFN. In category energy value :
[0104]
[0105] in, and This is the output logits of TabPFN in the proxy binary classification task.
[0106] S42: Determine the normalized energy of the candidate sample based on the average energy of the real samples in the category and the energy value of the candidate sample in the category in the first sample set.
[0107] Specifically, the normalized energy satisfies the following formula:
[0108]
[0109] in, Categories in real data The average energy, This is the normalization coefficient, usually set to 1.2.
[0110] S43: The statistical fidelity and normalized energy values of the candidate samples in the first sample set are weighted and fused to obtain the joint score of the candidate samples.
[0111] Specifically, the joint score satisfies the following formula:
[0112]
[0113] in, As a balancing parameter, it is usually set to 0.4.
[0114] S44: Filter candidate samples in the first sample set whose joint scores are lower than the preset joint screening threshold to obtain the second sample set.
[0115] Specifically, the sample retention conditions are as follows:
[0116]
[0117] in, The joint screening threshold is usually set to 0.65.
[0118] S5: Perform cross-domain consistency verification on the second sample set to obtain the third sample set.
[0119] Specifically, it includes the following steps:
[0120] S51: Use a pre-trained large language model to evaluate candidate samples in the second sample set and obtain expert scores.
[0121] Specifically, the system constructs domain expert scoring prompts and calls a large language model to obtain scores. .
[0122] S52: Standardize the expert scores to obtain standardized scores.
[0123] Specifically, the standardized score satisfies the following formula:
[0124]
[0125] in, For the field of large language model simulation Experts on the sample Logical consistency score, ranging from 0 to 100. The highest score across all areas The lowest score across all areas.
[0126] S53: Calculate the cross-domain fusion score of candidate samples based on domain contribution weights and standardized scores.
[0127] Specifically, the cross-domain fusion score satisfies the following formula:
[0128]
[0129] in, For the field Contribution weight, For standardized scoring.
[0130] S54: Filter out candidate samples in the second sample set whose cross-domain fusion scores are lower than the cross-domain verification threshold to obtain the third sample set.
[0131] Specifically, the sample retention conditions are as follows:
[0132]
[0133] in, This is the cross-domain verification threshold, which is usually set to 0.6.
[0134] S6: Standardize the format of the third sample set to obtain the augmented dataset.
[0135] Specifically, it unifies the feature data type, fills in missing values, verifies the validity of labels, and outputs a standard CSV file with a timestamp in the filename.
[0136] In an optional embodiment, the method further includes: performing quality verification on the output samples and generating a JSON-formatted quality report, which includes core information such as screening statistics, energy distribution comparison, statistical fidelity indicators (KL divergence 0.08, KS test p-value 0.92, etc.), domain consistency score (DESF mean 0.73), and category distribution verification (default rate 30% consistent with the real data).
[0137] For example, quality assessment metrics include downstream task accuracy:
[0138]
[0139] in, For the number of categories, and Categories The number of true positive and false negative samples;
[0140] Statistical fidelity:
[0141]
[0142] in, For the normalized KL divergence, The p-value is the result of the Kolmogorov-Smirnov test. For feature relevance preservation, weights , , It is usually set to 0.4, 0.4, 0.2;
[0143] Domain consistency score:
[0144]
[0145] in, To enhance the total number of samples in the dataset, the logical consistency of the generated data across multiple domain perspectives is quantified. When the statistical fidelity is greater than 0.8 and the domain consistency score is greater than 0.7, the enhanced data quality is deemed acceptable.
[0146] In an optional embodiment, the mathematical expression of the optimization objective of this method is:
[0147]
[0148] The following constraints must be met:
[0149]
[0150] in, For downstream classifiers, To balance accuracy, and The distributions of augmented data and real data are respectively. Statistical fidelity tolerance This is the tolerance for the category distribution.
[0151] This application provides a cross-domain knowledge-guided large model-energy collaborative structured data augmentation method. It acquires a small sample dataset, inputs it into a pre-trained large language model for analysis, and identifies the main domain and related domains corresponding to the small sample data. The pre-trained large language model generates multiple sets of candidate samples in parallel for different domains, forming a candidate sample set. The candidate sample set is statistically filtered to obtain a first sample set. The first sample set undergoes fusion statistics and energy evaluation to obtain a second sample set. Cross-domain consistency verification is performed on the second sample set to obtain a third sample set. The third sample set is then format-standardized to obtain the augmented dataset, improving the data augmentation effect in small sample scenarios. This invention is applicable to data-sensitive and scarce fields such as healthcare and finance, overcoming the data volume limitations of traditional methods, solving the problem of insufficient data in small sample scenarios, and avoiding the risks of distribution shift and pattern collapse. It provides interpretable and high-fidelity data support for model training and business decision-making.
[0152] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.
[0153] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.
Claims
1. A method for augmenting structured data based on a cross-domain knowledge-guided large model-energy synergy, characterized in that, The method includes: Obtain a small sample dataset, input the small sample dataset into a pre-trained large language model for analysis and processing, and identify the main domain and related domains corresponding to the small sample data; A pre-trained large language model is used to generate multiple sets of candidate samples in parallel for different domains, and the generated multiple sets of candidate samples are combined into a candidate sample set. Statistical filtering is performed on the candidate sample set to obtain the first sample set; The first sample set is subjected to fusion statistics and energy assessment to obtain the second sample set; Perform cross-domain consistency verification on the second sample set to obtain the third sample set; The third sample set is format-normalized to obtain the augmented dataset.
2. The method according to claim 1, characterized in that, The step of inputting the small sample dataset into a pre-trained large language model for analysis and processing, and identifying the main domain and related domains corresponding to the small sample data, includes: Extract feature metadata from a small sample dataset, the feature metadata including: feature type, numerical range, and statistical distribution; A pre-trained large language model is used to perform feature semantic understanding and task target mapping on the extracted feature metadata, and output the main domain and related domains corresponding to small sample data. A mapping relationship between features and domain knowledge is constructed, and transferable knowledge points from each domain are extracted to form a structured domain information database. The structured domain information database includes domain relevance, transferable knowledge points, and feature-domain mapping.
3. The method according to claim 2, characterized in that, The domain relevance satisfies the following formula: ； in, For semantic weights, Cosine similarity of the domain text. Main field, Candidate fields, For feature weights, For the feature coverage of domain knowledge, Main domain characteristics.
4. The method according to claim 3, characterized in that, The method utilizes a pre-trained large language model to generate multiple sets of candidate samples in parallel across different domains, including: Calculate the domain feature coverage based on the feature coverage of the main domain features in the candidate domain and the total number of main domain features; The domain contribution weight is determined based on the domain feature coverage and domain relevance; The number of candidate samples for each domain is determined based on the domain contribution weight, the preset target sample size, and the redundancy coefficient.
5. The method according to claim 1, characterized in that, The statistical filtering of the candidate sample set to obtain the first sample set includes: Calculate the feature statistics of the real data, and then calculate the feature weights based on the feature statistics of the real data. The statistical bias penalty value of the candidate sample is calculated based on the feature weights, the feature values of the candidate sample, and the feature statistics of the real data. Candidate samples whose statistical bias penalty value exceeds the preset statistical bias threshold are filtered to obtain the first sample set.
6. The method according to claim 5, characterized in that, After filtering candidate samples whose statistical bias penalty values exceed a preset statistical bias threshold to obtain a first sample set, the method further includes: Calculate the class deviation between the candidate samples in the first sample set and the true data; If the class deviation value is greater than the preset tolerance threshold, the candidate samples of the first sample set are adjusted until the class deviation value is no greater than the preset tolerance threshold, thus obtaining the adjusted first sample set.
7. The method according to claim 1, characterized in that, The process of performing fusion statistics and energy assessment on the first sample set to obtain the second sample set includes: Calculate the statistical fidelity of the candidate samples; For any sample category, calculate the average energy of the real samples in that category, and calculate the energy value of the candidate samples in the first sample set in that category based on the trained energy model; The normalized energy of the candidate sample is determined based on the average energy of the real samples in the category and the energy value of the candidate sample in the first sample set in the category. The statistical fidelity and normalized energy values of the candidate samples in the first sample set are weighted and fused to obtain the joint score of the candidate samples. Candidate samples in the first sample set whose joint scores are lower than the preset joint screening threshold are filtered to obtain the second sample set.
8. The method according to claim 4, characterized in that, The process of performing cross-domain consistency verification on the second sample set to obtain the third sample set includes: The candidate samples in the second sample set are evaluated using a pre-trained large language model to obtain expert scores; The expert scores are standardized to obtain standardized scores. The cross-domain fusion score of the candidate samples is calculated based on the domain contribution weight and the standardized score; Candidate samples in the second sample set whose cross-domain fusion scores are lower than the cross-domain verification threshold are filtered to obtain the third sample set.