Data generation and model optimization method and device based on large language model adversarial training
By training the generator and discriminator in adversarial training of a large language model, the problem of scarce labeled data in professional fields is solved, high-quality synthetic data is generated, and the performance and applicability of the model are improved.
Patent Information
- Application Number
- CN202511184692.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-22
- Publication Date
- 2025-11-25
AI Technical Summary
Existing technologies face challenges in specialized fields, including a scarcity of labeled data, high acquisition costs, and limited quality and consistency of generated data, making it difficult to generate high-quality, highly diverse synthetic data for model training.
An adversarial training method based on large language models is adopted, in which two large language models play the roles of generator and discriminator respectively, and engage in multiple rounds of adversarial interaction to generate high-quality synthetic data, which is then used to optimize the performance of the target model.
It generates high-quality, highly reliable synthetic data, significantly improving the accuracy and generalization ability of target models in specific domains, reducing dependence on labeled data, and is suitable for a variety of application scenarios.
Smart Images
Figure CN121009953A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of artificial intelligence and natural language processing, and in particular to a data generation method based on adversarial interactive training of a large language model, and an apparatus for optimizing the performance of a target model using the generated data. Background Technology
[0002] Large Language Models (LLMs) have demonstrated outstanding performance in numerous natural language processing tasks. However, their effective application in specific downstream domains (such as healthcare, finance, and law) heavily relies on fine-tuning with large amounts of high-quality labeled data. In these specialized fields, data often faces challenges such as high acquisition costs, highly specialized annotation requirements, and privacy sensitivities, leading to a scarcity of labeled data and constituting a major bottleneck for improving model performance.
[0003] Traditional solutions, such as data augmentation techniques, are often limited to superficial transformations and struggle to generate samples that rival real data in semantics, logic, and professionalism. While Generative Adversarial Networks (GANs) can generate data, their discriminators are typically binary classifiers, unable to understand deep semantic information, resulting in limited quality and coherence of the generated text.
[0004] Therefore, there is an urgent need in this field for a new method that can overcome the limitation of scarce labeled data and generate high-quality, highly diverse synthetic data for efficient training of domain-specific models. Summary of the Invention
[0005] In view of this, the present invention aims to overcome the aforementioned deficiencies of the prior art and provide a data generation and model optimization method and apparatus based on adversarial training of large language models. The present invention uses two large language models, acting as a generator and a discriminator respectively, to generate high-quality synthetic data through multiple rounds of adversarial interaction, which is then used to optimize the performance of the target model.
[0006] To achieve the above objectives, this invention provides a data generation and model optimization method based on adversarial training of a large language model, comprising the following steps: Step 1: Obtain a real training dataset for a specific domain; Step 2: Based on the first pre-trained large language model, train the generator through instruction fine-tuning so that it can generate synthetic data similar to the real training data; Step 3: Based on the second pre-trained large language model, train the discriminator so that it can distinguish whether the input data is real data or synthetic data; Step 4: Alternately execute the adversarial training process between the generator and the discriminator. The generator optimizes its generation ability based on the feedback from the discriminator, and the discriminator optimizes its discrimination ability based on the synthetic data output by the generator. Step 5: After multiple rounds of adversarial training, use the optimized generator to generate large-scale synthetic data; Step 6: Merge the real training data with the generated synthetic data to construct a hybrid training dataset; Step 7: Fine-tune the target model using the hybrid training dataset; Step 8: Evaluate the performance of the fine-tuned target model.
[0007] Furthermore, the generator and discriminator are open-source or closed-source large language models with the same or different structures.
[0008] Furthermore, in step 2, the generator is fine-tuned using a sample containing task instructions. The instruction template is: "Please generate a [real / fake] data about [domain], which should be similar in style and content to the following real data: [real data example]".
[0009] Furthermore, in step 3, the data used to train the discriminator includes real data and synthetic data generated by the generator, and the training is performed through instruction fine-tuning. The instruction template is: "Please determine whether the following data is real data and provide an analysis: [Data to be judged]".
[0010] Furthermore, the adversarial training process in step 4 includes: the generator generating a batch of synthetic data; the discriminator judging the synthetic data against the real data and generating feedback; calculating the loss based on the discriminator's feedback and updating the generator parameters; and calculating the loss based on the judgment result and updating the discriminator parameters.
[0011] Furthermore, the target model can be any machine learning model or large language model to be applied to the specific domain.
[0012] The present invention also provides a data generation and model optimization device based on adversarial training of a large language model, comprising: The data acquisition module is used to acquire and preprocess real training data in a specific domain. The generator module, consisting of the first fine-tuned large language model, is used to generate synthetic data; The discriminator module, consisting of a second fine-tuned large language model, is used to determine the authenticity of data and provide feedback. The adversarial training control module is used to control the alternating training process of the generator and discriminator modules; The data synthesis module is used to integrate real data with the synthetic data finally output by the generator to construct a hybrid dataset; The target model training module is used to fine-tune the target model using a mixed dataset; The performance evaluation module is used to evaluate the performance metrics of the target model.
[0013] Compared with the prior art, the technical solution provided by the present invention has the following main advantages: 1. High-quality generated data: By using a large language model as a generator and discriminator, it can deeply understand semantics and domain knowledge, and generate high-quality, highly credible synthetic data, far exceeding traditional data augmentation methods.
[0014] 2. Reduced data dependence: It effectively alleviates the dependence on a large amount of labeled data, and is especially suitable for highly specialized fields where data is scarce.
[0015] 3. High versatility: This method does not depend on a specific model structure, and the generator and target model can be flexibly selected, making it suitable for a variety of application scenarios.
[0016] 4. Significant performance improvement: By injecting high-quality synthetic data, the accuracy, generalization ability and robustness of the target model in a specific domain can be significantly improved. Attached Figure Description
[0017] Figure 1 This is a flowchart of a method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the device structure according to an embodiment of the present invention. Detailed Implementation
[0018] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments. Example
[0019] like Figure 1 As shown, the specific implementation of the method of the present invention includes the following stages: Phase 1: Component Initialization Choose two pre-trained large language models, such as the LLaMA series models, and initialize them as a generator (G) and a discriminator (D), respectively. Use prompt engineering to assign roles and instructions to each model, clarifying their tasks.
[0020] Phase Two: Independent Fine-tuning G and D were fine-tuned using a small amount of real data. When fine-tuning G, the instruction template was used: "Give a fake news story that matches the style of the following real news story: [Real News Example]". When fine-tuning D, the instruction template was used: "Determine the veracity of the following news story and provide your reasoning: [News Text]", with the data consisting of a mix of real news and the initial fake news generated by G.
[0021] Phase 3: Adversarial Interaction Training This stage is an iterative process: 1.G generates a batch of synthetic data.
[0022] 2. Mix the synthetic data with the real data and input it into D.
[0023] 3.D makes a judgment (true / false) on each data point and generates an analysis reason.
[0024] 4. For G: Based on D's judgment that the generated data is "false" (i.e., it has been detected), construct a loss function and update the parameters. The goal is to make the generated data closer to the real data in order to "deceive" D.
[0025] 5. For D: Based on whether its judgment is correct or not (compared with the true label), construct a loss function, update the parameters, and improve its discrimination ability.
[0026] 6. Repeat steps 1-5 until the preset number of rounds is reached or performance convergence is achieved.
[0027] Phase Four: Data Generation and Model Optimization High-quality synthetic data is generated in batches using the trained G. The synthetic data is then mixed with the original real data at a certain ratio. This mixed dataset is used to perform full parameter fine-tuning or efficient parameter fine-tuning of the target domain model (such as a classifier or another LLM) using LoRA or other methods.
[0028] Phase 5: Assessment The performance of the fine-tuned target model was evaluated on an independent test set, and compared with the performance of the model fine-tuned using only the original real data, to verify the effectiveness of the synthetic data. Example
[0029] like Figure 2 As shown, the device of the present invention can be integrated into a computing device in the form of a software module. The adversarial training control module is responsible for scheduling the interaction process between the generator module and the discriminator module, and calling the data synthesis module to perform data fusion. The target model training module can load different pre-trained models, and the performance evaluation module outputs evaluation reports such as accuracy and F1 score.
[0030] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, those skilled in the art will find that… Any technical solution that can be obtained from the prior art through logical analysis, reasoning, or limited experimentation based on the concept of this invention should be within the scope of protection defined by the claims.
Claims
1. A data generation and model optimization method based on adversarial training of a large language model, characterized in that, Includes the following steps: Step S1: Obtain a real training dataset for a specific domain; Step S2: Train based on the first pre-trained large language model to obtain a generator, which is used to generate synthetic data that is semantically and structurally similar to the real training data; Step S3: Train the second pre-trained large language model to obtain a discriminator, which is used to distinguish whether the input data is real data or synthetic data; Step S4: Perform the interactive adversarial training process between the generator and the discriminator, wherein the generator optimizes its generation ability based on the feedback from the discriminator, and the discriminator optimizes its discrimination ability based on the synthetic data output by the generator; Step S5: Generate synthetic data using the optimized generator; Step S6: Combine the real training data with the synthetic data generated in step S5 to construct a hybrid training dataset; Step S7: Train and optimize the target model using the hybrid training dataset; Step S8: Evaluate the performance of the trained and optimized target model.
2. The method according to claim 1, characterized in that, The first pre-trained large language model and the second pre-trained large language model are open-source or closed-source models with the same or different structures.
3. The method according to claim 1, characterized in that, In step S2, the generator is trained by fine-tuning the instructions. The training data includes samples from the real training dataset and corresponding task instructions. The task instructions are used to guide the generator to imitate the features of the real data to generate synthetic data.
4. The method according to claim 1, characterized in that, In step S3, the discriminator is trained by fine-tuning the instructions. The training data includes the real training data, the synthetic data generated by the generator, and the authenticity label corresponding to each data point. The task instructions are used to guide the discriminator to judge and analyze the authenticity of the input data.
5. The method according to claim 1, characterized in that, The interactive adversarial training process in step S4 is an iterative process, and a single iteration includes: S4.1: The generator generates a batch of synthesized data; S4.2: The synthesized data is mixed with some real data and then input into the discriminator; S4.3: The discriminator judges the mixed data and generates feedback information; S4.4: Calculate the generator loss and update the generator parameters based on the output feedback of the discriminator; S4.5: Calculate the discriminator loss and update the parameters of the discriminator based on the difference between the discriminator's judgment result and the authenticity label.
6. The method according to claim 1, characterized in that, In step S6, the real training data and the generated synthetic data are fused according to a preset ratio to construct the hybrid training dataset.
7. The method according to claim 1, characterized in that, The target model is a large language model or machine learning model used to perform the domain-specific task.
8. A data generation and model optimization apparatus for implementing the method of any one of claims 1 to 7 based on adversarial training of a large language model, characterized in that, include: The data acquisition module (110) is used to acquire and preprocess real training data in a specific domain; The generator module (120), consisting of a trained first large language model, is used to generate synthetic data; The discriminator module (130), consisting of a trained second-largest language model, is used to determine the authenticity of data and provide feedback. An adversarial training control module (140) is connected to the generator module (120) and the discriminator module (130) and is used to control the interactive adversarial training process between the two. A data synthesis module (150), connected to the generator module (120) and the data acquisition module (110), is used to integrate real data and synthetic data to construct a hybrid dataset; The target model training module (160) is connected to the data synthesis module (150) and is used to train and optimize the target model using the hybrid dataset; The performance evaluation module (170) is connected to the target model training module (160) and is used to evaluate the performance of the target model.
9. The apparatus according to claim 8, characterized in that, The performance evaluation module (170) evaluates the effectiveness of the synthetic data and the performance improvement of the target model by comparing the performance metrics of a model trained using only real data and a model trained using the hybrid dataset.
10. A computing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method as described in any one of claims 1 to 7.