LLM-based data set production method, system and device, and medium

Abstract

The invention discloses an LLM-based data set production method, system and device and a medium, and the method comprises the steps: carrying out the data collection, forming a data set containing diversified labels, and guaranteeing that the data in the data set has diversified sources; processing the data set by adopting a fair resampling algorithm: firstly, carrying out statistical analysis on data in the data set, and identifying a data category or attribute with deviation; according to the number and importance of the data samples of the data categories or attributes, weighting or re-sampling the data in the data set, so that the data set is more balanced on the whole; the diversity and fairness of a data set are enhanced by adopting an adversarial training technology; and evaluation and adjustment are carried out through indexes. According to the scheme, the diversity and fairness of the data set can be enhanced, and the performance of the LLM in practical application is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of dataset production, and in particular to a dataset production method, system, device and medium based on LLM. Background Art

[0002] With the advancement of computing power and data storage capabilities in the fields of artificial intelligence and machine learning, machine learning models have evolved from simple linear regression and decision trees to complex deep neural networks and large language models. This process has also led to an increasing demand for large-scale, high-quality datasets. In the field of natural language processing (NLP), in particular, the effectiveness of model training depends heavily on the size and quality of the dataset. However, traditional data collection and annotation methods are often time-consuming and labor-intensive, making them unable to meet the demands of rapidly evolving technologies. To address this challenge, researchers and engineers have begun exploring automated and efficient methods for dataset creation.

[0003] The development of automated data collection technologies has made it easier to access massive amounts of data. Web crawlers and APIs allow for the rapid collection of diverse data from various sources. However, raw data often suffers from noise and inconsistencies, necessitating data cleaning and preprocessing to ensure data quality and usability. Data cleaning techniques, including deduplication, correction of erroneous data, and filling in missing data, have significantly improved the efficiency and effectiveness of data processing.

[0004] Data labeling is another crucial step in dataset creation. Traditional manual labeling methods are not only time-consuming but also prone to subjective bias. With the advancement of deep learning and natural language processing technologies, automated labeling tools have emerged. Automatic labeling using pre-trained models can replace manual labeling in most cases, significantly improving labeling efficiency. Furthermore, crowdsourced labeling platforms offer an efficient solution, enabling rapid completion of large-scale data labeling tasks through the collaboration of numerous annotators.

[0005] Data augmentation techniques are an important means of increasing dataset diversity and model robustness. By transforming, generating, and expanding existing data, we can create more diverse training samples, thereby improving the model's generalization ability. Common data augmentation methods include synonym replacement and word order transformation in text data, as well as new data samples for generative adversarial networks (GANs). The application of these techniques not only expands the size of datasets but also enhances their diversity and representativeness.

[0006] However, efficient dataset creation technology faces numerous challenges in practical application. Data privacy protection and ethical issues require significant attention. During automated data collection and annotation, ensuring user privacy and avoiding bias and discrimination in the data are issues that require in-depth research and resolution. Furthermore, as data volumes continue to expand, data storage and management are becoming increasingly challenging. Effectively organizing and managing massive amounts of data, as well as efficient querying and analysis in big data environments, are also key challenges that dataset creation technology urgently needs to address.

[0007] Efficient dataset creation technology has played a significant role in promoting the development and application of LLM. Through automated and intelligent approaches, it significantly improves the efficiency and quality of dataset creation, meeting the needs of large-scale data-driven model training. In the future, with the continuous advancement of technology and the deepening of its application, dataset creation technology will further develop, providing a more solid foundation for the training and application of large language models.

[0008] Due to the high degree of automation in the data collection and annotation process, bias and errors in the original data are inevitably introduced and amplified. This bias not only affects the fairness of the model but can also lead to poor performance in specific populations or scenarios. If the training dataset contains too many samples from a certain language or cultural background, the model may overfit to these samples and lack sufficient generalization ability for samples from other languages ​​or cultural backgrounds. This data bias is particularly prominent in the practical application of LLM and can lead to serious social and ethical issues. Summary of the Invention

[0009] The purpose of the present invention is to provide a dataset production method, system, device and medium based on LLM, which adopts a dataset enhancement algorithm based on diversity and fairness, balances the data distribution of different categories and attributes through resampling and weighting, and combines adversarial training technology to generate adversarial samples, thereby enhancing the diversity and fairness of the dataset.

[0010] To achieve the above object, the technical solution provided by the present invention is:

[0011] The first aspect of the present application provides a method for preparing a dataset based on LLM, comprising the following steps:

[0012] Conduct data collection to form a dataset with diverse annotations, ensuring that the data in the dataset comes from diverse sources;

[0013] The fair resampling algorithm is used to process the dataset: first, statistical analysis is performed on the data in the dataset to identify data categories or attributes with deviations; then, based on the number and importance of data samples of these data categories or attributes, the data in the dataset is weighted or resampled to make the dataset more balanced overall;

[0014] Adopt adversarial training techniques to enhance the diversity and fairness of datasets;

[0015] Evaluate and adjust through indicators.

[0016] To optimize the above technical solutions, specific measures taken also include:

[0017] After data collection, the data is cleaned and preprocessed, including removing duplicate data, correcting spelling errors, standardizing text formats, and labeling and annotating the data; it also includes using data augmentation technology to convert text data into multiple languages ​​through translation and de-translation technology to increase language diversity; establishing a feedback mechanism for the dataset with diverse annotations, collecting feedback from users and experts on the dataset, identifying and correcting possible biases and errors, and improving diversity and fairness through iterative optimization of the dataset and training process.

[0018] Furthermore, the use of a fair resampling algorithm to process the data set includes: refining the data set through multi-level classification; regularly performing data statistical analysis to identify the latest deviations in the data set and dynamically adjusting the weights of each category; combining oversampling and undersampling techniques to balance the data set; building a more comprehensive data set by fusing multi-source data and resampling it; and establishing a continuous monitoring and feedback optimization mechanism.

[0019] Furthermore, the adversarial training technique is used to enhance the diversity and fairness of the dataset. Adversarial examples are generated through adversarial training, forcing the training process to face more complex and variable inputs, thereby improving generalization and robustness. The adversarial training method uses FGSM to generate adversarial examples by calculating the gradient of the input data and adding small perturbations in the gradient direction. The specific process includes:

[0020] Calculate gradient: For each training sample, calculate the gradient of the loss function with respect to the input data;

[0021] Add perturbation: Add a small perturbation to the input data in the direction of the gradient. The size of the perturbation is controlled by a hyperparameter ε.

[0022] Generate adversarial samples: Use the perturbed data as adversarial samples;

[0023] The formula for generating adversarial samples using FGSM is:

[0024]

[0025] x is the original input, ε is the perturbation amplitude, is the gradient of the loss function with respect to the input data, x adv For adversarial samples, sign() represents a mathematical function that returns an integer variable, which is used to indicate the positive and negative signs of the parameters.

[0026] The methods for evaluating and adjusting through indicators include manual evaluation, automatic evaluation, and hybrid evaluation, and the indicators used include accuracy, sentiment analysis, and fairness.

[0027] The second aspect of the present application provides a data set production model based on LLM, including:

[0028] The data collection module is used to collect data and form a dataset with diverse annotations to ensure that the data in the dataset has diverse sources;

[0029] The fair resampling module is used to process the dataset using a fair resampling algorithm. First, the data in the dataset is statistically analyzed to identify data categories or attributes with deviations. Then, based on the number and importance of data samples in these data categories or attributes, the data in the dataset is weighted or resampled to make the dataset more balanced overall.

[0030] Adversarial training module, which uses adversarial training techniques to enhance the diversity and fairness of datasets;

[0031] Evaluation module, used for evaluation and adjustment through indicators.

[0032] The third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the LLM-based dataset production method as described in the first aspect of the present application is implemented.

[0033] A fourth aspect of the present application provides a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the LLM-based dataset production method as described in the first aspect of the present application.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] This method uses a dataset enhancement algorithm based on diversity and fairness. It balances the distribution of data across categories and attributes through resampling and weighting, and combines it with adversarial training techniques to generate adversarial examples, enhancing both dataset diversity and fairness. This approach not only effectively reduces data bias but also improves model performance in diverse scenarios.

[0036] The present invention can quickly generate large-scale, high-quality data sets through automated data collection, data cleaning, data labeling, and data enhancement, meeting the LLM's demand for large amounts of training data.

[0037] The present invention adopts a dataset enhancement algorithm based on diversity and fairness. By resampling or weighting the dataset, the data distribution of different categories and attributes is balanced, thereby reducing bias. First, the dataset is statistically analyzed to identify the categories or attributes with bias, and then the data is weighted or resampled according to the number and importance of samples of these categories or attributes, so that the dataset is more balanced as a whole. In the specific implementation process, the adversarial training technology in machine learning can be combined to further enhance the diversity and fairness of the dataset by generating adversarial samples. This method can not only effectively alleviate the problem of data bias, but also improve the performance of the model in diverse scenarios.

[0038] In addition, the present invention can also realize customized data generation for specific fields or tasks, ensuring that the model performs better in specific application scenarios. For example, in natural language processing (NLP) tasks, through automated labeling and verification tools, manual participation can be greatly reduced, and the consistency and accuracy of data labeling can be improved, thereby making the model training process more efficient and ultimately improving the performance of LLM in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 : Schematic diagram of the process of the data set preparation method based on LLM of the present invention.

[0040] Figure 2 : Schematic diagram of the process of processing a data set using a fair resampling algorithm in the present invention.

[0041] Figure 3 : A schematic structural diagram of the data set production model based on LLM of the present invention. DETAILED DESCRIPTION

[0042] The above contents of the present invention are further described in detail below in the form of embodiments, but this should not be understood as the scope of the above subject matter of the present invention being limited to the following embodiments. All technologies implemented based on the above contents of the present invention fall within the scope of the present invention.

[0043] Some of the glossary terms in this invention are explained as follows:

[0044] LLM (Large Language Model) is a deep learning-based artificial intelligence algorithm that models natural language text by training on large amounts of text data to learn the grammar, semantics, and context of a language. This model has a wide range of applications in natural language processing, including but not limited to text generation, text classification, machine translation, and sentiment analysis.

[0045] Traditional methods create datasets through manual labeling and simple data collection, which can easily introduce bias and be inefficient. This method uses a dataset enhancement algorithm based on diversity and fairness. This algorithm balances the distribution of data across categories and attributes through resampling and weighting, and combines adversarial training techniques to generate adversarial examples, enhancing both dataset diversity and fairness. This approach not only effectively reduces data bias but also improves model performance in diverse scenarios.

[0046] In one embodiment, the present invention provides a method for preparing a data set based on LLM, such as Figure 1 As shown, the following steps are included: data collection to form a dataset with diverse annotations, ensuring that the data in the dataset has diverse sources;

[0047] The dataset is processed using a fair resampling algorithm, such as Figure 2 As shown in the figure: First, statistical analysis is performed on the data in the dataset to identify the data categories or attributes with deviations; then, based on the number and importance of data samples of these data categories or attributes, the data in the dataset is weighted or resampled to make the dataset more balanced overall;

[0048] Adopt adversarial training techniques to enhance the diversity and fairness of datasets;

[0049] Evaluate and adjust through indicators.

[0050] The present invention is further described in detail below through the following key aspects: diversity of data collection and annotation, fair resampling algorithm, adversarial training techniques, data enhancement and diversification, and model evaluation and adjustment:

[0051] (1) Diversity of data collection and annotation

[0052] First, ensure that the dataset comes from diverse sources, including different languages, cultures, and social backgrounds. The goal is to reduce the impact of single-source bias by increasing the diversity of data sources. By obtaining text data from various websites, books, social media platforms, etc., ensure that the model is exposed to diverse content, thereby better balancing different social and cultural perspectives during the training process and ensuring the diversity of the dataset. The specific implementation process is as follows:

[0053] First, identify and select diverse data sources; these should encompass diverse languages, cultures, and social backgrounds. Research and investigation can identify websites, books, social media platforms, and other sources with representative and diverse content, and then collect data from these sources. For websites, web crawlers can be used to regularly capture text data; for books, e-book databases and digital libraries can be utilized; and for social media, APIs can be used to access publicly available user-generated content. During this process, ensure compliance with relevant data privacy and usage policies.

[0054] Collected data often contains noise and inconsistent information, requiring data cleaning and preprocessing. This includes removing duplicate data, correcting spelling errors, and standardizing text formats. Natural language processing techniques can also be used for grammatical analysis and annotation to improve data quality. To enhance the model's understanding and generation capabilities, data should be carefully labeled and annotated. Text data can be subjected to sentiment analysis, topic classification, and entity recognition. The annotation process can combine automated tools with manual review to ensure label accuracy and diversity.

[0055] To further ensure data diversity, data augmentation techniques can be used. Translation and detranslation techniques can be used to convert text data into multiple languages ​​to increase linguistic diversity. Data augmentation techniques can also be used to generate new data samples to enrich the dataset. Furthermore, care should be taken to balance the different categories and labels within the dataset, avoiding excessive or insufficient data for certain categories. Data diversification is an ongoing process that requires regular monitoring and updating of datasets. Regularly reviewing data sources and collection strategies, and introducing new data sources and content, can ensure that the dataset reflects the latest social and cultural changes.

[0056] During model training, use diverse datasets and regularly evaluate model performance. This evaluation process should focus on the model's performance across different languages, cultures, and social contexts to ensure it can fairly process and generate diverse content. Establish a feedback mechanism to collect user and expert feedback on model outputs to identify and correct potential biases and errors. Continuously improve model diversity and fairness through iterative optimization of datasets and model training.

[0057] Through the above steps, the dataset can be diversified, thereby effectively reducing the impact of single-source bias and improving the performance of large-scale language models in diverse contexts.

[0058] (2) Fair Resampling Algorithm

[0059] After data collection, the dataset is processed using a fair resampling algorithm. First, the dataset is statistically analyzed to identify biased categories or attributes. Then, based on the number and importance of samples in these categories or attributes, the data is weighted or resampled to make the dataset more balanced overall. This method adjusts the sample distribution to reduce the overrepresentation of certain categories or attributes in the dataset, thereby reducing bias in model training.

[0060] In the fields of data science and machine learning, fair resampling algorithms aim to reduce bias in datasets by adjusting sample distributions, thereby improving model fairness and accuracy. For operators, datasets often cover multiple dimensions, such as user behavior, network usage, and service quality, making ensuring the fairness of this data crucial. The following are some approaches to improving fair resampling algorithms and discuss their adaptation within operators:

[0061] The first step in this improvement is to refine the data classification and weighting strategies. Carrier datasets often contain complex, multi-level classifications, such as user geography, device type, and network usage behavior. Multi-level classification can be used to refine datasets and categorize users into more granular categories, such as urban, suburban, and rural by region, and smartphones, tablets, and laptops by device type. By calculating the number of samples for each sub-category and assigning weights based on these sub-categories, the weighting strategy becomes more precise. This allows for more effective balancing of data across different user groups and ensures that the model is not biased by excessive amounts of data from specific categories.

[0062] Traditional resampling methods often apply fixed weights to the entire dataset, ignoring the time-varying nature of data. For operators, user behavior and network usage patterns can change significantly over time. Therefore, a dynamic weight adjustment mechanism can be introduced to adjust weights in real time based on dataset changes. This involves regular statistical analysis of the data to identify recent deviations and dynamically adjust the weights of each category. This approach not only adapts to dataset changes but also improves the model's responsiveness to real-time data.

[0063] The third step in the improvement process is to combine oversampling and undersampling techniques to balance the dataset. For categories with a small number of samples, oversampling can be used, such as using SMOTE (Synthetic Minority Over-sampling Technique) to generate synthetic samples, or using data augmentation techniques to increase sample diversity. For categories with a large number of samples, undersampling can be used, such as randomly deleting some samples or using clustering algorithms to remove redundant samples. Furthermore, these techniques can be combined to achieve a more balanced dataset by adjusting the ratio of oversampling to undersampling.

[0064] Operators' datasets contain a variety of user behavior patterns, such as call logs, internet usage, and data usage. To comprehensively balance the dataset, we can classify and resample it based on different behavioral characteristics. We can perform statistical analysis on call logs, internet usage, and data usage, and resample each characteristic. This approach more comprehensively considers the diversity of user behavior and ensures fairness in the model's handling of different user behaviors.

[0065] To further improve the diversity and balance of datasets, multi-source data fusion can be incorporated into resampling algorithms. For operators, multi-source data may include user social media activity, geolocation data, customer service records, and more. By fusing these multi-source data, a more comprehensive dataset can be constructed and resampled. This not only helps reduce bias in single-source data but also improves the model's adaptability to diverse data sources.

[0066] The final step in improvement is to establish a continuous monitoring and feedback optimization mechanism. After the model is deployed, its performance and fairness should be continuously monitored, user and system feedback should be collected, and new biases and issues should be identified. Based on this feedback, the dataset and resampling strategy should be regularly updated to ensure the long-term fairness and accuracy of the model. User satisfaction surveys and network performance monitoring can be used to assess the model's performance in real-world applications and enable timely adjustments and optimizations.

[0067] Therefore, in the specific implementation process in the application scenario, operators can take the following steps: First, refine the data set according to the multi-level classification method, conduct statistical analysis, and assign initial weights; regularly (such as monthly or quarterly) update the data and conduct statistical analysis, and dynamically adjust the weights of each category; based on the statistical analysis results, use a combination of oversampling and undersampling techniques to balance the data set; collect and integrate multi-source data, build a comprehensive data set, and perform resampling processing; establish a monitoring and feedback mechanism, regularly evaluate model performance, collect user and system feedback, and make adjustments and optimizations.

[0068] The fair resampling algorithm can more effectively adapt to the data needs of operators, ensure the diversity and balance of data sets, and thus improve the fairness and accuracy of the model. This not only helps improve the service quality and user experience of operators, but also maintains technological leadership in the fierce market competition.

[0069] (3) Adversarial training technology

[0070] After resampling, adversarial training techniques are used to further enhance the diversity and fairness of the dataset. Adversarial training generates adversarial examples, forcing the model to face more complex and variable inputs during training, thereby improving the model's generalization and robustness. This approach not only effectively reduces the model's overfitting to specific categories or attributes, but also improves the model's performance in various scenarios. The following is the detailed implementation process:

[0071] Adversarial training is a method for enhancing model robustness by introducing adversarial examples to train the model. Adversarial examples are generated by adding small perturbations to the original input, with the goal of enabling the model to maintain good performance despite these perturbations. This technique forces the model to learn more robust features, thereby improving its generalization ability.

[0072] After resampling, the dataset is somewhat balanced, but further diversity improvement is needed. First, the resampled dataset is normalized to ensure consistency of the input data. Then, the dataset is divided into training and validation sets to ensure a balanced distribution of data within each set.

[0073] We use the adversarial generation algorithm to generate adversarial samples. We use FGSM (Fast Gradient Sign Method) to calculate the gradient of the input data and add a small perturbation in the gradient direction to generate adversarial samples. The specific steps are as follows:

[0074] Calculate gradients: For each training example, calculate the gradient of the model loss function with respect to the input data.

[0075] Add perturbation: Add a small perturbation to the input data in the direction of the gradient. The size of the perturbation is controlled by a hyperparameter (such as ε).

[0076] Generate adversarial samples: Use the perturbed data as adversarial samples.

[0077] The formula for generating adversarial samples using FGSM is:

[0078]

[0079] x is the original input, ε is the perturbation amplitude, is the gradient of the loss function with respect to the input data, x advadversarial examples. sign() is a mathematical function that returns an integer variable, indicating the sign of its argument. Its syntax is sign(number), where the number argument can be any valid numeric expression. The sign function returns a different integer depending on the value of number: 1 if number is greater than 0; 0 if number is equal to 0; and -1 if number is less than 0. The sign function has applications in both mathematics and computer science, for example, in finding the zeros of a function and discussing absolute value problems.

[0080] During adversarial training, the model must learn not only the original samples but also the adversarial samples. The specific training steps are as follows: The original samples and adversarial samples are mixed to form a new training set; a certain proportion of adversarial samples is included in the training set to ensure the model's robustness to different inputs; the model is trained using the mixed training set; during training, the model continuously adjusts its parameters to adapt to the perturbations of the adversarial samples, thereby improving its generalization ability; and the loss function is optimized during training to ensure that the model performs well on both the original and adversarial samples. This can be achieved by adjusting the weights of the loss function to maintain a balanced contribution to the original and adversarial samples.

[0081] After adversarial training is completed, the model needs to be verified and evaluated to ensure its performance on diverse inputs. This includes: using an independent validation set to evaluate model performance to ensure the model performs well on unseen data; evaluating the model's performance on adversarial samples to verify its robustness; and statistically analyzing the model's performance on different categories and attributes to ensure its fairness.

[0082] Based on the evaluation results, we continuously optimize the adversarial training strategy to ensure the performance and fairness of the model in real-world applications. Specific optimization measures include: dynamically adjusting the proportion of adversarial examples in the training set based on the model's performance and needs; optimizing the hyperparameters of the adversarial generation algorithm (such as the perturbation amplitude ε) to achieve optimal results; and regularly updating the training data and model, iterating training with new adversarial examples to continuously improve the model's robustness and fairness.

[0083] By combining these steps with adversarial training techniques, we can significantly enhance the diversity and fairness of datasets, thereby improving the generalization and robustness of the model. This not only effectively reduces the model's overfitting to specific categories or attributes, but also ensures the model's stable performance in different scenarios, providing operators with more reliable and fair technical support.

[0084] (4) Data enhancement and diversification

[0085] We further employ data augmentation techniques, such as synonym replacement and word order transformation, to increase the diversity of the dataset. These techniques can generate more training samples without changing the semantics of the data, thereby improving the robustness and generalization of the model. Generating more training samples without changing the semantics of the data enriches the dataset and improves model performance. The following is the specific implementation process:

[0086] Data augmentation techniques generate new samples by performing various transformations on the original data while keeping the semantics unchanged. These techniques can significantly increase the diversity of the dataset. Common data augmentation methods include: replacing certain words in a sentence with their synonyms; replacing "beautiful" with "beautiful"; changing the order of words or phrases in a sentence without changing the basic meaning of the sentence; transforming "he went to the store" into "he who went to the store"; randomly inserting context-related words into the sentence; and randomly deleting certain non-key words in the sentence to keep the semantics of the sentence intact.

[0087] Data augmentation through Word Net:

[0088] Select appropriate data augmentation strategies based on data characteristics and task requirements. Write code to implement data augmentation algorithms such as synonym replacement, word order transformation, random insertion, and random deletion. Process the original dataset to generate augmented samples and expand the training dataset.

[0089] A generative adversarial network (GAN) is a deep learning model that generates realistic new samples through adversarial training of two neural networks (a generator and a discriminator). GANs are widely used in image generation, but can also be used to generate text data. The specific implementation steps are as follows: Design the GAN's generator network and discriminator network. The generator is responsible for generating new samples, while the discriminator is responsible for distinguishing between real and generated samples. Through adversarial training, the generator's samples become increasingly realistic, while the discriminator can effectively distinguish between real and generated samples. After training, the generator is used to generate new samples to expand the dataset.

[0090] In language model (LLM) scenarios, data augmentation is a key step in improving the generalization and robustness of the model. Generative adversarial network (GAN) technology can provide an effective solution for data augmentation. The following are the specific implementation steps and methods:

[0091] The raw text data is cleaned to remove noise, correct typos, and standardize to ensure data quality. Basic text data enhancement techniques, such as synonym replacement, word order transformation, random insertion, and random deletion, are applied to generate multiple enhanced versions of the sample.

[0092] The generator is responsible for generating new text samples. The Transformer architecture can be used as a generator to generate high-quality text. The discriminator is responsible for distinguishing between real samples and generated samples. A bidirectional LSTM can be used as a discriminator to determine the authenticity of the input text.

[0093] Through adversarial training, the text samples generated by the generator become increasingly realistic, while the discriminator can effectively distinguish between real and generated text. The generator generates text samples, and the discriminator judges the samples and provides feedback on the loss. The generator and discriminator are optimized alternately, and through iterative training, the text samples generated by the generator gradually approach real text.

[0094] A fair resampling algorithm is applied to the augmented dataset to ensure a balanced data distribution. This is particularly effective in addressing biased datasets, making the generated samples more balanced across categories and attributes. Adversarial examples are generated by combining the original and augmented samples. These examples are used for adversarial training to improve the model's robustness to diverse input types. Training using the generated adversarial examples ensures that the model maintains high accuracy and robustness in the face of adversarial examples.

[0095] After training is complete, the generator is used to generate new text samples to further enrich the dataset. These samples can be used to expand the training data and improve the model's generalization ability. Combining the GAN-generated samples with the original and augmented data creates a more diverse dataset.

[0096] We use a comprehensive set of augmented and expanded datasets to train models and improve the generalization and accuracy of LLM. We validate and evaluate the trained models to ensure their good performance on diverse data. Evaluation criteria include accuracy, robustness, and fairness.

[0097] The above process, combined with improved GAN technology, significantly improves dataset diversity and model robustness. This approach not only effectively reduces overfitting of the model to specific categories and attributes, but also enhances model performance across diverse scenarios, providing more reliable and impartial technical support. Especially when working with large-scale language models, improved GAN technology provides a powerful tool for data augmentation, driving LLM's superior performance on diverse data.

[0098] In order to achieve a comprehensive enhancement effect, data augmentation technology, adversarial training and GAN generation samples can be combined to form a complete process: clean and standardize the original data set to ensure data quality; apply synonym replacement, word order transformation, random insertion and random deletion and other techniques to generate enhanced samples and expand the data set; apply fair resampling algorithm to the enhanced data set to ensure balanced data distribution; generate adversarial samples, combine the original samples and enhanced samples for adversarial training to improve model robustness; use GAN to generate new samples to further enrich the data set; use the enhanced and expanded data set for model training to improve the generalization ability and accuracy of the model; verify and evaluate the trained model to ensure its good performance on diverse data.

[0099] In some embodiments, the process is as follows:

[0100] Write and run a data augmentation algorithm to generate multiple enhanced versions of samples; implement a fair resampling algorithm to process the augmented dataset; implement adversarial sample generation and adversarial training; design and train a GAN model to generate new samples; and use all generated samples comprehensively for final model training and evaluation.

[0101] By combining data augmentation technology and GAN generation samples, the diversity of the dataset and the robustness of the model can be significantly improved; this can not only effectively reduce the model's dependence on specific categories, but also improve the model's performance in different scenarios, providing more reliable and fair technical support for practical applications.

[0102] (5) Model evaluation and adjustment

[0103] Finally, the model is evaluated using a variety of methods and indicators to detect and mitigate bias. The methods include manual evaluation, automatic evaluation, and hybrid evaluation, and the indicators include accuracy, sentiment analysis, fairness, etc. These evaluations and indicators can provide feedback on bias in the model output, help continuously improve the model, and ensure that it performs more fairly and accurately in diverse scenarios.

[0104] To ensure fair and accurate performance across diverse scenarios, models need to be evaluated using a variety of methods and metrics to detect and mitigate bias. These evaluation methods include manual, automatic, and hybrid evaluations, while the metrics used include accuracy, sentiment analysis, fairness, and more. The following is a detailed implementation process:

[0105] Manual evaluation is a subjective assessment of model outputs by human reviewers. Reviewers are typically domain experts or trained personnel who can conduct a detailed analysis of the model's performance based on the specific requirements and standards of the task. The advantage of manual evaluation is its flexibility and depth, which can detect subtle deviations and problems that are difficult to detect through automatic evaluation. Specific steps include: selecting representative samples from the model's output, including outputs of different categories and attributes. Establishing evaluation criteria: Developing detailed evaluation criteria and scoring rules based on task requirements to ensure the consistency and fairness of the evaluation process. Reviewers score each sample according to the evaluation criteria and record detailed review comments and suggestions. Summarize the reviewers' scores and opinions, analyze the deviations and problems in the model output, and provide improvement suggestions.

[0106] Automatic evaluation uses predefined algorithms and metrics to objectively assess model outputs. The advantages of automatic evaluation lie in its high efficiency and wide coverage. It can quickly process large numbers of samples and provide quantitative evaluation results. Commonly used automatic evaluation metrics include: Measuring the consistency between model predictions and actual results, which is a fundamental indicator for evaluating model performance. Sentiment analysis algorithms are used to assess the sentiment in model outputs and detect any emotional bias. Fairness metrics (such as the balanced error rate and inequality index) are used to assess the fairness of the model across different categories or attributes.

[0107] Hybrid evaluation combines the strengths of both manual and automated evaluation, providing both detailed subjective analysis and extensive quantitative assessment. The steps include: Using automated evaluation methods to initially screen model outputs, flagging potential biases and problematic samples. Manual review of these flagged samples provides a detailed analysis of the causes of biases and issues. The results of automated and manual review are then combined to produce a comprehensive evaluation report.

[0108] Accuracy is a fundamental metric for measuring model performance, reflecting the correctness of the model's predictions. In actual evaluation, methods such as confusion matrices can be used to analyze the model's accuracy across different categories and attributes, identifying areas with significant deviations. Sentiment analysis can detect the emotional tendencies in model output and assess the fairness of its emotional expression. Specific methods include sentiment classification of model-generated text, statistically analyzing the sentiment distribution across different categories or attributes, and detecting significant sentiment bias. If the model tends to use negative sentiment terms when describing certain groups, further adjustments and optimization are necessary.

[0109] Fairness metrics are used to assess a model's fairness across different categories or attributes, ensuring that its performance across diverse scenarios is not significantly biased. Common fairness metrics include the balanced error rate (which measures whether prediction errors across categories are balanced) and the inequality index (which measures the degree of inequality in prediction results across different groups). These metrics can be used to quantify the fairness of a model and identify areas for improvement.

[0110] Develop a detailed evaluation plan based on the specific application scenarios and requirements of the model, and determine the evaluation methods, indicators, and processes. The evaluation plan should include sample selection criteria, evaluation cycle, reviewer training, etc. Prepare the data samples required for the evaluation, including model output and actual results. For sentiment analysis and fairness assessment, additional labeled data may be required. Clean and preprocess the data to ensure data quality. Execute the evaluation process according to the evaluation plan. Automatic evaluation can be processed using predefined algorithms and tools, while manual evaluation requires reviewers to conduct scoring and analysis. For hybrid evaluation, automatic evaluation screening can be performed first, followed by manual review.

[0111] Summarize the evaluation results and conduct detailed analysis and interpretation. Identify model biases and issues, and provide specific improvement suggestions. Provide feedback to the model development team to guide subsequent optimization efforts. Based on the evaluation results and feedback, continuously optimize the model and dataset. Conduct regular evaluations to ensure the model performs more fairly and accurately in diverse scenarios. Improve the model's performance and fairness through continuous iteration and improvement.

[0112] By using a variety of evaluation methods and metrics, we can comprehensively detect and mitigate bias in models, ensuring fair and accurate performance across different scenarios. This not only helps improve model reliability and user experience, but also enables broader societal benefits in practical applications.

[0113] The following is a specific application example of this application method:

[0114] S101: Diversity in Data Collection and Annotation

[0115] First, ensure that the dataset comes from a diverse set of sources, encompassing different languages, cultures, and social backgrounds. This step aims to reduce the impact of single-source bias by increasing the diversity of data sources. This involves obtaining text data from a variety of websites, books, social media platforms, and more, ensuring that the model is exposed to diverse content, thereby better balancing different social and cultural perspectives during training.

[0116] S102: Fair Resampling Algorithm

[0117] After data collection, the dataset is processed using a fair resampling algorithm. First, a statistical analysis of the dataset is performed to identify biased categories or attributes. Then, based on the number and importance of samples in these categories or attributes, the data is weighted or resampled to make the dataset more balanced overall. This step aims to reduce bias in model training by adjusting the sample distribution to reduce the overrepresentation of certain categories or attributes in the dataset.

[0118] S103: Adversarial Training Techniques

[0119] After resampling, adversarial training techniques are incorporated to further enhance the diversity and fairness of the dataset. Adversarial training generates adversarial examples, forcing the model to face more complex and varied inputs during training, thereby improving the model's generalization and robustness. This approach not only effectively reduces the model's overfitting to specific categories or attributes, but also improves its performance in various scenarios.

[0120] S104: Data Augmentation and Diversification

[0121] Data augmentation techniques, such as synonym replacement and word order permutation, can be used to increase the diversity of the dataset. These techniques can generate more training samples without changing the semantics of the data, thereby improving the robustness and generalization of the model. Furthermore, techniques such as generative adversarial networks (GANs) can be used to generate new samples, further enriching the dataset.

[0122] S105: Model Evaluation and Adjustment

[0123] Finally, models are evaluated using a variety of methods and metrics to detect and mitigate bias. Methods include manual, automated, and hybrid evaluation, and metrics include accuracy, sentiment analysis, and fairness. These evaluations and metrics provide feedback on bias in model outputs, helping to continuously improve the model and ensure its fair and accurate performance in diverse scenarios.

[0124] In another embodiment, the present invention provides a data set production model based on LLM, such as Figure 3 As shown, including:

[0125] The data collection module is used to collect data and form a dataset with diverse annotations to ensure that the data in the dataset has diverse sources;

[0126] The fair resampling module is used to process the dataset using a fair resampling algorithm. First, the data in the dataset is statistically analyzed to identify data categories or attributes with deviations. Then, based on the number and importance of data samples in these data categories or attributes, the data in the dataset is weighted or resampled to make the dataset more balanced overall.

[0127] Adversarial training module, which uses adversarial training techniques to enhance the diversity and fairness of datasets;

[0128] Evaluation module, used for evaluation and adjustment through indicators.

[0129] In another embodiment, the present invention provides an electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the LLM-based dataset production method as described above is implemented.

[0130] In another embodiment, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the above-mentioned LLM-based dataset production method.

[0131] In the embodiments disclosed herein, computer storage media can be tangible media that can contain or store programs for use by or in conjunction with an instruction execution system, device, or apparatus. Computer storage media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. More specific examples of computer storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0132] This paper employs a dataset enhancement algorithm that prioritizes diversity and fairness. By resampling and weighting to balance the distribution of data across categories and attributes, combined with adversarial training techniques to generate adversarial examples, this method significantly reduces data bias. Data enhancement techniques and multiple evaluation methods are used to continuously improve the model, ensuring its fairness and robustness in diverse scenarios, thereby enhancing its performance in practical applications.

[0133] This application's solution primarily targets businesses that require large-scale, high-quality datasets to train natural language processing models, such as chatbots, language translation, and content generation. It addresses the inefficiency, bias, and difficulty handling diverse scenarios of traditional manual data annotation and collection methods, thereby improving the fairness and robustness of model training. The commercial value of this technology lies in its ability to significantly reduce dataset production costs, improve data quality and model training efficiency, ultimately enhancing the competitiveness and market value of products and services, while also bringing higher user satisfaction and trust to businesses.

[0134] In the future, this efficient LLM-based dataset production technology may be widely used in fields such as automatic customer service systems, intelligent assistants, medical diagnosis, legal consultation, and educational counseling, improving the intelligence and service levels of various industries by providing fairer, more accurate, and more diverse language models.

[0135] The above description is only a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Any simple modification, equivalent replacement and improvement made by any technician familiar with the profession to the above embodiment without departing from the scope of the technical solution of the present invention and based on the technical essence of the present invention shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A method for preparing a data set based on LLM, characterized in that: The following steps are involved: Conduct data collection to form a data set with diverse annotations to ensure that the data in the data set comes from diverse sources; The fair resampling algorithm is used to process the data set: first, statistical analysis is performed on the data in the data set to identify the data categories or attributes with deviations; then, based on the number and importance of data samples of these data categories or attributes, the data in the data set is weighted or resampled to make the data set more balanced overall; Adopt adversarial training techniques to enhance the diversity and fairness of datasets; Evaluate and adjust through indicators.

2. The method for preparing a data set based on LLM according to claim 1, characterized in that: After data collection, the data is cleaned and preprocessed, including removing duplicate data, correcting spelling errors, standardizing text formats, and labeling and annotating data. It also includes using data augmentation technology to convert text data into multiple languages ​​through translation and back-translation technology to increase language diversity. A feedback mechanism is established for the dataset with diverse annotations to collect feedback from users and experts on the dataset, identify and correct possible biases and errors, and improve diversity and fairness through iterative optimization of the dataset and training process.

3. The method for preparing a data set based on LLM according to claim 1, characterized in that: The fair resampling algorithm is used to process the data set, including: refining the data set through multi-level classification; regularly performing data statistical analysis to identify the latest deviations in the data set and dynamically adjusting the weights of each category; combining oversampling and undersampling techniques to balance the data set; building a more comprehensive data set by fusing multi-source data and resampling it; and establishing a continuous monitoring and feedback optimization mechanism.

4. The method for preparing a data set based on LLM according to claim 1, characterized in that: The method of using adversarial training technology to enhance the diversity and fairness of the data set is to generate adversarial samples through adversarial training, forcing the training process to face more complex and variable inputs, thereby improving generalization ability and robustness; the adversarial training method uses FGSM to generate adversarial samples by calculating the gradient of the input data and adding small perturbations in the gradient direction. The specific process includes: Calculate gradient: For each training sample, calculate the gradient of the loss function with respect to the input data; Add perturbation: Add a small perturbation to the input data in the direction of the gradient. The size of the perturbation is controlled by a hyperparameter ε. Generate adversarial samples: Use the perturbed data as adversarial samples; The formula for generating adversarial samples using FGSM is: x is the original input, ε is the perturbation amplitude, is the gradient of the loss function with respect to the input data, x adv For adversarial samples, sign() represents a mathematical function that returns an integer variable, which is used to indicate the positive and negative signs of the parameters.

5. The method for preparing a data set based on LLM according to claim 1, characterized in that: The method of evaluating and adjusting through indicators includes manual evaluation, automatic evaluation and hybrid evaluation, and the indicators used include accuracy, sentiment analysis and fairness.

6. A data set production model based on LLM, characterized in that: include: The data collection module is used to collect data and form a data set with diverse annotations to ensure that the data in the data set has diverse sources; The fair resampling module is used to process the data set using a fair resampling algorithm: first, statistical analysis is performed on the data in the data set to identify the data categories or attributes with deviations; then, based on the number and importance of data samples of these data categories or attributes, the data in the data set is weighted or resampled to make the data set more balanced overall; Adversarial training module, which uses adversarial training techniques to enhance the diversity and fairness of datasets; Evaluation module for evaluation and adjustment through indicators.

7. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for producing a data set based on LLM as described in any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the LLM-based dataset production method according to any one of claims 1 to 5.

Images