Method for generating questions and answers based on large model

Abstract

The invention discloses a method for generating questions and answers based on a large model, and the method comprises the steps: firstly carrying out the pre-training of the large model, then carrying out the fine tuning of the pre-trained model through the construction of a fine tuning sample and the optimization of a data combination mode, and employing a parameter fine tuning technology, and finally enabling the obtained large model to generate stable questions and answers. According to the method, the large model in the question and answer field can be efficiently trained under the condition that resources are limited, and high-quality output is achieved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a method for generating questions and answers based on a large model. Background Art

[0002] With the rapid development of artificial intelligence technology, deep learning models have shown excellent performance in many fields. However, model optimization for specific tasks remains a challenge.

[0003] At present, the mainstream methods for generating question answers mainly include the following categories:

[0004] (1) Full Fine-tuning

[0005] This is the most basic fine-tuning method, which updates all parameters of the model to adapt to downstream tasks. Although the effect is good, it requires storing a complete copy of the model parameters, which has high computational and storage overhead. For example, for a large model with billions of parameters, full fine-tuning often requires hundreds of GB of GPU memory.

[0006] (2) LoRA

[0007] This method represents the weight update by adding a low-rank decomposition next to the original weight matrix. Specifically, it decomposes the weight matrix to be learned into the product of two smaller matrices, significantly reducing the number of parameters that need to be optimized. This method maintains good performance while greatly reducing storage and computational overhead.

[0008] (3)QLoRA

[0009] Based on LoRA, 4-bit quantization technology is introduced to further reduce memory usage by quantizing the model backbone parameters to a lower numerical precision. This improvement enables fine-tuning of large models on consumer-grade GPUs while maintaining a performance level comparable to the original LoRA.

[0010] (4) Adapter Tuning

[0011] Fine-tuning is achieved by inserting a new adapter layer into the converter layer of the pre-trained model. These adapters usually adopt a bottleneck structure, that is, first reduce the dimension and then increase the dimension, thus limiting the number of parameters that need to be trained. During the training process, the parameters of the original model are frozen and only the parameters of the adapter layer are updated.

[0012] (5) Prefix Tuning

[0013] This method adds some trainable continuous vectors (prefixes) to the front of the input sequence, and optimizes these prefix vectors to achieve adaptation to specific tasks. Compared with full fine-tuning, prefix adjustment only requires training a small number of parameters.

[0014] (6) Prompt Tuning

[0015] Similar to prefix adjustment, but only adds a small number of trainable tokens to the input. These soft prompts can be optimized by gradient descent to make the model better fit the target task. This method is particularly suitable for natural language generation tasks.

[0016] (7) P-Tuning and P-Tuning v2

[0017] P-Tuning introduces a cue encoder based on cue tuning to convert discrete cue tokens into continuous representations. P-Tuning v2 further expands this idea by adding trainable cue parameters to each layer in the deep transformer, improving the model's performance on various downstream tasks.

[0018] These traditional methods have their own characteristics: LoRA / QLoRA achieves a good balance between parameter efficiency and performance; adapter adjustment supports multi-task learning through modular design; prefix adjustment and prompt adjustment achieve task adaptation while keeping the main body of the model unchanged; P-Tuning series methods improve model performance through an improved prompt mechanism.

[0019] However, the traditional method of generating question and answer has the problems of low efficiency and high resource consumption. Therefore, how to further improve the efficiency of generating question and answer while maintaining the performance of the model? Summary of the invention

[0020] In view of the shortcomings of the prior art, the present invention proposes a fine-tuning training method for a large model in the question-answering field. The specific technical solution is as follows:

[0021] A method for generating questions and answers based on a large model, the method comprising the following steps:

[0022] Step 1: Build a pre-training dataset;

[0023] Collect large-scale, high-quality data related to the target question-answering domain and pre-process the data to ensure data quality; use data enhancement technology to increase data diversity and thus construct a pre-training data set;

[0024] Step 2: Select and use the pre-training data set to train a pre-trained large model suitable for the target question-answering field. The pre-trained large model is a Transformer-based model structure, including an input layer, multiple hidden layers, and an output layer; the input layer is used to receive the original input data and convert it into a numerical vector that the large model can understand; the hidden layer contains multiple attention heads, each of which is responsible for focusing on different parts of the input sequence; the outputs of these attention heads are merged by splicing and passed to a feedforward neural network to further extract and convert features; the output layer is used to convert the output of the hidden layer into the final task result;

[0025] Step 3: Construct fine-tuning samples;

[0026] Collect electronic documents and knowledge graphs containing professional knowledge in the target question-answering field, extract the professional knowledge, and standardize the format of the extracted content; then use natural language processing technology to identify key information and concepts in the standardized text content; then perform data cleaning to remove irrelevant or redundant information and retain data that is useful for large model learning; finally, based on the results of analysis and cleaning, build a corpus suitable for model learning; optimize the constructed corpus, and the optimization measures include sample balance, feature selection, and sample weighting according to the characteristics of the target question-answering field to ensure that large model training is close to actual application scenarios;

[0027] Step 4: Optimize the combination of different data sources or data sets through data integration, feature engineering, data dimension reduction, data balancing and data fusion;

[0028] Step 5: Use parameter fine-tuning technology LoRA or Q-LoRA to update the weight matrix of the model pre-trained in step 2 by introducing a low-rank structure, and use the weights of the updated model to generate questions and answers.

[0029] Furthermore, in step 2, for classification tasks, the output layer is a softmax layer for calculating the probability distribution of each category; for generation tasks, the output layer is a linear layer or an embedding layer for generating words or characters in the vocabulary.

[0030] Furthermore, in the step five, when training the pre-trained large model suitable for the target question-answering field, the PEFT tool is used to supervisely fine-tune the model parameters, and only the most critical parameters for the target question-answering field are fine-tuned, without updating all the parameters of the model.

[0031] Furthermore, in step five, the PEFT tool is used to supervisely fine-tune the model parameters, including full-layer fine-tuning, top-level fine-tuning, layer-by-layer fine-tuning and transfer learning.

[0032] Furthermore, after executing step 4, the data is first divided into different qualities according to the quality of the data resources, and then the large model is gradually trained from low-quality data to high-quality data to enhance the large model's understanding and adaptability to the target task.

[0033] Furthermore, in the step 2, when training the pre-trained large model, for long sequence data, the sequence parallelism technology of model parallelism or data parallelism is adopted to improve the model training efficiency.

[0034] A device for generating questions and answers based on a large model includes one or more processors for implementing a method for generating questions and answers based on a large model.

[0035] An electronic device, comprising:

[0036] one or more processors;

[0037] A storage device is used to store one or more programs. When the one or more programs are executed by the electronic device, the electronic device implements a method for generating questions and answers based on a large model.

[0038] A computer-readable storage medium stores a program, which, when executed by a processor, implements a method for generating questions and answers based on a large model.

[0039] The beneficial effects of the present invention are as follows:

[0040] The method for generating questions and answers based on a large model of the present invention has a fast training and reasoning speed for specific question and answer tasks, and can also achieve high-quality output under limited computing resources. Especially for complex tasks that process sequence data, the large model of the question and answer field obtained by the training method of the present invention can provide users with a high-quality and convenient conversation experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A flowchart of a method for generating questions and answers based on a large model according to an embodiment of the present invention.

[0042] Figure 2 Schematic diagram of the principle of fine-tuning training of a large model in an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The present invention will be described in detail below based on the accompanying drawings and preferred embodiments, and the purpose and effects of the present invention will become more clear. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0044] like Figure 1 As shown, the method for generating questions and answers based on a large model in this embodiment includes the following steps:

[0045] Step 1: Build a pre-training dataset;

[0046] This step is the basis of the entire SFT process. Collect large-scale, high-quality datasets related to specific tasks. Data preprocessing includes operations such as denoising, standardization, and normalization to ensure data quality. In addition, data enhancement techniques such as random cropping, flipping, and rotation are used to increase data diversity and improve the generalization ability of the model.

[0047] Step 2: Model selection and initialization;

[0048] Select and use the pre-training data set to train a pre-trained large model suitable for the target question-answering field. The pre-trained large model is a Transformer-based model structure, which is a neural network structure that performs well in processing sequence data. The core of the Transformer architecture is the self-attention mechanism, which allows the model to pay attention to other elements in the input sequence when processing each input element, thereby capturing the global dependencies in the sequence. This mechanism enables the model to understand contextual information more accurately when processing sequence data such as natural language and time series, thereby improving the performance of the model. In terms of the specific model structure, the model consists of an input layer, multiple hidden layers, and an output layer.

[0049] Input layer: Responsible for receiving raw input data, which can be text, images, or other forms of information, depending on the application scenario of the model. For text data, the input layer usually performs word embedding processing to convert the text into a numerical vector that the model can understand. This process not only retains the semantic information of the text, but also provides a basis for subsequent model processing.

[0050] Hidden layer: The hidden layer is based on the Transformer architecture, the core of which is the self-attention mechanism, which allows the model to focus on other elements in the input sequence while processing each input element, thereby capturing global context information. This mechanism greatly improves the model's ability to understand and process complex tasks.

[0051] In the Transformer architecture, each hidden layer usually contains multiple attention heads, each of which is responsible for focusing on a different part of the input sequence. The outputs of these attention heads are combined (by concatenation) and then passed to a feedforward neural network to further extract and transform features.

[0052] In addition, the Transformer architecture also introduces the concept of positional encoding to make up for the defect that the self-attention mechanism cannot directly handle the order of elements in the sequence. Positional encoding is usually added to the model together with the input embedding to ensure that the model can understand the relative position relationship between elements.

[0053] Output layer: The final output layer is responsible for converting the output of the hidden layer into the final task result. For classification tasks, the output layer may be a softmax layer to calculate the probability distribution of each category. For generation tasks, the output layer may be a linear layer or an embedding layer to generate words or characters in the vocabulary.

[0054] Step 3: Construct fine-tuning samples;

[0055] First, determine the data sources for fine-tuning sample construction, including electronic files such as design documents, employee training materials, and industry knowledge graphs. For knowledge graphs, extract triples from the knowledge graph and assemble them into training texts containing professional knowledge in the target field to inject industry-specific knowledge and improve the professionalism and pertinence of the model in the industry. There are several sub-stages for processing electronic file data samples:

[0056] Sub-phase 1: Data reading and format standardization

[0057] Use file reading tools to read electronic files in the target field, automatically identify file formats and select corresponding reading strategies to ensure that electronic files of different formats can be effectively read and processed. Standardize the format of the extracted text content, including removing redundant formatting marks, unifying encoding, etc., to facilitate subsequent processing. For example, convert image text in PDF files into editable text format.

[0058] Sub-stage 2: Semantic analysis and corpus construction

[0059] Conduct in-depth semantic analysis on the standardized text content, using natural language processing techniques such as named entity recognition and dependency syntax analysis to identify key information and concepts. Then perform data cleaning to remove irrelevant or redundant information and retain data useful for model learning. Based on the results of analysis and cleaning, build a corpus suitable for model learning, including annotated data, training samples, etc.

[0060] Sub-stage 3: Corpus optimization

[0061] The constructed corpus is optimized to improve the learning efficiency and effect of the model. The optimization measures include sample balance, feature selection, etc., and sample weighting is performed according to the characteristics of the target field to ensure that the model training is closer to the actual application scenario and improve the applicability and effectiveness of the model in specific fields.

[0062] Step 4: Data combination optimization

[0063] Optimize the combination of different data sources or data sets to improve the overall value of the data and the accuracy of the analysis results. Including data integration, feature engineering, data dimensionality reduction, data balance, data fusion, etc.

[0064] After executing step 4, in order to enhance the big model's understanding and adaptability to the target task, the data can be divided into different quality data according to the quality of the data resources, and then the big model can be gradually trained from low-quality data to high-quality data.

[0065] Step 5: Efficiently fine-tune parameters and generate questions and answers using the updated model weights.

[0066] like Figure 2 As shown, parameter-efficient fine-tuning techniques such as LoRA or Q-LoRA are applied to update the weight matrix of the model by introducing a low-rank structure. This method only updates a small part of the model's parameters, thereby reducing the demand for computing resources while maintaining model performance. The fine-tuning process uses the PEFT (Parameter-Efficient Fine-Tuning) tool, which uses an innovative fine-tuning strategy to update only a small part of the model's parameters while keeping the remaining parameters fixed, thereby achieving an effect close to full parameter fine-tuning while significantly reducing the demand for computing and storage resources. This optimization method is particularly suitable for resource-constrained environments, such as small devices or edge computing scenarios, where models that require a lot of computing resources may not be deployed and run. The working principle and key features of the PEFT tool include:

[0067] Selective parameter updates: The PEFT tool uses an algorithm to determine which parameters are most critical to a specific task and only fine-tunes those parameters instead of updating all parameters of the model.

[0068] Transfer learning optimization: Using the principle of transfer learning, the PEFT tool retains the common language features in the pre-trained model, while improving the adaptability and performance of the model by fine-tuning key parameters to adapt to new tasks.

[0069] Computational efficiency: By updating only a subset of parameters, the PEFT tool reduces the amount of computation required during the model training and inference phases, allowing the model to run on devices with lower computing power.

[0070] Optimized memory usage: The PEFT tool reduces the memory usage during model updates, allowing the model to be fine-tuned on memory-constrained devices.

[0071] Flexibility and scalability: The design of the PEFT tool allows it to be compatible with different types of pre-trained models, supporting fine-tuning for a variety of natural language processing tasks.

[0072] Cost-effectiveness: In resource-constrained environments, the PEFT tool Lora reduces the cost of model deployment and operation by reducing the required hardware resources.

[0073] Supervised fine-tuning includes but is not limited to full-layer fine-tuning, top-layer fine-tuning, layer-by-layer fine-tuning, and transfer learning. When fine-tuning the top layer, keep the bottom-layer parameters of the pre-trained model unchanged to take advantage of transfer learning.

[0074] In addition, for long sequence data processing, sequence parallelism technology, such as model parallelism or data parallelism, is used to improve training efficiency. This technology can make full use of the parallel computing capabilities of modern GPUs to accelerate the model training process.

[0075] During the training process, we used methods such as cross-validation and continuous integration, and adopted a series of evaluation indicators to monitor the model training process in real time to ensure the quality of model optimization. Based on the evaluation results, we adjusted the model parameters and training strategies to optimize the model performance. Finally, we used the weights of the optimized model to generate questions and answers.

[0076] Through the training method of the present invention, an efficient, stable and intelligent system that can adapt to dialogue tasks can be obtained, which can provide users with high-quality and convenient dialogue experience.

[0077] The method for generating question and answer based on a large model in this embodiment can significantly improve the adaptability and performance of the model for specific tasks by adopting a multi-stage training strategy, efficient parameter fine-tuning techniques LoRA and Q-LoRA, and multi-task learning. The training efficiency is improved by introducing different learning rates for matrices A and B in LoRA, especially setting the learning rate of matrix B higher so that parameters can be adjusted faster, thereby speeding up the training speed and slightly improving the model accuracy.

[0078] In addition, the introduction of sequence parallelism technology optimizes the processing of long sequence data, improves the training and reasoning speed, and enables high-quality output even with limited computing resources. The present invention demonstrates excellent performance in an unsupervised learning environment, and can generate content that meets user needs without relying on pre-labeled data.

[0079] Corresponding to the above-mentioned fine-tuning training method for a large model in the question and answer field, the present invention also provides an embodiment of a device for generating questions and answers based on a large model.

[0080] The apparatus for generating questions and answers based on a large model in this embodiment includes one or more processors for implementing the method for generating questions and answers based on a large model in the above-mentioned embodiment.

[0081] The embodiment of the device for generating questions and answers based on a large model of the present invention can be applied to any device with data processing capabilities, and the any device with data processing capabilities can be a device or apparatus such as a computer. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of any device with data processing capabilities in which it is located reading the corresponding computer program instructions in the non-volatile memory into the memory and running. From the hardware level, the device for generating questions and answers based on a large model of the present invention may also include other hardware in addition to the processor, memory, network interface, and non-volatile memory, which will not be described in detail. For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0082] An embodiment of the present invention also provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the method for generating questions and answers based on a large model in the above embodiment is implemented.

[0083] The computer-readable storage medium may be an internal storage unit of any device with data processing capability described in any of the aforementioned embodiments, such as a hard disk or a memory. The computer-readable storage medium may also be an external storage device, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of any device with data processing capability and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capability, and may also be used to temporarily store data that has been output or is to be output.

[0084] Those skilled in the art can understand that the above are only preferred examples of the invention and are not intended to limit the invention. Although the invention is described in detail with reference to the above examples, those skilled in the art can still modify the technical solutions recorded in the above examples or replace some of the technical features therein with equivalents. Any modification, equivalent replacement, etc. made within the spirit and principle of the invention shall be included in the protection scope of the invention.

Claims

1. A method for generating questions and answers based on a large model, characterized in that: The method comprises the following steps: Step 1: Build a pre-training dataset; Collect large-scale, high-quality data related to the target question-answering domain and pre-process the data to ensure data quality; use data enhancement technology to increase data diversity and thus construct a pre-training data set; Step 2: Select and use the pre-training data set to train a pre-trained large model suitable for the target question-answering field. The pre-trained large model is a Transformer-based model structure, including an input layer, multiple hidden layers, and an output layer; the input layer is used to receive the original input data and convert it into a numerical vector that the large model can understand; the hidden layer contains multiple attention heads, each of which is responsible for focusing on different parts of the input sequence; the outputs of these attention heads are merged by splicing and passed to a feedforward neural network to further extract and convert features; the output layer is used to convert the output of the hidden layer into the final task result; Step 3: Construct fine-tuning samples; Collect electronic documents and knowledge graphs containing professional knowledge in the target question-answering field, extract the professional knowledge, and standardize the format of the extracted content; then use natural language processing technology to identify key information and concepts in the standardized text content; then perform data cleaning to remove irrelevant or redundant information and retain data that is useful for large model learning; finally, based on the results of analysis and cleaning, build a corpus suitable for model learning; optimize the constructed corpus, and the optimization measures include sample balance, feature selection, and sample weighting according to the characteristics of the target question-answering field to ensure that large model training is close to actual application scenarios; Step 4: Optimize the combination of different data sources or data sets through data integration, feature engineering, data dimension reduction, data balancing and data fusion; Step 5: Use parameter fine-tuning technology LoRA or Q-LoRA to update the weight matrix of the model pre-trained in step 2 by introducing a low-rank structure, and use the weights of the updated model to generate questions and answers.

2. The method for generating questions and answers based on a large model according to claim 1, characterized in that: In step 2, for classification tasks, the output layer is a softmax layer for calculating the probability distribution of each category; for generation tasks, the output layer is a linear layer or an embedding layer for generating words or characters in the vocabulary.

3. The method for generating questions and answers based on a large model according to claim 1, characterized in that: In the step five, when training the pre-trained large model suitable for the target question-answering field, the PEFT tool is used to supervisely fine-tune the model parameters, and only the most critical parameters for the target question-answering field are fine-tuned, without updating all the parameters of the model.

4. The method for generating questions and answers based on a large model according to claim 3, characterized in that: In the step five, the PEFT tool is used to supervise the fine-tuning of model parameters, including full-layer fine-tuning, top-level fine-tuning, layer-by-layer fine-tuning and transfer learning.

5. The method for generating questions and answers based on a large model according to claim 1, characterized in that: After executing step 4, first sort the data into different qualities according to the source based on the quality of the data resources, and then gradually train the large model from low-quality data to high-quality data to enhance the large model's understanding and adaptability to the target task.

6. The method for generating questions and answers based on a large model according to claim 1, characterized in that: In the step 2, when training the pre-trained large model, for long sequence data, the sequence parallelism technology of model parallelism or data parallelism is adopted to improve the model training efficiency.

7. A device for generating questions and answers based on a large model, characterized in that: The method comprises one or more processors for implementing the method for generating questions and answers based on a large model as described in any one of claims 1 to 6.

8. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the electronic device, enables the electronic device to implement the method for generating questions and answers based on a large model as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, the method for generating questions and answers based on a large model as described in any one of claims 1 to 6 is implemented.

Images