Improvement method and improvement device of Lora-based large model

Abstract

The embodiment of the invention provides an improvement method and an improvement device for a large model based on Lora, and solves the technical problems that training data for preparing model training is purely manually, and automatic upgrading is difficult to realize after model training is completed. According to the Lora-based large model improvement method provided by the invention, in the upgrading and improvement process of the large model, the new questions and answers are input into the large model for calculation to obtain the reasoning answers corresponding to the new questions, and the questions and the corresponding reasoning answers are used as the training set to further train the large model to improve the large model. A large model is adopted to perform a question and answer method, so that the large model does not need a large amount of manual training data, and the large model can be automatically upgraded in the use process.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to an improved method and an improved device based on a Lora large model. Background Art

[0002] Training a large model is a complex process involving multiple stages and a large amount of computing resources. The general steps for training a large model (such as a large language model) are: data collection and preprocessing, model design, computing resource preparation, model training, model evaluation, tuning and optimization, etc. The most important factors affecting large models are data collection, cleaning, and preprocessing. The quantity and quality of data have a profound impact on the output of large models. First, 1) Sufficient data: More data can help the model learn more complex patterns and improve its generalization ability, so that it can perform better on unseen data, and it is more helpful to reduce model bias because the model can learn from a wider range of perspectives. High-quality data: Ensure that the model learns accurate patterns and outputs more accurate results, so that it can be better generalized to new data sets. Secondly, 2) Sufficient high-quality data is the basis for training a high-performance, generalized and fair model. Insufficient or low-quality data may lead to reduced model performance, increased bias, and even misleading results in practical applications. Collecting high-quality data requires avoiding interference from irrelevant data, but it requires a lot of manual work or the use of algorithms to correct errors in the data, such as using a professional annotation team to ensure the quality of annotations and ensure that different annotation personnel have consistent annotation results for the same data. Then, when preparing training data, people in the computing field have to cooperate continuously, and there is no automated processing capability, which reduces the efficiency of model training and makes it difficult to achieve automatic model upgrades. Summary of the invention

[0003] In view of this, the embodiments of the present invention provide an improved method and an improved device for a large model based on Lora, which solves the technical problems that the preparation of training data for model training depends purely on manual work and it is difficult to achieve automatic upgrading after the model training is completed.

[0004] As a first aspect of the present invention, the present invention provides an improved method based on a large model of Lora, comprising:

[0005] Get multiple question data;

[0006] Input the multiple question data into the Lora-based large model for calculation to generate an inference answer corresponding to each question data, wherein the question data and the corresponding inference answer constitute first training data, and the multiple first training data constitute a first training data set;

[0007] Based on preset hyperparameters, the first training data set is input into the large model for training to improve the large model.

[0008] In one embodiment of the present invention, the first training data set includes:

[0009] A first forward training data set, wherein the first forward training data set includes a plurality of first forward training data, and the first forward training data includes a first question related to a business and a first inference answer corresponding to the first question;

[0010] A first reference training data set, wherein the first reference training data set includes a plurality of first reference training data, wherein the first reference training data includes a second question unrelated to the business and a second inference answer corresponding to the second question.

[0011] In one embodiment of the present invention, based on preset hyperparameters, inputting the first training data set into the large model for training includes:

[0012] Inputting the first reference training data set and the first forward training data set into the large model for Lora fine-tuning training to obtain the trained first Lora parameters;

[0013] The large model is upgraded based on the first Lora parameters.

[0014] In one embodiment of the present invention, based on preset hyperparameters, the first training data set is input into the large model for training, further comprising:

[0015] Performing positive annotation on the first question to generate a label answer corresponding to the first question, wherein the first question and the corresponding label answer constitute second positive training data, and a plurality of second positive training data constitute a second positive training data set;

[0016] Performing positive annotation on the second question to generate a label answer corresponding to the second question, wherein the second question and the corresponding label answer constitute second reference training data, and a plurality of second reference training data constitute a second reference training data set;

[0017] Inputting the second reference training data set and the second forward training data set into the large model for Lora fine-tuning training to obtain trained second Lora parameters;

[0018] Merging the first Lora parameter and the second Lora parameter to determine a merged Lora parameter;

[0019] Based on the merged Lora parameters, the large model is upgraded.

[0020] In one embodiment of the present invention, the improved method based on the Lora large model further includes:

[0021] Save the upgraded large model to the target directory.

[0022] In one embodiment of the present invention, before inputting the first training data set into the large model for training based on preset hyperparameters to improve the large model, the method for improving the large model based on Lora further includes:

[0023] The inference answer corresponding to each of the question data is verified. If the verification is successful, the question data and the corresponding inference answer constitute the first training data.

[0024] In one embodiment of the present invention, before inputting the first training data set into the large model for training based on preset hyperparameters to improve the large model, the method for improving the large model based on Lora further includes:

[0025] Data filtering is performed on the first training data in the first training data set to generate a filtered first training data set.

[0026] As a second aspect of the present invention, the present invention also provides an improved device based on a large model of Lora, comprising:

[0027] A data acquisition module, used to acquire multiple problem data;

[0028] A data reasoning module, used for inputting the plurality of question data into a large model based on Lora for calculation, so as to generate a reasoning answer corresponding to each of the question data, wherein the question data and the corresponding reasoning answer constitute first training data, and the plurality of first training data constitute a first training data set;

[0029] The model training module is used to input the first training data set into the large model for training based on preset hyperparameters to improve the large model.

[0030] In one embodiment of the present invention, the data reasoning module includes:

[0031] A forward reasoning data module, used for inputting a plurality of first question data related to the business into a large model based on Lora for training, so as to generate a first reasoning answer corresponding to each of the first question data, wherein the first question and the corresponding first reasoning answer constitute first forward training data;

[0032] The reverse reasoning data module is used to input multiple business-independent second question data into the Lora-based large model for training to generate a second reasoning answer corresponding to each of the second question data. The second question and the corresponding second reasoning answer constitute the first reference training data.

[0033] In one embodiment of the present invention, the data reasoning module further includes:

[0034] A forward label data module, used to positively label the first question to generate a label answer corresponding to the first question, wherein the first question and the corresponding label answer constitute second forward training data, and a plurality of second forward training data constitute a second forward training data set;

[0035] A reverse label data module is used to positively annotate the second question to generate a label answer corresponding to the second question, wherein the second question and the corresponding label answer constitute second reference training data, and multiple second reference training data constitute a second reference training data set.

[0036] The Lora-based large model improvement method provided by the present invention, in the process of upgrading and improving the large model, inputs new questions and answers into the large model for calculation to obtain the reasoning answers corresponding to the new questions, and uses the questions and the corresponding reasoning answers as training sets to further train the large model to improve the large model. A question-answering method is performed using the large model, so that the large model does not require a large amount of manual training data, and can be automatically upgraded during the use of the large model. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 The figure is a flow chart of an improved method of a large model based on Lora provided by an embodiment of the present invention.

[0038] Figure 2 The figure is a flow chart of an improved method of a large model based on Lora provided by an embodiment of the present invention.

[0039] Figure 3 The figure is a flow chart of an improved method of a large model based on Lora provided by an embodiment of the present invention.

[0040] Figure 4 The figure shows a working block diagram of an improved device based on a large Lora model provided by an embodiment of the present invention.

[0041] Figure 5 Shown is a working block diagram of an improved device based on a Lora large model provided by another embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to enable those skilled in the art to better understand the technical solution of the present disclosure, and to fully understand and implement how the present disclosure applies technical means to solve technical problems and achieve the corresponding technical effects, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only embodiments of a part of the present disclosure, not all of the embodiments. The embodiments of the present disclosure and the various features in the embodiments can be combined with each other without conflict, and the technical solutions formed are all within the scope of protection of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present disclosure.

[0043] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0044] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] Exemplary Methods

[0047] Figure 1 FIG. 1 is a flow chart of an improved method based on a large Lora model according to an embodiment of the present invention. Figure 1 As shown, an improved method based on the large model of Lora specifically includes the following steps:

[0048] S1: Get multiple question data;

[0049] Specifically, the Lora (Low-Rank Adaptation of Large Language) model is a low-rank adaptation of a large model (i.e., a large language model), and is a fine-tuning technology for large models.

[0050] Problem data refers to specific problems, and the problems also include first problems related to the business and second problems unrelated to the business.

[0051] S2: Input multiple question data into the large model based on Lora for calculation to generate an inference answer corresponding to each question data, wherein the question data and the corresponding inference answer constitute first training data, and multiple first training data constitute a first training data set;

[0052] Specifically, multiple question data are input into the large model one by one for inference calculation to obtain the inference answer corresponding to each question data. For example, the first question corresponds to the first inference answer. The second question corresponds to the second inference answer. When multiple question data are input into the large model one by one for calculation, the inference answer corresponding to each question can be obtained. Each question and the corresponding answer are used as a first training data, and multiple first training data constitute a first training data set.

[0053] Optionally, the first training data set includes:

[0054] The first forward training data set includes multiple first forward training data, and the first forward training data includes a first question related to the business and a first inference answer corresponding to the first question; the first inference answer is the inference answer corresponding to the first question output after the first question is input into the large model.

[0055] The first forward training data includes a new question (ie, the first question) and the corresponding reasoning answer (formed through reasoning with a large model).

[0056] The first reference training data set includes multiple first reference training data, and the first reference training data includes a second question that is not related to the business and a second reasoning answer corresponding to the second question. The second reasoning answer is the reasoning answer corresponding to the second question output after the second question is input into the large model. The first reference training data includes a new question (i.e., the second question) and the corresponding reasoning answer (formed by reasoning with the large model). If the second question in the first reference training data is a question that is not related to the business, then the first reference training data can also be used as noise data.

[0057] S3: Based on preset hyperparameters, the first training data set is input into the large model for training to improve the large model.

[0058] Specifically, the preset hyperparameters are parameters that are not learned through model training. For example, the preset hyperparameters include: learning rate, regularization parameter, batch size, number of iterations, hidden layer size and number of hidden layers, locked hidden layers, optimizer selection (such as Adam, SGD, RMSprop, etc.), Dropout rate, etc.

[0059] When the first training data set (the first forward training data set and the first reference training data set) is obtained through S2, it is input into the large model for training to improve the large model.

[0060] The Lora-based large model improvement method provided by the present invention, in the process of upgrading and improving the large model, inputs new questions and answers into the large model for calculation to obtain the reasoning answers corresponding to the new questions, and uses the questions and the corresponding reasoning answers as training sets to further train the large model to improve the large model. A question-answering method is performed using the large model, so that the large model does not require a large amount of manual training data, and can be automatically upgraded during the use of the large model.

[0061] In one embodiment of the present invention, Figure 2 As shown, S3 (based on preset hyperparameters, inputting the first training data set into the large model for training) specifically includes the following steps:

[0062] S31: Inputting the first reference training data set and the first forward training data set into the large model for Lora fine-tuning training to obtain the trained first Lora parameters;

[0063] The large model is further trained using the thrust answers corresponding to the questions calculated and inferred by the large model and the training data set composed of the questions to obtain the first Lora parameters after fine-tuning.

[0064] S310: Upgrading the large model based on the first Lora parameter.

[0065] The large model is upgraded by using the fine-tuned first Lora parameter to play the role of upgrading the large model.

[0066] Optional, such as Figure 3 As shown, S3 (based on preset hyperparameters, inputting the first training data set into the large model for training) also includes the following steps:

[0067] S32: positively annotating the first question to generate a label answer corresponding to the first question, wherein the first question and the corresponding label answer constitute second positive training data, and a plurality of second positive training data constitute a second positive training data set;

[0068] That is to say, the first question is input into the large model for calculation to infer the inference answer, and at the same time the first question is labeled to generate a label answer corresponding to the first question.

[0069] It should be noted that the number of first questions positively annotated in S32 may be the same as the number of first questions for which reasoning answers are calculated in the input model, or may be different (i.e., multiple first questions are selected from multiple first questions calculated in the input large model for annotation).

[0070] S33: Forward annotating the second question to generate a label answer corresponding to the second question, wherein the second question and the corresponding label answer constitute second reference training data, and multiple second reference training data constitute a second reference training data set;

[0071] Similarly, the second question is input into the large model for calculation to infer the answer, and at the same time the second question is labeled to generate a label answer corresponding to the second question.

[0072] It should be noted that the number of second questions positively annotated in S33 may be the same as the number of second questions for which reasoning answers are calculated in the input model, or may be different (i.e., multiple second questions are selected from multiple second questions calculated in the input large model for annotation).

[0073] S34: inputting the second reference training data set and the second forward training data set into the large model for Lora fine-tuning training to obtain trained second Lora parameters;

[0074] The large model is trained using the annotated second training data set (i.e., including the second forward training data set and the second reference training data set) to obtain the fine-tuned second Lora parameters;

[0075] S35: merging the first Lora parameter and the second Lora parameter to determine a merged Lora parameter;

[0076] The fine-tuned Lora parameters obtained after training the large model using the above two different training data sets are merged to obtain the merged Lora parameters.

[0077] S36: Upgrade the large model based on the merged Lora parameters.

[0078] In one embodiment of the present invention, after S3, the improved method based on the large model of Lora further includes the following steps:

[0079] S4: Save the upgraded large model to the target directory. Save the upgraded large model to the agreed directory, and perform cyclic upgrade training without changing the name, thereby reducing the storage of the model. In the next upgrade, the upgraded large model can be directly obtained as the initial large model for the next upgrade.

[0080] In one embodiment of the present invention, before S3 (based on preset hyperparameters, inputting the first training data set into the large model for training to improve the large model), the method for improving the large model based on Lora also includes the following steps:

[0081] S21: verifying the inference answer corresponding to each question data. If the verification is successful, the question data and the corresponding inference answer constitute the first training data.

[0082] Input the question into the big model for calculation and output of the inference answer. If the answers corresponding to multiple questions are compared with the standard answers, the generalization ability or similarity of the big model can be obtained. If the generalization ability is less than the preset value or the similarity is less than the preset similarity, it means that the answer obtained by reasoning has not been verified. In this case, the question and the corresponding inference answer are abandoned, the question is replaced, and the question is input into the big model for inference calculation.

[0083] In one embodiment of the present invention, before S3 (based on preset hyperparameters, inputting the first training data set into the large model for training to improve the large model), the method for improving the large model based on Lora also includes the following steps:

[0084] S20: Filter the first training data in the first training data set to generate a filtered first training data set.

[0085] Specifically, the data filtering method can be:

[0086] A script is used to determine whether the data in the reasoning answer corresponding to each question is too long or whether illegal data appears (for example, Chinese should be output, but it contains code, English, etc.). If the data is too long or illegal data appears, the data in the reasoning answer is filtered, for example, the illegal data is modified to normal data, or the overlong data is deleted so that the data is not too long.

[0087] Improved device

[0088] As a second aspect of the present invention, the present invention also provides an improved device based on the large model of Lora, Figure 4FIG. 1 is a working principle diagram of an improved device based on a large Lora model provided by an embodiment of the present invention, as shown in FIG. Figure 4 As shown, an improved device 1 based on the large model of Lora includes:

[0089] A data acquisition module 10 is used to acquire multiple problem data;

[0090] Specifically, the data acquisition module 10 is used to execute S1 (acquire multiple problem data) in the above-mentioned improved method based on the large model of Lora.

[0091] A data reasoning module 20 is used to input multiple question data into the large model based on Lora for calculation to generate a reasoning answer corresponding to each question data, wherein the question data and the corresponding reasoning answer constitute the first training data, and the multiple first training data constitute the first training data set;

[0092] Specifically, the data reasoning module 20 is used to execute S2 in the above-mentioned improved method based on the Lora big model (used to input multiple question data into the Lora-based big model for calculation to generate the reasoning answer corresponding to each question data).

[0093] Optionally, the first training data set includes:

[0094] The first forward training data set includes multiple first forward training data, and the first forward training data includes a first question related to the business and a first inference answer corresponding to the first question; the first inference answer is the inference answer corresponding to the first question output after the first question is input into the large model.

[0095] The first forward training data includes a new question (ie, the first question) and the corresponding reasoning answer (formed through reasoning with a large model).

[0096] The first reference training data set includes multiple first reference training data, and the first reference training data includes a second question that is not related to the business and a second reasoning answer corresponding to the second question. The second reasoning answer is the reasoning answer corresponding to the second question output after the second question is input into the large model. The first reference training data includes a new question (i.e., the second question) and the corresponding reasoning answer (formed by reasoning with the large model). If the second question in the first reference training data is a question that is not related to the business, then the first reference training data can also be used as noise data.

[0097] The model training module 30 is used to input the first training data set into the large model for training based on preset hyperparameters to improve the large model.

[0098] Specifically, the model training module 30 is used to execute S3 in the above-mentioned method for improving the large model based on Lora (used to input the first training data set into the large model for training based on preset hyperparameters to achieve improvement of the large model).

[0099] Specifically, the preset hyperparameters are parameters that are not learned through model training. For example, the preset hyperparameters include: learning rate, regularization parameter, batch size, number of iterations, hidden layer size and number of hidden layers, locked hidden layers, optimizer selection (such as Adam, SGD, RMSprop, etc.), Dropout rate, etc.

[0100] The Lora-based large model improvement device provided by the present invention, in the process of upgrading and improving the large model, inputs new questions and answers into the large model for calculation to obtain the reasoning answers corresponding to the new questions, and uses the questions and the corresponding reasoning answers as training sets to further train the large model to improve the large model. A question-answering method is performed using the large model, so that the large model does not require a large amount of manual training data, and can be automatically upgraded during the use of the large model.

[0101] In one embodiment of the present invention, Figure 5 As shown, the data reasoning module 20 includes:

[0102] A forward reasoning data module 21 is used to input a plurality of first question data related to the business into a large model based on Lora for training, so as to generate a first reasoning answer corresponding to each first question data, and the first question and the corresponding first reasoning answer constitute the first forward training data;

[0103] The reverse reasoning data module 22 is used to input multiple business-independent second question data into the Lora-based large model for training to generate a second reasoning answer corresponding to each second question data. The second question and the corresponding second reasoning answer constitute the first reference training data.

[0104] The large model is further trained using the thrust answers corresponding to the questions calculated and inferred by the large model and the training data set composed of the questions to obtain the first Lora parameters after fine-tuning.

[0105] A positive label data module 23 is used to positively label the first question to generate a label answer corresponding to the first question, wherein the first question and the corresponding label answer constitute second positive training data, and multiple second positive training data constitute a second positive training data set;

[0106] The first question is input into the large model for calculation to infer the inference answer, and at the same time the first question is annotated to generate a label answer corresponding to the first question.

[0107] The reverse label data module 24 is used to positively label the second question to generate a label answer corresponding to the second question, wherein the second question and the corresponding label answer constitute the second reference training data, and multiple second reference training data constitute the second reference training data set.

[0108] The second question is input into the large model for calculation to infer the answer, and at the same time the second question is annotated to generate a label answer corresponding to the second question.

[0109] The large model is trained using the annotated second training data set (i.e., including the second forward training data set and the second reference training data set) to obtain the fine-tuned second Lora parameters;

[0110] Then the first Lora parameter and the second Lora parameter are merged to determine the merged Lora parameters; the fine-tuning Lora parameters obtained after training the large model using the above two different training data sets are merged to obtain the merged Lora parameters, and the large model is upgraded based on the merged Lora parameters.

[0111] Computer equipment

[0112] On the basis of the above embodiments, this embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in the above embodiments.

[0113] In some implementations of this embodiment, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that when the computer program is executed by a processor, the steps of the method described in the above embodiment are implemented.

[0114] In some implementations of this embodiment, a computer program product is provided, including a computer program / instructions, characterized in that when the computer program is executed by a processor, the steps of the method described in the above embodiment are implemented.

[0115] The processor may include, but is not limited to, one or more processors or microprocessors. Each processor may be an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor or other electronic components to execute the method in the above embodiments.

[0116] The computer-readable storage medium may be implemented by any type of volatile or non-volatile storage device or a combination thereof, and the computer-readable storage medium may include but is not limited to, for example, random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, computer storage media (e.g., hard disk, floppy disk, solid-state drive, removable disk, CD-ROM, DVD-ROM, Blu-ray disc, etc.).

[0117] The computer-readable storage medium may also store at least one computer executable program / instruction, which may be, for example, a computer-readable instruction. The computer-readable storage medium includes, but is not limited to, for example, a volatile memory and / or a non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The computer-readable storage medium may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. For example, a non-transitory computer-readable storage medium may be connected to a computing device such as a computer, and then, when the computing device runs the computer-readable instructions stored on the computer-readable storage medium, the various methods described above may be performed.

[0118] In addition, the computer device may also include (but not limited to) a data bus, an input / output (I / O) bus, a display, and input / output devices (eg, keyboard, mouse, speaker, etc.), etc.

[0119] The processor may communicate with external devices via an I / O bus via a wired or wireless network.

[0120] In one embodiment, the at least one computer executable instruction may also be compiled into or constitute a software product / computer program product, wherein one or more computer executable instructions are executed by a processor to perform the various functions and / or method steps in the embodiments described in the present technology.

[0121] In the embodiments provided in the present disclosure, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0122] It should be noted that in the present disclosure, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element limited by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0123] Although the embodiments disclosed in the present disclosure are as above, the above contents are only embodiments adopted for facilitating the understanding of the present disclosure and are not intended to limit the present disclosure. Any technician in the technical field to which the present disclosure belongs can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in the present disclosure, but the scope of patent protection of the present disclosure shall still be subject to the scope defined in the attached claims.

Claims

1. An improved method based on a large model of Lora, characterized in that: include: Get multiple question data; Input the multiple question data into the Lora-based large model for calculation to generate an inference answer corresponding to each question data, wherein the question data and the corresponding inference answer constitute first training data, and the multiple first training data constitute a first training data set; Based on preset hyperparameters, the first training data set is input into the large model for training to improve the large model.

2. The improved method based on the Lora large model according to claim 1 is characterized in that: The first training data set includes: A first forward training data set, wherein the first forward training data set includes a plurality of first forward training data, and the first forward training data includes a first question related to a business and a first reasoning answer corresponding to the first question; A first reference training data set, wherein the first reference training data set includes a plurality of first reference training data, wherein the first reference training data includes a second question irrelevant to the business and a second inference answer corresponding to the second question.

3. The improved method based on the Lora large model according to claim 2 is characterized in that: Based on preset hyperparameters, inputting the first training data set into the large model for training includes: Inputting the first reference training data set and the first forward training data set into the large model for Lora fine-tuning training to obtain the trained first Lora parameters; The large model is upgraded based on the first Lora parameters.

4. The improved method based on the Lora large model according to claim 3 is characterized in that: Based on preset hyperparameters, the first training data set is input into the large model for training, further comprising: Performing positive annotation on the first question to generate a labeled answer corresponding to the first question, wherein the first question and the corresponding labeled answer constitute second positive training data, and a plurality of second positive training data constitute a second positive training data set; Forward annotating the second question to generate a label answer corresponding to the second question, wherein the second question and the corresponding label answer constitute second reference training data, and multiple second reference training data constitute a second reference training data set; Inputting the second reference training data set and the second forward training data set into the large model for Lora fine-tuning training to obtain trained second Lora parameters; Merging the first Lora parameter and the second Lora parameter to determine a merged Lora parameter; Based on the merged Lora parameters, the large model is upgraded.

5. The improved method based on the Lora large model according to claim 4 is characterized in that: Also includes: Save the upgraded large model to the target directory.

6. The improved method based on the Lora large model according to claim 1 is characterized in that: Before inputting the first training data set into the large model for training based on preset hyperparameters to improve the large model, the method for improving the large model based on Lora also includes: The inference answer corresponding to each of the question data is verified. If the verification is successful, the question data and the corresponding inference answer constitute the first training data.

7. The improved method based on the Lora large model according to claim 1 is characterized in that: Before inputting the first training data set into the large model for training based on preset hyperparameters to improve the large model, the method for improving the large model based on Lora also includes: Data filtering is performed on the first training data in the first training data set to generate a filtered first training data set.

8. An improved device based on a large model of Lora, characterized in that: include: A data acquisition module, used to acquire multiple problem data; A data reasoning module, used for inputting the plurality of question data into a large model based on Lora for calculation, so as to generate a reasoning answer corresponding to each of the question data, wherein the question data and the corresponding reasoning answer constitute first training data, and the plurality of first training data constitute a first training data set; The model training module is used to input the first training data set into the large model for training based on preset hyperparameters to improve the large model.

9. The improved device based on the Lora large model according to claim 8 is characterized in that: The data reasoning module comprises: A forward reasoning data module, used for inputting a plurality of first question data related to the business into a large model based on Lora for training, so as to generate a first reasoning answer corresponding to each of the first question data, wherein the first question and the corresponding first reasoning answer constitute first forward training data; The reverse reasoning data module is used to input multiple business-independent second question data into the Lora-based large model for training to generate a second reasoning answer corresponding to each of the second question data. The second question and the corresponding second reasoning answer constitute the first reference training data.

10. The improved device based on the Lora large model according to claim 9 is characterized in that: The data reasoning module further includes: A forward label data module, used to positively label the first question to generate a label answer corresponding to the first question, wherein the first question and the corresponding label answer constitute second forward training data, and a plurality of second forward training data constitute a second forward training data set; A reverse label data module is used to positively annotate the second question to generate a label answer corresponding to the second question, wherein the second question and the corresponding label answer constitute second reference training data, and multiple second reference training data constitute a second reference training data set.

Images