LLM size model-based collaborative training method, medium and device

Abstract

The invention discloses a collaborative training method, medium and device based on an LLM large and small model, and the method comprises the steps: S1, obtaining a huge knowledge system through the combination of the language understanding capability and pre-training capability of a general large model, and carrying out the cold start of an AI service 0 sample needed by a business scene, and enabling the AI service 0 sample to be online; s2, performing a small amount of annotation on sample data generated by a business scene, performing fine tuning on the general large model to form a scene large model, performing continuous learning to enable the scene large model to have knowledge in the field, and quickly improving an algorithm effect; and S3, distilling knowledge in the field obtained by the scene large model into a plurality of small models, and fusing results of the plurality of small models by using a scoring mechanism to realize collaborative training of the large and small models. According to the method, any scene text service can be subjected to cold start online under the condition of limited hardware resources, large and small model cooperative training of large model knowledge can be obtained through a small amount of labeling, and the model learning ability and the working efficiency are greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of big data and AI technology, and in particular to a method, medium, and device for collaborative training of LLM-based large and small models. Background Art

[0002] With the advent of ChatGPT and large models, AI development has entered the era of general artificial intelligence. Large models, with their vast number of parameters and training data, have achieved outstanding performance in natural language processing. In this field, the emergence of large models has significantly enhanced machines' ability to recognize and understand language. They can more accurately capture the subtle nuances of human language, achieving near-human or even superior performance in a variety of application scenarios, including translation, summarization, question-answering, and writing. This advancement has not only transformed human-computer interaction but has also significantly promoted the widespread adoption of intelligent services and the advancement of intelligence.

[0003] In existing technologies, training small models requires collecting and annotating large amounts of data. The scenarios are complex and diverse, requiring separate data collection and training for each scenario task, and each scenario requires custom development. Large models, on the other hand, offer excellent language understanding capabilities, a vast knowledge base, and are versatile across scenarios. Therefore, this paper proposes a method, medium, and device for collaborative training of large and small models based on LLM. Summary of the Invention

[0004] The present invention provides a method, medium and device based on LLM large and small model collaborative training that can reduce the amount of annotation and quickly improve the algorithm effect, which can solve at least one of the above technical problems.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0006] The method for collaborative training of large and small models based on LLM includes the following steps:

[0007] S1. Combining the language understanding and pre-training capabilities of general large models, we can acquire a vast knowledge system and launch AI services required by business scenarios with zero-sample cold start.

[0008] S2. A small amount of annotation is performed on sample data generated by business scenarios, and the general large model is fine-tuned to form a large scenario model. Through continuous learning, the large scenario model is equipped with domain knowledge, which quickly improves the algorithm effect.

[0009] S3. Distill the domain knowledge obtained from the large scene model into multiple small models, and use the scoring mechanism to fuse the results of multiple small models to achieve collaborative training of large and small models.

[0010] Furthermore, the S1 further includes:

[0011] S1.1. Collect corpus data x and determine the scenario tasks, and determine the label space Y for the collected corpus data x;

[0012] S1.2. The corpus data x and the corresponding label space Y form a text prompt. The text prompt is fed into the general large model M. The general large model M outputs the corresponding answer q1 based on the text prompt.

[0013] S1.3. The service consisting of the general large model M and the application scenario is collectively referred to as service Z. At the same time, the answer space Q of service Z is determined.

[0014] S1.4. Deploy service Z online to meet the needs of rapid implementation of AI services in scenarios, and implement a zero-sample cold start for the scenario service.

[0015] Furthermore, the S2 further includes:

[0016] S2.1. Establish an online backtracking mechanism to save and backtrack each set of corpus data x and the corresponding universal model M output answer q1 (x, q1). Manual intervention is performed on each set (x, q1) to correct the mismatch between the universal model M output answer q1 and the input text prompt containing the corpus data x. After correction, the result is (x, q2);

[0017] S2.2. Use the corrected (x, q2) to feed into the general large model M, and use the Lora algorithm to fine-tune the general large model M to obtain the fine-tuned scene large model M1;

[0018] S2.3. Re-input the text prompt model built based on the corpus data x and the corresponding answer space Q(x, Q) into the scenario model M1. Find the answer with the highest prediction score for the scenario model M1 in the answer space Q. Record the process of searching the answer space Q for the given corpus data x as f(x′, q). For each answer in the answer space Q, calculate the probability of the scenario model M1 outputting that answer, thereby finding the best answer q. The calculation is as follows:

[0019] q=find q∈Q Max(P(f(x′, q); θ));

[0020] S2.4. Set a mapping between the answer space Q and the label space Y so that the answer q accurately corresponds to the required label space Y.

[0021] Furthermore, the S3 further includes:

[0022] S3.1. Distill the scene model M1 into small models m1, m2, m3, and m4.

[0023] S3.2. Use the scene model M1 ontology transformer structure head plus a linear layer to train a multi-model fusion scoring model f(x′, w). The process of searching for the scoring weight w for a given corpus data x is recorded as f(x′, w; θ). The process of obtaining the scoring weights of the small models m1, m2, m3, and m4 is as follows:

[0024] Σm(f(x′, w; θ));

[0025] S3.3. Replace the scene large model M1 with multiple small models. For the service Z(x, Z) corresponding to the same set of corpus data x, use the multi-model fusion scoring mechanism to obtain the final result and complete the collaborative training of large and small models.

[0026] A computer-readable storage medium stores a computer program, which, when executed by a processor, causes the processor to perform the steps of the above-mentioned method for collaborative training of LLM-based large and small models.

[0027] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the above-mentioned method based on LLM large and small model collaborative training.

[0028] The beneficial effects of the present invention are embodied in:

[0029] In the present invention, in the early stage, the excellent language understanding and pre-training capabilities of the general large model are utilized to obtain a huge knowledge system to realize the cold start and launch of the AI service required for the business scenario with 0 samples; in the mid-term, a small amount of annotation is performed on the sample data generated by the business scenario and the large model is fine-tuned, and continuous learning is carried out to enable the large model to have domain knowledge and quickly improve the algorithm effect; in the later stage, the domain knowledge obtained by the large model is distilled into multiple small models, and the results of multiple small models are integrated using a multi-model scoring mechanism to realize the collaborative training of large and small models. In this way, a method of collaborative training of large and small models that can obtain large model knowledge through a small amount of annotation for the cold start and launch of text services for any scenario under limited hardware resources is realized, which greatly improves the model learning ability and work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The drawings described herein are used to provide further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0031] Figure 1 It is a schematic diagram of the overall process of the large and small model collaborative training method according to an embodiment of the present invention.

[0032] Figure 2 Detailed flowchart of the large and small model collaborative training method according to an embodiment of the present invention.

[0033] Figure 3 It is a structural block diagram of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. In the absence of conflict, the embodiments in this application and the features in the embodiments can be combined with each other. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0035] It should be noted that the meaning of "and / or" appearing throughout the text includes three parallel solutions. Taking "A and / or B" as an example, it includes solution A, solution B, or solutions in which both A and B are satisfied. In addition, "multiple" refers to more than two. In addition, the technical solutions between the various embodiments can be combined with each other, but this must be based on the fact that ordinary technicians in this field can implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0036] The key terms involved in this application are explained as follows:

[0037] A large language model (LLM) is a natural language processing model based on deep learning technology. It has billions or even hundreds of billions of parameters, and through pre-training on massive amounts of text data, it can capture the complex structure and deep semantic information of language. The emergence of LLM marks a major breakthrough in the field of artificial intelligence in natural language processing. LLM is usually trained based on a large amount of text data, including books, articles, web pages, conversations, etc., covering a wide range of topics and language styles, which enables the model to capture the complexity and diversity of language. LLM usually uses deep neural networks, especially the Transformer architecture, which can handle long-range dependencies and complex language patterns, improving the model's ability to understand language. LLM first learns on a large amount of unlabeled data through pre-training, and then optimizes on specific tasks through fine-tuning. This two-stage approach enables the model to generalize to a variety of NLP tasks. LLM pre-training typically uses self-supervised learning methods such as Masked Language Modeling (MLM) and Next Sentence Prediction (NSP), which allow models to learn language patterns from unlabeled data. LLMs are able to understand and utilize contextual information, which makes them more accurate and coherent when understanding and generating text. LLMs implicitly learn a variety of language tasks during pre-training, which allows them to easily adapt to new tasks such as text classification, question answering, and text generation. The attention mechanism in the Transformer architecture enables the model to dynamically focus on different parts when processing the input sequence, which is crucial for understanding and generating natural language.

[0038] Lora fine-tuning is a method for fine-tuning pre-trained language models for natural language processing (NLP) tasks. It aims to adapt to specific downstream tasks by introducing a small number of trainable parameters without having to fully fine-tune the entire pre-trained model. The core idea of Lora fine-tuning is to add a trainable sidecar to each layer of the pre-trained model, which minimizes the impact on the original model parameters while enabling rapid adaptation to new tasks.

[0039] Knowledge distillation is a machine learning technique used to improve model efficiency. It transfers knowledge from a larger, more complex model to a smaller, simpler model. The large model is often a single complex network or a collection of several networks, with good performance and generalization capabilities. The small model has a smaller network size and limited expressive power. Therefore, the knowledge learned by the large model can be used to guide the training of the small model, so that the small model has comparable performance to the large model.

[0040] See also Figure 1-Figure 2 , an embodiment of the present invention provides a method for collaborative training of large and small models based on LLM, comprising the following steps:

[0041] S1. Combining the language understanding and pre-training capabilities of general large models, we can acquire a vast knowledge system and launch AI services required by business scenarios with zero-sample cold start.

[0042] S2. A small amount of annotation is performed on sample data generated by business scenarios, and the general large model is fine-tuned to form a large scenario model. Through continuous learning, the large scenario model is equipped with domain knowledge, which quickly improves the algorithm effect.

[0043] S3. Distill the domain knowledge obtained from the large scene model into multiple small models, and use the scoring mechanism to fuse the results of multiple small models to achieve collaborative training of large and small models.

[0044] In this embodiment, the S1 further includes:

[0045] S1.1. Collect corpus data x and determine the scenario tasks, and determine the label space Y for the collected corpus data x;

[0046] S1.2. The corpus data x and the corresponding label space Y form a text prompt. The text prompt is fed into the general large model M. The general large model M outputs the corresponding answer q1 based on the text prompt.

[0047] S1.3. The service consisting of the general large model M and the application scenario is collectively referred to as service Z. At the same time, the answer space Q of service Z is determined.

[0048] S1.4. Deploy service Z online to meet the needs of rapid implementation of AI services in scenarios, and implement a zero-sample cold start for the scenario service.

[0049] At this time, the general large model M is just a general model that has learned general knowledge. It can only provide general understanding and answers to texts, and has not learned the scene corpus data x. Therefore, a backtracking mechanism needs to be established, as shown below.

[0050] In this embodiment, the S2 further includes:

[0051] S2.1. Establish an online backtracking mechanism to save and backtrack each set of corpus data x and the corresponding universal model M output answer q1 (x, q1). Manual intervention is performed on each set (x, q1) to correct the mismatch between the universal model M output answer q1 and the input text prompt containing the corpus data x. After correction, the result is (x, q2);

[0052] S2.2. Use the corrected (x, q2) to feed the general large model M. Use the Lora algorithm to fine-tune the general large model M to obtain the fine-tuned scenario large model M1. The scenario large model M1 is a proprietary model that has learned the corpus data x and is more targeted than the general large model M.

[0053] S2.3. Re-input the text prompt model built based on the corpus data x and the corresponding answer space Q(x, Q) into the scenario model M1. Find the answer with the highest prediction score for the scenario model M1 in the answer space Q. Record the process of searching the answer space Q for the given corpus data x as f(x′, q). For each answer in the answer space Q, calculate the probability of the scenario model M1 outputting that answer, thereby finding the best answer q. The calculation is as follows:

[0054] q=find q∈Q Max(P(f(x′, q); θ));

[0055] S2.4. Set a mapping between the answer space Q and the label space Y so that the answer q accurately corresponds to the required label space Y. Take text classification as an example:

[0056] if q∈{“happy”, “excited”}y=“positive”

[0057] if q∈{“lost”, “sad”}y=“negative”

[0058] if q∈{“bland”}y=“neutral”.

[0059] In this embodiment, the S3 further includes:

[0060] S3.1. Distill the scene model M1 into small models m1, m2, m3, and m4.

[0061] S3.2. Use the scene model M1 ontology transformer structure head plus a linear layer to train a multi-model fusion scoring model f(x′, w). The process of searching for the scoring weight w for a given corpus data x is recorded as f(x′, w; θ). The process of obtaining the scoring weights of the small models m1, m2, m3, and m4 is as follows:

[0062] Σm(f(x′, w; θ));

[0063] S3.3. Replace the scene large model M1 with multiple small models. For the service Z(x, Z) corresponding to the same set of corpus data x, use the multi-model fusion scoring mechanism to obtain the final result and complete the collaborative training of large and small models.

[0064] Thus, this embodiment realizes a method for collaborative training of large and small models under limited hardware resource conditions, which can obtain large model knowledge by cold-starting any scene text service with a small amount of annotation.

[0065] An embodiment of the present invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor performs the steps of the method for collaborative training based on LLM large and small models as described above.

[0066] See also Figure 3 An embodiment of the present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method for collaborative training based on LLM large and small models as described above.

[0067] An embodiment of the present invention also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the steps of the above-mentioned method for collaborative training based on LLM large and small models.

[0068] It is understandable that the system, device and storage medium provided by the embodiments of the present invention correspond to the method provided by the embodiments of the present invention. The explanation, examples and beneficial effects of the relevant content can refer to the corresponding parts of the above-mentioned method based on collaborative training of LLM large and small models.

[0069] It should be noted that those skilled in the art will understand that all or part of the steps implemented in the embodiments of the present invention can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using hardware, it can be implemented in whole or in part in the form of purchased standard parts or modified parts. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiments of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).

[0070] In summary, in order to solve the low efficiency problems of traditional small model training, such as the need to collect and label a large amount of data, complex and diverse scenarios, and the need to collect data for training for each scenario task separately, and the need for customized development for each scenario, the present invention proposes a method for collaborative training of large and small models based on LLM, which distills the domain knowledge obtained by the large model into multiple small models, and uses a multi-model scoring mechanism to fuse the results of multiple small models, realizing the cold start and launch of any scenario text service under limited hardware resources, and the method of collaborative training of large and small models that can obtain large model knowledge through a small amount of annotation.

[0071] It should be understood that the examples and implementation methods described herein are for illustrative purposes only and are not intended to limit the present invention. Those skilled in the art may make various modifications or changes based on them. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method based on collaborative training of LLM large and small models, characterized in that: The following steps are involved: S1. Combining the language understanding and pre-training capabilities of general large models, we can acquire a vast knowledge system and launch AI services required by business scenarios with zero-sample cold start. S2. A small amount of annotation is performed on sample data generated by business scenarios, and the general large model is fine-tuned to form a large scenario model. Through continuous learning, the large scenario model is equipped with domain knowledge, which quickly improves the algorithm effect. S3. Distill the domain knowledge obtained from the large scene model into multiple small models, and use the scoring mechanism to fuse the results of multiple small models to achieve collaborative training of large and small models.

2. The method for collaborative training based on LLM large and small models according to claim 1, characterized in that: Said S1 further comprises: S1.

1. Collect corpus data x and determine the scenario tasks, and determine the label space Y for the collected corpus data x; S1.

2. The corpus data x and the corresponding label space Y form a text prompt. The text prompt is fed into the general large model M. The general large model M outputs the corresponding answer q1 based on the text prompt. S1.

3. The service consisting of the general large model M and the application scenario is collectively referred to as service Z. At the same time, the answer space Q of service Z is determined. S1.

4. Deploy service Z online to meet the needs of rapid implementation of AI services in scenarios, and implement a zero-sample cold start for the scenario service.

3. The method for collaborative training based on LLM large and small models according to claim 1, characterized in that: Said S2 further comprises: S2.

1. Establish an online backtracking mechanism to save and backtrack each set of corpus data x and the corresponding universal model M output answer q1 (x, q1). Manual intervention is performed on each set (x, q1) to correct the mismatch between the universal model M output answer q1 and the input text prompt containing the corpus data x. After correction, the result is (x, q2); S2.

2. Use the corrected (x, q2) to feed into the general large model M, and use the Lora algorithm to fine-tune the general large model M to obtain the fine-tuned scene large model M1; S2.

3. Re-input the text prompt model built based on the corpus data x and the corresponding answer space Q(x, Q) into the scenario model M1. Find the answer with the highest prediction score for the scenario model M1 in the answer space Q. Record the process of searching the answer space Q for the given corpus data x as f(x′, q). For each answer in the answer space Q, calculate the probability of the scenario model M1 outputting that answer, thereby finding the best answer q. The calculation is as follows: q=find q∈Q Max(P(f(x′, q); θ)); S2.

4. Set a mapping between the answer space Q and the label space Y so that the answer q accurately corresponds to the required label space Y.

4. The method for collaborative training based on LLM large and small models according to claim 1, characterized in that: Said S3 further comprises: S3.

1. Distill the scene model M1 into small models m1, m2, m3, and m4. S3.

2. Use the scene model M1 ontology transformer structure head plus a linear layer to train a multi-model fusion scoring model f(x′, w). The process of searching for the scoring weight w for a given corpus data x is recorded as f(x′, w; θ). The process of obtaining the scoring weights of the small models m1, m2, m3, and m4 is as follows: Σm(f(x′, w; θ)); S3.

3. Replace the scene large model M1 with multiple small models. For the service Z(x, Z) corresponding to the same set of corpus data x, use the multi-model fusion scoring mechanism to obtain the final result and complete the collaborative training of large and small models.

5. A computer-readable storage medium, characterized in that A computer program is stored, and when the computer program is executed by a processor, the processor is caused to perform the steps of the method for collaborative training based on LLM large and small models as described in any one of claims 1 to 4.

6. Computer equipment, characterized in that It includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method based on LLM large and small model collaborative training as described in any one of claims 1 to 4.

Images