A large model training method and device
By constructing sample data from different types of prompt words, supervised fine-tuning training of the large model is performed, which solves the problem of insufficient model professional understanding caused by insufficient data volume and achieves effective improvement in the supervised fine-tuning training stage.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING ACAD OF ARTIFICIAL INTELLLIGENCE
- Filing Date
- 2026-01-29
- Publication Date
- 2026-06-05
AI Technical Summary
Existing large-scale model training methods cannot continue pre-training for domains with insufficient data, resulting in insufficient understanding of the specific domain by the model.
By constructing first cue words indicating the text paragraphs to be read and second cue words indicating sample questions, first sample data and second sample data are generated. These data are then used to perform supervised fine-tuning training on the large model, enhancing its domain knowledge learning.
The supervised fine-tuning training phase effectively improves the learning ability of large models on domain data, achieving similar results to continued pre-training and solving the problem of insufficient data.
Smart Images
Figure CN122154773A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of model training technology, and in particular to a method and apparatus for training large models. Background Technology
[0002] In the existing large model training process, in order to enable the model to have professional understanding and domain knowledge in specific domains, it is usually necessary to first perform continuous pre-training (CPT) on the basic large model and then perform supervised fine-tuning training (SFT). The essence of continuous pre-training is to use domain-specific corpora (such as finance, medicine, law) or task-related corpora to supervise and fine-tune the large model on the basis of general pre-training, so that the large model learns the vocabulary, grammar and semantic features of the professional domain, while retaining the original general capabilities.
[0003] However, continued pre-training has a clear minimum data requirement; the larger the model and the more specialized the domain, the more data is needed. For some domains with scarce data, continued pre-training is not feasible. Therefore, a new training method for large models is urgently needed for domains where continued pre-training is not possible. Summary of the Invention
[0004] This invention provides a training method and apparatus for a large-scale model. By constructing a first cue word indicating a text paragraph and a second cue word indicating a sample question, first sample data and second sample data are constructed based on the first and second cue words, respectively. This yields sample data containing both text paragraphs and question-answer pairs, which are then used as training data for supervised fine-tuning training of the large-scale model. This effectively improves the large-scale model's learning of domain data during the supervised fine-tuning training phase. In other words, for domains where further pre-training is not feasible, the large-scale model training method and apparatus provided in this invention can achieve the same training effect directly through supervised fine-tuning training.
[0005] This invention provides a training method for a large model, comprising the following steps: constructing a first prompt word and a second prompt word according to different types of prompt word architectures; wherein the first prompt word indicates a reading text paragraph, and the second prompt word indicates a sample question; generating first sample data corresponding to the first prompt word and second sample data corresponding to the second prompt word; and performing supervised fine-tuning training of the large model using the first sample data and the second sample data.
[0006] Optionally, generating the first sample data corresponding to the first prompt word and the second sample data corresponding to the second prompt word respectively includes: acquiring domain corpus data; constructing at least one text paragraph based on the domain corpus data; and generating the first sample data based on the first prompt word and the text paragraph.
[0007] Optionally, constructing at least one text paragraph based on the domain corpus data includes: sliding the domain corpus data according to a preset window to obtain multiple text segments; wherein the preset window includes at least one of a fixed window, a random window, and an overlapping window; and generating the text paragraph based on one or more of the text segments.
[0008] Optionally, generating the text paragraph based on one or more of the text fragments includes: when there are multiple text fragments, concatenating at least two consecutive text fragments to obtain a text paragraph.
[0009] Optionally, generating the first sample data corresponding to the first prompt word and the second sample data corresponding to the second prompt word further includes: acquiring domain corpus data; extracting sample answers corresponding to the sample question from the domain corpus data; and generating the second sample data based on the second prompt word and the sample answer.
[0010] Optionally, the step of supervising and fine-tuning the large model using the first sample data and the second sample data includes: inputting the first sample data and the second sample data into the large model respectively; calculating a first loss corresponding to the first sample data and a second loss corresponding to the second sample data based on the output of the large model; wherein the first loss and the second loss are calculated in the same way; and updating the parameters in the large model based on the first loss and the second loss.
[0011] Optionally, after acquiring the domain corpus data, the method further includes: preprocessing the domain corpus data; the preprocessing includes at least one of the following: domain relevance processing, accuracy processing, standardization processing, grammatical correction, and redundant information removal.
[0012] The present invention also provides a training device for a large model, comprising the following modules: The construction module is used to construct a first prompt word and a second prompt word according to different types of prompt word architectures; wherein the first prompt word indicates the reading text paragraph, and the second prompt word indicates the sample question; The generation module is used to generate first sample data corresponding to the first prompt word and second sample data corresponding to the second prompt word, respectively; The training module is used to perform supervised fine-tuning training of the large model using the first sample data and the second sample data.
[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the training method for a large model as described in any of the above.
[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the training method for a large model as described above.
[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the training method for a large model as described above.
[0016] This invention provides a training method and apparatus for a large-scale model. By constructing a first cue word indicating a text paragraph and a second cue word indicating a sample question, first sample data and second sample data are constructed based on the first and second cue words, respectively. This yields sample data containing both text paragraphs and question-answer pairs, which are then used as training data for supervised fine-tuning training of the large-scale model. This effectively improves the large-scale model's learning of domain data during the supervised fine-tuning training phase. In other words, for domains where further pre-training is not feasible, the large-scale model training method and apparatus provided in this invention can achieve the same training effect directly through supervised fine-tuning training. Attached Figure Description
[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0018] Figure 1 This is one of the flowcharts illustrating the training method for large models provided by this invention.
[0019] Figure 2 This is a schematic diagram of the process for generating the first sample data provided by the present invention.
[0020] Figure 3 This is a schematic diagram of another process for generating first sample data provided by the present invention.
[0021] Figure 4 This is a schematic diagram of the process for generating the second sample data provided by the present invention.
[0022] Figure 5 This is the second flowchart illustrating the training method for large models provided by this invention.
[0023] Figure 6 This is a schematic diagram of the structure of the training device for the large model provided by the present invention.
[0024] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation
[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0026] Before describing the large model training method provided in the embodiments of the present invention, a brief explanation of the supervised fine-tuning training stage in the prior art will be given to distinguish the training process in the embodiments of the present invention from the training process in the prior art. Specifically, in the existing supervised fine-tuning training stage, it is usually necessary to construct question-answer pairs as training sample data, that is, to define the sample questions and the sample answers corresponding to the sample questions. Then, the question-answer pairs are input into the large model to allow the model to learn, so that when the user's actual question is received later, a predicted answer corresponding to the actual question can be generated.
[0027] However, in typical cases, question-answer pairs are extracted from domain data, meaning that they cannot inherently cover all domain data. For example, from a complete domain document, 100 question-answer pairs can be extracted, each covering one of the 100 points included in the document. However, some domain documents still cannot yield question-answer pairs. This results in large models, even after learning question-answer pairs, still lacking some domain-specific understanding. Therefore, in existing technologies, to compensate for this deficiency in the supervised fine-tuning stage, the large model is pre-trained before the supervised fine-tuning stage, allowing it to absorb massive amounts of domain data in unsupervised learning.
[0028] However, as described in the background section of this invention, for some areas where the amount of data is insufficient, it is impossible to continue pre-training. Therefore, this invention has made corresponding adjustments to the supervised fine-tuning training stage. By constructing different types of first sample data and second sample data, additional first sample data is added on the basis of the existing question-answer pairs (second sample data) to achieve a training objective similar to continuing pre-training.
[0029] Figure 1 This is one of the flowcharts illustrating the training method for large models provided by this invention, such as... Figure 1 As shown, the method includes the following: Step 101: Construct a first cue word and a second cue word according to different types of cue word structures; wherein, the first cue word indicates the reading text paragraph, and the second cue word indicates the sample question; Specifically, different types of prompt word architectures determine the different generation modes of the large model. For example, the first prompt word could be "Read the following content." When the model recognizes the first prompt word, it activates the original text generation mode. In this mode, the goal of the large model is to accurately replicate the input text paragraph, learning the expression style, terminology, sentence structure, and language expression habits of the domain text, thus effectively injecting domain knowledge. The second prompt word could be "Question" or "Please answer." When the model recognizes the second prompt word, it activates the regular question-and-answer generation mode. In this mode, the model, based on its learned domain knowledge, accurately understands the core needs of the user's question, quickly retrieves and generates a suitable answer.
[0030] Step 102: Generate the first sample data corresponding to the first prompt word and the second sample data corresponding to the second prompt word respectively; It's understandable that sample data consists of prompts and corresponding target outputs; the two are bound together as carrier and content, framework and information, and neither can be omitted. Simply put, prompts help the model understand the task intent, while the target output is the specific content the model needs to learn.
[0031] In an alternative embodiment, the process of generating the first sample data can be as follows: Figure 2 As shown, it includes: Step 201: Obtain domain-specific corpus data; For example, domain corpus data can be patents, academic papers, and articles in forums in that domain. This invention does not specifically limit this, but it is usually text data and cannot consist only of images or non-text data.
[0032] Step 202: Construct at least one text paragraph based on the domain corpus data; Step 203: Generate the first sample data based on the first prompt word and the text paragraph.
[0033] As can be seen, the first sample data is generated from the first prompt word and the text paragraph. In this way, after the large model is trained, it can learn more comprehensive domain knowledge based on the first sample data composed of text paragraphs.
[0034] In a further optional embodiment, the process of generating the first sample data described above can be as follows: Figure 3 As shown, it includes: Step 301: Obtain domain corpus data; Step 302: Perform sliding segmentation on the domain corpus data according to the preset window to obtain multiple text fragments; wherein, the preset window includes at least one of a fixed window, a random window, and an overlapping window; A fixed window refers to a window with a fixed length (which can be understood as containing a fixed number of words), such as 512 words or 1024 words. This can be set and adjusted according to the characteristics of the domain corpus data. During the segmentation process, the domain corpus data is segmented sequentially according to the set fixed length, resulting in multiple consecutive text segments containing a fixed number of words. Specifically, these multiple text segments can be combined to obtain a complete domain corpus data.
[0035] A random window is a given range of window lengths, such as 300 to 1500 words. During segmentation, a random value is selected from the range of window lengths to obtain multiple text segments containing different numbers of words that are consecutive to each other.
[0036] Overlapping windows refer to the practice of not segmenting the data sequentially, but rather using a portion of the previous window's length as a step size for sliding sampling. This means that the text segments sampled by two adjacent windows overlap (repeated), thus ensuring the coherence and coverage integrity of the domain corpus data.
[0037] Understandably, in order to help large models learn better from domain corpus data, this embodiment of the invention performs sliding sampling through the above-mentioned multiple preset windows to construct as many text fragments as possible, thereby increasing the number of samples during the training process of large models.
[0038] In one alternative embodiment, the process of generating the second sample data can be as follows: Figure 4 As shown, it includes: Step 401: Obtain domain corpus data; Step 402: Extract sample answers corresponding to the sample questions from the domain corpus data; Step 403: Generate second sample data based on the second prompt word and the sample answer.
[0039] As can be seen from steps 201 to 203 and 401 to 403 above, the first sample data is generated from the first prompt word and at least one text paragraph, while the second sample data is generated from the second prompt word indicating the sample question and the corresponding sample answer. Their compositions are fundamentally different. In other words, by training the first and second sample data together, this embodiment of the invention enables the large model to accurately understand the user's core needs based on the second sample data, and further learn more comprehensive domain knowledge based on the first sample data composed of text paragraphs.
[0040] Step 303: Generate a text paragraph based on one or more text fragments.
[0041] Each text fragment can be used as a separate text paragraph, or multiple text fragments can be combined to form a separate text paragraph.
[0042] In one alternative embodiment, when there are multiple text segments, at least two consecutive text segments are concatenated to obtain a text paragraph.
[0043] Step 304: Generate the first sample data based on the first prompt word and the text paragraph.
[0044] For example, the first sample data generated is shown below: Plain Text <user>Read the following content.
[0045] <assistant>{Original text paragraph}.
[0046] The generated second sample data is shown below: Plain Text <user>Question: xxxx.
[0047] <assistant>{Answer text}.
[0048] It can be seen that, through the above Figure 2 as well as Figure 3 The process involves constructing different types of first and second sample data based on the first and second prompt words. The first sample data ensures that the large model learns sufficient domain knowledge, especially the parts that cannot be covered by the second sample data, thereby improving the overall training accuracy of the large model.
[0049] In a further optional embodiment, after acquiring the domain corpus data, and before constructing at least one text paragraph based on the domain corpus data and extracting sample answers corresponding to sample questions from the domain corpus data, the method further includes: preprocessing the domain corpus data; the preprocessing includes at least one of the following: domain relevance processing, accuracy processing, standardization processing, grammatical correction, and redundant information removal. Specifically, domain relevance processing ensures that the relevance between different domain corpus data is not lower than a preset threshold, such as 90%; accuracy processing ensures that the domain corpus data does not contain factual errors, i.e., the data itself is correct; standardization processing unifies the expression habits of the domain corpus data, such as unifying proper nouns; grammatical correction corrects grammatical errors in the domain corpus data; and redundant information removal removes meaningless spaces and repetitive sentences from the domain corpus data. Through these multiple preprocessing steps, the quality of the domain corpus data can be guaranteed, ensuring the training effect of the subsequent large model.
[0050] Step 103: Use the first sample data and the second sample data to perform supervised fine-tuning training on the large model.
[0051] The first and second sample data can be mixed in a set ratio to form the sample training dataset, and the mixing is done by random shuffling to ensure the randomness of the sample training dataset and avoid the large model learning the arrangement pattern between the first and second sample data, thus avoiding learning bias.
[0052] Specifically, in one optional embodiment, such as Figure 5 As shown, the first sample data and the second sample data are input into the large model, respectively. Based on the output of the large model, the first loss corresponding to the first sample data and the second loss corresponding to the second sample data are calculated. The first and second losses are calculated in the same way, both being cross-entropy losses. The parameters in the large model are updated based on the first and second losses. The process of updating the parameters of the large model based on the cross-entropy loss can adopt conventional update methods in existing technologies, and this invention does not impose specific limitations on this.
[0053] In summary, the large-scale model training method provided in this embodiment of the invention constructs a first cue word indicating a text paragraph and a second cue word indicating a sample question. Based on the first and second cue words, first and second sample data are constructed respectively, resulting in sample data containing both text paragraphs and question-answer pairs. This sample data is then used as training data for supervised fine-tuning training of the large-scale model, effectively improving the model's learning of domain data during the supervised fine-tuning training phase. In other words, for domains where further pre-training is not feasible, the large-scale model training method and apparatus provided in this embodiment of the invention can achieve the same training effect directly through supervised fine-tuning training.
[0054] The training apparatus for large models provided by the present invention will be described below. The training apparatus for large models described below can be referred to in correspondence with the training method for large models described above.
[0055] like Figure 6 As shown, the large model training device 600 provided in this embodiment of the invention specifically includes: The construction module 601 is used to construct a first prompt word and a second prompt word according to different types of prompt word architectures; wherein the first prompt word indicates a reading text paragraph, and the second prompt word indicates a sample question; The generation module 602 is used to generate first sample data corresponding to the first prompt word and second sample data corresponding to the second prompt word, respectively; Training module 603 is used to perform supervised fine-tuning training of a large model using the first sample data and the second sample data.
[0056] In an optional embodiment, the generation module 602 is further configured to: acquire domain corpus data; construct at least one text paragraph based on the domain corpus data; extract sample answers corresponding to the sample questions from the domain corpus data; generate first sample data based on the first prompt word and the text paragraph; and generate second sample data based on the second prompt word and the sample answers.
[0057] In an optional embodiment, the generation module 602 is further configured to perform sliding segmentation of the domain corpus data according to a preset window to obtain multiple text segments; wherein the preset window includes at least one of a fixed window, a random window, and an overlapping window; and generate the text paragraph based on one or more of the text segments.
[0058] In an optional embodiment, the generation module 602 is further configured to, when there are multiple text segments, concatenate at least two consecutive text segments to obtain a text paragraph.
[0059] In an optional embodiment, the training module 603 is further configured to input the first sample data and the second sample data into the large model respectively, calculate the first loss corresponding to the first sample data and the second loss corresponding to the second sample data based on the output of the large model; wherein the first loss and the second loss are calculated in the same way; and update the parameters in the large model based on the first loss and the second loss.
[0060] In an optional embodiment, the generation module 602 is further configured to preprocess the domain corpus data after acquiring the domain corpus data and before constructing at least one text paragraph based on the domain corpus data and extracting sample answers corresponding to the sample questions from the domain corpus data; the preprocessing includes at least one of the following: domain relevance processing, accuracy processing, standardization processing, grammar correction, and redundant information removal.
[0061] In summary, the large-scale model training apparatus provided in this embodiment of the invention constructs first prompt words indicating the text paragraphs to be read and second prompt words indicating the sample questions. Based on these first and second prompt words, it constructs first and second sample data, respectively, thus obtaining sample data that includes both text paragraphs and question-answer pairs. This sample data is then used as training data for supervised fine-tuning training of the large-scale model, effectively improving the large-scale model's learning of domain data during the supervised fine-tuning training phase. In other words, for domains where further pre-training is not feasible, the large-scale model training method and apparatus provided in this embodiment of the invention can achieve the same training effect directly through supervised fine-tuning training.
[0062] Figure 7 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7 As shown, the electronic device may include a processor 710, a communications interface 720, a memory 730, and a communication bus 740, wherein the processor 710, communications interface 720, and memory 730 communicate with each other via the communication bus 740. The processor 710 can call logical instructions in the memory 730 to execute a training method for a large model. This method includes: constructing a first prompt word and a second prompt word according to different types of prompt word architectures; wherein the first prompt word indicates a reading text paragraph, and the second prompt word indicates a sample question; generating first sample data corresponding to the first prompt word and second sample data corresponding to the second prompt word; and performing supervised fine-tuning training of the large model using the first sample data and the second sample data.
[0063] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0064] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the training method for the large model provided by the above methods. The method includes: constructing a first prompt word and a second prompt word according to different types of prompt word architectures; wherein the first prompt word indicates a reading text paragraph and the second prompt word indicates a sample question; generating first sample data corresponding to the first prompt word and second sample data corresponding to the second prompt word; and performing supervised fine-tuning training of the large model using the first sample data and the second sample data.
[0065] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a training method for a large model provided by the methods described above. This method includes: constructing a first cue word and a second cue word according to different types of cue word architectures; wherein the first cue word indicates a reading text paragraph, and the second cue word indicates a sample question; generating first sample data corresponding to the first cue word and second sample data corresponding to the second cue word; and performing supervised fine-tuning training of the large model using the first sample data and the second sample data.
[0066] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0067] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0068] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.< / assistant> < / user> < / assistant> < / user>
Claims
1. A training method for a large model, characterized in that, Applied to the supervised fine-tuning training phase, including: Based on different types of cue word architectures, a first cue word and a second cue word are constructed respectively; wherein, the first cue word indicates the reading text paragraph, and the second cue word indicates the sample question; Generate first sample data corresponding to the first prompt word and second sample data corresponding to the second prompt word respectively; The large model is trained under supervised supervision using the first sample data and the second sample data.
2. The training method for a large model according to claim 1, characterized in that, The step of generating the first sample data corresponding to the first prompt word and the second sample data corresponding to the second prompt word includes: Acquire domain-specific corpus data; Construct at least one text paragraph based on the domain corpus data; First sample data is generated based on the first prompt word and the text paragraph.
3. The training method for a large model according to claim 2, characterized in that, The construction of at least one text paragraph based on the domain corpus data includes: The domain corpus data is slidably segmented according to a preset window to obtain multiple text fragments; wherein the preset window includes at least one of a fixed window, a random window, and an overlapping window; The text paragraph is generated based on one or more of the text fragments.
4. The training method for a large model according to claim 3, characterized in that, The step of generating the text paragraph based on one or more of the text fragments includes: When there are multiple text segments, at least two consecutive text segments are concatenated to obtain a text paragraph.
5. The training method for a large model according to claim 1, characterized in that, The step of generating the first sample data corresponding to the first prompt word and the second sample data corresponding to the second prompt word respectively further includes: Acquire domain-specific corpus data; Extract sample answers corresponding to the sample questions from the domain corpus data; A second sample data is generated based on the second prompt word and the sample answer.
6. The training method for a large model according to claim 1, characterized in that, The supervised fine-tuning training of the large model using the first sample data and the second sample data includes: The first sample data and the second sample data are respectively input into the large model, and the first loss corresponding to the first sample data and the second loss corresponding to the second sample data are calculated based on the output of the large model; wherein the first loss and the second loss are calculated in the same way; The parameters in the large model are updated based on the first loss and the second loss.
7. The training method for a large model according to claim 2 or 3, characterized in that, After acquiring the domain corpus data, the following is also included: The domain corpus data is preprocessed; The preprocessing includes at least one of the following: domain relevance processing, accuracy processing, standardization processing, syntax correction, and redundant information removal.
8. A training device for a large model, characterized in that, Applied to the supervised fine-tuning training phase, including: The construction module is used to construct a first prompt word and a second prompt word according to different types of prompt word architectures; wherein the first prompt word indicates the reading text paragraph, and the second prompt word indicates the sample question; The generation module is used to generate first sample data corresponding to the first prompt word and second sample data corresponding to the second prompt word, respectively; The training module is used to perform supervised fine-tuning training of the large model using the first sample data and the second sample data.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the training method for the large model as described in any one of claims 1 to 7.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the training method for the large model as described in any one of claims 1 to 7.