Language model training method and device, storage medium and computer program product

Abstract

The invention discloses a language model training method and device, a storage medium and a computer program product, and relates to the technical field of natural language process.The method comprises the steps that after a second device trains a small language model, text data are input into the small language model to obtain a second prediction result; the first device inputs the text data into the large language model to obtain a first prediction result, calculates first knowledge distillation loss and optimizes the large language model based on the aligned first prediction result and second prediction result, and inputs the text data into the large language model to obtain a third prediction result; the second equipment calculates second knowledge distillation loss and optimizes the small language model based on the aligned second prediction result and third prediction result; and obtaining a trained large language model until a preset training ending condition is met. According to the method, under the condition that enterprise computing power resources are limited, the big language model absorbs privacy knowledge of the field where each enterprise is located, and meanwhile, the data privacy of each enterprise is also protected.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and particularly relates to a language model training method and device, a storage medium and a computer program product. BACKGROUND

[0002] With the rapid development of large language model (LLM, Large Language Model) technology, their powerful performance has brought significant productivity improvement in various industries. In the training and use of large language models, a large amount of computing resources are required, which leads to only a few large enterprises being able to train and fully utilize these models. Due to the limitation of computing resources, small and medium-sized enterprises cannot directly run large language models with particularly large model parameter sizes, and can only run large language models with relatively limited model parameter sizes (for distinction, also referred to as small language models). At the same time, as the model parameter size of large language models becomes larger and larger, more and more training text data is required. At present, the open source training text data set has gradually failed to meet the training and fine-tuning of large language models, especially some text data in specific fields in various enterprises, but these text data involve privacy and cannot be exposed. Therefore, how to enable large language models to absorb the private knowledge of the fields of various enterprises in the case of limited computing resources of enterprises to improve the natural language processing capability of large language models, while also protecting the data privacy of various enterprises is a problem to be solved at present.

[0003] The above content is only used to assist in understanding the technical solutions of the present application and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0004] The main purpose of the present application is to provide a language model training method, device, storage medium and computer program product, to realize the training of large language models in the case of limited computing resources of enterprises, to enable large language models to absorb the private knowledge of the fields of various enterprises, to improve the natural language processing capability of large language models, and also to protect the data privacy of various enterprises.

[0005] To achieve the above purpose, the present application provides a language model training method applied to a first device, wherein the first device is in communication connection with each second device, a large language model to be trained is deployed in the first device, and a small language model to be trained is deployed in each second device, the large language model is heterogeneous with the small language model in at least one second device, and the method comprises the following steps:

[0006] Receive the second training results sent by each of the second devices respectively, wherein the second training results include the second prediction results, which are obtained by the second devices training the small language model with local private text data and then inputting the input text data from the public text data into the small language model for prediction;

[0007] The input text data from the publicly available text data is input into the large language model for prediction, and a first prediction result is obtained;

[0008] The first prediction result and the second prediction result are aligned with text components. Based on the aligned first prediction result and the second prediction result, the first knowledge distillation loss from the small language model to the large language model is calculated.

[0009] The large language model is optimized based on the first knowledge distillation loss, and the first training result is sent to each of the second devices respectively. The first training result includes a third prediction result, which is obtained by inputting the input text data in the public text data into the optimized large language model for prediction. The second devices align the second prediction result with the third prediction result to perform text unit alignment. Based on the aligned second prediction result and the third prediction result, a second knowledge distillation loss is calculated from the large language model to the small language model. The small language model is optimized based on the second knowledge distillation loss.

[0010] Return to the step of receiving the second training results sent by each of the second devices until the preset training end condition is met, and obtain the large language model that has been trained.

[0011] Optionally, there are multiple pieces of publicly available text data, each piece of publicly available text data including input text data as model input data and label text data as model training labels;

[0012] For each piece of public text data, the second training result also includes a second prediction loss corresponding to the public text data. The second prediction loss is a prediction loss calculated by the second device based on the tag text data in the public text data and the corresponding second prediction result.

[0013] After the step of inputting the input text data from the publicly available text data into the large language model for prediction and obtaining the first prediction result, the method further includes:

[0014] The first prediction loss is calculated based on the tag text data in the publicly available text data and the corresponding first prediction result.

[0015] Based on the publicly available text data in each of the aforementioned publicly available text data where the second prediction loss is less than the first prediction loss, the step of aligning the first prediction result and the second prediction result into text components is performed. Based on the aligned first prediction result and second prediction result, the first knowledge distillation loss from the small language model to the large language model is calculated.

[0016] Optionally, for each piece of publicly available text data, the first training result further includes a third prediction loss corresponding to the publicly available text data;

[0017] After optimizing the large language model based on the first knowledge distillation loss, the method further includes:

[0018] The third prediction loss is calculated based on the tag text data in the public text data and the corresponding third prediction result. Each of the second devices performs the step of aligning the second prediction result with the third prediction result based on the public text data in each piece of public text data for which the third prediction loss is less than the second prediction loss. Based on the aligned second prediction result and the third prediction result, the second knowledge distillation loss from the large language model to the small language model is calculated.

[0019] Optionally, the first prediction result includes the prediction probability corresponding to each text unit in the first vocabulary of the large language model, and the second prediction result includes the prediction probability corresponding to each text unit in the second vocabulary of the small language model.

[0020] The step of aligning the first prediction result with the second prediction result into text components includes:

[0021] Calculate the minimum edit distance between each text unit in the first vocabulary and each text unit in the second vocabulary;

[0022] For any target text unit in the first vocabulary, align the text unit with the smallest minimum edit distance from the target text unit in the second vocabulary with the target text unit;

[0023] Align the prediction probability corresponding to the target text component in the first prediction result with the prediction probability of the text component aligned with the target text component in the second prediction result to obtain the aligned first prediction result and second prediction result.

[0024] Optionally, the large language model includes a model ontology and an adapter module;

[0025] The steps for optimizing the large language model based on the first knowledge distillation loss include:

[0026] The model parameters of the adapter module are updated based on the first knowledge distillation loss to optimize the large language model.

[0027] Optionally, after the step of inputting the input text data from the publicly available text data into the large language model for prediction to obtain a first prediction result, the method further includes:

[0028] The first prediction loss is calculated based on the tag text data in the publicly available text data and the corresponding first prediction result.

[0029] The steps for optimizing the large language model based on the first knowledge distillation loss include:

[0030] The large language model is optimized based on the first prediction loss and the first knowledge distillation loss.

[0031] Furthermore, to achieve the above objectives, this application also proposes a language model training method, applied to any one of a plurality of second devices, each second device being communicatively connected to a first device, wherein a large language model to be trained is deployed in the first device, and a small language model to be trained is deployed in each of the second devices, wherein the large language model and the small language model in at least one of the second devices are heterogeneous, and the method comprises:

[0032] The small language model was trained using local private text data.

[0033] The input text data from the publicly available text data is fed into the locally trained small language model to obtain a second prediction result;

[0034] The first device sends a second training result, including the second prediction result, to the first device. After receiving the second training results sent by each of the second devices, the first device inputs the input text data in the public text data into the large language model for prediction to obtain a first prediction result. The first prediction result and the second prediction result are aligned to form text units. Based on the aligned first prediction result and the second prediction result, a first knowledge distillation loss is calculated from the small language model to the large language model. The large language model is optimized based on the first knowledge distillation loss, and the first training result is sent to each of the second devices.

[0035] The system receives the first training result sent by the first device, aligns the second prediction result with the third prediction result in the first training result using text components, and calculates the second knowledge distillation loss from the large language model to the small language model based on the aligned second prediction result and the third prediction result. The third prediction result is obtained by the first device inputting the input text data from the public text data into the optimized large language model for prediction.

[0036] The small language model is optimized based on the second knowledge distillation loss, and the step of training the small language model with local private text data is returned until the preset training termination condition is met, and the trained small language model is obtained.

[0037] Optionally, there are multiple pieces of publicly available text data, each piece of publicly available text data including input text data as model input data and label text data as model training labels;

[0038] For each piece of public text data, the first training result also includes a third prediction loss corresponding to the public text data, which is calculated by the first device based on the tag text data in the public text data and the corresponding third prediction result;

[0039] After the step of inputting the input text data from the publicly available text data into the locally trained small language model to obtain the second prediction result, the method further includes:

[0040] The second prediction loss is calculated based on the tag text data in the publicly available text data and the corresponding second prediction result;

[0041] Based on the publicly available text data in each of the aforementioned publicly available text data where the third prediction loss is less than the second prediction loss, the step of aligning the second prediction result with the third prediction result into text components is performed. Based on the aligned second prediction result and the third prediction result, the second knowledge distillation loss from the large language model to the small language model is calculated.

[0042] Optionally, the step of sending the second training result, including the second prediction result, to the first device includes:

[0043] The second training result, including the second prediction result and the second prediction loss, is sent to the first device so that after the first device calculates the first prediction loss based on the labeled text data in the public text data and the corresponding first prediction result, it performs the step of aligning the first prediction result and the second prediction result into text components based on the public text data in each piece of public text data for which the second prediction loss is less than the first prediction loss, and calculates the first knowledge distillation loss from the small language model to the large language model based on the aligned first prediction result and the second prediction result.

[0044] Optionally, the second prediction result includes the prediction probability corresponding to each text unit in the second vocabulary of the small language model, and the third prediction result includes the prediction probability corresponding to each text unit in the first vocabulary of the large language model.

[0045] The step of aligning the second prediction result with the third prediction result in the first training result for text components includes:

[0046] Calculate the minimum edit distance between each text unit in the first vocabulary and each text unit in the second vocabulary;

[0047] For any target text unit in the second vocabulary, align the text unit with the smallest minimum edit distance from the target text unit among all text units in the first vocabulary with the target text unit;

[0048] Align the prediction probability corresponding to the target text component in the second prediction result with the prediction probability of the text component aligned with the target text component in the third prediction result to obtain the aligned second prediction result and the third prediction result.

[0049] Optionally, the small language model includes a model ontology and an adapter module;

[0050] The steps for optimizing the small language model based on the second knowledge distillation loss include:

[0051] The model parameters of the adapter module are updated based on the second knowledge distillation loss to optimize the small language model.

[0052] Optionally, after the step of inputting the input text data from the publicly available text data into the locally trained small language model to obtain the second prediction result, the method further includes:

[0053] The second prediction loss is calculated based on the tag text data in the publicly available text data and the corresponding second prediction result;

[0054] The steps for optimizing the small language model based on the second knowledge distillation loss include:

[0055] The small language model is optimized based on the second prediction loss and the second knowledge distillation loss.

[0056] In addition, to achieve the above objectives, this application also proposes a language model training device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the language model training method described above.

[0057] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the language model training method described above.

[0058] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the language model training method described above.

[0059] One or more technical solutions proposed in this application have at least the following technical effects:

[0060] By deploying a large language model to be trained on a first device and a small language model to be trained on a second device, the first device uses publicly available text data, while the second devices use both publicly available and their respective private text data to collaboratively train the large and small language models. This collaborative training architecture allows enterprises with limited computing resources to participate in training the large language model, leveraging their private, domain-specific text data to enhance its natural language processing capabilities. In each training round, after each second device trains its small language model locally using its private text data, the input text data from the publicly available text data is fed into the small language model for prediction, yielding a second prediction result. This second prediction result is then sent to the first device. This allows the private text data from the second devices to help optimize the large language model on the first device without exposing the privacy of the private text data. Furthermore, since the second devices deploy small language models, the computing power requirements are lower, enabling most enterprises with limited computing resources to participate. The process involves training a language model to obtain a language model with stronger natural language processing capabilities. Then, a first device receives second prediction results sent by various second devices, and inputs the input text data from the public text data into the large language model for prediction, obtaining a first prediction result. After aligning the first and second prediction results with text components, a first knowledge distillation loss is calculated based on the aligned first and second prediction results, moving from the small language model to the large language model. This first knowledge distillation loss is used to optimize the large language model, enabling further optimization of the large language model using the small language model optimized locally by the second device. The text information (which can be understood as a kind of "knowledge" learned by the model) in the sub-domains corresponding to each enterprise, learned by the small language model, is transferred to the large language model, allowing the large language model to absorb the privacy knowledge of each enterprise's domain. Since this process is performed on the first device, it avoids the disadvantage of limited computing resources for most enterprises and fully utilizes the computing resources of each second device to distribute the computational pressure of optimizing the large language model on a single device.Based on the large language model optimized by knowledge distillation, the first device inputs the public text data into the large language model for prediction, obtaining a third prediction result. This third prediction result is then sent to the second device, where each second device aligns the text components of the second and third prediction results. Based on the aligned second and third prediction results, a second knowledge distillation loss is calculated. This loss is used to optimize the small language model, enhancing its natural language processing capabilities through the large language model. Furthermore, in the next training round, the enhanced small language model learns more privacy knowledge from each enterprise's domain and feeds it back into the large language model. This iterative cycle continues until, after multiple rounds of training, a larger and smaller language model with stronger natural language processing capabilities is obtained. Moreover, before knowledge distillation, the first and second devices align the text components of the prediction results from different language models, enabling the heterogeneous large and small language models to train collaboratively. This supports the current situation where most language models deployed or supported by various enterprises are heterogeneous. Therefore, in this embodiment, the computing resources of each device and the corresponding enterprise's private, segmented text data are fully utilized to train the model language. This enables the large language model to absorb the private knowledge of each enterprise's domain, even with limited enterprise computing resources, thereby improving the natural language processing capabilities of the large language model while protecting the data privacy of each enterprise. Attached Figure Description

[0061] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0062] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 This is a flowchart illustrating the first embodiment of the language model training method of this application.

[0064] Figure 2 This is a flowchart illustrating the fourth embodiment of the language model training method of this application.

[0065] Figure 3 A schematic diagram of the architecture of a language model training method provided in one embodiment of this application;

[0066] Figure 4This is a schematic diagram of the device structure of the hardware operating environment involved in the language model training method in the embodiments of this application.

[0067] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0068] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0069] The following presents a first embodiment of the language model training method of this application. (Refer to...) Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the language model training method of this application. In this embodiment, the language model training method is applied to a first device, which is communicatively connected to each of the second devices. Both the first and second devices can be computing service devices with data processing, network communication, and program execution functions, such as servers, tablets, personal computers, and mobile phones. The first and second devices can be deployed in different enterprises; that is, each enterprise uses its own devices to jointly train the language model, so as to perform natural language processing tasks using the trained language model. In specific implementations, specific natural language processing tasks can be set according to the specific needs of each enterprise. For example, in a dialogue scenario, a task to generate a response based on the preceding text can be set; in a summary generation scenario, a task to generate a text summary based on the text can be set. The number of second devices is at least two. The operations performed or functions implemented by the first and second devices during the language model training process are different, while the operations performed or functions implemented by each second device are the same. Therefore, in the following embodiments, the two types of devices are distinguished as "first" and "second." In some application scenarios, the first device can also be called a server or coordinator, and the second device can be called a client or participant.

[0070] The first device deploys a large language model to be trained, and each of the second devices deploys a small language model to be trained. A language model is a model that can be used to perform natural language processing tasks, such as LLaMa213B, LLaMa2 7B, OPT 7B, etc. In this embodiment, both the large and small language models belong to the category of language models. The large language model has a more complex structure and more parameters, while the small language model has a relatively simpler structure and fewer parameters. The language models that various enterprises can deploy or support are likely to be heterogeneous. Based on this, in this embodiment, the large language model deployed in the first device and the small language model deployed in at least one of the second devices are heterogeneous. The model structures of the small language models deployed in each of the second devices can be the same or different; if they are different model structures, they can be heterogeneous or homogeneous, and this embodiment does not impose any restrictions on this. Heterogeneity refers to the different model architectures and parameter scales of the two language models. For example, in a feasible implementation, the first device can deploy LLaMa2 13B, and the second device can deploy OPT 7B. Due to differences in structure and parameter quantity, a large language model, once trained, generally outperforms a small language model in processing natural language tasks. However, the training and operation of a large language model require more computational resources than a small language model. Therefore, the first and second devices can have different computational resources; the first device can have superior computing power compared to the second. The quality of these resources is reflected by parameters such as the number of CPU cores, clock frequency, memory bandwidth, and capacity. This embodiment does not impose any restrictions on these parameters. In practice, a company with strong computing resources can provide the first device, while a company with text data in various sub-fields but limited computing resources can provide the second device. These companies can collaborate to train the language model, leveraging their respective strengths to jointly obtain a language model with higher natural language processing capabilities.For example, banks and insurance companies need to train language models to perform natural language processing tasks in their business scenarios to improve the effectiveness of their services. While they possess text data in specific areas such as banking and insurance, their computing resources are limited. These companies cannot independently train large language models requiring significant computing power using their own equipment, making it difficult to successfully deploy language models that improve service effectiveness in their business scenarios. Conversely, companies or institutions with powerful computing resources lack text data in these specific areas, making it difficult to train language models with stronger natural language processing capabilities. Therefore, a collaborative approach could be adopted. Enterprises' devices can be used to collaboratively train language models, leveraging the strengths of both parties to jointly train a language model with higher natural language processing capabilities. For example, a large customer service question-and-answer language model can be deployed in the first device, and a small customer service question-and-answer language model can be deployed in the second device. The large and small customer service question-and-answer language models are used to respond to the customer's above statements to realize the function of robot customer service. By collaboratively training the large customer service question-and-answer language model in the first device and the small customer service question-and-answer language model in the second device, it is possible to combine the devices of these enterprises and leverage the strengths of both parties to jointly train a customer service question-and-answer language model with higher accuracy.

[0071] The model to be trained refers to the model whose parameters need to be optimized through the training process. Its initial model parameters can be obtained after model pre-training (or, a pre-trained model can be used as the model to be trained), or they can be initialized empirically or randomly. Using a pre-trained model as the model to be trained and training it on a specific natural language processing task is also called "fine-tuning," and the term "training" is used to represent this process in the following embodiments. In a specific implementation, using a pre-trained language model as the language model to be trained and further training it can achieve better training results, meaning the trained language model has stronger natural language processing capabilities. In this embodiment, it is not limited to using a pre-trained language model as the language model to be trained, nor is the specific pre-training method limited.

[0072] The first and second devices each deploy text data for training language models. Generally, there are multiple pieces of text data used for training language models, forming a training dataset. Each piece of text data includes input text data as input to the model and label text data as training labels for the model. For example, in a dialogue scenario, a language model needs to be trained to generate corresponding answers based on specific questions. In this case, the input text data can be the question, and the label text data can be the answer. The first and second devices can deploy publicly available text data (hereinafter referred to as "public text data") used for training language models within the industry, for joint training of the large and small language models. Each second device deploys its own proprietary text data (hereinafter referred to as "proprietary text data") for local training of the small language model within the second device. The proprietary text data of each enterprise can come from the enterprise's business data. Due to the differences in the specific business directions of each enterprise, the proprietary text data of each enterprise belongs to different sub-domains. Each second device participates in the training of the language model through its own proprietary text data in its respective sub-domain, which can provide the language model with knowledge from different sub-domains, thereby enabling the language model trained in the end to have stronger natural language processing capabilities. For example, the first device can be a device with computing power capable of supporting the training and operation of large language models, provided by a few enterprises or institutions capable of supporting such devices; the publicly available text data can be publicly available text data used to train language models within the industry; the various second devices can be deployed in enterprises such as banks and insurance companies, and the private text data can be business data from various enterprises, such as dialogue data from customer service personnel of banks and insurance companies when serving customers. This private text data belongs to the text data of sub-sectors such as banking business and insurance business. It should be noted that in specific implementations, the business data used needs to be approved by the customer and can be anonymized to avoid disclosing customer privacy.

[0073] In this embodiment, the language model training method includes steps S10 to S50:

[0074] Step S10: Receive the second training results sent by each of the second devices respectively, wherein the second training results include a second prediction result, which is the result obtained by the second device training the small language model with local private text data and then inputting the input text data from the public text data into the small language model for prediction.

[0075] The first device and each of the second devices jointly conduct multiple rounds of iterative training on the large language model and the small language model to be trained, based on their respective computing resources. Each round of training is based on the previous round. During each round of training, each second device trains its own small language model using its local private text data, optimizes the small language model, and then inputs the input text data from the public text data into the optimized small language model. After processing by the small language model, the predicted result is output (hereinafter referred to as the "second prediction result" for distinction). In this embodiment, the data format of the second prediction result is not limited. For example, the second prediction result can be the probability distribution corresponding to the text, that is, the probability distribution of the output text predicted by the small language model.

[0076] Each second device sends its second training result, including its own second prediction result, to the first device. In a specific implementation, the second training result may only include the second prediction result, or it may include other training results in addition to the second prediction result. The specific training results to be included can be set as needed, and this embodiment does not impose any restrictions on this.

[0077] It should be noted that the private text data on each second device comes from the specific sub-fields of each enterprise. Each second device trains its own small language model using its local private text data, enabling each small language model to learn the text information in the corresponding sub-field (which can be understood as a kind of "knowledge" acquired by the model). The second prediction result obtained by each second device processing the same public text data using its locally trained small language model is a prediction result made by each small language model based on its own acquired "knowledge".

[0078] The first device receives the second training results sent by each of the second devices and proceeds with the next training process based on these results. Since the second prediction result is merely a prediction made by a small language model trained and optimized using private text data on public text data within the second device, the second device does not need to send its private text data during collaborative training with the first device. Therefore, it does not directly expose its private text data to the first device. This allows the second device to utilize its private text data to help optimize the large language model in the first device, enabling the large language model to learn text information from various sub-domains while ensuring the privacy and security of the private text data within the second device. Furthermore, because the small language model has a relatively simple structure and fewer parameters, it requires less computing power from the second device. This allows even companies with limited computing resources to participate in language model training based on their own sub-domain text data, resulting in language models with stronger natural language processing capabilities.

[0079] For example, various second devices deployed in enterprises such as banks and insurance companies can use private text data from their dialogue business data to train their respective small language models locally. This allows each small language model to learn text information from specific sub-domains such as banking and insurance. The training results are then sent to the first device for training the large language model on the first device. This process does not expose the private text data of each enterprise, thus protecting the privacy and security of their private text data. At the same time, the computing power requirements of the second devices of each enterprise are also low. As a result, text data from specific sub-domains such as banking and insurance can participate in the training of the large language model, resulting in a language model with stronger natural language processing capabilities.

[0080] In one feasible implementation, the second device can first train a small language model using its own local private text data, and then further train the small language model using public text data to obtain an optimized small language model. This optimized small language model fully learns the text information in both the private and public text data. Then, the input text data from the public text data is input into the optimized small language model, processed by the small language model, and a second prediction result is output.

[0081] Step S20: Input the input text data from the publicly available text data into the large language model for prediction to obtain the first prediction result.

[0082] The first device, based on its computing power, inputs the input text data from the publicly available text data into a large language model, processes it through the large language model, and outputs the predicted result (hereinafter referred to as the "first prediction result" for distinction). In this embodiment, there are no restrictions on the data format of the first prediction result. For example, the first prediction result can be the probability distribution corresponding to the text, that is, the probability distribution of the output text predicted by the large language model.

[0083] It should be noted that the publicly available text data deployed in the first and second devices is the same. That is, the first and second devices use their respective language models to process the same text data to obtain prediction results. Since the structure and parameter scale of each language model are different, the training progress and the data used for training are different, so the "knowledge" learned by each language model is also different. Therefore, the prediction results obtained by processing the same text data may also be different.

[0084] Step S30: Align the first prediction result with the second prediction result using text components. Based on the aligned first prediction result and second prediction result, calculate the first knowledge distillation loss from the small language model to the large language model.

[0085] Knowledge distillation (KD) is a model training method that transfers knowledge from the teacher's model to the student's model.

[0086] After the first device obtains the first and second prediction results, it can utilize these results based on its computing resources to perform knowledge distillation from the small language model to the large language model. That is, the small language model acts as the teacher model, and the large language model as the student model. This allows the small language model, locally optimized by the second device, to further optimize the large language model, transferring the knowledge learned by the small language model to the large language model. This enables the large language model to also learn the text information corresponding to the specific sub-domains of each enterprise. Specifically, the first device uses the first and second prediction results, after text component alignment, to calculate the knowledge distillation loss (hereinafter referred to as the "first knowledge distillation loss" for distinction) for knowledge distillation from the small language model to the large language model. The specific calculation method for the first knowledge distillation loss is not limited in this embodiment; for example, it can be obtained by calculating the cross-entropy or KL (Kullback-Leibler) distance between the first and second prediction results.

[0087] A text token is the basic unit that makes up text. It is artificially divided, and the various text tokens divided according to a set of division rules constitute a vocabulary. Since the large language model and the small language model are heterogeneous in this embodiment, the division rules used by the large language model and the small language model may be different. That is, the vocabulary used by the large language model and the small language model may be different. In other words, the number of text tokens included in the two vocabularys may be different, and the text tokens themselves may be different. For example, for Chinese text, one vocabulary divides text tokens by each character, while the other vocabulary divides text tokens by each word (a word may include multiple characters).

[0088] Because the vocabulary is different, the text components involved in the first prediction result and the second prediction result are different. Therefore, in this embodiment, the first device can align the text components of the first prediction result and the second prediction result, matching the same text components and matching different text components according to a certain mapping rule, so that the text components involved in the first prediction result and the second prediction result can be matched, that is, the aligned first prediction result and the second prediction result are obtained. Then, the aligned first prediction result and the second prediction result are used to perform knowledge distillation, so that the heterogeneous large language model and small language model can also be trained collaboratively.

[0089] In this embodiment, the specific implementation of text unit alignment is not limited; that is, the mapping rules between text units in two vocabulary lists are not limited, and mapping rules can be preset as needed.

[0090] Step S40: Optimize the large language model based on the first knowledge distillation loss, and send the first training result to each of the second devices respectively. The first training result includes a third prediction result, which is obtained by inputting the input text data in the public text data into the optimized large language model for prediction. The second devices align the second prediction result with the third prediction result to perform text unit alignment. Based on the aligned second prediction result and the third prediction result, calculate the second knowledge distillation loss from the large language model to the small language model, and optimize the small language model based on the second knowledge distillation loss.

[0091] If the preset training termination conditions are met, then proceed to step S50: obtain the large language model that has been trained.

[0092] If the preset training termination condition is not met, return to step S10.

[0093] After calculating the first knowledge distillation loss, the first device can optimize the large language model based on its computing resources. Specifically, it can update the model parameters in the large language model with the goal of reducing the first knowledge distillation loss. For example, it can use the gradient descent algorithm to calculate the gradient values ​​corresponding to the model parameters in the large language model based on the first knowledge distillation loss, and update the model parameters in the large language model based on the gradient values, thereby achieving the purpose of optimizing the large language model.

[0094] The first and second prediction results are the results obtained by the large language model and the small language model respectively, processing the same input text data. The first knowledge distillation loss represents the difference between the first and second prediction results. The greater the difference, the greater the first knowledge distillation loss. By reducing the first knowledge distillation loss, the difference between the prediction results output by the large language model and the prediction results output by the small language model can be reduced. Since the small language model is trained on the private text data of the second device, it learns the mapping relationship between the input text data and the label text data in the second text data of the second device (which can be understood as a kind of "knowledge"). Therefore, by reducing the first knowledge distillation loss, the large language model can indirectly learn the "knowledge" possessed by the small language model, thereby optimizing the natural language processing capability of the large language model.

[0095] Understandably, by using the second prediction results sent by the second device to perform knowledge distillation from the small language model to the large language model, the first device can protect data privacy while avoiding the disadvantage of limited computing resources in most enterprises. This allows for the joint training of large language models using text data from various sub-fields owned by enterprises with limited computing resources, in order to obtain language models with stronger natural language processing capabilities. At the same time, it can make full use of the computing resources of each second device to distribute the computational pressure of optimizing large language models on a single device.

[0096] After optimizing the large language model, the first device inputs the input text data from the publicly available text data into the optimized large language model. After processing by the large language model, the predicted result is output (hereinafter referred to as the "third prediction result" for distinction). In this embodiment, there are no restrictions on the data format of the third prediction result. For example, the third prediction result can be the probability distribution corresponding to the text, that is, the probability distribution of the output text predicted by the small language model.

[0097] The first device sends a first training result, including the third prediction result, to each of the second devices. In a specific implementation, the first training result may only include the third prediction result, or it may include other training results in addition to the third prediction result. The specific training results to be included can be set as needed, and this embodiment does not impose any restrictions on this.

[0098] After receiving the first training result from the first device, the second device, similarly, finds differences in the text components involved in the second and third prediction results due to the different vocabulary. Therefore, in this embodiment, the second prediction result is aligned with the third prediction result in the first training result using text component alignment. The alignment method is the same as the method used by the first device to align the text components of the first and second prediction results, and will not be elaborated here. The second device obtains the aligned second and third prediction results, and then uses them to perform knowledge distillation, thereby enabling the heterogeneous large language model and small language model to be trained collaboratively.

[0099] The second device uses the second and third prediction results to perform knowledge distillation from the large language model to the small language model. That is, the large language model is used as the teacher model and the small language model is used as the student model. This can enhance the natural language processing capabilities of the small language model through the large language model, so that even enterprises with limited computing resources can obtain the small language model enhanced by the large language model. Finally, after multiple rounds of training, a small language model with stronger natural language processing capabilities can be obtained.

[0100] Specifically, the second device uses the second and third prediction results after text component alignment to calculate the knowledge distillation loss (hereinafter referred to as "second knowledge distillation loss" for distinguishing between the two) for knowledge distillation from the large language model to the small language model. The specific calculation method for the second knowledge distillation loss is not limited in this embodiment; for example, it can be obtained by calculating the cross-entropy or KL (Kullback-Leibler) distance between the second and third prediction results.

[0101] After calculating the second knowledge distillation loss, the second device can optimize the small language model based on the second knowledge distillation loss. Specifically, it can update the model parameters in the small language model with the goal of reducing the second knowledge distillation loss. For example, it can use the gradient descent algorithm to calculate the gradient values ​​corresponding to the model parameters in the small language model based on the second knowledge distillation loss, and update the model parameters in the small language model based on the gradient values, thereby achieving the goal of optimizing the large language model.

[0102] Because the large language model is enhanced through the knowledge distillation process, it learns the mapping relationship between input text data and label text data in the private text data of each second device (which can be understood as a kind of "knowledge"). At the same time, the more complex model structure of the large language model also enables it to learn different "knowledge" from the small language model through the knowledge distillation process. Therefore, by reducing the loss of the second knowledge distillation, the small language model can indirectly learn the new "knowledge" possessed by the large language model, thereby optimizing the natural language processing capability of the small language model.

[0103] After the first device sends the first training result to the second device, it can be checked whether the preset training termination condition is met. The preset training termination condition is the condition that must be met to end the pre-training, and it can be set as needed. In this embodiment, there is no limitation on it. For example, in a specific implementation, it can be set to the convergence of the loss function of the large language model or the convergence of the loss function of the small language model, or it can be set to the number of training rounds reaching a set number of rounds, or the training duration reaching a set duration, etc.

[0104] If the first device detects that the preset training termination condition is met, it can end the iterative training process and use the final updated large language model as the trained large language model. Correspondingly, each of the second devices also ends its iterative training process and uses the final updated small language model as the trained small language model. The first device can use the trained large language model to complete the corresponding natural language processing task, and each of the second devices can use the trained small language model to complete the corresponding natural language processing task.

[0105] If the first device detects that the preset training termination condition is not met, it can enter the next round of training. That is, the second device performs the next round of local training based on the small language model after knowledge distillation in this round, and obtains the second training result based on the public text data again, and returns it to the first device. The first device receives the second training result again and executes the next training process. This process is repeated until the preset training termination condition is met.

[0106] In this embodiment, a large language model to be trained is deployed on a first device, and a small language model to be trained is deployed on a second device. The first device uses publicly available text data, while the second devices use both publicly available text data and their respective private text data to collaboratively train the large and small language models. This achieves a collaborative training architecture for the large and small language models, enabling enterprises with limited computing resources to participate in training the large language model and utilize their private, segmented text data to enhance the natural language processing capabilities of the large language model. For example, banks and insurance companies can use their limited computing power to participate in the collaborative training of the large and small language models, utilizing text data from segmented fields such as banking and insurance to enhance the natural language processing capabilities of the large language model. In each training round, after each second device trains the small language model locally using its own private text data, the input text data from the publicly available text data is input into the small language model for prediction to obtain a second prediction result. This second prediction result is then sent to the first device. This allows the private text data from the second devices to help optimize the large language model in the first device without exposing the privacy of the private text data of each second device. Furthermore, each second device... The system deploys a small language model, thus requiring less computing power from the second device. This allows companies with limited computing resources to participate in language model training, resulting in a language model with stronger natural language processing capabilities. The first device receives the second prediction results from the second devices and inputs the public text data into the large language model for prediction, obtaining a first prediction result. After aligning the first and second prediction results with their text components, the system calculates the first knowledge distillation loss from the small language model to the large language model based on the aligned first and second prediction results. This first knowledge distillation loss is then used to optimize the large language model, enabling further optimization of the large language model using the small language model optimized locally on the second device. The system transfers text information (which can be understood as "knowledge" learned by the model) from the specific domains of each company learned by the small language model to the large language model, allowing the large language model to absorb the privacy knowledge of each company's domain. Since this process is performed on the first device, it avoids the limitation of limited computing resources for most companies and fully utilizes the computing resources of the second devices, distributing the computational pressure of optimizing the large language model from a single device.Based on the large language model optimized by knowledge distillation, the first device inputs the public text data into the large language model for prediction, obtaining a third prediction result. This third prediction result is then sent to the second device, where each second device aligns the text components of the second and third prediction results. Based on the aligned second and third prediction results, a second knowledge distillation loss is calculated. This loss is used to optimize the small language model, enhancing its natural language processing capabilities through the large language model. Furthermore, in the next training round, the enhanced small language model learns more privacy knowledge from each enterprise's domain and feeds it back into the large language model. This iterative cycle continues until, after multiple rounds of training, a larger and smaller language model with stronger natural language processing capabilities is obtained. Moreover, before knowledge distillation, the first and second devices align the text components of the prediction results from different language models, enabling the heterogeneous large and small language models to train collaboratively. This supports the current situation where most language models deployed or supported by various enterprises are heterogeneous. Therefore, this embodiment fully utilizes the computing resources of each device and the corresponding enterprise's private, segmented text data to train the model language. This allows the large language model to absorb the private knowledge of each enterprise's domain, even with limited enterprise computing resources, thereby improving the large language model's natural language processing capabilities while protecting the data privacy of each enterprise. For example, it enables banks, insurance companies, and other enterprises with limited computing power to utilize their text data from their respective banking and insurance business segments, and collaborate with devices provided by enterprises or institutions with strong computing resources to train both large and small language models. This allows the large language model to absorb the private knowledge from its banking and insurance business segments, improving its natural language processing capabilities while protecting the data privacy of the banks, insurance companies, and other enterprises.

[0107] In one feasible implementation, a specific method for aligning text components of the first and second prediction results is proposed. Specifically, for distinction, the vocabulary corresponding to the large language model is referred to as the first vocabulary, and the vocabulary corresponding to the small language model is referred to as the second vocabulary. The first prediction result includes the prediction probability corresponding to each text component in the first vocabulary, i.e., the probability distribution of the output text. Based on the probability distribution of the output text, the output text predicted by the large language model can be determined. For example, the text component with the highest prediction probability among all text components can be determined as the output text. The second prediction result includes the prediction probability corresponding to each text component in the second vocabulary. Similarly, the output text predicted by the small language model can be determined based on the second prediction result.

[0108] The step of aligning the first prediction result with the second prediction result in text components in step S30 includes steps S301 to S303:

[0109] Step S301: Calculate the minimum edit distance between each text unit in the first vocabulary and each text unit in the second vocabulary.

[0110] For each text unit in the first vocabulary and for each text unit in the second vocabulary, the first device can calculate the minimum edit distance between the text unit in the first vocabulary and the text unit in the second vocabulary. Minimum edit distance is an algorithm for measuring the similarity between two strings. It is obtained by calculating the minimum number of operations required to transform one string into another, including insertions, deletions, and substitutions. In this embodiment, the minimum edit distance between the text units in the first and second vocabulary is obtained by calculating the minimum number of operations required to transform the text units in the first vocabulary into the text units in the second vocabulary.

[0111] Step S302: For any target text unit in the first vocabulary, align the text unit with the target text unit that has the smallest minimum edit distance from the target text unit in the second vocabulary.

[0112] Since the operation of aligning each text unit in the first vocabulary with the text units in the second vocabulary is the same, we will use a text unit in the first vocabulary as an example for explanation, and refer to this text unit as the target text unit for distinction.

[0113] After calculating the minimum edit distance between each text unit in the second vocabulary and the target text unit, the first device aligns the text unit with the smallest minimum edit distance to the target text unit.

[0114] Thus, for each text unit in the first vocabulary, a text unit that aligns with it is found in the second vocabulary.

[0115] Step S303: Align the prediction probability corresponding to the target text component in the first prediction result with the prediction probability of the text component aligned with the target text component in the second prediction result to obtain the aligned first prediction result and the second prediction result.

[0116] Then, the first device aligns the predicted probability corresponding to the target text unit in the first prediction result with the predicted probability of the text unit aligned with the target text unit in the second prediction result. Thus, for each predicted probability in the first prediction result, after finding its aligned prediction probability in the second prediction result, the aligned first and second prediction results are obtained. Furthermore, a first knowledge distillation loss can be calculated based on the aligned first and second prediction results; specifically, for example, cross-entropy or KL distance can be calculated based on the aligned prediction probabilities.

[0117] In this embodiment, the minimum edit distance is used to align the text components of the first and second prediction results, so that the finally aligned text components have a high and reasonable similarity. This makes the first knowledge distillation loss calculated based on the aligned first and second prediction results more accurate, and thus makes the knowledge transfer from the small language model to the large language model more accurate.

[0118] In one feasible implementation, when performing knowledge distillation from a small language model to a large language model, the loss function of the large language model may include a fine-tuning loss function of the large language model and a knowledge distillation loss function from the small language model to the large language model. The fine-tuning loss function enables the large language model to learn textual information from the text data, while the knowledge distillation loss function enables knowledge transfer from the small language model to the large language model. Following step S20, a step S60 is included, where step S40, which optimizes the large language model based on the first knowledge distillation loss, includes step S401.

[0119] Step S60: Calculate the first prediction loss based on the tag text data in the publicly available text data and the corresponding first prediction result.

[0120] The labeled text data is the standard output text data set for the input text data. The first prediction result is the output of the large language model based on the input text data. The fine-tuning loss of the large language model (also known as the first prediction loss) represents the difference between the first prediction result and the labeled text data. The greater the difference, the greater the fine-tuning loss. By reducing the fine-tuning loss, the difference between the prediction result output by the large language model and the standard result can be reduced, allowing the large language model to learn the mapping relationship between the input text data and the labeled text data in the public text data, thereby making more accurate predictions. In other words, the natural language processing capability of the large language model is optimized. The specific calculation method of the fine-tuning loss is not limited in this embodiment. For example, the fine-tuning loss can be obtained by calculating the cross-entropy between the first prediction result and the labeled text data.

[0121] Step S401: Optimize the large language model based on the first prediction loss and the first knowledge distillation loss.

[0122] In a specific implementation, the first prediction loss and the first knowledge distillation loss can be added together or weighted to obtain the total loss. With the goal of reducing the total loss, the model parameters in the large language model can be updated. For example, the gradient value corresponding to the model parameter in the large language model can be calculated based on the total loss using the gradient descent algorithm. The model parameter in the large language model can be updated based on the gradient value, thereby achieving the purpose of optimizing the large language model.

[0123] In one feasible implementation, the large language model and the small language model can be applied to dialogue scenarios to generate responses to a given preamble (e.g., a question), thus achieving dialogue functionality. Accordingly, public and private text data can include preamble text data as model input and response text data as model training labels. For example, a preamble and corresponding response from a bank's dialogue business data could be "What should I pay attention to when applying for a loan?" and "I need to pay attention to loan interest rates, loan amounts, repayment methods, and the consequences of overdue payments...". Preamble text data and response text data can be obtained from this preamble and response, serving as private text data deployed on a second device within the bank. Multiple pieces of private text data can be obtained from the dialogue business data and deployed on the second device for local training of the small language model. After obtaining a trained large language model, the first device can deploy it in a device used for dialogue tasks. By acquiring target context text data (e.g., user-input context text data), it processes the target context text data using the trained large language model to obtain target response text data and output it (e.g., through a display device on the device to the user). Similarly, after obtaining a trained small language model, the second device can also deploy it in a device used for dialogue tasks (which may be a device with limited computing resources) to process the target context text data, obtain target response text data, and output it.

[0124] Based on the first embodiment described above, a second embodiment of the language model training method of this application is proposed. In this embodiment, content that is the same as or similar to that in the first embodiment can be referred to the above description and will not be repeated hereafter. In this embodiment, by selecting the samples used in the knowledge distillation process, the accuracy of knowledge transfer during knowledge distillation is improved, thereby further enhancing the natural language processing capabilities of the trained large and small language models. Specifically, if there are multiple publicly available text data, when performing knowledge distillation from the small language model to the large language model, or from the large language model to the small language model, the publicly available text data can be selected, and a portion of the publicly available text data can be used for knowledge distillation.

[0125] In one feasible implementation, when performing knowledge distillation from a small language model to a large language model, the publicly available text data used is selected to improve the accuracy of knowledge transfer from the small language model to the large language model. Specifically, for each piece of publicly available text data, the second training result also includes a second prediction loss corresponding to the publicly available text data. The second prediction loss is calculated by the second device based on the tag text data in the publicly available text data and the corresponding second prediction result. The specific calculation method of the second prediction loss is not limited in this embodiment; for example, the second prediction loss can be obtained by calculating the cross-entropy between the second prediction result and the tag text data.

[0126] Following step S20, steps S70 to S80 are also included:

[0127] Step S70: Calculate the first prediction loss based on the tag text data in the publicly available text data and the corresponding first prediction result.

[0128] The label text data is the standard output text data set for the input text data. The first prediction result is the output of the large language model based on the input text data. The first prediction loss represents the difference between the first prediction result and the label text data; the greater the difference, the greater the first prediction loss. The specific calculation method for the first prediction loss is not limited in this embodiment; for example, it can be obtained by calculating the cross-entropy between the first prediction result and the label text data.

[0129] Step S80: Based on the public text data in each of the aforementioned public text data where the second prediction loss is less than the first prediction loss, perform the step of aligning the first prediction result and the second prediction result into text components, and calculate the first knowledge distillation loss from the small language model to the large language model based on the aligned first prediction result and the second prediction result.

[0130] For each piece of public text data, through the aforementioned steps, the first prediction loss of the large language model and the second prediction loss of the small language model can be calculated. The first prediction loss and the second prediction loss are compared. If the second prediction loss is less than the first prediction loss, then the public text data is selected for the knowledge distillation process from the small language model to the large language model. That is, the aligned first and second prediction results corresponding to the public text data are selected to calculate the first knowledge distillation loss.

[0131] In this embodiment, since the second prediction loss corresponding to the public text data is less than the first prediction loss, it indicates that the result obtained by the small language model processing the public text data is more accurate than the result obtained by the large language model processing the public text data. That is, it indicates that the "knowledge" based on the small language model when processing the public text data is more accurate than the "knowledge" based on the large language model when processing the public text data. Therefore, selecting public text data with a second prediction loss less than the first prediction loss for the knowledge distillation process from the small language model to the large language model can improve the accuracy of knowledge transfer during the knowledge distillation process. That is, it transfers the positive "knowledge" in the small language model to the large language model and avoids transferring the "negative" knowledge in the small language model to the large language model, thereby further improving the natural language processing capability of the finally trained large language model.

[0132] In one feasible implementation, when performing knowledge distillation from a large language model to a small language model, the publicly available text data used is selected to improve the accuracy of knowledge transfer from the large language model to the small language model. Specifically, after the first device optimizes the large language model based on the first knowledge distillation loss, the method further includes step S90:

[0133] The third prediction loss is calculated based on the tag text data in the public text data and the corresponding third prediction result. Each of the second devices performs the step of aligning the second prediction result with the third prediction result based on the public text data in each piece of public text data for which the third prediction loss is less than the second prediction loss. Based on the aligned second prediction result and the third prediction result, the second knowledge distillation loss from the large language model to the small language model is calculated.

[0134] For each piece of publicly available text data, the first training result also includes a third prediction loss corresponding to the publicly available text data. The labeled text data is the standard output text data set for the input text data. The third prediction result is the output of the large language model, optimized by knowledge distillation, for the input text data. The third prediction loss represents the difference between the third prediction result and the labeled text data; the greater the difference, the greater the third prediction loss. The specific calculation method for the third prediction loss is not limited in this embodiment; for example, it can be obtained by calculating the cross-entropy between the third prediction result and the labeled text data.

[0135] The first device sends the first training result, which includes the third prediction result and the third prediction loss, to each of the second devices.

[0136] For each piece of public text data, through the aforementioned steps, the third prediction loss of the large language model after knowledge distillation optimization and the second prediction loss of the small language model can be calculated. The second device compares the third prediction loss with the second prediction loss. If the third prediction loss is less than the second prediction loss, then the public text data is selected for the knowledge distillation process from the large language model to the small language model. That is, the aligned third prediction result and the second prediction result corresponding to the public text data are selected to calculate the second knowledge distillation loss.

[0137] In this embodiment, since the third prediction loss corresponding to the public text data is less than the second prediction loss, it indicates that the result obtained by the large language model after knowledge distillation optimization of the public text data is more accurate than the result obtained by the small language model after the public text data is processed. That is, it indicates that the "knowledge" on which the large language model after knowledge distillation optimization of the public text data is based is more accurate than the "knowledge" on which the small language model is based when processing the public text data. Therefore, selecting public text data with a third prediction loss less than the second prediction loss for the knowledge distillation process from the large language model to the small language model can improve the accuracy of knowledge transfer during the knowledge distillation process. That is, it transfers the positive "knowledge" in the large language model to the small language model and avoids transferring the "negative" knowledge in the large language model to the small language model, thereby further improving the natural language processing capability of the finally trained small language model.

[0138] Based on the first and / or second embodiments described above, a third embodiment of the language model training method of this application is proposed. In this embodiment, content that is the same as or similar to the first and second embodiments described above can be referred to the above description and will not be repeated hereafter. In this embodiment, the step of optimizing the large language model based on the first knowledge distillation loss in step S40 includes S402:

[0139] Step S402: Update the model parameters of the adapter module based on the first knowledge distillation loss to optimize the large language model.

[0140] A large language model can include a model ontology and an adapter module. The pre-trained large language model can be used as the model ontology, and the adapter module can be attached to the model ontology. The structure of the adapter module and the specific method of attachment are not limited in this embodiment. For example, in one feasible implementation, the LoRA Adapter method can be used to attach the adapter module.

[0141] The first device can maintain the model parameters of the model ontology in the large language model unchanged while updating the model parameters of the adapter module when optimizing the large language model using the first knowledge distillation loss. This reduces the computational complexity of the first device when performing knowledge distillation optimization on the large language model, thereby reducing the computing power requirements of the first device.

[0142] Based on the first, second, and / or third embodiments described above, a fourth embodiment of the language model training method of this application is proposed. (Refer to...) Figure 2 , Figure 2 This is a flowchart illustrating the fourth embodiment of the language model training method of this application. In this embodiment, content that is the same as or similar to the first, second, and third embodiments described above can be referred to the above description and will not be repeated hereafter. In this embodiment, the language model training method is applied to any one of a plurality of second devices; that is, the executing entity is any one of the second devices, and the method includes steps A10 to A60:

[0143] Step A10: Train the small language model using local private text data.

[0144] Step A20: Input the input text data from the publicly available text data into the locally trained small language model to obtain a second prediction result.

[0145] Step A30: Send the second training result, including the second prediction result, to the first device, so that after receiving the second training results sent by each of the second devices, the first device inputs the input text data in the public text data into the large language model for prediction to obtain a first prediction result, and aligns the first prediction result with the second prediction result into text components. Based on the aligned first prediction result and second prediction result, a first knowledge distillation loss is calculated from the small language model to the large language model, and the large language model is optimized based on the first knowledge distillation loss. The first training result is then sent to each of the second devices.

[0146] Step A40: Receive the first training result sent by the first device, align the second prediction result with the third prediction result in the first training result for text component units, and calculate the second knowledge distillation loss from the large language model to the small language model based on the aligned second prediction result and the third prediction result. The third prediction result is obtained by the first device inputting the input text data from the public text data into the optimized large language model for prediction.

[0147] Step A50: Optimize the small language model based on the second knowledge distillation loss.

[0148] If the preset training termination conditions are met, proceed to step A60: obtain the small language model that has been trained.

[0149] If the preset training termination condition is not met, return to step A10.

[0150] The specific implementation of steps A10 to A60 above can refer to the specific implementation of steps S10 to S50 in the first embodiment above, and the achieved effects can also refer to the first embodiment above, which will not be repeated here.

[0151] In one feasible implementation, a specific method for aligning text components of the second and third prediction results is proposed. Specifically, for distinction, the vocabulary corresponding to the large language model is referred to as the first vocabulary, and the vocabulary corresponding to the small language model is referred to as the second vocabulary. The third prediction result includes the prediction probability corresponding to each text component in the first vocabulary, i.e., the probability distribution of the output text. Based on the probability distribution of the output text, the output text predicted by the large language model can be determined. For example, the text component with the highest prediction probability among all text components can be determined as the output text. The second prediction result includes the prediction probability corresponding to each text component in the second vocabulary. Similarly, the output text predicted by the small language model can be determined based on the second prediction result.

[0152] The step A40, which aligns the second prediction result with the third prediction result in the first training result for text component units, includes A401 to A403:

[0153] Step A401: Calculate the minimum edit distance between each text unit in the first vocabulary and each text unit in the second vocabulary.

[0154] For each text unit in the first vocabulary and for each text unit in the second vocabulary, the second device can calculate the minimum edit distance between the text unit in the first vocabulary and the text unit in the second vocabulary. Minimum edit distance is an algorithm for measuring the similarity between two strings. It is obtained by calculating the minimum number of operations required to transform one string into another, including insertions, deletions, and substitutions. In this embodiment, the minimum edit distance between the text units in the first and second vocabulary is obtained by calculating the minimum number of operations required to transform the text units in the first vocabulary into the text units in the second vocabulary.

[0155] Step A402: For any target text unit in the second vocabulary, align the text unit with the smallest minimum edit distance from the target text unit among the text units in the first vocabulary.

[0156] Since the operation of aligning each text unit in the second vocabulary with the text units in the first vocabulary is the same, we will use a text unit in the second vocabulary as an example for explanation, and refer to this text unit as the target text unit for distinction.

[0157] After calculating the minimum edit distance between each text unit in the first vocabulary and the target text unit, the second device aligns the text unit with the smallest minimum edit distance to the target text unit.

[0158] Thus, for each text unit in the second vocabulary, a text unit that aligns with it is found in the first vocabulary.

[0159] Step A403: Align the prediction probability corresponding to the target text component in the second prediction result with the prediction probability of the text component aligned with the target text component in the third prediction result to obtain the aligned second prediction result and the third prediction result.

[0160] Then, the second device aligns the predicted probabilities corresponding to the target text units in the second prediction result with the predicted probabilities of the text units aligned with the target text units in the third prediction result. Thus, for each predicted probability in the second prediction result, after finding its aligned prediction probability in the third prediction result, the aligned second and third prediction results are obtained. Furthermore, a second knowledge distillation loss can be calculated based on the aligned second and third prediction results; specifically, for example, cross-entropy or KL distance can be calculated based on the aligned prediction probabilities.

[0161] In this embodiment, the minimum edit distance is used to align the text components of the second and third prediction results, so that the finally aligned text components have a high and reasonable similarity. This makes the second knowledge distillation loss calculated based on the aligned second and third prediction results more accurate, and thus makes the knowledge transfer from the large language model to the small language model more accurate.

[0162] In one feasible implementation, step A50 includes A501:

[0163] Step A501: Update the model parameters of the adapter module based on the second knowledge distillation loss to optimize the small language model.

[0164] A small language model can include a model ontology and an adapter module. The pre-trained small language model can be used as the model ontology, and the adapter module can be attached to the model ontology. The structure of the adapter module and the specific method of attachment are not limited in this embodiment. For example, in one feasible implementation, the LoRA Adapter method can be used to attach the adapter module.

[0165] The second device can maintain the model parameters of the model ontology within the small language model while updating the model parameters of the adapter module when optimizing the small language model using the second knowledge distillation loss. This reduces the computational complexity of the second device when performing knowledge distillation optimization on the small language model, thereby reducing the computing power requirements of the second devices for various enterprises.

[0166] In one feasible implementation, when performing knowledge distillation from a large language model to a small language model, the loss function of the small language model may include a fine-tuning loss function for the small language model and a knowledge distillation loss function from the large language model to the small language model. The fine-tuning loss function enables the small language model to learn textual information from the text data, while the knowledge distillation loss function enables knowledge transfer from the large language model to the small language model. Following step A20, step A70 is also included, and step A50 includes A502:

[0167] Step A70: Calculate the second prediction loss based on the tag text data in the publicly available text data and the corresponding second prediction result.

[0168] The labeled text data is the standard output text data set for the input text data. The second prediction result is the output of the small language model based on the input text data. The fine-tuning loss of the small language model (also known as the second prediction loss) represents the difference between the second prediction result and the labeled text data. The greater the difference, the greater the fine-tuning loss. By reducing the fine-tuning loss, the difference between the predicted result output by the small language model and the standard result can be reduced, allowing the small language model to learn the mapping relationship between the input text data and the labeled text data in the public text data, thereby making more accurate predictions. In other words, the natural language processing capability of the large language model is optimized. The specific calculation method of the fine-tuning loss is not limited in this embodiment. For example, the fine-tuning loss can be obtained by calculating the cross-entropy between the second prediction result and the labeled text data.

[0169] Step A502: Optimize the small language model based on the second prediction loss and the second knowledge distillation loss.

[0170] In a specific implementation, the second prediction loss and the second knowledge distillation loss can be added together or weighted to obtain the total loss. With the goal of reducing the total loss, the model parameters in the small language model are updated. For example, the gradient value corresponding to the model parameter in the small language model can be calculated based on the total loss using the gradient descent algorithm. The model parameter in the small language model is then updated based on the gradient value, thereby achieving the purpose of optimizing the small language model.

[0171] Based on the fourth embodiment described above, a fifth embodiment of the language model training method of this application is proposed. In this embodiment, content that is the same as or similar to that in the fourth embodiment can be referred to the above description and will not be repeated hereafter. In this embodiment, by selecting the samples used in the knowledge distillation process, the accuracy of knowledge transfer during knowledge distillation is improved, thereby further enhancing the natural language processing capabilities of the trained large and small language models. Specifically, if there are multiple publicly available text data, when performing knowledge distillation from the small language model to the large language model, or from the large language model to the small language model, the publicly available text data can be selected, and a portion of the publicly available text data can be used for knowledge distillation.

[0172] In one feasible implementation, when performing knowledge distillation from a large language model to a small language model, the publicly available text data used is selected to improve the accuracy of knowledge transfer from the large language model to the small language model. Specifically, for each piece of publicly available text data, the first training result also includes a third prediction loss corresponding to the publicly available text data, which is calculated by the first device based on the labeled text data in the publicly available text data and the corresponding third prediction result.

[0173] After step A20, steps A80 to A90 are also included:

[0174] Step A80: Calculate the second prediction loss based on the tag text data in the publicly available text data and the corresponding second prediction result.

[0175] Step A90: Based on the public text data in each of the aforementioned public text data where the third prediction loss is less than the second prediction loss, perform the step of aligning the second prediction result with the third prediction result into text components, and calculate the second knowledge distillation loss from the large language model to the small language model based on the aligned second prediction result and the third prediction result.

[0176] The specific implementation of steps A80 to A90 above can refer to the specific implementation of step S90 in the second embodiment above, and the achieved effect can also refer to the second embodiment above, which will not be repeated here.

[0177] In one feasible implementation, when performing knowledge distillation from a small language model to a large language model, the publicly available text data used is selected to improve the accuracy of knowledge transfer from the small language model to the large language model. Specifically, step A30 includes A301:

[0178] Step A301: Send the second training result, including the second prediction result and the second prediction loss, to the first device so that after the first device calculates the first prediction loss based on the labeled text data in the public text data and the corresponding first prediction result, it performs the step of aligning the first prediction result and the second prediction result into text components based on the public text data in each piece of public text data for which the second prediction loss is less than the first prediction loss, and calculates the first knowledge distillation loss from the small language model to the large language model based on the aligned first prediction result and the second prediction result.

[0179] The specific implementation of step A301 above can refer to the specific implementation of steps S70-S80 in the second embodiment above, and the achieved effect can also refer to the second embodiment above, which will not be repeated here.

[0180] To aid in understanding the above embodiments of the language model training method of this application, an example combining the language model training method of the above embodiments is proposed. (Refer to...) Figure 3 The first device can be called the Federated Server, and each of the second devices can be called a client. 1 D 2 ...Dk ... D K D represents the private text data of each of the K clients. P This represents publicly available text data, and "Adapter" represents the adapter module. Adapter modules are attached to pre-trained Large Language Model (LLM) and Small Language Model (SLM). The server, in collaboration with various clients, fine-tunes the models. During this fine-tuning process, the parameters of the LLM and SLM models are frozen; only the parameters of the Adapter are updated and interacted with. For example, the server-side LLM could use the LLaMa2 13B model, and the client-side SLM could use models such as GPT-2, OPT 7B, or LLaMa2 7B.

[0181] In each round of collaborative fine-tuning, the client fine-tunes the SLM using local private text data, then fine-tunes it again using public text data, and finally uses the fine-tuned SLM to predict the input text data for each piece of public text data, outputting a second prediction result p. k The second prediction loss is calculated based on the second prediction result and the tag text data in the publicly available text data. k (Specifically, cross-entropy loss, CE loss), p k and l k Upload server (corresponding) Figure 3 (Step 1 in the text).

[0182] On the server side, LLM is used to predict the input text data for each piece of publicly available text data, outputting a first prediction result. The first and second prediction results are then aligned to their corresponding text components. Figure 3 (Step 2 in the text).

[0183] The server performs knowledge distillation from SLM to LLM based on the aligned first and second prediction results to optimize the LLM (corresponding to...). Figure 3 (Step 3 in the process). The server calculates the first prediction loss based on the first prediction result and the labeled text data in the public text data. It then compares the first and second prediction losses corresponding to the public text data, selecting public text data with a second prediction loss less than the first prediction loss for knowledge distillation. During knowledge distillation, the LLM model parameters are frozen, and the Adapter model parameters are updated.

[0184] After optimizing the LLM through knowledge distillation, the server uses LLM to predict the input text data for each piece of public text data, outputting a third prediction result p. Based on the third prediction result and the tag text data in the public text data, the server calculates a third prediction loss l (specifically, cross-entropy loss, CE loss), and sends p and l to each client (corresponding to...).Figure 3 (Step 4 in the text).

[0185] After receiving p and l, the client performs a second prediction on p. k Align the text components with the third prediction result p (corresponding to) Figure 3 (Step 5 in the text).

[0186] The client bases the second prediction result p after alignment. k And the third prediction result p, perform knowledge distillation from LLM to SLM to optimize SLM (corresponding to Figure 3 (Step 6 in the text). Specifically, the server compares the second prediction loss corresponding to the publicly available text. k For the third prediction loss l, publicly available text data with a third prediction loss less than the second prediction loss are selected for knowledge distillation. During knowledge distillation, the model parameters of the SLM are frozen, and the model parameters of the Adapter are updated.

[0187] When performing knowledge distillation from SLM to LLM on the server side, the following loss function is used:

[0188] L LLM =a*L LLM FT +(1-a)*CE(p LLM p SLM )

[0189] When performing knowledge distillation from LLM to SLM on the client side, the following loss function is used:

[0190] L SLM =a*L SLM FT +(1-a)*CE(p SLM p LLM )

[0191] Where L LLM L represents the loss function of a large language model. LLM FT L represents the fine-tuning loss function for a large language model (cross-entropy can be used). SLM L represents the loss function of the small model. SLM FT p represents the fine-tuning loss function for the small language model. SLM and p LLM Let CE(p) represent the probability distributions of the output text predicted by the small language model and the large language model based on the same publicly available text data, respectively. LLM p SLM ) and CE(p SLM p LLMThe expression represents the calculation of the cross-entropy loss between two probability distributions, which is also the knowledge distillation loss function of the large language model and the small language model. 'a' is a control parameter, for example, set to 0.9.

[0192] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the language model training method of this application. Any simple transformations based on this technical concept are within the protection scope of this application.

[0193] This application provides a language model training device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the language model training method in the first embodiment described above.

[0194] The following is for reference. Figure 4 The diagram illustrates a structural schematic suitable for implementing the language model training device of the embodiments of this application. The language model training device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 4 The language model training device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0195] like Figure 4As shown, the language model training device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the language model training device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. The communication device 1009 allows the language model training device to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows language model training devices with various systems, it should be understood that it is not required to implement or possess all of the systems shown. More or fewer systems may be implemented alternatively.

[0196] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0197] Compared with the prior art, the beneficial effects of the language model training device provided in this application embodiment are the same as the beneficial effects of the language model training method provided in the above embodiment, and other technical features in the language model training device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0198] It should be understood that the various parts disclosed in the embodiments of this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0199] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0200] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the language model training method described in the above embodiments.

[0201] The computer-readable storage medium provided in this application embodiment may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0202] The aforementioned computer-readable storage medium may be included in the language model training device; or it may exist independently and not be assembled into the language model training device.

[0203] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by a language model training device, cause the language model training device to perform the functions defined in the methods of the embodiments disclosed in this application.

[0204] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0205] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0206] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0207] The readable storage medium provided in this application embodiment is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-described language model training method. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application embodiment are the same as the beneficial effects of the language model training method provided in the above-described embodiments, and will not be repeated here.

[0208] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the language model training method described above.

[0209] Compared with the prior art, the beneficial effects of the computer program product provided in this application embodiment are the same as the beneficial effects of the language model training method provided in the above embodiments, and will not be repeated here.

[0210] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A language model training method, characterized in that, The method, applied to a first device, which is communicatively connected to each of the second devices, includes: a large language model to be trained deployed in the first device, and a small language model to be trained deployed in each of the second devices; the large language model and the small language model in at least one of the second devices are heterogeneous; and the method comprises: Receive the second training results sent by each of the second devices respectively, wherein the second training results include the second prediction results, which are obtained by the second devices training the small language model with local private text data and then inputting the input text data from the public text data into the small language model for prediction; The input text data from the publicly available text data is input into the large language model for prediction, and a first prediction result is obtained; The first prediction result and the second prediction result are aligned with text components. Based on the aligned first prediction result and the second prediction result, the first knowledge distillation loss from the small language model to the large language model is calculated. The large language model is optimized based on the first knowledge distillation loss, and the first training result is sent to each of the second devices respectively. The first training result includes a third prediction result, which is obtained by inputting the input text data in the public text data into the optimized large language model for prediction. The second devices align the second prediction result with the third prediction result to perform text unit alignment. Based on the aligned second prediction result and the third prediction result, a second knowledge distillation loss is calculated from the large language model to the small language model. The small language model is optimized based on the second knowledge distillation loss. Return to the step of receiving the second training results sent by each of the second devices until the preset training end condition is met, and obtain the large language model that has been trained.

2. The method as described in claim 1, characterized in that, The publicly available text data consists of multiple pieces, each piece of publicly available text data including input text data as model input data and label text data as model training labels; For each piece of public text data, the second training result also includes a second prediction loss corresponding to the public text data. The second prediction loss is a prediction loss calculated by the second device based on the tag text data in the public text data and the corresponding second prediction result. After the step of inputting the input text data from the publicly available text data into the large language model for prediction and obtaining the first prediction result, the method further includes: The first prediction loss is calculated based on the tag text data in the publicly available text data and the corresponding first prediction result. Based on the publicly available text data in each of the aforementioned publicly available text data where the second prediction loss is less than the first prediction loss, the step of aligning the first prediction result and the second prediction result into text components is performed. Based on the aligned first prediction result and second prediction result, the first knowledge distillation loss from the small language model to the large language model is calculated.

3. The method as described in claim 2, characterized in that, For each piece of publicly available text data, the first training result also includes a third prediction loss corresponding to the publicly available text data; After optimizing the large language model based on the first knowledge distillation loss, the method further includes: The third prediction loss is calculated based on the tag text data in the public text data and the corresponding third prediction result. Each of the second devices performs the step of aligning the second prediction result with the third prediction result based on the public text data in each piece of public text data for which the third prediction loss is less than the second prediction loss. Based on the aligned second prediction result and the third prediction result, the second knowledge distillation loss from the large language model to the small language model is calculated.

4. The method as described in claim 1, characterized in that, The first prediction result includes the prediction probability corresponding to each text unit in the first vocabulary of the large language model, and the second prediction result includes the prediction probability corresponding to each text unit in the second vocabulary of the small language model. The step of aligning the first prediction result with the second prediction result into text components includes: Calculate the minimum edit distance between each text unit in the first vocabulary and each text unit in the second vocabulary; For any target text unit in the first vocabulary, align the text unit with the smallest minimum edit distance from the target text unit in the second vocabulary with the target text unit; Align the prediction probability corresponding to the target text component in the first prediction result with the prediction probability of the text component aligned with the target text component in the second prediction result to obtain the aligned first prediction result and second prediction result.

5. The method as described in claim 1, characterized in that, The large language model includes a model ontology and an adapter module; The steps for optimizing the large language model based on the first knowledge distillation loss include: The model parameters of the adapter module are updated based on the first knowledge distillation loss to optimize the large language model.

6. The method according to any one of claims 1 to 5, characterized in that, After the step of inputting the input text data from the publicly available text data into the large language model for prediction and obtaining the first prediction result, the method further includes: The first prediction loss is calculated based on the tag text data in the publicly available text data and the corresponding first prediction result. The steps for optimizing the large language model based on the first knowledge distillation loss include: The large language model is optimized based on the first prediction loss and the first knowledge distillation loss.

7. A language model training method, characterized in that, The method, applied to any one of a plurality of second devices, each second device being communicatively connected to a first device, wherein a large language model to be trained is deployed in the first device, and a small language model to be trained is deployed in each of the second devices, wherein the large language model and the small language model in at least one of the second devices are heterogeneous, comprises: The small language model was trained using local private text data. The input text data from the publicly available text data is fed into the locally trained small language model to obtain a second prediction result; The first device sends a second training result, including the second prediction result, to the first device. After receiving the second training results sent by each of the second devices, the first device inputs the input text data in the public text data into the large language model for prediction to obtain a first prediction result. The first prediction result and the second prediction result are aligned to form text units. Based on the aligned first prediction result and the second prediction result, a first knowledge distillation loss is calculated from the small language model to the large language model. The large language model is optimized based on the first knowledge distillation loss, and the first training result is sent to each of the second devices. The system receives the first training result sent by the first device, aligns the second prediction result with the third prediction result in the first training result using text components, and calculates the second knowledge distillation loss from the large language model to the small language model based on the aligned second prediction result and the third prediction result. The third prediction result is obtained by the first device inputting the input text data from the public text data into the optimized large language model for prediction. The small language model is optimized based on the second knowledge distillation loss, and the step of training the small language model with local private text data is returned until the preset training termination condition is met, and the trained small language model is obtained.

8. The method as described in claim 7, characterized in that, The publicly available text data consists of multiple pieces, each piece of publicly available text data including input text data as model input data and label text data as model training labels; For each piece of public text data, the first training result also includes a third prediction loss corresponding to the public text data, which is calculated by the first device based on the tag text data in the public text data and the corresponding third prediction result; After the step of inputting the input text data from the publicly available text data into the locally trained small language model to obtain the second prediction result, the method further includes: The second prediction loss is calculated based on the tag text data in the publicly available text data and the corresponding second prediction result; Based on the publicly available text data in each of the aforementioned publicly available text data where the third prediction loss is less than the second prediction loss, the step of aligning the second prediction result with the third prediction result into text components is performed. Based on the aligned second prediction result and the third prediction result, the second knowledge distillation loss from the large language model to the small language model is calculated.

9. The method as described in claim 8, characterized in that, The step of sending the second training result, including the second prediction result, to the first device includes: The second training result, including the second prediction result and the second prediction loss, is sent to the first device so that after the first device calculates the first prediction loss based on the labeled text data in the public text data and the corresponding first prediction result, it performs the step of aligning the first prediction result and the second prediction result into text components based on the public text data in each piece of public text data for which the second prediction loss is less than the first prediction loss, and calculates the first knowledge distillation loss from the small language model to the large language model based on the aligned first prediction result and the second prediction result.

10. The method as described in claim 7, characterized in that, The second prediction result includes the prediction probability corresponding to each text unit in the second vocabulary of the small language model, and the third prediction result includes the prediction probability corresponding to each text unit in the first vocabulary of the large language model. The step of aligning the second prediction result with the third prediction result in the first training result for text components includes: Calculate the minimum edit distance between each text unit in the first vocabulary and each text unit in the second vocabulary; For any target text unit in the second vocabulary, align the text unit with the smallest minimum edit distance from the target text unit among all text units in the first vocabulary with the target text unit; Align the prediction probability corresponding to the target text component in the second prediction result with the prediction probability of the text component aligned with the target text component in the third prediction result to obtain the aligned second prediction result and the third prediction result.

11. The method as described in claim 7, characterized in that, The small language model includes a model ontology and an adapter module; The steps for optimizing the small language model based on the second knowledge distillation loss include: The model parameters of the adapter module are updated based on the second knowledge distillation loss to optimize the small language model.

12. The method according to any one of claims 7 to 11, characterized in that, After the step of inputting the input text data from the publicly available text data into the locally trained small language model to obtain the second prediction result, the method further includes: The second prediction loss is calculated based on the tag text data in the publicly available text data and the corresponding second prediction result; The steps for optimizing the small language model based on the second knowledge distillation loss include: The small language model is optimized based on the second prediction loss and the second knowledge distillation loss.

13. A language model training device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the language model training method as described in any one of claims 1 to 12.

14. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the language model training method as described in any one of claims 1 to 12.

15. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the language model training method as described in any one of claims 1 to 12.

Images