Large language model training method, electronic equipment, storage medium and program product

Abstract

The invention discloses a large language model training method, electronic equipment, a storage medium and a program product, and relates to the technical field of computers. Comprising the following steps: inputting a target training sample in a training sample set into a to-be-trained large language model to obtain a sample result; training samples in the training sample set comprise private domain corpus samples and general corpus samples, the private domain corpus samples are corpus samples associated with to-be-adapted tasks of the to-be-trained large language model, and the general corpus samples are corpus samples associated with adapted tasks of the to-be-trained large language model; determining a sample loss value of the sample result based on a target loss function group corresponding to the result type of the sample result; based on the sample loss value, performing back propagation to update model parameters of the to-be-trained large language model until a training stop condition is reached, and obtaining a target large language model; the target large language model is aligned with the to-be-adapted task and the adapted task. According to the method, the effect of the large language model in practical application can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a training method for a large language model, an electronic device, a storage medium, and a program product. Background Technology

[0002] In practical applications, companies or specific fields often face the problem of limited computing power and corpus resources, making it difficult to train a completely new large language model with their own resources. To solve this problem, the usual approach is to build upon an open-source large language model and further train it using specialized task corpora (i.e., private domain corpora) to adapt it to specific needs.

[0003] Currently, during the training process based on open-source large language models, a "catastrophic forgetting" phenomenon occurs. This means that when the large language model learns new tasks or adapts to new data distributions, it will significantly forget the knowledge and abilities it has learned before and will not be able to effectively maintain its response to old task data, resulting in poor performance of the large language model in practical applications. Summary of the Invention

[0004] This application provides a training method, electronic device, storage medium, and program product for a large language model, aiming to improve the effectiveness of large language models in practical applications.

[0005] A first aspect of this application provides a method for training a large language model, comprising: inputting target training samples from a training sample set into a large language model to be trained to obtain sample results of the target training samples; wherein the training samples in the training sample set include private domain corpus samples and general corpus samples, the private domain corpus samples are associated with the task to be adapted by the large language model to be trained, the general corpus samples are associated with the adapted task of the large language model to be trained, and the result type of the sample results includes at least one of a first sample result and a second sample result, the first sample result being an output result based on the private domain corpus samples, and the second sample result being an output result based on the general corpus samples; determining the sample loss value of the sample results based on a target loss function group corresponding to the result type of the sample results; the weights for the same loss function in the loss function group corresponding to different result types are different; updating the model parameters of the large language model to be trained by backpropagation based on the sample loss value until the training stopping condition is reached to obtain the target large language model; and aligning the target large language model with the task to be adapted and the adapted task.

[0006] A second aspect of this application provides an electronic device, comprising: at least one processor; at least one memory for storing at least one program; and, when the at least one program is executed by the at least one processor, implementing a training method for a large language model as provided in any of the above aspects.

[0007] A third aspect of the embodiments of this application provides a computer-readable storage medium storing computer-executable instructions for performing a training method for a large language model as provided in any of the above aspects.

[0008] A fourth aspect of the embodiments of this application provides a computer program product, including a computer program or computer instructions, the computer program or computer instructions being stored in a computer-readable storage medium, a processor of an electronic device reading the computer program or computer instructions from the computer-readable storage medium, and the processor executing the computer program or computer instructions to cause the electronic device to perform the training method for a large language model as provided in any of the above aspects.

[0009] In the large language model training method provided in this application, the training sample set includes private domain corpus samples and general corpus samples. Private domain corpus samples are associated with the task to be adapted, while general corpus samples are associated with the already adapted tasks. This allows the large language model to learn new task knowledge while simultaneously consolidating its capabilities for adapted tasks using general corpus samples. Furthermore, different loss function groups correspond to different result types, and the weights for the same loss function differ across these groups, allowing for a more accurate measurement of the sample loss values ​​for different types of training samples. Finally, the model parameters are updated through backpropagation based on the sample loss values ​​until the training stopping condition is met. In this way, both new task learning and old task consolidation are considered during training, effectively mitigating the "catastrophic forgetting" problem and aligning the target large language model with the tasks to be adapted and those already adapted, thereby improving the performance of the large language model in practical applications. Attached Figure Description

[0010] The accompanying drawings are used to provide a further understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.

[0011] Figure 1 This is a flowchart illustrating a training method for a large language model provided in an embodiment of this application; Figure 2 This is a schematic diagram of the training process of a large language model provided in an embodiment of this application; Figure 3 This is a schematic diagram of a training system for a large language model provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0013] It should be understood that in the description of the embodiments of this application, the use of terms such as "first" and "second" is only for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of technical features indicated, or implicitly indicating the order of the technical features indicated. "At least one" refers to one or more, and "more" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, the simultaneous existence of A and B, or the existence of B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any group of these items, including any group of singular or plural items. For example, at least one of a, b, and c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0014] To facilitate understanding of the solutions in the embodiments of this application, some contents involved in the embodiments of this application are described below: In the traditional training process of existing large language models, when training based on open-source large language models using private domain corpus samples, the use of a unified cross-entropy loss function to process all training samples leads to knowledge overwriting when learning new task data distributions. Specifically, the model parameters over-adapt to the features of the private domain corpus samples during optimization, weakening the already adapted task-related knowledge representations. This significantly reduces the large language model's responsiveness to older task data, thus affecting the overall functional integrity and output stability of the large language model in multi-task scenarios.

[0015] For example, in a corporate application scenario in the financial sector, an organization uses an open-source large language model to adapt to internal financial report generation tasks. The training sample set contains the organization's private financial corpus (such as financial statement analysis data). When trained using conventional methods, the large language model prioritizes fitting specific patterns of the private corpus during the iteration process, causing its ability to handle common-sense issues (such as basic mathematical operations or descriptions of historical events) in the general corpus to degrade. In actual operation, the large language model can generate report content that conforms to financial norms, but it produces logical errors or irrelevant responses to cross-domain queries proposed by users (such as calculating simple interest rates or explaining economic terms), thereby leading to a decrease in the reliability of the system's output results.

[0016] If the above issues are not addressed, large language models will be unable to retain knowledge of old tasks while continuously adapting to new tasks. The continuous weakening of their general capabilities will cause large language models to lose their adaptability to multi-task processing in actual deployment environments. The resulting functional limitations not only restrict the application scope of large language models in complex business scenarios, but may also affect the overall service continuity of the system due to the failure of critical task response, ultimately hindering the effective deployment of large language models under resource-constrained conditions.

[0017] Furthermore, to alleviate the knowledge overlay phenomenon, the training process needs to separate the optimization objectives for different corpus types. Specifically, private domain corpus samples correspond to knowledge acquisition for the task to be adapted, while general corpus samples correspond to knowledge retention for already adapted tasks. This allows for the synergistic optimization of capabilities for both new and old tasks through differentiated loss calculation. In detail, the large language model needs to generate sample results for both types of training samples separately and independently determine the sample loss value, then guide parameter updates based on dual-path loss. As a preferred implementation method, this approach ensures that the large language model consolidates existing capabilities while absorbing new task knowledge.

[0018] Based on this, embodiments of this application provide a training method, electronic device, storage medium, and program product for a large language model, which can improve the effectiveness of the large language model in practical applications.

[0019] The large language model training method provided in this application can be applied to different electronic devices. Specifically, the method can be executed by an electronic device, or by components of the electronic device, such as its processor, chip, or chip system. It can also be implemented by logic modules or software that implement all or part of the functions of the electronic device. In practical applications, electronic devices can be terminals, servers, service platforms, cloud platforms, distributed systems, Internet of Things (IoT) systems, vehicle-to-everything (V2X) systems, etc. Furthermore, terminals can be smartphones, tablets, laptops, desktop computers, etc.

[0020] The large language model training method provided in this application is applicable to various scenarios requiring accurate adaptation of the large language model to different tasks, ensuring the accuracy, effectiveness, and practicality of the output results. Large language models are widely needed in numerous business scenarios. For example, in the field of intelligent customer service, enterprises expect large language models to accurately answer private domain questions related to specific customer businesses, such as product usage details and after-sales policies, while also handling general inquiries, such as frequently asked questions and basic industry information queries. If the large language model is poorly trained, inaccurate answers to private domain questions will prevent customers from receiving effective assistance, reducing customer satisfaction; unprofessional answers to general questions will also damage the company's image. In the field of intelligent writing assistance, large language models need to be able to generate professional and accurate text content based on private domain corpora in specific fields, such as medical literature and legal provisions, while also using general corpora, such as daily news and general literary works, to output text that conforms to popular language habits and expression norms. If the large language model cannot adapt well to private and general tasks, the generated text will either lack professionalism or lack universality and readability, failing to meet the diverse writing needs of users. In the field of intelligent education, large language models should be able to provide precise teaching guidance for specific subjects and grade levels using private domain teaching materials, such as mathematical formula derivations and historical event analyses. They should also be able to offer comprehensive learning suggestions based on general educational corpora, such as educational philosophy explanations and learning method introductions. If the training effect of the large language model is not ideal and it cannot simultaneously address both private and general tasks, it will affect the quality of teaching and hinder students' comprehensive knowledge acquisition and effective skill enhancement.

[0021] The training method for the large language model provided in this application will be described in detail below with reference to the accompanying drawings, through some embodiments and application scenarios.

[0022] like Figure 1 As shown, this application provides a flowchart of a training method for a large language model, which may include the following steps S110 to S130: S110, Input the target training samples from the training sample set into the large language model to be trained to obtain the sample results of the target training samples; wherein, the training samples in the training sample set include private domain corpus samples and general corpus samples, the private domain corpus samples are associated with the task to be adapted by the large language model to be trained, the general corpus samples are associated with the adapted task of the large language model to be trained, and the result type of the sample results includes at least one of the first sample results and the second sample results, the first sample results are the output results based on the private domain corpus samples, and the second sample results are the output results based on the general corpus samples.

[0023] In this step, the training sample set is the collection of all training samples used to train the large language model to be trained. It includes private domain corpus samples and general corpus samples, providing a data foundation for model learning. The target training sample refers to the specific training sample selected from the training sample set and input into the large language model to be trained for the current round of training.

[0024] Private domain corpus samples refer to corpus samples closely related to the task to be adapted to the large language model being trained. These samples focus on specific domains, specific businesses, or specific needs. For example, in the medical field, corpus is used to train a large language model to answer medical professional questions, or in the financial field, corpus is used to train a large language model to perform risk assessment analysis. By learning from private domain corpus samples, the large language model can acquire professional knowledge and skills related to the task to be adapted to, thereby better performing the specific task.

[0025] General corpus samples refer to corpus samples associated with the adapted tasks of the large language model to be trained. Adapted tasks are those that the large language model has already been able to handle well during previous training or application. General corpus samples are broad and universal, covering common language phenomena, basic knowledge, and general expressions, such as daily conversations and general news reports. Learning from general corpus samples helps the large language model maintain its memory of acquired knowledge and abilities, avoiding the "catastrophic forgetting" phenomenon when training new tasks.

[0026] The first sample result refers to the output obtained by the large language model under training after processing and outputting samples from the private domain corpus. It reflects the large language model's understanding and ability to handle the private domain corpus, and demonstrates the model's initial performance on the task to be adapted. For example, if the private domain corpus is text about the interpretation of legal provisions, the first sample result is the interpretation given by the large language model after analyzing these texts.

[0027] The second sample result refers to the output obtained by the large language model under training after processing and outputting samples from a general corpus. It demonstrates the large language model's processing level on the general corpus and reflects its performance capabilities in the adapted task. For example, if the general corpus consists of everyday dialogue texts, the second sample result is the large language model's response to these dialogues.

[0028] Specifically, the first step is to collect and organize a training sample set that includes both private domain corpus samples and general corpus samples. The private domain corpus samples focus on the tasks to be adapted to the large language model being trained, such as answering medical questions in the medical field or conducting risk assessment analysis in the financial field. The general corpus samples are associated with already adapted tasks, covering common language phenomena such as daily conversations and general news reports. Target training samples are selected from this integrated training sample set and input into the large language model being trained. The large language model then processes and outputs the selected private domain corpus samples and general corpus samples respectively, thus obtaining a first sample result based on the private domain corpus samples and a second sample result based on the general corpus samples.

[0029] Among them, the large language model has the ability to process and generate language. When inputted with different types of corpus samples, it uses its existing language knowledge and pattern recognition capabilities to analyze and interpret the samples. Private domain corpus samples allow the large language model to access information from specific domains or tasks, enabling it to learn relevant knowledge and skills to adapt to the task to be adapted; general corpus samples allow the large language model to consolidate its existing language processing capabilities and maintain familiarity with the adapted tasks. In this way, the large language model can simultaneously acquire information from both private and general corpora, providing a foundation for subsequent loss calculation and model optimization.

[0030] S120, Based on the target loss function set corresponding to the result type of the sample result, determine the sample loss value of the sample result; the weights for the same loss function are different in the loss function set corresponding to different result types.

[0031] In this step, the target loss function set is used to characterize the loss function set determined according to the result type of the sample results. The loss function set includes at least one loss function. The loss functions included in the loss function set corresponding to different result types can be different. For example, for the first sample result of the private domain corpus, which is associated with the task to be adapted by the large language model to be trained, only the first loss function (e.g., the cross-entropy loss function) used to measure the difference between the model's predicted result and the true label can be selected to form the corresponding loss function set. For the second sample result of the general corpus, which is associated with the adapted task of the large language model to be trained, only the second loss function (e.g., the KL divergence regularization term) used to measure the difference between the model's output distribution during training and the original model's output distribution can be selected to form the corresponding loss function set.

[0032] Furthermore, the loss functions included in the loss function groups corresponding to different result types can be the same, but in this case, the weights of the same loss function in the loss function groups corresponding to different result types are different. For example, the first sample result of the private domain corpus sample has a loss function group that includes a first loss function and a second loss function, with the first loss function having a larger weight and the second loss function having a smaller weight; the second sample result of the private domain corpus sample also has a loss function group that includes a first loss function and a second loss function, with the first loss function having a smaller weight and the second loss function having a larger weight.

[0033] Specifically, based on the result type of the sample results, a corresponding set of target loss functions is matched, and each set contains at least one loss function. The composition of the loss function sets corresponding to different result types is diverse. It can be different combinations of loss functions; for example, the first sample result of the private domain corpus sample might use only the first loss function, while the second sample result of the general corpus sample might use only the second loss function. Alternatively, it can be the same loss function with different weights; for example, the loss function set corresponding to the first sample result of the private domain corpus sample might contain both the first and second loss functions, with the first loss function having a larger weight, while the loss function set corresponding to the second sample result might have a smaller weight for the first loss function and a larger weight for the second loss function. Finally, based on the determined set of target loss functions, the sample loss value of the sample results is calculated.

[0034] Different result types correspond to different loss function sets, or different result types correspond to different weights of the same loss function, which can more accurately measure the difference between the model output and the expectation. For private domain corpus samples, the focus is on optimizing the model's performance on the task to be adapted; for general corpus samples, the focus is on maintaining the model's ability to adapt to already adapted tasks. By reasonably setting the loss function and weights, the large language model to be trained can be guided to balance the learning of new tasks and the consolidation of old tasks during the training process, enabling the large language model to be trained to develop in a direction that better meets the needs.

[0035] S130: Based on the sample loss value, backpropagation is used to update the model parameters of the large language model to be trained until the training stopping condition is met, and the target large language model is obtained; the target large language model is aligned with the task to be adapted and the adapted task.

[0036] In this step, the target large language model refers to the large language model obtained after iteratively updating the model parameters of the large language model to be trained using sample loss values. This target large language model can not only adapt well to the task to be adapted and meet the needs of specific domains or businesses, but also maintain its ability to process already adapted tasks, achieving alignment with both the task to be adapted and the already adapted tasks. In practical applications, it can provide more accurate and reliable language processing results.

[0037] Specifically, based on the calculated sample loss value, the model parameters of the large language model to be trained are updated using the backpropagation algorithm. The backpropagation process calculates the gradient of the sample loss value with respect to each parameter along the model's network structure, from the output layer to the input layer. Then, the parameter values ​​are adjusted according to the gradient, and this process is repeated multiple times for iterations. During the iteration process, the training stopping condition is continuously monitored, such as reaching a preset number of training rounds, the sample loss value being less than a set threshold, or the model performance no longer improving. When the training stopping condition is met, training stops, and the target large language model is obtained.

[0038] The backpropagation process updates model parameters based on the gradient descent principle. By continuously adjusting the parameters, the model output is made as close to the expectation as possible, reducing sample loss. Multiple iterative updates allow the model to gradually learn knowledge from different types of samples, enabling it to adapt to the task at hand, meeting specific domain or business needs, while maintaining its ability to process already adapted tasks. This alignment with both the task at hand and the adapted tasks leads to more accurate and reliable language processing results in practical applications.

[0039] In implementing this large language model training method, the target training samples from the training sample set are first input into the large language model to be trained to generate sample results. The training sample set consists of private domain corpus samples and general corpus samples. The private domain corpus samples are associated with the task to be adapted to by the large language model, while the general corpus samples are associated with the already adapted task. The sample results are classified into first sample results and second sample results. The first sample result corresponds to the output based on the private domain corpus samples, and the second sample result corresponds to the output based on the general corpus samples. Further, based on the type of sample result, a matching target loss function group is selected to calculate the sample loss value. Within the loss function group corresponding to different result types, the weights for the same loss function are set differently. Subsequently, the model parameters are updated through a backpropagation mechanism based on the sample loss values ​​until the training stopping condition is met, ultimately obtaining a target large language model aligned with both the task to be adapted and the already adapted task.

[0040] For example, in a specific scenario adapted to the financial field, the task to be adapted is financial risk assessment, while the task already adapted is general text generation. The private domain corpus samples come from financial regulatory reports, and the general corpus samples come from public news corpora. When a financial regulatory report sample is input, the generated first sample result triggers an increase in the weight of the loss function related to the task to be adapted in the target loss function group, so as to strengthen the model's learning of financial professional content. When a public news sample is input, the generated second sample result triggers an increase in the weight of the loss function related to the task already adapted in the target loss function group, so as to consolidate the model's ability to process general text.

[0041] Therefore, this training method for the large language model effectively mitigates the phenomenon of knowledge forgetting from old tasks caused by learning new tasks during continued training by dynamically matching sample result types with loss function weights. This ensures that the large language model under limited resources can adapt to both the task to be adapted and maintain its processing capability for already adapted tasks, thereby improving the overall performance and stability of the model in practical applications. Specifically, this weight differentiation mechanism based on sample result types enables the model to adaptively balance the learning intensity of new and old tasks, avoiding the performance degradation of old tasks in traditional unified loss function training, and ultimately achieving synchronous alignment of the target large language model for both tasks.

[0042] like Figure 2 The diagram illustrates a training process for a Large Language Model (LLM). First, a mixed training sample set containing private domain and general domain samples is shuffled and fed into the LLM for forward inference, outputting the corresponding sample results. Then—in one embodiment—the private domain samples are enhanced with a cross-entropy loss function to improve their adaptability to new tasks, while the general domain samples maintain their performance on older tasks through KL divergence regularization. Finally, the total loss is calculated by fusing the two types of losses, and the model parameters of the LLM are iteratively updated via backpropagation. This allows the LLM to absorb private domain knowledge while consolidating its general capabilities, achieving dual alignment between the task to be adapted and the already adapted tasks.

[0043] In this embodiment, the training sample set includes private domain corpus samples and general corpus samples. Private domain corpus samples are associated with the task to be adapted, while general corpus samples are associated with already adapted tasks. This allows the large language model to learn new task knowledge during training while simultaneously consolidating its capabilities for already adapted tasks using general corpus samples. Furthermore, different loss function groups correspond to different result types, with varying weights for the same loss function across different result types. This allows for more accurate measurement of the sample loss values ​​for different types of training samples. Finally, the model parameters are updated via backpropagation based on the sample loss values ​​until the training stopping condition is met. In this way, both new task learning and old task consolidation are considered during training, effectively mitigating the "catastrophic forgetting" problem and aligning the target large language model with both the task to be adapted and the already adapted tasks, thereby improving the performance of the large language model in practical applications.

[0044] In some of the embodiments described above in this application, a training process using a set of loss functions to distinguish between private domain corpora and general corpora is proposed. However, if the dynamic adjustment mechanism of the loss function weights is unreasonable during this process, it can easily lead to the model focusing excessively on the task to be adapted while neglecting the already adapted task when training new task samples, or failing to effectively consolidate existing knowledge when training old task samples, thereby exacerbating the catastrophic forgetting phenomenon and making it difficult for the model to simultaneously achieve a performance balance between new and old tasks.

[0045] In this regard, this application further proposes that the loss function set includes a first loss function and a second loss function. The first loss function is used to constrain the alignment of the large language model to be trained with the task to be adapted, and the second loss function is used to constrain the alignment of the large language model to be trained with the adapted task; S120 includes: Based on the result type of the sample result, determine the first loss weight corresponding to the first loss function and the second loss weight corresponding to the second loss function in the target loss function group; both the first loss weight and the second loss weight are non-negative and are accumulated to one; when the result type indicates that the sample result is the first sample result, the first loss weight is greater than the second loss weight; when the result type indicates that the sample result is the second sample result, the second loss weight is greater than the first loss weight. The sample loss value of the sample result is determined based on the target loss function set, the first loss weight, and the second loss weight.

[0046] In this embodiment, the loss function set refers to a constraint system composed of multiple loss functions, which can be implemented using a combination of a first loss function, a second loss function, or a task-specific loss function. The purpose is to establish a differentiated task learning objective framework. The first loss function is used to constrain the alignment of the large language model to be trained with the task to be adapted. It can be implemented using loss functions with "alignment with the standard answer" functions, such as cross-entropy loss function and mean squared error loss function. Its purpose is to directly measure the difference between the model output and the target of the task to be adapted, ensuring the effectiveness of training on the new task and enabling the model output to align with the provided standard. The second loss function can be understood as a constraint function used to constrain the large language model to be trained. The loss function that aligns the language model with the adapted task can be implemented using loss functions that have the function of "aligning with the model itself", such as the KL divergence regularization term. Its purpose is to evaluate the degree of matching between the model output and the adapted task objective and to prevent the forgetting of old task knowledge. The first loss weight can be understood as a dynamic adjustment coefficient assigned to the learning of new tasks. Specifically, it can be determined through a threshold comparison mechanism triggered by sample type. Its purpose is to adaptively adjust the intensity of new knowledge absorption according to the current sample attributes. The second loss weight refers to the adjustment parameter used to balance the retention of old task knowledge. Specifically, it can be generated through a normalization algorithm that complements the first loss weight. Its purpose is to ensure that historical knowledge is not overwritten in parameter updates.

[0047] Specifically, the proposed solution establishes a weight decision mechanism by identifying the result type of the sample results. When the system receives the sample results of the target training sample, it first parses the result type attribute: if the result type indicates a first sample result (i.e., output based on private domain corpus samples), the first loss weight is automatically increased to the dominant level, and the model parameter update process will focus on strengthening the feature learning of the task to be adapted; if the result type indicates a second sample result (i.e., output based on general corpus samples), the second loss weight is increased accordingly, so that the parameter update focuses on the knowledge consolidation of the adapted task. This mechanism ensures the precise ratio of the learning intensity of the new and old tasks in each sample processing process by forcing the first and second loss weights to be non-negative and summed to one, enabling the model to dynamically switch the task learning focus in a single iteration, thereby constructing an optimized path for the co-evolution of new and old knowledge at the micro-training level.

[0048] As a specific implementation method, the solution of this application is implemented as follows: In the training scenario of a large language model in the financial field, when a bank credit policy document (private domain corpus sample) is input to generate the first sample result, the system identifies the result type and sets the first loss weight to 0.65 and the second loss weight to 0.35. At this time, the model prioritizes optimizing the professional expression ability of the credit task. When a general news corpus is input to generate the second sample result, the system automatically adjusts the second loss weight to 0.7 and the first loss weight is correspondingly reduced to 0.3, so that the model maintains its general language ability while avoiding interference from the new task. In this embodiment, the switching of loss weights is automatically triggered based on the sample result type, and the real-time shift of the training focus can be achieved without manual intervention.

[0049] Through the above technical solution, this application establishes a sample type-driven dynamic weight allocation mechanism during model training, enabling the large language model to be trained to adaptively adjust its learning strategy according to the task attributes of the current sample being processed. This strengthens the adaptability to new tasks while effectively suppressing the forgetting of historical knowledge, thereby significantly improving the model's knowledge retention ability in multi-task scenarios and achieving synergistic optimization of the performance of new and old tasks.

[0050] In some of the embodiments described above in this application, an accumulation constraint of loss weights is proposed to ensure the global balance between learning new and old tasks. However, in its implementation, if the weights are set independently for only a single sample result, and there is a lack of a complementary mechanism between weights of different sample results, the model may over-focus on new tasks during training, weakening knowledge of old tasks and exacerbating the catastrophic forgetting problem.

[0051] In this regard, this application further proposes that the loss weights for the first loss function in the loss function group corresponding to the first sample result and the loss weights for the first loss function in the loss function group corresponding to the second sample result are summed to one. The loss weights for the second loss function in the loss function group corresponding to the first sample result are summed with the loss weights for the second loss function in the loss function group corresponding to the second sample result, and the sum is one.

[0052] In this embodiment, the constraint that the weights of the first loss function are summed to one means that the sum of the weights corresponding to the first sample result and the second sample result is 1. This can be achieved using a preset fixed value. The purpose is to ensure that while the model strengthens the learning of new tasks when processing private domain corpora, it automatically reduces the weights of new tasks when training with general corpora. Similarly, the constraint that the weights of the second loss function are summed to one means that the sum of the weights corresponding to the first sample result and the second sample result is 1. This can also be achieved using a preset fixed value. The purpose is to enable the model to strengthen the old task constraints in training with general corpora when it weakens the old task constraints in training with private domain corpora.

[0053] Specifically, the proposed scheme establishes a complementary constraint relationship between loss weights for different sample results. When the model processes private domain corpus samples to obtain the first sample result, the loss weight for the first loss function is larger to enhance learning for the new task, while the loss weight for the second loss function is smaller to allow the model to adapt to the new task. Conversely, when the model processes general corpus samples to obtain the second sample result, the loss weight for the first loss function is smaller to avoid overemphasizing the new task, while the loss weight for the second loss function is larger to enhance knowledge retention for the old task. This design dynamically adjusts the weight distribution throughout the overall training process, allowing model parameter updates to naturally integrate with the dual objectives of task adaptability and knowledge retention, thereby effectively alleviating the problem of imbalance between learning new and old tasks.

[0054] As a specific implementation method, the scheme of this application is implemented as follows: During the training process, when inputting private domain corpus samples, the loss weight for the first loss function is set to be relatively large, and the loss weight for the second loss function is set to be relatively small; when inputting general corpus samples, the loss weight for the first loss function is set to be relatively small, and the loss weight for the second loss function is set to be relatively large, wherein the weight for the same loss function is cumulatively equal to one between private domain samples and general samples.

[0055] Through the above technical solutions, this application effectively reduces the forgetting of adapted task knowledge by the model during training, and improves the stability and generalization ability of the model in multi-task scenarios.

[0056] In some of the embodiments described above in this application, a first preset weight coefficient or a second preset weight coefficient is proposed to perform a weighted summation of the sub-loss values. However, in its implementation, if the weight coefficient is a fixed value set in advance and cannot be adaptively adjusted according to the changes in the loss value during training, it may lead to the model focusing too much on new tasks and ignoring old tasks during training, or conversely, exacerbating the catastrophic forgetting phenomenon and affecting the model's ability to retain adapted tasks.

[0057] In this regard, this application further proposes that S120 includes: The target loss value of the sample result is compared with the preset loss threshold to obtain a loss comparison result indicating the relationship between the target loss value and the preset loss threshold. The target loss value is the loss value of the sample result calculated based on the target loss function. The target loss function is any loss function in the target loss function group. The preset loss threshold corresponds to the target loss function. Based on the loss comparison results, determine the loss weights corresponding to the target loss function; Based on the loss weights corresponding to the target loss function, the loss weights corresponding to the auxiliary loss function are determined; the auxiliary loss function is the loss function other than the target loss function in the loss function group, and the loss weights corresponding to the target loss function and the auxiliary loss function are summed to one. The sample loss value of the sample result is determined based on the target loss function set, the loss weight corresponding to the target loss function, and the loss weight corresponding to the auxiliary loss function.

[0058] In this embodiment, the loss comparison result refers to the indicative output generated by comparing the target loss value with a preset loss threshold. This can be implemented using a numerical comparison circuit or software logic module. Its purpose is to quantify the deviation between the current training state and the expected threshold in real time, providing an objective basis for dynamic weight adjustment. The determination of the loss weight can be understood as a mechanism for adaptive adjustment based on the loss comparison result. This can be implemented using a rule-based weight mapping table or a real-time optimization algorithm. Its purpose is to strengthen the learning intensity of the model when training weak tasks and suppress overfitting tendencies when performing well. Specifically, the auxiliary loss weight refers to the complementary weight value derived from the target loss weight. This can be implemented using a weight normalization method or linear complementary calculation. Its purpose is to ensure that the total weight sum of the loss function group is always one, maintaining the numerical stability of the training process. In practical applications, the determination of the sample loss value refers to the process of calculating the final loss by comprehensively considering the dynamic weights. This can be implemented using a weighted summation or nonlinear combination function. Its purpose is to ensure that the backpropagation update accurately reflects the current training needs and optimizes the direction of model parameter adjustment.

[0059] Specifically, the proposed solution compares the target loss value of the sample results with a preset loss threshold to generate a loss comparison result for real-time evaluation of the training status. Based on this comparison result, the loss weights corresponding to the target loss function are dynamically determined, thereby adaptively adjusting the task focus according to the actual loss level. Subsequently, auxiliary loss weights are automatically derived based on the target loss weights, ensuring that the weights sum to one, thus ensuring the overall coordination of the loss function group. Finally, the sample loss value is calculated based on the dynamic weight combination, enabling the model parameter updates to respond specifically to the learning needs of different tasks, forming a closed-loop feedback mechanism, effectively balancing the training intensity of new and old tasks, and avoiding the training rigidity problem caused by fixed weights.

[0060] As one embodiment, the specific implementation of the solution in this application is as follows: When training a large language model to adapt to financial domain tasks, when the sample result is the first sample result based on private domain corpus samples, the target loss function is selected as the first loss function, and the preset loss threshold is set as a reference value that matches the first loss function; if the target loss value is greater than the preset loss threshold, it indicates that the model is not performing well on the new task, and the first loss weight can be increased to 0.8, and the second loss weight can be reduced to 0.2 accordingly; if the target loss value is less than the preset loss threshold, it indicates that the model may be overfitting the new task, and the first loss weight can be reduced to 0.3, and the second loss weight can be increased to 0.7; the sample loss value is calculated based on the adjusted dynamic weights and used for backpropagation to update the model parameters.

[0061] Through the above approach, this application enables large language models to dynamically adjust the loss weight allocation based on the actual loss value during training, avoiding overfitting or underfitting problems caused by fixed weight mechanisms, and effectively improving the model's ability to balance new and old tasks. This significantly alleviates catastrophic forgetting and ensures that the model maintains performance stability on already adapted tasks while adapting to new ones.

[0062] In some of the embodiments described above in this application, a loss weight corresponding to the target loss function is determined based on the loss comparison results. However, in the implementation process, if the method of determining the loss weight based on the loss comparison results to optimize the calculation of the sample loss value does not fully consider the degree of difference between the loss value and the threshold, it will be impossible to achieve fine adjustment of the weight when the loss value deviates significantly from the threshold. This will affect the model's ability to retain knowledge of the old task when adapting to the new task and exacerbate the catastrophic forgetting problem.

[0063] In this regard, this application further proposes to determine the loss weights corresponding to the target loss function based on the loss comparison results, including: In response to the loss comparison result indicating that the target loss value is equal to the preset loss threshold, the loss weight corresponding to the target loss function is determined as the preset weight value; In response to the loss comparison result indicating that the target loss value is greater than the preset loss threshold, the loss weight corresponding to the target loss function is determined as the sum of the preset weight value and the first correction value; the first correction value is determined based on the difference between the target loss value and the preset loss threshold. In response to the loss comparison result indicating that the target loss value is less than the preset loss threshold, the loss weight corresponding to the target loss function is determined as the difference between the preset weight value and the second correction value; the second correction value is determined based on the difference between the preset loss threshold and the target loss value.

[0064] In this embodiment, the loss comparison result refers to the state indication signal generated after comparing the target loss value with the preset loss threshold. It can be implemented using a numerical comparison circuit or software logic module, with the aim of reflecting the degree of matching between the current training state and the expected benchmark in real time. The preset loss threshold can be understood as a reference value output by the loss function pre-set according to the task characteristics. It can be dynamically adjusted based on the statistical distribution of historical training data, with the aim of providing a stable benchmark for weight calculation. The preset weight value refers to the basic weight parameter used when the loss value is within the normal range. It can be initialized by the task importance ratio, with the aim of maintaining the benchmark rhythm of model training. The first correction value and the second correction value are adjustment amounts dynamically generated based on the loss deviation. They can be calculated according to the difference using a linear proportional function or a nonlinear mapping function, with the aim of achieving a precise match between the weight adjustment magnitude and the severity of the problem.

[0065] Specifically, the proposed solution triggers the execution of different conditional branches based on the loss comparison results: when the target loss value equals a preset loss threshold, a preset weight value is used to ensure training stability; when the target loss value is greater than the preset loss threshold, the loss weight is increased by accumulating a first correction value based on the difference, enhancing the model's ability to correct problem regions; when the target loss value is less than the preset loss threshold, the loss weight is reduced through difference operations to prevent the model from over-focusing on the current task and weakening the learning of other tasks. This mechanism establishes a positive correlation between the adjustment magnitude of the loss weight and the degree of loss bias, thereby avoiding training oscillations caused by sudden weight changes during the dynamic balancing of the learning intensity of new and old tasks, and achieving continuous optimization of the model's knowledge retention capability.

[0066] As a specific embodiment, the solution of this application is implemented as follows: When the target loss function is the first loss function, the preset loss threshold is set to a moderate level according to the task data distribution; if the sample loss value is equal to the preset loss threshold, the preset weight value is directly used for parameter update; if the sample loss value is significantly higher than the preset loss threshold, the system automatically calculates the difference and generates a positive correction value to increase the loss weight to a reasonable range to strengthen error correction; if the sample loss value is significantly lower than the preset loss threshold, the system generates a negative correction value to reduce the loss weight, ensuring that the model does not overfit the current task and sacrifice the performance of historical tasks. This implementation method, by dynamically responding to changes in loss state, enables the model to effectively retain knowledge of old tasks while adapting to new tasks.

[0067] Through the above scheme, this application achieves refined adaptive adjustment of loss weights, enabling the large language model to intelligently allocate learning resources for new and old tasks according to the current learning state during training, significantly alleviating the catastrophic forgetting phenomenon and improving the model's comprehensive alignment ability for adapted and unadapted tasks.

[0068] In some of the embodiments described above in this application, a method for training a large language model based on private domain corpora and general corpora is proposed. However, in its implementation, due to the limitation of device capacity, it may not be possible to process all corpus samples at once; and if only one corpus sample is processed at a time, the training efficiency of the large language model is low.

[0069] In this regard, this application further proposes that the training sample set includes multiple training sample batches, and each training sample batch includes a first preset number of randomly selected training samples; S130 includes: Based on the sample loss value of each sample result in each training sample batch, determine the batch loss value of each training sample batch. The first comprehensive loss value is determined based on the batch loss values ​​of a second preset number of consecutive training sample batches. Based on the first comprehensive loss value, backpropagation is used to update the model parameters of the large language model to be trained until the training stopping condition is met, thus obtaining the target large language model.

[0070] In this embodiment, a training sample batch refers to the organization of the training sample set into multiple subsets. A random sampling algorithm can be used to ensure a balanced distribution of private and general corpus samples within each subset, aiming to avoid excessive bias from a single sample influencing the training direction. The first preset quantity refers to the number of training samples included in each training sample batch. For example, assuming a total of 10,000 corpus samples are used, and eight graphics cards are used during training, with each graphics card processing two corpus samples at a time, the total number of corpus samples involved in loss calculation and backpropagation in each batch is the sum of the number of corpus samples processed by each of the eight graphics cards, i.e., 8 multiplied by 2 equals 16 corpus samples. In other words, each training sample batch can process 16 corpus samples simultaneously. To ensure that all 10,000 corpora participate in a single training run, 10,000 divided by 16 rounds of batch training are required. The batch loss value is an aggregated metric calculated based on the sample loss values ​​of all samples in a training sample batch. It can be achieved using methods such as arithmetic mean, weighted average, mode, and median. The first comprehensive loss value is a comprehensive metric calculated based on the batch loss values ​​of multiple consecutive training sample batches. It can be achieved using methods such as arithmetic mean, weighted average, mode, and median. The second preset quantity refers to the number of consecutive training sample batches used to calculate the first comprehensive loss value, which can be set to an integer value such as 5 or 10.

[0071] In one specific implementation, the first preset number of training sample batches can be set to 16, meaning each batch contains 16 randomly selected training samples; the second preset number can be set to 5, meaning the first comprehensive loss value is calculated based on the batch loss values ​​of the most recent 5 consecutive training sample batches. During training, the system first calculates the arithmetic mean of the sample loss values ​​of each sample result in each training sample batch as the batch loss value; then, it performs a sliding window average on the batch loss values ​​of the most recent 5 consecutive training sample batches to obtain the first comprehensive loss value; finally, it performs a backpropagation operation based on the first comprehensive loss value to update the model parameters of the large language model to be trained until the training stopping condition is met.

[0072] By using the above technical solution, the training sample set is divided into multiple batches, each batch containing a first preset number of randomly selected samples, which makes full use of device capacity resources. This improves training efficiency and enables more reasonable and effective model training under memory constraints.

[0073] In some embodiments described above in this application, a method is proposed to train a large language model by independently inputting private domain corpus samples and general corpus samples into the model and calculating the loss separately. However, in its implementation, due to the lack of semantic correlation between private domain corpus samples and general corpus samples, the large language model to be trained has difficulty effectively maintaining knowledge retention of old tasks when adapting to new tasks, leading to an exacerbation of catastrophic forgetting. Specifically, when learning the task to be adapted, the large language model to be trained over-focuses on the private domain corpus while ignoring the contextual consistency of the general corpus, thereby weakening its responsiveness to the adapted task.

[0074] In this regard, this application further proposes that the training samples in the training sample set include private domain corpus samples and general corpus samples constructed from the same target corpus information; The target corpus information is corpus information that has a complete semantic structure and can independently express a complete meaning.

[0075] In this embodiment, the target corpus information refers to corpus information that has a complete semantic structure and can independently express a complete meaning. It can be implemented in the form of complete dialogue records, independent paragraphs or question-and-answer pairs, etc. The purpose is to ensure the contextual coherence and information integrity of the corpus, and to provide a high-quality semantic foundation for the large language model to be trained. The first corpus information constituting the private domain corpus sample and the second corpus information constituting the general corpus sample in the target corpus information refer to two different parts of the corpus information. They can be implemented in parallel, such as the question part and answer part in the target corpus information, or professional domain content and general background description. The purpose is to ensure that the private domain corpus shares background knowledge when it is for the task to be adapted and the general corpus is for the adapted task. Furthermore, the private domain corpus samples and general corpus samples belonging to the same target corpus information can also be further formed into sample pairs. They can be implemented in a generalized way, such as the user input part and model response part in the same dialogue, or the professional terminology explanation and general definition in the same article. The purpose is to force the large language model to be trained to process semantically related corpora simultaneously in a single forward propagation.

[0076] As an example, in a medical adaptation task, the target corpus information could be a complete doctor-patient dialogue record. This dialogue record possesses a complete semantic structure and can independently express the diagnosis and treatment process. The description of the patient's detailed explanation of their symptoms to the doctor within this dialogue record is constructed as the first corpus information and serves as a private domain corpus sample. This private domain corpus sample focuses on the key information required for disease diagnosis tasks, enabling the model to learn how to make accurate diagnoses based on symptoms. Simultaneously, the basic health consultation background information mentioned by the doctor before responding to the patient, such as common disease prevention knowledge and general knowledge of human physiology, is constructed as the second corpus information and serves as a general corpus sample. This general corpus is used to maintain the model's ability to respond to general medical knowledge, ensuring that the model has a broad foundation of medical common sense.

[0077] The training samples consist of symptom descriptions and health consultation backgrounds from the same dialogue record. After being input into the large language model to be trained, when generating the first sample result for the symptom description (i.e., the model's response to the patient's symptom diagnosis), the model can refer to the contextual information of the health consultation background, avoiding over-focusing on professional symptoms while ignoring general medical knowledge. For example, it will not give one-sided or unreasonable diagnostic suggestions based solely on symptoms, but will comprehensively consider the overall health status. When generating the second sample result for the health consultation background (i.e., the model's further expanded response to basic health consultation), it can further enhance the accuracy of responses to general tasks by leveraging the relevance of the symptom descriptions. For example, it can provide more targeted explanations and supplements to the basic health consultation content based on the patient's specific symptoms.

[0078] Finally, the second loss function is applied to calculate the loss value for the basic health consultation background information (i.e., the second corpus information) in the dialogue record; the first loss function is applied to calculate the loss value for the patient symptom description information (i.e., the first corpus information) in the dialogue record.

[0079] In this embodiment, private and general corpus samples are constructed using the same target corpus information and paired as training samples. These are then input into the large language model to be trained, allowing both to share background knowledge. When the model processes tasks, it is forced to process semantically related corpora simultaneously during a single forward propagation. This allows the large language model to refer to the context of the general corpus samples when generating results for private corpus samples, avoiding one-sided processing. When generating results for general corpus samples, the relevance of the private corpus samples can be utilized to improve accuracy. Furthermore, different loss functions are used to calculate loss values ​​for different corpus samples, enabling targeted optimization of the model's ability to process different corpus samples. In this way, the large language model to be trained focuses on key information for new tasks while maintaining responsiveness for older tasks. Without performance degradation, it effectively internalizes reference documents and external feedback-type knowledge, similar to how reference documents provide comprehensive knowledge and external feedback guides optimization, thereby enhancing overall performance and adaptability.

[0080] In some of the embodiments described above in this application, monitoring of the target feature extraction layer and local loss calculation are proposed to reduce catastrophic forgetting. However, in the implementation process, the changes in features within the model are not effectively constrained, which causes the knowledge adapted to the task to be overwritten when training new tasks.

[0081] In this regard, this application further proposes that the training method for this large language model also includes: Obtain the target feature extraction layer that matches the large language model to be trained and the original large language model; the original large language model is the model of the large language model to be trained before it was trained on the task to be adapted. The training samples are input into the target feature extraction layer of the large language model to be trained and the target feature extraction layer of the original large language model respectively to obtain the first sample features and the second sample features. Based on the features of the first sample and the features of the second sample, the local loss value of the training sample is determined by the second loss function; the second loss function is used to constrain the large language model to be trained to align with the adapted task. The second comprehensive loss value is determined based on the sample loss value and the local loss value; Based on the second comprehensive loss value, backpropagation is used to update the model parameters of the large language model to be trained until the training stopping condition is met, thus obtaining the target large language model.

[0082] In this embodiment, the target feature extraction layer refers to the neural network layer in the large language model responsible for extracting the internal representation of the input corpus. It can be implemented using a self-attention layer or a feedforward neural network layer in the Transformer architecture. Its purpose is to ensure that the large language model focuses on the core parts that may affect knowledge retention during the update process. The first sample feature refers to the feature vector output by the large language model to be trained at the target feature extraction layer. It can be represented by a high-dimensional vector, and its purpose is to reflect the internal state changes of the large language model during the adaptation to new tasks. The second sample feature refers to the feature vector output by the original large language model at the target feature extraction layer. It can be represented by a vector with the same dimension as the first sample feature. The purpose is to provide a baseline feature representation of the large language model before it is trained on a new task; the second loss function is a loss function used to calculate feature differences, which can be implemented using the second loss function, and its purpose is to constrain the large language model to be trained to align with the adapted task; the local loss value is the loss value calculated based on feature differences, which can be obtained by comparing the first sample features and the second sample features using the second loss function, and its purpose is to quantify the knowledge retention of the large language model at the target feature extraction layer; the second comprehensive loss value is a weighted combination of the sample loss value and the local loss value, which can be implemented using a linear weighted sum, and its purpose is to form an optimization objective that balances the adaptation effect of the new task and the knowledge retention of the old task.

[0083] Specifically, the proposed solution ensures structural consistency of the benchmark by obtaining a target feature extraction layer that matches the original large language model and avoids invalid monitoring due to differences in model architecture. Then, identical training samples are input into the target feature extraction layers of both models to obtain first and second sample features, allowing the internal representation drift of the model during adaptation to the new task to be directly captured. Based on these features, a local loss value is calculated using a second loss function, forcing the model to maintain internal representation stability for the old task during updates. The local loss value is integrated with the sample loss value reflecting the adaptation effect to the new task into a second comprehensive loss value, forming an optimization objective that balances task adaptability and knowledge stability. Finally, the backpropagation process is guided by this second comprehensive loss value, ensuring that model parameter updates not only optimize the performance of the new task but also suppress the overwriting of knowledge from the old task, thereby effectively mitigating catastrophic forgetting.

[0084] As a specific implementation method, the scheme of this application is implemented as follows: the target feature extraction layer can be specified as an intermediate Transformer block in a large language model; after inputting training samples, the first sample feature and the second sample feature are both represented in the form of 768-dimensional vectors; the local loss value is determined by calculating the mean square error of the two vectors; the second comprehensive loss value is the weighted sum of the sample loss value and the local loss value, wherein the weights can be dynamically adjusted according to the training stage; based on this second comprehensive loss value, the model parameters are updated by backpropagation, so that the model can maintain its responsiveness to general corpus tasks while adapting to private domain corpus tasks.

[0085] This embodiment effectively constrains changes in the model's internal features, preventing the knowledge of already adapted tasks from being overwritten when training new tasks, thereby mitigating the catastrophic forgetting problem and ensuring that the large language model can maintain good performance on already adapted tasks while adapting to tasks to be adapted.

[0086] In some of the embodiments described above in this application, a set of loss functions based on the result type of the sample results is proposed to dynamically adjust the training weights of private domain corpora and general corpora. However, in its implementation, if the training samples do not carry clear type identification information, the system cannot accurately distinguish whether the samples belong to private domain corpora or general corpora at the input stage, resulting in a deviation in the allocation of loss weights. This exacerbates the risk of the model forgetting the adapted task and affects the stability of the model in multi-task scenarios.

[0087] In this regard, this application further proposes that the training samples in the training sample set carry sample type labels; The sample type label is used to indicate the type to which the training sample belongs, including private domain corpus samples or general corpus samples.

[0088] In this embodiment, the sample type label refers to the metadata identifier attached to the training sample. It can be implemented using binary flags, string identifiers, or enumeration types. Specifically, it can be a pre-defined type field in the sample data structure. Its purpose is to provide the system with a clear indication of the sample category, avoiding the unreliability of inference based on implicit corpus sources, and ensuring that a reliable sample classification basis is established at the input stage. The attribution type is used to characterize the sample type to which the training sample belongs, that is, whether the training sample belongs to the private domain corpus sample or the general corpus sample.

[0089] Specifically, the proposed solution embeds explicit type labels before the training samples are input into the large language model to be trained. This allows the system to directly distinguish between private domain corpus samples and general corpus samples based on the sample type labels, thereby accurately matching the corresponding weights when determining the loss function set: when the sample type label indicates a private domain corpus sample, the system automatically assigns a higher first loss weight to strengthen the alignment of the task to be adapted; when it indicates a general corpus sample, a higher second loss weight is assigned to maintain the ability to adapt to the already adapted task. This label-based deterministic recognition mechanism effectively prevents mismatch of loss weights due to sample type confusion, ensuring that the model does not over-cover historical knowledge during the learning of new tasks, while maintaining adaptability to multi-task distributions.

[0090] As a specific implementation method, the solution of this application is implemented as follows: In practical application scenarios, the following operations can be performed to construct a training sample set carrying sample type labels. On the one hand, the user-specified general corpus set and private domain corpus set are read. The user will provide the storage folder paths corresponding to the general corpus and private domain corpus respectively. Based on these paths, all corpus files under the folder are read, and each text content is bound to the corresponding domain label "general" or "private domain", thereby constructing a training sample set carrying sample type labels. For example, the text content of general corpus A corresponds to the "general" label, and the text content of private domain corpus B corresponds to the "private domain" label, etc.

[0091] It's important to note that the key to this approach lies in adding supplementary information as sample type labels to the samples. These labels are replaceable and extensible. They are not limited to simple domain labels like "general" or "private domain," but can be extended to other types. For example, using 0 / 1 labels, where 0 represents general corpora and 1 represents private domain corpora, allows for a more concise sample classification.

[0092] Furthermore, more refined labeling can be applied to the same corpus. For example, different labels can be used for different parts of the same corpus information. The first half of the corpus, which involves professional domain knowledge, can be labeled with "private domain," while the second half, which is a general background description, can be labeled with "general." This guides the model's learning and improves its performance and adaptability in multi-task scenarios.

[0093] Through the above technical solution, this application solves the problem of mismatched loss weights caused by fuzzy sample type recognition, ensuring that the model effectively retains its responsiveness to the adapted task when adapting to new tasks, thereby maintaining the stability and adaptability of the large language model in multi-task scenarios.

[0094] It should be noted that the various embodiments described in this application can be combined with each other or implemented individually without conflict, and this application does not limit this.

[0095] To facilitate understanding, the training of the large language model in this application is illustrated below with a specific embodiment: like Figure 3 The diagram illustrates a training system for a large language model. Specifically, this training system achieves multi-task collaborative optimization through a modular architecture. Its core is a training framework that coordinates the entire process. First, it parses the user-input private corpus path and general corpus path, as well as the large language model to be trained, and then transmits the data to the corpus processing module.

[0096] This corpus processing module generates standardized input sequences with domain labels through domain label binding (used to mark whether training samples belong to private or general corpus samples), global random shuffling, and tokenization transformation. The shuffling operation ensures that each training batch mixes private and general corpus samples, avoiding category distribution bias; for example, a single batch might contain both medical private corpus and encyclopedic general corpus. The training framework divides batches according to GPU memory capacity, supporting immediate loss calculation for a single training sample or averaging the loss across multiple training samples to balance computational efficiency and gradient stability. The batched data is then input into the large language model to be trained for forward inference. Mapping text to a model-recognizable token sequence completes the standardized conversion from raw text to model input format. Specifically, the text portion of the corpus is converted into the model's input tokens, mapping the content of text segments to integer inputs that the model can process. For example, if the model's vocabulary is {Today: 0, Weather: 1, You: 2, Real: 3, Good: 4}, then the phrase "Today the weather is really good" will be tokenized as [0, 1, 3, 4].

[0097] The loss calculation module implements differentiated optimization strategies based on domain labels: for private domain corpus samples, cross-entropy loss or combined weighted loss (e.g., M-cross-entropy loss + NKL divergence, where M>N) is applied to force the output of the large language model to be trained to align with the standard answer and master the task to be adapted; for general corpus samples, KL divergence regularization or combined weighted loss (e.g., (1-M) cross-entropy loss + (1-N) KL divergence, where M>N) is applied to constrain the output of the large language model to be trained to be consistent with the distribution of the original large language model, preventing performance degradation of the adapted task. This loss calculation module also supports dynamic weight adjustment, optimizing the M / N weight ratio within a preset range. In addition, KL divergence regularization can be applied to specific layers of the model through intermediate target feature extraction layer alignment technology to ensure that the feature space is aligned with the original model, further consolidating the old task capabilities; or a corpus pair training mode can be adopted to construct associated sample pairs between private domain corpus samples and general corpus samples belonging to the same target corpus information, enhancing the domain knowledge transfer effect.

[0098] The backpropagation module automatically calculates parameter gradients based on the first comprehensive loss value using the chain rule, and iteratively updates model parameters through an optimizer. Its implementation is directly compatible with open-source frameworks such as PyTorch. The entire process forms a closed loop: the training framework schedules corpus processing to generate input; after forward inference, the loss calculation module calculates the loss according to domain labels; and the backpropagation module drives parameter optimization until the large language model converges. This design effectively solves the catastrophic forgetting problem in private domain adaptation through mechanisms such as hybrid batch training, dynamic weighted loss, and feature layer alignment. It ensures that the large language model maintains high performance on both new tasks (such as domain-specific question answering) and old tasks (such as general dialogue), making it particularly suitable for industry scenarios with limited computing power and requiring multi-task deployment, such as the development of customized medical or financial large-scale models.

[0099] Based on the training method for the large language model provided in the embodiments of this application, correspondingly, the embodiments of this application also provide an electronic device, such as... Figure 4 As shown, the electronic device 400 includes: One or more processors 410; The memory 420 stores one or more programs that, when executed by one or more processors 410, enable the one or more processors 410 to implement the training method for the large language model described in any of the above embodiments.

[0100] Memory 420, as a non-transitory network system, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory 420 may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 420 may optionally include remotely located memories 420 relative to processor 410, which can be connected to processor 410 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0101] The memory 420 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 420 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 420 and called by the processor 410 to execute the training method of the large language model of the embodiments of this application.

[0102] The processor 410 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0103] In some embodiments, the electronic device further includes: Input / output interfaces are used to implement information input and output; The communication interface is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). The bus transmits information between various components of the device (e.g., processor 410, memory 420, input / output interface, and communication interface); The processor 410, memory 420, input / output interface, and communication interface can communicate with each other within the device via a bus.

[0104] One embodiment of this application also provides a computer-readable storage medium storing computer-executable instructions for performing a training method for a large language model as provided in any embodiment of this application.

[0105] An embodiment of this application also provides a computer program product, including a computer program or computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer program or computer instructions from the computer-readable storage medium and executes the computer program or computer instructions, causing the computer device to perform a training method for a large language model as provided in any embodiment of this application.

[0106] The system architecture and application scenarios described in this application are intended to more clearly illustrate the technical solutions of this application and do not constitute a limitation on the technical solutions provided in this application. Those skilled in the art will understand that as system architectures evolve and new application scenarios emerge, the technical solutions provided in this application are also applicable to similar technical problems.

[0107] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0108] In hardware implementations, the division between functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0109] As used in this specification, the terms "component," "module," "system," etc., are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, or a computer. As illustrated, applications running on computing devices and computing devices can both be components. One or more components may reside in a process or execution thread, and components may be located on a single computer or distributed among two or more computers. Furthermore, these components can be executed from various computer-readable media on which various data structures are stored. Components can communicate, for example, via local or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, or a network, such as the Internet interacting with other systems via signals).

[0110] The above description, with reference to the accompanying drawings, illustrates some embodiments of this application, but does not limit the scope of this application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and spirit of this application shall be within the scope of this application.

Claims

1. A training method for a large language model, comprising: The target training samples in the training sample set are input into the large language model to be trained to obtain the sample results of the target training samples; wherein, the training samples in the training sample set include private domain corpus samples and general corpus samples, the private domain corpus samples are associated with the task to be adapted by the large language model to be trained, and the general corpus samples are associated with the adapted task of the large language model to be trained. The result type of the sample results includes at least one of the first sample results and the second sample results, wherein the first sample result is the output result based on the private domain corpus samples, and the second sample result is the output result based on the general corpus samples; Based on the target loss function set corresponding to the result type of the sample result, the sample loss value of the sample result is determined; the weights for the same loss function are different in the loss function set corresponding to different result types; Based on the sample loss value, the model parameters of the large language model to be trained are updated through backpropagation until the training stopping condition is met, thus obtaining the target large language model; the target large language model is aligned with the task to be adapted and the adapted task.

2. The method according to claim 1, characterized in that, The loss function group includes a first loss function and a second loss function. The first loss function is used to constrain the large language model to be trained to align with the task to be adapted, and the second loss function is used to constrain the large language model to be trained to align with the adapted task. Determining the sample loss value of the sample result based on the target loss function set corresponding to the result type of the sample result includes: Based on the result type of the sample result, a first loss weight corresponding to the first loss function and a second loss weight corresponding to the second loss function are determined in the target loss function group; both the first loss weight and the second loss weight are non-negative and are accumulated to one; when the result type indicates that the sample result is the first sample result, the first loss weight is greater than the second loss weight; when the result type indicates that the sample result is the second sample result, the second loss weight is greater than the first loss weight. Based on the target loss function set, the first loss weight, and the second loss weight, the sample loss value of the sample result is determined.

3. The method according to claim 2, characterized in that, The loss weights for the first loss function in the loss function group corresponding to the first sample result are summed with the loss weights for the first loss function in the loss function group corresponding to the second sample result to one. The loss weights for the second loss function in the loss function group corresponding to the first sample result are summed with the loss weights for the second loss function in the loss function group corresponding to the second sample result to form one.

4. The method according to claim 1, characterized in that, Determining the sample loss value of the sample result based on the target loss function set corresponding to the result type of the sample result includes: The target loss value of the sample result is compared with a preset loss threshold to obtain a loss comparison result indicating the magnitude relationship between the target loss value and the preset loss threshold; the target loss value is the loss value calculated by the sample result based on the target loss function, the target loss function is any loss function in the target loss function group, and the preset loss threshold corresponds to the target loss function; Based on the loss comparison results, determine the loss weights corresponding to the target loss function; Based on the loss weights corresponding to the target loss function, the loss weights corresponding to the auxiliary loss function are determined; the auxiliary loss function is a loss function other than the target loss function in the loss function group, and the loss weights corresponding to the target loss function and the auxiliary loss function are summed to one. The sample loss value of the sample result is determined based on the target loss function set, the loss weight corresponding to the target loss function, and the loss weight corresponding to the auxiliary loss function.

5. The method according to claim 4, characterized in that, The step of determining the loss weight corresponding to the target loss function based on the loss comparison result includes: In response to the loss comparison result indicating that the target loss value is equal to the preset loss threshold, the loss weight corresponding to the target loss function is determined as the preset weight value; In response to the loss comparison result indicating that the target loss value is greater than the preset loss threshold, the loss weight corresponding to the target loss function is determined as the sum of the preset weight value and the first correction value; the first correction value is determined based on the difference between the target loss value and the preset loss threshold. In response to the loss comparison result indicating that the target loss value is less than the preset loss threshold, the loss weight corresponding to the target loss function is determined as the difference between the preset weight value and the second correction value; the second correction value is determined based on the difference between the preset loss threshold and the target loss value.

6. The method according to claim 1, characterized in that, The training sample set includes multiple training sample batches, and each training sample batch includes a first preset number of training samples randomly selected. The process of backpropagating and updating the model parameters of the large language model to be trained based on the sample loss value until the training stopping condition is met, thereby obtaining the target large language model, includes: Based on the sample loss value of each sample result in each training sample batch, the batch loss value of each training sample batch is determined. Based on the batch loss values ​​of a second preset number of consecutive training sample batches, a first comprehensive loss value is determined; Based on the first comprehensive loss value, the model parameters of the large language model to be trained are updated through backpropagation until the training stopping condition is met, thus obtaining the target large language model.

7. The method according to claim 1, characterized in that, The training samples in the training sample set include the private domain corpus samples and the general corpus samples constructed from the same target corpus information; The target corpus information is corpus information that has a complete semantic structure and can independently express a complete meaning.

8. The method according to claim 1, characterized in that, The process of backpropagating and updating the model parameters of the large language model to be trained based on the sample loss value until the training stopping condition is met, thereby obtaining the target large language model, includes: Obtain the target feature extraction layer that matches the large language model to be trained and the original large language model; the original large language model is the model of the large language model to be trained before it was trained on the task to be adapted. The training samples are respectively input into the target feature extraction layer of the large language model to be trained and the target feature extraction layer of the original large language model to obtain the first sample features and the second sample features. Based on the first sample features and the second sample features, the local loss value of the training sample is determined by the second loss function; the second loss function is used to constrain the large language model to be trained to align with the adapted task. Based on the sample loss value and the local loss value, a second comprehensive loss value is determined; Based on the second comprehensive loss value, the model parameters of the large language model to be trained are updated through backpropagation until the training stopping condition is met, thus obtaining the target large language model.

9. The method according to claim 1, characterized in that, The training samples in the training sample set carry sample type labels; The sample type label is used to indicate the type to which the training sample belongs, and the type to which the training sample belongs includes the private domain corpus sample or the general corpus sample.

10. An electronic device, comprising: At least one processor; At least one memory for storing at least one program; The training method for a large language model according to any one of claims 1 to 9 is implemented when at least one of the programs is executed by at least one of the processors.

11. A computer-readable storage medium storing computer-executable instructions for performing the training method of a large language model according to any one of claims 1 to 9.

12. A computer program product comprising a computer program or computer instructions stored in a computer-readable storage medium, wherein a processor of a communication device reads the computer program or computer instructions from the computer-readable storage medium, and the processor executes the computer program or computer instructions to cause the communication device to perform the training method for a large language model according to any one of claims 1 to 9.

Images