Language large model training method and device, equipment and storage medium

Abstract

This application discloses a training method, apparatus, device, and storage medium for a large language model, belonging to the field of computer technology. The method specifically includes: acquiring training sample data, which includes sample task data and corresponding labels; performing feature extraction processing on the sample task data using a preset causal knowledge graph to obtain causal feature information; performing transformation processing on the causal feature information using a preset causal embedding encoder to obtain a causal augmentation vector; and performing fine-tuning training on the large language model to be trained based on the training sample data and the causal augmentation vector to obtain a trained large language model.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to the fields of big data analysis technology, intelligent agent technology, etc., and particularly to a training method, apparatus, device and storage medium for a large language model. Background Technology

[0002] Large Language Models (LMMs) are deep learning models based on a large-scale Transformer architecture. They achieve powerful language understanding and generation capabilities through self-supervised learning on massive amounts of text data. Their core principle is learning the statistical relationships between words in a text.

[0003] Currently, most deep reasoning fine-tuning training schemes for large language models in related technologies are based on instruction tuning or chain-of-thought (CoT) deep reasoning training. However, these schemes often lack sufficient statistical correlation in learning and memorizing the co-occurrence of words in a surface sequence during training, making it difficult to support genuine logical deduction. The output thought chains may be seemingly plausible statistical patterns, failing to fundamentally distinguish between correlation and causality, thus leading to high error rates and illusion rates in large language model reasoning. Summary of the Invention

[0004] This application provides a training method, apparatus, device, and storage medium for a large language model, which can solve the problem of poor ability of large language models to learn and recognize the relationship between events or behaviors during training. The technical solution is as follows: Firstly, a method for training a large language model is provided, the method comprising: Acquire training sample data, which includes sample task data and labels corresponding to the sample task data; Using a pre-defined causal knowledge graph, feature extraction processing is performed on the sample task data to obtain causal feature information; The causal feature information is transformed using a preset causal embedding encoder to obtain a causal enhancement vector; Based on the training sample data and causal augmentation vectors, the language model to be trained is fine-tuned to obtain the trained language model.

[0005] In one possible implementation, the step of using a pre-defined causal knowledge graph to perform feature extraction processing on the sample task data to obtain causal feature information includes: Entity analysis processing is performed on the sample task data to obtain entity information of the sample task data; Using a pre-defined causal knowledge graph, causal retrieval is performed on the entity information to obtain the causal relationship triplet corresponding to the entity information; The causal feature information is obtained based on the causal relationship triples corresponding to the entity information.

[0006] In one possible implementation, the step of using a preset causal embedding encoder to transform the causal feature information to obtain a causal enhancement vector includes: The causal feature information is transformed using a preset causal embedding encoder to obtain the embedded feature information corresponding to the causal feature information. Based on the embedded feature information corresponding to the causal feature information, a causal enhancement vector is obtained.

[0007] In one possible implementation, the fine-tuning training process based on the training sample data and the causal augmentation vector to obtain the trained language model includes: The training sample data and causal augmentation vectors are input into the language model to be trained to output inference results and intermediate hidden state information. Based on the inference results and training sample data, determine the language modeling loss function; Based on the intermediate hidden state information and the causal enhancement vector, determine the causal consistency loss function; Based on the language modeling loss function and the causal consistency loss function, the total loss function of the language model to be trained is determined; Based on the total loss function, training sample data, and causal augmentation vector, the language model to be trained is iteratively trained until the preset training termination condition is met, thus obtaining the trained language model.

[0008] In one possible implementation, determining the causal consistency loss function based on intermediate hidden state information and the causal enhancement vector includes: The intermediate hidden state information and the causal enhancement vector are compared and processed. Based on the results of the comparison process, the causal consistency loss function is determined.

[0009] In one possible implementation, the method further includes: Retrieve task data to be processed; The task data is input into the trained language model to obtain the inference results output by the trained language model.

[0010] Secondly, a training device for a large language model is provided, the device comprising: An acquisition unit is used to acquire training sample data, wherein the training sample data includes sample task data and labels corresponding to the sample task data; The extraction unit is used to perform feature extraction processing on the sample task data using a preset causal knowledge graph to obtain causal feature information; The encoding unit is used to transform the causal feature information using a preset causal embedding encoder to obtain a causal enhancement vector; The training unit is used to fine-tune the language model to be trained based on the training sample data and the causal augmentation vector, so as to obtain the trained language model.

[0011] Thirdly, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement the aspects and any possible implementations described above.

[0012] Fourthly, an electronic device is provided, comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the methods described above and any possible implementations.

[0013] Fifthly, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the aspects and any possible implementations described above.

[0014] The beneficial effects of the technical solution provided in this application include at least the following: As can be seen from the above technical solution, the embodiments of this application can obtain training sample data, which includes sample task data and corresponding labels. Using a preset causal knowledge graph, feature extraction processing is performed on the sample task data to obtain causal feature information. Using a preset causal embedding encoder, the causal feature information is transformed to obtain causal enhancement vectors. Based on the training sample data and the causal enhancement vectors, fine-tuning training is performed on the language model to be trained to obtain a trained language model. Since fine-tuning training can be performed on the language model to be trained based on the training sample data and the causal enhancement vectors to obtain a trained language model, structured and explicit causal knowledge constraints and enhancement signals can be introduced during the training process of the language model. This improves the language model's ability to learn causal relationships in task data, thereby significantly reducing the reasoning error rate and illusion rate of the language model during deep reasoning and improving the reasoning performance of the language model.

[0015] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating a method for training a large language model according to an embodiment of this application; Figure 2 This is a schematic diagram of a training method for a large language model provided in another embodiment of this application; Figure 3 This is a structural block diagram of a language large model training device provided in another embodiment of this application; Figure 4 This is a block diagram of an electronic device used to implement the language large model training method of the embodiments of this application. Detailed Implementation

[0018] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0019] Obviously, the described embodiments are only some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0020] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0021] Deep reasoning in large language models refers to the process by which large models perform multi-step logical deduction, contextual analysis, or knowledge integration when answering complex questions, rather than simply restating facts from the training data. Causal reasoning in large language models is the ability to identify and determine cause-and-effect relationships between events or behaviors—that is, A leads to B. In natural language processing, this requires models to understand not only "what" and "how," but also "why."

[0022] In related technologies, deep reasoning enhancement training methods based on instruction fine-tuning or thought chains typically involve first collecting datasets with complex reasoning tasks, then adding manually or model-generated intermediate reasoning steps—i.e., thought processes—to the data to determine thought chain cues. During model training, the large language model is fine-tuned using data with thought chain cues. Here, the large language model primarily learns statistical associations (lexical co-occurrence patterns), and the thought chain-based training scheme is trained through text sequences, with its reasoning process relying on the statistical associations of the text sequences.

[0023] However, the language large model in the related technology is not good at capturing and memorizing the statistical correlation of the statistical regularity of the co-occurrence of words in the surface sequence of the text. It is difficult to support true logical deduction. The output thought chain may be a seemingly plausible statistical pattern. It cannot fundamentally distinguish the correlation and causality between words, resulting in a high reasoning error rate and illusion rate.

[0024] Please refer to Figure 1This illustration shows a flowchart of a language large model training method according to an embodiment of this application. The language large model training method may specifically include: Step 101: Obtain training sample data, which includes sample task data and the labels corresponding to the sample task data.

[0025] Step 102: Using a preset causal knowledge graph, perform feature extraction processing on the sample task data to obtain causal feature information.

[0026] Step 103: Use a preset causal embedding encoder to transform the causal feature information to obtain a causal enhancement vector.

[0027] Step 104: Based on the training sample data and causal augmentation vector, perform fine-tuning training on the language model to be trained to obtain the trained language model.

[0028] It should be noted that sample task data can include sample data from complex reasoning tasks. Task data can be task data from hallucination-sensitive domains, such as education, law, and medicine. Task data can also include questions posed by users to the language model.

[0029] It should be noted that the default Causal Knowledge Graph (Causal KG) focuses on storing and expressing causal relationships between entities or events.

[0030] It should be noted that the preset causal embedding encoder can be an encoder used to convert causal structures into embedding features.

[0031] In this way, by acquiring training sample data, which includes sample task data and corresponding labels, and using a preset causal knowledge graph, feature extraction processing is performed on the sample task data to obtain causal feature information. Then, using a preset causal embedding encoder, the causal feature information is transformed to obtain causal enhancement vectors. Based on the training sample data and the causal enhancement vectors, the language model to be trained is fine-tuned to obtain a trained language model. Since the language model to be trained can be fine-tuned based on the training sample data and the causal enhancement vectors to obtain a trained language model, structured and explicit causal knowledge constraints and enhancement signals can be introduced into the training process of the language model. This improves the language model's ability to learn causal relationships in task data, thereby significantly reducing the reasoning error rate and illusion rate of the language model in deep reasoning and improving its reasoning performance.

[0032] Optionally, in one possible implementation of this embodiment, in step 102, firstly, entity analysis processing can be performed on the sample task data to obtain entity information of the sample task data. Secondly, a preset causal knowledge graph can be used to perform causal retrieval on the entity information to obtain causal relationship triples corresponding to the entity information. Thirdly, the causal feature information can be obtained based on the causal relationship triples corresponding to the entity information.

[0033] In this implementation, the preset causal knowledge graph can be a pre-constructed causal knowledge graph of the domain to which the task belongs.

[0034] Understandably, a knowledge graph (KG) is a structured method of knowledge representation that stores entities and relations as triples, clearly expressing the relationships between facts and concepts. A causal knowledge graph (Causal KG), on the other hand, can be used to store and represent causal relationships between entities or events. A pre-defined causal knowledge graph can include causal relationship triples (entity 1, relation_causation, entity 2).

[0035] In one specific implementation of this method, a preset causal knowledge graph can be queried based on the entity information of the sample task data. All causal triples related to the entity information can be extracted from the preset causal knowledge graph, and then the causal triples related to the entity information can be used as the causal feature information.

[0036] In this way, entity information of the sample task data can be obtained by performing entity analysis on the sample task data. Then, by using a preset causal knowledge graph, the causal relationship triples corresponding to the entity information can be retrieved and extracted to obtain more accurate and reliable causal feature information, thereby further ensuring that the subsequent model can better learn the causal information corresponding to the task data.

[0037] Optionally, in one possible implementation of this embodiment, in step 103, a preset causal embedding encoder is used to transform the causal feature information to obtain the embedding feature information corresponding to the causal feature information, and then a causal enhancement vector can be obtained based on the embedding feature information corresponding to the causal feature information.

[0038] In this implementation, a pre-defined causal embedding encoder can be used to convert causal information into causal-related embedded feature information.

[0039] In one specific implementation of this approach, a pre-defined causal embedding encoder is used to convert the causal feature information into high-dimensional embedding feature information, which is then used as a causal enhancement vector.

[0040] Here, the causal enhancement vector can characterize the associated causal structure required for the task.

[0041] In this way, by using a pre-defined causal embedding encoder, causal feature information can be converted into high-dimensional embedding feature information, and then the embedding feature information can be used as a causal enhancement vector. This causal enhancement vector can be feature information that is easy for large language models to understand, thereby further ensuring that subsequent models can better learn the causal information corresponding to the task data.

[0042] It should be noted that the specific implementation process provided in this embodiment can be combined with various specific implementation processes provided in the aforementioned implementation methods to implement the language large model training method of this embodiment. Detailed descriptions can be found in the relevant content of the aforementioned implementation methods, and will not be repeated here.

[0043] Optionally, in one possible implementation of this embodiment, in step 104, firstly, the training sample data and causal augmentation vector can be input into the language model to be trained to output inference results and intermediate hidden state information. Secondly, a language modeling loss function can be determined based on the inference results and training sample data. Thirdly, a causal consistency loss function can be determined based on the intermediate hidden state information and causal augmentation vector. Fourthly, a total loss function of the language model to be trained can be determined based on the language modeling loss function and the causal consistency loss function. Fifthly, the language model to be trained can be iteratively trained based on the total loss function, training sample data, and causal augmentation vector until a preset training termination condition is met, resulting in a trained language model.

[0044] In this implementation, the training sample data includes sample task data and the labels corresponding to the sample task data.

[0045] In one specific implementation of this approach, firstly, the inference result and the labels corresponding to the sample task data can be compared. Secondly, based on the result of the comparison, the language modeling loss function can be determined.

[0046] One specific implementation method involves calculating the matching degree between the inference result and the label corresponding to the sample task data, and obtaining the language modeling loss function based on the matching degree.

[0047] In another specific implementation of this approach, firstly, the intermediate hidden state information and the causal enhancement vector can be compared. Secondly, based on the result of the comparison, the causal consistency loss function can be determined.

[0048] One specific implementation involves calculating the distance between the intermediate hidden state information and the causal enhancement vector, and then obtaining the causal consistency loss function based on the distance.

[0049] Another specific implementation involves calculating the similarity between the intermediate hidden state information and the causal enhancement vector, and then obtaining the causal consistency loss function based on the similarity.

[0050] In another specific implementation of this method, based on the language modeling loss function and the causal consistency loss function, the total loss function of the language model to be trained is calculated using a weighted summation algorithm, as shown in formula (1): (1) in, It can be the total loss function. A loss function can be modeled for the language. It can be a causal consistency loss function. It can be a balancing parameter.

[0051] In another specific implementation of this approach, firstly, the model parameters of the large language model to be trained can be optimized based on the total loss function. Secondly, the large language model to be trained can be updated based on the optimized model parameters. Thirdly, based on the training sample data and causal augmentation vectors, the updated large language model to be trained is iteratively trained until a preset training termination condition is met, resulting in a fully trained large language model.

[0052] One specific implementation method involves using an optimizer to optimize and update the model parameters of the large language model to be trained, based on the total loss function.

[0053] Here, the preset training termination conditions may include satisfying the preset loss convergence condition, the number of iterations of training reaching the preset number of iterations threshold, etc.

[0054] In this way, the total loss function of the language model to be trained can be calculated based on the language modeling loss function determined by the inference results and training sample data, and the causal consistency loss function determined by the intermediate hidden state information and the causal augmentation vector. Then, based on the total loss function, the training sample data, and the causal augmentation vector, the language model to be trained is iteratively trained to obtain the trained language model. By using a dual loss function, namely the language modeling loss and the constraint loss based on causal knowledge, the model parameters are optimized, which further improves the reliability of the language model in learning causal information from task data, thereby improving the reliability and performance of the trained language model during inference.

[0055] It should be noted that the specific implementation process provided in this embodiment can be combined with various specific implementation processes provided in the aforementioned implementation methods to implement the language large model training method of this embodiment. Detailed descriptions can be found in the relevant content of the aforementioned implementation methods, and will not be repeated here.

[0056] Optionally, in one possible implementation of this embodiment, after step 104, further, task data to be processed can be obtained, and then the task data can be input into the trained language model to obtain the inference result output by the trained language model.

[0057] In this implementation, the task data to be processed may include user-submitted question data, task data, etc.

[0058] In this implementation, the reasoning result may include response data, task processing method data, etc.

[0059] In this way, a well-trained large language model can be used to perform reasoning analysis on the task data. This trained large language model can better understand the causal relationships in the task data, and can reason according to the correct causal chain, reducing the possibility of hallucination errors and improving the reliability and accuracy of the model's reasoning results. It should be noted that the specific implementation process provided in this embodiment can be combined with various specific implementation processes provided in the aforementioned implementation methods to implement the language large model training method of this embodiment. Detailed descriptions can be found in the relevant content of the aforementioned implementation methods, and will not be repeated here.

[0060] To better understand the methods of the embodiments of this application, the methods of the embodiments of this application will be described below in conjunction with the accompanying drawings and specific application scenarios.

[0061] Figure 2 This is a flowchart illustrating a method for training a large language model according to another embodiment of this application, as shown below. Figure 2 As shown.

[0062] Step 201: Obtain the sample task data and the corresponding labels for the sample task data.

[0063] In this embodiment, the sample task data may include deep inference task data to be trained.

[0064] Step 202: Use the preset causal knowledge graph to perform feature extraction processing on the sample task data to obtain causal feature information.

[0065] In this embodiment, firstly, entity analysis is performed on the sample task data to obtain entity information. Secondly, a pre-defined causal knowledge graph is used to perform causal retrieval on the entity information to obtain causal triples corresponding to the entity information. Thirdly, causal feature information is obtained based on the causal triples corresponding to the entity information.

[0066] For example, given sample task data P, a pre-built causal knowledge graph (CK Graph) can be queried based on the entities and concepts in the input P. All causal triples (entity 1, relation_causation, entity 2) related to P can be extracted from the CK Graph.

[0067] Step 203: Use a preset causal embedding encoder to transform the causal feature information to obtain a causal enhancement vector.

[0068] In this embodiment, the extracted triples can be transformed into a high-dimensional causal augmentation vector Ce using a causal embedding encoder. The causal augmentation vector Ce can represent the true causal structure required for the task.

[0069] Step 204: Input the sample task data, labels, and causal augmentation vectors into the language model to be trained to output the inference results and intermediate hidden state information.

[0070] Step 205: Determine the language modeling loss function based on the inference results and labels.

[0071] Step 206: Determine the causal consistency loss function based on the intermediate hidden state information and the causal enhancement vector.

[0072] Step 207: Calculate the total loss function based on the language modeling loss function and the causal consistency loss function.

[0073] Step 208: Using the optimizer, based on the total loss function, optimize the model parameters of the language model to be trained. Then, based on the sample task data, labels, and causal augmentation vectors, iteratively train the language model to be trained after parameter optimization to obtain the trained language model.

[0074] In this embodiment, firstly, the sample task data P, the corresponding label Y, and the causal augmentation vector Ce are simultaneously input into the language big model. The causal augmentation vector Ce can be used as external knowledge input through concatenation or injection. The language big model performs forward propagation to generate the inference result of the current round, i.e., the inference output O and the intermediate hidden state H_LLM.

[0075] Secondly, the loss function is calculated. First, the language modeling loss function and the causal consistency loss function are calculated. Then, the language modeling loss function and the causal consistency loss function are weighted and summed to obtain the total loss function.

[0076] Here, it can be understood that the language modeling loss function can be used to measure the degree of matching between O and the true labeled answer. The causal consistency loss function can be used to measure the distance or similarity between the intermediate hidden state H_LLM and the causal augmentation vector Ce. The causal consistency loss function can be used to force the internal representation of the large language model to align with the true causal logic.

[0077] Next, model parameter optimization. Here, the backpropagation algorithm can be used to iteratively update the parameters of the large language model to be trained by minimizing the total loss function.

[0078] Step 209: Using the trained language model, reason about the acquired real-time task data to obtain the reasoning results corresponding to the task data.

[0079] In this embodiment, for example, in a medical diagnostic reasoning scenario, if the input task P is: "A 60-year-old male, a long-term smoker (20 years), has recently experienced persistent cough, hemoptysis, and chest pain. What is the most likely cause and the next diagnostic step?", using related technologies, the language model might give the incorrect answer "bronchitis," which has the problem of high-frequency co-occurrence.

[0080] Here, using the method in this embodiment, firstly, training sample data for a medical diagnostic reasoning scenario is obtained. This training sample data can include sample task data and corresponding labels. Secondly, the entity information of the sample task data is determined. Thirdly, a causal knowledge graph can be used to retrieve and extract causal feature information corresponding to the sample task data. For example, key causal triples such as (long-term smoking leads to lung cell carcinogenesis) and (carcinogenesis causes hemoptysis) are extracted, i.e., causal feature information. Next, a causal embedding encoder is used to transform the causal feature information to obtain a causal enhancement vector. Here, high-weight causal triples can also be encoded as causal enhancement vectors Ce, for example, strongly pointing to the "lung cancer" path. Finally, the LLM is trained. During LLM training, when the LLM attempts to generate "bronchitis," the causal consistency loss function calculates the distance between its intermediate hidden state H_LLM and the causal enhancement vector Ce. The causal enhancement vector Ce constrains the "lung cancer" path; deviation results in a large causal consistency loss function. By penalizing the LLM with gradients, its internal representation can be forced to turn towards a path that better conforms to the true causal structure. Finally, using the trained LLM to process the above input task P, the LLM can generate the inference: "Combining the smoking history and symptoms of coughing up blood and chest pain, based on causal knowledge, the most likely diagnosis is lung cancer, and the next step should be a CT scan." Thus, by adopting the scheme in this embodiment, the external structured causal knowledge graph information can be encoded into a causal enhancement vector during the fine-tuning training of the language big model. This vector can then be used as a regularization target to construct and optimize the causal consistency loss. This explicitly constrains the hidden states of the internal reasoning paths of the big model, ensuring that they are consistent with the true causal structure. This achieves explicit encoding and constraint of essential causal relationships, enabling the language big model to effectively distinguish between correlation and causality. Consequently, it significantly reduces the model's reasoning error rate and illusion rate in deep reasoning, especially in error- and illusion-sensitive domains.

[0081] Furthermore, by adopting the scheme in this embodiment, high-quality external causal knowledge based on causal knowledge graphs can be transformed into causal embedding vectors that the language big model can understand. The training process minimizes the total loss that references the language modeling loss function and the causal consistency loss function, which can force the model to maintain a high degree of alignment between its internal hidden state and the real causal logic derived from the external causal knowledge graph when performing deep reasoning. That is, the causal embedding vectors are highly aligned. The causal embedding vectors are used to calibrate the model's internal reasoning path during training, thereby improving the reasoning performance of the language big model.

[0082] Furthermore, by adopting the scheme in this embodiment, the causal consistency loss can be introduced, which provides a hard constraint on the causal structure, forcing the internal inference representation H_LLM of the model to align with the external real causal logic, avoiding erroneous inferences based on statistical co-occurrence, and significantly reducing the inference error rate.

[0083] Furthermore, by employing the scheme in this embodiment, the model's reasoning path can be constrained by a high-quality, pre-validated causal knowledge graph. The causal knowledge graph limits the range of legitimate causal relationships, making it difficult for the model to deviate from the correct causal chain, thereby reducing the possibility of fabricating false facts and significantly lowering the hallucination rate.

[0084] Furthermore, in the scheme of this embodiment, the causal embedding vector is traceable. By analyzing the activation state of the causal embedding vector and the internal reasoning representation of the model, it is possible to trace back which causal chain(s) dominated the reasoning result, thereby improving the transparency of decision-making and the interpretability of complex multi-step reasoning.

[0085] Furthermore, in the scheme of this embodiment, the total loss function can be calculated through the balancing parameter, the language modeling loss function, and the causal consistency loss function. The language modeling loss function can ensure general capabilities, while the causal consistency loss function can specifically optimize causal reasoning capabilities. By adjusting the balancing parameter, a balance can be achieved, enabling the trained language model to enhance professional reasoning capabilities without sacrificing general capabilities.

[0086] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0087] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0088] Figure 3 This invention provides a structural block diagram of a language large model training device according to one embodiment of the present application, as shown below. Figure 3As shown. The language large model training device 300 of this embodiment may include an acquisition unit 301, an extraction unit 302, an encoding unit 303, and a training unit 304. The acquisition unit 301 is used to acquire training sample data, which includes sample task data and corresponding labels. The extraction unit 302 is used to perform feature extraction processing on the sample task data using a preset causal knowledge graph to obtain causal feature information. The encoding unit 303 is used to perform transformation processing on the causal feature information using a preset causal embedding encoder to obtain a causal enhancement vector. The training unit 304 is used to perform fine-tuning training processing on the language large model to be trained based on the training sample data and the causal enhancement vector to obtain a trained language large model.

[0089] Optionally, in one possible implementation of this embodiment, the extraction unit 302 is used to perform entity analysis processing on the sample task data to obtain entity information of the sample task data; use a preset causal knowledge graph to perform causal retrieval on the entity information to obtain causal relationship triples corresponding to the entity information; and obtain the causal feature information based on the causal relationship triples corresponding to the entity information.

[0090] Optionally, in one possible implementation of this embodiment, the encoding unit 303 is used to perform transformation processing on the causal feature information using a preset causal embedding encoder to obtain the embedding feature information corresponding to the causal feature information; and to obtain a causal enhancement vector based on the embedding feature information corresponding to the causal feature information.

[0091] Optionally, in one possible implementation of this embodiment, the training unit 304 is configured to input the training sample data and causal augmentation vector into the language model to be trained, and output inference results and intermediate hidden state information; determine a language modeling loss function based on the inference results and training sample data; determine a causal consistency loss function based on the intermediate hidden state information and causal augmentation vector; determine the total loss function of the language model to be trained based on the language modeling loss function and the causal consistency loss function; and perform iterative training on the language model to be trained based on the total loss function, training sample data, and causal augmentation vector until a preset training termination condition is met, thereby obtaining a trained language model.

[0092] Optionally, in one possible implementation of this embodiment, the training unit 304 is used to perform a comparison process on the intermediate hidden state information and the causal augmentation vector; and to determine the causal consistency loss function based on the result of the comparison process.

[0093] Optionally, in one possible implementation of this embodiment, an inference unit (not shown in the figure) may also be included. The inference unit is used to acquire task data to be processed and input the task data into the trained language model to obtain the inference result output by the trained language model.

[0094] In this embodiment, training sample data can be acquired by an acquisition unit. The training sample data includes sample task data and corresponding labels. An extraction unit uses a preset causal knowledge graph to perform feature extraction processing on the sample task data to obtain causal feature information. An encoding unit uses a preset causal embedding encoder to transform the causal feature information to obtain causal enhancement vectors. A training unit performs fine-tuning training on the language model to be trained based on the training sample data and causal enhancement vectors to obtain a trained language model. Since the language model to be trained can be fine-tuned based on the training sample data and causal enhancement vectors to obtain a trained language model, structured and explicit causal knowledge constraints and enhancement signals can be introduced into the training process of the language model. This improves the language model's ability to learn causal relationships in task data, thereby significantly reducing the reasoning error rate and illusion rate of the language model in deep reasoning and improving the reasoning performance of the language model.

[0095] The technical solution of this application involves the collection, storage, use, processing, transmission, provision, and disclosure of user personal information, such as user image and attribute data, which comply with relevant laws and regulations and do not violate public order and good morals.

[0096] According to embodiments of this application, this application also provides an electronic device, a readable storage medium, and a computer program product.

[0097] Figure 4 A schematic block diagram of an example electronic device 400 that can be used to implement embodiments of this application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0098] like Figure 4As shown, the electronic device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 into a random access memory (RAM) 403. The RAM 403 may also store various programs and data required for the operation of the electronic device 400. The computing unit 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0099] Multiple components in electronic device 400 are connected to I / O interface 405, including: input unit 406, such as keyboard, mouse, etc.; output unit 407, such as various types of displays, speakers, etc.; storage unit 408, such as disk, optical disk, etc.; and communication unit 409, such as network card, modem, wireless transceiver, etc. Communication unit 409 allows electronic device 400 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0100] The computing unit 401 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 401 performs the various methods and processes described above, such as the language large model training method. For example, in some embodiments, the language large model training method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 408. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 400 via ROM 402 and / or communication unit 409. When the computer program is loaded into RAM 403 and executed by the computing unit 401, one or more steps of the language large model training method described above can be performed. Alternatively, in other embodiments, the computing unit 401 can be configured to perform the language large model training method by any other suitable means (e.g., by means of firmware).

[0101] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0102] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0103] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0104] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0105] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0106] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0107] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0108] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for training a large language model, characterized in that, The method includes: Acquire training sample data, which includes sample task data and labels corresponding to the sample task data; Using a pre-defined causal knowledge graph, feature extraction processing is performed on the sample task data to obtain causal feature information; The causal feature information is transformed using a preset causal embedding encoder to obtain a causal enhancement vector; Based on the training sample data and causal augmentation vectors, the language model to be trained is fine-tuned to obtain the trained language model.

2. The method according to claim 1, characterized in that, The step of using a pre-defined causal knowledge graph to perform feature extraction processing on the sample task data to obtain causal feature information includes: Entity analysis processing is performed on the sample task data to obtain entity information of the sample task data; Using a pre-defined causal knowledge graph, causal retrieval is performed on the entity information to obtain the causal relationship triplet corresponding to the entity information; The causal feature information is obtained based on the causal relationship triples corresponding to the entity information.

3. The method according to claim 1, characterized in that, The step of using a preset causal embedding encoder to transform the causal feature information to obtain a causal enhancement vector includes: The causal feature information is transformed using a preset causal embedding encoder to obtain the embedded feature information corresponding to the causal feature information. Based on the embedded feature information corresponding to the causal feature information, a causal enhancement vector is obtained.

4. The method according to claim 1, characterized in that, The process of fine-tuning the language model to be trained based on the training sample data and the causal augmentation vector to obtain the trained language model includes: The training sample data and causal augmentation vectors are input into the language model to be trained to output inference results and intermediate hidden state information. Based on the inference results and training sample data, determine the language modeling loss function; Based on the intermediate hidden state information and the causal enhancement vector, the causal consistency loss function is determined; Based on the language modeling loss function and the causal consistency loss function, the total loss function of the language model to be trained is determined; Based on the total loss function, training sample data, and causal augmentation vector, the language model to be trained is iteratively trained until the preset training termination condition is met, thus obtaining the trained language model.

5. The method according to claim 4, characterized in that, The step of determining the causal consistency loss function based on the intermediate hidden state information and the causal enhancement vector includes: The intermediate hidden state information and the causal enhancement vector are compared and processed. Based on the results of the comparison process, the causal consistency loss function is determined.

6. The method according to any one of claims 1-5, characterized in that, The method further includes: Retrieve task data to be processed; The task data is input into the trained language model to obtain the inference results output by the trained language model.

7. A training device for a large language model, characterized in that, The device includes: An acquisition unit is used to acquire training sample data, wherein the training sample data includes sample task data and labels corresponding to the sample task data; The extraction unit is used to perform feature extraction processing on the sample task data using a preset causal knowledge graph to obtain causal feature information; The encoding unit is used to transform the causal feature information using a preset causal embedding encoder to obtain a causal enhancement vector; The training unit is used to fine-tune the language model to be trained based on the training sample data and the causal augmentation vector, so as to obtain the trained language model.

8. An electronic device, characterized in that, include: At least one processor; as well as A memory that is communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1-6.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method according to any one of claims 1-6.

Images