Large model fine tuning method based on causal graph and thinking chain enhancement and related device

Abstract

The invention discloses a large model fine tuning method based on a causal graph and thinking chain enhancement and a related device, and relates to the technical field of large model fine tuning in the power industry, and the method comprises the steps: carrying out the causal mining of power equipment data, and constructing a power equipment causal graph containing causal weight information; disassembling the input of the large model into a thinking chain, correspondingly generating a chain type causal pair according to the thinking chain, and performing path retrieval matching and causal consistency check through the chain type causal pair and the constructed electrical equipment causal graph to realize alignment of the reasoning process; and exciting a reinforcement learning process through an alignment result of the reasoning process, optimizing a pre-established reinforcement learning reward model, constraining a thinking chain generation process, guiding the large model to generate a thinking chain under a causal constraint condition, and realizing fine tuning of the large model. According to the method, causal reasoning and causality are embedded into a reinforcement learning feedback process of large model fine tuning, so that the large model can learn a basic causal reasoning rule, and the logicality, the interpretability and the robustness of thinking chain reasoning can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of large model fine-tuning in the power industry, and in particular to a large model fine-tuning method and related devices based on causal graphs and thought chain enhancement. Background Art

[0002] The power big model is a smart grid solution based on artificial intelligence technology. It combines traditional power grids with modern information technology, and realizes the intelligence, automation and informatization of power grids through big data, cloud computing, the Internet of Things and other means.

[0003] With the rapid development of new energy power generation technology, the energy landscape has undergone profound adjustments. The energy and power industry is facing many challenges. Traditional technologies are difficult to solve problems such as accurate source and load prediction and complex system optimization. Therefore, it is urgent to accelerate the integration and application of the new generation of artificial intelligence technologies represented by large models to promote the intelligent transformation and upgrading of the power industry.

[0004] Large-model fine-tuning technology involves further training a pre-trained large language model for a specific task or domain. The goal of fine-tuning is to better adapt the large model to downstream tasks while reducing computational costs and the risk of overfitting. Common fine-tuning methods include full-parameter fine-tuning, efficient parameter fine-tuning, instruction fine-tuning, and reinforcement learning with human feedback. The power equipment sector, characterized by multimodal data, scarce defect data, and clear causal mechanisms, is well-suited for instruction fine-tuning or reinforcement learning with human feedback, leveraging small-scale, high-quality data to achieve model fine-tuning. However, current large models rely on statistical correlations between data and lack explicit causal mechanisms. Intermediate steps in the generation of large-model inferences may contain errors or redundant inferences, leading to poor performance in reasoning or prediction tasks. Even considering using external knowledge graphs for semantic retrieval, large models still face challenges such as missing causal edges, misaligned reasoning steps, and severe noise in large-scale graphs, resulting in reduced accuracy, limited generalization, and weakened robustness. Summary of the Invention

[0005] The purpose of the present invention is to address the problems in the above-mentioned prior art and provide a large model fine-tuning method and related devices based on causal graphs and thought chain enhancement. Considering the combination of causal graphs and thought chain enhancement mechanisms, causal reasoning and causality are embedded in the reinforcement learning feedback process of large model fine-tuning, so that the large model can learn basic causal reasoning rules, continuously optimize and update reasoning strategies, and thereby improve the logic, interpretability and robustness of thought chain reasoning.

[0006] In order to achieve the above object, the present invention has the following technical solutions:

[0007] First, a large model fine-tuning method based on causal diagrams and thought chain enhancement is provided, including:

[0008] Conduct causal mining on power equipment data and construct a causal graph of power equipment containing causal weight information;

[0009] The input of the large model is decomposed into thought chains, and chain causal pairs are generated according to the thought chains. The chain causal pairs are then compared with the constructed power equipment causal graph for path retrieval and matching, and causal consistency checks are performed to achieve alignment of the reasoning process.

[0010] The alignment results of the reasoning process are used to stimulate the reinforcement learning process, optimize the pre-established reinforcement learning reward model, constrain the thought chain generation process, guide the large model to generate thought chains under causal constraints, and achieve fine-tuning of the large model.

[0011] As a preferred solution, the causal mining of the power equipment data and the construction of the power equipment causal graph containing causal weight information include the following steps:

[0012] Entity extraction, relationship extraction, and graph construction are performed on power equipment data to form an original knowledge graph G = (V, E), where V is all nodes in the graph and E is all edges in the graph. Each relationship edge in the graph, edge = (u, r, v) ∈ E, is defined as a triple with a starting point u, an end point v, and a relationship r. The relationship r contains three relationship labels: "related", "causal", and "belongs to". The power equipment data includes power equipment operation information and fault analysis reports.

[0013] Define the causal function Causality(r) to calculate the causal numerical weight of the relationship r. If the weight is lower than the threshold θ, it will be discarded.

[0014] The weight values ​​of causal edges are iteratively updated through the causal screening and causal discovery PC algorithm. If the new weight of the edge (u, r, v) is greater than the threshold θ, the new weight edge is added or updated to the causal graph to construct a causal graph of power equipment containing causal weight information.

[0015] As a preferred solution, the input of the large model is decomposed into thought chains, and chain causal pairs are generated according to the thought chains. The chain causal pairs are then used to perform path retrieval matching and causal consistency checking with the constructed power equipment causal graph to achieve alignment of the reasoning process, which includes the following steps:

[0016] When breaking down the input of the large model into thought chains, prompts are given to the input of the large model, requiring the large model to generate a thought chain P=(P1, P2, ..., P N ); Each thinking chain reasoning step P i Generate a corresponding causal pair EPi = (start i , end i ), start i represents the cause entity of the cause-effect pair, and end i represents the effect entity of the cause-effect pair.

[0017] In the constructed power equipment cause-effect graph, a path retrieval match is performed on the cause-effect pair EP i = (start i , end i ), to obtain a candidate path set Path = {path1, …, path i} with a start point start i and an end point end m .

[0018] The path scoring method is used to respectively evaluate the CUI overlap degree, the semantic type matching degree, and the path length heuristic score of the path and the cause-effect pair, so as to evaluate the path. The total score is calculated based on the following formula to reflect the relevance of the path and the cause-effect pair:

[0019] Score (path) = aCUIoverlap (path) + bsemantic_overlap (path) + gLength (path)

[0020] Wherein, a, b, g are the weight proportions of the CUI overlap degree, the semantic type matching degree, and the path length heuristic score of the three types of scoring methods, respectively. The paths are sorted according to the total score, and the paths with a set proportion keep_ratio x |Path| are reserved.

[0021] As a preferred scheme, the alignment result through the reasoning process stimulates the reinforcement learning process, and the pre-established reinforcement learning reward model is optimized, which includes:

[0022] The process of retrieving the chain cause-effect pair EP i on the constructed power equipment cause-effect graph is modeled as a Markov decision process, which includes a state space S, an action space A, a state transition function d, and a reward function R.

[0023] At each retrieval time t, the state of the agent is defined as s t = (EP, e t , h t , end) e S, EP represents the retrieved thought chain reasoning step cause-effect pair, e t represents the current entity vector, and h trepresents the hidden state of the visited path, and end represents the result entity of the retrieved causal pair; at the starting time t = 0, e0 is initialized to the cause entity start in the causal pair, and h0 is initialized to the all-zero vector;

[0024] In state s t Next, the action space is defined as the current entity e t The set of all neighbor nodes in the causal graph, namely A(s t )={e|(e t ,r,e)∈K}, where K represents the triple set of the causal graph and r represents the causal relationship;

[0025] At time t, the agent follows the policy π θ (a t |s t ) Execute action a t ∈A(s t ), the state is based on the transfer function δ(s t ,a t )Transfer to s t+1 =(EP,e t+1 ,h t+1 ,end), where e t+1 =a t ;The environment is based on the reward function R(s t ,a t )=t t Feedback reward value; in the last step T-1, if e T =end means reaching the target node, and the environment returns a reward of 1, otherwise it returns 0; the rewards at other times are all 0;

[0026] Use the policy network-value function network hybrid Actor-Critic algorithm to train the policy network π θ (a t |s t ) and value function network The long short-term memory network LSTM is used to model state features, and at each step the retrieved thought chain reasoning step causal pair EP and the current entity vector e t Concatenate the input LSTM to generate the hidden state h t :

[0027]

[0028] The feedforward networks of the policy network Actor and the value function network Critic are built on the LSTM layer respectively; the policy network Actor is responsible for all possible actions a t ∈A(s t ) is scored and the action probability distribution is generated through the softmax function:

[0029] π θ (a t |s t )=softmax(A t W2·ReLU(W1·h t ))

[0030] Among them, W1 and W2 are learnable weight matrices, Indicates that the edge information in the causal graph is also encoded into each action a t In the embedding vector of

[0031] The value function network Critic outputs a scalar value to evaluate the state s t The value function of:

[0032]

[0033] Among them, W3 and W4 are learnable weight matrices.

[0034] As a preferred solution, in the step of guiding the large model to generate a thinking chain under causal constraints during the generation of the constrained thinking chain, causal constraints are performed based on guided prompts or structured prompts to guide the large model to generate a thinking chain under causal constraints;

[0035] Guided prompts are to guide the generation of thought chains by outlining the specific steps to be followed in the thinking process of the big model;

[0036] Structured prompts refer to the use of extensible markup language (XML) tags to separate reasoning steps and final answers, helping large models to parse prompt words and thus generate thought chains.

[0037] As a preferred solution, in the step of implementing the fine-tuning of the large model, the thinking chain sample data generated under the causal constraint conditions are checked and verified with the mechanism rules to ensure the credibility of the thinking chain sample data of the thinking process; according to the model distillation idea, the checked thinking chain sample data is used as cold start data to fine-tune the large model, so that the large model can learn the causal reasoning rules of the thinking chain sample data, and improve the logic, interpretability and robustness of the thinking process of the large model by optimizing and updating the reasoning strategy.

[0038] Secondly, a large model fine-tuning system based on causal diagrams and thought chain enhancement is provided, including:

[0039] The power equipment causal graph construction module is used to perform causal mining on power equipment data and construct a power equipment causal graph containing causal weight information;

[0040] The causal thinking chain alignment module is used to decompose the input of the large model into thinking chains and generate chained causal pairs based on the thinking chains. The chained causal pairs are then compared with the constructed power equipment causal graph for path retrieval and matching, and causal consistency checks are performed to achieve alignment of the reasoning process.

[0041] The causal enhanced reinforcement learning module is used to stimulate the reinforcement learning process through the alignment results of the reasoning process, optimize the pre-established reinforcement learning reward model, constrain the thought chain generation process, guide the large model to generate thought chains under causal constraints, and realize large model fine-tuning.

[0042] As a preferred solution, the power equipment causal graph construction module performs entity extraction, relationship extraction, and graph construction on the power equipment data to form an original knowledge graph G = (V, E), where V is all nodes in the graph and E is all edges in the graph; each relationship edge edge = (u, r, v) ∈ E in the graph is defined as a triple with a starting point u, an end point v, and a relationship r, where the relationship r contains three relationship labels: "related", "causal", and "belongs to"; the power equipment data includes power equipment operation information and fault analysis reports;

[0043] Define the causal function Causality(r) to calculate the causal numerical weight of the relationship r. If the weight is lower than the threshold θ, it will be discarded.

[0044] The weight values ​​of causal edges are iteratively updated through the causal screening and causal discovery PC algorithm. If the new weight of the edge (u, r, v) is greater than the threshold θ, the new weight edge is added or updated to the causal graph to construct a causal graph of power equipment containing causal weight information.

[0045] As a preferred solution, the causal thinking chain alignment module prompts the input of the large model when decomposing the input of the large model into thinking chains, requiring the large model to generate a specific format of thinking chain P = (P1, P2, ..., P N ); Each thinking chain reasoning step P i Generate a corresponding causal pair EP i =(start i ,end i ), start i Indicates causal relationship between cause entities, end i Represents cause-effect pairs of entities;

[0046] In the constructed power equipment causal diagram, the causal pair EP i =(start i ,end i ) Perform path search and matching, and get the starting point as start i 、End point is endi a set of candidate paths Path = {path1,..., path m};

[0047] The path scoring method is used to evaluate the path and the CUI overlap degree, semantic type matching degree and path length heuristic score of the causal pair respectively, so as to evaluate the path, and the total score reflecting the relevance of the path and the causal pair is calculated based on the following formula:

[0048] Score(path) = aCUIoverlap(path) + bsemantic_overlap(path) + gLength(path)

[0049] Wherein, a, b, g are the weight proportions of CUI overlap degree, semantic type matching degree and path length heuristic score of the three types of scoring methods respectively; the paths are sorted according to the total score, and the paths with a set proportion of keep_ratio x |Path| are retained.

[0050] As a preferred scheme, the causal reinforcement learning module stimulates the reinforcement learning process through the alignment result of the reasoning process, and when optimizing the pre-established reinforcement learning reward model, the process of searching the chain causal pair EP i on the constructed power equipment causal graph is modeled as a Markov decision process, which includes state space S, action space A, state transition function d and reward function R;

[0051] At each search time t, the state of the agent is defined as s t = (EP, e t , h t , end) e S, EP represents the search step causal pair of the reasoning chain, e t represents the current entity vector, h t represents the hidden state of the visited path, and end represents the result entity of the searched causal pair; at the starting time t = 0, e0 is initialized as the reason entity start in the causal pair, and h0 is initialized as a zero vector;

[0052] In state s t , the action space is defined as the set of all neighbor nodes of the current entity e t in the causal graph, that is, A(s t ) = {e | (e t , r, e) e K}, where K represents the triple set of the causal graph, and r represents the causal relationship;

[0053] At time t, the agent selects an action a θ according to the policy p t (a t) Execute action a t ∈A(s t ), the state is based on the transfer function δ(s t ,a t )Transfer to s t+1 =(EP,e t+1 ,h t+1 ,end), where e t+1 =a t ;The environment is based on the reward function R(s t ,a t )=t t Feedback reward value; in the last step T-1, if e T =end means reaching the target node, and the environment returns a reward of 1, otherwise it returns 0; the rewards at other times are all 0;

[0054] Use the policy network-value function network hybrid Actor-Critic algorithm to train the policy network π θ (a t |s t ) and value function network The long short-term memory network LSTM is used to model state features, and at each step the retrieved thought chain reasoning step causal pair EP and the current entity vector e t Concatenate the input LSTM to generate the hidden state h t :

[0055]

[0056] The feedforward networks of the policy network Actor and the value function network Critic are built on the LSTM layer respectively; the policy network Actor is responsible for all possible actions a t ∈A(s t ) is scored and the action probability distribution is generated through the softmax function:

[0057] π θ (a t |s t )=softmax(A t W2·ReLU(W1·h t ))

[0058] Among them, W1 and W2 are learnable weight matrices, Indicates that the edge information in the causal graph is also encoded into each action a t In the embedding vector of

[0059] The value function network Critic outputs a scalar value to evaluate the state s t The value function of:

[0060]

[0061] Among them, W3 and W4 are learnable weight matrices.

[0062] As a preferred solution, the causal enhancement reinforcement learning module performs causal constraints based on guided prompts or structured prompts, guiding the large model to generate a thinking chain under causal constraints;

[0063] Guided prompts are to guide the generation of thought chains by outlining the specific steps to be followed in the thinking process of the big model;

[0064] Structured prompts refer to the use of extensible markup language (XML) tags to separate reasoning steps and final answers, helping large models to parse prompt words and thus generate thought chains.

[0065] As a preferred solution, the large model fine-tuning system based on causal diagram and thought chain enhancement further includes a large model fine-tuning module for executing large model fine-tuning actions;

[0066] When fine-tuning the large model, the thinking chain sample data generated under causal constraints are checked and verified with mechanism rules to ensure the credibility of the thinking chain sample data in the thinking process; according to the model distillation idea, the checked thinking chain sample data is used as cold start data to fine-tune the large model, so that the large model can learn the causal reasoning rules of the thinking chain sample data, and improve the logic, interpretability and robustness of the thinking process of the large model by optimizing and updating the reasoning strategy.

[0067] In a third aspect, an electronic device is provided, comprising a processor and a memory, wherein the processor is configured to execute a computer program stored in the memory to implement the large model fine-tuning method based on causal graphs and enhanced thinking chains.

[0068] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores at least one instruction, and when the at least one instruction is executed by a processor, the large model fine-tuning method based on causal diagram and thinking chain enhancement is implemented.

[0069] Compared with the prior art, the first aspect of the present invention has at least the following beneficial effects:

[0070] This invention, based on a large-scale model fine-tuning method enhanced by causal graphs and thought chains, conducts causal mining on relevant data such as power equipment operating information and fault analysis, constructs a causal graph for power equipment containing causal weight information, and dynamically updates the causal graph through causal consistency testing. The large-scale model input is decomposed into thought chains and chained causal pairs. The chained causal pairs are searched and matched with the causal graph, and causal consistency is checked to align the reasoning process. The alignment results incentivize the reinforcement learning process, optimize the pre-established reinforcement learning reward model, constrain the thought chain generation process, and continuously optimize and update the reasoning strategy. Through causal thought chain alignment, the invention dynamically aligns the thought chains obtained by parsing the large-scale model input with the causal graph containing causal knowledge, achieving step-by-step and precise retrieval of chained thought reasoning. A causal-enhanced reinforcement learning process is proposed. By embedding causal knowledge and causality into the reinforcement learning feedback process of large-scale model fine-tuning, the large-scale model's understanding of causal relationships and causal rules is enhanced. During the large-scale model fine-tuning phase, the causal graph and thought chain enhancement mechanism are combined to enable the large-scale model to learn basic causal reasoning rules and continuously optimize and update the reasoning strategy, improving the logic, interpretability, and robustness of the large-scale model's thinking process.

[0071] It can be understood that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0073] Figure 1 Flowchart of a large model fine-tuning method based on causal diagrams and enhanced thought chains according to an embodiment of the present invention;

[0074] Figure 2 Schematic diagram of the design architecture of a large model fine-tuning method based on causal diagrams and thought chain enhancement in an embodiment of the present invention;

[0075] Figure 3 The embodiment of the present invention is based on a structural block diagram of a large model fine-tuning system enhanced by a cause-effect diagram and a thinking chain. DETAILED DESCRIPTION

[0076] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0077] See also Figure 1 and Figure 2 The embodiment of the present invention considers combining causal graphs with the thought chain enhancement mechanism, embedding causal reasoning and causality into the reinforcement learning feedback process of large model fine-tuning, so that the large model can learn basic causal reasoning rules, continuously optimize and update the reasoning strategy, and improve the logic, interpretability and robustness of the thought chain reasoning.

[0078] The embodiment of the present invention provides a large model fine-tuning method based on a causal graph and enhanced thought chain, which mainly includes the following steps:

[0079] S1. Perform causal mining on power equipment data and construct a power equipment causal graph containing causal weight information;

[0080] S2. Decompose the input of the large model into thought chains and generate chained causal pairs based on the thought chains. Path retrieval and matching of these chained causal pairs and causal consistency checks are performed on the constructed power equipment causal graph to achieve alignment of the reasoning process.

[0081] S3. Inspire the reinforcement learning process through the alignment results of the reasoning process, optimize the pre-established reinforcement learning reward model, constrain the thought chain generation process, guide the large model to generate thought chains under causal constraints, and achieve fine-tuning of the large model.

[0082] In one possible implementation, step S1 performs entity extraction, relationship extraction, and graph construction based on relevant monitoring data and text data, such as power equipment operation information and fault analysis reports, to form an original knowledge graph G = (V, E), where V represents all nodes in the graph and E represents all edges in the graph. Each edge in the graph, edge = (u, r, v) ∈ E, is defined as a triple with a starting point u, an end point v, and a relationship r, where the relationship r includes relationship labels such as "related," "causal," and "belongs to."

[0083] The causal function Causality(r) is defined to calculate the causal numerical weight of the relationship r. If the weight is lower than the threshold θ, it is discarded and only strong weight edges with clear causal relationships are retained.

[0084] When considering new data to update causal information, the weight values ​​of the causal edges are continuously updated iteratively through causal screening and the PC algorithm. If the new weight of the edge (u, r, v) is greater than the threshold θ, the new weight edge is added or updated to the causal graph, and finally a causal graph of power equipment containing causal weight information is constructed.

[0085] In a possible implementation, step S2 performs thought chain decomposition and alignment, including the following steps:

[0086] S2.1. Thinking chain decomposition. When decomposing the input of the large model into thinking chains, prompt the input of the large model, requiring the large model to generate a specific format of thinking chain P = (P1, P2, ..., P N ); Each thinking chain reasoning step P i Generate a corresponding causal pair EP i =(start i ,end i ), start i Indicates causal relationship between cause entities, end i Represents cause-effect pairs of entities;

[0087] S2.2. Retrieve alignment. In the power equipment causal graph constructed in step S1, for the causal pair EP i =(start i ,end i ) Perform path search and matching, and get the starting point as start i 、End point is end i The candidate path set Path={path1,…,path m};

[0088] For each path, path scoring methods such as CUI overlap (CUI overlap), sentiment overlap (semantic type matching), and length (path length heuristic score) are used to evaluate the CUI overlap, semantic type matching, and path length heuristic score of the path and the causal pair, thereby evaluating the path;

[0089] CUI overlap is used to evaluate the degree of overlap between the concepts or entities involved in the path and the causal pair. If the nodes in the path (such as entities, concepts, commands, etc.) overlap with the nodes in the causal pair, then the overlap is high, indicating that the path is directly related to the causal pair. Semantic type matching is used to evaluate the consistency of the sentiment or semantic types involved in the path with the semantic types in the causal pair. This is often used in natural language processing or text analysis to ensure that the path matches the causal pair in sentiment or semantics. The path length heuristic score is used to evaluate the simplicity and effectiveness of the path. Generally, shorter paths are considered more effective because they involve fewer steps and transfer information more directly.

[0090] The total score is calculated based on the following formula, reflecting the relevance of the path to the causal pair:

[0091] Score(path)=αCUIoverlap(path)+βsemantic_overlap(path)+γLength(path)

[0092] Among them, α, β, and γ are the weight proportions of the three scoring methods: CUI overlap, semantic type matching, and path length heuristic score. Paths are sorted according to the total score, and the number of paths with a set ratio of keep_ratio×|Path| before ranking is retained.

[0093] In one possible implementation, step S3 stimulates the reinforcement learning process by using the alignment results of the reasoning process, and optimizing the pre-established reinforcement learning reward model specifically includes:

[0094] The chain causal pairs EP will be retrieved on the constructed power equipment causal graph i The process is modeled as a Markov decision process, which includes state space S, action space A, state transfer function δ and reward function R;

[0095] At each retrieval time t, the state of the agent is defined as s t =(EP,e t ,h t ,end)∈S, EP represents the causal pair of the thought chain reasoning steps retrieved, e t Represents the current entity vector, h t represents the hidden state of the visited path, and end represents the result entity of the retrieved causal pair; at the starting time t = 0, e0 is initialized to the cause entity start in the causal pair, and h0 is initialized to the all-zero vector;

[0096] In state s t Next, the action space is defined as the current entity e t The set of all neighbor nodes in the causal graph, namely A(s t)={e|(e t ,r,e)∈K}, where K represents the triple set of the causal graph and r represents the causal relationship;

[0097] At time t, the agent follows the policy π θ (a t |s t ) Execute action a t ∈A(s t ), the state is based on the transfer function δ(s t ,a t )Transfer to s t＋1 =(EP,e t+1 ,h t+1 ,end), where e t+1 =a t ;The environment is based on the reward function R(s t ,a t )=t t Feedback reward value; in the last step T-1, if e T =end means reaching the target node, and the environment returns a reward of 1, otherwise it returns 0; the rewards at other times are all 0;

[0098] Use the policy network-value function network hybrid Actor-Critic algorithm to train the policy network π θ (a t |s t ) and value function network The long short-term memory network LSTM is used to model state features, and at each step the retrieved thought chain reasoning step causal pair EP and the current entity vector e t Concatenate the input LSTM to generate the hidden state h t :

[0099]

[0100] The feedforward networks of the policy network Actor and the value function network Critic are built on the LSTM layer respectively; the policy network Actor is responsible for all possible actions a t ∈A(s t ) is scored and the action probability distribution is generated through the softmax function:

[0101] π θ (a t |s t )=softmax(A t W2·ReLU(W1·h t ))

[0102] Among them, W1 and W2 are learnable weight matrices, represents that the edge information in the causal graph is also encoded into each action a t embedding vector of the action a

[0103] The value function network Critic outputs a scalar value to evaluate the state s t value function of the state s

[0104]

[0105] where W3, W4 are learnable weight matrices.

[0106] Actor-Critic algorithm is a classic hybrid algorithm of policy gradient and value function method in reinforcement learning (RL), aiming to solve the high variance problem of traditional policy gradient method (such as REINFORCE reinforcement learning algorithm) and the limitation of pure value function method (such as Q-Learning) in handling continuous action space. The core idea is to realize a more efficient learning process through the collaborative work of Actor (policy network) and Critic (value function network) two modules.

[0107] In a possible implementation, step S3 performs causal constraints based on guided prompts or structured prompts, guiding the large model to generate the thought chain under the causal constraint condition;

[0108] Guided prompts refer to guiding the generation of thought chains by outlining specific steps to be followed by the large model in the thinking process;

[0109] Structured prompts refer to separating reasoning steps and final answers by using XML tags to help the large model parse prompt words and generate thought chains.

[0110] XML (eXtensible Markup Language) is a markup language used to store and transmit data. XML tags are the basic components of XML documents, used to define the structure and meaning of data. Unlike HTML tags, XML tags are user-defined and aim to describe the data itself, rather than control the way data is displayed.

[0111] In a possible implementation, in the step of implementing fine-tuning of the large model, the thought chain sample data generated under the causal constraint condition is checked and mechanism rule verified to ensure the credibility of the thought chain sample data in the thinking process; according to the model distillation idea, the checked thought chain sample data is used as cold start data to fine-tune the large model, so that the large model can learn the causal reasoning rules of the thought chain sample data, and through optimization and update of the reasoning strategy, the logicality, interpretability and robustness of the large model in the thinking process are improved.

[0112] Compared with the existing large model fine-tuning technology, the large model fine-tuning method based on the causal graph and thought chain enhancement of the embodiment of the application has outstanding advantages in the following aspects:

[0113] (1) Causal thought chain alignment: dynamically aligning the thought chain obtained by analyzing the large model input with the causal graph containing causal knowledge, and realizing step-by-step accurate retrieval of chain thought reasoning;

[0114] (2) Causal enhancement reinforcement learning: by embedding causal knowledge into the reinforcement learning feedback process of large model fine-tuning, the understanding ability of the large model to causal relationship and causal rule is improved;

[0115] (3) Large model fine-tuning: combined with the causal graph and thought chain enhancement mechanism, the large model can learn basic causal reasoning rules, continuously optimize and update the reasoning strategy, and improve the logicality, interpretability and robustness of the thinking process of the large model.

[0116] Please refer to Figure 3 Another embodiment of the application also proposes a large model fine-tuning system based on causal graph and thought chain enhancement, comprising:

[0117] The power equipment causal graph construction module 310 is used for causal mining of power equipment data, and constructs a power equipment causal graph containing causal weight information;

[0118] The causal thought chain alignment module 320 is used for decomposing the input of the large model into a thought chain, and generating a chain causal pair according to the thought chain, and performing path retrieval matching and causal consistency checking with the constructed power equipment causal graph through the chain causal pair, to realize alignment of the reasoning process;

[0119] The causal enhancement reinforcement learning module 330 is used for stimulating the reinforcement learning process through the alignment result of the reasoning process, optimizing the pre-established reinforcement learning reward model, constraining the thought chain generation process, guiding the large model to generate the thought chain under the causal constraint condition, and realizing the large model fine-tuning.

[0120] In a possible implementation, the power equipment causal graph construction module 310 performs entity extraction, relation extraction and graph construction on the power equipment data to form an original knowledge graph G=(V,E), wherein V is all nodes in the graph, and E is all edges in the graph; each relationship edge edge=(u,r,v)∈E in the graph is defined as a triple with u as the starting point, v as the ending point and r as the relationship; the relationship r includes three relationship labels of "related", "causal" and "belongs to"; the power equipment data includes power equipment operation information and fault analysis report;

[0121] Define the causal function Causality(r) to calculate the causal numerical weight of the relationship r. If the weight is lower than the threshold θ, it will be discarded.

[0122] The weight values ​​of causal edges are iteratively updated through the causal screening and causal discovery PC algorithm. If the new weight of the edge (u, r, v) is greater than the threshold θ, the new weight edge is added or updated to the causal graph to construct a causal graph of power equipment containing causal weight information.

[0123] In a possible implementation, when the causal thinking chain alignment module 320 decomposes the input of the large model into thinking chains, it prompts the input of the large model and requires the large model to generate a specific format of thinking chain P = (P1, P2, ..., P N ); Each thinking chain reasoning step P i Generate a corresponding causal pair EP i =(start i ,end i ), start i Indicates causal relationship between cause entities, end i Represents cause-effect pairs of entities;

[0124] In the constructed power equipment causal diagram, the causal pair EP i =(start i ,end i ) Perform path search and matching, and get the starting point as start i 、End point is end i The candidate path set Path={path1,…,path m};

[0125] The path scoring method is used to evaluate the CUI overlap, semantic type matching, and path length heuristic score between the path and the causal pair. The total score is calculated based on the following formula to reflect the relevance of the path to the causal pair:

[0126] Score(path)=αCUIoverlap(path)+βsemantic_overlap(path)+γLength(path)

[0127] Among them, α, β, and γ are the weight proportions of the three scoring methods: CUI overlap, semantic type matching, and path length heuristic score. Paths are sorted according to the total score, and the number of paths with a set ratio of keep_ratio×|Path| before ranking is retained.

[0128] In one possible implementation, the causal reinforcement reinforcement learning module 330 stimulates the reinforcement learning process by aligning the results of the inference process, and optimizes the pre-established reinforcement learning reward model, retrieves the chainwise causal pairs EP i from the constructed power equipment causal graph when modeling the process as a Markov decision process, which contains a state space S, an action space A, a state transition function δ, and a reward function R;

[0129] At each retrieval time t, the state of the agent is defined as s t =(EP,e t ,h t ,end)∈S, EP represents the retrieved chainwise inference step causal pair, e t represents the current entity vector, h t represents the hidden state of the visited path, and end represents the result entity of the retrieved causal pair; at the starting time t = 0, e0 is initialized as the reason entity start in the causal pair, and h0 is initialized as a zero vector;

[0130] In the state s t , the action space is defined as the set of all neighbor nodes of the current entity e t in the causal graph, that is, A(s t ) = {e | (e t ,r,e) = K}, where K represents the set of triples of the causal graph, and r represents the causal relationship;

[0131] At time t, the agent performs action a θ according to the strategy π t (a t |s t ), the state is transferred to s t according to the transition function δ(s t ,a t ), that is, s t =(EP,e t+1 ,h t+1 ,end), where e t+1 =a t+1 ; the environment feeds back the reward value according to the reward function R(s t ,a t ) = t t ; at the last step T-1, if e t =end represents reaching the target node, the environment returns a reward of 1, otherwise returns 0; the reward at other times is 0;

[0132] The strategy network π T (a θ |s t ) is trained using the hybrid Actor-Critic algorithm of the strategy network-value function network.t ) and value function network The long short-term memory network LSTM is used to model state features, and at each step the retrieved thought chain reasoning step causal pair EP and the current entity vector e t Concatenate the input LSTM to generate the hidden state h t :

[0133]

[0134] The feedforward networks of the policy network Actor and the value function network Critic are built on the LSTM layer respectively; the policy network Actor is responsible for all possible actions a t ∈A(s t ) is scored and the action probability distribution is generated through the softmax function:

[0135] π θ (a t |s t )=softmax(A t W2·ReLU(W1·h t ))

[0136] Among them, W1 and W2 are learnable weight matrices, Indicates that the edge information in the causal graph is also encoded into each action a t In the embedding vector of

[0137] The value function network Critic outputs a scalar value to evaluate the state s t The value function of:

[0138]

[0139] Among them, W3 and W4 are learnable weight matrices.

[0140] In one possible implementation, the causal enhancement reinforcement learning module 330 performs causal constraints based on guided prompts or structured prompts, guiding the large model to generate a thinking chain under causal constraints;

[0141] Guided prompts are to guide the generation of thought chains by outlining the specific steps to be followed in the thinking process of the big model;

[0142] Structured prompts refer to the use of extensible markup language (XML) tags to separate reasoning steps and final answers, helping large models to parse prompt words and thus generate thought chains.

[0143] In a possible implementation, the large model fine-tuning system based on the causal graph and thought chain enhancement of the embodiment of the present invention further includes a large model fine-tuning module 340 for performing large model fine-tuning actions;

[0144] When fine-tuning the large model, the thinking chain sample data generated under causal constraints are checked and verified with mechanism rules to ensure the credibility of the thinking chain sample data in the thinking process; according to the model distillation idea, the checked thinking chain sample data is used as cold start data to fine-tune the large model, so that the large model can learn the causal reasoning rules of the thinking chain sample data, and improve the logic, interpretability and robustness of the thinking process of the large model by optimizing and updating the reasoning strategy.

[0145] Another embodiment of the present invention further provides an electronic device, comprising a processor and a memory, wherein the processor is configured to execute a computer program stored in the memory to implement the large model fine-tuning method based on causal graph and thought chain enhancement.

[0146] Another embodiment of the present invention further proposes a computer-readable storage medium, which stores at least one instruction. When the at least one instruction is executed by a processor, it implements the large model fine-tuning method based on causal graph and thinking chain enhancement.

[0147] The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device, medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electric carrier signal, telecommunication signal and software distribution medium that can carry the computer program code. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals. For ease of explanation, the above content only shows the part related to the embodiment of the present invention. For specific technical details not disclosed, please refer to the method part of the embodiment of the present invention. The computer-readable storage medium is non-transitory and can be stored in a storage device formed by various electronic devices, and can implement the execution process recorded in the method of the embodiment of the present invention.

[0148] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0149] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flows and / or block diagram block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 one or more flowcharts and / or blocks

[0150] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart or flows and / or block diagram block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 one or more flowcharts and / or blocks

[0151] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flows and / or block diagram block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 one or more flowcharts and / or blocks

[0152] Finally, it should be noted that the above-mentioned embodiments are merely intended to illustrate the technical solutions of the present application, rather than limit the technical solutions of the present application. Although the present application has been described in detail with reference to the above-mentioned embodiments, those skilled in the art should understand that the specific embodiments of the present application can be modified or replaced, and any modification or replacement without departing from the spirit and scope of the present application should be covered within the protection scope of the claims of the present application.

Claims

1. A large model fine-tuning method based on causal diagram and thought chain enhancement, characterized by: include: Conduct causal mining on power equipment data and construct a causal graph of power equipment containing causal weight information; The input of the large model is decomposed into thought chains, and chain causal pairs are generated according to the thought chains. The chain causal pairs are then used to perform path retrieval matching and causal consistency checks with the constructed power equipment causal graph to achieve alignment of the reasoning process. The alignment results of the reasoning process are used to stimulate the reinforcement learning process, optimize the pre-established reinforcement learning reward model, constrain the thought chain generation process, guide the large model to generate thought chains under causal constraints, and achieve fine-tuning of the large model.

2. The large model fine-tuning method based on causal diagram and thought chain enhancement according to claim 1 is characterized in that: The method of performing causal mining on the power equipment data and constructing a power equipment causal graph containing causal weight information includes the following steps: Entity extraction, relationship extraction, and graph construction are performed on power equipment data to form an original knowledge graph G = (V, E), where V represents all nodes in the graph and E represents all edges in the graph. Each edge in the graph, edge = (u, r, v) ∈ E, is defined as a triple with a starting point u, an end point v, and a relationship r. The relationship r contains three relationship labels: "related", "causal", and "belongs to". Power equipment data includes power equipment operation information and fault analysis reports. Define the causal function Causality(r) to calculate the causal numerical weight of the relationship r. If the weight is lower than the threshold θ, it will be discarded. The weight values ​​of causal edges are iteratively updated through the causal screening and causal discovery PC algorithm. If the new weight of the edge (u, r, v) is greater than the threshold θ, the new weight edge is added or updated to the causal graph to construct a causal graph of power equipment containing causal weight information.

3. The large model fine-tuning method based on causal diagram and thought chain enhancement according to claim 1 is characterized in that: The process of decomposing the input of the large model into thought chains, generating chain causal pairs according to the thought chains, performing path retrieval matching and causal consistency check on the chain causal pairs and the constructed power equipment causal graph to achieve alignment of the reasoning process includes the following steps: When breaking down the input of the large model into thought chains, prompts are given to the input of the large model, requiring the large model to generate a thought chain P = (P1, P2, ..., P N ); Each thinking chain reasoning step P i Generate a corresponding causal pair EP i =(start i ,end i ), start i Indicates causal relationship between cause entities, end i Represents cause-effect pairs of entities; In the constructed power equipment causal diagram, the causal pair EP i =(start i ,end i ) Perform path search and matching, and get the starting point as start i 、End point is end i The candidate path set Path={path1,…,path m }; The path scoring method is used to evaluate the CUI overlap, semantic type matching, and path length heuristic score between the path and the causal pair. The total score is calculated based on the following formula to reflect the relevance of the path to the causal pair: Score(path)=αCUIoverlap(path)+βsemantic_overlap(path)+γLength(path) Among them, α, β, and γ are the weight proportions of the three scoring methods: CUI overlap, semantic type matching, and path length heuristic score. Paths are sorted according to the total score, and the number of paths with a set ratio of keep_ratio×|Path| before ranking is retained.

4. The large model fine-tuning method based on causal diagram and thought chain enhancement according to claim 1 is characterized in that: The method of stimulating the reinforcement learning process by using the alignment results of the reasoning process and optimizing the pre-established reinforcement learning reward model includes: The chain causal pairs EP will be retrieved on the constructed power equipment causal graph i The process is modeled as a Markov decision process, which includes state space S, action space A, state transfer function δ and reward function R; At each retrieval time t, the state of the agent is defined as s t =(EP,e t ,h t ,end)∈S, EP represents the causal pair of the thought chain reasoning steps retrieved, e t Represents the current entity vector, h t represents the hidden state of the visited path, and end represents the result entity of the retrieved causal pair; at the starting time t = 0, e0 is initialized to the cause entity start in the causal pair, and h0 is initialized to the all-zero vector; In state s t Next, the action space is defined as the current entity e t The set of all neighbor nodes in the causal graph, namely A(s t )={e|(e t ,r,e)∈K}, where K represents the triple set of the causal graph and r represents the causal relationship; At time t, the agent follows the policy π θ (a t |s t ) Execute action a t ∈A(s t ), the state is based on the transfer function δ(s t ,a t )Transfer to s t+1 =(EP,e t+1 ,h t+1 ,end), where e t+1 =a t ;The environment is based on the reward function R(s t ,a t )=t t Feedback reward value; in the last step T-1, if e T =end means reaching the target node, and the environment returns a reward of 1, otherwise it returns 0; the rewards at other times are all 0; Use the policy network-value function network hybrid Actor-Critic algorithm to train the policy network π θ (a t |s t ) and value function network The long short-term memory network LSTM is used to model state features, and at each step the retrieved thought chain reasoning step causal pair EP and the current entity vector e t Concatenate the input LSTM to generate the hidden state h t : The feedforward networks of the policy network Actor and the value function network Critic are built on the LSTM layer respectively; the policy network Actor is responsible for all possible actions a t ∈A(s t ) is scored and the action probability distribution is generated through the softmax function: π θ (to t |s t )=softmax(A t ·W2·ReLU(W1·h t )) Among them, W1 and W2 are learnable weight matrices, Indicates that the edge information in the causal graph is also encoded into each action a t In the embedding vector of The value function network Critic outputs a scalar value to evaluate the state s t The value function of: Among them, W3 and W4 are learnable weight matrices.

5. The large model fine-tuning method based on causal diagram and thought chain enhancement according to claim 1 is characterized in that: In the step of guiding the large model to generate a thinking chain under causal constraints during the generation of the constraint thinking chain, causal constraints are performed based on guided prompts or structured prompts to guide the large model to generate a thinking chain under causal constraints; Guided prompts are to guide the generation of thought chains by outlining the specific steps to be followed in the thinking process of the big model; Structured prompts refer to the use of extensible markup language (XML) tags to separate reasoning steps and final answers, helping large models to parse prompt words and thus generate thought chains.

6. The large model fine-tuning method based on causal diagram and thought chain enhancement according to claim 1 is characterized in that: In the steps of implementing the fine-tuning of the large model, the thought chain sample data generated under the causal constraint conditions are checked and verified with the mechanism rules to ensure the credibility of the thought chain sample data of the thinking process; according to the model distillation idea, the checked thought chain sample data is used as cold start data to fine-tune the large model, so that the large model can learn the causal reasoning rules of the thought chain sample data, and improve the logic, interpretability and robustness of the thinking process of the large model by optimizing and updating the reasoning strategy.

7. A large model fine-tuning system based on causal diagram and thought chain enhancement, characterized by: include: The power equipment causal graph construction module is used to perform causal mining on power equipment data and construct a power equipment causal graph containing causal weight information; The causal thinking chain alignment module is used to decompose the input of the large model into thinking chains and generate chained causal pairs based on the thinking chains. The chained causal pairs are then compared with the constructed power equipment causal graph for path retrieval and matching, and causal consistency checks are performed to achieve alignment of the reasoning process. The causal enhanced reinforcement learning module is used to stimulate the reinforcement learning process through the alignment results of the reasoning process, optimize the pre-established reinforcement learning reward model, constrain the thought chain generation process, guide the large model to generate thought chains under causal constraints, and realize large model fine-tuning.

8. The large model fine-tuning system based on causal diagram and thought chain enhancement according to claim 7 is characterized in that: The power equipment causal graph construction module extracts entities, extracts relationships, and constructs a graph from power equipment data to form an original knowledge graph G = (V, E), where V represents all nodes in the graph and E represents all edges in the graph. Each edge in the graph, edge = (u, r, v) ∈ E, is defined as a triple with a starting point u, an end point v, and a relationship r, where the relationship r contains three relationship labels: "related", "causal", and "belongs to". The power equipment data includes power equipment operation information and fault analysis reports. Define the causal function Causality(r) to calculate the causal numerical weight of the relationship r. If the weight is lower than the threshold θ, it will be discarded. The weight values ​​of causal edges are iteratively updated through the causal screening and causal discovery PC algorithm. If the new weight of the edge (u, r, v) is greater than the threshold θ, the new weight edge is added or updated to the causal graph to construct a causal graph of power equipment containing causal weight information.

9. The large model fine-tuning system based on causal diagram and thought chain enhancement according to claim 7 is characterized in that: When the causal thinking chain alignment module decomposes the input of the large model into thinking chains, it prompts the input of the large model and requires the large model to generate a specific format of thinking chain P = (P1, P2, ..., P N ); Each thinking chain reasoning step P i Generate a corresponding causal pair EP i =(start i ,end i ), start i Indicates causal relationship between cause entities, end i Represents cause-effect pairs of entities; In the constructed power equipment causal diagram, the causal pair EP i =(start i ,end i ) Perform path search and matching, and get the starting point as start i 、End point is end i The candidate path set Path={path1,...,path m }; The path scoring method is used to evaluate the CUI overlap, semantic type matching, and path length heuristic score between the path and the causal pair. The total score is calculated based on the following formula to reflect the relevance of the path to the causal pair: Score(path)=αCUIoverlap(path)+βsemantic_overlap(path)+γLength(path) Among them, α, β, and γ are the weight proportions of the three scoring methods: CUI overlap, semantic type matching, and path length heuristic score. Paths are sorted according to the total score, and the number of paths with a set ratio of keep_ratio×|Path| before ranking is retained.

10. The large model fine-tuning system based on causal diagram and thought chain enhancement according to claim 7 is characterized in that: The causal reinforcement learning module stimulates the reinforcement learning process through the alignment results of the reasoning process, and when optimizing the pre-established reinforcement learning reward model, it retrieves the chain causal pairs EP on the constructed power equipment causal graph. i The process is modeled as a Markov decision process, which includes state space S, action space A, state transfer function δ and reward function R; At each retrieval time t, the state of the agent is defined as s t =(EP,e t ,h t ,end)∈S, EP represents the causal pair of the thought chain reasoning steps retrieved, e t Represents the current entity vector, h t represents the hidden state of the visited path, and end represents the result entity of the retrieved causal pair; at the starting time t = 0, e0 is initialized to the cause entity start in the causal pair, and h0 is initialized to the all-zero vector; In state s t Next, the action space is defined as the current entity e t The set of all neighbor nodes in the causal graph, namely A(s t )={e|(e t ,r,e)∈K}, where K represents the triple set of the causal graph and r represents the causal relationship; At time t, the agent follows the policy π θ (a t |s t ) Execute action a t ∈A(s t ), the state is based on the transfer function δ(s t ,a t )Transfer to s t+1 =(EP,e t+1 ,h t+1 ,end), where e t+1 =a t ;The environment is based on the reward function R(s t ,a t )=t t Feedback reward value; in the last step T-1, if e T =end means reaching the target node, and the environment returns a reward of 1, otherwise it returns 0; the rewards at other times are all 0; Use the policy network-value function network hybrid Actor-Critic algorithm to train the policy network π θ (a t |s t ) and value function network The long short-term memory network LSTM is used to model state features, and at each step the retrieved thought chain reasoning step causal pair EP and the current entity vector e t Concatenate the input LSTM to generate the hidden state h t : The feedforward networks of the policy network Actor and the value function network Critic are built on the LSTM layer respectively; the policy network Actor is responsible for all possible actions a t ∈A(s t ) is scored and the action probability distribution is generated through the softmax function: π θ (to t |s t )=softmax(A t ·W2·ReLU(W1·h t )) Among them, W1 and W2 are learnable weight matrices, Indicates that the edge information in the causal graph is also encoded into each action a t In the embedding vector of The value function network Critic outputs a scalar value to evaluate the state s t The value function of: Among them, W3 and W4 are learnable weight matrices.

11. The large model fine-tuning system based on causal diagram and thought chain enhancement according to claim 7 is characterized in that: The causal enhancement reinforcement learning module performs causal constraints based on guided prompts or structured prompts, guiding the large model to generate a thinking chain under causal constraints; Guided prompts are to guide the generation of thought chains by outlining the specific steps to be followed in the thinking process of the big model; Structured prompts refer to the use of extensible markup language (XML) tags to separate reasoning steps and final answers, helping large models to parse prompt words and thus generate thought chains.

12. The large model fine-tuning system based on causal diagram and thought chain enhancement according to claim 7 is characterized in that: It also includes a large model fine-tuning module for performing large model fine-tuning actions; When fine-tuning the large model, the thinking chain sample data generated under causal constraints are checked and verified with mechanism rules to ensure the credibility of the thinking chain sample data in the thinking process; according to the model distillation idea, the checked thinking chain sample data is used as cold start data to fine-tune the large model, so that the large model can learn the causal reasoning rules of the thinking chain sample data, and improve the logic, interpretability and robustness of the thinking process of the large model by optimizing and updating the reasoning strategy.

13. An electronic device, characterized in that: The system comprises a processor and a memory, wherein the processor is used to execute a computer program stored in the memory to implement the large model fine-tuning method based on causal diagram and thinking chain enhancement as described in any one of claims 1 to 6.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium stores at least one instruction, and when the at least one instruction is executed by the processor, it implements the large model fine-tuning method based on causal diagram and thinking chain enhancement as described in any one of claims 1 to 6.

Images