Big language model reasoning method and big language model training method

Abstract

The invention provides a big language model training method, a big language model reasoning method and device, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence, in particular to the technical field of deep learning, big language models, natural language processing and content auditing. According to the specific implementation scheme, for input content, target nodes corresponding to the input content in an inference tree are determined, and the inference tree comprises multiple levels of nodes; according to the input content and the target node, thinking content, a thinking result and a reasoning path for reasoning the input content are generated, the thinking result is a target child node determined from multiple candidate child nodes of the target node, and the reasoning path is a path from the target node to the target child node; determining the correctness of the reasoning path according to the thinking content and the thinking result; and in response to determining that the reasoning path is incorrect, generating updated thinking content, thinking result and reasoning path according to the input content and the target node.

Description

Technical Field

[0001] The present disclosure relates to the fields of artificial intelligence technology, and in particular to the fields of deep learning, large language models, natural language processing, and content review technology. More specifically, the present disclosure provides a large language model inference method, a large language model training method, an apparatus, an electronic device, a storage medium, and a computer program product. Background Art

[0002] With the rapid development of artificial intelligence, large language models are becoming increasingly widely used. In particular, their reasoning capabilities have demonstrated significant results in various fields. Summary of the Invention

[0003] The present disclosure provides an inference method for a large language model, a training method for a large language model, an apparatus, an electronic device, a storage medium, and a computer program product.

[0004] According to a first aspect, a reasoning method for a large language model is provided, the method comprising: determining, for input content, a target node in an inference tree corresponding to the input content, wherein the inference tree comprises multi-level nodes; generating, based on the input content and the target node, thinking content, thinking results and reasoning paths for reasoning about the input content, wherein the thinking result is a target child node determined from a plurality of candidate child nodes of the target node, and the reasoning path is a path from the target node to the target child node; determining, based on the thinking content and the thinking result, the correctness of the reasoning path; and in response to determining that the reasoning path is incorrect, generating updated thinking content, thinking results and reasoning paths based on the input content and the target node.

[0005] According to a second aspect, a training method for a large language model is provided, the method comprising: determining, for sample content, a target node corresponding to the sample content in an inference tree, wherein the inference tree comprises multiple levels of nodes; generating, based on the sample content and the target node, thinking content, thinking results, and inference paths for reasoning about the sample content, wherein the thinking result is a target child node determined from multiple candidate child nodes of the target node, and the inference path is a path from the target node to the target child node; determining, based on the thinking content and the thinking result, the correctness of the inference path; in response to determining that the inference path is correct, determining the inference path as a historical path, determining the target child node as an updated target node, and returning to the step of generating, based on the sample content and the target node, thinking content, thinking results, and inference paths for reasoning about the sample content, until determining that the target node corresponding to the sample content in the inference tree is the last-level node, obtaining a multi-level historical path as the inference path of the large language model; and adjusting the parameters of the large language model according to the evaluation result of the inference path for the large language model.

[0006] According to a third aspect, an inference device for a large language model is provided, which includes: a first target node determination module, which is used to determine, for input content, a target node in an inference tree corresponding to the input content, wherein the inference tree includes multi-level nodes; a first generation module, which is used to generate, based on the input content and the target node, thinking content, thinking results and reasoning paths for reasoning about the input content, wherein the thinking result is a target child node determined from multiple candidate child nodes of the target node, and the reasoning path is a path from the target node to the target child node; a first reflection module, which is used to determine the correctness of the reasoning path based on the thinking content and the thinking result; and an update module, which is used to generate updated thinking content, thinking results and reasoning paths based on the input content and the target node in response to determining that the reasoning path is incorrect.

[0007] According to a fourth aspect, a training device for a large language model is provided, the device comprising: a second target node determination module for determining, for sample content, a target node in an inference tree corresponding to the sample content, wherein the inference tree comprises multiple levels of nodes; a second generation module for generating, based on the sample content and the target node, a thinking content, a thinking result, and an inference path for reasoning about the sample content, wherein the thinking result is a target child node determined from multiple candidate child nodes of the target node, and the inference path is a path from the target node to the target child node; a second reflection module for determining, based on the thinking content and the thinking result, the correctness of the inference path; an inference path determination module for, in response to determining that the inference path is correct, determining the inference path as a historical path and the target child node as an updated target node, and returning to the step of generating, based on the sample content and the target node, a thinking content, a thinking result, and an inference path for reasoning about the sample content, until determining that the target node corresponding to the sample content in the inference tree is the last-level node, and obtaining the multi-level historical path as the inference path of the large language model; and an adjustment module for adjusting the parameters of the large language model according to the evaluation result of the inference path for the large language model.

[0008] According to a fifth aspect, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method provided according to the present disclosure.

[0009] According to a sixth aspect, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to execute the method provided according to the present disclosure.

[0010] According to a seventh aspect, a computer program product is provided, comprising a computer program stored on at least one of a readable storage medium and an electronic device, wherein the computer program implements the method provided according to the present disclosure when executed by a processor.

[0011] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0013] Figure 1 is a schematic diagram of an exemplary system architecture to which a large language model inference method and a large language model training method can be applied according to an embodiment of the present disclosure;

[0014] Figure 2 is a flowchart of a method for reasoning a large language model according to an embodiment of the present disclosure;

[0015] Figure 3 is a schematic diagram of an inference method of a large language model according to an embodiment of the present disclosure;

[0016] Figure 4 is a flowchart of a method for training a large language model according to one embodiment of the present disclosure;

[0017] Figure 5 is a schematic diagram of a method for training a large language model according to an embodiment of the present disclosure;

[0018] Figure 6 is a block diagram of an inference apparatus for a large language model according to an embodiment of the present disclosure;

[0019] Figure 7 is a block diagram of a training apparatus for a large language model according to an embodiment of the present disclosure; and

[0020] Figure 8 The present invention is a block diagram of an electronic device according to at least one of a method for inferring a large language model and a method for training a large language model according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0021] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0022] In the areas of content review and content risk control, large language models can be used to determine the security of content. For example, the content to be reviewed can be directly input into the large language model, which can then determine the security category of the content.

[0023] However, large language models may also make misunderstandings. This method of directly allowing large language models to classify content lacks error correction capabilities and cannot guarantee the accuracy of the judgment results.

[0024] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0025] In the technical solution disclosed herein, the user's authorization or consent is obtained before obtaining or collecting the user's personal information.

[0026] Figure 1 This is a schematic diagram of an exemplary system architecture to which a large language model inference method and a large language model training method can be applied according to an embodiment of the present disclosure. It should be noted that, Figure 1 The examples shown are merely examples of system architectures to which the embodiments of the present disclosure may be applied, to help those skilled in the art understand the technical content of the present disclosure, but do not mean that the embodiments of the present disclosure may not be used in other devices, systems, environments or scenarios.

[0027] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0028] Users can use terminal devices 101, 102, 103 to interact with server 105 via network 104 to receive or send messages, etc. Terminal devices 101, 102, 103 can be various electronic devices, including but not limited to smartphones, tablet computers, laptop computers, etc.

[0029] Server 105 can be a server that provides various services, such as a backend management server (for example only) that supports websites browsed by users using terminal devices 101, 102, and 103. The backend management server can analyze and process received data such as requests and provide feedback to terminal devices 101, 102, and 103. Server 105 can be deployed with a pre-trained model, which can be a Large Language Model (LLM).

[0030] At least one of the large language model inference method and the large language model training method provided in the embodiments of the present disclosure can generally be executed by the server 105. Accordingly, the large language model inference device and the large language model training device provided in the embodiments of the present disclosure can generally be set in the server 105.

[0031] Figure 2 The figure is a flowchart of a method for inferring a large language model according to an embodiment of the present disclosure.

[0032] like Figure 2 As shown, the large language model inference method 200 includes operations S210 to S240.

[0033] In operation S210 , a target node corresponding to the input content in an inference tree is determined for the input content, wherein the inference tree includes multiple levels of nodes.

[0034] In operation S220 , based on the input content and the target node, a thinking content, a thinking result, and a reasoning path for reasoning on the input content are generated.

[0035] In operation S230 , the correctness of the reasoning path is determined based on the thinking content and the thinking result.

[0036] In operation S240 , in response to determining that the reasoning path is incorrect, updated thinking content, thinking results, and reasoning path are generated according to the input content and the target node.

[0037] The input content can be content awaiting review. The security of the content can be categorized as safe or unsafe. The unsafe category can include multiple types, and each category can be further divided into specific subcategories. Therefore, the security risk categories of the input content can be represented by a tree structure, with each node in the tree representing a category. The inference process used by the large language model to determine the security of the input content can be the process of determining nodes in the tree structure. Therefore, the tree structure can also be called an inference tree.

[0038] For example, an inference tree includes multiple levels of nodes. A first-level node can include "safe" and "unsafe." The child nodes (second-level nodes) of the "unsafe" node can include nodes A, B, and C, where nodes A, B, and C represent different risk categories within the "unsafe" category. Node A's child nodes (third-level nodes) can include nodes A1, A2, and A3, where nodes A1, A2, and A3 represent different risk subcategories within category A, and so on.

[0039] The large language model can determine the security of input content on a level-by-level basis. For example, the large language model first determines which level-one node the input content belongs to, that is, whether the input content is safe or unsafe. If it is safe, it can directly output a safe determination result. If it is unsafe, it can further determine which level-two node the input content belongs to, and so on. The final level-one node determined by the large language model can be the final classification result for the input content.

[0040] The large language model's discriminative process is also known as the inference process. Each level of reasoning generates thought steps (i.e., thought content), judgment results (i.e., thought outcomes), and reasoning paths. For example, when the large language model infers which of the first-level nodes the input content belongs to, it can generate a thought process that determines whether the input content is safe or unsafe. If the thought process at this level is "The input content contains sensitive words, and the input content poses a security risk," and the result is "The input content belongs to an unsafe node," since the reasoning at the first-level node is the first inference of the large language model, this level of reasoning can have no reasoning path.

[0041] For example, after the large language model infers the first-level node, it can infer which node in the second-level node the input content belongs to based on the first-level node. This level of reasoning can generate thinking content that identifies the input content as one of node A, node B, and node C. If the thinking content at this level is "the input content contains sensitive words XXX", and the thinking result is "the input content belongs to node A", the reasoning path is from unsafe node to node A. By analogy, the reasoning path of the large language model can be obtained. For example, the reasoning path of the large language model is unsafe node → node A → node A1.

[0042] The target node corresponding to the input content in the inference tree is the node to which the input content belongs as inferred by the large language model. After determining the target node to which the input content currently belongs, the thinking content and thinking results of the next level of the large language model can be generated based on the target node, and the thinking results can be the target child node determined from multiple candidate child nodes of the target node. For example, after determining that the input content belongs to an unsafe target node, the large language model next determines the target child node A from multiple candidate child nodes A, B, and C of the target node. Next, the target child node A can be used as the new target node, and the target child node can be determined from the child nodes A1, A2, and A3 of the target node A.

[0043] While the reasoning process of the aforementioned large language model can generate the thought content, results, and reasoning path for each step (i.e., each level), it lacks a reflection mechanism. Therefore, if the large language model makes a reasoning error, it cannot be corrected. In particular, reasoning errors in early reasoning steps can lead to the accumulation of subsequent reasoning errors, making it difficult to obtain a correct judgment result.

[0044] Therefore, after generating the reasoning path at each step, the embodiment of the present disclosure determines the accuracy of the reasoning path based on the thinking content and thinking results of that step. If it is determined that the reasoning path is wrong, an updated thinking content, thinking results and reasoning path are generated.

[0045] For example, when the large language model infers from node A to node A1, the logical consistency between the thinking content generated by inferring from node A to node A1 and the category represented by node A1 can be judged to determine the accuracy of the reasoning path "node A→node A1". If it is determined that the reasoning path "node A→node A1" is correct, the next step of reasoning can be performed based on node A1. If it is determined that the reasoning path "node A→node A1" is incorrect, it can return to node A and regenerate new thinking content, thinking results and reasoning path based on node A. For example, the new thinking result can be determined from nodes A2 and A3. For the new reasoning path, the step of determining the correctness of the reasoning path based on the thinking content and thinking results can be returned until the correct reasoning path is obtained. For example, the correct reasoning path can be "node A→node A2".

[0046] The embodiments of the present disclosure generate the thinking content, thinking results and reasoning path of the large language model reasoning, determine the accuracy of the reasoning path based on the thinking content and thinking results, so that the large language model has the ability of self-reflection. When it is determined that the reasoning path is wrong, it returns to the previous node and generates updated thinking content, thinking results and reasoning path. This enables the large model to have the ability of backtracking and error correction, find the wrong node, generate a new reasoning path, and improve the accuracy of reasoning.

[0047] According to an embodiment of the present disclosure, the above operation S230 includes determining the logical consistency of the thinking content and the thinking result based on the semantics of the thinking content and the semantics of the target sub-node represented by the thinking result; and determining the correctness of the reasoning path based on the logical consistency of the thinking content and the thinking result.

[0048] The semantic consistency between the semantics of the thought content and the semantics of the target child node represented by the thought result can represent the logical consistency of the thought content and the thought result. For example, the thought content from node A to node A1 is "the input content contains sensitive word XXX", and the thought result is "the input content belongs to node A". However, if "sensitive word XXX" is a sensitive word belonging to category A2, which is inconsistent with category A1 represented by the target child node A1, then the logical inconsistency between the thought content and the thought result can be determined to be an error in the reasoning path.

[0049] The embodiments of the present disclosure judge the logical accuracy of each step of the reasoning path, which can enable the large language model to have the ability of self-reflection, thereby improving the accuracy of the large language model reasoning.

[0050] Figure 3 Schematic diagram of a large language model inference method according to an embodiment of the present disclosure.

[0051] like Figure 3 As shown, an inference tree with a first-level node of the unsafe category is shown, wherein the unsafe node is marked with R, indicating risk.

[0052] According to an embodiment of the present disclosure, in response to determining that the reasoning path is correct, the reasoning path is determined to be a historical path, the target child node is determined to be an updated target node, and the steps of generating thinking content, thinking results and reasoning path for reasoning based on the input content and the target node are returned until it is determined that the target node corresponding to the input content in the reasoning tree is the last-level node.

[0053] According to an embodiment of the present disclosure, in response to determining that the reasoning path is incorrect, updated thinking content, thinking results and reasoning path are generated based on the input content and target node; for the updated thinking content, thinking results and reasoning path, the step of determining the correctness of the reasoning path based on the thinking content and thinking results is returned until the reasoning path is determined to be correct.

[0054] For example, after the large language model infers that the target node of the input content is node R, it infers the target child node A based on node R, generating the thought content and reasoning path from node R to node A. The correctness of the reasoning path from node R to node A is then determined based on the logical consistency between the thought content and node A. If the reasoning path from node R to node A is correct, the reasoning path from node R to node A is determined as the historical path. The target child node A is then determined as the new target node, and reasoning continues based on node A.

[0055] For example, the large language model infers the target child node A1 based on node A, generating the thought content and reasoning path from node A to node A1. If the thought content from node A to node A1 is inconsistent with the logic of node A1, it can be determined that the reasoning path from node A to node A1 is incorrect. Node A1 then returns to node A, and reasoning continues with node A as the target node, determining a new target child node and generating a new reasoning path.

[0056] For example, if the new reasoning path is from node A to node A2, and the new thinking content is consistent with the logic of node A2, then the reasoning path from node A to node A2 is correct. The reasoning path from node A to node A2 can be determined as the historical path, and node A2 can be used as the new target node to continue reasoning. Repeat the above reasoning process until the last level target node is obtained. The multi-level historical paths obtained during the reasoning process can constitute the reasoning path of the large language model. For example, the reasoning path of the large language model can be node R → node A → node A2 → node A22.

[0057] According to an embodiment of the present disclosure, a node represents a security risk category of input content, and a child node represents a security risk subcategory of the input content. Generating thought content, thought results, and a reasoning path for reasoning about the input content includes: generating thought content for security risk discrimination based on the semantics of the input content; determining, based on the thought content, a target category subnode corresponding to the input content from multiple candidate child nodes of the target category node as a thought result; and determining a reasoning path based on the target category node and the target category subnode.

[0058] For example, in the field of content review, the reasoning process of a large language model is the process of assessing the security of input content. Based on the semantics of the input content, this process generates intermediate reasoning steps, namely, the thought process. The final node (e.g., node A22) in the complete reasoning path generated by the large language model can be used to determine the final risk category of the input content.

[0059] The disclosed embodiments utilize multi-level reasoning within a large language model to generate intermediate steps and reasoning paths. This allows for reflection and error correction based on these intermediate steps, resulting in a complete and accurate reasoning path. This improves the accuracy of large language model reasoning. In scenarios involving auditing input content, this can improve the accuracy of audit results and the sophistication of category determinations.

[0060] Figure 4 4 is a flowchart of a method for training a large language model according to an embodiment of the present disclosure.

[0061] like Figure 4 As shown, the large language model training method 400 includes operations S410 to S460.

[0062] In operation S410 , a target node corresponding to the sample content in an inference tree is determined for the sample content, wherein the inference tree includes multiple levels of nodes.

[0063] In operation S420 , based on the sample content and the target node, a thinking content, a thinking result, and a reasoning path for reasoning on the sample content are generated. The thinking result is a target child node determined from a plurality of candidate child nodes of the target node.

[0064] In operation S430 , the correctness of the reasoning path is determined based on the thinking content and the thinking result.

[0065] In operation S440 , in response to determining that the inference path is correct, the inference path is determined as a historical path, and the target child node is determined as an updated target node.

[0066] In operation S450 , it is determined whether the target node is the last-level node. If so, the reasoning path of the large language model is obtained and operation S460 is executed. Otherwise, the process returns to execute operation S420 .

[0067] In operation S460 , parameters of the large language model are adjusted according to the evaluation result of the inference path for the large language model.

[0068] The above operations S410 to S440 are similar to the corresponding contents of the reasoning method of the above-mentioned large language model. In the training phase of this embodiment, the large language model also generates the thinking content, thinking results and reasoning path of each step of reasoning, and determines the accuracy of the reasoning path based on the logical consistency of the thinking content and thinking results. When the reasoning path is correct, the reasoning path is determined as the historical path, and the target sub-node indicated by the thinking result is determined as the new target node, and the reasoning continues. Until the last level of target node is inferred, the multi-level historical paths obtained in the reasoning process can constitute the reasoning path for the large language model to reason about the sample content.

[0069] Next, the inference path of the large language model can be evaluated. For example, the evaluation model can be used to evaluate the inference path of the large language model based on the reference path. The reference path can be a pre-annotated inference path for the sample content, and can include multiple intermediate steps in the inference process (i.e., reference content) and the inference results at each level (i.e., reference nodes).

[0070] For example, the inference path of a large language model can be evaluated based on the differences between it and a labeled reference path. The evaluation results can indicate the inference capability and accuracy of the large language model. Based on the evaluation results, the parameters of the large language model can be adjusted to achieve stronger inference capability and higher accuracy.

[0071] The embodiments of the present disclosure generate an inference path of a large language model. By evaluating the inference path of the large language model and adjusting the parameters of the large language model, the inference ability and inference accuracy of the large language model can be improved.

[0072] Figure 5 4 is a schematic diagram of a method for training a large language model according to an embodiment of the present disclosure.

[0073] like Figure 5 As shown, the first annotated data 501 may be a data set annotated with correspondences between sample content and categories. The categories may be security risk categories. The initial model 510 may be an initialized large language model. By training the initial model 510 using the first annotated data 501, a first model 520 may be obtained. The first model 520 may be a large language model that has learned knowledge in the field of content review.

[0074] The second annotated data 502 may be a dataset annotated with a reference path for the sample content. The reference path may include multiple levels of reasoning steps for determining the security risk category of the sample content, as well as the inference results of each level of reasoning steps. The content of each level of reasoning steps may serve as reference content, and the inference results of each level, i.e., the category nodes, may serve as reference nodes.

[0075] According to an embodiment of the present disclosure, an evaluation result is determined based on at least one of the difference between a multi-level reference path and a multi-level historical path, the difference between the reference content of a multi-level reference path and the thinking content of a multi-level historical path, and the difference between a reference node in a multi-level reference path and a target node in a multi-level historical path.

[0076] For example, the second data set 502 can be input into the first model 520 , so that the first model 520 generates a reasoning path and multiple levels of intermediate steps (ie, thinking content) and intermediate results (target nodes) included in the reasoning path based on the second data set 502 .

[0077] Next, the inference path generated by the first model 520 can be evaluated using an evaluation model or manually based on the reference path to obtain an evaluation result 503. The evaluation result 503 may include the difference between the reference path and the inference path, the difference between the reference content and the thought content, and the difference between the reference node and the target node. The loss of the first model 520 can be determined based on at least one of these differences, and the parameters of the first model 520 can be adjusted based on the loss to improve the inference capability and accuracy of the first model 520.

[0078] The aforementioned method of evaluating the reasoning path generated by first model 520 based on the reference path and adjusting the parameters of first model 520 based on the evaluation results is based on the principles of Proximal Policy Optimization (PPO) or Direct Preference Optimization (DPO). Compared to conventional PPO or DPO, this embodiment performs reinforcement learning or optimization on reasoning paths that include intermediate steps, thereby improving the reasoning capabilities of first model 520.

[0079] According to an embodiment of the present disclosure, the present disclosure also provides an inference device for a large language model and a training device for a large language model.

[0080] Figure 6 FIG. 4 is a block diagram of an inference apparatus for a large language model according to an embodiment of the present disclosure.

[0081] like Figure 6 As shown, the reasoning device 600 of the large language model includes a first target node determination module 610 , a first generation module 620 , a first reflection module 630 , and a first update module 640 .

[0082] The first target node determination module 610 is used to determine, with respect to the input content, a target node corresponding to the input content in the inference tree, wherein the inference tree includes multiple levels of nodes.

[0083] The first generation module 620 is used to generate thinking content, thinking results and reasoning paths for reasoning about the input content based on the input content and the target node, wherein the thinking result is a target child node determined from multiple candidate child nodes of the target node, and the reasoning path is a path from the target node to the target child node.

[0084] The first reflection module 630 is used to determine the correctness of the reasoning path based on the thinking content and thinking results.

[0085] The first updating module 640 is configured to generate updated thinking content, thinking results, and reasoning path according to the input content and the target node in response to determining that the reasoning path is incorrect.

[0086] According to an embodiment of the present disclosure, the inference apparatus 600 for a large language model further includes a first inference path determination module.

[0087] The first reasoning path determination module is used to, in response to determining that the reasoning path is correct, determine the reasoning path as a historical path, determine the target child node as an updated target node, and return to the steps of generating thinking content, thinking results and reasoning path for reasoning with respect to the input content based on the input content and the target node, until it is determined that the target node corresponding to the input content in the reasoning tree is the last-level node, and obtain a multi-level historical path as the reasoning path of the large language model.

[0088] The reasoning device 600 for a large language model further includes a first return module.

[0089] The first return module is used to return to the first reflection module 630 to execute the step of determining the correctness of the reasoning path based on the updated thinking content, thinking results and reasoning path, until the reasoning path is determined to be correct.

[0090] The first reflection module 630 includes a first reflection unit and a second reflection unit.

[0091] The first reflection unit is used to determine the logical consistency of the thinking content and the thinking result according to the semantics of the thinking content and the semantics of the target sub-node represented by the thinking result.

[0092] The second reflection unit is used to determine the correctness of the reasoning path based on the logical consistency of the thinking content and thinking results.

[0093] According to an embodiment of the present disclosure, a node represents a security risk category of input content, and a sub-node represents a security risk sub-category of input content. The first generating module 620 includes a thinking content generating unit, a thinking result generating unit, and a reasoning path generating unit.

[0094] The thinking content generating unit is used to generate thinking content for security risk judgment of the input content according to the semantics of the input content.

[0095] The thinking result generating unit is used to determine, according to the thinking content, a target category sub-node corresponding to the input content from a plurality of candidate sub-nodes of the target category node as a thinking result.

[0096] The reasoning path generating unit is used to determine a reasoning path according to the target category node and the target category sub-node.

[0097] Figure 74 is a block diagram of a large language model training apparatus according to an embodiment of the present disclosure.

[0098] like Figure 7 As shown, the large language model training device 700 includes a second target node determination module 710 , a second generation module 720 , a second reflection module 730 , a second reasoning path determination module 740 and an adjustment module 750 .

[0099] The second target node determination module 710 is used to determine, with respect to the sample content, a target node corresponding to the sample content in the inference tree, wherein the inference tree includes multiple levels of nodes.

[0100] The second generation module 720 is used to generate thinking content, thinking results and reasoning paths for reasoning about the sample content based on the sample content and the target node, wherein the thinking result is the target child node determined from multiple candidate child nodes of the target node, and the reasoning path is the path from the target node to the target child node.

[0101] The second reflection module 730 is used to determine the correctness of the reasoning path based on the thinking content and thinking results.

[0102] The second reasoning path determination module 740 is used to determine the reasoning path as a historical path in response to determining that the reasoning path is correct, determine the target child node as an updated target node, and return to the steps of generating thinking content, thinking results and reasoning path for reasoning based on the sample content and the target node, until it is determined that the target node corresponding to the sample content in the reasoning tree is the last-level node, and obtain a multi-level historical path as the reasoning path of the large language model.

[0103] The adjustment module 750 is configured to adjust parameters of the large language model according to the evaluation result of the inference path of the large language model.

[0104] The large language model training apparatus 700 further includes a second updating module and a second returning module.

[0105] The second updating module is configured to generate updated thinking content, thinking results, and reasoning path based on the sample content and the target node in response to determining that the reasoning path is incorrect;

[0106] The second return module is used to return to the second reflection module 730 based on the updated thinking content, thinking results and reasoning path, and to determine the correctness of the reasoning path based on the thinking content and thinking results until the reasoning path is determined to be correct.

[0107] The second reflection module 730 includes a third reflection unit and a fourth reflection unit.

[0108] The third reflection unit is used to determine the logical consistency of the thinking content and the thinking result according to the semantics of the thinking content and the semantics of the target sub-node represented by the thinking result.

[0109] The fourth reflection unit is used to determine the correctness of the reasoning path based on the logical consistency of the thinking content and thinking results.

[0110] The large language model training apparatus 700 further includes an evaluation module.

[0111] The evaluation module is used to evaluate the reasoning path of the large language model based on the reference path to obtain the evaluation result.

[0112] According to an embodiment of the present disclosure, the reference path includes multiple levels of reference paths, and each level of the reference path has corresponding reference content and reference nodes.

[0113] The evaluation module is used to determine the evaluation result based on at least one of the difference between the multi-level reference path and the multi-level historical path, the difference between the reference content of the multi-level reference path and the thinking content of the multi-level historical path, and the difference between the reference node in the multi-level reference path and the target node in the multi-level historical path.

[0114] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0115] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0116] like Figure 8 As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. Computing unit 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to bus 804.

[0117] Various components in device 800 are connected to I / O interface 805, including an input unit 806, such as a keyboard, mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, optical disk, etc.; and a communication unit 809, such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0118] The computing unit 801 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as at least one of the large language model inference method and the large language model training method. For example, in some embodiments, at least one of the large language model inference method and the large language model training method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of at least one of the large language model inference method and the large language model training method described above can be performed. Alternatively, in other embodiments, the computing unit 801 may be configured in any other appropriate manner (for example, by means of firmware) to execute at least one of the inference method of the large language model and the training method of the large language model.

[0119] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-a-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0120] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0121] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fibers, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0122] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0123] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0124] Computer systems may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The client and server relationship arises through computer programs running on the respective computers and having a client-server relationship to each other.

[0125] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0126] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for reasoning about a large language model, comprising: For the input content, determining a target node corresponding to the input content in an inference tree, wherein the inference tree includes multiple levels of nodes; generating, based on the input content and the target node, a thinking content, a thinking result, and a reasoning path for reasoning with respect to the input content, wherein the thinking result is a target child node determined from a plurality of candidate child nodes of the target node, and the reasoning path is a path from the target node to the target child node; Determining the correctness of the reasoning path based on the thinking content and thinking results; and In response to determining that the reasoning path is incorrect, an updated thinking content, thinking result, and reasoning path are generated based on the input content and the target node.

2. The method according to claim 1, further comprising: In response to determining that the reasoning path is correct, the reasoning path is determined as a historical path, the target sub-node is determined as an updated target node, and the step of generating thinking content, thinking results and reasoning paths for reasoning with respect to the input content based on the input content and the target node is returned, until it is determined that the target node corresponding to the input content in the reasoning tree is the last-level node, and a multi-level historical path is obtained as the reasoning path of the large language model.

3. The method according to claim 1, further comprising: For the updated thinking content, thinking result and reasoning path, return to the step of determining the correctness of the reasoning path based on the thinking content and thinking result until it is determined that the reasoning path is correct.

4. The method according to any one of claims 1 to 3, wherein Determining the correctness of the reasoning path based on the thinking content and thinking results includes: Determining the logical consistency of the thought content and the thought result based on the semantics of the thought content and the semantics of the target sub-node represented by the thought result; and The correctness of the reasoning path is determined based on the logical consistency of the thinking content and the thinking result.

5. The method according to any one of claims 1 to 3, wherein The node represents the security risk category of the input content, and the sub-node represents the security risk sub-category of the input content; The generating, based on the input content and the target node, a thinking content, a thinking result, and a reasoning path for reasoning about the input content includes: generating, based on the semantics of the input content, a thought content for performing security risk assessment on the input content; According to the thought content, determining a target category child node corresponding to the input content from a plurality of candidate child nodes of the target category node as a thought result; and The reasoning path is determined according to the target category node and the target category sub-node.

6. A method for training a large language model, comprising: For the sample content, determining a target node corresponding to the sample content in an inference tree, wherein the inference tree includes multiple levels of nodes; generating, based on the sample content and the target node, a thought content, a thought result, and a reasoning path for reasoning about the sample content, wherein the thought result is a target child node determined from a plurality of candidate child nodes of the target node, and the reasoning path is a path from the target node to the target child node; Determining the correctness of the reasoning path based on the thinking content and thinking results; In response to determining that the reasoning path is correct, determining the reasoning path as a historical path, determining the target child node as an updated target node, and returning to the step of generating, based on the sample content and the target node, a thinking content, a thinking result, and a reasoning path for reasoning with respect to the sample content, until determining that the target node corresponding to the sample content in the reasoning tree is the last-level node, thereby obtaining a multi-level historical path as the reasoning path of the large language model; and Adjust parameters of the large language model according to an evaluation result of the inference path of the large language model.

7. The method according to claim 6, further comprising: In response to determining that the reasoning path is incorrect, generating updated thinking content, thinking results, and reasoning path based on the sample content and the target node; For the updated thinking content, thinking result and reasoning path, return to the step of determining the correctness of the reasoning path based on the thinking content and thinking result until it is determined that the reasoning path is correct.

8. The method according to claim 6 or 7, wherein: Determining the correctness of the reasoning path based on the thinking content and thinking results includes: Determining the logical consistency of the thought content and the thought result based on the semantics of the thought content and the semantics of the target sub-node represented by the thought result; and The correctness of the reasoning path is determined based on the logical consistency of the thinking content and the thinking result.

9. The method according to claim 6, further comprising: The inference path of the large language model is evaluated according to the reference path to obtain the evaluation result.

10. The method according to claim 9, wherein: The reference path includes multiple levels of reference paths, and each level of reference path has corresponding reference content and reference nodes; The evaluating the inference path of the large language model according to the reference path to obtain the evaluation result includes: The evaluation result is determined based on at least one of the difference between the multi-level reference path and the multi-level historical path, the difference between the reference content of the multi-level reference path and the thinking content of the multi-level historical path, and the difference between the reference node in the multi-level reference path and the target node in the multi-level historical path.

11. A large language model inference device, comprising: A first target node determination module is configured to determine, based on the input content, a target node corresponding to the input content in an inference tree, wherein the inference tree includes multiple levels of nodes; a first generating module, configured to generate, based on the input content and the target node, a thinking content, a thinking result, and a reasoning path for reasoning with respect to the input content, wherein the thinking result is a target child node determined from a plurality of candidate child nodes of the target node, and the reasoning path is a path from the target node to the target child node; A first reflection module is used to determine the correctness of the reasoning path based on the thinking content and thinking results; and An updating module is configured to generate updated thinking content, thinking results, and reasoning path based on the input content and the target node in response to determining that the reasoning path is incorrect.

12. A large language model training device, comprising: a second target node determination module, configured to determine, with respect to the sample content, a target node corresponding to the sample content in an inference tree, wherein the inference tree includes multiple levels of nodes; a second generating module, configured to generate, based on the sample content and the target node, a thought content, a thought result, and a reasoning path for reasoning about the sample content, wherein the thought result is a target child node determined from a plurality of candidate child nodes of the target node, and the reasoning path is a path from the target node to the target child node; A second reflection module is used to determine the correctness of the reasoning path based on the thinking content and thinking results; an inference path determination module, configured to, in response to determining that the inference path is correct, determine the inference path as a historical path, determine the target child node as an updated target node, and return to the step of generating, based on the sample content and the target node, a thought content, a thought result, and a reasoning path for reasoning with respect to the sample content, until determining that the target node corresponding to the sample content in the inference tree is the last-level node, thereby obtaining a multi-level historical path as the inference path of the large language model; and An adjustment module is used to adjust parameters of the large language model according to an evaluation result of the inference path of the large language model.

13. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 10.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 10. 15 . A computer program product, comprising a computer program, wherein the computer program is stored on at least one of a readable storage medium and an electronic device, and when the computer program is executed by a processor, implements the method according to claim 1 .

Images