Inference method and related device

Abstract

An inference method and a device. The inference method comprises: acquiring input data; generating a plurality of pieces of prompt information on the basis of the input data, the plurality of pieces of prompt information being used for instructing a large language model to infer the input data from different perspectives; using each piece of prompt information among the plurality of pieces of prompt information as a root node, and invoking the large language model to construct a plurality of trees of thoughts, each tree of thoughts representing an inference path of the large language model for inferring the input data; and obtaining a target inference result of the input data on the basis of a plurality of reference inference results corresponding to the plurality of trees of thoughts. In the present application, a plurality of parallel trees of thoughts are set, each tree of thoughts represents a unique inference path, and thus by exploring in parallel the plurality of paths, a problem can be examined from multiple perspectives, thereby improving the accuracy and robustness of a target inference result of input data.

Description

A reasoning method and related apparatus

[0001] This application claims priority to Chinese Patent Application No. 2024117509582, filed on November 29, 2024, entitled “A Reasoning Method and Related Apparatus”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of artificial intelligence technology, and in particular to a reasoning method and related apparatus. Background Technology

[0003] In the field of Artificial Intelligence (AI), Large Language Models (LLMs) are deep learning-based models that learn language patterns, grammatical rules, semantic understanding, and logical reasoning abilities by processing and learning from large amounts of text data. They are widely used in tasks such as text generation, translation, dialogue systems, and question answering. The reasoning mechanism of LLMs in task processing can be understood as a step-by-step construction process to understand complex sentences or solve mathematical problems, similar to human thought processes. By simulating this step-by-step reasoning, LLMs can more effectively understand and respond to complex problems. Therefore, a reasoning method is needed to further improve the understanding capabilities of LLMs. Summary of the Invention

[0004] To address the aforementioned problems, embodiments of this application provide a reasoning method and related apparatus, which can improve the ability of large language models to understand input data.

[0005] Therefore, the following technical solutions are adopted in the embodiments of this application:

[0006] In a first aspect, embodiments of this application provide a reasoning method, comprising: acquiring input data; generating multiple prompts based on the input data, the multiple prompts being used to guide a large language model to reason about the input data from different perspectives; using each of the multiple prompts as a root node, calling the large language model to construct multiple thought trees, the thought trees representing the reasoning path of the large language model in reasoning about the input data; and obtaining the target reasoning result of the input data based on the multiple reference reasoning results corresponding to the multiple thought trees.

[0007] In this embodiment, multiple prompts from different perspectives are obtained based on the input data. Then, a large language model is guided to construct a thought tree corresponding to each prompt. Based on the thought tree of each prompt, a reference reasoning result for each prompt can be obtained. Based on the reference reasoning results of multiple prompts, the target reasoning result for the input data is determined. This reasoning framework is equivalent to multiple parallel thought trees, which can be understood as a kind of thought forest. In the structure of this thought forest, each thought tree symbolizes a unique reasoning path, thus exploring multiple paths in parallel. This thought forest reasoning framework can examine the problem from multiple angles, thereby improving the accuracy and robustness of the target reasoning result for the input data. Furthermore, this multi-path reasoning mechanism enables the large language model to capture information from various aspects of the input data from different dimensions, making the target reasoning result for the input data more comprehensive and accurate. It avoids problems such as fixed thinking patterns and local optima caused by a single thought tree. The thought forest can easily solve and reason about problems by "changing perspectives" or "changing approaches."

[0008] In one possible implementation, the above uses each of the multiple prompts as the root node, calls the large language model, and constructs multiple thought trees, including: in the process of constructing each thought tree, obtaining the root node of the thought tree based on the prompts; inputting the prompts into the large language model to output the corresponding response content; and obtaining the child nodes corresponding to the root node based on the response content.

[0009] This implementation provides the process for constructing each thought tree. The prompt information is used as the root node of the thought tree; the prompt information is input into a large language model to output the response; and the response is used as the child node corresponding to the root node. This implementation utilizes the analytical capabilities of the large language model to construct the nodes of the thought tree. It should be noted that generating the child nodes of the root node is merely an example of using a large language model to construct the nodes of the thought tree. This implementation can be considered applicable to the generation process of any node, such as using the large language model to generate the child nodes of a given child node.

[0010] In one possible implementation, the method further includes: correcting child nodes according to preset error correction rules to obtain corrected child nodes.

[0011] In this implementation, during the construction of the mind tree, after the large language model generates child nodes, the child nodes are corrected by pre-setting error correction rules. This can prevent erroneous child nodes from entering the next step of the calculation, thereby reducing the amount of computation, improving computational efficiency, and saving computational resources.

[0012] In one possible implementation, the aforementioned preset error correction rules include fixed rules, which are obtained through existing rule requirements of the input data. The process of correcting child nodes according to the preset error correction rules to obtain corrected child nodes includes: determining whether a child node conforms to an inherent rule; when a child node does not conform to an inherent rule, correcting the child node according to the inherent rule to obtain a corrected child node.

[0013] In this implementation, child nodes generated by a large language model can be quickly corrected according to fixed rules. These fixed rules represent the relevant rules corresponding to the input data, such as the rules of Go in a Go scenario or the rules of Chinese chess in a chess scenario. Fixed rule error correction is fast, preventing erroneous information from nodes from entering subsequent calculations and saving computational resources.

[0014] In one possible implementation, the above-mentioned correction of child nodes according to preset error correction rules to obtain corrected child nodes includes: calling a large language model to score the child nodes to obtain a score for the child nodes; determining whether the score of the child nodes is less than a preset score threshold; when the score of the child nodes is less than the preset score threshold, calling the large language model so that the large language model can generate corrected child nodes based on historical failure cases and child nodes.

[0015] In this implementation, the learning ability of a large language model to study historical failure cases can be used to correct child nodes. Specifically, child nodes can be selected from a pool of child nodes based on the large language model's scores. Therefore, different score thresholds can be set to determine the number of child nodes requiring correction. In other words, by adjusting the score thresholds, the frequency of error correction by the large language model can be dynamically adjusted, improving correction efficiency and enhancing the robustness and adaptability of the large language model in complex reasoning tasks.

[0016] In one possible implementation, the method further includes: determining whether a child node meets a preset stopping condition; and stopping the construction of branches for the child node when the child node meets the preset stopping condition.

[0017] This implementation provides a sparse activation method for the thought forest. This means activating only a portion of the thought trees or nodes within those trees, avoiding unnecessary computations and wasting computational resources. Furthermore, if the node is an intermediate node, construction of branches below it is stopped, discarding subtrees related to the intermediate node; if the node is a leaf node, construction of that leaf node is stopped, discarding subtrees with that leaf node as the reasoning result; if the leaf node is the only leaf node, construction of that leaf node is stopped, discarding the entire thought tree.

[0018] In one possible implementation, the aforementioned preset stopping condition includes at least one of a first condition and a second condition, wherein the first condition indicates that the score of the child node is lower than a preset threshold, and the second condition indicates that the information of the child node contains errors.

[0019] This implementation provides an example of a preset stop condition. However, this does not constitute a limitation on the preset stop condition itself.

[0020] In one possible implementation, the above method further includes: when there are multiple child nodes, calling a large language model to score the multiple child nodes to obtain scores for the multiple child nodes; and retaining the target number of child nodes based on the scores of the multiple child nodes.

[0021] This implementation provides an alternative sparse activation method for generating mind trees. This sparse activation method is equivalent to a pruning method, retaining only the child nodes with higher scores, reducing computational load, improving computational efficiency, and saving computational resources.

[0022] In one possible implementation, obtaining the target reasoning result of the object to be reasoned based on multiple reference reasoning results corresponding to multiple mind trees includes: obtaining the target reasoning result from multiple reference reasoning results according to a preset strategy.

[0023] This implementation provides a preset strategy. Based on this preset strategy, the target inference result can be obtained from multiple reference inference results. The preset strategy can be a combination of multiple strategies, thereby improving the accuracy of the target inference result.

[0024] In one possible implementation, the aforementioned preset strategy includes a first strategy and a second strategy. The first strategy represents a strategy based on the voting principle, and the second strategy represents a strategy based on judgment by a large language model. Obtaining the target inference result from multiple reference inference results according to the preset strategy includes: voting on the multiple reference inference results according to the first strategy to obtain a voting result; invoking the large language model according to the second strategy to judge the multiple reference inference results to obtain a judgment result; and obtaining the target inference result from the multiple reference inference results based on the voting result and the judgment result.

[0025] This implementation provides a reasoning strategy that combines voting principles and expert judgment. Two strategies are used to determine whether the target reasoning result for the input data exists among the reference reasoning results of multiple prompts. Upon obtaining the target reasoning result for the input data, the reasoning process for that input data is halted. The first strategy is equivalent to a strategy embodying collective wisdom (a majority consensus voting strategy), ensuring broad participation of opinions. The second strategy is equivalent to an expert judgment strategy (such as a selection strategy using a large language model), guaranteeing the professionalism and high quality of the result. Furthermore, by synthesizing the reference reasoning results from different prompts, the large language model can stop the reasoning process after reaching a certain confidence level, ensuring the optimal decision result is obtained with limited resources.

[0026] In one possible implementation, obtaining the target inference result from multiple reference inference results based on the voting results and the judgment results includes: when the voting results conform to the majority consensus principle, obtaining the target inference result from multiple reference inference results based on the voting results; when the voting results do not conform to the majority consensus principle, obtaining the target inference result from multiple reference inference results based on the judgment results.

[0027] This implementation provides a specific example of a reasoning strategy that combines voting principles and expert judgment, but does not constitute a limitation on its implementation.

[0028] In one possible implementation, the aforementioned preset strategy further includes a third strategy, which represents a strategy based on real data, where the real data is data related to the target inference result. The process of obtaining the target inference result from multiple reference inference results according to the preset strategy includes: matching multiple reference inference results with real data according to the third strategy; and obtaining the target inference result from the multiple reference inference results based on the matching results.

[0029] This implementation provides a third strategy. Based on this third strategy, the target inference result for the input data can be obtained from multiple reference inference results. When the reference inference results of multiple prompts are related to the real data, the system can determine whether to stop the inference process and obtain the target inference result for the input data based on the real data.

[0030] In one possible implementation, the method further includes: determining whether the nodes in the mind tree conform to preset processing rules; when the nodes in the mind tree conform to the preset processing rules, generating child nodes corresponding to the nodes in the mind tree according to the preset processing rules.

[0031] This implementation provides another method for generating child nodes. Some child nodes can be generated through preset processing rules. In this case, it is not necessary to use a large language model to generate child nodes, thereby reducing the amount of computation, improving computational efficiency, and saving computing resources.

[0032] In one possible implementation, the above-mentioned generation of multiple prompt messages based on input data includes: combining multiple prompt word templates with the input data to obtain corresponding multiple prompt messages.

[0033] In this implementation, multiple prompts are obtained through multiple prompt word templates with different content, which can better guide the large language model to analyze the input data from different perspectives. In this way, the large language model can examine the problem from multiple perspectives, thereby improving the accuracy and robustness of the target inference results of the input data.

[0034] Secondly, embodiments of this application provide a reasoning device, comprising: an acquisition module for acquiring input data; a processing module for generating multiple prompts based on the input data, the multiple prompts being used to guide a large language model to reason about the input data from different perspectives, such as through different prompt word templates; and, using each of the multiple prompts as a root node, calling the large language model to construct multiple thought trees, the thought trees representing the reasoning path of the large language model in reasoning about the input data; and, obtaining the target reasoning result of the input data based on the multiple reference reasoning results corresponding to the multiple thought trees.

[0035] In one possible implementation, the above processing module is specifically used to: obtain the root node of the thought tree based on the prompt information during the construction of each thought tree; input the prompt information into the large language model to output the response content corresponding to the prompt information; and obtain the child nodes corresponding to the root node based on the response content.

[0036] In one possible implementation, the above processing module is further used to: correct the child nodes according to preset error correction rules to obtain the corrected child nodes.

[0037] In one possible implementation, the aforementioned preset error correction rules include fixed rules, which are obtained through existing rule requirements of the input data. Specifically, the aforementioned processing module is used to: determine whether a child node conforms to the inherent rules; when a child node does not conform to the inherent rules, correct the child node according to the inherent rules to obtain the corrected child node.

[0038] In one possible implementation, the above processing module is specifically used to: call the large language model to score the child nodes to obtain the scores of the child nodes; determine whether the scores of the child nodes are less than a preset score threshold; when the scores of the child nodes are less than the preset score threshold, call the large language model so that the large language model can generate corrected child nodes based on historical failure cases and child nodes.

[0039] In one possible implementation, the above processing module is further used to: determine whether the child node meets the preset stopping condition; when the child node meets the preset stopping condition, stop building the branch of the child node.

[0040] In one possible implementation, the aforementioned preset stopping condition includes at least one of a first condition and a second condition, wherein the first condition indicates that the score of the child node is lower than a preset threshold, and the second condition indicates that the information of the child node contains errors.

[0041] In one possible implementation, the above processing module is further configured to: when there are multiple child nodes, call the large language model to score the multiple child nodes to obtain scores for the multiple child nodes; and retain the target number of child nodes based on the scores of the multiple child nodes.

[0042] In one possible implementation, the above processing module is specifically used to: obtain the target inference result from multiple reference inference results according to a preset strategy.

[0043] In one possible implementation, the aforementioned preset strategy includes a first strategy and a second strategy. The first strategy represents a strategy based on the voting principle, and the second strategy represents a strategy based on the judgment of a large language model. Specifically, the aforementioned processing module is used to: vote on multiple reference inference results according to the first strategy to obtain a voting result; invoke the large language model according to the second strategy to judge the multiple reference inference results to obtain a judgment result; and obtain the target inference result from the multiple reference inference results based on the voting result and the judgment result.

[0044] In one possible implementation, the above processing module is specifically used to: when the voting result conforms to the majority consensus principle, obtain the target inference result from multiple reference inference results based on the voting result; when the voting result does not conform to the majority consensus principle, obtain the target inference result from multiple reference inference results based on the judgment result.

[0045] In one possible implementation, the aforementioned preset strategy further includes a third strategy, which represents a strategy based on real data, where the real data is data related to the target inference result. Specifically, the aforementioned processing module is used to: match multiple reference inference results with the real data according to the third strategy; and obtain the target inference result from the multiple reference inference results based on the matching results.

[0046] In one possible implementation, the above processing module is further used to: determine whether the nodes in the mind tree conform to the preset processing rules; when the nodes in the mind tree conform to the preset processing rules, generate the child nodes corresponding to the nodes in the mind tree according to the preset processing rules.

[0047] In one possible implementation, the above processing module is specifically used to: obtain multiple corresponding prompt messages by combining multiple prompt word templates according to the input data.

[0048] Thirdly, this application provides a chip system including a processor and a power supply circuit, the power supply circuit being used to supply power to the processor, and the processor being used to execute the method described in any possible implementation of the first aspect and / or the algorithmic functions embodied by the apparatus described in any possible implementation of the second aspect.

[0049] Fourthly, this application provides a computing device including a processor and a memory. The processor is used to execute instructions stored in the memory to cause the computing device to perform the methods described in any possible implementation of the first aspect and / or the algorithmic functions embodied by the means described in any possible implementation of the second aspect.

[0050] Fifthly, this application provides a computing device cluster, including at least one computing device, each computing device including a processor and a memory; the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster performs the method described in any possible implementation of the first aspect and / or the algorithmic function embodied by the apparatus described in any possible implementation of the second aspect.

[0051] In a sixth aspect, this application provides a computer-readable storage medium including computer program instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method described in any possible implementation of the first aspect and / or the algorithmic function embodied by the apparatus described in any possible implementation of the second aspect, wherein the cluster of computing devices includes at least one computing device.

[0052] In a seventh aspect, this application provides a computer program product containing instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method described in any possible implementation of the first aspect and / or the algorithmic function embodied by the apparatus described in any possible implementation of the second aspect, wherein the cluster of computing devices includes at least one computing device.

[0053] It is understood that the beneficial effects of the second to seventh aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0054] The accompanying drawings used in the embodiments or technical description are briefly introduced below.

[0055] Figure 1 is a schematic diagram of the composition of a reasoning system provided in an embodiment of this application;

[0056] Figure 2 is a flowchart illustrating a reasoning method provided in an embodiment of this application;

[0057] Figure 3 is a schematic diagram of a mind forest provided in an embodiment of this application;

[0058] Figure 4 is a schematic diagram of an example of correcting child nodes according to inherent rules provided in the embodiments of this application;

[0059] Figure 5 is a schematic diagram of an example of correcting child nodes according to learnable rules provided in the embodiments of this application;

[0060] Figure 6 is a flowchart illustrating an example of a reasoning method provided in an embodiment of this application;

[0061] Figure 7 is a schematic diagram of the relevant optimizations to the reasoning process of a single mind tree provided in the embodiments of this application;

[0062] Figure 8 is a schematic diagram of the computational load required to achieve different accuracies for the thought forest and thought tree provided in the embodiments of this application;

[0063] Figure 9 is a schematic diagram of the accuracy changes corresponding to different strategies of the thinking forest provided in the embodiments of this application;

[0064] Figure 10 is a schematic diagram of the composition of a reasoning device provided in an embodiment of this application;

[0065] Figure 11 is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0066] Figure 12 is a schematic diagram of the structure of a computing device cluster provided in an embodiment of this application;

[0067] Figure 13 is a schematic diagram of another computing device cluster provided in an embodiment of this application. Detailed Implementation

[0068] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0069] In this article, the term "and / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three cases: A exists alone, A and B exist simultaneously, and B exists alone. The symbol " / " in this article indicates that the related objects have an "or" relationship; for example, A / B means A or B.

[0070] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.

[0071] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0072] In the description of the embodiments in this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, etc.; multiple elements means two or more elements, etc. "Several" means one or more.

[0073] To facilitate understanding of the solutions provided in the embodiments of this application, a brief introduction to some of the terms involved in this solution will be given first.

[0074] Large Language Models (LLMs): Large language models are natural language processing (NLP) models trained using deep learning techniques. They are capable of understanding, generating, and manipulating human language. These models learn language patterns, grammatical rules, semantic understanding, and logical reasoning abilities by processing and learning from large amounts of text data. They are widely used in tasks such as text generation, translation, dialogue systems, and question answering. Large language models are also known as large models or large language models.

[0075] Thinking: The basic semantic unit in the task-solving process. In the task-solving process, a step can be a statement, a plan, a piece of text, a set of documents, or a series of numbers, etc.

[0076] Chain of Thought (COT): This term describes a reasoning mechanism used by Large Language Models (LLMs) when processing tasks. In Natural Language Processing (NLP), a model might understand complex sentences or solve mathematical problems through a step-by-step construction process, similar to a human thought chain. By simulating this step-by-step reasoning, Artificial Intelligence (AI) systems can more effectively understand and respond to complex problems.

[0077] Tree of Thought (TOT): This application proposes a method for organizing and representing reasoning processes, primarily used in Large Language Models (LLMs) for path exploration during complex reasoning. It simulates the branching structure of human thought, gradually generating multiple reasoning directions and possible answers when solving problems. Unlike linear "chains of thought," the "tree of thought" emphasizes starting from a single point, allowing thought to develop in multiple different directions, forming a tree-like structure.

[0078] Forest of Thought (FOT): A forest of thought is a method for exploring multiple independent reasoning paths in parallel. Unlike mind trees, forest of thought does not require branches to develop from the same initial node, but rather allows for simultaneous attempts at multiple reasoning directions, much like exploring the thought process across multiple trees.

[0079] Monte Carlo Tree Search (MCTS), also known as random sampling or statistical experimentation, is a branch of computational mathematics. It was developed in the mid-1940s to meet the needs of the atomic energy industry at the time. Traditional empirical methods, unable to approximate real physical processes, rarely yield satisfactory results. In contrast, Monte Carlo Tree Search realistically simulates actual physical processes, thus providing solutions that closely resemble reality and yielding highly satisfactory results. This computational method, based on probability and statistical theory, uses random numbers (or more commonly, pseudo-random numbers) to solve many computational problems. It connects the problem to a certain probability model and uses a computer to perform statistical simulation or sampling to obtain an approximate solution.

[0080] The Twenty-Four Math Problem: Given four integers ranging from one to thirteen, calculate the result of a problem involving addition, subtraction, multiplication, and division to arrive at the number twenty-four. The Twenty-Four Math Problem asks you to determine which addition, subtraction, multiplication, and division operations, given four integers, will result in a total of twenty-four. This problem is also known as the Game of 24.

[0081] Ground truth (GT) early stopping mechanism: Also known as truth value early stopping mechanism, this refers to a mechanism that uses the true value as the stopping condition. Taking the 24 math problem as an example, the stopping condition can be the true number "24", which is an example of a GT early stopping mechanism.

[0082] Input data refers to any information or data to be inferred. For example, in the field of intelligent customer service, input data could be a user's voice information, which is then used to design multi-turn dialogue processes based on a thought forest, enabling intelligent customer service to better understand user needs and provide more accurate services. In the financial field, input data could be an applicant's background information and credit score, used to build a thought forest between the applicant's background information and credit score; by constructing a thought forest, financial institutions can use diverse inference results to assist in analyzing transaction patterns and abnormal behaviors, quickly identifying potential fraudulent activities, and protecting platform security. In the medical field, input data could be information such as a patient's disease and symptoms, used to build a thought forest model between diseases and symptoms; by constructing a thought forest, diverse inference results from information such as the user's lifestyle habits and genetic factors can be analyzed, assisting doctors in quickly selecting the most suitable treatment plan based on the patient's specific condition, improving diagnostic accuracy and treatment efficiency. In the field of autonomous driving, input data could be road condition information, used to formulate vehicle route planning and decisions; by constructing a thought forest, autonomous driving systems can analyze road conditions in real time, make optimal driving decisions, and improve driving safety and efficiency.

[0083] In the field of Artificial Intelligence (AI), Large Language Models (LLMs) are deep learning-based models that learn language patterns, grammatical rules, semantic understanding, and logical reasoning abilities by processing and learning from large amounts of text data. They are widely used in tasks such as text generation, translation, dialogue systems, and question answering. The reasoning mechanism of LLMs in task processing can be understood as a step-by-step construction process to understand complex sentences or solve mathematical problems, similar to human thought processes. By simulating this step-by-step reasoning, LLMs can more effectively understand and respond to complex problems. Therefore, a reasoning method is needed to further improve the understanding capabilities of LLMs.

[0084] Some solutions offer methods such as Chain of Thought (CoT), ToT (ToT), and Monte Carlo Tree Search (MCTS). In these solutions, large language models can decompose complex problems into multiple hints and solve them step-by-step. However, these solutions are prone to getting stuck in a fixed mindset when dealing with single problems and may not be able to handle non-linear or complex multi-path problems. For example, while Chain of Thought (CoT) improves the inference performance of large language models to some extent, it may not be able to handle non-linear or complex mathematical and logical reasoning problems. As the complexity of the problem increases, the number of branches in ToT may grow exponentially, leading to a significant increase in computational and memory requirements. However, due to the uniqueness of the input, the inference results still lack diversity and are prone to getting stuck in a fixed mindset, preventing further improvement in accuracy. In Monte Carlo Tree Search (MCTS) solutions, performance bottlenecks may still be encountered in extremely large search spaces, with multiple trees getting trapped in local optima.

[0085] In view of this, embodiments of this application provide a reasoning method that constructs a thought forest to diversify reasoning path searches. In this structure, the thought forest comprises multiple thought trees, each representing a unique reasoning path. By exploring multiple paths in parallel, the framework can examine the problem from multiple perspectives, thereby improving the accuracy and robustness of the reasoning. Furthermore, this multi-path reasoning mechanism enables the model to capture various aspects of the problem from different dimensions.

[0086] In one possible implementation, the inference method of this application embodiment also incorporates sparse activation strategies, fixed rule-based and learnable dynamic error correction strategies, and flexible and diverse strategies, aiming to improve the accuracy and efficiency of solving complex problems.

[0087] For example, compared to the Chain of Thought (CoT) solution, the embodiments of this application explore multiple paths in parallel through multiple thought trees of a thought forest, which can analyze nonlinear or complex mathematical and logical reasoning problems from multiple perspectives and improve the accuracy of reasoning results for nonlinear or complex problems.

[0088] For example, compared to the ToT (Total Time) solution, this application embodiment reduces the problem of a large number of branches in a single thought tree by increasing the number of thought trees, thus reducing computational resources. This application embodiment uses a thought forest to analyze input data from multiple perspectives, thereby increasing the diversity of reasoning results, avoiding the risk of falling into "fixed mindsets," and improving the accuracy of reasoning results.

[0089] For example, compared to the Monte Carlo Tree Search (MCTS) solution, the embodiments of this application can analyze the input data from multiple perspectives through the mind forest. Each mind tree represents a unique reasoning path, which can find the optimal solution of the reasoning result in a wider range and avoid the occurrence of the "local optimum" problem.

[0090] To better understand the feature filtering method of this application embodiment, the application scenarios of the feature filtering method of this application embodiment are described below, but the description of the application scenarios does not constitute a limitation on the embodiments of this application.

[0091] Please refer to Figure 1, which is a schematic diagram of the composition of a reasoning system provided in an embodiment of this application. As shown in Figure 1, in this embodiment, a reasoning system 100 includes a terminal 110 and a server 120. For example, the terminal 110 acquires user input data and sends it to the server 120; the server 120 deploys a large language model, and based on the reasoning method of this embodiment, the large language model outputs a reasoning result, and the server 120 sends the target reasoning result of the input data to the user; the user obtains the target reasoning result of the input data through the terminal 110. The reasoning method of this embodiment utilizes the reasoning structure of a thought forest, which can analyze input data from multiple perspectives, making the reasoning result more comprehensive and accurate.

[0092] Optionally, server 120 may include one or more servers (Figure 1 illustrates this by including one server as an example), and server 120 may provide the methods or apparatus provided in the embodiments of this application to one or more terminals.

[0093] Optionally, a relevant application may be installed on the terminal 110. This application or webpage can provide an interface that displays the inference results. The terminal 110 can receive input data entered by the user on the interface and send the input data to the server 120. The server 120 is equipped with the large language model of this application embodiment, which can analyze the input data from multiple perspectives and output more accurate inference results.

[0094] It should be understood that in some optional implementations, terminal 110 may deploy the large language model and complete the work on its own without the cooperation of server 120, and this application embodiment is not limited in this regard. In some optional implementations, server 120 may also rely on input data received from other devices outside the terminal or locally stored, without the cooperation of terminal 110, and this application embodiment is not limited in this regard.

[0095] The product form of terminal 110 in Figure 1 will be described next. In this embodiment, terminal 110 can be a mobile phone, tablet computer, wearable device, in-vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc. This embodiment does not impose any limitations on this.

[0096] The product form of server 120 in Figure 1 will be described next. It can be further understood that server 120 can be various types of servers, such as x89 architecture servers, specifically rack servers, blade servers, high-density servers, platform servers, or high-performance servers. In other words, this application embodiment does not specifically limit the specific type of server. Furthermore, it can be understood that the server structure shown in Figure 1 does not constitute a limitation on the server structure; a server may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0097] Furthermore, server 120 can be configured as an independent physical server, or as a server cluster or distributed system consisting of multiple physical servers. It can also be configured as a cloud server or cloud server cluster that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The cloud server cluster is deployed in several cloud data centers. The software can be an application that implements object control methods, but it is not limited to the above forms.

[0098] Next, the communication connection method between terminal 110 and server 120 is described. For example, terminal 110 and server 120 are connected via a network, enabling terminal 110 to access the cloud management platform deployed on the cloud server cluster. The network can be a wired network or a wireless network. For example, a wired network can be a cable network, fiber optic network, Digital Data Network (DDN), etc., while a wireless network can be a telecommunications network, intranet, Internet, Local Area Network (LAN), Wide Area Network (WAN), Wireless Local Area Network (WLAN), Metropolitan Area Network (MAN), Public Service Telephone Network (PSTN), Bluetooth network, ZigBee network, Global System for Mobile Communications (GSM), CDMA (Code Division Multiple Access) network, CPRS (General Packet Radio Service) network, etc., or any combination thereof.

[0099] Understandably, a network can use any known network communication protocol to enable communication between different terminal layers and gateways. These network communication protocols can be various wired or wireless communication protocols, such as Ethernet, Universal Serial Bus (USB), FireWire, Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), New Radio (NR), Bluetooth, Wireless Fidelity (Wi-Fi), and other communication protocols.

[0100] In one possible scenario, server 120 can function as a cloud platform (a software platform employing application virtualization technology, integrating multiple functions such as software search, download, use, management, and backup). In practical use, server 120 can deploy a cloud management platform and a data center, with terminal 110 interacting with the cloud through the cloud management platform. Additionally, the data center can deploy nodes, which can be virtual machine instances, container instances, physical servers, etc.

[0101] In another possible scenario, the method provided in this application embodiment can be implemented by software. The software has a terminal and a server; the terminal 110 runs the software, and the server 120 runs the software's server. While the terminal 110 is running the software, it can call the server running on the server 120 to implement the method provided in this application embodiment.

[0102] In other words, the method provided in this application embodiment can be applied to the terminal 110 or the server 120. In specific implementation, it can run as software on the terminal 110 or the server 120; for example, the software can be a service or an application. This application embodiment can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application embodiment can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can reside in local and remote computer storage media, including storage devices.

[0103] Furthermore, the system that can implement the technical solution of this application embodiment includes cloud servers, web pages, application program interfaces (APIs), mobile application software, etc. The large language model of this application embodiment can be deployed on the server side, website, or mobile application software to meet the text generation needs of enterprise users and individual users, and can also be provided to developers for secondary development and use through open API interfaces. Furthermore, the implementation of this application embodiment mainly involves deploying the relevant platforms and applications of the large language model of this application embodiment, such as cloud platforms, websites, applications (APPs), etc.

[0104] This application provides a reasoning framework based on a thought forest-based LLM (Limited Learning Model). This framework can handle complex mathematical and logical problems and provide accurate solutions. The reasoning system using this framework can solve problems using a thought forest structure. The solution process is optimized through node selection and splitting, as well as sparse activation and early stopping mechanisms within the forest, improving efficiency and accuracy. Furthermore, the system incorporates a dynamic error correction strategy based on fixed and learnable rules, enhancing robustness and adaptability in complex reasoning tasks.

[0105] This application can be applied to various products. For example, in a Software as a Service (SaaS) model, users can subscribe to the service via the internet and use the intelligent inference system's functions on demand, without needing to install the software locally. In cloud platform services, users can directly access and use the intelligent inference system on the cloud platform and pay based on usage. It can also be used for custom development, providing customized solution development services tailored to the specific needs of users. Furthermore, in hardware products, if specific hardware requirements are involved, corresponding hardware products such as servers and dedicated chips can be developed.

[0106] Exemplary examples illustrate the system architecture or scenarios to which the embodiments of this application can be applied. For logical reasoning problems, such as mathematical reasoning or puzzle solving, the model can generate different solution paths and select the best solution based on the rationality of each path. For path optimization and exploration problems, in scenarios requiring the search for the optimal solution, the thought forest allows the model to try multiple possibilities and then select the most suitable solution path through subsequent steps. Specifically, for example, in the field of intelligent customer service, designing multi-turn dialogue processes based on thought forests enables intelligent customer service to better understand user needs and provide more accurate services. Specifically, for example, in the financial field, the thought forest construction method of the embodiments of this application can be applied to build a thought forest model between applicant background information and credit scores; by constructing a thought forest, financial institutions can use diverse reasoning results to assist in analyzing transaction patterns and abnormal behaviors, quickly identify potential fraudulent activities, and protect platform security. Specifically, for example, in the medical field, the embodiments of this application can be applied to build a thought forest model between diseases and symptoms; by constructing a thought forest and analyzing diverse reasoning results of user lifestyle habits, genetic factors, etc., doctors can quickly select the most suitable treatment plan based on the patient's specific condition, improving the accuracy of diagnosis and the efficiency of treatment. Specifically, for example in the field of autonomous driving, the mind forest construction method of this application embodiment can be used for vehicle path planning and decision making; by constructing a mind forest, the autonomous driving system can analyze road conditions in real time, make optimal driving decisions, and improve driving safety and efficiency.

[0107] Furthermore, the aforementioned application scenarios can be modified accordingly to adapt to new usage requirements. For example, in the field of intelligent customer service, the integration strategy of the thought forest can be adjusted to adapt to different types of dialogues; for example, more rules can be introduced, or the learning mechanism can be adjusted to better adapt to specific dialogue needs. In the financial field, the construction method of the thought forest can be modified to adapt to the risk assessment models and credit approval processes of different financial institutions; for example, the number of certain thought trees can be increased or decreased, or the rules for node selection and splitting can be adjusted to better adapt to specific risk assessment needs. In the medical field, the dynamic error correction strategy of the thought forest can be adjusted to adapt to different medical diagnosis and treatment processes; for example, more medical rules can be introduced, or the learning mechanism can be adjusted to better adapt to specific medical needs. In the field of autonomous driving, the strategy of the thought forest can be modified to adapt to different driving environments and decision-making needs; for example, more environmental perception data may be introduced, or the stopping conditions of the thought forest may be adjusted to better adapt to specific driving scenarios.

[0108] Furthermore, the performance of the reasoning framework of the Mind Forest can be further improved for the aforementioned scenarios. For example, in the financial field, more large language models can be integrated to improve the accuracy and efficiency of risk assessment. In the medical field, more biomedical data can be integrated to improve the accuracy and efficiency of disease diagnosis and treatment. In the field of autonomous driving, more sensor data and advanced algorithms can be integrated to improve the accuracy and efficiency of path planning and decision-making.

[0109] Please refer to Figure 2, which is a flowchart illustrating a reasoning method provided in an embodiment of this application. As shown in Figure 2, this embodiment of the application provides a reasoning method, which mainly includes the following steps:

[0110] Step S210: Obtain input data.

[0111] Here, input data represents any information or data to be reasoned about. For example, in the field of intelligent customer service, input data can be the user's voice information, which is then used to design a multi-turn dialogue process based on a thought forest, enabling the intelligent customer service to better understand user needs and provide more accurate services.

[0112] Step S220: Based on the input data, generate multiple prompts. These prompts are used to guide the large language model to reason about the input data from different perspectives.

[0113] In this embodiment, instead of directly reasoning from the input data, the input data is preprocessed to obtain multiple prompts. These prompts then form the reasoning question for each thought tree. This approach better guides the large language model to analyze the input data from different perspectives, enabling it to examine the problem from multiple angles and thus improving the accuracy and robustness of the target reasoning results.

[0114] Optionally, based on the input data and multiple prompt word templates, the prompt information corresponding to each prompt word template can be obtained, where the multiple prompt word templates are different in content. Multiple prompt messages are obtained by processing these multiple prompt word templates with different content. In other implementations, multiple prompt messages can also be obtained based on the input data using other methods, such as manual writing.

[0115] For example, in the medical field, multiple prompt templates with different content could be: Prompt Template 1: "Based on *'s description of their condition, is disease A caused by lifestyle habits?"; Prompt Template 2: "Based on *'s description of their condition, is disease A related to genetic factors?"; Prompt Template 3: "Based on *'s description of their condition, is disease A related to their living environment?". * represents input data, i.e., the patient's specific medical record information or description of their condition. Based on the medical record information and the above three prompt templates, three prompt messages can be obtained. Under the guidance of a thought forest, the large language model can analyze the patient's condition from multiple perspectives, such as "lifestyle habits," "genetic factors," and "living environment," corresponding to the above three prompt templates, to determine whether the patient has disease A. This yields diverse inference results from the user's lifestyle habits, genetic factors, and living environment, thereby assisting doctors in quickly selecting the most suitable treatment plan based on the patient's specific condition, improving diagnostic accuracy and treatment efficiency.

[0116] Step S230: Using each of the multiple prompts as the root node, call the large language model to construct multiple thought trees. The thought tree represents the reasoning path of the large language model in reasoning about the input data.

[0117] It should be noted that, for ease of description, the structure of the multiple thought trees corresponding to the above prompts can also be understood as a kind of "thought forest," which is a reasoning framework that represents a larger scope than a "single thought tree" and includes multiple thought trees.

[0118] Please refer to Figure 3, which is a schematic diagram of a thought forest provided in an embodiment of this application. As shown in Figure 3, the thought forest includes multiple thought trees, each symbolizing a unique reasoning path. By exploring multiple paths in parallel, the reasoning framework of the thought forest can examine the problem from multiple perspectives, thereby improving the accuracy and robustness of the reasoning. Furthermore, this multi-path reasoning mechanism enables the model to capture various aspects of the problem from different dimensions. For example, user input data can be described from different angles.

[0119] It should be noted that, taking a mind tree as an example, the construction process of a mind tree mainly includes: using the prompt information as the root node of the mind tree; using the root node as the parent node, generating several child nodes of the parent node through a large language model; then using the child nodes as new parent nodes, executing the above steps of generating several child nodes of the parent node based on the parent node through the large language model, until the condition for stopping reasoning is met. For the sake of simplicity, the detailed construction process of the mind tree will not be elaborated in this application embodiment. The following only provides examples of the relevant optimizations made to the mind tree construction process in this application embodiment.

[0120] Optionally, the parent node is used as input to the large language model to generate several child nodes; then, according to the preset error correction rules, the several child nodes are corrected to obtain several corrected child nodes; based on the parent node and the several corrected child nodes, a mind tree is obtained.

[0121] In other words, during the process of constructing a mind tree, after the large language model generates child nodes, the child nodes are corrected by pre-setting error correction rules. This is a real-time error correction method, rather than correcting errors after generating the entire mind tree. This can prevent erroneous child nodes from entering the next step of the calculation work, reduce the amount of calculation, improve the calculation efficiency, and save computing resources.

[0122] In the first implementation, the preset error correction rules include fixed rules, which represent the relevant rules corresponding to the prompt information. Then, based on the fixed rules, several child nodes can be corrected to obtain corrected child nodes. The fixed rules represent the relevant rules corresponding to the prompt information, such as the rules of Go in a Go scenario, or the rules of Chinese chess in a Chinese chess scenario. Fixed rule error correction is fast and can improve the efficiency of the error correction process.

[0123] For example, please refer to Figure 4, which is a schematic diagram of an example of correcting child nodes according to inherent rules provided by an embodiment of this application. As shown in Figure 4, in one example, taking the 24 mathematical operation problem as an example, the parent node is the prompt information "4, 5, 6, 10", such as "If the four given integers are 4, 5, 6, 10, what four arithmetic operations are required to make the calculation result 24?".

[0124] As shown in Figure 4, in this example, among the child nodes, child node 1 is "4*6=24(left: 10, 9, 24)", child node 2 is "5+6=11(left: 10, 11, 25)", and child node 3 is "10+4=14(left: 6, 14, 24)". Here, "left" represents the remaining numbers to be calculated. For example, "left: 10, 9, 24" means that we need to calculate what arithmetic operations need to be performed on 10, 9, and 24 to get the result 24. In other words, the question corresponding to this child node is "What arithmetic operations need to be performed on 10, 9, and 24 to get the result 24?".

[0125] As shown in Figure 4, in this example, child nodes 1, 2, and 3 are corrected according to inherent rules. For example, the inherent rule could be "the content of left can only include numbers that did not participate in the operation of this node and the result of the operation of this node".

[0126] As shown in Figure 4, this example illustrates the corrected child nodes 1, 2, and 3 according to the inherent rules. The corrected child node 1 is "4*6=24(left: 10, 5, 24)", the corrected child node 2 is "5+6=11(left: 10, 11, 4)", and the corrected child node 3 is "10+4=14(left: 6, 14, 5)". The corrected numbers are represented by the underscore "_" in Figure 4.

[0127] In the second implementation, the pre-set error correction rules can also include a rule for the large language model to correct errors through learning. Specifically, child nodes are determined from a pool of child nodes using methods such as child node scoring and pre-defined conditions; historical failure cases and child nodes are used as input to the large language model, and the corrected child nodes are output. Examples of pre-defined conditions include, for instance, outputting child nodes whose values ​​are greater than a threshold.

[0128] In this implementation, the learning ability of a large language model to study historical failure cases can be used to correct child nodes. Specifically, child nodes can be selected from a pool of child nodes based on the large language model's scores. Therefore, different score thresholds can be set to determine the number of child nodes requiring correction. In other words, the frequency of error correction by the large language model can be dynamically adjusted through scoring, improving correction efficiency and enhancing the robustness and adaptability of the large language model in complex reasoning tasks.

[0129] For example, please refer to Figure 5, which is a schematic diagram of an example of error correction of child nodes based on learnable rules provided by an embodiment of this application. As shown in Figure 5, the input is a prompt message; a large language model (such as a large language model LLM) infers the prompt message; during the inference process, several child nodes are generated; each child node undergoes fast thinking, i.e., fast error correction based on fixed rules; each child node undergoes slow thinking, i.e., error correction based on learnable rules; until the large language model completes the inference, the reference inference result of the prompt message is output. Here, "fast thinking" refers to the fast error correction based on fixed rules in the first implementation method described above. "Slow thinking" refers to the second implementation method, which is a more complex error correction process because it requires the large language model to learn and then correct the error.

[0130] As shown in Figure 5, error correction based on learnable rules, i.e., the "slow thinking" process, firstly determines child nodes from a number of child nodes. For example, each child node is scored (0 to 100) using prompt word template 2, and child nodes with scores below 60 are identified as child nodes. Then, historical failure cases and child nodes are processed through prompt word template 1 as shown in Figure 5, and the resulting prompt words are used as input to the large language model. The large language model outputs the corrected child nodes.

[0131] This application proposes a method combining two error correction mechanisms. On one hand, the system relies on pre-defined fixed rules (such as mathematical rules) for rapid error correction, quickly identifying and correcting errors (such as those in mathematical expressions). On the other hand, during the reasoning process, the large language model scores each reasoning step in real time—an innovative self-monitoring mechanism. When the score of a step falls below a preset threshold (e.g., 60 points), the system triggers an error correction procedure: the large language model learns from historical failure cases and immediately adjusts the current reasoning path. This feedback-based self-correction mechanism allows the large language model to learn from past errors, optimize reasoning schemes, avoid repeating the same mistakes, and thus improve the overall accuracy and efficiency of reasoning.

[0132] In this application, a rule-based and learning-based dynamic error correction strategy is proposed. It can perform rapid error correction based on fixed rules ("fast thinking"), and reflective error correction using a large language model ("slow thinking"). Furthermore, it can dynamically adjust the frequency of reflective error correction in real time based on the reasoning score, thereby improving error correction efficiency and enhancing the system's robustness and adaptability in complex reasoning tasks.

[0133] As shown in Figure 5, in one example, a historical failure case could be: "Input data: (2x+3=7). We have a weak (i.e., incorrect) answer: [2x+3=7]; [2x=7+3]; (this step is incorrect); [2x=10]; [x=5]. Can you provide some reflection or feedback to better correct this weak answer? Analyze and critique this answer rigorously, ending with {ans_format}. Let's think about it step by step. Answer (LLM's answer): The student made a mistake in the transformation process. The correct way is to move 3 to the right side of the equation; it should be subtracting 3, not adding 3. #### Correct answer: \[2x+3=7]; [2x=7-3]; [2x=4]; [x=\frac{4}{2}=2]". Where, frac represents a fraction, frac{4}{2} represents 4 / 2; {ans_format} represents the formatted answer. In other words, a historical failure case mainly consists of "input data", "incorrect answer", "error reason analysis", "correct answer" and related prompts. The above historical failure case is only an example for analysis and explanation.

[0134] As shown in Figure 5, in one example, the function of prompt word template 1 is to obtain the prompt words for child nodes to input into the large language model. An example of prompt word template 1 is “f'Question:{Question}\nAnswer:{Answer}\nPlease refine your answer based on your reflection or feedback. The answer should begin with [reasoning process]...[verification]... and end with {ans_format}.\nLet's think about it step by step.'” Here, {Question} represents the content of the parent node, and {Answer} represents the content of the child node.

[0135] As shown in Figure 5, in one example, the function of prompt word template 2 is to obtain prompt words for scoring each positive child node, which are then input into the large language model so that the large language model can score each positive child node. An example of prompt word template 1 is: "f'Question:{Question}\nAnswer:{Answer}\nPlease rigorously analyze and criticize this answer, pointing out every possible imperfection to deduct every point! When calculating the score, you need to be very strict and demanding, never giving a perfect score to ensure the authority of the score.\nOutput a score between [0,+100], i.e., from 0 to +100.\nReply format:\n[Analyst]...[Score]...'".

[0136] Optionally, the parent node is used as input to the large language model to generate several child nodes. The parent node represents the node of the previous level in the mind tree, and the child node represents the node of the next level corresponding to the parent node. The several child nodes are used as input to the large language model to generate scores for the several child nodes. Based on the scores of the several child nodes, the target child node is determined from the several child nodes. Based on the parent node and the target child node, the mind tree is generated.

[0137] Furthermore, this optional implementation provides a sparse activation method during the generation of the mind tree. This sparse activation method is equivalent to a pruning method, retaining only the child nodes with higher scores, reducing computational load, improving computational efficiency, and saving computational resources.

[0138] Optionally, when the parent node meets the preset requirements, the method further includes: generating several child nodes based on the parent node according to the preset processing rules, wherein the parent node represents the node of the previous level in the mind tree, and the child node represents the node of the next level corresponding to the parent node.

[0139] Furthermore, this optional implementation provides another method for generating child nodes. Some child nodes can be generated through preset processing rules. In this case, it is not necessary to use a large language model to generate child nodes, which reduces the amount of computation, improves computational efficiency, and saves computing resources.

[0140] For example, taking the 24-digit mathematical operation problem as an example, if the number of given numbers is 4, after several steps of reasoning, the number of numbers to be reasoned is 2. Then, the calculation can be performed by four arithmetic operations, without having to input the 2 numbers to be reasoned into the large language model for reasoning, thus saving computing resources.

[0141] Step S240: Based on the multiple reference reasoning results corresponding to multiple mind trees, obtain the target reasoning result of the input data.

[0142] Optionally, by combining the first strategy and the second strategy, the target reasoning result of the input data is selected from the reference reasoning results of multiple prompts. The first strategy represents a strategy based on the voting principle, and the second strategy represents a strategy based on the expert judgment principle.

[0143] Furthermore, this optional implementation provides a reasoning strategy that combines voting principles and expert judgment. Two strategies are used to determine whether the target reasoning result for the input data exists among the reference reasoning results of multiple prompts; upon obtaining the target reasoning result for the input data, the reasoning process for that input data is halted. The first strategy is equivalent to a strategy embodying collective wisdom (a majority consensus voting strategy), ensuring broad participation of opinions. The second strategy is equivalent to an expert judgment strategy (such as a selection strategy using a large language model), guaranteeing the professionalism and high quality of the result. Furthermore, by integrating the reference reasoning results from different prompts, the large language model can stop the reasoning process after reaching a certain confidence level, ensuring the optimal decision result is obtained with limited resources.

[0144] For example, in the field of autonomous driving, the obtained inference results include: Inference result 1 is "rushing at speed a", Inference result 2 is "rushing at speed b", and Inference result 3 is "rushing at speed c". Combining the voting principle, the first strategy is executed. Taking 100 votes as an example, Inference result 1 receives 55 votes, Inference result 2 receives 40 votes, and Inference result 3 receives 5 votes. Inference results 1 and 2 with higher votes, or all inference results, are used as the inference result to be judged. Combining the relevant rules of expert judgment, the second strategy is executed. For example, the inference result to be judged is input into a large language model, which processes the inference result to obtain the optimal inference result. The results of the first and second strategies are weighed using methods such as weighting to obtain the final inference result.

[0145] It should be noted that, compared with methods based on a single strategy (such as large model selection or majority consensus voting), the fusion method based on model selection and majority consensus can improve accuracy by nearly 2%. For the sake of brevity, the specific implementation details of the voting principle and expert judgment principle will not be elaborated upon in the embodiments of this application.

[0146] Optionally, when the reference inference results of multiple prompts are related to real data, the above-mentioned determination of the target inference result of the input data based on the reference inference results of multiple prompts includes: selecting the target inference result of the input data from the reference inference results of multiple prompts according to a third strategy, wherein the third strategy represents a strategy based on real data.

[0147] Furthermore, this optional implementation provides a third strategy to determine whether to stop the inference process and obtain the target inference result for the input data. When the reference inference results of multiple prompts are related to the real data, the decision to stop the inference process and obtain the target inference result for the input data can be made based on the real data.

[0148] For example, taking the 24-operation problem as an example, the goal is to calculate the result of a given number as "24". Therefore, "24" belongs to the ground truth (GT), and the third strategy mentioned above can be obtained based on "24". In other words, when reasoning in the 24-operation problem, if the calculation result is "24", the reasoning can be stopped directly without executing other strategies. This results in higher reasoning efficiency, avoids unnecessary calculations, and saves computing resources.

[0149] Optionally, the method further includes: during the construction of the mind tree, if a node in the mind tree meets a preset stopping condition, then the mind tree is discarded. Furthermore, this optional implementation provides a sparse activation method for the mind forest. That is, only a portion of the mind trees in the mind forest are activated, which avoids useless computation and the waste of computational resources.

[0150] Optionally, the aforementioned preset stopping condition includes at least one of a first condition and a second condition. The first condition indicates that the score of a node in the mind tree is lower than a preset threshold, and the second condition indicates that the node in the mind tree contains erroneous content. Further, this optional implementation provides an example of discarding (i.e., not activating) the preset stopping condition of the mind tree. However, this does not constitute a limitation on the preset stopping condition itself.

[0151] For example, taking the 24-point mathematical operation problem, the data type of the nodes should be numbers. If the result of a node is a character, then that child node is discarded. Furthermore, if the node is an intermediate node, then the construction of branches below that intermediate node is stopped, i.e., the subtree related to the intermediate node is discarded; if the node is a leaf node, then the construction of that leaf node is stopped (i.e., not activated), i.e., the subtree with that leaf node as the reasoning result is discarded; if the leaf node is the only leaf node, then the construction of that leaf node is stopped (i.e., not activated), i.e., the entire thought tree is discarded. This avoids useless calculations and the waste of computational resources.

[0152] In this embodiment, multiple prompts from different descriptive perspectives are obtained based on the input data, such as through different prompt word templates. Then, a large language model is guided to construct a thought tree corresponding to each prompt. Based on the thought tree of each prompt, a reference reasoning result for each prompt can be obtained. Based on the reference reasoning results of multiple prompts, the target reasoning result for the input data is determined. This reasoning framework is equivalent to multiple parallel thought trees, which can be understood as a kind of thought forest. In the structure of this thought forest, each thought tree symbolizes a unique reasoning path, thus exploring multiple paths in parallel. This thought forest reasoning framework can examine the problem from multiple angles, thereby improving the accuracy and robustness of the target reasoning result for the input data. Furthermore, this multi-path reasoning mechanism allows the large language model to capture various aspects of the input data from different dimensions, making the target reasoning result for the input data more comprehensive and accurate. It avoids the problems of fixed thinking and local optima caused by a single thought tree. The thought forest can easily solve problems by "changing perspectives" or "changing approaches."

[0153] In this embodiment, the reasoning framework of Mind Forest aims to address the technical challenges faced by Large Language Models (LLMs) in mathematical reasoning tasks, including improving multi-path reasoning capabilities for complex problems, optimizing error detection and correction mechanisms, and dynamically adjusting errors based on efficient and accurate real-time scoring. By integrating sparse activation, dynamic error correction strategies, and flexible and diverse strategies, this framework strives to improve the accuracy, efficiency, and overall performance of LLMs in mathematical reasoning, especially when dealing with complex problems requiring high levels of logical reasoning and mathematical computation.

[0154] This application proposes an innovative thought forest reasoning framework aimed at significantly improving the performance of Large Language Models (LLMs) in mathematical reasoning tasks. By integrating sparse activation, dynamic error correction strategies, and a variety of flexible strategies, this framework greatly enhances the ability of LLMs to handle complex reasoning problems.

[0155] In this embodiment, the core of the framework is a thought forest reasoning structure built based on a sparse activation strategy. In this thought forest reasoning structure, each thought tree symbolizes a unique reasoning path. By exploring multiple paths in parallel, the framework can examine the problem from multiple perspectives, thereby improving the accuracy and robustness of the reasoning. Furthermore, this multi-path reasoning mechanism enables the model to capture various aspects of the problem from different dimensions.

[0156] This application also proposes a method combining two error correction mechanisms. Firstly, the system relies on preset mathematical rules for rapid error correction, quickly identifying and correcting errors in mathematical expressions. Secondly, during the inference process, the system scores each inference step in real time, an innovative self-monitoring mechanism. When the score of a step falls below a preset threshold (e.g., 60 points), the system triggers an error correction procedure, learning from historical failure cases and immediately adjusting the current inference path. This feedback-based self-correction mechanism allows the model to learn from past errors, optimize inference schemes, avoid repeating the same mistakes, and thus improve the overall accuracy and efficiency of inference.

[0157] This application also proposes a strategy that combines collective intelligence (such as voting mechanisms) and expert judgment (such as large language model selection). This strategy ensures both broad participation of opinions and the professionalism and high quality of the results. By integrating inputs from different sources, the model can stop the reasoning process after reaching a certain confidence level, ensuring that the optimal decision result is obtained with limited resources.

[0158] In this embodiment, the Mind Forest reasoning framework, through its unique design, not only improves the performance of LLM in mathematical reasoning tasks but also provides a new perspective and method for solving complex problems. The implementation of this framework is expected to play a significant role in the field of artificial intelligence, particularly in areas requiring high levels of reasoning and computation, such as automated theorem proving and solving complex problems.

[0159] This application provides a method for constructing a thought forest. Under the same parameter conditions, compared to thought tree-based reasoning, the reasoning method using diverse paths through a thought forest can capture multiple solutions to a problem, improving the model's comprehensiveness and fault tolerance when dealing with complex problems, thus significantly improving reasoning accuracy. For example, in the 24-point game, the reasoning thinking of a thought tree (ToT) is limited; despite expanding the depth of the thought tree, some problems remain unsolved due to a mental bottleneck. Conversely, the thought forest (FoT) can easily solve problems by "changing perspectives" or "changing approaches." For instance, when integrating eight subtrees, the thought forest (FoT) can achieve an accuracy rate close to 97%.

[0160] This application also provides a dynamic error correction strategy based on fixed rules and learnable rules. That is, it can both rely on fixed rules for rapid error correction and dynamically adjust the frequency of reflective error correction in real time based on the reasoning score, thereby improving error correction efficiency and enhancing the system's robustness and adaptability in complex reasoning tasks.

[0161] In this embodiment, a strategy for thinking forests is also provided. Furthermore, compared to a single strategy (such as large model selection or majority consensus voting), the fusion method based on model selection and majority consensus can improve accuracy by nearly 2%.

[0162] The following example will further illustrate the reasoning method described above.

[0163] Please refer to Figure 6, which shows a flowchart illustrating an example of a reasoning method. As shown in Figure 6, this application embodiment provides an example of a reasoning method, mainly including the following steps:

[0164] Step S610, given the basic model.

[0165] Furthermore, the base model is the large language model used for reasoning. This base model can be any large language model, and the embodiments of this application do not impose any restrictions on it.

[0166] Step S620: Randomly import the input data into the prompt template and input it into each of the n trees.

[0167] Furthermore, input data refers to the data entered by the user. Multiple different prompt templates are obtained from the dataset; combining the input data with different prompt templates yields different prompt messages. Different prompt templates refer to prompt templates with different descriptive perspectives, allowing multiple prompt messages to describe the input data from different angles.

[0168] Furthermore, the root node of each of the n thought trees is a hint. These n hints are input into the n thought trees respectively, thus enabling a multi-path parallel reasoning process from n different perspectives, resulting in a more comprehensive and accurate reasoning outcome.

[0169] Step S630: Construct the reasoning paths for the n trees.

[0170] Furthermore, in each thought tree, the root node is a prompt; using the prompt as the parent node, the large language model is used to generate one or more child nodes based on the parent node; then, using each of the child nodes as the parent node, the large language model is used to generate one or more child nodes based on the parent node; until the reasoning result is obtained.

[0171] Step S640: Score the results of the single-step reasoning, identify and correct errors in the mathematical expression.

[0172] Furthermore, this step is performed each time the large language model generates a child node (i.e., single-step reasoning), which can ensure timely correction of errors, prevent meaningless calculations, and avoid wasting computing resources.

[0173] Step S650: When the score is below 60, the reasoning result with errors in the mathematical expression is corrected again.

[0174] Furthermore, assuming the score range is 0 to 100, a second correction is performed on child nodes with scores below 60, i.e., correction based on learnable rules. It's understandable that the low scores of child nodes could be due to errors in the fixed rules or other errors (such as incorrect wording or phrasing). The combination of steps S640 and S650 allows for two rounds of correction for child nodes, reducing the probability of discarding them and improving the accuracy of the final inference result.

[0175] Step S660: Invoke the dynamic error correction mechanism.

[0176] Furthermore, this step utilizes the learning capabilities of the large language model to learn from historical failure cases, performing a second round of error correction on child nodes with scores below 60, and regenerating corrected child nodes until the child node's score is greater than or equal to 60. Optionally, the number of times the dynamic error correction mechanism can be invoked can be set; that is, if the child node's score is still less than 60 after the number of invocations exceeds a threshold, the child node is discarded.

[0177] Step S670: Discard subtrees with low scores or invalid outputs, and do not participate in the final decision. A subtree represents a thought tree in the thought forest.

[0178] For example, taking the 24-point mathematical operation problem, the data type of the child nodes should be numbers. If the result of a node is a character, then the child node is discarded. Furthermore, if the child node is an intermediate node, then the branch corresponding to that intermediate node is discarded; if the child node is a leaf node, then the subtree whose reasoning result is the content of that leaf node is discarded; if the leaf node is the only leaf node, then the entire thought tree is discarded. Discarded branches or thought trees, such as the inactive thought trees 2 and 4 shown in Figure 3, do not participate in the final decision, thus avoiding useless calculations and wasting computational resources.

[0179] Step S680: Select the optimal reasoning result according to the strategy of the mind forest.

[0180] For example, the strategies of the thought forest may be the first strategy, the second strategy, the third strategy, etc., or a combination of multiple strategies. As shown in Figure 3, the optimal reasoning result is selected from the reasoning result 1 corresponding to thought tree 1 and the reasoning result 2 corresponding to thought tree 2, based on the majority consensus voting method or, for example, the expert judgment method of large language model selection.

[0181] The following example further illustrates the reasoning process of a single thought tree in a thought forest. For simplicity, the detailed construction process of a single thought tree will not be elaborated upon in this embodiment. Instead, only the optimized parts of the single thought tree construction process will be illustrated using a specific example.

[0182] Please refer to Figure 7, which illustrates the optimization of the reasoning process for a single thought tree. As shown in Figure 7, taking the 24-point mathematical operation problem as an example, the reasoning process of a single thought tree mainly includes the following steps:

[0183] Step S710: Obtain input data, i.e., prompt information.

[0184] Furthermore, as shown in Figure 7, the input data is 4, 5, 6, 10, which means that the 24 mathematical operations need to be performed based on 4, 5, 6, 10.

[0185] Step S720: Based on the root node of the input data, generate multiple child nodes using the large language model.

[0186] Furthermore, as shown in Figure 7, the large language model is used to perform arithmetic operations on any two of the four numbers. For example, the input data 4, 5, 6, and 10 are given prompts and then input into the large language model. After the large language model performs reasoning, it outputs the reasoning result of performing arithmetic operations on the two numbers.

[0187] For example, the output child nodes of the large language model are as follows: Node 1: 5*6=30, (left: 30 4 10); Node 2: 4+5=9, (left: 6 10 9); Node 3: 4*6=24, (left: 10 9 24); Node 4: 5+6=11, (left: 10 11 24); Node 5: 5-6=-1, (left: 4 -1 10); Node 6: 10+4=14, (left: 6 14 24); Node 7: 10-5=5, (left: 5 4 6). Here, left represents the remaining number to be calculated. For example, in node 1, left: 30 4 10 means that we need to calculate 30 4 10 and perform what arithmetic operations to make the result 24. In other words, the question corresponding to this child node is "What arithmetic operations need to be performed on 30 4 10 to make the result 24?".

[0188] Step S730: Perform error correction on each child node according to preset principles.

[0189] Furthermore, if nodes 3, 4, and 6 contain errors, as shown in Figure 7 (the incorrect parts), these errors can be quickly corrected based on mathematical calculation principles. As shown in Figure 7, the corrected results could be: Node 3: 4*6 = 24 (left: 10 5 24); Node 4: 5+6 = 11 (left: 10 11 4); Node 6: 10+4 = 14 (left: 6 14 5). Optionally, the child nodes can also be corrected by leveraging the learning capabilities of the large language model and incorporating historical failure cases.

[0190] Step S740: Use the large language model to score each child node.

[0191] For example, after adding prompt words to each child node, the data is input into a large language model. After inference, the large language model outputs a score for each child node. As shown in Figure 7, the large language model's score for each child node could be: Node 1: 20 points; Node 2: 1 point; Node 3: 20 points; Node 4: 0.001 points; Node 5: 0.001 points; Node 6: 0.001 points; Node 7: 1 point. Assuming a maximum score of 30, a higher score indicates a higher probability that the branch of that child node will yield an inference result.

[0192] Step S750: Based on the scoring results, retain the nodes with higher scores.

[0193] For example, based on the scoring results, the top 5 nodes are retained, as shown in Figure 7. The top 5 nodes are node 1, node 3, node 2, node 7, and node 4. Optionally, for nodes with the same score, nodes are selected and retained from the beginning to the end according to their serial numbers.

[0194] Step S760: Based on the nodes retained above, use the large language model to generate multiple child nodes corresponding to each node.

[0195] For example, after adding prompt words to the nodes that are retained above, they are input into a large language model. After the large language model performs inference, it outputs multiple child nodes corresponding to each node.

[0196] For example, the child nodes of the aforementioned reserved nodes in the output of the large language model are described. For node 1, its child nodes are: node 11: 5*6=30, (left: 30 4 10), 10-4=6, (left: 6 30); node 12: 5*6=30, (left: 30 4 10), 4*10=40, (left: 40 30); node 13: 5*6=30, (left: 30 4 10), 30+4=34, (left: 34 10). For node 7, its child nodes are: Node 71: 10-5=5, (left: 5 4 6), 5+4=9, (left: 9 6); Node 72: 10-5=5, (left: 5 4 6), 4*5=20, (left: 20 6); Node 73: 10-5=5, (left: 5 4 6), 5-4=1, (left: 1 6). For nodes 3, 2, and 4, please refer to the relevant descriptions of nodes 1 and 7, which will not be repeated here.

[0197] In step S770, the child nodes generated in step S760 are scored using four arithmetic operations.

[0198] For example, since the remaining numbers are few and the calculation is relatively simple, the final score of each child node can be determined through four arithmetic operations. As shown in Figure 7, the score results of each child node could be: Node 11: 0.001 points; Node 12: 0.001 points; Node 13: 20 points; Node 71: 0.001 points; Node 72: 0.001 points; Node 73: 0.001 points. Assuming the full score is 30, the higher the score, the higher the probability that the branch of that child node will yield a reasoning result.

[0199] Step S780: Based on the above strategy, determine the final reasoning result and stop the reasoning process.

[0200] For example, as shown in Figure 7, the criterion for stopping inference is: whether the calculation result is 24, in which case the GT early stopping strategy can be selected. According to the GT early stopping strategy, the content of node 13 is determined to be the final inference result, and the inference process is stopped.

[0201] In this embodiment, the large language model can be a large language model, and this embodiment provides a reasoning method based on a large language model with a thought forest. Based on the thought forest, the large language model can perform multi-path exploration and reasoning. An example application scenario is the 24-point mathematical operation problem. In this scenario, the user inputs a set of numbers and hopes to get the result of 24 points for these numbers. The reasoning system of this embodiment can automatically perform calculations and give the correct answer. To achieve this goal, a thought forest is first constructed, which contains multiple thought trees. During the construction process, the complex problem is decomposed into simpler solution steps, and the splitting nodes are selected according to the scores. Among them, this embodiment optimizes the search strategy of the thought trees. For example, in the 24-point mathematical operation problem, if the score of a certain step is lower than a certain threshold or is an invalid output, the construction of the current subtree is stopped immediately to avoid unnecessary calculations; in the reasoning result of a single tree, when the calculation of the mathematical expression meets the termination condition, the termination condition of the thought forest is also met, and voting and large model selection are no longer needed to obtain the optimal result.

[0202] To further illustrate the effectiveness of the method in the embodiments of this application, related experimental results are also provided. In these embodiments, experiments were conducted based on the reasoning method based on the thought forest provided in these embodiments. The results of these experiments are merely illustrative of the beneficial effects of the reasoning method in these embodiments and do not constitute a limitation on these embodiments. The 24-point mathematical operation problem is also known as the Game of 24.

[0203] In the reasoning process of the 24-point game, this application embodiment introduces a dynamic error correction strategy to ensure the accuracy of each step of reasoning, thereby improving the precision of the final result. Specifically, we first adopt a mathematical rule-based error correction mechanism, which quickly detects and corrects calculation errors through preset mathematical rules, and adjusts the reasoning path in a timely manner. This can avoid path deviations caused by erroneous reasoning, thereby saving resources and time.

[0204] As shown in Table 1, the inference methods used in the experiment included the baseline method (Standard), CoT (CoT), ToT (ToT), and FoT (Forest of Reasoning). The baseline method (Standard) means that no processing is done; the input data is directly combined with prompt words and input into the large language model, and the large language model outputs the inference result. * indicates that the corresponding method uses the dynamic error correction strategy based on fixed rules and learnable rules in the embodiments of this application. b represents the number of sparsely activated nodes, for example, in step S750, the top 5 nodes with sparse activation scores are activated. t represents the number of thought trees in the FoT, for example, in Figure 3, the number of thought trees in the FoT is four. The test set represents the category of the test set. The test set for the 24-point game includes several test samples, which are four different integers less than 13. As shown in Table 1, the accuracy of FoT is significantly higher than other methods. Among them, applying the dynamic error correction strategy in ToT and FoT can improve the accuracy of the large language model inference by 56.25%.

[0205] Table 1. Experimental results of Mind Forest (FoT) in the 24-point game.

[0206] Please refer to Figure 8, which illustrates the computational requirements for achieving different accuracies using the thought forest and thought tree methods. As shown in Figure 8, applying the thought forest inference method not only significantly improves the accuracy of large language models, but also compares the computational requirements of the thought forest and thought tree methods. This computational cost is the average number of inferences across multiple trials. For example, the computational cost for FoT (b=5, t=4) is 23.64, meaning that when the parameters of FoT are (b=5, t=4), the average number of inferences across multiple trials is 23.64, and the accuracy of the obtained inference result is 91.58%. The experimental results, as shown in Figure 8, demonstrate that without significantly increasing computational cost, the FoT method based on diversified inference paths shows a more significant improvement in accuracy compared to the extended method based on ToT. By increasing the number of subtrees and processing multiple inference paths in parallel, FoT can explore more potential solutions and inference schemes, improving the model's comprehensiveness and fault tolerance when dealing with complex problems. In contrast, ToT mainly relies on extending the depth of a single path. When computing resources are limited, a single path may not be able to fully capture multiple solutions to a problem, leading to bottlenecks in problem-solving and preventing further improvement in accuracy.

[0207] Please refer to Figure 9, which illustrates the accuracy variations corresponding to different strategies in the Mind Forest framework. As shown in Figure 9, experiments were conducted to compare strategies in the Mind Forest framework, including consensus voting, large model judgment, and early stopping mechanisms. Experimental results show that when the GT early stopping mechanism is introduced, the model achieves an accuracy of 93.33%. Without the GT early stopping mechanism in the test set, the inference method of this application, which integrates large model judgment and consensus voting, achieves an accuracy of 85.44% for the large language model, higher than the 83.44% accuracy of the single strategy of large model judgment and the 83.7% accuracy of the single strategy of majority consensus voting. In Figure 9, t represents the number of mind trees in the Mind Forest.

[0208] Based on the same concept as the foregoing embodiments, this application also provides an inference device.

[0209] Please refer to Figure 10, which shows a schematic diagram of the composition of an inference device. As shown in Figure 10, this application provides a second aspect, namely, an inference device 1000, which mainly includes:

[0210] The acquisition module 1010 is used to acquire input data.

[0211] The processing module 1020 is used to generate multiple prompts based on the input data. These prompts guide the large language model to reason about the input data from different perspectives, such as through different prompt word templates. It also calls the large language model to construct multiple thought trees, with each prompt as the root node. The thought trees represent the reasoning path of the large language model in reasoning about the input data. Finally, it obtains the target reasoning result of the input data based on the multiple reference reasoning results corresponding to the multiple thought trees.

[0212] In one possible implementation, the aforementioned processing module 1020 is specifically used for: obtaining the root node of the thought tree based on the prompt information during the construction of each thought tree; inputting the prompt information into the large language model to output the response content corresponding to the prompt information; and obtaining the child nodes corresponding to the root node based on the response content.

[0213] In one possible implementation, the above-mentioned processing module 1020 is further configured to: correct the child node according to the preset error correction rules, and obtain the corrected child node.

[0214] In one possible implementation, the aforementioned preset error correction rules include fixed rules, which are obtained through existing rule requirements of the input data. The aforementioned processing module 1020 is specifically used to: determine whether a child node conforms to the inherent rules; when a child node does not conform to the inherent rules, correct the child node according to the inherent rules to obtain the corrected child node.

[0215] In one possible implementation, the aforementioned processing module 1020 is specifically used to: call a large language model to score the child nodes to obtain the scores of the child nodes; determine whether the scores of the child nodes are less than a preset score threshold; when the scores of the child nodes are less than the preset score threshold, call the large language model so that the large language model generates corrected child nodes based on historical failure cases and child nodes.

[0216] In one possible implementation, the processing module 1020 is further configured to: determine whether a child node meets a preset stopping condition; and stop building a branch of the child node when the child node meets the preset stopping condition.

[0217] In one possible implementation, the aforementioned preset stopping condition includes at least one of a first condition and a second condition, wherein the first condition indicates that the score of the child node is lower than a preset threshold, and the second condition indicates that the information of the child node contains errors.

[0218] In one possible implementation, the processing module 1020 is further configured to: when there are multiple child nodes, call the large language model to score the multiple child nodes to obtain scores for the multiple child nodes; and retain the target number of child nodes based on the scores of the multiple child nodes.

[0219] In one possible implementation, the processing module 1020 is specifically used to: obtain the target inference result from multiple reference inference results according to a preset strategy.

[0220] In one possible implementation, the aforementioned preset strategy includes a first strategy and a second strategy. The first strategy represents a strategy based on the voting principle, and the second strategy represents a strategy based on the judgment of a large language model. The aforementioned processing module 1020 is specifically used for: voting on multiple reference inference results according to the first strategy to obtain a voting result; calling the large language model according to the second strategy to judge the multiple reference inference results to obtain a judgment result; and obtaining the target inference result from the multiple reference inference results based on the voting result and the judgment result.

[0221] In one possible implementation, the processing module 1020 is specifically used to: when the voting result conforms to the majority consensus principle, obtain the target inference result from multiple reference inference results based on the voting result; when the voting result does not conform to the majority consensus principle, obtain the target inference result from multiple reference inference results based on the judgment result.

[0222] In one possible implementation, the aforementioned preset strategy further includes a third strategy, which represents a strategy based on real data, where the real data is data related to the target inference result. The aforementioned processing module 1020 is specifically used to: match multiple reference inference results with real data according to the third strategy; and obtain the target inference result from the multiple reference inference results based on the matching results.

[0223] In one possible implementation, the processing module 1020 is further configured to: determine whether the nodes in the mind tree conform to the preset processing rules; when the nodes in the mind tree conform to the preset processing rules, generate the child nodes corresponding to the nodes in the mind tree according to the preset processing rules.

[0224] In one possible implementation, the processing module 1020 is specifically used to: obtain multiple prompt messages by combining multiple prompt word templates according to the input data.

[0225] The software and hardware implementation of the inference device 1000 shown in Figure 10 will be further explained below.

[0226] As an example of a software functional unit, a module can include code running on a computing instance. A computing instance can include at least one of a physical host (computing device), a virtual machine, or a container. Furthermore, the aforementioned computing instance can be one or more. For example, a module can include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code can be distributed within the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code can be distributed within the same availability zone (AZ) or in different AZs, each AZ comprising one or more geographically proximate data centers. Typically, a region can include multiple AZs.

[0227] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0228] As an example of a hardware functional unit, a module may include at least one computing device, such as a server. Alternatively, a module may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD can be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0229] The multiple computing devices included in the module can be distributed within the same region or in different regions. Similarly, the multiple computing devices included in the module can be distributed within the same Availability Zone (AZ) or in different AZs. Likewise, the multiple computing devices included in the module can be distributed within the same Virtual Private Cloud (VPC) or multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0230] It should be noted that, in other embodiments, the inference device 1000 is additionally provided with one or more modules for performing any of the steps included in the above implementation. The steps implemented by one or more modules in the inference device 1000 can be specified as needed, and more or fewer modules than in the embodiments of this application can be obtained to implement different steps in the above method, thereby realizing all the functions of the inference device 1000.

[0231] This application also provides a computing device 1100. As shown in FIG11, the computing device 1100 includes: a bus 1102, a processor 1104, a memory 1106, and a communication interface 1108. The processor 1104, the memory 1106, and the communication interface 1108 communicate with each other via the bus 1102. The computing device 1100 can be a server, such as a central server, an edge server, or a local server in a local data center, or it can be an electronic device such as a desktop computer, a laptop computer, or a smartphone. It should be understood that this application does not limit the number of processors and memories in the computing device 1100.

[0232] Bus 1102 can be a Peripheral Component Interconnect (PCI) bus, a Peripheral Component Interconnect Express (PCIe) bus, an Extended Industry Standard Architecture (EISA) bus, a Unified Bus (Ubus or UB), a Compute Express Link (CXL), a Cache Coherent Interconnect for Accelerators (CCIX), etc. The Unified Bus is also known as the Lingqu Bus. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 11 uses only one line, but this does not imply that there is only one bus or one type of bus. Bus 1104 can include pathways for transmitting information between various components of computing device 1100 (e.g., memory 1106, processor 1104, communication interface 1108).

[0233] The processor 1104 may include any one or more of the following computing devices: central processing unit (CPU), graphics processing unit (GPU), microprocessor (MP), digital signal processor (DSP), ASIC, FPGA, CPLD, NPU, SoC, offload card, accelerator card, etc.

[0234] Memory 1106 may include volatile memory, such as random access memory (RAM). Processor 1104 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD). Furthermore, memory 1106 may also be implemented using storage class memory (SCM), phase change memory (PCM), or other types of storage media.

[0235] It is worth noting that the same type of storage medium can be configured in the same computing device to realize the function of memory 1106, or two or more types of storage media can be configured to realize the function of memory 1106. This application does not limit this.

[0236] The memory 1106 stores executable program code, and the processor 1104 executes the executable program code to implement the functions of the apparatus in the embodiments of this application, thereby implementing the methods described in the above embodiments. That is, the memory 1106 stores instructions for executing the methods described in the above embodiments.

[0237] Alternatively, the memory 1106 may store executable code, which the processor 1104 executes to implement the functions of the apparatus in the embodiments of this application, thereby implementing the methods described in the above embodiments. That is, the memory 1106 stores instructions for executing the methods described in the above embodiments.

[0238] The communication interface 1108 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the computing device 1100 and other devices or communication networks.

[0239] As one possible implementation, the computing device 1100 may also include a chip system, which includes a processor and a power supply circuit. The power supply circuit supplies power to the processor, and the processor executes the operation steps corresponding to the methods of the embodiments of this application. For simplicity, further details are omitted here. The processor can be implemented using a GPU, or it can be implemented using computing devices or AI chips such as a DPU, NPU, XPU, SoC, offload card, or accelerator card.

[0240] As one possible implementation, the computing device 1100 may include various types of processors 1104, that is, the computing device 1100 is a heterogeneous device. For example, the computing device 1100 includes a CPU and a GPU, and at least one of the processors 1104 can execute the operation steps corresponding to the method of the embodiments of this application. For the sake of brevity, further details are omitted here.

[0241] This application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be an electronic device such as a desktop computer, a laptop computer, or a smartphone.

[0242] As shown in Figure 12, the computing device cluster includes at least one computing device 1100. The memory 1106 of one or more computing devices 1100 in the computing device cluster may store the same instructions for performing the methods described in the above embodiments.

[0243] In some possible implementations, the memory 1106 of one or more computing devices 1100 in the computing device cluster may also store partial instructions for executing the methods described in the above embodiments. In other words, a combination of one or more computing devices 1100 can jointly execute instructions for executing the methods described in the above embodiments.

[0244] It should be noted that the memory 1106 in different computing devices 1100 within the computing device cluster can store different instructions, which are used to execute some functions of the inference device 1000 (hereinafter referred to as "the device of this application embodiment"). That is, the instructions stored in the memory 1106 in different computing devices 1100 can implement the functions of one or more modules of the device of this application embodiment.

[0245] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 13 illustrates one possible implementation. As shown in Figure 13, two computing devices 1100A and 1100B are connected via a network. Specifically, they are connected to the network through communication interfaces in each computing device. In this type of possible implementation, the memory 1106 in computing device 1100A stores instructions for the functions of one or more modules of the apparatus according to the embodiments of this application. Simultaneously, the memory 1106 in computing device 1100B stores instructions for the functions of another one or more modules of the apparatus according to the embodiments of this application.

[0246] It should be understood that the functions of computing device 1100A shown in Figure 13 can also be performed by multiple computing devices 1100. Similarly, the functions of computing device 1100B can also be performed by multiple computing devices 1100.

[0247] This application also provides another computing device cluster. The connection relationship between the computing devices in this computing device cluster can be similarly referred to the connection method of the computing device cluster described in Figures 12 and 13. The difference is that the memory 1106 of one or more computing devices 1100 in this computing device cluster can store the same instructions for executing the methods in the above embodiments.

[0248] In some possible implementations, the memory 1106 of one or more computing devices 1100 in the computing device cluster may also store partial instructions for executing the aforementioned data processing method. In other words, a combination of one or more computing devices 1100 can jointly execute instructions for performing the aforementioned method.

[0249] Based on the methods in the above embodiments, this application provides a computer-readable storage medium including computer program instructions. When executed by a cluster of computing devices including at least one computing device, the computer program instructions cause the cluster of computing devices to perform the methods in the above embodiments. Exemplarily, the computer-readable storage medium can be any available medium capable of being stored in the cluster of computing devices or a data storage device such as a data center containing one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives).

[0250] Based on the methods in the above embodiments, this application provides a computer program product containing instructions. The computer program product may be software or program products containing instructions, capable of running on a computing device or stored on any available medium. When the instructions are executed by a cluster of computing devices containing at least one computing device, at least one computing device in the cluster of computing devices performs the methods in the above embodiments.

[0251] It is understood that the processor in the embodiments of this application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor may be a microprocessor or any conventional processor.

[0252] The method steps in the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0253] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0254] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application.

[0255] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. An inference method, characterized by, The method comprises: obtaining input data; generating a plurality of prompt information according to the input data, the plurality of prompt information being used to guide a large language model to reason the input data from different angles; calling the large language model with each of the plurality of prompt information as a root node, and constructing a plurality of thinking trees, the plurality of thinking trees representing reasoning paths of the large language model for reasoning the input data; obtaining a target reasoning result of the input data according to a plurality of reference reasoning results corresponding to the plurality of thinking trees.

2. The method of claim 1, wherein, The method of calling the large language model with each of the plurality of prompt information as a root node and constructing a plurality of thinking trees comprises: obtaining a root node of each thinking tree according to the prompt information during the construction process of the thinking tree; inputting the prompt information into the large language model to output reply content corresponding to the prompt information; obtaining a child node corresponding to the root node according to the reply content.

3. The method of claim 2, wherein, The method further comprises: correcting the child node according to a preset error correction rule to obtain a corrected child node.

4. The method of claim 3, wherein, The preset error correction rule comprises a fixed rule obtained by an existing rule of the input data; The method of correcting the child node according to the preset error correction rule to obtain a corrected child node comprises: judging whether the child node conforms to the fixed rule; when the child node does not conform to the fixed rule, correcting the child node according to the fixed rule to obtain a corrected child node.

5. The method according to claim 3 or 4, characterized in that, The method of correcting the child node according to the preset error correction rule to obtain a corrected child node comprises: calling the large language model to score the child node to obtain a score of the child node; judging whether the score of the child node is less than a preset score threshold; when the score of the child node is less than the preset score threshold, calling the large language model to enable the large language model to generate a corrected child node according to a historical failure case and the child node.

6. The method according to any one of claims 2-5, characterized in that, The method further comprises: judging whether the child node conforms to a preset stopping condition; when the child node conforms to the preset stopping condition, stopping the construction of a branch of the child node.

7. The method of claim 6, wherein, The preset stopping condition comprises at least one of a first condition and a second condition, the first condition indicating that the score of the child node is lower than a preset threshold, and the second condition indicating that information of the child node has an error.

8. The method according to any one of claims 2 to 7, characterized in that, The method further comprises: when the number of the child nodes is a plurality, calling the large language model to score a plurality of child nodes to obtain scores of the plurality of child nodes; retaining a target number of child nodes according to the scores of the plurality of child nodes.

9. The method according to any one of claims 2 to 8, characterized in that, The method of obtaining a target reasoning result of the input data according to a plurality of reference reasoning results corresponding to the plurality of thinking trees comprises: obtaining the target reasoning result from the plurality of reference reasoning results according to a preset strategy.

10. The method of claim 9, wherein, The preset strategy comprises a first strategy and a second strategy, the first strategy indicating a strategy based on a voting principle, and the second strategy indicating a strategy based on a judgment of the large language model; The method of obtaining the target reasoning result from the plurality of reference reasoning results according to the preset strategy comprises: According to the first strategy, a vote is taken on the multiple reference inference results to obtain the voting result; According to the second strategy, the large language model is invoked to judge the multiple reference inference results in order to obtain the judgment result; Based on the voting results and the judgment results, the target reasoning result is obtained from the multiple reference reasoning results.

11. The method of claim 10, wherein, The step of obtaining the target reasoning result from the plurality of reference reasoning results based on the voting results and the judgment results includes: When the voting result meets the majority consensus principle, the target inference result is obtained from the plurality of reference inference results based on the voting result; When the voting result does not conform to the majority consensus principle, the target reasoning result is obtained from the multiple reference reasoning results based on the judgment result.

12. The method of claim 9, wherein, The preset strategy also includes a third strategy, which represents a strategy based on real data, where the real data is data related to the target reasoning result. The step of obtaining the target inference result from the plurality of reference inference results according to a preset strategy includes: According to the third strategy, the multiple reference inference results are matched with the real data; Based on the matching results, the target inference result is obtained from the plurality of reference inference results.

13. The method according to any one of claims 1 to 12, characterized in that, Also includes: Determine whether the nodes in the mind tree conform to the preset processing rules; When a node in the mind tree conforms to the preset processing rule, a child node corresponding to the node in the mind tree is generated according to the preset processing rule.

14. The method of claims 1-13, wherein, The step of generating multiple prompt messages based on the input data includes: Based on the input data, and by combining multiple prompt word templates, the corresponding multiple prompt messages are obtained.

15. An inference apparatus characterized by comprising: include: The acquisition module is used to acquire input data; The processing module is configured to generate multiple prompts based on the input data, the multiple prompts being used to guide the large language model to reason about the input data from different perspectives; and, using each of the multiple prompts as a root node, calling the large language model to construct multiple thought trees, the thought trees representing the reasoning path of the large language model in reasoning about the input data. Furthermore, based on the multiple reference reasoning results corresponding to the multiple mind trees, the target reasoning result of the input data is obtained.

16. The apparatus of claim 15, wherein, The processing module is specifically used for: During the construction of each mind tree, the root node of the mind tree is obtained based on the prompt information. The prompt information is input into the large language model to output the response content corresponding to the prompt information; Based on the response, the child nodes corresponding to the root node are obtained.

17. The apparatus of claim 16, wherein, The processing module is further configured to: The child nodes are corrected according to the preset error correction rules to obtain the corrected child nodes.

18. The apparatus of claim 16 or 17, wherein, The processing module is specifically used for: According to a preset strategy, the target reasoning result is obtained from the multiple reference reasoning results.

19. A chip system, characterized by The chip system includes a processor and a power supply circuit, the power supply circuit being used to supply power to the processor, the processor being used to perform the method as described in any one of claims 1-14.

20. A computing device, comprising: The computing device includes a processor and memory; The processor is configured to execute instructions stored in the memory to cause the computing device to perform the method of any of claims 1-14.

21. A cluster of computing devices, characterized in that, comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method of any of claims 1-14.

22. A computer-readable storage medium, characterized in that, comprising computer program instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method of any of claims 1-14, wherein the cluster of computing devices comprises at least one computing device.

23. A computer program product comprising instructions, characterized in that, comprising computer program instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method of any of claims 1-14, wherein the cluster of computing devices comprises at least one computing device.

Images