Big language model logical reasoning method and device, equipment, storage medium and product

Abstract

The invention discloses a big language model logical reasoning method, device and equipment, a storage medium and a product, and relates to the technical field of big language models.The method comprises the steps that a logical reasoning tree is constructed according to a preset problem, an intermediate step generated by a logic big language model and a reasoning path; alternately training the logic big language model and the logic tree scoring model to obtain a trained logic big language model and a trained logic tree scoring model; and according to the trained logic tree scoring model, evaluating a plurality of next-step reasoning paths generated by the trained logic big language model according to the logic reasoning tree to obtain an optimal reasoning path. According to the method, the training logic tree scoring model is used for replacing an existing prompt large model scoring strategy, a prompt or a high-level large language large model does not need to be preset in the scoring process, and the logic tree scoring model and the logic large model are alternately trained, so that the accuracy of model reasoning is improved in the face of complex problems.

Description

Technical Field

[0001] The present application relates to the technical field of large language models, and in particular to a large language model logical reasoning method, device, equipment, storage medium and product. Background Art

[0002] With the rapid growth of model size, Transformer-based Large Language Models (LLMs) have demonstrated impressive results in areas such as instruction following and coding assistance. In these tasks, how to solve complex problem reasoning is a major challenge for existing LLM methods. Usually, the score of sub-step reasoning is obtained by prompting LLM. The way such algorithms obtain scores lacks universal applicability and relies heavily on carefully designed prompts and the powerful functions of advanced LLMs. Secondly, the scoring process repeatedly calls the prompt LLM, which incurs huge computational overhead. Existing algorithms are usually limited to 10 sub-step scoring. As a result, its capabilities are limited to relatively simple and superficial tasks, making it difficult to solve complex problems that require a lot of analytical depth and long-term planning vision, reducing scalability. Summary of the invention

[0003] The main purpose of this application is to provide a large language model logical reasoning method, device, equipment, storage medium and product, aiming to solve the technical problem of low accuracy of existing large language models when facing complex problems.

[0004] To achieve the above objectives, the present application proposes a large language model logical reasoning method, the method comprising: Construct a logical reasoning tree based on preset questions, intermediate steps and reasoning paths generated by the logical large language model; Alternately training the logical large language model and the logical tree scoring model to obtain a trained logical large language model and a trained logical tree scoring model; The trained logic tree scoring model is used to evaluate multiple next-step reasoning paths generated by the trained logic large language model according to the logic reasoning tree to obtain the best reasoning path.

[0005] In one embodiment, the step of alternately training the logical large language model and the logical tree scoring model to obtain the trained logical large language model and the trained logical tree scoring model includes: The logic tree scoring model is trained according to the preset scoring data set and the logic large language model to obtain a trained logic tree scoring model, wherein the parameters of the logic large language model are fixed; The logical large language model is trained according to the trained logical tree scoring model to obtain a trained logical large language model, wherein the parameters of the trained logical tree scoring model are fixed.

[0006] In one embodiment, the step of training the logical large language model according to the trained logical tree scoring model to obtain the trained logical large language model includes: Acquire an instruction output data set, wherein the instruction output data set includes a preset instruction and an expected output corresponding to the preset instruction; The logical large language model is subjected to instruction fine-tuning training according to the trained logical tree scoring model and the instruction output data set to obtain a trained logical large language model.

[0007] In one embodiment, the step of evaluating the multiple next-step reasoning paths generated by the trained logic large language model according to the logic reasoning tree according to the trained logic tree scoring model to obtain the best reasoning path includes: Determine a current reasoning path corresponding to a current task according to the logical reasoning tree; Iteratively evaluate multiple next-step reasoning paths generated by the trained logical large language model according to the current task and the current reasoning path according to the trained logic tree scoring model to obtain the best reasoning path.

[0008] In one embodiment, the step of iteratively evaluating multiple next-step reasoning paths generated by the trained logical large language model according to the current task and the current reasoning path according to the trained logic tree scoring model to obtain the best reasoning path includes: Iteratively generate multiple next-step reasoning paths according to the current task and the current reasoning path through the trained logical large language model; Evaluate the multiple next-step reasoning paths according to the current reasoning path according to the trained logic tree scoring model, and obtain a scoring score corresponding to each of the next-step reasoning paths; When the preset number of iterations is reached, the next reasoning path with the highest score is taken as the best reasoning path.

[0009] In one embodiment, the step of constructing a logical reasoning tree according to the preset question, the intermediate steps generated by the logical large language model, and the reasoning path includes: Construct the root node of the logical reasoning tree according to the preset questions; Generate intermediate steps according to the preset question through a logical large language model, and use the intermediate steps as edges of a logical reasoning tree; A reasoning path is determined according to the intermediate steps, and child nodes of a logical reasoning tree are constructed according to the reasoning path.

[0010] In addition, to achieve the above purpose, the present application also proposes a large language model logic reasoning device, the large language model logic reasoning device comprising: A logical reasoning tree construction module is used to construct a logical reasoning tree based on preset questions, intermediate steps and reasoning paths generated by the logical large language model; A model alternating training module is used to perform alternating training on the logical large language model and the logical tree scoring model to obtain a trained logical large language model and a trained logical tree scoring model; The reasoning path module scoring is used to evaluate multiple next-step reasoning paths generated by the trained logic large language model according to the logic reasoning tree according to the trained logic tree scoring model to obtain the best reasoning path.

[0011] In addition, to achieve the above-mentioned objectives, the present application also proposes a large language model logical reasoning device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the large language model logical reasoning method described above.

[0012] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the large language model logical reasoning method described above are implemented.

[0013] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the large language model logical reasoning method described above.

[0014] The present application provides a large language model logical reasoning method, which constructs a logical reasoning tree according to preset questions, intermediate steps and reasoning paths generated by the logical large language model; alternately trains the logical large language model and the logical tree scoring model to obtain a trained logical large language model and a trained logical tree scoring model; and evaluates the multiple next-step reasoning paths generated by the trained logical large language model according to the logical reasoning tree according to the trained logical tree scoring model to obtain the best reasoning path. Since the present application replaces the existing prompt large model scoring strategy by training the logical tree scoring model, there is no need to preset prompts or advanced large language large models during the scoring process, and alternately trains the logical tree scoring model and the logical large model, thereby improving the accuracy of model reasoning when facing complex problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0017] Figure 1 A flowchart diagram of the first embodiment of the large language model logical reasoning method of the present application; Figure 2 A flowchart diagram of the second embodiment of the large language model logical reasoning method of the present application; Figure 3 This is a flow chart of model alternating training in the large language model logical reasoning method of this application; Figure 4 A flowchart diagram of the third embodiment of the large language model logical reasoning method of the present application; Figure 5 A flowchart of the iterative reasoning process in the large language model logical reasoning method of this application; Figure 6 This is a schematic diagram of the module structure of the large language model logic reasoning device according to an embodiment of the present application; Figure 7 This is a schematic diagram of the device structure of the hardware operating environment involved in the large language model logical reasoning method in the embodiment of the present application.

[0018] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0019] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0020] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0021] The main solution of the embodiment of the present application is: to build a logical reasoning tree according to preset problems, intermediate steps and reasoning paths generated by the logical large language model; to alternately train the logical large language model and the logical tree scoring model to obtain a trained logical large language model and a trained logical tree scoring model; to evaluate multiple next-step reasoning paths generated by the trained logical large language model according to the logical reasoning tree according to the trained logical tree scoring model to obtain the best reasoning path.

[0022] As the model size grows rapidly in the prior art, Transformer-based Large Language Models (LLMs) have demonstrated impressive results in areas such as instruction following and coding assistance. In these tasks, how to solve complex problem reasoning is a major challenge for existing LLM methods. Usually, the score of sub-step reasoning is obtained by prompting the LLM. The way such algorithms obtain scores lacks universal applicability and is heavily dependent on well-designed prompts and the powerful functions of advanced LLMs. Secondly, multiple calls to the prompt LLM during the scoring process will bring huge computational overhead, and existing algorithms are usually limited to 10 sub-step scoring. As a result, its capabilities are limited to relatively simple and superficial tasks, and it is difficult to solve complex problems that require a lot of analytical depth and long-term planning vision, which reduces scalability.

[0023] The present application provides a solution, which constructs a logical reasoning tree according to preset questions, intermediate steps and reasoning paths generated by a logical large language model; alternately trains the logical large language model and the logical tree scoring model to obtain a trained logical large language model and a trained logical tree scoring model; and evaluates the multiple next-step reasoning paths generated by the trained logical large language model according to the logical reasoning tree according to the trained logical tree scoring model to obtain the best reasoning path. Since the present application replaces the existing prompt large model scoring strategy by training the logical tree scoring model, there is no need to preset prompts or advanced large language large models during the scoring process, and alternately trains the logical tree scoring model and the logical large model, thereby improving the accuracy of model reasoning when facing complex problems.

[0024] It should be noted that the execution subject of the method of this embodiment can be a computing service device with large language model logic reasoning, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc.; it can also be a large language model logic reasoning device with the same or similar functions. This embodiment and the following embodiments will be described by taking the large language model logic reasoning device as an example.

[0025] Based on this, the embodiment of the present application provides a large language model logical reasoning method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the large language model logical reasoning method of the present application.

[0026] In this embodiment, the large language model logical reasoning method includes steps S10 to S30: Step S10, constructing a logical reasoning tree according to preset questions, intermediate steps generated by the logical large language model and reasoning paths.

[0027] It can be understood that the logic tree reasoning method decomposes complex problems into several small and easy-to-handle units, solves these unit problems one by one, and gradually derives the final conclusion. This method introduces a multi-step reasoning mechanism to enhance the model's ability to handle complex problems and generalizes the thinking chain method, which helps to explore and solve a series of sub-problems in a coherent manner.

[0028] It should be understood that the symbol represents the logical large language model (logical LLM), where Represents the trainable parameters of the logical LLM. A logical reasoning tree is constructed based on the preset questions, the intermediate steps generated by the logical large language model, and the reasoning path. It is used to express the reasoning path corresponding to the preset questions through a tree structure, so as to solve the logical reasoning of complex problems.

[0029] In a feasible implementation, step S10 may include steps S101 to S103: Step S101, constructing a root node of a logical reasoning tree according to a preset question.

[0030] Step S102: Generate intermediate steps according to the preset question through a large logical language model, and use the intermediate steps as edges of a logical reasoning tree.

[0031] Step S103, determining a reasoning path according to the intermediate steps, and constructing child nodes of a logical reasoning tree according to the reasoning path.

[0032] It should be noted that the root node of the logical reasoning tree can be constructed according to the preset problem to be solved, and the intermediate steps can be generated according to the preset problem through the logical large language model, and the intermediate steps are used as the edges of the logical reasoning tree. Finally, the reasoning path is determined according to the intermediate steps, and the child nodes of the logical reasoning tree are constructed according to the reasoning path. That is, the root node in the logical reasoning tree represents the problem, the edge represents the intermediate step generated by the large language model, and each child node represents an inference path, which is composed of the edges in the path from the root node to the child node, marked as , Indicates depth The edge at (sub-step), Represents a concatenation operation.

[0033] Step S20, alternately train the logical large language model and the logical tree scoring model to obtain a trained logical large language model and a trained logical tree scoring model.

[0034] It should be understood that in this embodiment, the existing prompt large model scoring strategy is replaced by training the logic tree scoring model. The goal is to evaluate the intermediate step of generating the logic large language model by training the logic tree scoring model to help the model select the correct reasoning path according to the scoring score. Specifically, a scoring model can be defined , Represents the trainable parameters of the scoring model. Then the logical large language model and the logical tree scoring model are trained alternately. First, the logical LLM is fixed, and a large number of scoring data sets are collected for training the logical tree scoring model. Once the logical tree scoring model is trained, the logical tree scoring model is fixed, and the logical LLM is trained using instruction fine-tuning, thereby improving the performance of each of the two models by alternately fixing and updating them. Thus, a trained logical large language model and a trained logical tree scoring model with better performance are obtained.

[0035] Step S30, evaluating multiple next-step reasoning paths generated by the trained large logic language model according to the logic reasoning tree according to the trained logic tree scoring model to obtain the best reasoning path.

[0036] It is understandable that the trained logical large language model can be used to generate multiple next-step reasoning paths based on the logical reasoning tree, and then the scoring scores corresponding to each next-step reasoning path can be evaluated based on the trained logical tree scoring model, and the best reasoning path can be selected based on the scoring scores to obtain results for complex problems.

[0037] This embodiment provides a large language model logical reasoning method, which constructs a logical reasoning tree according to preset questions, intermediate steps and reasoning paths generated by the logical large language model; alternately trains the logical large language model and the logical tree scoring model to obtain a trained logical large language model and a trained logical tree scoring model; evaluates the multiple next-step reasoning paths generated by the trained logical large language model according to the logical reasoning tree according to the trained logical tree scoring model to obtain the best reasoning path. Since this application replaces the existing prompt large model scoring strategy by training the logical tree scoring model, there is no need to preset prompts or advanced large language large models during the scoring process, and alternately trains the logical tree scoring model and the logical large model, thereby improving the accuracy of model reasoning when facing complex problems.

[0038] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can refer to the above introduction, and will not be repeated later. Figure 2 , step S20, the large language model logical reasoning method further includes steps S201~S202: Step S201: training a logic tree scoring model according to a preset scoring data set and a logic large language model to obtain a trained logic tree scoring model, wherein the parameters of the logic large language model are fixed.

[0039] It should be understood that when the logical large language model and the logical tree scoring model are alternately trained, the logical large language model is fixed first. At this stage, the parameters of the logical large language model are fixed and will not be updated. That is, when training the logical tree scoring model, the parameters of the logical large language model will not change. When the parameters of the logical large language model are fixed, a large number of scoring data sets are collected for training the logical tree scoring model. The scoring data set may include various inputs and corresponding scores, which reflect certain characteristics or qualities of the inputs. The collected scoring data set is then used to train the logical tree scoring model to obtain a trained logical tree scoring model. In this process, the parameters of the logical tree scoring model will be adjusted to better fit the scoring data.

[0040] Step S202: training the logical large language model according to the trained logical tree scoring model to obtain a trained logical large language model, wherein the parameters of the trained logical tree scoring model are fixed.

[0041] It is understandable that after the training of the logical tree scoring model is completed, the parameters of the trained logical tree scoring model are fixed and no further updates are performed. When the parameters of the trained logical tree scoring model are fixed, the logical large language model is trained by fine-tuning the instructions so that the model can better understand and execute natural language instructions. The specific process of model alternating training can be referred to Figure 3 .

[0042] In a feasible implementation, step S202 may include steps S2021-S2022: Step S2021, obtaining an instruction output data set, wherein the instruction output data set includes a preset instruction and an expected output corresponding to the preset instruction.

[0043] It should be understood that before fine-tuning the logical large language model by means of instruction fine-tuning, an instruction output dataset is first obtained, and the instruction output dataset includes instruction-output pairs, wherein the instruction represents the human instruction of the model and the output represents the expected output following the instruction.

[0044] Step S2022, performing instruction fine-tuning training on the logic large language model according to the trained logic tree scoring model and the instruction output data set to obtain a trained logic large language model.

[0045] It is understandable that the logical large language model is fine-tuned according to the instruction output data set of the trained logical tree scoring model, and the logical large language model is fine-tuned according to the specified instructions or goals to improve its performance on specific tasks. The trained logical tree scoring model and the trained logical large language model can be iteratively and alternately trained to improve the performance of both.

[0046] In this embodiment, the logic tree scoring model and the logic large language model are trained alternately. First, the logic large language model is fixed, and a large amount of scoring data sets are collected for training the logic tree scoring model. When the logic tree scoring model is trained, the logic tree scoring model is fixed, and the logic large language model is trained by instruction fine-tuning. The two models are alternately fixed and updated to improve their respective performance. This alternating training method can ensure that each model can benefit from the improvement of the other model, thereby improving the performance of the overall system.

[0047] Based on the first embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the first embodiment can be referred to the above description, and will not be described in detail later. Figure 2 , step S30, the large language model logical reasoning method further includes steps S301~S302: Step S301: Determine the current reasoning path corresponding to the current task according to the logical reasoning tree.

[0048] It can be understood that since each child node in the logical reasoning tree represents a reasoning path, which is composed of edges in the path from the root node to the child node, the child node corresponding to the current task can be found in the logical reasoning tree to obtain the current reasoning path.

[0049] Step S302: iteratively evaluating multiple next-step reasoning paths generated by the trained large logic language model according to the current task and the current reasoning path according to the trained logic tree scoring model to obtain the best reasoning path.

[0050] It should be understood that the logical large language model generates k next-step reasoning paths based on the current task and the current reasoning path obtained above, and then the logical tree scoring model performs scoring, selects the path with the highest score from the k next-step reasoning paths, and uses the path with the highest score as the current reasoning path for the next round of the algorithm, iterating to find the final optimal reasoning path.

[0051] In a feasible implementation, step S302 may include steps S3021 to S3023: Step S3021, iteratively generating multiple next-step reasoning paths according to the current task and the current reasoning path through the trained logical large language model.

[0052] It is understandable that the reference here Figure 5 The iterative reasoning process of the algorithm is explained. First, the trained logical large language model is used to generate k next reasoning paths according to the current task and the current reasoning path.

[0053] Step S3022: Evaluate the multiple next-step reasoning paths according to the current reasoning path according to the trained logic tree scoring model, and obtain a scoring score corresponding to each of the next-step reasoning paths.

[0054] It should be understood that the current reasoning path n i and k next-step reasoning paths e i They are respectively used as the input of the trained logic tree scoring model. After scoring, the logic tree scoring model outputs the scores between them, which are expressed as , Represents the trainable parameters of the logic tree scoring model, which is used as the score corresponding to each next step of the reasoning path. When evaluating a new step, the scoring model takes into account the previous reasoning steps (that is, the current reasoning path mentioned above), thereby encouraging the logical process to be consistent throughout the path. A high reward value indicates may be The correct next step, suggesting the trajectory is worth exploring. Conversely, a smaller reward value can be considered a wrong step, indicating that the trajectory It is most likely wrong.

[0055] Step S3023, when the preset number of iterations is reached, the next reasoning path with the highest score is used as the best reasoning path.

[0056] It is understandable that since a higher score indicates a higher correctness, the next reasoning path with the highest score can be used as the best in the current algorithm round, and then the best is used as the input of the next iteration, and the next reasoning path of the next round is iteratively generated and scored until the preset number of iterations is reached. The final best reasoning path can be obtained, and the output of the LLM model can be obtained. The number of iterations is related to the task. For example, for the 24-point task, the number of iterations is 3. For example, given the input (2 2 8 14), the final answer of the 24-point game is (14-8=6)-->(2*6=12)-->(2*12=24).

[0057] In this embodiment, during the model reasoning stage, a logic tree scoring model is used to score the next step options output by the logic large language model, and the option with the highest score is selected as the input for the next iteration until the number of iterations is reached, and the final optimal reasoning path is obtained, thereby improving the accuracy of the model, and no carefully designed prompts or advanced large LLMs are required in the scoring process.

[0058] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the large language model logical reasoning method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0059] This application also provides a large language model logic reasoning device, please refer to Figure 6 , the large language model logical reasoning device comprises: A logic reasoning tree construction module 10 is used to construct a logic reasoning tree according to preset questions, intermediate steps and reasoning paths generated by the logic large language model; A model alternating training module 20 is used to perform alternating training on the logical large language model and the logical tree scoring model to obtain a trained logical large language model and a trained logical tree scoring model; The reasoning path module scoring 30 is used to evaluate multiple next-step reasoning paths generated by the trained logic large language model according to the logic reasoning tree according to the trained logic tree scoring model to obtain the best reasoning path.

[0060] The large language model logic reasoning device provided by the present application adopts the large language model logic reasoning method in the above embodiment to solve the technical problem. Compared with the prior art, the beneficial effects of the large language model logic reasoning device provided by the present application are the same as the beneficial effects of the large language model logic reasoning method provided by the above embodiment, and the other technical features in the large language model logic reasoning device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0061] The present application provides a large language model logical reasoning device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the large language model logical reasoning method in the above-mentioned embodiment one.

[0062] Reference below Figure 7, which shows a schematic diagram of the structure of a large language model logic reasoning device suitable for implementing the embodiment of the present application. The large language model logic reasoning device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions: tablet computers), PMPs (Portable Media Players: portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The large language model logical reasoning device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0063] like Figure 7 As shown, the large language model logic reasoning device may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the large language model logic reasoning device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the large language model logic reasoning device to communicate with other devices wirelessly or by wire to exchange data. Although the large language model logic reasoning device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0064] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0065] The large language model logic reasoning device provided by the present application adopts the large language model logic reasoning method in the above embodiment, which can solve the technical problems of large language model logic reasoning. Compared with the prior art, the beneficial effects of the large language model logic reasoning device provided by the present application are the same as the beneficial effects of the large language model logic reasoning method provided by the above embodiment, and the other technical features in the large language model logic reasoning device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0066] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0067] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0068] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the large language model logical reasoning method in the above-mentioned embodiment.

[0069] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0070] The computer-readable storage medium may be included in the large language model logic reasoning device; or may exist independently without being assembled into the large language model logic reasoning device.

[0071] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the large language model logic reasoning device, the large language model logic reasoning device: builds a logic reasoning tree according to preset questions, intermediate steps and reasoning paths generated by the logical large language model; alternately trains the logical large language model and the logic tree scoring model to obtain a trained logical large language model and a trained logic tree scoring model; and evaluates multiple next-step reasoning paths generated by the trained logical large language model according to the logic reasoning tree according to the trained logic tree scoring model to obtain the best reasoning path.

[0072] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0073] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0074] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0075] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned large language model logical reasoning method, and can solve technical problems. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the large language model logical reasoning method provided in the above-mentioned embodiment, and will not be repeated here.

[0076] The present application also provides a computer program product, including a computer program, which implements the steps of the large language model logical reasoning method as described above when executed by a processor.

[0077] The computer program product provided by this application can solve the technical problem. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the large language model logical reasoning method provided by the above embodiment, which will not be repeated here.

[0078] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A large language model logical reasoning method, characterized in that: The method includes: Construct a logical reasoning tree based on preset questions, intermediate steps and reasoning paths generated by the logical large language model; Alternately training the logical large language model and the logical tree scoring model to obtain a trained logical large language model and a trained logical tree scoring model; The trained logic tree scoring model is used to evaluate multiple next-step reasoning paths generated by the trained logic large language model according to the logic reasoning tree to obtain the best reasoning path.

2. The method according to claim 1, characterized in that The step of alternately training the logical large language model and the logical tree scoring model to obtain the trained logical large language model and the trained logical tree scoring model comprises: The logic tree scoring model is trained according to the preset scoring data set and the logic large language model to obtain a trained logic tree scoring model, wherein the parameters of the logic large language model are fixed; The logical large language model is trained according to the trained logical tree scoring model to obtain a trained logical large language model, wherein the parameters of the trained logical tree scoring model are fixed.

3. The method according to claim 2, characterized in that The step of training the logical large language model according to the trained logical tree scoring model to obtain the trained logical large language model comprises: Acquire an instruction output data set, wherein the instruction output data set includes a preset instruction and an expected output corresponding to the preset instruction; The logical large language model is subjected to instruction fine-tuning training according to the trained logical tree scoring model and the instruction output data set to obtain a trained logical large language model.

4. The method according to claim 1, characterized in that The step of evaluating the multiple next-step reasoning paths generated by the trained logic large language model according to the logic reasoning tree according to the trained logic tree scoring model to obtain the best reasoning path includes: Determine a current reasoning path corresponding to a current task according to the logical reasoning tree; Iteratively evaluate multiple next-step reasoning paths generated by the trained logical large language model according to the current task and the current reasoning path according to the trained logic tree scoring model to obtain the best reasoning path.

5. The method according to claim 4, characterized in that The step of iteratively evaluating multiple next-step reasoning paths generated by the trained large logic language model according to the current task and the current reasoning path according to the trained logic tree scoring model to obtain the best reasoning path includes: Iteratively generate multiple next-step reasoning paths according to the current task and the current reasoning path through the trained logical large language model; Evaluate the multiple next-step reasoning paths according to the current reasoning path according to the trained logic tree scoring model, and obtain a scoring score corresponding to each of the next-step reasoning paths; When the preset number of iterations is reached, the next reasoning path with the highest score is taken as the best reasoning path.

6. The method according to claim 1, characterized in that The step of constructing a logical reasoning tree according to the preset questions, the intermediate steps generated by the logical large language model and the reasoning path includes: Construct the root node of the logical reasoning tree according to the preset questions; Generate intermediate steps according to the preset question through a logical large language model, and use the intermediate steps as edges of a logical reasoning tree; A reasoning path is determined according to the intermediate steps, and child nodes of a logical reasoning tree are constructed according to the reasoning path.

7. A large language model logical reasoning device, characterized in that: The large language model logical reasoning device comprises: A logical reasoning tree construction module is used to construct a logical reasoning tree based on preset questions, intermediate steps and reasoning paths generated by the logical large language model; A model alternating training module is used to perform alternating training on the logical large language model and the logical tree scoring model to obtain a trained logical large language model and a trained logical tree scoring model; The reasoning path module scoring is used to evaluate multiple next-step reasoning paths generated by the trained logic large language model according to the logic reasoning tree according to the trained logic tree scoring model to obtain the best reasoning path.

8. A large language model logical reasoning device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the large language model logical reasoning method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the large language model logical reasoning method as described in any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the large language model logical reasoning method according to any one of claims 1 to 6 are implemented.

Images