Long-distance reasoning method based on large model and symbolic logic and related equipment
By introducing symbolic logic and knowledge base into the big model inference technology, automatically planning inference steps and retrieving relevant data, the problem of insufficient reliability and interpretability in long-distance in big model inference is solved, and more efficient and credible reasoning is achieved. process.
Patent Information
- Application Number
- CN202411561199.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-04
- Publication Date
- 2025-02-18
AI Technical Summary
Existing long-distance inference techniques based on large models are prone to ‘illusion’ when facing complex problems, resulting in insufficient reliability and interpretability of inference results, and the problem of explosion in search space combination and stopping search criteria.
Using long-distance inference methods based on big models and symbolic logic, by building a knowledge base and fact database, using big models to perform semantic understanding and automatic planning of inference steps, forming several inference sub-objectives, and obtaining relevant rules and fact data through retrieval technology. Perform short-distance deductive reasoning and gradually advance the reasoning process.
It improves the reliability and interpretability of long-distance inference of large models, alleviates the ‘illusion’ phenomenon, solves the problem of difficulty in combining search space and stopping search criteria, and enhances the causality and interpretability of the inference results.
Smart Images
Figure CN119476509A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to artificial intelligence technology, and in particular to long-distance reasoning technology for large models. Background Art
[0002] With the rapid iteration of generative artificial intelligence, whether large models can demonstrate human-like reasoning ability has attracted more and more attention. When faced with long-distance reasoning problems, large models generally find it difficult to directly give the correct answer. Therefore, researchers have proposed a series of optimization algorithm frameworks based on large models, including chain of thoughts (CoT), least-to-most prompting, selection-inference, and backward reasoning (LAMBADA). Among them:
[0003] 1. Chain of Thought (CoT): The basic idea is to decompose the reasoning process into multiple steps, guide the large model to perform step-by-step reasoning, and describe each step in natural language so that the model can understand and perform the operations required at each reasoning stage. The specific process is as follows: Figure 1 shown.
[0004] 2. Least-to-Most Prompting: The basic idea is to use the planning ability of the large model to decompose complex problems into a series of sub-problems and solve them accordingly. It is mainly divided into two stages: 1) decomposing a complex problem into a series of simpler problems; 2) solving these sub-problems in turn, and the solution of each sub-problem depends on the answer to the previously solved sub-problem. The specific process is as follows: Figure 2 shown.
[0005] 3. Selection-Inference: This algorithm consists of the Selection module and the Inference module. The task of the Selection module is to select a subset of relevant information s from the context. t , this subset is sufficient for a single-step reasoning; the task of the Inference module is to use the information provided by the selection module to reason and form an intermediate conclusion i t This is an iterative process, and each step SI generates an intermediate conclusion i t will be added to the context until the answer to question q is found. The specific process is as follows Figure 3 shown.
[0006] 4. LAMBADA: The basic idea of this algorithm is to start from the goal and recursively decompose the goal into sub-goals until the sub-goals can be proved or disproven based on facts. The specific process is as follows: Figure 4 shown.
[0007] When faced with long-distance reasoning problems, the current optimization algorithm framework based on large models still has limitations, as follows:
[0008] 1. When the number of reasoning steps increases, the "hallucination" of the large model itself becomes more serious, and it is easy to independently write false rules, relationships and other information for reasoning, which seriously affects the reliability of the reasoning results;
[0009] 2. In the existing optimization algorithm framework, Chain of Thought (CoT), Least-to-Most Prompting, and Selection-Inference belong to forward chain reasoning, which starts from facts and rules, iterates between making new inferences and adding them to the theory until the target statement can be proved or disproven. This type of technology has the following problems:
[0010] 1) There is no clear causal relationship in the reasoning process, or there is a situation where the final result is correct but the reasoning path is wrong;
[0011] 2) There is a combinatorial explosion problem in the search space, and the failure rate is high when the reasoning is long;
[0012] 3) In forward chaining, it is very difficult to decide when to stop searching and declare failure;
[0013] 3. LAMBADA is a backward chain reasoning method. Starting from the goal, it does not need to select a subset from a large number of combined searches. It can stop searching more naturally, effectively solving the problem that there is no stopping standard for forward chain reasoning. However, when reasoning with this type of method, all facts and rules are included in the given prompt word, and the rule set is small enough. When the problem involves many facts and rules, due to the limitation of the length of the prompt word, all facts and rules will not be included in the prompt word, which will affect the accuracy of the reasoning result. Summary of the invention
[0014] In view of the limitations of existing artificial intelligence large model technology in dealing with long-distance reasoning problems, the purpose of the present invention is to provide a long-distance reasoning solution based on large models and symbolic logic, thereby further improving the reliability and explainability of large-model long-distance reasoning.
[0015] In order to achieve the above object, the present invention provides a long-distance reasoning method based on a large model and symbolic logic, comprising:
[0016] Build a knowledge base and fact database at the data level, the knowledge base contains stable conceptual knowledge, rules, etc.;
[0017] The input question is used as the trigger for initiating reasoning. The large model will understand the semantics of the input question and automatically plan the reasoning steps to form several reasoning sub-goals.
[0018] For each sub-goal, the most relevant rules or knowledge (Rules) are retrieved from the knowledge base through retrieval technology; at the same time, relevant factual data (Facts) are extracted by interacting with the database;
[0019] The large model performs short-distance deductive reasoning based on the reasoning goal (SubGoal), factual data (Facts) and rules (Rules) of the subtask, and returns the reasoning result (Inference).
[0020] The returned inference result (Inference) will be added to the next step as fact data for the next inference after inspection, and the inference of the next sub-goal will begin.
[0021] In some embodiments of the present invention, the large model in the long-distance reasoning method will perform semantic understanding of the input problem and automatically plan the reasoning steps. First, the N expert experiences with the highest relevance to the input problem are retrieved from the knowledge base. The N expert experiences will be input into the large model together with the problem as Few-shots. The large model will automatically decompose the reasoning steps to form several reasoning sub-goals (SubGoals).
[0022] In some embodiments of the present invention, when searching for the most relevant rules or knowledge (Rules) in the long-distance reasoning method, a retrieval enhancement generation (RAG) method is adopted, and the retrieval results are re-ranked in combination with Rerank to improve the relevance of the retrieval results.
[0023] In some embodiments of the present invention, when extracting relevant fact data (Facts) in the long-distance reasoning method, based on the NL2SQL technology, the large model first understands the fields that need to be extracted, such as entity, time, content, etc., and then based on the understanding of natural language, checks the database framework and content, identifies the relevant forms and columns required to generate SQL, and finally writes the corresponding SQL query statement to extract relevant fact data (Facts) from the database.
[0024] In some embodiments of the present invention, in the long-distance reasoning method, the corresponding reasoning results obtained by performing short-distance deductive reasoning based on the reasoning goal (SubGoal), factual data (Facts) and rules (Rules) of the subtask have a causal relationship and are highly reliable and interpretable.
[0025] In order to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium on which a program is stored, and when the program is executed by a processor, the steps of the above-mentioned long-distance reasoning method based on large models and symbolic logic are implemented.
[0026] In order to achieve the above-mentioned purpose, the present invention also provides a processor, which is used to run a program, and when the program is running, the steps of the above-mentioned long-distance reasoning method based on large models and symbolic logic are executed.
[0027] In order to achieve the above-mentioned purpose, the present invention also provides a terminal device, which includes a processor, a memory, and a program stored in the memory and run on the processor, and the program code is loaded and executed by the processor to implement the steps of the above-mentioned long-distance reasoning method based on large models and symbolic logic.
[0028] In order to achieve the above objectives, the present invention also provides a computer program product, which, when executed on a data processing device, is suitable for executing the steps of the above-mentioned long-distance reasoning method based on large models and symbolic logic.
[0029] The long-distance reasoning solution based on big models and symbolic logic provided by the present invention can allow the system to automatically obtain the context most relevant to the problem (including rules, factual data, etc.) when facing long-distance reasoning problems. The big model answers according to the given context, thereby alleviating the "illusion" of the big model to a certain extent.
[0030] The long-distance reasoning solution based on large models and symbolic logic provided by the present invention rationally plans the long-distance reasoning steps, solves the problems of combinatorial explosion of search space and no stopping search criteria, and further improves the interpretability of large model reasoning. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.
[0032] Figure 1 This is an example diagram of the thinking chain process in the prior art;
[0033] Figure 2 This is an example diagram of a prompt process from few to many in the prior art;
[0034] Figure 3 An example diagram of a selection-reasoning algorithm framework in the prior art;
[0035] Figure 4 This is an example diagram of the reverse reasoning process in the prior art;
[0036] Figure 5 This is an example diagram of the principle framework of the long-distance reasoning method in the present invention;
[0037] Figure 6 This is a flowchart of long-distance reasoning based on large models and symbolic logic in the present invention. DETAILED DESCRIPTION
[0038] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below with reference to specific diagrams.
[0039] In view of the shortcomings of existing large artificial intelligence models when facing long-distance reasoning problems, the present invention provides a long-distance reasoning method based on large models and symbolic logic, thereby improving the reliability and explainability of long-distance reasoning of large models.
[0040] The long-distance reasoning method based on large models and symbolic logic provided by the present invention takes advantage of symbolic language reasoning and controllable logic, and uses the semantic understanding ability, inductive reasoning ability and short-distance deductive reasoning ability of the large model as an aid to achieve complete complex event reasoning.
[0041] See also Figure 5 , which shows a schematic diagram of the scheme of the present invention for implementing long-distance reasoning based on large models and symbolic logic.
[0042] With reference to the diagram, the long-distance reasoning method based on a large model and symbolic logic provided by the present invention mainly includes three parts, namely, data, a basic large model and an algorithm framework, when implemented specifically.
[0043] Among them, at the data level, it specifically includes stable conceptual knowledge, irregularly updated conceptual knowledge and factual data.
[0044] The stable conceptual knowledge mentioned here includes rules and regulations in a specific field;
[0045] The conceptual knowledge described here includes rules, relationships, etc.
[0046] At the basic big model level, we select appropriate open source big models as the foundation of the reasoning algorithm framework through model evaluation.
[0047] At the algorithm framework level, a long-distance reasoning algorithm framework based on large models and symbolic logic is constructed.
[0048] Specifically, the long-distance reasoning algorithm framework contains seven modules, namely automatic planning, selection, reasoning, inspection, toolset, data and prediction of reasoning steps.
[0049] Further, the seven modules can be divided into three parts:
[0050] 1) The backbone consists of automatic planning and prediction of reasoning steps. The core of the backbone is the agent ReAct mechanism, which generates the final answer by observing the input and intermediate processes;
[0051] 2) The branch part consists of three modules: selection, reasoning, and checking. The core task of the branch part is to process multiple subtasks output by the trunk part and return the results of each subtask to the trunk part;
[0052] 3) In the toolset and data part, the toolset brings together the tools needed for reasoning, and the data mainly includes vector knowledge base and database, which store the rule knowledge and fact data required for reasoning.
[0053] On this basis, further combining Figure 6 As shown, the implementation process of the long-distance reasoning method based on large models and symbolic logic provided by the present invention is as follows:
[0054] First, build a knowledge base and fact database; the knowledge base built here contains stable conceptual knowledge, rules, etc.
[0055] Next, based on the determined basic big model (LLM), the input question is used as the trigger to initiate reasoning. The big model will semantically understand the input question and automatically plan the reasoning steps to form several reasoning sub-goals (SubGoals).
[0056] Then, for each sub-goal, the most relevant rules or knowledge (Rules) are retrieved from the knowledge base through retrieval technology; at the same time, it interacts with the database to extract relevant factual data (Facts).
[0057] Next, the large model performs short-distance deductive reasoning based on the reasoning goal (SubGoal), factual data (Facts) and rules (Rules) of the subtask, and returns the reasoning result (Inference).
[0058] Finally, the returned inference result (Inference) will be added to the next step as the fact data for the next inference after inspection, and the inference of the next sub-goal will begin. The process will be looped until all sub-goals are completed and the final inference result will be generated.
[0059] As a further explanation, the big model in the solution of the present invention performs semantic understanding on the input question and automatically plans the reasoning steps. First, the N expert experiences with the highest relevance to the input question are retrieved from the knowledge base. The N expert experiences are input into the big model together with the question as Few-shots. The big model automatically decomposes the reasoning steps to form several reasoning sub-goals (SubGoals).
[0060] As a further explanation, in the solution of the present invention, when searching for the most relevant rules or knowledge (Rules), the retrieval enhancement generation (RAG) technology is adopted, and the retrieval results are re-ranked in combination with Rerank to improve the relevance of the retrieval results.
[0061] As a further explanation, when extracting relevant fact data (Facts) in the solution of the present invention, based on the NL2SQL technology, the large model first understands the fields that need to be extracted, such as entity, time, content, etc., and then based on the understanding of natural language, checks the database framework and content, identifies the relevant forms and columns required to generate SQL, and finally writes the corresponding SQL query statement to extract relevant fact data (Facts) from the database.
[0062] As a further illustration, in the scheme of the present invention, the corresponding reasoning results obtained by performing short-distance deductive reasoning based on the reasoning goal (SubGoal), factual data (Facts) and rules (Rules) of the subtask have a causal relationship and are highly reliable and interpretable.
[0063] As a further explanation, the solution of the present invention is based on another large model different from the aforementioned large inference model to check the returned inference results (Inference). That is, the returned inference results (Inference) are sent to another large model for inspection, and the other large model will determine whether the inference results (Inference) can meet the inference sub-goal (SubGoal) and whether they can be correctly deduced from the facts (Facts) and rules (Rules).
[0064] At the same time, the solution of the present invention firstly reasonably plans the long-distance reasoning steps, thereby solving the problems of combinatorial explosion of the search space and the lack of a temporary stopping search criterion, and further improving the interpretability of large model reasoning.
[0065] In order to further illustrate the solution of the present invention, the implementation process of the solution of the present invention and the corresponding technical features are further illustrated below through specific application examples.
[0066] Here we take the automated reasoning of cases in the public security field as an example.
[0067] Case analysis in the public security field usually requires long-distance reasoning. The present invention can semantically understand the clues provided and reasonably plan the reasoning steps based on the clues. It can also flexibly call database, analysis, query and other tools to retrieve the facts and rules most relevant to the clues. The large model then reasons based on the provided goals, facts and rules, identifies suspects, determines criminal suspects and expands the line, thereby assisting front-line police officers in realizing automated reasoning of cases.
[0068] Specifically, we first construct a virtual knowledge base and fact database for the experiment.
[0069] Next, based on the constructed virtual knowledge base and fact database, as an example, the clue here is "The victim Cao** received a call, the other party claimed that his package was seized due to contraband and he needed to pay a deposit to get it back."
[0070] Input the clues into the big model. First, the big model can determine that the type of case involved is "fraud-related", and complete the suspect identification, determine the criminal suspect and expand the line. The specific program running results are as follows, and the running results have been checked and are correct.
[0071] As an example, here are the results:
[0072] Step 1: The task of identifying the suspects based on the clues has been completed. The details are as follows:
[0073] "According to the information you provided, the people who contacted the victim are: [Liang**; Zhou**; Dai**]."
[0074] Step 2: The task of identifying the suspect [highly suspected] from the scope of suspects has been completed. The specific situation is as follows:
[0075] "Based on the information you provided, the suspects identified are: [Liang**]."
[0076] ["Liang**'s role is a front-line operator."]
[0077] Step 3: "Based on the known rules and conditions and the results of the task, the following conclusions can be drawn: \n\n1. The superior of the criminal suspect Liang** may be Nong**. \n2. Nong** may be related to the fraud case."
[0078] "According to the information you provided, the suspect's superior: [Nong**]
[0079] ["Nong**'s role is to be the call center manager."]
[0080] >Finished chain.
[0081] Since it is known that Chi** may be Nong**'s superior, Nong**'s superiors include Chi**. "
[0082] Based on the above-mentioned long-distance reasoning scheme based on large models and symbolic logic, an embodiment of the present invention also provides a computer-readable storage medium on which a program is stored, and when the program is executed by a processor, the steps of the above-mentioned long-distance reasoning method based on large models and symbolic logic are implemented.
[0083] An embodiment of the present invention further provides a processor, which is used to run a program, wherein the program executes the steps of the above-mentioned long-distance reasoning method based on large models and symbolic logic when running.
[0084] An embodiment of the present invention also provides a terminal device, which includes a processor, a memory, and a program stored in the memory and executable on the processor, wherein the program code is loaded and executed by the processor to implement the steps of the above-mentioned long-distance reasoning method based on large models and symbolic logic.
[0085] The present invention also provides a computer program product, which, when executed on a data processing device, is suitable for executing the steps of the above-mentioned long-distance reasoning method based on large models and symbolic logic.
[0086] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0087] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0088] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0089] The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products of the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0090] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0091] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0092] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0093] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0094] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.
[0095] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0096] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
[0097] The above-mentioned method of the present invention, or a specific system unit, or a part of the same, is a pure software architecture, which can be arranged on a physical medium, such as a hard disk, an optical disk, or any electronic device (such as a smart phone, a computer-readable storage medium) through program code. When the machine loads the program code and executes it (such as a smart phone loading and executing it), the machine becomes a device for implementing the present invention. The above-mentioned method and device of the present invention can also be transmitted in the form of program code through some transmission media, such as cables, optical fibers, or any transmission mode. When the program code is received, loaded and executed by a machine (such as a smart phone), the machine becomes a device for implementing the present invention.
[0098] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.
Claims
1. A long-distance reasoning method based on large models and symbolic logic, characterized in that: include: Build a knowledge base and fact database at the data level, the knowledge base contains stable conceptual knowledge, rules, etc.; The input question is used as the trigger for initiating reasoning. The large model will understand the semantics of the input question and automatically plan the reasoning steps to form several reasoning sub-goals. For each sub-goal, the most relevant rules or knowledge are retrieved from the knowledge base through retrieval technology; at the same time, relevant factual data are extracted by interacting with the database; The large model performs short-distance deductive reasoning based on the reasoning goals, factual data, and rules of the subtasks and returns the reasoning results; After checking, the returned reasoning results will be added to the next step as fact data for the next reasoning, and the reasoning of the next sub-goal will begin.
2. The long-distance reasoning method based on large models and symbolic logic according to claim 1, characterized in that: In the long-distance reasoning method, when the large model will semantically understand the input problem and automatically plan the reasoning steps, first, the N expert experiences with the highest relevance to the input problem are retrieved from the knowledge base. The N expert experiences will be input into the large model together with the problem as Few-shots, and the large model will automatically decompose the reasoning steps to form several reasoning sub-goals.
3. The long-distance reasoning method based on large models and symbolic logic according to claim 1, characterized in that: In the long-distance reasoning method, when searching for the most relevant rules or knowledge, a search enhancement generation method is adopted, and the search results are re-ranked in combination with Rerank.
4. The long-distance reasoning method based on large models and symbolic logic according to claim 1, characterized in that: When extracting relevant factual data in the long-distance reasoning method, the large model first understands the fields that need to be extracted, and then based on the understanding of natural language, checks the database framework and content, identifies the relevant forms and columns required to generate SQL, and finally writes the corresponding SQL query statement to extract relevant factual data from the database.
5. The long-distance reasoning method based on large models and symbolic logic according to claim 1, characterized in that: In the long-distance reasoning method, the corresponding reasoning results obtained by performing short-distance deductive reasoning based on the reasoning goals, factual data and rules of the subtasks have a causal relationship.
6. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the steps of the long-distance reasoning method based on large models and symbolic logic described in any one of claims 1 to 5 are implemented.
7. A processor for running a program, characterized in that: When the program is running, the steps of the long-distance reasoning method based on large models and symbolic logic described in any one of claims 1 to 5 are executed.
8. A terminal device, comprising a processor, a memory, and a program stored in the memory and executable on the processor, characterized in that: The program code is loaded and executed by the processor to implement the steps of the long-distance reasoning method based on large models and symbolic logic as described in any one of claims 1-5.
9. A computer program product, characterized in that When executed on a data processing device, it is suitable for executing the steps of the long-distance reasoning method based on large models and symbolic logic described in any one of claims 1 to 5.