Hybrid context neural-symbolic collaborative reasoning method driven by large language model
Patent Information
- Application Number
- CN202410280051.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-12
- Publication Date
- 2026-10-23
- Estimated Expiration
- 2044-03-12
AI Technical Summary
[0004]然而，由于长文本-多表格混合上下文的复杂性，现有的技术方法存在着知识融合不足(非结构化文本和结构化表格难以有效地融合)，复杂推理困难(针对故障分析和决策推断，大模型缺乏准确的推理规则和结构化知识)，数值计算能力有待提升(传感器数据中存在温度、压力、流量等参数计算，大语言模型处理复杂的数值计算表现不佳)等问题
[0017]本方法通过利用大语言模型的语义理解和文本生成能力，联合知识图谱符号推理策略，在交通基础设施运维等领域智能决策复杂场景下，实现神经-符号协同的复杂推理以及数值计算。
Smart Images
Figure CN118170913B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of intelligent computing technology, and in particular relates to a hybrid contextual neural-symbolic collaborative reasoning method driven by a large language model. Background Technology
[0002] As a crucial data source in the field of transportation infrastructure operation and maintenance, equipment status records, maintenance records, and sensor data contain a wealth of information and knowledge. Maintenance personnel can use this data to gain a granular understanding of equipment health, causes of failures, and maintenance needs. However, this data primarily exists in a mixed context of long texts and multiple tables. The complexity and diversity of this data make it difficult for maintenance personnel to effectively and directly analyze and utilize key information.
[0003] In recent years, with the development of deep learning and big data processing technologies, natural language processing algorithms and models have been gradually applied to various fields. Thanks to the significant improvement in computing resources, large language models, with their powerful semantic understanding and text generation capabilities, have been widely used in vertical industries such as search engines, recommendation systems, and intelligent customer service.
[0004] However, due to the complexity of long text-multi-table mixed contexts, existing technical methods suffer from problems such as insufficient knowledge fusion (unstructured text and structured tables are difficult to integrate effectively), difficulties in complex reasoning (large models lack accurate reasoning rules and structured knowledge for fault analysis and decision inference), and insufficient numerical computation capabilities (sensor data involves parameter calculations such as temperature, pressure, and flow, and large language models perform poorly in handling complex numerical calculations). Existing technical solutions struggle to perform effective reasoning and numerical computation in complex intelligent decision-making scenarios such as transportation infrastructure operation and maintenance.
[0005] Therefore, how to achieve effective complex reasoning and numerical calculation in complex scenarios of intelligent decision-making in fields such as transportation infrastructure operation and maintenance has become an urgent problem to be solved. Summary of the Invention
[0006] To address the shortcomings of the existing technologies, this invention provides a hybrid contextual neural-symbolic collaborative reasoning method driven by a large language model, which can achieve effective complex reasoning and numerical computation in complex intelligent decision-making scenarios such as transportation infrastructure operation and maintenance.
[0007] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:
[0008] A hybrid contextual neural-symbolic co-reasoning method driven by a large language model includes the following steps:
[0009] S1. Acquire and process the corpus of transportation infrastructure operation and maintenance, and construct a domain dataset and knowledge graph G related to transportation infrastructure operation and maintenance; then encode the constructed domain dataset and knowledge graph to obtain the dataset encoding and knowledge graph embedding encoding V. G ;
[0010] S2. Construct a collaborative reasoning model; the collaborative reasoning model includes an answer generation module M. gen Problem Classification Module M class It includes multiple classification problem processing modules; the problem classification module is used to classify simple problems; the classification problem module is used to combine the dataset encoding obtained from S1, solve the simple problems it is responsible for, and obtain the corresponding answers;
[0011] Answer generation module M gen Based on a large language model, it is used to summarize the answers to simple questions by learning from prompts, and gradually generate the final answer to complex questions.
[0012] S3. Using the thinking chain technique of the large language model, the complex problem text Q to be decided is decomposed into a sequential combination of multiple simple problems as a set of simple problems;
[0013] S4. Using the Problem Classification Module M class The simple problems in the set of simple problems obtained in S3 are classified; then, the simple problems in each classification are solved using the respective classification problem processing modules; and the obtained answers are embedded into the knowledge graph obtained in S1 and encoded as V. G The inputs are fed into the knowledge graph reasoning engine RDFox for deductive reasoning. The engine determines whether the output of each simple question conforms to logical rules. If any incorrect answers do not conform to the rules, the corresponding classification question processing module solves the simple question again and checks its output for logical rules again, until the correct answer is generated. This process yields the correct set of simple answers, A. set ;
[0014] S5. Set A to the simplest answers set The corresponding set of simple questions and the dataset encoding are input into the answer generation module M. gen In the text, the final answer A is generated step by step using a learning prompt.
[0015] The order of S1 and S2 can be interchanged.
[0016] Compared with the prior art, the present invention has the following advantages:
[0017] This method leverages the semantic understanding and text generation capabilities of large language models, combined with knowledge graph symbolic reasoning strategies, to achieve complex reasoning and numerical computation through neural-symbolic collaboration in complex intelligent decision-making scenarios such as transportation infrastructure operation and maintenance.
[0018] When dealing with complex problems, this method breaks them down into a sequential combination of multiple simple problems using the mind chain technique; and uses the problem classification module M. class These simple problems are categorized, and then each category's problem-solving module is used to solve the corresponding simple problems. This way, when there is a need to solve a particular type of problem, the corresponding category is set up and the corresponding problem-solving module is built. It can solve various types of simple problems (such as numerical calculation problems) and is highly flexible.
[0019] Furthermore, if the simple answers are directly combined into an answer set and input into a large language model, the simple prompting learning method is highly likely to fail to obtain the correct final answer. In this method, when solving simple questions of the corresponding categories using the respective classification question processing modules, simple answers are validated. Through validation using the knowledge graph reasoning engine RDFox, neural-symbolic collaborative reasoning is achieved, which can identify incorrect answers and generate secondary or multiple answers to maximize the accuracy of the final answer.
[0020] In summary, this method enables effective complex reasoning and numerical computation in complex intelligent decision-making scenarios such as transportation infrastructure operation and maintenance.
[0021] Preferably, in S1, when processing the transportation infrastructure operation and maintenance corpus, the processing procedure includes: using spaCy, Camelot, and DeepKE tools to extract paragraphs P, tables T, and triples from the transportation infrastructure operation and maintenance corpus; and using random generation functions and fuzzing functions to desensitize the identity information of operation and maintenance personnel, equipment identification information, and precise geographical location information.
[0022] In this way, we can obtain the necessary data while ensuring the privacy of important data.
[0023] Preferably, in S1, the domain dataset includes multiple hybrid contexts; each context includes at least one table T and two paragraphs P describing the table T.
[0024] In this way, while ensuring the legitimacy of the domain dataset, rich semantic relationships can also be guaranteed.
[0025] Preferably, in S1, when encoding the domain dataset, for each paragraph P in the Context, firstly, spaCy is used to segment paragraph P into multiple sentences S, obtaining the text sequence of the paragraph, P = (S1, S2, ..., S...). t ), where t is the number of sentences; then, a large language model is used to encode the text sequence of the paragraph, converting the symbolic text into vectorized numerical data, to obtain the text sequence encoding V of the paragraph. P V P =F(S1,S2,…,S) t ).
[0026] Glossary:
[0027] spaCy is a Python and CPython NLP (Natural Language Processing) library.
[0028] Preferably, in S1, when encoding the domain dataset, for each table T in the Context, the table pre-trained model SDCUP is used as the encoder, and the table T is encoded using a multi-granularity encoding method of Table-Row-Cell; the tables in the Context are divided into three granular representations, and the overall information of table T is obtained through Table-Level encoding; the row features of table T are obtained through Row-Level encoding; and the fine-grained information of the cell data of table T is obtained through Cell-Level encoding.
[0029] Preferably, in S1, the process of encoding the knowledge graph G includes: using the knowledge graph reasoning engine RDFox to deeply mine the rules in the knowledge graph G, thereby constructing a rule set R. G ; and the Concept2Box knowledge graph embedding model is used to map discrete symbolic knowledge to dense continuous vectorized knowledge, resulting in the knowledge graph embedding encoding V. G V G =Concept2Box(G,R G The rules include equivalence, subclassing, inverse, reflexivity, symmetry, and / or transitivity.
[0030] Compared with traditional quantitative data analysis, this method combines contextual interpretation, subjective judgment, and objective evaluation to analyze the contextual characteristics of heterogeneous text-table information from multiple perspectives, which can ensure the effectiveness of the coding.
[0031] Preferably, the problem classification module M class Based on heuristic strategies, it is used to classify simple problems into four types: retrieval, numerical reasoning, logical judgment, and induction.
[0032] Preferably, the classification problem processing module includes a fragment extraction module M. ext Numerical calculation module M cal Logic Judgment Module M jud And the summarization module M rea Fragment Extraction Module M ext Used to solve simple retrieval-type problems; Numerical computation module M cal Used for solving simple numerical reasoning problems; Logic judgment module M jud Used for solving simple logical reasoning problems; Inductive summary module M rea Used to solve simple inductive problems.
[0033] This setup allows for the solution of four representative types of simple problems. Taking numerical computation problems as an example, in this method, the numerical computation module M... cal It will generate the corresponding numerical calculation expression, which can then be solved using an external calculator or program code interpreter to obtain the correct numerical calculation result.
[0034] Preferably, in S5, the accuracy and loyalty of the answers generated in the intermediate steps are also ensured through a self-suggestion method.
[0035] Definition: Self-suggestion refers to the process in machine learning and natural language processing tasks where the model automatically provides suggestions and hints through autonomous learning and reasoning.
[0036] Preferably, in S3, the sequential combination of the simple problems obtained is: [q1,q2,…,q x ] = F CoT (Q); where x represents the number of simple problems. Attached Figure Description
[0037] To make the objectives, technical solutions, and advantages of the invention clearer, the invention will now be described in further detail with reference to the accompanying drawings, wherein:
[0038] Figure 1 This is a flowchart of the present invention;
[0039] Figure 2 This is a schematic diagram of the model's operation in the embodiment. Detailed Implementation
[0040] The following detailed explanation illustrates the specific implementation methods:
[0041] Example:
[0042] like Figure 1 , Figure 2 As shown, this embodiment discloses a hybrid contextual neural-symbolic collaborative reasoning method driven by a large language model, including the following steps:
[0043] S1. Acquire and process the corpus of transportation infrastructure operation and maintenance, and construct a domain dataset and knowledge graph G related to transportation infrastructure operation and maintenance; then encode the constructed domain dataset and knowledge graph to obtain the dataset encoding and knowledge graph embedding encoding V. G .
[0044] In practice, the corpus of transportation infrastructure operation and maintenance data is obtained, including reports, structured tables, and semi-structured knowledge graphs of existing roads, bridges, and other transportation infrastructure, covering daily inspections, periodic testing, and special inspections. The processing of this corpus includes: using spaCy, Camelot, and DeepKE tools to extract paragraphs (P), tables (T), and triples; and employing random generation and fuzzification functions to anonymize personnel identification information, equipment identification information, and precise geographical location information.
[0045] Next, paragraph P and table T are combined into multiple hybrid contexts. To ensure rich semantic relationships, a valid context must consist of at least one table T and two or more paragraphs P describing that table, thus constructing a domain dataset. For the triples (h,r,t), type constraints and relational constraints are applied for limitation, and the data is imported into the Neo4j graph database for storage, thereby initially constructing the domain's knowledge graph G.
[0046] When encoding the domain dataset, for each paragraph P in the Context, firstly, spaCy (a Python and CPython NLP natural language text processing library) is used to segment paragraph P into multiple sentences S, resulting in a text sequence of paragraphs, P = (S1, S2, ..., S...). t ), where t is the number of sentences; then, a large language model is used to encode the text sequence of the paragraph, converting the symbolic text into vectorized numerical data, to obtain the text sequence encoding V of the paragraph. P That is: V P =F(S1,S2,…,S) t In practical implementation, those skilled in the art can directly select the large language model published by a domestic company.
[0047] For each table T in the Context, the pre-trained table model SDCUP is used as the encoder, employing a multi-granularity Table-Row-Cell encoding approach to encode table T. Compared to traditional linear table encoding or graph neural network encoding strategies, this method uses a multi-granularity Table-Row-Cell encoding approach to divide one or more tables in a Context into three granular representations. Specifically:
[0048] The tables in the Context are divided into three granular representations, and the overall information of table T is obtained through Table-Level encoding; thus, T is obtained. t =(T′1,T′2,…,T′) l ), where l is the number of tables;
[0049] Row-Level encoding is used to obtain the row features of table T; thus, T is obtained. r =(R1,R2,…,R) n ), where n is the total number of rows in the table;
[0050] Fine-grained information about the cell data in table T is obtained through Cell-Level encoding; thus, T is obtained. c =(C1,C2,…,C m ), where m is the total number of cells in the table.
[0051] After the data format is constructed, the table pre-trained model SDCUP is used to encode the three granular representations to obtain the corresponding encoding results V. TT V TR V TC :
[0052] V TT′ =Encoder SDCUP (T t );
[0053] V TR =Encoder SDCUP (T r );
[0054] V TC =Encoder SDCUP (T c ).
[0055] For a knowledge graph G, the knowledge graph inference engine RDFox is used to deeply mine the rules in knowledge graph G, thereby constructing a rule set R. G ; and the Concept2Box knowledge graph embedding model is used to map discrete symbolic knowledge to dense continuous vectorized knowledge, resulting in the knowledge graph embedding encoding V.G V G =Concept2Box(G,R G The rules include equivalence, subclassing, inverse, reflexivity, symmetry, and / or transitivity.
[0056] It should be noted that, in practical implementation, those skilled in the art can choose a table-based pre-trained model based on their familiarity with the modeling tools, replacing SDCUP with other table-based pre-trained models such as TAPAS, TabBERT, and TabNet. Similarly, the knowledge graph embedding model Concept2Box can be replaced with TransE, ComplEx, RotatE, etc. After the above model replacements, the corresponding performance will decrease slightly, but overall, the impact is not significant.
[0057] S2. Construct a collaborative reasoning model; the collaborative reasoning model includes an answer generation module M. gen Problem Classification Module M class And multiple classification problem processing modules.
[0058] The problem classification module is used to categorize simple problems; in specific implementation, the problem classification module M... class Based on heuristic strategies, it is used to classify simple problems into four types: retrieval, numerical reasoning, logical judgment, and induction.
[0059] The classification problem module combines the dataset encoding obtained from S1 to solve a simple problem of its assigned class, yielding the corresponding answer. In specific implementation, the classification problem processing module includes a fragment extraction module M. ext Numerical calculation module M cal Logic Judgment Module M jud And the summarization module M rea Fragment Extraction Module M ext Used to solve simple retrieval-type problems; Numerical computation module M cal Used for solving simple numerical reasoning problems; Logic judgment module M jud Used for solving simple logical reasoning problems; Inductive summary module M rea Used to solve simple inductive problems.
[0060] Answer generation module M gen Based on a large language model, it is used to summarize the answers to simple questions by learning from prompts, and gradually generate the final answer to complex questions.
[0061] S3. Using the thought chain technique of large language model, decompose the complex problem text Q to be decided into a sequential combination of multiple simple problems: [q1,q2,…,q x ] = FCoT (Q) represents the set of simple problems. In the formula, x represents the number of simple problems. In practical implementation, a problem decomposition module can be built to automate the corresponding functions.
[0062] It should be noted that for question texts with specific formatting specifications, the mind chain technique in question classification in S3 can be replaced by a rule-based template method or a segmentation-based method. However, for question texts without fixed formats and specifications, the mind chain technique will be more flexible and more tolerant of errors.
[0063] S4. Using the Problem Classification Module M class The simple problems [q1,q2,…,q3] obtained from S3 are... x The problems are categorized into four types: retrieval, numerical reasoning, logical judgment, and induction. Then, guided by the problem classification results, the fragment extraction module M is used... ext Numerical calculation module M cal Logic Judgment Module M jud And the summarization module M rea Combined with the text sequence encoding V obtained in the second step p The corresponding table structure code V T Each person is responsible for solving a class of simple problems to obtain the answers to those simple problems.
[0064] To verify the correctness of the answer, the obtained answer is embedded into the knowledge graph obtained in S1 and encoded as V. G All inputs are fed into the knowledge graph reasoning engine RDFox for deductive reasoning, determining whether the output of each simple question conforms to logical rules, i.e.:
[0065]
[0066] If there are incorrect answers that do not conform to the trajectory specifications, the corresponding classification problem processing module will solve the corresponding simple problem again and judge whether its output conforms to the logical specifications again, until the correct answer is generated; thus obtaining the set of correct simple answers.
[0067] S5. Set A to the simplest answers set The corresponding set of simple questions and the dataset encoding are input into the answer generation module M. gen The algorithm generates the final answer A step-by-step using prompts. It also employs a self-suggestion mechanism to ensure the accuracy and loyalty of the answers generated in intermediate steps.
[0068] A = F prompt (A set,[q1,q2,…,q x ],P,T);
[0069] accuracy|loyalty=F self-suggestion (A set ,[q1,q2,…,q x ],P,T).
[0070] Self-suggestion refers to the process in machine learning and natural language processing tasks where models automatically provide suggestions and hints through autonomous learning and reasoning.
[0071] The order of S1 and S2 can be interchanged.
[0072] To help those skilled in the art better understand the meaning of this method, a complex problem-answer example based on a dataset is provided for illustration, as shown in Table 1.
[0073] Table 1
[0074]
[0075] This method leverages the semantic understanding and text generation capabilities of large language models, combined with knowledge graph symbolic reasoning strategies, to achieve complex reasoning and numerical computation through neural-symbolic collaboration in complex intelligent decision-making scenarios such as transportation infrastructure operation and maintenance. When dealing with complex problems, this method decomposes them into a sequential combination of multiple simpler problems using a thought chain technique; and utilizes a problem classification module M... class These simple problems are categorized, and then each category's problem processing module is used to solve the corresponding simple problems. This way, when there is a need to solve a particular type of problem, the corresponding category is set up and the corresponding problem processing module is built. It can solve various types of simple problems (such as numerical calculation problems) and is very flexible. However, if the individual simple answers are directly input into a large language model, the simple hint-based learning method is very likely to fail to obtain the correct final answer. In this method, when using each category's problem processing module to solve the corresponding category of simple problems, the simple answers are verified. Through the verification of the knowledge graph reasoning engine RDFox, neural-symbolic collaborative reasoning is achieved, which can identify incorrect answers and generate secondary or multiple answers to ensure the accuracy of the final answer to the greatest extent.
[0076] Using this method, effective complex reasoning and numerical computation can be achieved in complex scenarios of intelligent decision-making in fields such as transportation infrastructure operation and maintenance.
[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit the technical solutions. Those skilled in the art should understand that any modifications or equivalent substitutions to the technical solutions of the present invention without departing from the spirit and scope of the present invention should be covered within the scope of the claims of the present invention.
Claims
1. A hybrid contextual neural-symbolic collaborative reasoning method driven by a large language model, characterized in that, Includes the following steps: S1. Acquire and process the corpus of transportation infrastructure operation and maintenance, and construct a domain dataset and knowledge graph G related to transportation infrastructure operation and maintenance; then encode the constructed domain dataset and knowledge graph to obtain the dataset encoding and knowledge graph embedding encoding V. G ; S2. Construct a collaborative reasoning model; the collaborative reasoning model includes an answer generation module M. gen Problem Classification Module M class And multiple classification problem processing modules; the problem classification module is used to classify simple problems; the classification problem module is used to combine the dataset encoding obtained from S1 to solve the simple problems of the class it is responsible for, and obtain the corresponding answers; the answer generation module M gen Based on a large language model, it is used to summarize the answers to simple questions by learning from prompts, and gradually generate the final answer to complex questions. S3. Using the thinking chain technique of the large language model, the complex problem text Q to be decided is decomposed into a sequential combination of multiple simple problems, which is a set of simple problems. S4. Using the Problem Classification Module M class The simple problems in the set of simple problems obtained in S3 are classified; then, the simple problems in each classification are solved using the respective classification problem processing modules; and the obtained answers are embedded into the knowledge graph obtained in S1 and encoded as V. G The inputs are fed into the knowledge graph reasoning engine RDFox for deductive reasoning. The engine determines whether the output of each simple question conforms to logical rules. If any incorrect answers do not conform to the rules, the corresponding classification question processing module solves the simple question again and checks its output for logical rules again, until the correct answer is generated. This process yields the correct set of simple answers, A. set ; S5. Set A of simple answers set The corresponding set of simple questions and the dataset encoding are input into the answer generation module M. gen In the text, the final answer A is generated step by step using a learning prompt. The order of S1 and S2 can be interchanged.
2. The hybrid contextual neural-symbolic collaborative reasoning method driven by a large language model as described in claim 1, characterized in that: In S1, when processing the transportation infrastructure operation and maintenance corpus, the process includes: using spaCy, Camelot, and DeepKE tools to extract paragraphs P, tables T, and triples from the transportation infrastructure operation and maintenance corpus; and using random generation and fuzzing functions to desensitize the identity information of operation and maintenance personnel, equipment identification information, and precise geographical location information.
3. The hybrid contextual neural-symbolic collaborative reasoning method driven by a large language model as described in claim 2, characterized in that: In S1, the domain dataset includes multiple hybrid contexts; each context includes at least one table T and two paragraphs P describing the table T.
4. The hybrid contextual neural-symbolic collaborative reasoning method driven by a large language model as described in claim 3, characterized in that: In S1, when encoding the domain dataset, for each paragraph P in the Context, firstly, spaCy is used to segment paragraph P into multiple sentences S, resulting in the text sequence of the paragraph, P = (S1, S2, ..., S...). t ), where t is the number of sentences; then, a large language model is used to encode the text sequence of the paragraph, converting the symbolic text into vectorized numerical data, to obtain the text sequence encoding V of the paragraph. P V P =F(S1,S2,…,S) t ).
5. The hybrid contextual neural-symbolic collaborative reasoning method driven by a large language model as described in claim 4, characterized in that: In S1, when encoding the domain dataset, for each table T in the Context, the table pre-trained model SDCUP is used as the encoder, and the table T is encoded using a multi-granularity encoding method of Table-Row-Cell, which divides the tables in the Context into three granular representations: the overall information of table T is obtained through Table-Level encoding; the inter-row features of table T are obtained through Row-Level encoding; and the fine-grained information of the cell data of table T is obtained through Cell-Level encoding.
6. The hybrid contextual neural-symbolic collaborative reasoning method driven by a large language model as described in claim 2, characterized in that: In S1, the process of encoding the knowledge graph G includes: using the knowledge graph reasoning engine RDFox to deeply mine the rules in the knowledge graph G, thereby constructing a rule set R. G ; and the Concept2Box knowledge graph embedding model is used to map discrete symbolic knowledge to dense continuous vectorized knowledge, resulting in the knowledge graph embedding encoding V. G V G =Concept2Box(G,R G The rules include equivalence, subclassing, inverse, reflexivity, symmetry, and / or transitivity.
7. The hybrid contextual neural-symbolic collaborative reasoning method driven by a large language model as described in claim 1, characterized in that: Problem Classification Module M class Based on heuristic strategies, it is used to classify simple problems into four types: retrieval, numerical reasoning, logical judgment, and induction.
8. The hybrid contextual neural-symbolic collaborative reasoning method driven by a large language model as described in claim 7, characterized in that: The classification problem processing module includes the fragment extraction module M. ext Numerical calculation module M cal Logic Judgment Module M jud And the summarization module M rea Fragment Extraction Module M ext Used to solve simple retrieval-type problems; Numerical computation module M cal Used for solving simple numerical reasoning problems; Logic judgment module M jud Used for solving simple logical reasoning problems; Inductive summary module M rea Used to solve simple inductive problems.
9. The hybrid contextual neural-symbolic collaborative reasoning method driven by a large language model as described in claim 1, characterized in that: In S5, the accuracy and loyalty of the answers generated in intermediate steps are also ensured through self-suggestion.
10. The hybrid contextual neural-symbolic collaborative reasoning method driven by a large language model as described in claim 1, characterized in that: In S3, the sequential combination of the simple problems is: [q1,q2,…,q x ] = F CoT (Q); where x represents the number of simple problems.