A bridge maintenance question-answering method based on a large model and knowledge graph

Abstract

The present invention discloses a bridge maintenance question-and-answer method in which a large model and a knowledge graph are coordinated, including: constructing a prompt for complex bridge maintenance questions and answers; injecting complex questions and prompts into a large dialogue model of bridge maintenance knowledge to decompose the complex questions; understanding the first sub-question under the current number of iterations, generating a Cypher query statement, and retrieving the answer in the bridge maintenance knowledge graph; inputting the answer to the first sub-question, the second sub-question, and the prompt into the large dialogue model to decompose the second sub-question again, and executing the next iteration; if the decomposition fails again, rewriting the second sub-question; retrieving the corresponding answer according to the Cypher query statement of the second sub-question; and combining the answers to all sub-questions to generate the final answer to the complex question. The present invention realizes the question-and-answer of complex questions by utilizing the coordination and integration of the large dialogue model and the bridge maintenance knowledge graph.

Description

Technical Field

[0001] The present invention relates to the field of Internet and new generation information technology, and in particular to a bridge maintenance question-answering method in which a large model and a knowledge graph are coordinated. Background Art

[0002] Bridge maintenance is an important stage that lasts the longest in the entire life cycle of a bridge. The industry has formed a maintenance business system that focuses on structural health monitoring, damage identification and prediction, defect detection and evaluation, maintenance decision-making, and repair and maintenance. However, the large-scale multi-source heterogeneous data generated during the maintenance process are distributed and stored independently, and the data island problem seriously hinders the correlation analysis of facility performance in similar structures and the same environment. Therefore, bridge maintenance data integration and information fusion have received widespread attention in recent years.

[0003] With the development of new generation information technology, big data and artificial intelligence have entered the public's field of vision. Inspired by the application of natural language processing and big data knowledge engineering in vertical fields, related work such as named entity recognition, relationship extraction, ontology construction, knowledge graph question answering, etc. for bridge management and maintenance has been further studied and mainly applied to specific scenarios such as construction specification review and maintenance project management. However, due to excessive reliance on expert experience, knowledge barriers to industry rules, and complex spatiotemporal relationships, the current level of digitalization and knowledge services for bridge management and maintenance is relatively low, and there is still a huge gap in intelligent management and maintenance decision-making.

[0004] At present, knowledge graph question answering methods for bridge maintenance decision-making mainly include methods based on semantic parsing, information retrieval, subgraph matching or deep learning, and the model is equipped with domain knowledge by constructing domain rules and templates. As the capabilities of pre-trained language models become more prominent, knowledge graph question answering methods based on pre-trained language models such as BERT and GPT have been proposed and achieved better performance. However, the field of bridge maintenance involves a large number of professional terms and complex concepts. For complex problems or multi-step reasoning problems, existing models are still unable to fully understand the intention of complex problems, let alone update and adjust knowledge reasoning strategies in real time.

[0005] With the success of ChatGPT, the big dialogue model has quickly become a hot topic in academic and industrial research and has been widely used in all walks of life. The big dialogue model has significant performance in information extraction, text generation, machine translation, knowledge graph question answering, etc., showing strong context perception, deep semantic representation and few-sample learning capabilities, and also brought new opportunities and challenges to bridge maintenance decision assistance.

[0006] The applicant found in the research that the dialogue model can learn knowledge from massive data and generate corresponding answers as required given questions and instruction prompts. The knowledge graph can organically organize bridge maintenance data and provide richer semantic information and evidence support for understanding and answering questions. Therefore, how to design a bridge maintenance question-answering method that combines the dialogue model with the knowledge graph is a technical problem that needs to be solved urgently. Summary of the invention

[0007] In view of the above-mentioned deficiencies of the prior art, the technical problem to be solved by the present invention is: how to provide a bridge maintenance question and answer method that collaborates with a big model and a knowledge graph, and utilizes the collaboration and integration of the dialogue big model and the bridge maintenance knowledge graph to achieve question and answer of complex issues, which can conduct in-depth analysis of the bridge maintenance field, and then fully understand the complex issues of bridge maintenance, thereby improving the accuracy of bridge maintenance question and answer.

[0008] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0009] The bridge maintenance question-answering method based on the collaboration of big model and knowledge graph includes:

[0010] S1: Get complex questions to be answered;

[0011] S2: Construct prompts for complex questions and answers in bridge maintenance;

[0012] S3: The complex problem and prompt input are injected into the dialogue model of bridge maintenance knowledge. The complex problem is decomposed through the dialogue model and two sub-problems are output;

[0013] S4: Use the big dialogue model to understand the first sub-problem at the current iteration and generate a Cypher query statement.

[0014] S5: Search the bridge maintenance knowledge graph according to the Cypher query statement and output the answer to the first sub-question;

[0015] S6: Input the answer to the first sub-question, the second sub-question, and the prompt into the dialogue model, and decompose the second sub-question again through the dialogue model: if the decomposition is successful, output two sub-questions, and return to step S4 for the next iteration; if the decomposition fails, end the iteration, rewrite the second sub-question according to the answer to the first sub-question through the dialogue model, and execute step S7;

[0016] S7: Generate a Cypher query statement for the second sub-question through the dialogue model and retrieve the corresponding answer. Finally, combine the answers to all sub-questions to generate the final answer to the complex question.

[0017] Preferably, in step S2, the Prompt instruction prompt includes sub-problem decomposition, entity recognition, relationship recognition, hop count judgment, Cypher query generation and answer summary generation.

[0018] Preferably, in step S3 or step S6, when the dialogue model decomposes the complex problem or sub-problem, the complex problem or sub-problem is decomposed into two sub-problems;

[0019] The decomposition set of complex problems is expressed as:

[0020]

[0021] The first decomposition of the complex problem is the 0th iteration, then represents the first subproblem and the second subproblem obtained in the i-th iteration; n represents the number of iterations; represents the mth retrieval subproblem of the first subproblem obtained in the i-th iteration;

[0022] In each iteration, the first subproblem The answer is retrieved in the current iteration, and the second sub-question As the main issue of the next iteration, we need to Rewrite or break down the answer to the question;

[0023] Sub-questions According to the number of Cypher queries m, it is decomposed into m retrieval sub-problems.

[0024] Preferably, in step S3, the dialogue model implements problem decomposition through the following steps:

[0025] S301: Match the problem to be decomposed with the prompt decomposition prompts of different problems in the prompt instruction library to obtain the similarity between the problem to be decomposed and each prompt decomposition prompt;

[0026] S302: taking the Prompt decomposition hint with the highest similarity as the optimal Prompt decomposition hint of the problem to be decomposed;

[0027] S303: parsing the optimal Prompt decomposition prompt to extract key information, including the question to be answered, the required information type, and possible reasoning paths;

[0028] S304: Based on the analyzed key information, further understand the specific content and context of the problem to be decomposed;

[0029] S305: Decompose the problem to be decomposed into several sub-problems based on the specific content and context understood.

[0030] Preferably, in step S3, the dialogue model is based on ChatGLM3-6B, and a phased fine-tuning method is used to successively inject bridge maintenance knowledge graph related information and domain background knowledge into the dialogue model to form a graph dialogue model BKG-GPT for bridge maintenance questions and answers;

[0031] The processing steps of the staged fine-tuning method are as follows:

[0032] 1) The first stage: convert the bridge maintenance knowledge graph into a format that can be understood by the dialogue big model; then construct domain problem pair data based on the bridge maintenance knowledge graph to perform P-tuning v2 training on the dialogue big model, realize parameter tuning of the dialogue big model, and enable the dialogue big model to understand and apply the knowledge of the bridge maintenance knowledge graph;

[0033] 2) The second stage: For the dialogue model that has been injected with the bridge maintenance knowledge graph in the first stage, design appropriate training tasks, and use the P-tuning method to inject professional information such as maintenance suggestions and causes of diseases as domain knowledge background to fine-tune the dialogue model;

[0034] In the P-tuning method, the dialogue model input is:

[0035] {h0,h1,…,h i ,e(x),h i+1 ,…,h n ,e(y));

[0036]

[0037] Where: h is the continuous prompt word, h i is the Prompt embedding input into the large dialogue model after encoding using LSTM+MLP; e(x) represents the embedding vector of input x; e(y) represents the embedding vector of label t; h 0:i represents the hidden state set from the 0th element to the i-th element in the Prompt sequence, which contains the context information from the beginning of the sequence to the current i-th virtual token position; h i:n Represents the hidden state set from the i-th element to the n-th element, which contains the context information from the current virtual token position to the end of the sequence; represents the positive hidden state of the i-th element; Represents the reverse hidden state of the i-th element;

[0038] The optimization objectives during fine-tuning are as follows:

[0039]

[0040] Where: N is the number of samples in the training set, f llm (e(x j ),h j ) is a pre-trained dialogue model with a prompt vector for the input e(x j )’s predicted output, e(y j ) is the corresponding sample embedding e(x j ) is the true label embedding, and l is the loss function.

[0041] Preferably, in step S4, the dialogue model performs entity recognition, relationship recognition, and hop count judgment on the first sub-problem under the current iteration number, and generates a corresponding Cypher query statement based on the results of entity recognition, relationship recognition, and hop count judgment.

[0042] Preferably, in step S5, the answer is retrieved in the Neo4j knowledge graph of bridge maintenance according to the Cypher query statement of the sub-question, and the corresponding answer is output; wherein the Neo4j knowledge graph of bridge maintenance includes basic information of different bridge types and bridge instances, structural composition and disease-related information of specific important structural parts, and technical status of the bridge.

[0043] Preferably, in step S6, when the answer to the first sub-question, the next sub-question and the Prompt instruction are input into the dialogue model, the answer to the first sub-question is injected into the dialogue model as an external knowledge enhancement through the RAG retrieval enhancement generation mechanism, that is, the dialogue model is enhanced through the bridge maintenance knowledge graph.

[0044] Preferably, in step S6, in each iteration, before decomposing the second sub-problem again, first determine whether the number of iterations is greater than the maximum number of iterations: if so, end the iteration and execute step S7; otherwise, continue to calculate the confidence of the answer to the first sub-problem, if the confidence of the answer is greater than the confidence threshold, end the iteration and execute step S7, otherwise, decompose the second sub-problem again;

[0045] The formula for calculating the answer confidence is as follows:

[0046]

[0047] P(w i |w1,w2…,w i-1 ,q2,a1,p)=softmax(out_linear(h i-1 ));

[0048] Where: α represents the confidence of the answer; P(w i |w1,w2…,w i-1,q2,a1,p) is the given context, i.e. the second sub-question q2, the answer a1 to the first sub-question and the corresponding prompt instruction p and the previously generated words (w1,w2…,w i-1 ) to generate the i-th word w i The conditional probability of ; n is the number of words in the answer; h i-1 It is the previous hidden layer state of the Transformer model in the large dialogue model. After a linear layer and softmax function transformation, the probability distribution prediction of the next word is obtained.

[0049] Preferably, in step S6, during the iteration process, the dialogue model adds the answer to each sub-question to the evidence chain to continuously improve the evidence chain during the iteration process, and after the iteration, summarizes the answers on the evidence chain to generate the final answer to the complex question.

[0050] Compared with the prior art, the bridge maintenance question-answering method based on the collaboration of the large model and the knowledge graph in the present invention has the following beneficial effects:

[0051] The present invention uses a large dialogue model to decompose and understand complex problems and sub-problems to generate Cypher query statements, and then retrieves the answers to sub-problems in the bridge maintenance knowledge graph through Cypher query statements, that is, the collaboration and integration of the large dialogue model and the bridge maintenance knowledge graph is used to achieve the question and answer of complex problems. First, the large dialogue model can understand complex language structures and contextual information, while the knowledge graph provides rich domain knowledge. Therefore, through the collaboration of the large dialogue model and the knowledge graph, it is possible to achieve an in-depth understanding and analysis of the bridge maintenance field, and then fully understand the complex problems of bridge maintenance, thereby improving the accuracy of bridge maintenance questions and answers. Secondly, the combination of the large dialogue model and the knowledge graph can decompose and simplify the problem, adaptively adjust and generate dynamic instruction prompts, and through the iterative interaction of the large model and the knowledge graph, obtain the problem evidence chain, infer the cause of the disease and maintenance suggestions, thereby achieving the generation of inductive answers to complex bridge maintenance problems, and providing intelligent decision-making assistance for management and maintenance. Finally, when the dialogue model decomposes complex questions and sub-questions and generates Cypher query statements for sub-questions, the present invention provides a bridge for the dialogue model to manage complex questions and answers with a Prompt instruction prompt. The constructed Prompt instruction prompt can provide a framework or template for decomposing a question for the dialogue model, so that the large model can split the complex question into multiple sub-questions according to this framework, thereby reducing the difficulty of the large model in processing complex questions. At the same time, the Prompt instruction prompt can also constrain the style and logic of the Cypher query statement, thereby ensuring that the generated Cypher query statement meets the expected format and specifications, thereby improving the accuracy and efficiency of subsequent answer retrieval.

[0052] Based on the collaborative dialogue model and knowledge graph, the present invention uses the dialogue model and knowledge graph to iteratively decompose and retrieve answers to complex problems and sub-problems until the sub-problems cannot be decomposed or the iteration stop condition is reached. First, by iteratively decomposing complex problems through the dialogue model, a complex problem can be split into multiple simpler sub-problems, and then the answer is retrieved for each sub-problem, which reduces the difficulty and complexity of processing bridge maintenance questions and answers. Secondly, the present invention rewrites and further decomposes other sub-problems according to the answer to the current sub-problem during the iteration process, so that various complex problems, including those involving multiple aspects or requiring reasoning, can be handled more flexibly, and the interpretability of bridge maintenance complex problem questions and answers can be improved, thereby improving the effect of bridge maintenance questions and answers. Finally, the present invention retrieves the answer to each sub-problem independently, ensuring that each sub-problem is accurately answered, and at the same time gradually constructs the complete answer to the complex problem in an iterative manner, thereby ensuring the integrity of the final answer, and further ensuring the improvement of the interpretability of bridge maintenance complex problem questions and answers. This method of gradually approaching the final answer in an iterative manner can reduce unnecessary calculations and resource consumption, thereby improving the processing efficiency of bridge maintenance questions and answers. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to make the purpose, technical solution and advantages of the invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which:

[0054] Figure 1 This is a working principle diagram of the bridge maintenance question and answer method;

[0055] Figure 2 Schematic diagram for constructing the large dialogue model BKG-GPT;

[0056] Figure 3 A schematic diagram of the Neo4j knowledge graph for bridge maintenance;

[0057] Figure 4 The following is a workflow diagram for the bridge maintenance question and answer method. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but only represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0059] It should be noted that similar numbers and letters represent similar items in the following figures, so once an item is defined in one figure, it does not need to be further defined and explained in the subsequent figures. In the description of the present invention, it should be noted that the orientation or position relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inside", "outside", etc. is based on the orientation or position relationship shown in the figures, or the orientation or position relationship in which the invention product is usually placed when used, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance. In addition, the terms "horizontal", "vertical", etc. do not mean that the components are required to be absolutely horizontal or suspended, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", and does not mean that the structure must be completely horizontal, but can be slightly tilted. In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0060] The following is a further detailed description through specific implementation methods:

[0061] Example:

[0062] This embodiment discloses a bridge maintenance question-answering method that collaborates with a large model and a knowledge graph.

[0063] like Figure 1As shown in the figure, the bridge maintenance question-answering method based on the collaboration of the big model and the knowledge graph includes:

[0064] S1: Get complex questions to be answered;

[0065] S2: Construct prompts for complex questions and answers in bridge maintenance;

[0066] S3: The complex problem and prompt input are injected into the dialogue model of bridge maintenance knowledge. The complex problem is decomposed through the dialogue model and two sub-problems are output;

[0067] In this embodiment, the large model mainly refers to a large language model, which is an artificial intelligence model with a large number of parameters and powerful language understanding and generation capabilities. This type of model uses deep learning technology to learn the statistical laws and semantic relationships of language by training on a large text corpus, so that it can understand and generate natural language text.

[0068] S4: Use the big dialogue model to understand the first sub-problem at the current iteration and generate a Cypher query statement.

[0069] S5: Search the bridge maintenance knowledge graph according to the Cypher query statement and output the answer to the first sub-question;

[0070] S6: Input the answer to the first sub-question, the second sub-question, and the prompt into the dialogue model, and decompose the second sub-question again through the dialogue model: if the decomposition is successful, output two sub-questions, and return to step S4 for the next iteration; if the decomposition fails, end the iteration, rewrite the second sub-question according to the answer to the first sub-question through the dialogue model, and execute step S7;

[0071] S7: Generate a Cypher query statement for the second sub-question through the dialogue model and retrieve the corresponding answer. Finally, combine the answers to all sub-questions to generate the final answer to the complex question.

[0072] The present invention uses a large dialogue model to decompose and understand complex problems and sub-problems to generate Cypher query statements, and then retrieves the answers to sub-problems in the bridge maintenance knowledge graph through Cypher query statements, that is, the collaboration and integration of the large dialogue model and the bridge maintenance knowledge graph is used to achieve the question and answer of complex problems. First, the large dialogue model can understand complex language structures and contextual information, while the knowledge graph provides rich domain knowledge. Therefore, through the collaboration of the large dialogue model and the knowledge graph, it is possible to achieve an in-depth understanding and analysis of the bridge maintenance field, and then fully understand the complex problems of bridge maintenance, thereby improving the accuracy of bridge maintenance questions and answers. Secondly, the combination of the large dialogue model and the knowledge graph can decompose and simplify the problem, adaptively adjust and generate dynamic instruction prompts, and through the iterative interaction of the large model and the knowledge graph, obtain the problem evidence chain, infer the cause of the disease and maintenance suggestions, thereby achieving the generation of inductive answers to complex bridge maintenance problems, and providing intelligent decision-making assistance for management and maintenance. Finally, when the dialogue model decomposes complex questions and sub-questions and generates Cypher query statements for sub-questions, the present invention provides a bridge for the dialogue model to manage complex questions and answers with a Prompt instruction prompt. The constructed Prompt instruction prompt can provide a framework or template for decomposing a question for the dialogue model, so that the large model can split the complex question into multiple sub-questions according to this framework, thereby reducing the difficulty of the large model in processing complex questions. At the same time, the Prompt instruction prompt can also constrain the style and logic of the Cypher query statement, thereby ensuring that the generated Cypher query statement meets the expected format and specifications, thereby improving the accuracy and efficiency of subsequent answer retrieval.

[0073] Based on the collaborative dialogue model and knowledge graph, the present invention uses the dialogue model and knowledge graph to iteratively decompose and retrieve answers to complex problems and sub-problems until the sub-problems cannot be decomposed or the iteration stop condition is reached. First, by iteratively decomposing complex problems through the dialogue model, a complex problem can be split into multiple simpler sub-problems, and then the answer is retrieved for each sub-problem, which reduces the difficulty and complexity of processing bridge maintenance questions and answers. Secondly, the present invention rewrites and further decomposes other sub-problems according to the answer to the current sub-problem during the iteration process, so that various complex problems, including those involving multiple aspects or requiring reasoning, can be handled more flexibly, and the interpretability of bridge maintenance complex problem questions and answers can be improved, thereby improving the effect of bridge maintenance questions and answers. Finally, the present invention retrieves the answer to each sub-problem independently, ensuring that each sub-problem is accurately answered, and at the same time gradually constructs the complete answer to the complex problem in an iterative manner, thereby ensuring the integrity of the final answer, and further ensuring the improvement of the interpretability of bridge maintenance complex problem questions and answers. This method of gradually approaching the final answer in an iterative manner can reduce unnecessary calculations and resource consumption, thereby improving the processing efficiency of bridge maintenance questions and answers.

[0074] During the specific implementation process, the Prompt instructions include sub-problem decomposition, entity recognition, relationship recognition, hop count judgment, Cypher query generation and answer inductive generation.

[0075] In this embodiment, the prompt instruction consists of six aspects, including sub-problem decomposition, entity recognition, relationship recognition, hop count judgment, Cypher query generation, and answer inductive generation. The detailed explanation is shown in Table 1:

[0076] Table 1 Prompt command prompt classification

[0077]

[0078] In the present invention, the prompt instruction includes sub-problem decomposition, entity recognition, relationship recognition, hop count judgment, Cypher query generation and answer inductive generation. First, entity recognition and relationship recognition can ensure that the entities and relationships involved in the query statement match the description in the question, which helps to reduce the query deviation caused by misunderstanding or recognition error, thereby improving the accuracy of the query result. Secondly, hop count judgment is crucial in graph query. It determines the depth that the query needs to traverse in the graph database. Through hop count judgment, unnecessary deep traversal can be avoided, the query path can be optimized, and the query efficiency can be improved. Further, for complex queries involving multiple entities and relationships, the capabilities of entity recognition and relationship recognition enable the dialogue large model to generate more complex Cypher query statements, thereby supporting a wider range of query requirements. Finally, the Cypher query statement generated by clear entity, relationship recognition and hop count judgment has higher interpretability, which helps users understand the logic and process of the query and improves the transparency and credibility of the system.

[0079] In the specific implementation process, when the dialogue model decomposes a complex problem or sub-problem, it decomposes the complex problem or sub-problem into two sub-problems, that is, the decomposition set of the complex problem is expressed as:

[0080]

[0081] The first decomposition of the complex problem is the 0th iteration, then represents the first subproblem and the second subproblem obtained in the i-th iteration; n represents the number of iterations; represents the mth retrieval subproblem of the first subproblem obtained in the i-th iteration;

[0082] In each iteration, the first subproblem The answer can be retrieved in the current iteration, and the second sub-question As the main issue of the next iteration, we need to Rewrite or break down the answer to the question;

[0083] question It can be decomposed into m retrieval sub-problems according to the number of Cypher queries m.

[0084] The prompt is dynamically and adaptively generated under the iterative interaction between the large dialogue model and the knowledge graph. The content of each prompt step will include six parts: sub-problem decomposition, entity recognition, relationship recognition, hop count judgment, Cypher query generation, and answer inductive generation. The iterative prompt design mechanism is shown in Table 2.

[0085] Table 2 Complex problem decomposition strategies

[0086]

[0087] The overall process of Table 2 is as follows:

[0088] In the 0th iteration, the complex problem is decomposed into sub-problems (0.1) and (0.2), and the answer to sub-problem (0.1) is retrieved, while question (0.2) is used as the problem for the next iteration;

[0089] In the first iteration, subproblem (0.2) is rewritten and decomposed into subproblem (1.1) and subproblem (1.2). Subproblem (1.1) is analyzed and retrieved, and subproblem (1.2) is used as the problem for the next iteration;

[0090] The second iteration is the last iteration. Since subproblem (1.2) cannot be decomposed any further, we only need to rewrite subproblem (1.2) according to the answer to subproblem (1.1) to obtain subproblem (2.1), and finally retrieve the answer to subproblem (2.1).

[0091] In subproblem (1.1), in order to distinguish the crack-type diseases corresponding to different component elements, different Cypher query statements need to be constructed for retrieval. Therefore, subproblem (1.1) can be divided into two Cypher queries, corresponding to two different retrieval subproblems. Similarly, subproblem (2.1) can be divided into three Cypher queries. The idea of ​​task planning is applied in both problem decomposition and Cypher generation, but the two have different focuses.

[0092] In the specific implementation process, the dialogue model achieves problem decomposition through the following steps:

[0093] S301: Match the problem to be decomposed (i.e., a complex problem or a sub-problem) with the Prompt decomposition prompts of different problems in the Prompt instruction library to obtain the similarity between the problem to be decomposed and each Prompt decomposition prompt;

[0094] The matching function is expressed as:

[0095] f(Q,P)=similarity(llm(Q),llm(P));

[0096] Where: llm(Q) and llm(P) represent the results of the dialogue model encoding the input question Q and prompt P; the similarity function is used to calculate the similarity between the encoding vectors;

[0097] The output of the dialogue model is (question, prompt, score), that is, S = (q i ,p j ,score t ), which means that the similarity score between the i-th question and the j-th prompt is score t ;

[0098] The sorting function is:

[0099] Rank(S) = S′;

[0100] Among them, S′ is based on the similarity score t The S sequence is sorted from high to low; the dialogue model will decompose the complex problem into sub-problems according to the matched Prompt instructions.

[0101] S302: taking the Prompt decomposition hint with the highest similarity as the optimal Prompt decomposition hint of the problem to be decomposed;

[0102] S303: parsing the optimal Prompt decomposition prompt to extract key information, including the question to be answered, the required information type, and possible reasoning paths;

[0103] S304: Based on the analyzed key information, further understand the specific content and context of the problem to be decomposed;

[0104] In this embodiment, it includes identifying entities, relationships and events in the problem, and understanding the logical relationship between these elements.

[0105] S305: Decompose the problem to be decomposed into several sub-problems based on the specific content and context understood.

[0106] In this embodiment, this can be achieved in a variety of ways, such as decomposing by space, time, structure or process, so that each sub-problem or task is more specific and easier to handle, which helps the model gradually approach the final answer.

[0107] In the specific implementation process, the dialogue model is based on ChatGLM3-6B. The phased fine-tuning method is used to inject bridge maintenance knowledge graph related information (including knowledge graph mode, entity type, relationship type, entity instance, etc.) and domain background knowledge (maintenance suggestion formulation, disease cause analysis and other professional information) into the dialogue model to form a graph dialogue model BKG-GPT for bridge maintenance questions and answers. Figure 2 shown.

[0108] The processing steps of the staged fine-tuning method are as follows:

[0109] 1) The first stage: convert the bridge maintenance knowledge graph into a format that the dialogue model can understand (such as triples (entity-relationship-entity) or graph structures); then construct domain problem pairs based on the bridge maintenance knowledge graph to perform P-tuning v2 training on the dialogue model, realize parameter tuning of the dialogue model, and enable the dialogue model to understand and apply the knowledge of the bridge maintenance knowledge graph;

[0110] like Figure 3 As shown in the figure, the instance layer of the bridge maintenance knowledge graph follows the concept layer model. The concept graph summarizes the basic information of different bridge types, structural composition and disease-related information of specific important structural parts, including the causes of diseases, detection methods and maintenance suggestions. The instance data layer strictly abides by the concept model and stores the relevant information of the actual bridge, structural composition, structural diseases and other related information in an associative manner. According to the storage mode of the bridge maintenance knowledge graph, the causes of defects of the current bridge can be logically deduced based on the historical knowledge analysis of the same bridge type and similar cases, and corresponding maintenance suggestions can be provided.

[0111] An example of question-answer pair data construction is shown in Table 3.

[0112] Table 3. Example of constructing prompt-turning question-answer pair data in the large dialogue model

[0113]

[0114] In the first stage, the P-tuning v2 method combines its advantages in multi-task learning. In the first stage, this method is used to perform parameter tuning on the large dialogue model in order to inject relevant information from the bridge maintenance knowledge graph, and can simultaneously process multiple knowledge graph question-answering related tasks such as named entity recognition, relationship extraction, hop count judgment, and Cypher query generation through a custom Prompt prefix, so that the large dialogue model can more accurately adapt to various task requirements. The input of the large dialogue model consists of a fixed Prompt prefix and a sample pair of a specific task, that is, z = [p; x; y], where p is the Prompt virtual Token prefix, x is the actual question input, and the label y is usually the output that needs to be predicted by the model, which can be a complete answer or partial information (such as a masked token). The input is encoded as E z =[h;e x ;e y ], where the Prompt prefix is ​​embedded in h and is represented as follows:

[0115] h=(h0,h1,…,h i );

[0116]

[0117] Among them, h i is the embedding vector of the ith virtual token in the Prompt sequence. These embeddings are pre-set or learned during training. i is the activation at time step i, which is the concatenation of all activation layers at that time step, is the activation of the jth layer at time step i. Attention is the self-attention mechanism, FeedForward is the feedforward network part of this layer, and LayerNorm is the layer normalization operation, which helps stabilize the training process.

[0118] 2) The second stage: For the dialogue model that has been injected with relevant information from the bridge maintenance knowledge graph in the first stage, design appropriate training tasks (such as text classification, question answering, summarization, etc.), and use the P-tuning method to inject professional information such as maintenance recommendations and causes of diseases as domain knowledge background.

[0119] In the second stage, the P-tuning method is used to inject bridge maintenance background knowledge such as maintenance suggestions and disease causes. The LSTM embedding in P-tuning can better capture the contextual dependencies and latent semantic structures of long texts on bridge maintenance.

[0120] The input of the dialogue model is:

[0121] {h0,h1,…,h i ,e(x),h i+1,…,h n ,e(y));

[0122]

[0123] Where: h is the continuous prompt word, h i is the Prompt embedding input into the large dialogue model after encoding using LSTM+MLP; e(x) represents the embedding vector of input x; e(y) represents the embedding vector of label y; h 0:i represents the hidden state set from the 0th element to the i-th element in the Prompt sequence, which contains the context information from the beginning of the sequence to the current i-th virtual token position; h i:n Represents the hidden state set from the i-th element to the n-th element, which contains the context information from the current virtual token position to the end of the sequence; represents the positive hidden state of the i-th element; represents the reverse hidden state of the i-th element.

[0124] The optimization objectives during fine-tuning are as follows:

[0125]

[0126] Among them, N is the number of samples in the training set, f llm (e(x j ),h j ) is a pre-trained dialogue model with a prompt vector for the input e(x j )’s predicted output, e(y j ) is the corresponding sample embedding e(x j ) is the true label embedding, and l is the loss function.

[0127] During the specific implementation process, the dialogue model performs entity recognition, relationship recognition, and hop count judgment on the first sub-problem under the current iteration number, and generates the corresponding Cypher query statement based on the results of entity recognition, relationship recognition, and hop count judgment.

[0128] The bridge maintenance entity type, relationship type, and hop count examples are injected into the dialogue model through prompts, telling the model what types of entities, relationships, and hop count relationships between different entities to look for in complex bridge maintenance issues. The dialogue model uses the built-in NER function to identify professional entities in the question text; determines entity relationships by analyzing the text descriptions between entities; predicts the number of hops required in the query based on the complexity of the entity relationship; and constructs one or more Cypher query statements using the identified entity, relationship, and hop count information. Prompt examples are shown in Table 4.

[0129] Table 4 Prompt Example

[0130]

[0131]

[0132] The prompt instructions for entity recognition, relationship extraction, and hop count judgment are static, while the prompt prompts for Cypher queries need to be dynamically matched. The matching model can refer to the prompt prompt matching model for problem decomposition.

[0133] In the present invention, the dialogue model generates corresponding Cypher query statements based on the results of entity recognition, relationship recognition and hop count judgment. First, entity recognition and relationship recognition can ensure that the entities and relationships involved in the query statement match the description in the question, which helps to reduce query deviations caused by misunderstandings or recognition errors, thereby improving the accuracy of query results. Secondly, hop count judgment is crucial in graph queries. It determines the depth that the query needs to traverse in the graph database. Through hop count judgment, unnecessary deep traversals can be avoided, the query path can be optimized, and the query efficiency can be improved. Furthermore, for complex queries involving multiple entities and relationships, the capabilities of entity recognition and relationship recognition enable the dialogue model to generate more complex Cypher query statements, thereby supporting a wider range of query requirements. Finally, the Cypher query statements generated by clear entity, relationship recognition and hop count judgment have higher interpretability, which helps users understand the logic and process of the query and improves the transparency and credibility of the system.

[0134] During the specific implementation process, the answer is retrieved in the Neo4j knowledge graph of bridge management and maintenance according to the Cypher query statement of the sub-question, and the corresponding answer is output; the Neo4j knowledge graph of bridge management and maintenance includes basic information of different bridge types and bridge instances, structural composition and disease-related information of specific important structural parts, and the technical status of the bridge.

[0135] In this embodiment, the py2neo library is installed in the Python environment, the URI, user name and password of the Neo4j database are obtained, and a connection with the Neo4j database is established through the code "graph = Graph ("bolt: / / localhost:7687", auth = ("neo4j", "your_password"))", and the Neo4j database stores the constructed bridge maintenance knowledge graph. The Cypher query statement generated by the dialogue model is executed using the Graph.run() method of py2neo, and the information returned by the query is sorted and output as the answer to the sub-question.

[0136] like Figure 3As shown in the figure, the bridge maintenance knowledge graph consists of a concept layer and an instance layer. The concept layer is a summary of common concepts and categories in the field, including the classification of bridge types, the general definition of structural composition, the common cognition of bridge diseases and disease-related parameters, etc. The concept layer provides a common graph architecture and terminology system for the instance layer. The instance layer contains specific bridge maintenance case data. Bridge instances belong to different bridge types in the concept layer, including basic attributes such as construction year, geographical location and pictures, center pile number, length, width, etc. Its structural composition is consistent with the structure of the bridge type to which it belongs in the concept layer. If bridge structural diseases such as cracks, corrosion, and spalling are not discovered and handled in time, they may lead to structural failure and even cause bridge collapse accidents. Therefore, the disease entity is the core entity of the bridge maintenance knowledge graph, and its associated entities include key information such as disease causes, maintenance recommendations, detection methods, detection parameters, disease degree, disease location, and disease pictures. The time attributes of the disease include discovery time, recording time, and repair time, which are used to record the evolution of the disease, track the historical development of the disease, analyze the speed and pattern of disease progression, and thus predict future disease trends and possible maintenance measures. Quantitative attributes are used to record the growth of the same type of disease, which helps to monitor the development trend of the disease. Component entities connect fine-grained diseases, and parts, components, and component elements are associated with the statistical information of the disease. In addition, the maintenance behavior of the bridge, such as detection, monitoring, recording, analysis, early warning, evaluation, and maintenance, is recorded in an event-based manner to form a knowledge graph with logical logic. The knowledge graph in this field can assist in bridge maintenance decision-making based on the same bridge type and similar case analysis.

[0137] The data sources of the knowledge graph mainly include relevant standards and specifications in the field, bridge inspection reports, maintenance records, and bridge maintenance cases. The above bridge maintenance text is preprocessed to filter out important information in the text and divide it into a set of fixed-length text paragraphs. Similarly, the fine-tuned BKG-GPT dialogue model is used to extract entities, relationships, and attributes from the maintenance text to form import files of different entity types and relationship types, and the file format is converted to CSV format for Neo4j's Load CSV import. Web document management of defective images is implemented through Python programming, and local images are uploaded to the web page to obtain the URI link format of the image, and it is saved as a knowledge triple format of <disease, defective image, image URI> to achieve visualization of defective images in Neo4j.

[0138] During the specific implementation process, when the answer to the first sub-question, the next sub-question and the Prompt instruction are input into the dialogue model, the answer to the first sub-question is injected into the dialogue model as external knowledge enhancement through the RAG retrieval enhancement generation mechanism, that is, the dialogue model is enhanced through the bridge maintenance knowledge graph.

[0139] RAG includes two processes: retrieval and generation. For domain intelligent question answering tasks, the RAG strategy first retrieves evidence related to the input question from the bridge maintenance knowledge graph. The retrieved evidence is then converted into text form and concatenated with the question to be decomposed and the prompt. Finally, the fused unknown question, known evidence, and prompt conditions are input into the BKG-GPT generative dialogue model, and the model performs the next step of question parsing based on this, that is, returns to step S4.

[0140] The present invention is based on the idea of ​​collaboration between the dialogue big model and the knowledge graph. It uses the dialogue big model to enhance the knowledge graph retrieval and the knowledge graph to enhance the dialogue big model reasoning in two stages. It meets the complex question and answer needs of bridge maintenance coordinated by the dialogue big model and the knowledge graph. Through multi-step iterative interactive feedback between the dialogue big model and the knowledge graph, it adaptively generates Prompt command prompts and RAG evidence chains, avoiding the problems of incomplete and inaccurate reasoning results caused by incomplete static Prompt prompt information and insufficient RAG enhancement. This can improve the interpretability of complex question and answer problems in bridge maintenance, and also make the provided maintenance decision-making assistance plan more in line with specific task scenarios.

[0141] In the specific implementation process, in each iteration, before decomposing the second sub-problem again, first determine whether the number of iterations is greater than the maximum number of iterations: if so, end the iteration and execute step S7; otherwise, continue to calculate the confidence of the answer to the first sub-problem, if the confidence of the answer is greater than the confidence threshold, end the iteration and execute step S7, otherwise, decompose the second sub-problem again;

[0142] like Figure 4 As shown in the figure, a judgment condition is needed to determine whether to continue the next iteration. Therefore, the dialogue model (BKG-GPT domain dialogue model) is used to judge the confidence α of the answer. If α ≥ 0.8, an iterative evidence chain is formed, otherwise the iteration continues. Considering the server resources and time cost, the maximum number of iterations k is set in advance. If k ≥ 5, the iteration will end even if α < 0.8. Define the evidence unit e for each iteration m is "reasoning path + retrieval result", that is, e m = {r m ,a m}, connect these evidences in logical order to form an evidence chain E = {e1, e2, …, e m In order to better explain the source and logic of the answer to the bridge maintenance users, the evidence chain is visualized, and the reasoning path from query to answer is displayed in the form of evidence subgraph. It is then input into the dialogue model for retrieval enhancement and induction and summary, and the summarized answer is given. Based on the evidence chain, a natural language explanation is generated to show that the answer given is well-founded.

[0143] The formula for calculating the confidence of the answer is as follows:

[0144]

[0145] P(w i |w1,w2…,w i-1 ,q2,a1,p)=softmax(out_linear(h i-1 ));

[0146] Where: α represents the confidence of the answer; P(w i |w1,w2…,w i-1 ,q2,a i ,p) is the given context (i.e., the question to be answered q2, the answer to the retrieved sub-question a1 and the corresponding prompt p) and the previously generated words (w1,w2…,w i-1 ) to generate the i-th word w i The conditional probability of ; n is the number of words in the answer; h i-1 It is the previous hidden layer state of the Transformer model in the large dialogue model. After a linear layer and softmax function transformation, the probability distribution prediction of the next word is obtained.

[0147] where w i It represents the i-th word of the answer a2 generated by the dialogue model based on q2, a1, p. The previous generated word is the word generated before the i-th word. a2=(w1, w2…, w i ,…,w n ), the principle is to let the big model generate answers based on q2, a1, p in each round of iteration. If the generative answer obtained by the big model according to the given conditions is greater than the confidence threshold, it means that the answer is close to the correct answer, and the iteration can be ended. S7 is executed to perform RAG enhancement according to the chain of evidence, guiding the big model to generate a more reasonable and explainable final inductive answer within a limited range.

[0148] During the specific implementation process, in the iteration process, the dialogue model adds the answer to each sub-question into the evidence chain to continuously improve the evidence chain during the iteration process. After the iteration, the answers on the evidence chain are summarized and summarized to generate the final answer to the complex question. Please refer to the process shown in Table 5.

[0149] Table 5 Prompt dynamic adaptive generation example

[0150]

[0151]

[0152]

[0153]

[0154] The present invention retrieves the answer to each sub-question independently, ensuring that each sub-question is answered accurately. At the same time, the complete answer to the complex question is gradually constructed in an iterative manner, thereby ensuring the integrity of the final answer, and further ensuring the improvement of the interpretability of the complex question and answer of bridge maintenance. This method of gradually approaching the final answer in an iterative manner can reduce unnecessary calculations and resource consumption, thereby improving the processing efficiency of bridge maintenance questions and answers.

[0155] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit the technical solution. Those skilled in the art should understand that those modifications or equivalent substitutions of the technical solution of the present invention that do not depart from the purpose and scope of the technical solution should be included in the scope of the claims of the present invention.

Claims

1. A bridge maintenance question-answering method based on a large model and knowledge graph, characterized in that: include: S1: Get complex questions to be answered; S2: Construct prompts for complex questions and answers in bridge maintenance; S3: Inject the complex problem and prompt command input into the dialogue model of bridge maintenance knowledge, decompose the complex problem through the dialogue model, and output two sub-problems; S4: Understand the first sub-problem at the current iteration through the dialogue model and generate a Cypher query statement; S5: Search the bridge maintenance knowledge graph according to the Cypher query statement and output the answer to the first sub-question; S6: Input the answer to the first sub-question, the second sub-question, and the prompt into the dialogue model, and decompose the second sub-question again through the dialogue model: if the decomposition is successful, output two sub-questions, and return to step S4 for the next iteration; If the decomposition fails again, the iteration ends, and the second sub-question is rewritten according to the answer to the first sub-question through the dialogue model, and step S7 is executed; In step S3 and step S6, when the dialogue model decomposes the complex problem or sub-problem, the complex problem or sub-problem is decomposed into two sub-problems; The decomposition set of complex problems is expressed as: The first decomposition of the complex problem is the 0th iteration, then represents the first subproblem and the second subproblem obtained in the i-th iteration; n represents the number of iterations; represents the mth retrieval subproblem of the first subproblem obtained in the i-th iteration; In each iteration, the first subproblem The answer is retrieved in the current iteration, and the second sub-question As the main issue of the next iteration, we need to Rewrite or break down the answer to the question; Sub-questions According to the number of Cypher queries m, it is decomposed into m retrieval sub-problems; S7: Generate a Cypher query statement for the second sub-question through the dialogue model and retrieve the corresponding answer. Finally, combine the answers to all sub-questions to generate the final answer to the complex question.

2. The bridge maintenance question-answering method based on the collaboration of a large model and a knowledge graph as claimed in claim 1 is characterized by: In step S2, the Prompt instruction prompts include sub-problem decomposition, entity recognition, relationship recognition, hop count judgment, Cypher query generation and answer summary generation.

3. The bridge maintenance question-answering method based on the collaboration of a large model and a knowledge graph as claimed in claim 1 is characterized by: In step S3, the dialogue model implements problem decomposition through the following steps: S301: Match the problem to be decomposed with the prompt decomposition prompts of different problems in the prompt instruction library to obtain the similarity between the problem to be decomposed and each prompt decomposition prompt; S302: taking the Prompt decomposition hint with the highest similarity as the optimal Prompt decomposition hint for the problem to be decomposed; S303: parsing the optimal Prompt decomposition prompt to extract key information, including the question to be answered, the type of information required, and possible reasoning paths; S304: Based on the analyzed key information, further understand the specific content and context of the problem to be decomposed; S305: Decompose the problem to be decomposed into several sub-problems based on the specific content and context understood.

4. The bridge maintenance question-answering method based on the collaboration of a large model and a knowledge graph as claimed in claim 1 is characterized by: In step S3, the dialogue model is based on ChatGLM3-6B, and a phased fine-tuning method is used to inject bridge maintenance knowledge graph related information and domain background knowledge into the dialogue model to form a graph dialogue model BKG-GPT for bridge maintenance questions and answers. The processing steps of the staged fine-tuning method are as follows: 1) The first stage: convert the bridge maintenance knowledge graph into a format that can be understood by the dialogue big model; then construct domain problem pair data based on the bridge maintenance knowledge graph to perform P-tuning v2 training on the dialogue big model, realize parameter tuning of the dialogue big model, and enable the dialogue big model to understand and apply the knowledge of the bridge maintenance knowledge graph; 2) The second stage: For the dialogue model that has been injected with the bridge maintenance knowledge graph in the first stage, design appropriate training tasks, and use the P-tuning method to inject professional information such as maintenance suggestions and causes of diseases as domain knowledge background to fine-tune the dialogue model; In the P-tuning method, the dialogue model input is: {h0,h1,…,h i ,e(x),h i+1 ,…,h n ,e(y); Where: h is the continuous prompt word, h i is the Prompt embedding input into the large dialogue model after encoding using LSTM+MLP; e(x) represents the embedding vector of input x; e(y) represents the embedding vector of label y; h 0:i represents the hidden state set from the 0th element to the i-th element in the Prompt sequence, which contains the context information from the beginning of the sequence to the current i-th virtual token position; h i:n Represents the hidden state set from the i-th element to the n-th element, which contains the context information from the current virtual token position to the end of the sequence; represents the positive hidden state of the i-th element; Represents the reverse hidden state of the i-th element; The optimization objectives during fine-tuning are as follows: Where: N is the number of samples in the training set, f llm (e(x j ),h j ) is a pre-trained dialogue model with a prompt vector for the input e(x j )’s predicted output, e(y j ) is the corresponding sample embedding e(x j ) is the true label embedding, and l is the loss function.

5. The bridge maintenance question-answering method based on the collaboration of a large model and a knowledge graph as claimed in claim 1 is characterized by: In step S4, the dialogue model performs entity recognition, relationship recognition, and hop count judgment on the first sub-problem under the current iteration number, and generates a corresponding Cypher query statement based on the results of entity recognition, relationship recognition, and hop count judgment.

6. The bridge maintenance question-answering method based on the collaboration of a large model and a knowledge graph as claimed in claim 1, characterized in that: In step S5, the answer is retrieved in the Neo4j knowledge graph of bridge maintenance according to the Cypher query statement of the sub-question, and the corresponding answer is output; wherein the Neo4j knowledge graph of bridge maintenance includes basic information of different bridge types and bridge instances, structural composition and disease-related information of specific important structural parts, and technical status of the bridge.

7. The bridge maintenance question-answering method based on the collaboration of a large model and a knowledge graph as claimed in claim 1 is characterized by: In step S6, when the answer to the first sub-question, the next sub-question and the Prompt instruction are input into the dialogue model, the answer to the first sub-question is injected into the dialogue model as external knowledge enhancement through the RAG retrieval enhancement generation mechanism, that is, the dialogue model is enhanced through the bridge maintenance knowledge graph.

8. The bridge maintenance question-answering method based on the collaboration of a large model and a knowledge graph as claimed in claim 1, characterized in that: In step S6, in each iteration, before decomposing the second sub-problem again, first determine whether the number of iterations is greater than the maximum number of iterations: if so, end the iteration and execute step S7; otherwise, continue to calculate the confidence of the answer to the first sub-problem, if the confidence of the answer is greater than the confidence threshold, end the iteration and execute step S7, otherwise, decompose the second sub-problem again; The formula for calculating the confidence of the answer is as follows: P(w i |w1,w2…,w i-1 ,q2,a1,p)=softmax(out_linear(h i-1 )); Where: α represents the confidence of the answer; P(w i |w1,w2…,w i-1 ,q2,a1,p) is the given context, i.e. the second sub-question q2, the answer a1 to the first sub-question and the corresponding prompt instruction p and the previously generated words (w1,w2…,w i-1 ) to generate the i-th word w i The conditional probability of ; n is the number of words in the answer; h i-1 It is the previous hidden layer state of the Transformer model in the large dialogue model. After a linear layer and softmax function transformation, the probability distribution prediction of the next word is obtained.

9. The bridge maintenance question-answering method based on the collaboration of a large model and a knowledge graph as claimed in claim 1, characterized in that: In step S6, during the iteration process, the dialogue model adds the answer to each sub-question to the evidence chain to continuously improve the evidence chain during the iteration process. After the iteration, the answers on the evidence chain are summarized and summarized to generate the final answer to the complex question.

Images