Knowledge graph driven pile bearing structure operation and maintenance man-machine interaction system and method

Through the knowledge graph-driven pile bearing structure operation and maintenance human-computer interaction system, deep learning models are used for semantic analysis and intention recognition, combined with the three-dimensional model display, the semantic analysis effect and maintenance of pile bearing structure question-and-answer system in the existing technology is solved. Complex problems achieve more efficient information processing and display.

CN118964569APending Publication Date: 2024-11-15DALIAN MARITIME UNIVERSITY
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411043437.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

The existing pile bearing structure question and answer system has poor results in deep semantic analysis of questions and knowledge, and is too complex to maintain, which is not conducive to non-professional operations.

Method used

A knowledge graph-driven pile bearing structure operation and maintenance human-computer interaction system is designed, including user language processing module, knowledge graph module and question and answer module. The user input text naming is performed through BERT model, BiLSTM model, CRF model, TextCNN model, etc. Entity recognition and intention recognition are combined with knowledge graphs to query and maintain information, and the results are displayed using three-dimensional models.

Benefits of technology

It improves the multi-semantic analysis capability of pile bearing structure information, simplifies the maintenance process of knowledge graphs, facilitates non-professional operations, and reduces the need for manual labeling and template library construction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118964569A_ABST
    Figure CN118964569A_ABST
Patent Text Reader

Abstract

The invention discloses a knowledge graph driven pile bearing structure operation and maintenance man-machine interaction system and method. The system comprises a user language processing module used for performing naming body recognition operation on a text input by a user; the knowledge graph construction module is used for constructing a pile bearing structure knowledge graph, and the knowledge graph operation and maintenance module is used for modifying the knowledge graph; the intention recognition unit is used for determining a user intention; the slot filling unit is used for determining pile bearing structure information inquired by the user and returning the pile bearing structure information to the user; the three-dimensional pile bearing structure module is used for establishing a three-dimensional pile bearing structure model and displaying pile bearing structure information inquired by a user. The intent recognition unit is combined with the slot filling unit to better solve the problems of multi-semantic analysis and need of a large number of templates, and meanwhile, the knowledge graph operation and maintenance module is used for matching according to different semantic templates so as to execute corresponding Cypher statements to complete maintenance operation of knowledge graph nodes and relation information. And great convenience is provided for the user to process the data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and knowledge graph question answering technology, and in particular to a knowledge graph-driven pile-supported structure operation and maintenance human-computer interaction system and method. Background Art

[0002] Pile construction technology has gone through thousands of years of development. A large number of ancient buildings in China now use pile foundation structures. At present, there have been great advances in pile foundation materials, pile types, and construction methods, and a modern foundation engineering system has been formed. Under certain conditions, the use of pile foundations can greatly reduce the workload and material consumption on the construction site. Pile foundations have small deformation and good stability under the action of seismic forces. They are an effective measure to solve the seismic problems of soft foundations and earthquake liquefied foundations in earthquake zones. Pile-bearing structures are deep foundations composed of piles and pile caps connected to the top of the piles. Pile foundations are widely used in coastal areas and soft soil areas. There are various sensors on the pile-bearing structure to obtain various signal information. The existing pile-bearing structure information displays complex and redundant information, and it is impossible to directly understand the relevant information of the pile-bearing structure components. The construction of the atlas information usually needs to be established one by one, which wastes time and is difficult to maintain in the later stage. In particular, as the atlas becomes larger and larger, manual editing and maintenance alone is difficult and prone to errors.

[0003] In the field of pile engineering technology, due to the high degree of professionalism, it is difficult for ordinary users to identify, select and obtain the correct knowledge of pile-bearing structures from a large amount of information. Even professionals in the field of pile-bearing structures need to spend a lot of time and energy to screen and acquire knowledge. In addition, there are few studies on knowledge graph question and answer and operation and maintenance in this field. Generally, they rely on templates for question and answer dialogues, and templates require a lot of manual annotations. They are limited to the increasing amount of data and require a lot of manpower and material resources to build a template library. In addition, the effect of deep semantic analysis of questions and knowledge is poor. At the same time, knowledge graph information maintenance is generally performed in the neo4j database through Cypher statements, which requires professional statement knowledge and each modification and maintenance is too complicated, which is not conducive to non-professionals. Summary of the invention

[0004] The present invention provides a knowledge graph-driven pile-supported structure operation and maintenance human-computer interaction system and method to overcome the technical problems that the existing pile-supported structure question-answering system has poor effect in deep semantic analysis of questions and knowledge, and is too complicated to maintain, making it difficult for non-professionals to operate.

[0005] In order to achieve the above object, the technical solution of the present invention is:

[0006] A knowledge graph-driven pile-supported structure operation and maintenance human-computer interaction system, comprising a user language processing module, a knowledge graph module, a question-answering module, and a three-dimensional pile-supported structure display module;

[0007] The user language processing module is used to perform a named entity recognition operation on the text input by the user, so as to extract entities, entity relationships and semantic information in the text;

[0008] The knowledge graph module includes a knowledge graph construction module and a knowledge graph operation and maintenance module; the knowledge graph construction module is used to construct a pile-bearing structure knowledge graph, and the knowledge graph operation and maintenance module is used to modify the knowledge graph according to the text input by the user;

[0009] The question-answering module includes an intention recognition unit and a slot filling unit; the intention recognition unit is used to determine the user intention based on the text input by the user; the slot filling unit is used to determine the pile-bearing structure information queried by the user based on the entities, entity relationships, semantic information, user intentions and pile-bearing structure knowledge graph in the text, and return it to the user;

[0010] The three-dimensional pile-bearing structure module is used to establish a three-dimensional pile-bearing structure model and display the pile-bearing structure information queried by the user in the three-dimensional pile-bearing structure model.

[0011] Further, the user language processing module includes a first BERT model, a BiLSTM model and a conditional random field CRF model connected in sequence;

[0012] The first BERT model splits the text input by the user according to spaces to obtain multiple tokens, each of which corresponds to a separate word, and maps each token to a low-dimensional vector space through the word embedding tool Word2Vec. The vector in the vector space is the distributed representation of the token, and then all the vector representations corresponding to all the tokens are sequentially spliced ​​to obtain the vector representation corresponding to the text input by the user, that is, the vector representation of the entire text, and transmit it to the BiLSTM model;

[0013] The BiLSTM model includes a forward LSTM network and a backward LSTM network; the vector representation of the entire text is input into the forward LSTM network and the backward LSTM network respectively to obtain a left-to-right context information vector and a right-to-left context information vector of the vector representation of the entire text, and then the two context information vectors are concatenated one by one to obtain a concatenated vector, and transmitted to the CRF model;

[0014] The CRF model calculates the probability of the label sequence corresponding to all characters in the text input by the user based on the concatenated vector. The process is: calculate the transition probability between the labels of the two entity labels and the emission probability of the feature representation being marked by a certain label; then based on the Viterbi algorithm and using the transition probability and the emission probability, find the label with the highest probability among all possible labels of the last character, and use the label as the end of the sequence, and then backtrack from the last character of the sequence to find the label path with the highest overall probability of a word composed of several characters until backtracking to the beginning of the sequence. The optimal path obtained by backtracking is the output sequence with the highest probability, that is, the most likely label sequence, and then generate entities, entity relationships and semantic information according to the most likely label sequence.

[0015] Furthermore, the process of constructing the pile-supported structure knowledge graph by the knowledge graph module is as follows:

[0016] 1) Obtain relevant knowledge in the field of pile-supported structures, and store the relevant knowledge in the form of web pages as data in json format; for relevant knowledge in the form of documents, convert it into unstructured text data through OCR recognition;

[0017] The relevant knowledge in the field of pile-bearing structures includes pile foundation structure data, pile foundation panel data, displacement sensor data, horizontal sensor data, acceleration sensor data, node types, node labels, reliability classification, safety technical indicators, durability technical indicators, applicability technical indicators, safety-level material performance indicators, durability-level material performance indicators, applicability-level material performance indicators, related diseases, treatment methods, preventive measures and causes of diseases;

[0018] 2) performing cleaning processing on the JSON formatted data and the unstructured text data to obtain processed JSON formatted data and the unstructured text data, wherein the cleaning processing includes removing spaces, encoding errors, and special symbols;

[0019] 3) Convert the processed JSON format data into triple data according to the key-value pairs, and extract triple data from the processed unstructured text data using the template matching method;

[0020] 4) Use Python program to write triple data in json format and unstructured text into Neo4j database for storage, thereby establishing a pile-bearing structure database.

[0021] Furthermore, the process of modifying the knowledge graph according to the text input by the user by the knowledge graph operation and maintenance module is as follows:

[0022] Perform Jieba word segmentation on the text input by the user, and then use the pattern matching rules of the AIML semantic template to match the result of Jieba word segmentation with the set semantic library template. When the semantic library template is matched, call the Cypher statement corresponding to the semantic library template to execute the corresponding knowledge graph modification operation, and feedback the execution result to the user.

[0023] Furthermore, the intention recognition unit includes a second BERT model and a TextCNN model; the intention recognition unit determines the user intention based on the text input by the user, and the slot filling unit determines the pile-bearing structure information queried by the user based on the entities, entity relationships, semantic information, user intentions and pile-bearing structure knowledge graph in the text, and returns it to the user in the following process:

[0024] The second BERT model splits the text input by the user according to spaces to obtain multiple tokens, and maps each token to a low-dimensional vector space through a word embedding tool. The vector in the vector space is the distributed representation of the token, and then all vector representations corresponding to all tokens are sequentially spliced ​​to obtain the vector representation corresponding to the text input by the user, that is, the vector representation of the entire text, and transmit it to the TextCNN model;

[0025] The TextCNN model includes a convolutional layer, a pooling layer, a fully connected layer, and a softmax layer;

[0026] The convolution layer uses three convolution kernels of different sizes to extract the text features of the vector representation of the entire text respectively. The operation process based on any convolution kernel is: the vector representation of the entire text is convolved with a convolution kernel to obtain a feature map C i Then, the convolution kernel is moved to the right by one step, and the vector representation of the entire text and the convolution kernel are convolved to obtain another feature map C i+1 ; Repeat this process until the convolution kernel moves to the rightmost end of the vector representation of the entire text, and then combine several feature maps to obtain a text feature; based on the three convolution kernels, a set of text features C is finally obtained j , j = 1, 2, 3, respectively, representing the text features obtained by using convolution kernels of sizes h1×d, h2×d, and h3×d, and the text features C j Transmitted to the pooling layer;

[0027] Among them, the calculation formula of convolution is:

[0028] C i =f(w·x i +b)

[0029] In the formula, x i represents the i-th row in the input matrix, b is the bias term, w is the convolution kernel parameter, and f is the activation function ReLU;

[0030] The pooling layer is used to transform the text features C output by the convolutional layer j Convert to a one-dimensional vector, that is, the text feature C j Each feature map in is subjected to maximum pooling to obtain a pooled feature vector p, and the feature vector p is transmitted to the fully connected layer;

[0031] The fully connected layer performs a linear transformation on the input feature vector p to obtain a transformed vector, and transmits the transformed vector z to the softmax layer. The transformed vector z is expressed as:

[0032] z=Wp+b

[0033] Among them, W is the weight parameter of the fully connected layer, and b is the bias term;

[0034] The softmax layer maps each component in the transformed vector z to the (0,1) interval and outputs it. The formula is as follows:

[0035]

[0036] The sum of all output values ​​of the softmax layer is 1. Each output value represents the confidence score of a classification label. The user intent is determined based on the confidence score of the classification label with the highest score, and different solutions are executed according to the user intent, including:

[0037] When the confidence score of the classification label is less than 0.4, the knowledge graph is operated and maintained by the knowledge graph operation and maintenance module;

[0038] When the confidence score of the classification label is 0.4-0.8, the user intention is further determined by returning information confirming the user intention. If the user replies to confirm, the user intention recognition is completed, and the pile bearing structure information queried by the user is returned through the slot filling unit; if the user replies to deny, the user intention recognition fails;

[0039] When the confidence score of the classification label is greater than 0.8, the pile-bearing structure information queried by the user is returned through the slot filling unit. The process includes: the slot filling unit first determines the key entities and their information based on the entities, entity relationships and semantic information in the text, and then analyzes and determines the user's question type based on the text input by the user and the user's intention in combination with the pile-bearing structure knowledge graph, and matches the user's question type with several pre-defined question types to determine the intent template corresponding to the user's question type, and then fills the key entities and their information into the vacant positions of the intent template to generate a complete Cypher query statement, and obtains the pile-bearing structure information queried by the user from the pile-bearing structure knowledge graph according to the Cypher query statement and returns it to the user.

[0040] Furthermore, the three-dimensional pile-bearing structure module includes a three-dimensional pile-bearing structure model building module and a display module;

[0041] The three-dimensional pile-supported structure model building module uses the revit modeling tool to build a three-dimensional pile-supported structure model;

[0042] The display module forms a correspondence between the entity and the component information in the three-dimensional pile-bearing structure model through identifier mapping and attribute relationship mapping, and forms a correspondence between the component information in the three-dimensional pile-bearing structure model and the node information body stored in the knowledge graph; and uses the three-dimensional visualization engine Three.js to find the corresponding components in the pile-bearing structure information queried by the user returned to the user by the question and answer module in the three-dimensional pile-bearing structure model, and highlight and display the information.

[0043] A knowledge graph-driven pile-supported structure operation and maintenance human-computer interaction method, the specific steps include:

[0044] S1: Get the text input by the user and extract the entities, entity relations and semantic information in the text;

[0045] S2: determining the user intention based on the text input by the user, wherein the user intention includes performing knowledge graph operation and maintenance and asking questions about the pile-bearing structure;

[0046] S3: determining the pile-bearing structure information queried by the user based on the entities, entity relationships, semantic information, user intentions and the established pile-bearing structure knowledge graph in the text, and highlighting and displaying the pile-bearing structure queried by the user through a three-dimensional pile-bearing structure model;

[0047] S4: modifying the pile-bearing structure knowledge graph based on the text input by the user.

[0048] Beneficial effect: The present invention proposes a knowledge graph-driven pile-bearing structure operation and maintenance human-computer interaction system, which performs named body recognition on the text input by the user through the user language processing module, and constructs a pile-bearing structure knowledge graph, determines the user's intention through the intention recognition unit, and determines the pile-bearing structure information queried by the user through the slot filling unit. The intention recognition unit and the slot filling unit are combined to better solve the problems of multi-semantic parsing and the need for a large number of templates. At the same time, the knowledge graph operation and maintenance module matches different semantic templates and then executes corresponding Cypher statements to complete the maintenance operations of the knowledge graph nodes and relationship information, which greatly facilitates users to process data. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0050] Figure 1 This is a flow chart of a knowledge graph-driven human-computer interaction method for pile-supported structure operation and maintenance in the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] This embodiment provides a knowledge graph driven pile support structure operation and maintenance human-computer interaction system, such as Figure 1As shown, it includes a user language processing module, a knowledge graph module, a question-and-answer module and a three-dimensional pile-bearing structure display module; the user language processing module is used to perform named entity recognition operations on the text input by the user, so as to extract entities, entity relationships and semantic information in the text; the knowledge graph module includes a knowledge graph construction module and a knowledge graph operation and maintenance module; the knowledge graph construction module is used to construct a pile-bearing structure knowledge graph, and the knowledge graph operation and maintenance module is used to modify the knowledge graph according to the text input by the user; the question-and-answer module includes an intention recognition unit and a slot filling unit; the intention recognition unit is used to determine the user's intention based on the text input by the user; the slot filling unit is used to determine the pile-bearing structure information queried by the user based on the entities, entity relationships, semantic information, user intentions and pile-bearing structure knowledge graph in the text, and return it to the user; the three-dimensional pile-bearing structure module is used to establish a three-dimensional pile-bearing structure model, and display the pile-bearing structure information queried by the user in the three-dimensional pile-bearing structure model.

[0053] In a specific embodiment, the user language processing module includes a first BERT model, a BiLSTM model, and a conditional random field CRF model connected in sequence;

[0054] The first BERT model splits the text input by the user according to spaces to obtain multiple tokens, each of which corresponds to a separate word, and maps each token to a low-dimensional vector space through the word embedding tool Word2Vec. The vector in the vector space is the distributed representation of the token, and then all the vector representations corresponding to all the tokens are sequentially spliced ​​to obtain the vector representation corresponding to the text input by the user, that is, the vector representation of the entire text, and transmit it to the BiLSTM model;

[0055] The BiLSTM model includes a forward LSTM network and a backward LSTM network; the vector representation of the entire text is input into the forward LSTM network and the backward LSTM network respectively to obtain a left-to-right context information vector and a right-to-left context information vector of the vector representation of the entire text, and then the two context information vectors are concatenated one by one to obtain a concatenated vector, and transmitted to the CRF model;

[0056] Specifically, the forward LSTM network will traverse each word in the input sequence and generate feature representations based on the left context of each word. These feature representations contain information about the current word and the words on its left, as well as the features carried by the current word itself. The backward LSTM network captures the right-to-left context information of each word, which means that when processing text, the feature vector of each word will contain information about the words on its right, as well as the features carried by the word itself. The backward LSTM network can understand the semantic and grammatical role of each word in its right context, helping the model to more comprehensively understand the context of the entire text sequence. By combining the outputs of the forward and backward LSTM networks, the sequence dependencies between words in the text can be captured more accurately.

[0057] The CRF model calculates the probability of the label sequence corresponding to all characters in the text input by the user based on the concatenated vector. The process is: calculate the transition probability between the labels of the two entity labels and the emission probability of the feature representation being marked by a certain label; then based on the Viterbi algorithm and using the transition probability and the emission probability, find the label with the highest probability among all possible labels of the last character, and use the label as the end of the sequence, and then backtrack from the last character of the sequence to find the label path with the highest overall probability of a word composed of several characters until backtracking to the beginning of the sequence. The optimal path obtained by backtracking is the output sequence with the highest probability, that is, the most likely label sequence, and then generate entities, entity relationships and semantic information according to the most likely label sequence.

[0058] Specifically, in the Viterbi algorithm, the calculation of the scores of possible label sequences is usually performed through dynamic programming. For each time step and each possible state (i.e., label), the score of the state is calculated. This score is usually the product of the emission probability of the state and the transition probability to the state.

[0059] Let the time step be t, the state be s, the observation sequence be x, and the state sequence be y. Then the Viterbi score of the possible labels is δ(t,s), which can be calculated recursively as follows:

[0060] δ(t,s)=max(δ(t-1,s')×transition probability(s'→s)×emission probability(s→x t ))

[0061] Where: δ(t-1,s') represents the maximum score of state s' in the previous time step t-1;

[0062] The transition probability (s'→s) represents the transition probability from state s' to state s;

[0063] Emission probability (s→x t) represents the generation of observation x in state s t The probability of emission;

[0064] By calculating the score of each state, we can find the state that maximizes the score at the current time step, that is, we get the Viterbi score of the possible label; the calculation is performed sequentially at each time step until the calculation of the entire sequence is completed, and finally the most likely label sequence is obtained.

[0065] In a specific embodiment, the process of constructing the pile-bearing structure knowledge graph by the knowledge graph module is as follows:

[0066] 1) Obtain relevant knowledge in the field of pile-supported structures, and store the relevant knowledge in the form of web pages as data in json format; for relevant knowledge in the form of documents, convert it into unstructured text data through OCR recognition;

[0067] The relevant knowledge in the field of pile-bearing structures includes pile foundation structure data, pile foundation panel data, displacement sensor data, horizontal sensor data, acceleration sensor data, node types, node labels, reliability classification, safety technical indicators, durability technical indicators, applicability technical indicators, safety-level material performance indicators, durability-level material performance indicators, applicability-level material performance indicators, related diseases, treatment methods, preventive measures and causes of diseases;

[0068] 2) performing cleaning processing on the JSON formatted data and the unstructured text data to obtain processed JSON formatted data and the unstructured text data, wherein the cleaning processing includes removing spaces, encoding errors, and special symbols;

[0069] 3) Convert the processed JSON format data into triple data according to the key-value pairs, and extract triple data from the processed unstructured text data using the template matching method;

[0070] 4) Use Python program to write triple data in json format and unstructured text into Neo4j database for storage, thereby establishing a pile-bearing structure database.

[0071] In a specific embodiment, the intention recognition unit includes a second BERT model and a TextCNN model; the intention recognition unit determines the user intention based on the text input by the user, and the slot filling unit determines the pile-bearing structure information queried by the user based on the entities, entity relationships, semantic information, user intentions and pile-bearing structure knowledge graph in the text, and returns it to the user in the following process:

[0072] The second BERT model splits the text input by the user according to spaces to obtain multiple tokens, and maps each token to a low-dimensional vector space through a word embedding tool. The vector in the vector space is the distributed representation of the token, and then all vector representations corresponding to all tokens are sequentially spliced ​​to obtain the vector representation corresponding to the text input by the user, that is, the vector representation of the entire text, and transmit it to the TextCNN model;

[0073] The TextCNN model includes a convolutional layer, a pooling layer, a fully connected layer, and a softmax layer;

[0074] The convolution layer uses three convolution kernels of different sizes to extract the text features of the vector representation of the entire text respectively. The operation process based on any convolution kernel is: the vector representation of the entire text is convolved with a convolution kernel to obtain a feature map C i Then, the convolution kernel is moved to the right by one step, and the vector representation of the entire text and the convolution kernel are convolved to obtain another feature map C i+1 ; Repeat this process until the convolution kernel moves to the rightmost end of the vector representation of the entire text, and then combine several feature maps to obtain a text feature; based on the three convolution kernels, a set of text features C is finally obtained j , j = 1, 2, 3, respectively, representing the text features obtained by using convolution kernels of sizes h1×d, h2×d, and h3×d, and the text features C j Transmitted to the pooling layer;

[0075] Among them, the calculation formula of convolution is:

[0076] C i =f(w·x i +b)

[0077] In the formula, x i represents the i-th row in the input matrix, b is the bias term, w is the convolution kernel parameter, and f is the activation function ReLU;

[0078] The pooling layer is used to transform the text features C output by the convolutional layer j Convert to a one-dimensional vector, that is, the text feature C j Each feature map in is subjected to maximum pooling to obtain a pooled feature vector p, and the feature vector p is transmitted to the fully connected layer;

[0079] The fully connected layer performs a linear transformation on the input feature vector p to obtain a transformed vector, and transmits the transformed vector z to the softmax layer. The transformed vector z is expressed as:

[0080] z=Wp+b

[0081] Among them, W is the weight parameter of the fully connected layer, and b is the bias term;

[0082] The softmax layer maps each component in the transformed vector z to the (0,1) interval and outputs it. The formula is as follows:

[0083]

[0084] Specifically, the sum of all output values ​​of the softmax layer is 1, and each output value represents the confidence score of the classification label. The user intention is determined based on the confidence score of the classification label with the highest score, and different solutions are executed according to the user intention, including:

[0085] When the confidence score of the classification label is less than 0.4, the knowledge graph is operated and maintained by the knowledge graph operation and maintenance module;

[0086] When the confidence score of the classification label is 0.4-0.8, the user intention is further determined by returning information confirming the user intention. If the user replies to confirm, the user intention recognition is completed, and the pile bearing structure information queried by the user is returned through the slot filling unit; if the user replies to deny, the user intention recognition fails;

[0087] When the confidence score of the classification label is greater than 0.8, the pile-bearing structure information queried by the user is returned through the slot filling unit. The process includes: the slot filling unit first determines the key entities and their information based on the entities, entity relationships and semantic information in the text, and then analyzes and determines the user's question type based on the user's input sentence and user intent in combination with the pile-bearing structure knowledge graph, and matches the user's question type with several pre-defined question types to determine the intent template corresponding to the user's question type, and then fills the key entities and their information into the vacant positions of the intent template to generate a complete Cypher query statement, and obtains the pile-bearing structure information queried by the user from the pile-bearing structure knowledge graph according to the Cypher query statement and returns it to the user.

[0088] Specifically, the identified user intent corresponds to a label_class tag, such as category, reliability classification, preventive measures, related diseases, etc.

[0089] Specifically, in this embodiment, based on intention inheritance and slot inheritance, multiple rounds of dialogue in the pile-bearing field can be completed; slot inheritance, according to user input and user unique identification, determines whether there is an entity in the dialogue user input. If not, the entity of the previous round is inherited. Intent inheritance means that if there is a clear intention in the previous round, and the intention intensity of the current round is lower than the set threshold and the slot related to the intention of the previous round exists, the intention inheritance will be carried out. For intentions below the set threshold, clarification and confirmation processing is required, support is provided for multiple rounds of questions and answers, and multiple rounds of state machines are designed and maintained, and the dialogue state tracking results of each round are saved. In order to control the scope of multiple rounds of dialogue, a multi-round time window strategy is set, in which the round window is limited to 3 rounds and the time window is limited to 4 minutes. Historical information will be stored in Redis and an expiration time will be set to quickly find context information during the dialogue process. For the same dialogue state management intention, only the most recent one is taken to process the returned results, query statement conversion and database return results. If no data is found, the bottom-up speech is returned.

[0090] In a specific embodiment, the process by which the knowledge graph operation and maintenance module modifies the knowledge graph according to the text input by the user is as follows: performing a jieba word segmentation operation on the text input by the user, and then using the pattern matching rules of the AIML semantic template to match the result after the jieba word segmentation with the set semantic library template, and when the semantic library template is matched, calling the Cypher statement corresponding to the semantic library template to execute the corresponding knowledge graph modification operation, and feeding back the execution result to the user.

[0091] Specifically, the text input by the user is segmented by Jieba, that is, the sentence is decomposed according to the Chinese representation method, and then processed by setting the AIML semantic template, including setting wildcards for the sentence template, replacing key information with *, and using category to solve the problem of the same sentence structure, for example:

[0092] <category>

[0093] <pattern> I want to modify the tag with the tag * and the attribute * to *< / pattern>

[0094] <template>

[0095] <think>

[0096] <set name="star1"><star index="1" / >

[0097] <set name="star2"><star index="2" / >

[0098] <set name="star3"><star index="3" / >

[0099] <set name="star4"><star index="4" / >

[0100] < / think>

[0101] xneo4j:<get name="star1" / >:<get name="star2" / >:<get name="star3"

[0102] / >:<get name="star4" / >

[0103] < / template>

[0104] < / category>

[0105] Then execute the Cypher statement in the corresponding matching function to perform related operations, "MATCH(m:{label})wherem.{name}='{name1}'REMOVE m:{label}set m:{name2}".format(

[0106] label=entity,name=entity1,name1=entity2,name2=entity3)) to complete an operation and maintenance task. At the same time, the knowledge graph operation and maintenance module supports the functional assistance of multiple labels, including <random>, increasing the diversity of responses, <if> <condition>Judgment conditions, <topic>Match by topic,

[0107] <system> learn< / system> The learning function learns the correct answers entered by the user. If there is no feedback on the operation the user wants to perform in the semantic library, the system automatically learns the operation and adds it to the semantic library so that the system can further improve the database next time it encounters the same user demand.

[0108] In this embodiment, the three-dimensional pile-supported structure module includes a three-dimensional pile-supported structure model construction module and a display module; specifically, the three-dimensional pile-supported structure model construction module adopts the revit modeling tool to construct a three-dimensional pile-supported structure model;

[0109] Specifically, the display module forms a corresponding relationship between the entity and the component information in the three-dimensional pile-bearing structure model through identifier mapping and attribute relationship mapping, and forms a corresponding relationship between the component information in the three-dimensional pile-bearing structure model and the node information body stored in the knowledge graph, and uses the three-dimensional visualization engine Three.js to find the corresponding components in the three-dimensional pile-bearing structure model in combination with the data input by the question-answering module, and performs highlighting and information display. Specifically, in this embodiment, each node in the knowledge graph and each component in the three-dimensional pile-bearing structure model have a unique identifier, and correspond one-to-one through the identifier. At the same time, the nodes in the knowledge graph also include attributes (such as material, size, position) and relationships (such as connection relationships, bearing relationships), which are also mapped with the attributes in the three-dimensional pile-bearing structure model and the relationship between the components, so as to ensure that the user can intuitively see the structural part information corresponding to the question-answering result through the display of the three-dimensional pile-bearing structure model during the question-answering process, thereby achieving a more three-dimensional display effect, so that the part information involved in the dialogue result is displayed more three-dimensionally and intuitively.

[0110] This embodiment also proposes a knowledge graph-driven pile-supported structure operation and maintenance human-computer interaction method, which is implemented based on the pile-supported structure operation and maintenance human-computer interaction system. The specific steps include:

[0111] S1: Get the text input by the user and extract the entities, entity relations and semantic information in the text;

[0112] S2: determining the user intention based on the text input by the user, wherein the user intention includes performing knowledge graph operation and maintenance and asking questions about the pile-bearing structure;

[0113] S3: determining the pile-bearing structure information queried by the user based on the entities, entity relationships, semantic information, user intentions and the established pile-bearing structure knowledge graph in the text, and highlighting and displaying the pile-bearing structure queried by the user through a three-dimensional pile-bearing structure model;

[0114] S4: modifying the pile-bearing structure knowledge graph based on the text input by the user.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.< / topic> < / condition> < / if> < / random>

Claims

1. A knowledge graph-driven pile-supported structure operation and maintenance human-computer interaction system. It is characterized in that It includes user language processing module, knowledge graph module, question-answering module and three-dimensional pile-bearing structure display module; The user language processing module is used to perform a named entity recognition operation on the text input by the user, so as to extract entities, entity relationships and semantic information in the text; The knowledge graph module includes a knowledge graph construction module and a knowledge graph operation and maintenance module; the knowledge graph construction module is used to construct a pile-bearing structure knowledge graph, and the knowledge graph operation and maintenance module is used to modify the knowledge graph according to the text input by the user; The question-answering module includes an intention recognition unit and a slot filling unit; the intention recognition unit is used to determine the user intention based on the text input by the user; The slot filling unit is used to determine the pile-bearing structure information queried by the user based on the entities, entity relationships, semantic information, user intentions and pile-bearing structure knowledge graph in the text, and return it to the user; The three-dimensional pile-bearing structure module is used to establish a three-dimensional pile-bearing structure model and display the pile-bearing structure information queried by the user in the three-dimensional pile-bearing structure model.

2. According to claim 1, the knowledge graph-driven pile-supported structure operation and maintenance human-computer interaction system, It is characterized in that The user language processing module includes a first BERT model, a BiLSTM model and a conditional random field CRF model connected in sequence; The first BERT model splits the text input by the user according to spaces to obtain multiple tokens, each of which corresponds to a separate word, and maps each token to a low-dimensional vector space through the word embedding tool Word2Vec. The vector in the vector space is the distributed representation of the token, and then all the vector representations corresponding to all the tokens are sequentially spliced ​​to obtain the vector representation corresponding to the text input by the user, that is, the vector representation of the entire text, and transmit it to the BiLSTM model; The BiLSTM model includes a forward LSTM network and a backward LSTM network; the vector representation of the entire text is input into the forward LSTM network and the backward LSTM network respectively to obtain a left-to-right context information vector and a right-to-left context information vector of the vector representation of the entire text, and then the two context information vectors are concatenated one by one to obtain a concatenated vector, and transmitted to the CRF model; The CRF model calculates the probability of the label sequence corresponding to all characters in the text input by the user based on the concatenated vector. The process is: calculate the transition probability between the labels of the two entity labels and the emission probability of the feature representation being marked by a certain label; then based on the Viterbi algorithm and using the transition probability and the emission probability, find the label with the highest probability among all possible labels of the last character, and use the label as the end of the sequence, and then backtrack from the last character of the sequence to find the label path with the highest overall probability of a word composed of several characters until backtracking to the beginning of the sequence. The optimal path obtained by backtracking is the output sequence with the highest probability, that is, the most likely label sequence, and then generate entities, entity relationships and semantic information according to the most likely label sequence.

3. According to claim 2, the knowledge graph-driven pile-supported structure operation and maintenance human-computer interaction system, It is characterized in that The process of constructing the pile-supported structure knowledge graph by the knowledge graph module is as follows: 1) Obtain relevant knowledge in the field of pile-supported structures, and store the relevant knowledge in the form of web pages as data in json format; for relevant knowledge in the form of documents, convert it into unstructured text data through OCR recognition; The relevant knowledge in the field of pile-bearing structures includes pile foundation structure data, pile foundation panel data, displacement sensor data, horizontal sensor data, acceleration sensor data, node types, node labels, reliability classification, safety technical indicators, durability technical indicators, applicability technical indicators, safety-level material performance indicators, durability-level material performance indicators, applicability-level material performance indicators, related diseases, treatment methods, preventive measures and causes of diseases; 2) performing cleaning processing on the JSON formatted data and the unstructured text data to obtain processed JSON formatted data and the unstructured text data, wherein the cleaning processing includes removing spaces, encoding errors, and special symbols; 3) Convert the processed JSON format data into triple data according to the key-value pairs, and extract triple data from the processed unstructured text data using the template matching method; 4) Use Python program to write triple data in json format and unstructured text into Neo4j database for storage, thereby establishing a pile-bearing structure database.

4. The knowledge graph-driven pile-supported structure operation and maintenance human-computer interaction system according to claim 3, It is characterized in that The process of modifying the knowledge graph according to the text input by the user by the knowledge graph operation and maintenance module is as follows: Perform Jieba word segmentation on the text input by the user, and then use the pattern matching rules of the AIML semantic template to match the result of Jieba word segmentation with the set semantic library template. When the semantic library template is matched, call the Cypher statement corresponding to the semantic library template to execute the corresponding knowledge graph modification operation, and feedback the execution result to the user.

5. According to claim 4, the knowledge graph-driven pile-supported structure operation and maintenance human-computer interaction system, It is characterized in that The intention recognition unit includes a second BERT model and a TextCNN model; the intention recognition unit determines the user intention based on the text input by the user, and the slot filling unit determines the pile-bearing structure information queried by the user based on the entities, entity relationships, semantic information, user intentions and pile-bearing structure knowledge graph in the text, and returns it to the user in the following process: The second BERT model splits the text input by the user according to spaces to obtain multiple tokens, and maps each token to a low-dimensional vector space through a word embedding tool. The vector in the vector space is the distributed representation of the token, and then all vector representations corresponding to all tokens are sequentially spliced ​​to obtain the vector representation corresponding to the text input by the user, that is, the vector representation of the entire text, and transmit it to the TextCNN model; The TextCNN model includes a convolutional layer, a pooling layer, a fully connected layer, and a softmax layer; The convolution layer uses three convolution kernels of different sizes to extract the text features of the vector representation of the entire text respectively. The operation process based on any convolution kernel is: the vector representation of the entire text is convolved with a convolution kernel to obtain a feature map C i Then, the convolution kernel is moved to the right by one step, and the vector representation of the entire text and the convolution kernel are convolved to obtain another feature map C i+1 ; Repeat this process until the convolution kernel moves to the rightmost end of the vector representation of the entire text, and then combine several feature maps to obtain a text feature; based on the three convolution kernels, a set of text features C is finally obtained j , j = 1, 2, 3, respectively, with the size of h 1 ×d、h 2 ×d and h 3 ×d convolution kernel, and convert the text feature C j Transmitted to the pooling layer; Among them, the calculation formula of convolution is: C i =f(w·x i +b) In the formula, x i represents the i-th row in the input matrix, b is the bias term, w is the convolution kernel parameter, and f is the activation function ReLU; The pooling layer is used to transform the text features C output by the convolutional layer j Convert to a one-dimensional vector, that is, the text feature C j Each feature map in is subjected to maximum pooling to obtain a pooled feature vector p, and the feature vector p is transmitted to the fully connected layer; The fully connected layer performs a linear transformation on the input feature vector p to obtain a transformed vector, and transmits the transformed vector z to the softmax layer. The transformed vector z is expressed as: z=Wp+b Among them, W is the weight parameter of the fully connected layer, and b is the bias term; The softmax layer maps each component in the transformed vector z to the (0,1) interval and outputs it. The formula is as follows: The sum of all output values ​​of the softmax layer is 1. Each output value represents the confidence score of a classification label. The user intent is determined based on the confidence score of the classification label with the highest score, and different solutions are executed according to the user intent, including: When the confidence score of the classification label is less than 0.4, the knowledge graph is operated and maintained by the knowledge graph operation and maintenance module; When the confidence score of the classification label is 0.4-0.8, the user intention is further determined by returning information confirming the user intention. If the user replies to confirm, the user intention recognition is completed, and the pile bearing structure information queried by the user is returned through the slot filling unit; if the user replies to deny, the user intention recognition fails; When the confidence score of the classification label is greater than 0.8, the pile-bearing structure information queried by the user is returned through the slot filling unit. The process includes: the slot filling unit first determines the key entities and their information based on the entities, entity relationships and semantic information in the text, and then analyzes and determines the user's question type based on the text input by the user and the user's intention in combination with the pile-bearing structure knowledge graph, and matches the user's question type with several pre-defined question types to determine the intent template corresponding to the user's question type, and then fills the key entities and their information into the vacant positions of the intent template to generate a complete Cypher query statement, and obtains the pile-bearing structure information queried by the user from the pile-bearing structure knowledge graph according to the Cypher query statement and returns it to the user.

6. The knowledge graph-driven pile-supported structure operation and maintenance human-computer interaction system according to claim 5, It is characterized in that The three-dimensional pile-bearing structure module includes a three-dimensional pile-bearing structure model building module and a display module; The three-dimensional pile-supported structure model building module uses the revit modeling tool to build a three-dimensional pile-supported structure model; The display module forms a corresponding relationship between the entity and the component information in the three-dimensional pile-bearing structure model through identifier mapping and attribute relationship mapping, and forms a corresponding relationship between the component information in the three-dimensional pile-bearing structure model and the node information body stored in the knowledge graph; The three-dimensional visualization engine Three.js is used to find the corresponding components in the pile-bearing structure information queried by the user returned to the user by the question-and-answer module in the three-dimensional pile-bearing structure model, and highlight and display the information.

7. A knowledge graph-driven pile-supported structure operation and maintenance human-computer interaction method, based on the system implementation of claims 1-6, It is characterized in that The specific steps include: S1: Get the text input by the user and extract the entities, entity relations and semantic information in the text; S2: determining a user intention based on the text input by the user, wherein the user intention includes performing knowledge graph operation and maintenance and asking questions about the pile-bearing structure; S3: determining the pile-bearing structure information queried by the user based on the entities, entity relationships, semantic information, user intentions and the established pile-bearing structure knowledge graph in the text, and highlighting and displaying the pile-bearing structure queried by the user through a three-dimensional pile-bearing structure model; S4: modifying the pile-bearing structure knowledge graph based on the text input by the user.

Citation Information

Patent Citations

  • Question and answer method based on combination of semantic analysis and vector modeling

    CN114896407A

  • Operation and maintenance knowledge graph updating method and device

    CN117171364A

  • Intelligent question and answer implementation method and system based on large model and semantic map

    CN117271724A

  • Operation and maintenance auxiliary decision-making method based on relay protection device defect knowledge graph

    CN117892814A

  • Operation and maintenance management system for full life cycle of knowledge graph

    CN118364121A