Gate hydraulic hoist knowledge graph construction and question-answering system based on BERT model

Abstract

The gate hydraulic hoist knowledge graph construction and question-answering system based on the BERT model comprises a data collection and preprocessing module, an entity and relation extraction module, a knowledge graph construction module, a natural language understanding module, a knowledge graph retrieval module, an answer generation module and a user interaction module. And the knowledge graph construction module integrates the entity and relation information, generates a visual knowledge graph and is used for supporting intelligent question and answer of the hydraulic hoist. The natural language understanding module is responsible for performing intention recognition and analysis on questions proposed by a user, querying related information through the knowledge graph retrieval module, generating answers and displaying the answers through the user interaction module. All the modules perform data interaction through an API (Application Program Interface), so that the construction, updating and question and answer functions of the knowledge graph are realized. According to the invention, an intelligent and automatic hydraulic hoist information query solution is provided, and the accuracy and efficiency of information retrieval are improved.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and mechanical engineering, and in particular to a method for constructing a knowledge graph of a hydraulic gate hoist based on BERT (Bidirectional Encoder Representations from Transformers) and its application in a question-answering system, and belongs to the field of natural language processing. Background Art

[0002] BERT is a self-supervised model that can learn a good feature representation by itself. As a general technology that has recently emerged, it uses the Transformer network. BERT uses a bidirectional Transformer, which generates downstream task features in series through two LSTMs from left to right and from right to left. Each segmented word has three Embedding concatenation representations, namely Token Embeddings, Segment Embeddings, and Position Embeddings. Among them, Token Embeddings is used for text classification tasks, which converts each word and special symbols into word embeddings; Segment Embeddings separates sentences and marks them; Position Embeddings encodes the position information of each word and perceives the position relationship between each word. It solves the long-standing dependency problem of NLP and the problem that Word2vec cannot solve. It can compare the context on both sides of all layers. It can do both classification tasks and question-answering tasks.

[0003] Knowledge graph is a structured knowledge representation method and data management model based on semantic network. It uses triples (head entity, relationship, tail entity) to represent and associate data. Knowledge graph question-answering system is a question-answering system using knowledge graph. It can answer questions accurately and efficiently, and realize intelligent retrieval of special domain knowledge through interactive question-answering. Knowledge graph question-answering system has been widely used in many fields. For example, Zhu et al. [2] developed a question-answering system based on traditional Chinese medicine knowledge graph and a series of deep learning models, and built an interactive platform to automatically answer user questions. Li and Zhao [3] et al. constructed a system knowledge graph based on crawler data and expert knowledge and proposed a physical fitness intelligent question-answering system based on the knowledge graph to answer physical fitness questions. Hanifah and Kusumaningrum [4] established a question-answering system specifically for natural science problems, using the long short-term memory (LSTM) model to handle answer selection problems. However, as a key field, the field of water conservancy engineering has not yet fully explored the potential advantages of knowledge graph question-answering system.

[0004] Hydraulic gate hoists are widely used in engineering construction, mainly for gate opening and closing, bridge lifting and other occasions in water conservancy projects. Its operation and maintenance involve a lot of professional knowledge, including hydraulic systems, mechanical structures, control systems, etc. In the existing technology, knowledge management and information retrieval of hydraulic gate hoists mainly rely on the following methods: 1. Paper documents and electronic documents: including operation manuals, maintenance manuals, technical standards, etc.

[0005] 2. Expert system: A rule-based system that provides basic fault diagnosis and operational guidance through pre-defined rules and knowledge base.

[0006] 3. Traditional question-answering system: uses keyword matching and simple natural language processing technology to answer users' common questions.

[0007] In the existing technology, there are still many deficiencies in the knowledge management and information retrieval of hydraulic gate hoists, which cannot efficiently meet the needs of users for knowledge acquisition and question answering. Most of the existing question-answering systems are based on rule matching or traditional machine learning methods, which are difficult to handle complex natural language questions. With the widespread application of the BERT model in the field of natural language processing, the knowledge graph of hydraulic gate hoists is constructed using the BERT model and applied to the question-answering system, which is expected to greatly improve the efficiency and accuracy of knowledge management and information retrieval.

[0008] At present, the knowledge management of hydraulic gate hoists in water conservancy projects mainly relies on the maintenance experience of engineers and fault maintenance manuals, which has problems such as insufficient support from professional technicians, high cost of acquiring fault knowledge, low inheritance, and poor accuracy. The details are as follows: 1. Information islands: Information in paper and electronic documents is scattered, making it difficult to centrally manage and quickly retrieve.

[0009] 2. Rigid rules: The rules of expert systems are fixed, making it difficult to handle complex and dynamically changing problems, and they lack the ability to self-learn and update.

[0010] 3. Poor retrieval accuracy: Traditional question-answering systems rely on simple keyword matching and have difficulty understanding the true intention of users’ questions, resulting in low answer accuracy.

[0011] 4. Difficulty in updating knowledge: The knowledge base in the existing system is updated slowly and cannot reflect the latest technologies and operating specifications in a timely manner. Summary of the invention

[0012] The present invention aims to provide a knowledge graph construction method and question-answering system for hydraulic gate hoists based on the BERT (Bidirectional Encoder Representations from Transformers) model, so as to solve the problems of low efficiency and poor accuracy of knowledge management and information retrieval in the prior art.

[0013] The technical solution of the present invention is specifically as follows: The knowledge graph construction and question-answering system for gate hydraulic hoists based on the BERT model includes data collection and preprocessing module, entity and relationship extraction module, knowledge graph construction module, natural language understanding module, knowledge graph retrieval module, answer generation module and user interaction module. Each module interacts with each other through API to form a complete knowledge graph construction and question-answering system. In the entity and relationship extraction module, the following processing steps are performed: Step (2.1) predefines the entity type and relationship type of the hydraulic gate hoist. The hydraulic gate hoist determines the operating equipment according to the current working condition engineering, and then determines the component information. Finally, the part information, state information, behavior information, fault information and sensor data are determined according to the component information. Step (2.2) Build a BERT-BiLSTM-CRF neural network model for entity extraction, divide the vectorized data into training set, test set and validation set as the input of the model, set the model parameters and start training, and finally save the trained model to model_path; Step (2.3) builds a BERT-BiLSTM-Attention neural network model for relation extraction. The model extracts the context representation of the input sentence through the BERT layer, captures the sequence dependency through the BiLSTM layer, and finally assigns weights in the Attention layer to enhance the model's attention to important features. Training is performed after inputting the data set, which includes entity and relationship annotations. The input data is passed to the model in batches, and the prediction value is obtained through forward propagation and the loss function (such as cross entropy loss) is calculated. Backpropagation is used to adjust the model parameters. The order of the data set is randomly shuffled in each round (epoch) of iteration to improve generalization ability. After multiple rounds of training and verification, the model parameters that achieve the best performance are saved in the specified path model_path for subsequent model loading and prediction tasks.

[0014] In the data collection and preprocessing module, the data is processed as follows: Step (1.1) collect text data related to the hydraulic gate hoist; Step (1.2) converts the text data into a unified txt format; Step (1.3) Use Python to clean the data: Use the Jieba word segmentation tool to segment the txt text, remove stop words, and remove special characters and punctuation marks; Step (1.4) defines entity labels and uses the sequence labeling tool LabelStudio to perform BIO sequence labeling on the cleaned text; Step (1.5) Text vectorization: Use the Word2Vec tool of the Gensim library in Python to convert text data into numerical vectors.

[0015] The BERT model is used to perform entity recognition and relationship extraction of documents related to hydraulic gate hoists, which specifically includes the following steps: (2.3.1) Use the pre-trained BERT model to extract features from the hydraulic gate hoist technical documentation; (2.3.2) Based on feature extraction, train a task-specific BERT-BiLSTM-CRF entity extraction model to identify key entities in the document; (2.3.3) Use the bidirectional encoder feature of BERT to extract the relationship between entities; Step (2.4) Prediction: Load the trained BERT-BiLSTM-CRF entity extraction model and BERT-BiLSTM-Attention relationship extraction model to predict the unlabeled data, extracting labeled entities and relationships from a large amount of unprocessed data. This step first extracts key entities from the text using the entity recognition model, and then uses the relationship extraction model to identify the potential relationships between entities. Step (2.5) Evaluation: Model evaluation indicators: Load the trained entity recognition and relationship extraction models to predict the test set, calculate the accuracy, recall, F1 value and precision based on the model prediction results, and calculate the evaluation indicators of the entity recognition model and relationship extraction model respectively to measure the performance of the model.

[0016] In the knowledge graph construction module, the following processing is performed: (3.1) Knowledge fusion: The hydraulic gate hoist fault diagnosis knowledge graph belongs to the vertical domain knowledge graph, which integrates the three entities of engineering, equipment, and components; (3.2) Knowledge storage - Neo4j graph database: Neo4j graph database is selected as the construction platform, and Cypher is used as the query language to realize the data storage, query, update, and visualization functions involved in the hydraulic hoist fault knowledge graph application; (3.3) Knowledge update: In Neo4j, we first compare the newly added data with the existing data to identify new entities and relationships, and then create new nodes and edges in the graph. For existing entities and relationships, we directly use the Cypher query language to perform corresponding update operations. Through regular knowledge updates, we continuously improve the fault knowledge graph, and also provide timely, accurate, and comprehensive information to the question-answering system.

[0017] The answer generation module and the user interaction module constitute the question-answering system. The question-answering system also includes a natural language understanding module. The natural language understanding module uses natural language understanding to convert text information into semantic representations that can be understood by machines. The process is as follows: (4.1) Define question types and construct question templates based on question types; (4.2) Collect and annotate a large number of questions, build training and test datasets, and annotate each question with its corresponding template type; (4.3) Use the BERT model to classify questions: 4.3.1) Segment and tag questions; 4.3.2) Convert the question sentence into BERT input format; 4.3.3) Use the pre-trained BERT model, add a classification layer, and output the question classification results; 4.3.4) Input question data for training and optimize the parameters of the classification layer so that the model can accurately classify questions and identify user intent.

[0018] In the answer generation module, the natural language generation module processes the results of the knowledge graph query into natural language and outputs it to the user; The user interaction module is responsible for receiving questions input by users, processing questions, performing knowledge graph retrieval, and feeding back answers to users.

[0019] The beneficial effects of the present invention are as follows: the present invention constructs a knowledge graph of a hydraulic gate hoist by using a BERT model and applies it to a question-answering system, which has the following advantages: 1. Improve information retrieval efficiency: Be able to quickly and accurately answer user questions and improve the efficiency of knowledge acquisition.

[0020] 2. Enhance the robustness of the system: Leveraging the powerful natural language understanding capabilities of the BERT model, it can handle complex and diverse user questions.

[0021] 3. Rich knowledge base: The question-answering system based on the knowledge graph has more comprehensive and structured knowledge storage and management capabilities.

[0022] 4. Real-time knowledge update: The system can self-learn and update according to new data to ensure the timeliness and accuracy of the knowledge base. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is the overall architecture diagram of the present invention; Figure 2 It is an entity relationship type diagram; Figure 3 This is the architecture diagram of the question-answering system; Figure 4 This is the source map of the knowledge graph dataset; Figure 5 Label the graph for the data; Figure 6 This is the interface diagram of the question-and-answer system. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] like Figure 1 As shown in the figure, the knowledge graph construction and question-answering system of the gate hydraulic opening and closing machine based on the BERT model includes a data collection and preprocessing module, an entity and relationship extraction module, a knowledge graph construction module, a natural language understanding module, a knowledge graph retrieval module, an answer generation module and a user interaction module. The modules interact with each other through APIs to form a complete knowledge graph construction and question-answering system.

[0026] Among them, in the data collection and preprocessing module, the data is processed as follows: Step (1.1) Collect text data such as technical documents, research papers, hydraulic hoist specification documents, International Organization for Standardization standard documents and maintenance manuals related to hydraulic hoists; Step (1.2) converts the text data into a unified txt format; Step (1.3) Use Python to clean the data: Use the Jieba word segmentation tool to segment the txt text, remove stop words, and remove special characters and punctuation marks; Step (1.4) defines entity labels and uses the sequence annotation tool LabelStudio to perform BIO sequence annotation on the cleaned text. The BIO sequence annotation format is shown in Table 1. Step (1.5) Text vectorization: Use the Word2Vec tool of the Gensim library in Python to convert text data into numerical vectors for subsequent deep learning model processing.

[0027] Table 1. BIO sequence annotation format. Entity Tags Start tag Middle Mark End Tag project B-PROJECT M-PROJECT E-PROJECT Fault B-FAULT M-FAULT E-FAULT Components B-COMPONENT M-COMPONENT E-COMPONENT sensor B-SENSOR M-SENSOR E-SENSOR equipment B-DEVICE M-DEVICE E-DEVICE Behavior B-BEHAVIOR M-BEHAVIOR E-BEHAVIOR state B-CONDITION M-CONDITION E-CONDITION Component B-PART M-PART E-PART other O O O

[0028] In the entity and relationship extraction module, the following processing steps are performed: Step (2.1) predefines the hydraulic gate hoist entity type and relationship type, such as Figure 2 As shown, the hydraulic gate hoist determines the operating equipment according to the current working condition engineering, and then determines the component information, and finally determines the part information, state information, behavior information, fault information and sensor data according to the component information; Step (2.2) Build a BERT-BiLSTM-CRF neural network model for entity extraction, divide the vectorized data into training set, test set and validation set as the input of the model, set the model parameters and start training, and finally save the trained model to model_path; Step (2.3) builds a BERT-BiLSTM-Attention neural network model for relationship extraction. The model extracts the context representation of the input sentence through the BERT layer, then captures the sequence dependency through the BiLSTM layer, and finally assigns weights in the Attention layer to enhance the model's attention to important features. After the data set is input, training is performed. The data set includes entity and relationship annotations. The input data is passed into the model in batches, and the predicted values ​​are obtained through forward propagation and the loss function (such as cross entropy loss) is calculated. The model parameters are adjusted using back propagation, and the order of the data set is randomly shuffled in each epoch iteration to improve generalization ability. After multiple rounds of training and verification, the model parameters that achieve the best performance are saved in the specified path model_path for subsequent model loading and prediction tasks; the present invention uses the BERT model to perform entity recognition and relationship extraction of documents related to hydraulic gate hoists, specifically including the following steps: (2.3.1) Use the pre-trained BERT model to extract features from the hydraulic gate hoist technical documentation; (2.3.2) Based on feature extraction, train a task-specific BERT-BiLSTM-CRF entity extraction model to identify key entities in the document, such as hydraulic system components, operating procedures, maintenance methods, etc. (2.3.3) Using the bidirectional encoder feature of BERT, extract the relationship between entities, including component connections, and causal relationships between operation steps and results; Step (2.4) Prediction: Load the trained BERT-BiLSTM-CRF entity extraction model and BERT-BiLSTM-Attention relationship extraction model to predict the unlabeled data, so as to extract labeled entities and relationships from a large amount of unprocessed data; Step (2.5) Evaluation: Model evaluation indicators: Load the trained entity recognition and relationship extraction models to predict the test set, calculate the accuracy, recall, F1 value and precision based on the model prediction results, and calculate the evaluation indicators of the entity recognition model and relationship extraction model respectively to measure the performance of the model.

[0029] In the knowledge graph construction module, the following processing is performed: (3.1) Knowledge fusion: Knowledge fusion is also called entity alignment. The data in the field of hydraulic gate hoist failure mainly have instance co-reference phenomenon. In the process of graph construction, co-reference elimination is required to obtain a standardized and unified description. The hydraulic gate hoist fault diagnosis knowledge graph belongs to the vertical field knowledge graph, and the data source is scarce. This graph only considers the entity fusion of three entities: engineering, equipment, and components. Figure 2 Taking "piston oil leakage" as an example, the fault entity "oil leakage" is extracted and expanded into "piston seal oil leakage" through context entity reference. A group of entities "seal oil leakage, piston rod oil leakage, oil seal oil leakage, etc." are obtained through candidate entity generation. Finally, the candidate entities are sorted and fused to be equivalent to the fault entity "seal oil leakage" in the graph.

[0030] (3.2) Knowledge storage - Neo4j graph database: There are many ways to store knowledge graphs, among which relational databases and graph databases are the most commonly used. Considering the complex entity relationships and frequent queries and updates of hydraulic gate hoists, it is more appropriate to choose a graph database for storage. The present invention selects the high-performance Neo4j graph database as the building platform and uses Cypher as the query language, which can realize the data storage, query, update, visualization and other functions involved in the hydraulic gate hoist fault knowledge graph application.

[0031] (3.3) Knowledge update: The update of the knowledge graph refers to the modification, addition or deletion of entities, relationships, attributes and other information in the graph on a regular basis or when necessary, to ensure that the knowledge graph keeps pace with changes in the real world. The update method adopted is incremental update, which uses the current new data as input to add new knowledge to the existing knowledge graph. In Neo4j, the new data is first compared with the existing data to identify new entities and relationships, and new nodes and edges are created in the graph. For existing entities and relationships, the Cypher query language can be used directly to perform the corresponding update operations. Through regular knowledge updates, the fault knowledge graph can be continuously improved, and timely, accurate and comprehensive information can also be provided to the question-answering system.

[0032] The answer generation module and the user interaction module constitute the question-answering system. The question-answering system also includes a natural language understanding module. The natural language understanding module uses natural language understanding (Natural Language Understanding): converting text information into semantic representations that can be understood by machines.

[0033] (4.1) Define question types and construct question templates based on question types, such as "How to operate XX", "What is the cause of XX's failure", and "How to maintain XX". As shown in Table 2, 8 question types are defined; Table 2. Hydraulic gate hoist fault query question template. Serial number category Question Template 1 Fault Overview Can you provide an overview of a particular fault? 2 Fault classification What type of fault is this? 3 Fault type What are the types of failures that could occur in a part or assembly? 4 sensor What are the possible failures of the sensors used in this equipment? 5 Failure Behavior What kind of failure occurred during the operation of the equipment? 6 Fault maintenance When a device fails, which parts should be repaired or replaced? 7 Failure Impact What impact will this failure cause? 8 Troubleshooting How to solve this problem?

[0034] (4.2) Collect and annotate a large number of questions, build training and test datasets, and annotate each question with its corresponding template type; (4.3) Use the BERT model to classify questions: 4.3.1) Segment and tag questions; 4.3.2) Convert the question into BERT input format, including input tokens, segment ids and attention masks; 4.3.3) Use the pre-trained BERT model, add a classification layer (fully connected layer), and output the question classification results; 4.3.4) Input question data for training and optimize the parameters of the classification layer so that the model can accurately classify questions and identify user intent.

[0035] In the answer generation module, the natural language generation module processes the results found by the knowledge graph into natural language and outputs them to the user. For example: Query result: Operation method: Start the hydraulic system, press the opening and closing button, and observe the opening and closing status. Generate answer: The operation method of the hydraulic opening and closing machine is to start the hydraulic system, press the opening and closing button, and observe the opening and closing status.

[0036] The user interaction module is the core part of the question-answering system. It is responsible for receiving questions input by users, processing questions, performing knowledge graph retrieval, and feeding back answers to users.

[0037] The question-answering system of the present invention is implemented based on the BERT model, and specifically includes the following steps: (1) Question understanding: Use the BERT model to identify the intent and extract keywords of questions entered by users to understand their query requirements.

[0038] (2) Knowledge retrieval: Based on the extracted keywords and intents, search the knowledge graph to find relevant entities and relationships.

[0039] (3) Answer generation: Combine the retrieval results and the pre-trained language generation model to generate natural language answers and provide feedback to the user.

[0040] The above-described embodiments are only preferred embodiments of the present invention. It should be noted that, for those skilled in the art, several changes and improvements may be made without departing from the overall concept of the present invention, and these should also be regarded as the protection scope of the present invention.

Claims

1. The knowledge graph construction and question-answering system of gate hydraulic hoist based on BERT model is characterized by: It includes data collection and preprocessing module, entity and relationship extraction module, knowledge graph construction module, natural language understanding module, knowledge graph retrieval module, answer generation module and user interaction module. Each module interacts with each other through API to form a complete knowledge graph construction and question-answering system. In the entity and relationship extraction module, the following processing steps are performed: Step (2.1) predefines the entity type and relationship type of the hydraulic gate hoist. The hydraulic gate hoist determines the operating equipment according to the current working condition engineering, and then determines the component information. Finally, the part information, state information, behavior information, fault information and sensor data are determined according to the component information. Step (2.2) Build a BERT-BiLSTM-CRF neural network model for entity extraction, divide the vectorized data into training set, test set and validation set as the input of the model, set the model parameters and start training, and finally save the trained model to model_path; Step (2.3) Build a BERT-BiLSTM-Attention neural network model for relationship extraction. The model extracts the context representation of the input sentence through the BERT layer, captures the sequence dependency through the BiLSTM layer, and finally assigns weights in the Attention layer to enhance the model's attention to important features. Training is performed after inputting a data set, which includes entity and relationship annotations. The input data is passed into the model in batches, and the prediction value is obtained and the loss function is calculated through forward propagation. The model parameters are adjusted using backpropagation, and the order of the data set is randomly shuffled in each iteration to improve generalization ability. After multiple rounds of training and verification, the model parameters that achieve the best performance are saved in the specified path model_path for subsequent model loading and prediction tasks.

2. According to the BERT model-based knowledge graph construction and question-answering system for gate hydraulic hoist according to claim 1, it is characterized by: In the data collection and preprocessing module, the data is processed as follows: Step (1.1) collect text data related to the hydraulic gate hoist; Step (1.2) converts the text data into a unified txt format; Step (1.3) Use Python to clean the data: Use the Jieba word segmentation tool to segment the txt text, remove stop words, and remove special characters and punctuation marks; Step (1.4) Define entity labels and use the sequence labeling tool Label Studio to label the cleaned text with BIO sequences; Step (1.5) Text vectorization: Use the Word2Vec tool of the Gensim library in Python to convert text data into numerical vectors.

3. The gate hydraulic hoist knowledge graph construction and question-answering system based on the BERT model according to claim 1 is characterized in that: The BERT model is used to perform entity recognition and relationship extraction of documents related to hydraulic gate hoists, which specifically includes the following steps: (2.3.1) Use the pre-trained BERT model to extract features from the hydraulic gate hoist technical documentation; (2.3.2) Based on feature extraction, train a task-specific BERT-BiLSTM-CRF entity extraction model to identify key entities in the document; (2.3.3) Use the bidirectional encoder feature of BERT to extract the relationship between entities; Step (2.4) Prediction: Load the trained BERT-BiLSTM-CRF entity extraction model and BERT-BiLSTM-Attention relationship extraction model to predict the unlabeled data, extracting labeled entities and relationships from a large amount of unprocessed data. This step first extracts key entities from the text using the entity recognition model, and then uses the relationship extraction model to identify the potential relationships between entities. Step (2.5) Evaluation: Model evaluation indicators: Load the trained entity recognition and relationship extraction models to predict the test set, calculate the accuracy, recall, F1 value and precision based on the model prediction results, and calculate the evaluation indicators of the entity recognition model and relationship extraction model respectively to measure the performance of the model.

4. The gate hydraulic hoist knowledge graph construction and question-answering system based on the BERT model according to claim 1 is characterized in that: In the knowledge graph construction module, the following processing is performed: (3.1) Knowledge fusion: The hydraulic gate hoist fault diagnosis knowledge graph belongs to the vertical domain knowledge graph, which integrates the three entities of engineering, equipment, and components; (3.2) Knowledge storage - Neo4j graph database: Neo4j graph database is selected as the construction platform, and Cypher is used as the query language to realize the data storage, query, update, and visualization functions involved in the hydraulic hoist fault knowledge graph application; (3.3) Knowledge update: In Neo4j, we first compare the newly added data with the existing data to identify new entities and relationships, and then create new nodes and edges in the graph. For existing entities and relationships, we directly use the Cypher query language to perform corresponding update operations. Through regular knowledge updates, we continuously improve the fault knowledge graph, and also provide timely, accurate, and comprehensive information to the question-answering system.

5. The gate hydraulic hoist knowledge graph construction and question-answering system based on the BERT model according to claim 1 is characterized in that: The answer generation module and the user interaction module constitute the question-answering system. The question-answering system also includes a natural language understanding module. The natural language understanding module uses natural language understanding to convert text information into semantic representations that can be understood by machines. The process is as follows: (4.1) Define question types and construct question templates based on question types; (4.2) Collect and annotate a large number of questions, build training and test datasets, and annotate each question with its corresponding template type; (4.3) Use the BERT model to classify questions: 4.3.1) Segment and tag questions; 4.3.2) Convert the question sentence into BERT input format; 4.3.3) Use the pre-trained BERT model, add a classification layer, and output the question classification results; 4.3.4) Input question data for training and optimize the parameters of the classification layer so that the model can accurately classify questions and identify user intent.

6. The gate hydraulic hoist knowledge graph construction and question-answering system based on the BERT model according to claim 1 is characterized by: In the answer generation module, the natural language generation module processes the results of the knowledge graph query into natural language and outputs it to the user; The user interaction module is responsible for receiving questions input by users, processing questions, performing knowledge graph retrieval, and feeding back answers to users.

Images