Knowledge question-answering system based on middle table operation manual and implementation and use method of knowledge question-answering system

Abstract

The invention relates to a knowledge question-answering system based on a middle table operation manual and an implementation and use method thereof, and belongs to the technical field of computer technology, data management and knowledge management. The system designed by the invention comprises five parts, namely data access and data analysis, entity definition, entity relationship definition and entity attribute definition, graph construction, question and answer system construction and user feedback. According to the method, an optimal and most efficient data processing mode is designed for current questions and customer requirements, operation manual data are integrated, and efficient operation manual content retrieval and intelligent question answering are realized for the user questions by combining natural language processing (NLP) and a knowledge graph technology, so that timely and accurate operation process guidance is provided for the user, and the user experience is improved. The device can be used for assisting a middle table operator to quickly master, and the training time and training cost are reduced.

Description

Technical Field

[0001] The present invention belongs to the fields of computer technology, data management and knowledge management technology, and specifically relates to a knowledge question and answer system based on a middle platform operation manual and its implementation and use method. Background Art

[0002] At present, the data middle platform has multiple locations, multiple versions, and different customization requirements. In response to the diverse needs of different customers, there are many inconsistent versions of the functions of the middle platform. The functions are complex, the operation and learning are difficult, and the learning cost is high. In response to customer deliverable requirements, an operation manual is provided to customers. The current question-and-answer system logic is simple query and basic search. The content that is similar cannot obtain accurate search content due to title changes. This leads to low answer efficiency and difficulty in data updating. It is impossible to effectively manage the content of multiple versions of operation manuals. This decentralized management method lacks the integration of the content of the operation manual, and cannot achieve efficient content retrieval and intelligent question-and-answer. Summary of the invention

[0003] 1. Technical issues to be resolved

[0004] The technical problem to be solved by the present invention is to propose a knowledge question-answering system and its implementation and use method, so as to realize efficient operation manual content retrieval and intelligent question-answering for user questions.

[0005] (II) Technical solution

[0006] In order to solve the above technical problems, the present invention provides a knowledge question answering system based on the middle platform operation manual, the system comprising:

[0007] Data access module: realizes automatic import and preprocessing of multi-version operation manual data. The data source includes operation manuals of different formats and sources, and converts the data into a structured format;

[0008] Knowledge graph construction module: Generates a knowledge graph based on the operation manual content, automatically extracts key entities and establishes associations to achieve unified management and display of multi-version content;

[0009] Natural language processing module: responsible for identifying the intent of query parsing questions input by users, including entity extraction and context understanding;

[0010] Question and answer engine: Provides intelligent question and answer services based on the processing results of the knowledge graph and natural language processing module, maps user questions processed by the natural language processing module to relevant information in the operation manual, and provides feedback on operation steps or specific answers.

[0011] The invention also provides a method for realizing and using the system.

[0012] (III) Beneficial effects

[0013] The system designed by the present invention includes five parts: data access and data analysis, entity definition, entity relationship definition and entity attribute definition, graph construction, question-answering system construction, and user feedback. The present invention designs the best and most efficient data processing method for current problems and customer requirements, integrates operation manual data, combines natural language processing (NLP) and knowledge graph technology, realizes efficient operation manual content retrieval and intelligent question-answering for user questions, and provides users with timely and accurate operation process guidance, which can be used to assist middle-office operators to quickly get started and reduce training time and training costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a system architecture diagram of the present invention. DETAILED DESCRIPTION

[0015] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the drawings and examples.

[0016] refer to Figure 1 The overall architecture of a knowledge question answering system based on the middle platform operation manual provided by the present invention includes the following modules:

[0017] Data access module: supports automatic import and preprocessing of multi-version operation manual data. The data sources include operation manuals of different formats and sources (PDF, Word, etc.), and converts the data into a structured format.

[0018] Knowledge graph construction module: Generates a knowledge graph based on the content of the operation manual, automatically extracts key entities (such as operation steps, parameters, precautions) and establishes associations to achieve unified management and display of multi-version content.

[0019] Natural language processing module: responsible for identifying the intent of query parsing questions input by users, including entity extraction and context understanding, to ensure efficient matching and accurate answers to user questions.

[0020] Question and answer engine: Provides intelligent question and answer services based on the processing results of the knowledge graph and natural language processing module, maps user questions processed by the natural language processing module to relevant information in the operation manual, and provides feedback on operation steps or specific answers.

[0021] User feedback module: supports user feedback so that the system can continuously optimize the accuracy of answers, and records common questions in the knowledge base to improve the system's question and answer quality.

[0022] The present invention also proposes a method for implementing and using the knowledge question answering system, which comprises the following steps:

[0023] S1. Define the module body

[0024] Defining the module ontology is a key step in generating the knowledge graph. Based on the actual application scenario of operation question answering, and referring to various versions of the existing middle platform, 10 entities, 4 entity relationships, and 8 entity attributes are defined as components of the module ontology;

[0025] 10 entity types: data lake management (datasource), data synchronization (data_synchronization), metadata (metadata), global search (elastic_search), unstructured data (file_manage), resource directory (resource), data governance (data_governance), shared exchange (share_exchange), data service (interface_manage), system management (system_control), their definitions are shown in Table 1:

[0026] Table 1 Entity type definition

[0027]

[0028]

[0029] There are four types of entity relationships: data lake management and module relationships (datasource_part), data synchronization and metadata, global search, resource directory, data governance, and sharing and exchange relationships (etl_tools), metadata and data governance relationships (metadata_part), and system management and module relationships (role_manage). See Table 2 for their definitions:

[0030] Table 2 Entity relationship definition

[0031]

[0032] 8 entity attributes: module location, operation process, current problems, key operation steps, data type, operation preparation, service, and technical person in charge. Their definitions are shown in Table 3:

[0033] Table 3: Entity attribute definition

[0034]

[0035] S2. Design a data access module for data collection: Bring out the deliverables of each project through the formal process and desensitize sensitive information. Since there are general templates in the operation manual, a fixed format program can be designed to extract data from the corresponding module content. Some content is dirty data caused by non-standard writing by personnel, which needs further processing. Regular expressions are used to remove useless tags, special characters or useless information. Finally, the data is converted into a structured format and stored in the Dream database to prepare for the construction of the knowledge graph.

[0036] S3. Design a knowledge graph construction module to build a knowledge graph and realize knowledge graph storage visualization: Determine the entity relationship, entity type and entity attribute of the structured data stored in the DAMO database, and combine the three types of data in the form of triples. Convert the data in the DAMO database into json format through a program and finally store it in a graph database based on Neo4j.

[0037] S4. Build a knowledge question-and-answer system based on the middle-end operation manual: The question-and-answer system mainly analyzes the questions input by users, retrieves answers from the constructed medical knowledge graph, and feeds back to users. The processing flow mainly includes user input questions, word segmentation, intent recognition, question parsing, Neo4j-based graph database query, and output answers. First, perform entity extraction and question classification on the questions input by users, convert the extracted results into cypher statements, and then use the converted cypher statements to query the results from the Neo4j-based graph database, and return the results to the user.

[0038] S5. Intent recognition: The natural language processing module receives questions raised by users and performs entity extraction to identify user intent. Intent recognition involves the following algorithms:

[0039] The text representation method TF-IDF is used to evaluate the importance of a word to a document set or one of the documents in a corpus.

[0040] The AC algorithm uses Trie tree and fail pointer for multi-mode string matching, which reduces a lot of time complexity compared to traditional algorithms. Among them, after the question is input, the AC automaton is first used to match the question, find all the domain feature words, filter the matched feature words, remove those repeated short words, and only keep the long words; it allows the pattern set of the AC automaton to be dynamically added or updated without rebuilding the entire Trie tree; dynamically insert new patterns when the AC algorithm is running, and cache the frequently queried patterns by partially adjusting the Trie tree and updating the fail pointer to improve the matching efficiency; when users frequently input specific questions (such as kettle operation related questions), the system can learn and add new patterns in real time; further, give weights or priorities to different patterns, and dynamically adjust the matching order according to the context; add weight fields when building patterns, and adjust the priority according to the context (user history, intent classification) during the matching process; when multiple patterns in a question are matched at the same time, more relevant answers are returned first; based on the matched (recognized) feature words, the specific type of the question is determined by further combining predefined question words (such as module operation process, how to fill in the link, what components are needed, etc.).

[0041] S6. Question and answer engine performs question parsing: Question parsing is to write the corresponding structured query statement template according to the question type determined after the natural language processing module's intention recognition and search it in the knowledge graph. Use Cypher statements to query the Neo4j-based graph database. Cypher is a declarative query language for querying the graph database Neo4j. Use the word segmentation tool to segment the question, and mark the part of speech (noun, verb, etc.) for each word, and perform named entity recognition to identify the entities in the question.

[0042] S7. The user feedback module provides users with feedback options for each answer. Customers can choose to feedback whether the answer is correct or wrong. And they can fill in the correct operation content for the wrong content. The data will be fed back to the system and stored in the DAMO database to facilitate regular maintenance and updates of the question-answering system.

[0043] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A knowledge question-answering system based on the middle platform operation manual, characterized in that: The system includes: Data access module: realizes automatic import and preprocessing of multi-version operation manual data. The data source includes operation manuals of different formats and sources, and converts the data into a structured format; Knowledge graph construction module: Generates a knowledge graph based on the operation manual content, automatically extracts key entities and establishes associations to achieve unified management and display of multi-version content; Natural language processing module: responsible for identifying the intent of query parsing questions input by users, including entity extraction and context understanding; Question and answer engine: Provides intelligent question and answer services based on the processing results of the knowledge graph and natural language processing module, maps user questions processed by the natural language processing module to relevant information in the operation manual, and provides feedback on operation steps or specific answers.

2. The method according to claim 1, characterized in that The system also includes a user feedback module: optimizing the accuracy of the feedback operation steps or specific answers, and recording preset questions in the knowledge base.

3. A method for implementing and using the system as claimed in claim 1 or 2.

4. The method according to claim 3, characterized in that The method comprises the following steps: S1. Define the module body Defining the module ontology is a key step in generating the knowledge graph. Based on the actual application scenario of operation question answering, and referring to various versions of the existing middle platform, 10 entities, 4 entity relationships, and 8 entity attributes are defined as components of the module ontology; The 10 entity types include: data lake management, data synchronization, metadata, global search, unstructured data, resource directory, data governance, sharing and exchange, data services, and system management; 4 types of entity relationships: the relationship between data lake management and entities, the relationship between data synchronization and metadata, global search, resource catalog, data governance, and sharing and exchange, the relationship between metadata and data governance, and the relationship between system management and entities; 8 entity attributes: module location, operation process, current problems, key operation steps, data type, operation preparation, service, and technical person in charge; S2. Design data access module for data collection: Design a fixed format program to extract data from the corresponding module content, further process some content containing dirty data, use regular expressions to remove useless tags, special characters, and useless information, and finally convert the data into a structured format and store it in the Dream database to prepare for the construction of the knowledge graph; S3. Design a knowledge graph construction module to build a knowledge graph and realize knowledge graph storage visualization: determine the entity relationship, entity type and entity attribute of the structured data stored in the DAMO database, and combine the three types of data in the form of triples. Convert the data in the DAMO database into json format through a program, and finally store it in a graph database based on Neo4j. S4. Form a knowledge question-answering system based on the middle-end operation manual. The knowledge question-answering system can analyze the questions input by users, retrieve answers from the constructed knowledge graph and feedback to users. The processing flow includes user input of questions, word segmentation, intent recognition, question parsing, Neo4j-based graph database query and answer output. S5, Intent recognition: The natural language processing module receives the question raised by the user and performs entity extraction to identify the user's intent; S6. Question-answering engine performs question parsing: Question parsing is to write a corresponding structured query statement template based on the question type determined after the natural language processing module's intent recognition and search the knowledge graph; use Cypher statements to query the Neo4j-based graph database, use the word segmentation tool to segment the questions, mark the part of speech for each word, and perform named entity recognition to identify the entities in the questions.

5. The method according to claim 4, characterized in that It also includes step S5, for each answer, giving the user feedback options, the customer chooses to feedback the correctness or errors of the answer content, and fills in the correct operation content for the incorrect content, and stores the feedback data in the DAMO database, and regularly maintains and updates the system.

6. The method according to claim 4, characterized in that The text representation method TF-IDF is used in the intent recognition process to evaluate the importance of a word to a document set or one of the documents in a corpus.

7. The method according to claim 4, characterized in that The Trie tree and fail pointer in the AC algorithm are used for multi-mode string matching during the intent recognition process. After a question is input, the AC automaton is first used to match the question to find all the domain feature words, and the matched feature words are filtered to remove repeated short words and retain only long words. The pattern set of the AC automaton can be dynamically added or updated without rebuilding the entire Trie tree. New patterns are dynamically inserted when the AC algorithm is running, and frequently queried patterns are cached by locally adjusting the Trie tree and updating the fail pointer. When users frequently input specific questions, new patterns are learned and added in real time. Furthermore, weights or priorities are assigned to different patterns, and the matching order is dynamically adjusted according to the context. A weight field is added when constructing the pattern, and the priority is adjusted according to the context during the matching process. When multiple patterns in a question are matched at the same time, more relevant answers are returned first. The specific type of the question is determined based on the matched feature words and further combined with the predefined question words.

8. The method according to claim 4, characterized in that The 10 types of entity definitions are shown in Table 1: Table 1 Entity type definition 9. The method according to claim 4, characterized in that The definitions of the four entity relationships are shown in Table 2: Table 2 Entity relationship definition 10. The method according to claim 4, characterized in that The definitions of the eight entity attributes are shown in Table 3: Table 3: Entity attribute definition

Images