Verification method and device based on graph structure data, electronic equipment and storage medium

Abstract

The application provides a verification method and device based on graph structure data, electronic equipment and a storage medium, and the method comprises the following steps: based on the graph structure data, for any two target words in the answer text, if the path between the two target words in the graph structure data is less than or equal to the first shortest search path, the relationship between the two target words is constructed, and a regression entity relationship network is obtained; the target substantive word, the target attribute word and the answer text are input into a semantic extraction model to obtain an intra-text entity relationship network; the regression entity relationship network and the intra-text entity relationship network are compared to determine the matching degree between the answer text and the graph structure data; the matching degree between the answer text and the preset answer is determined; and based on the matching degree between the answer text and the graph structure data and the matching degree between the answer text and the preset answer, the comprehensive ability of the question and answer system in processing the preset question is determined. Through the method, the accuracy of evaluating the ability of the intelligent question and answer system to answer questions is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of graph structure data, and particularly to a verification method and device based on graph structure data, an electronic device and a storage medium. BACKGROUND

[0002] An intelligent question answering system is a system that uses artificial intelligence technologies, particularly natural language processing (NLP) and machine learning, to understand, interpret and provide accurate answers to questions posed by users. Generally, by analyzing natural language text input by users, the system understands the semantics and intent of the input and retrieves, generates or infers the most suitable answer from a large amount of data. The development of intelligent question answering systems aims to simulate the ability of humans to understand and answer questions, in order to provide a more intuitive and effective user interaction experience.

[0003] At present, before an intelligent question answering system is put into use, the system usually needs to be verified to evaluate the ability of the system to answer questions, that is, to evaluate the effectiveness of the answers provided by the system. Therefore, there is an urgent need in the prior art for a method capable of evaluating the ability of an intelligent question answering system to answer questions. SUMMARY

[0004] In view of this, the present application aims to provide a verification method and device based on graph structure data, an electronic device and a storage medium, in order to improve the accuracy of evaluating the ability of an intelligent question answering system to answer questions.

[0005] In a first aspect, the embodiments of the present application provide a verification method based on graph structure data, which is used to verify a to-be-verified intelligent question answering system, the to-be-verified intelligent question answering system being configured to generate an answer text for a preset question according to the input of the preset question; the method comprising:

[0006] extracting target substantive words and target attribute words of each target substantive word from the answer text based on graph structure data; wherein the graph structure data contains knowledge information of multiple fields; the target substantive words are substantive words common to the graph structure data and the answer text; the target attribute words are attribute words common to the graph structure data and the answer text;

[0007] For any two target words, if the path between the two target words in the graph structure data is less than or equal to a preset first shortest search path, a relationship between the two target words is constructed based on the first shortest search path, so as to construct a regression entity relationship network according to the relationship between each two target words; wherein the any two target words include any two target substantive words, any one target substantive word and any one target attribute word;

[0008] inputting each of the target substantive vocabulary and the target attribute vocabulary of each of the target substantive vocabulary and the answer text into a semantic extraction model to obtain an intra-text entity relationship network containing a relationship between any two target vocabularies;

[0009] comparing the regression entity relationship network and the intra-text entity relationship network to determine a matching degree between the answer text and the graph structure data;

[0010] obtaining a preset answer to the preset question and determining a matching degree between the answer text and the preset answer;

[0011] determining a comprehensive capability of the question and answer system in processing the preset question based on the matching degree between the answer text and the graph structure data and the matching degree between the answer text and the preset answer.

[0012] With reference to the first aspect, in a first possible implementation manner of the first aspect, the extracting, based on the graph structure data, the target substantive vocabulary and the target attribute vocabulary of each of the target substantive vocabulary from the answer text comprises:

[0013] performing word segmentation processing on the answer text to obtain a vocabulary list and removing meaningless vocabularies in the vocabulary list to obtain a substantive vocabulary list;

[0014] querying whether each substantive vocabulary in the substantive vocabulary list is contained in a graph structure data entity list, and when the graph structure data entity list contains the substantive vocabulary, saving the substantive vocabulary as a target substantive vocabulary to a text entity list; wherein the graph structure data entity list contains each entity in the graph structure data;

[0015] searching, for each of the target substantive vocabulary in the text entity list, a target attribute vocabulary of the target substantive vocabulary in the graph structure data, and if the target attribute vocabulary appears in the vocabulary list, storing the target substantive vocabulary and the target attribute vocabulary to a text entity dictionary.

[0016] With reference to the first aspect, in a second possible implementation manner of the first aspect, the comparing the regression entity relationship network and the intra-text entity relationship network to determine the matching degree between the answer text and the graph structure data comprises:

[0017] counting a first total number of first target substantive vocabularies and first relationships contained in the intra-text entity relationship network; wherein the first target substantive vocabulary is a target substantive vocabulary that does not appear in the regression entity relationship network; and the first relationship is a relationship that does not appear in the regression entity relationship network.

[0018] For a relationship between any two target words contained in the text entity relationship net, if the relationship between the two target words in the text entity relationship net is different from the relationship between the two target words in the regression entity relationship net, the relationship between the two target words in the text entity relationship net is determined as a matching error relationship, the probability of the matching error relationship in the text entity relationship net is counted, and a regression error rate is obtained.

[0019] With reference to the second possible implementation manner of the first aspect, a third possible implementation manner of the first aspect is provided in the embodiments of the present application, and the determining the matching degree between the answer text and the preset answer for the preset question comprises:

[0020] The preset answer for the preset question is obtained, and according to any two preset keywords in the preset answer, a preset second shortest search path is used to construct a relationship between the two preset keywords if a path between the two preset keywords in the graph structure data is less than or equal to the second shortest search path, so as to construct a preset keyword relationship net according to the relationship between each two preset keywords; wherein the preset keywords include mandatory keywords, additional keywords and over-authorized keywords.

[0021] The relationship is used to construct the preset keyword relationship net, and the text entity relationship net is compared to determine the matching degree between the answer text and the preset answer.

[0022] With reference to the third possible implementation manner of the first aspect, a fourth possible implementation manner of the first aspect is provided in the embodiments of the present application, and the determining the matching degree between the answer text and the preset answer for the preset question comprises:

[0023] The coverage rate of the answer text to the preset keywords is counted to obtain a keyword recall rate.

[0024] According to any two mandatory keywords in the preset answer, a preset third shortest search path is used to construct a relationship between the two mandatory keywords if a path between the two mandatory keywords in the graph structure data is less than or equal to the third shortest search path, so as to construct a mandatory keyword relationship net according to the relationship between each two mandatory keywords.

[0025] The first matching degree between the mandatory keyword relationship net and the text entity relationship net is calculated.

[0026] According to any two additional keywords in the preset answer, a third shortest search path is preset, and if a path between the two additional keywords in the graph structure data is less than or equal to the third shortest search path, a relationship between the two additional keywords is constructed, so as to construct an additional keyword relationship network according to the relationship between each two additional keywords;

[0027] A second matching degree between the additional keyword relationship network and the entity relationship network in the text is calculated;

[0028] According to any two unauthorized keywords in the preset answer, a third shortest search path is preset, and if a path between the two unauthorized keywords in the graph structure data is less than or equal to the third shortest search path, a relationship between the two unauthorized keywords is constructed, so as to construct an unauthorized keyword relationship network according to the relationship between each two unauthorized keywords;

[0029] A third matching degree between the unauthorized keyword relationship network and the entity relationship network in the text is calculated.

[0030] With reference to the fourth possible implementation manner of the first aspect, the fourth possible implementation manner of the first aspect is provided, and the comprehensive ability of the question-answering system in processing the preset question is determined based on the matching degree between the answer text and the graph structure data and the matching degree between the answer text and the preset answer.

[0031] According to the keyword recall rate, the first matching degree, the total number of entities and the total number of relationships included in the answer text, and the total number of logics between the plurality of relationships included in the answer text, a coverage degree of the answer text on the preset question is calculated.

[0032] According to the first total number, the regression error rate, and the total number of entities and the total number of relationships included in the answer text, a reliability of the answer text is calculated.

[0033] A ratio of the first matching degree to the total number of necessary keyword logics is determined as a logic coverage rate of the answer text.

[0034] A ratio of the second matching degree to a preset upper limit number of the additional keywords is determined as an additional score amount of the answer text.

[0035] According to the third matching degree, the total number of entities and the total number of relationships included in the answer text, an unauthorized keyword amount score of the answer text is calculated.

[0036] calculating a ratio of the first total number and a total number of entities and a total number of relations contained in the answer text, determining a difference between 1 and the ratio as an illusion amount score of the answer text;

[0037] using the coverage of the answer text on the preset question, the logical coverage rate of the answer text, the additional score amount, the out-of-authority word amount score and the illusion amount score, evaluating a comprehensive ability of the question and answer system in processing the preset question.

[0038] In a second aspect, the embodiments of the present application also provide a verification device based on graph structure data, which is used to verify a to-be-verified intelligent question and answer system, and the to-be-verified intelligent question and answer system is used to generate an answer text for a preset question according to the input of the preset question. The device comprises:

[0039] The extraction module is used to extract target substantive words and target attribute words of each target substantive word from the answer text based on graph structure data, wherein the graph structure data contains knowledge information of multiple fields; the target substantive words are substantive words common to the graph structure data and the answer text; and the target attribute words are attribute words common to the graph structure data and the answer text.

[0040] The construction module is used to construct a relationship between any two target words based on a preset first shortest search path, if a path between the two target words in the graph structure data is less than or equal to the first shortest search path, to construct a regression entity relationship network according to the relationship between each two target words, wherein the any two target words include any two target substantive words, any one target substantive word and any one target attribute word.

[0041] The input module is used to input the target substantive words, the target attribute words of each target substantive word and the answer text into a semantic extraction model to obtain an intra-text entity relationship network containing the relationship between any two target words.

[0042] The comparison module is used to compare the regression entity relationship network and the intra-text entity relationship network to determine a matching degree between the answer text and the graph structure data.

[0043] The first determination module is used to obtain a preset answer for the preset question, and determine a matching degree between the answer text and the preset answer.

[0044] The second determining module is used to determine the comprehensive ability of the question-answering system in processing the preset question based on the degree of matching between the answer text and the graph structure data, and the degree of matching with the preset answer.

[0045] In conjunction with the second aspect, this application provides a first possible implementation of the second aspect, wherein the extraction module, when used to extract target semantic words and target attribute words of each target semantic word from the answer text based on graph structure data, is specifically used for:

[0046] The answer text is segmented to obtain a vocabulary list, and meaningless words are removed from the vocabulary list to obtain a semantic vocabulary list.

[0047] For each semantic word in the semantic vocabulary, the graph structure data entity list is queried to see if the semantic word is contained therein. If the graph structure data entity list contains the semantic word, the semantic word is saved as the target semantic word to the text entity list; wherein, the graph structure data entity list contains each entity in the graph structure data.

[0048] For each target semantic word in the text entity list, search for the target attribute word of the target semantic word in the graph structure data. If the target attribute word appears in the word list, store the target semantic word and the target attribute word in the text entity dictionary.

[0049] In conjunction with the second aspect, this application provides a second possible implementation of the second aspect, wherein the comparison module, when comparing the regressed entity relationship network with the textual entity relationship network to determine the degree of matching between the response text and the graph structure data, is specifically used for:

[0050] The first total number of first target content words and first relations contained in the entity relationship network within the text is counted; wherein, the first target content words are target content words that do not appear in the regressed entity relationship network; and the first relation is a relation that does not appear in the regressed entity relationship network.

[0051] For any two target words contained in the text-based entity relationship network, if the relationship between the two target words in the text-based entity relationship network is different from the relationship between the two target words in the regression entity relationship network, then the relationship between the two target words in the text-based entity relationship network is determined as a mismatched relationship, and the probability of mismatched relationships in the text-based entity relationship network is calculated to obtain the regression error rate.

[0052] In conjunction with the second possible implementation of the second aspect, this application provides a third possible implementation of the second aspect, wherein the first determining module, when used to obtain a preset answer to the preset question and determine the degree of matching between the answer text and the preset answer, is specifically used for:

[0053] Obtain a preset answer to the preset question. Based on any two preset keywords in the preset answer, and using a preset second shortest search path, if the path between the two preset keywords in the graph structure data is less than or equal to the second shortest search path, then construct a relationship between the two preset keywords to build a preset keyword relationship network based on the relationship between each pair of preset keywords. The preset keywords include required keywords, additional keywords, and overriding keywords.

[0054] A preset keyword relationship network is constructed for the relationship and compared with the entity relationship network in the text to determine the degree of matching between the answer text and the preset answer.

[0055] In conjunction with the third possible implementation of the second aspect, this application provides a fourth possible implementation of the second aspect, wherein the first determining module, when comparing the preset keyword relationship network constructed by the relationship with the in-text entity relationship network to determine the degree of matching between the answer text and the preset answer, is specifically used for:

[0056] The keyword recall rate is obtained by statistically analyzing the coverage rate of the answer text to the preset keywords.

[0057] Based on any two required keywords in the preset answer, and based on the preset third shortest search path, if the path between the two required keywords in the graph structure data is less than or equal to the third shortest search path, then the relationship between the two required keywords is constructed, so as to construct a required keyword relationship network based on the relationship between each pair of required keywords.

[0058] Calculate the first matching degree between the required keyword relationship network and the in-text entity relationship network;

[0059] Based on any two additional keywords in the preset answer, and based on the preset third shortest search path, if the path between the two additional keywords in the graph structure data is less than or equal to the third shortest search path, then the relationship between the two additional keywords is constructed, so as to construct an additional keyword relationship network based on the relationship between each pair of additional keywords.

[0060] Calculate the second matching degree between the additional keyword relationship network and the text-based entity relationship network;

[0061] Based on any two overriding keywords in the preset answer, and based on the preset third shortest search path, if the path between the two overriding keywords in the graph structure data is less than or equal to the third shortest search path, then the relationship between the two overriding keywords is constructed, so as to construct an overriding keyword relationship network based on the relationship between each pair of overriding keywords.

[0062] Calculate the third matching degree between the overriding keyword relationship network and the text-based entity relationship network.

[0063] In conjunction with the fourth possible implementation of the second aspect, this application provides a fifth possible implementation of the second aspect, wherein the second determining module, when determining the comprehensive ability of the question-answering system in processing the preset question based on the degree of matching between the answer text and the graph structure data, and the degree of matching with the preset answer, is specifically used for:

[0064] The coverage of the preset question by the answer text is calculated based on the keyword recall rate, the first matching degree, the total number of entities and relations contained in the answer text, and the total number of logical relationships between multiple relations contained in the answer text.

[0065] The reliability of the response text is calculated based on the first total number, the regression error rate, and the total number of entities and relations contained in the response text.

[0066] The ratio of the first matching degree to the total number of necessary keyword logics is determined as the logical coverage rate of the answer text;

[0067] The ratio of the second matching degree to the preset upper limit of the number of additional keywords is determined as the additional component of the answer text;

[0068] The score for the number of out-of-weight words in the answer text is calculated based on the third matching degree, the total number of entities and the total number of relations contained in the answer text.

[0069] Calculate the ratio of the first total number to the total number of entities and the total number of relations contained in the answer text, and determine the difference between 1 and the ratio as the illusion quantity score of the answer text;

[0070] The comprehensive ability of the question-answering system in processing the preset question is evaluated using the coverage of the answer text to the preset question, the logical coverage of the answer text, the additional component, the score of the number of overweight words, and the score of the number of illusions.

[0071] Thirdly, embodiments of this application also provide an electronic device, including: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and when the machine-readable instructions are executed by the processor, the steps in any of the possible implementations of the first aspect described above are performed.

[0072] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps in any of the possible implementations of the first aspect described above.

[0073] This application provides a verification method based on graph structure data. Graph structure data typically records relatively accurate knowledge. Therefore, in this embodiment, by combining graph structure data, a regression entity relationship network containing the relationships between target words in the answer text is constructed. The answer text is then directly input into a semantic extraction model to obtain an intratext entity relationship network containing the relationships between target words in the answer text. The regression entity relationship network and the intratext entity relationship network are then compared to determine the degree of matching between the answer text and the graph structure data. Furthermore, the degree of matching between the answer text and a preset answer is determined. Finally, the comprehensive ability of the question-answering system to process preset questions is determined by combining the degree of matching between the answer text and the graph structure data, and the degree of matching between the answer text and the preset answer. In this embodiment, using graph structure data to verify the comprehensive ability of the intelligent question-answering system to process preset questions helps improve the accuracy of the verification.

[0074] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0075] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0076] Figure 1 A flowchart of a verification method based on graph structure data provided in an embodiment of this application is shown;

[0077] Figure 2 This illustration shows a schematic diagram of some of the graph structure data provided in an embodiment of this application;

[0078] Figure 3This illustration shows a schematic diagram of a verification device based on graph structure data provided in an embodiment of this application;

[0079] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0080] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0081] Graph-structured data is a complex non-linear data structure consisting of a set of nodes (Vertex) and a set of edges (E), denoted as G = (V, E). Graph-structured data can be classified in several ways: according to whether the edges have direction, they can be divided into undirected graphs and directed graphs; according to whether the edges have weights, they can be divided into unweighted graphs and weighted graphs.

[0082] Intelligent question-answering systems are systems that utilize artificial intelligence technologies, particularly natural language processing (NLP) and machine learning, to understand and interpret user-generated questions and provide accurate answers. They typically analyze user-input natural language text to understand its semantics and intent, and retrieve, generate, or infer the most suitable answer from large amounts of data. The development of intelligent question-answering systems aims to mimic human abilities to understand and answer questions, providing a more intuitive and effective user interaction experience.

[0083] Currently, before an intelligent question-answering system is put into use, it usually needs to be validated to assess its ability to answer questions, that is, to evaluate the effectiveness of the answers provided by the intelligent question-answering system.

[0084] In existing technologies, the following methods are used to evaluate the ability of intelligent question-answering systems to answer questions:

[0085] 1. Manual Verification: This involves professionals verifying the accuracy of the responses output by the intelligent question-answering system. This method requires professionals to undergo prior training, resulting in high time and financial costs for verification.

[0086] 2. Automated Verification of Preset Answers: This method compares the text of the answer output by the intelligent question-answering system with preset answers to check their consistency. However, because the completeness of the preset answers cannot be guaranteed, the completeness of the text output by the intelligent question-answering system cannot be verified. For example, if the question input to the intelligent question-answering system is "Who is Zhang San?", the preset answer might be "Zhang San is the president of S University." This preset answer is actually incomplete, thus affecting the verification of the completeness of the text output by the intelligent question-answering system.

[0087] 3. User Feedback Analysis: Users assess the accuracy of the responses output by the intelligent question-and-answer system. However, this method is susceptible to subjective bias due to differing user evaluation criteria, leading to lower accuracy.

[0088] In view of the above problems, this application provides a verification method, apparatus, electronic device, and storage medium based on graph structure data. In the verification method of this embodiment, no manual verification by professionals is required, nor is judgment based on user feedback necessary. This automated verification method improves the accuracy of the intelligent question-answering system's evaluation while reducing labor costs. Furthermore, this embodiment not only compares the answer text output by the intelligent question-answering system with preset answers but also incorporates graph structure data, specifically knowledge graphs. Since graph structure data contains a complete knowledge system, comparing the relationships between target words in the graph structure data and the relationships between target words in the answer text further verifies the completeness and accuracy of the answer text, thereby improving the accuracy of the verification. The following is a description through embodiments.

[0089] To facilitate understanding of this embodiment, a verification method based on graph structure data disclosed in this application will first be described in detail. This verification method based on graph structure data is used to verify an intelligent question-answering system to be verified, which generates an answer text for a preset question based on an input preset question.

[0090] In this embodiment, the intelligent question-answering system is a question-answering system based on artificial intelligence technology. Through technologies such as natural language processing and machine learning, it can quickly and accurately answer questions raised by users.

[0091] In practical applications, users can input their questions into the intelligent question-answering system. The system then extracts information from a large amount of data based on the user's input, analyzes and processes it, and outputs the answer text to the user, thus quickly and accurately answering the user's questions.

[0092] Generally, before the intelligent question - answering system is put into use, it needs to be verified and evaluated. Only after passing the verification can the intelligent question - answering system be put into use.

[0093] As Figure 1 shown, when verifying the intelligent question - answering system to be verified, the following steps S101 - S106 are included:

[0094] S101: Based on the graph - structured data, extract the target content words and the target attribute words of each target content word from the answer text; among them, the graph - structured data contains knowledge information in multiple fields; the target content words are the common content words between the graph - structured data and the answer text; the target attribute words are the common attribute words between the graph - structured data and the answer text.

[0095] In this embodiment, the graph - structured data contains multiple nodes and multiple edges. The edges in the graph - structured data are used to connect two related nodes, that is, the edges are used to represent the relationship between two nodes.

[0096] Specifically, the nodes in the graph - structured data can represent entities such as natural persons, animals, objects, etc., and the edges represent the relationships between entities. For example, when node A represents Li Si and node B represents paper c, the edge between node A and node B can represent that the author of paper c is Li Si, and the attributes of the node can be adhesion, gender, interest, occupation, etc.

[0097] Furthermore, the nodes in the graph - structured data can also represent components of the power system, such as generators, substations, and the edges represent power transmission lines. The attributes of the nodes may include the capacity, efficiency, maintenance status, etc. of the equipment. This application will not give further examples here.

[0098] In this embodiment, the graph - structured data can specifically be a knowledge graph. The graph - structured data contains knowledge information in multiple fields, and moreover, the knowledge information contained in the graph - structured data is usually objective, comprehensive, and accurate.

[0099] In a possible implementation manner, when executing step S101, it can be specifically executed according to the following steps S1011 - S1013:

[0100] S1011: Perform word - segmentation processing on the answer text to obtain a vocabulary list, and remove the meaningless words in the vocabulary list to obtain a content - word list.

[0101] Use a word - segmenter to perform word - segmentation processing on the answer text to obtain a vocabulary list; compare the words in the vocabulary list with each meaningless word in the preset vocabulary cleaning list, and remove the meaningless words in the vocabulary list to obtain a content - word list. Among them, each meaningless word in the vocabulary cleaning list is, for example: de, di, de.

[0102] S1012: For each semantic word in the semantic vocabulary, query the graph structure data entity list to see if the semantic word is contained therein. If the semantic word is contained in the graph structure data entity list, save the semantic word as the target semantic word to the text entity list; wherein, the graph structure data entity list contains each entity in the graph structure data.

[0103] For example, when the preset question is "Who is Li Si?", if the answer text is "Li Si graduated from S University, and his graduation thesis c was supervised by his supervisor Zhang San", then the resulting semantic vocabulary list contains the following semantic terms: Li Si, S University, graduation thesis, supervisor, Zhang San. The system determines whether the graph structure data contains any of the semantic terms from the semantic vocabulary list, and then stores the semantic terms common to both the semantic vocabulary list and the graph structure data as target semantic terms in the text entity list.

[0104] S1013: For each target semantic word in the text entity list, search for the target attribute word of the target semantic word in the graph structure data. If the target attribute word appears in the word list, store the target semantic word and the target attribute word in the text entity dictionary.

[0105] For example, taking the target content word "Zhang San" as an example, when searching for the target attribute word of the target content word in the graph structure data, if the target attribute word found is "mentor", since the target attribute word "mentor" appears in the word list, the target content word "Zhang San" and the target attribute word "mentor" are stored in the text entity dictionary.

[0106] In this embodiment, the same word in the answer text can be both a content word and an attribute word. Content words and attribute words can be preset, and this application does not limit the definition of content words and attribute words.

[0107] S102: For any two target words, based on the preset first shortest search path, if the path between the two target words in the graph structure data is less than or equal to the first shortest search path, then construct the relationship between the two target words, so as to construct a regression entity relationship network based on the relationship between each pair of target words; wherein, any two target words include: any two target semantic words, any one target semantic word and any one target attribute word.

[0108] For example, taking two target words, "Li Si" and "Zhang San," as an example, Figure 2As shown, in the graph structure data, there are two paths between Li Si and Zhang San, both with a length of 1 (i.e., separated by one node). Assuming the preset first shortest search path has a length of 2 (i.e., separated by 2 nodes), in this example, the path between the two target words "Li Si" and "Zhang San" is shorter than the first shortest search path. Therefore, a relationship between the two target words "Li Si" and "Zhang San" can be constructed. After determining whether the path between any two target words is less than or equal to the first shortest search path, and after constructing the relationship between two target words whose paths are less than or equal to the first shortest search path, a regression entity relationship network is built based on the relationship between each pair of target words.

[0109] Among them, any two target words can be any two target semantic words, or one target semantic word and one target attribute word.

[0110] S103: Input each target semantic vocabulary and the target attribute vocabulary of each target semantic vocabulary, and the response text into the semantic extraction model to obtain an in-text entity relationship network containing the relationship between any two target words.

[0111] In this embodiment, the text entity dictionary and the response text are input into the semantic extraction model to obtain an intratext entity relationship network containing the relationship between any two target words.

[0112] S104: Compare the regressed entity relationship network with the textual entity relationship network to determine the degree of matching between the response text and the graph structure data.

[0113] In one possible implementation, when performing step S104, the following steps S1041-S1042 can be performed:

[0114] S1041: The first total number of first target content words and first relations contained in the entity relationship network within the statistical text; wherein, the first target content words are target content words that do not appear in the regression entity relationship network; and the first relation is a relation that does not appear in the regression entity relationship network.

[0115] In this embodiment, the first target content term is a target content term that appears in the in-text entity relationship network but does not appear in the regression entity relationship network. The first relationship is a relationship that appears in the in-text entity relationship network but does not appear in the regression entity relationship network. Here, a relationship refers to, for example, the relationship between "Li Si" and "S University" is that Li Si is a student of S University.

[0116] For example, assuming there are 100 target content words and relations in the statistical text entity relationship network, if 5 target content words (i.e., the first target content words) and 3 relations (the first relations) appear in the text entity relationship network but do not appear in the regression entity relationship network, then the total number of the first is 8.

[0117] S1042: For any two target words contained in the text entity relationship network, if the relationship between the two target words in the text entity relationship network is different from the relationship between the two target words in the regression entity relationship network, then the relationship between the two target words in the text entity relationship network is determined as a mismatched relationship. The probability of mismatched relationships in the text entity relationship network is calculated to obtain the regression error rate.

[0118] In this embodiment, since the regression entity relationship network is determined based on graph structure data, we assume that the relationship between any two target words in the regression entity relationship network is correct. Therefore, if the relationship between any two target words in the text entity relationship network is different from the relationship between the two target words in the regression entity relationship network, it indicates that the relationship between the two target words in the text entity relationship network is incorrect.

[0119] For example, suppose there are 50 relationships between any two target words in the entity relationship network within the text. If 5 of these relationships are determined to be mismatched, then the regression error rate is:

[0120]

[0121] S105: Obtain the preset answer to the preset question and determine the degree of matching between the answer text and the preset answer.

[0122] In one possible implementation, when performing step S105, the following steps S1051-S1052 can be specifically performed:

[0123] S1051: Obtain a preset answer to a preset question. Based on any two preset keywords in the preset answer, and based on the preset second shortest search path, if the path between the two preset keywords in the graph structure data is less than or equal to the second shortest search path, then construct the relationship between the two preset keywords to build a preset keyword relationship network based on the relationship between each pair of preset keywords. The preset keywords include required keywords, additional keywords, and overriding keywords.

[0124] In this embodiment, the preset answer is the answer to a preset question set in advance by the staff, and the preset keywords are the keywords contained in the preset answer.

[0125] In this embodiment, the process of constructing the preset keyword relationship network can refer to the process of constructing the regression entity relationship network in step S102. The second shortest search path (i.e., the length of the second shortest search path) can be the same as the first shortest search path (i.e., the length of the first shortest search path).

[0126] Among them, required keywords are keywords that are expected to appear in the answer; additional keywords are keywords that are expected to appear in the answer and a certain number are expected to be included; unauthorized keywords refer to keywords that should not appear in the answer, such as private information like phone numbers.

[0127] S1052: Compare the pre-defined keyword relationship network with the entity relationship network in the text to determine the degree of matching between the answer text and the pre-defined answer.

[0128] In one possible implementation, when performing step S1052, the following steps S10521-S10527 can be specifically performed:

[0129] S10521: Calculate the coverage of the response text with the preset keywords to obtain the keyword recall rate.

[0130] For example, assuming there are 10 preset keywords, if the answer text contains 6 of the preset keywords, then the keyword recall rate is 60%.

[0131] S10522: Based on any two required keywords in the preset answer, and based on the preset third shortest search path, if the path between the two required keywords in the graph structure data is less than or equal to the third shortest search path, then construct the relationship between the two required keywords, so as to construct a required keyword relationship network based on the relationship between each pair of required keywords.

[0132] In this embodiment, the process of constructing the required keyword relationship network can refer to the process of constructing the regression entity relationship network in step S102, and will not be repeated in this embodiment. The third shortest search path (i.e., the length of the third shortest search path) can be the same as the first shortest search path (i.e., the length of the first shortest search path).

[0133] S10523: Calculate the first match degree between the required keyword relationship network and the in-text entity relationship network.

[0134] S10524: Based on any two additional keywords in the preset answer, and based on the preset third shortest search path, if the path between the two additional keywords in the graph structure data is less than or equal to the third shortest search path, then construct the relationship between the two additional keywords, so as to construct an additional keyword relationship network based on the relationship between each pair of additional keywords.

[0135] Similarly, the process of constructing the additional keyword relationship network can refer to the process of constructing the regression entity relationship network in step S102, and will not be repeated in this embodiment.

[0136] S10525: Calculate the second matching degree between the additional keyword relationship network and the entity relationship network in the text.

[0137] S10526: Based on any two out-of-weight keywords in the preset answer, and based on the preset third shortest search path, if the path between the two out-of-weight keywords in the graph structure data is less than or equal to the third shortest search path, then construct the relationship between the two out-of-weight keywords, so as to construct an out-of-weight keyword relationship network based on the relationship between each pair of out-of-weight keywords.

[0138] Similarly, the process of constructing the overriding keyword relationship network can refer to the process of constructing the regression entity relationship network in step S102, and will not be repeated in this embodiment.

[0139] S10527: Calculate the third matching degree between the overriding keyword relationship network and the in-text entity relationship network.

[0140] S106: Determine the comprehensive ability of the question-answering system to process preset questions based on the degree of matching between the answer text and the graph structure data, as well as the degree of matching with the preset answer.

[0141] In one possible implementation, when performing step S106, the following steps S1061-S1067 can be specifically performed:

[0142] S1061: Calculate the coverage of the answer text to the preset question based on the keyword recall rate, first match degree, total number of entities and relations contained in the answer text, and total logical number of relations contained in the answer text.

[0143] In this embodiment, the coverage of the answer text to the preset question can be calculated using the following formula:

[0144]

[0145] Where e represents keyword recall rate; f represents first match degree; M represents the sum of the total number of entities and relations contained in the answer text; and N represents the total number of logical relationships between multiple relations contained in the answer text. The distinction between relation and logic is generally as follows: a relation refers to the relationship between two entities, while logic typically refers to a chain of multiple relations.

[0146] S1062: Calculate the reliability of the response text based on the first total number, the regression error rate, and the total number of entities and relations contained in the response text.

[0147] In this embodiment, the reliability of the response text can be calculated using the following formula:

[0148]

[0149] Where a represents the first total number; d represents the regression error rate.

[0150] S1063: The ratio of the first matching degree to the total number of necessary keyword logics is determined as the logical coverage rate of the answer text.

[0151] In this embodiment, the logical coverage of the answer text can be calculated using the following formula:

[0152]

[0153] Where Z represents the total number of necessary keyword logic.

[0154] S1064: The ratio of the second matching degree to the preset upper limit of additional keywords is determined as the additional component of the answer text.

[0155] In this embodiment, the additional component of the answer text can be calculated using the following formula:

[0156]

[0157] Where g represents the second matching degree; Y represents the maximum number of preset additional keywords.

[0158] S1065: Calculate the out-of-weight word score of the response text based on the third matching degree, the total number of entities and the total number of relations contained in the response text.

[0159] In this embodiment, the score for exceeding the authority of words can be calculated using the following formula:

[0160]

[0161] Where h represents the third degree of matching.

[0162] S1066: Calculate the ratio of the first total quantity to the total number of entities and relations contained in the response text, and determine the difference between 1 and this ratio as the illusion quantity score of the response text.

[0163] In this embodiment, the illusion score of the response text can be calculated using the following formula:

[0164]

[0165] S1067: Evaluate the overall ability of a question-answering system to handle a predefined question using the coverage of the answer text to the predefined question, the logical coverage of the answer text, the additional weighting, the overriding word count score, and the illusion count score.

[0166] Based on the same technical concept, this application also provides a verification device based on graph structure data. This device is used to verify an intelligent question-answering system to be verified. The intelligent question-answering system to be verified is used to generate an answer text for a preset question based on an input preset question. Figure 3 As shown, the device includes:

[0167] Extraction module 301 is used to extract target semantic words and target attribute words of each target semantic word from the answer text based on graph structure data; wherein, the graph structure data contains knowledge information from multiple domains; the target semantic words are semantic words common to the graph structure data and the answer text; the target attribute words are attribute words common to the graph structure data and the answer text.

[0168] The construction module 302 is used to construct the relationship between any two target words based on a preset first shortest search path, if the path between the two target words in the graph structure data is less than or equal to the first shortest search path, so as to construct a regression entity relationship network based on the relationship between each pair of target words; wherein, any two target words include: any two target semantic words, any one target semantic word and any one target attribute word;

[0169] The input module 303 is used to input each of the target semantic words and the target attribute words of each of the target semantic words, and the answer text into the semantic extraction model to obtain an intratext entity relationship network containing the relationship between any two target words;

[0170] The comparison module 304 is used to compare the regression entity relationship network with the text-based entity relationship network to determine the degree of matching between the response text and the graph structure data.

[0171] The first determining module 305 is used to obtain a preset answer to the preset question and determine the degree of matching between the answer text and the preset answer;

[0172] The second determining module 306 is used to determine the comprehensive ability of the question-answering system in processing the preset question based on the degree of matching between the answer text and the graph structure data, and the degree of matching with the preset answer.

[0173] Optionally, when the extraction module 301 extracts target semantic words and target attribute words of each target semantic word from the answer text based on graph structure data, it is specifically used for:

[0174] The answer text is segmented to obtain a vocabulary list, and meaningless words are removed from the vocabulary list to obtain a semantic vocabulary list.

[0175] For each semantic word in the semantic vocabulary, the graph structure data entity list is queried to see if the semantic word is contained therein. If the graph structure data entity list contains the semantic word, the semantic word is saved as the target semantic word to the text entity list; wherein, the graph structure data entity list contains each entity in the graph structure data.

[0176] For each target semantic word in the text entity list, search for the target attribute word of the target semantic word in the graph structure data. If the target attribute word appears in the word list, store the target semantic word and the target attribute word in the text entity dictionary.

[0177] Optionally, when the comparison module 304 compares the regressed entity relationship network with the textual entity relationship network to determine the degree of matching between the response text and the graph structure data, it is specifically used for:

[0178] The first total number of first target content words and first relations contained in the entity relationship network within the text is counted; wherein, the first target content words are target content words that do not appear in the regressed entity relationship network; and the first relation is a relation that does not appear in the regressed entity relationship network.

[0179] For any two target words contained in the text-based entity relationship network, if the relationship between the two target words in the text-based entity relationship network is different from the relationship between the two target words in the regression entity relationship network, then the relationship between the two target words in the text-based entity relationship network is determined as a mismatched relationship, and the probability of mismatched relationships in the text-based entity relationship network is calculated to obtain the regression error rate.

[0180] Optionally, when the first determining module 305 is used to obtain a preset answer to the preset question and determine the degree of matching between the answer text and the preset answer, it is specifically used to:

[0181] Obtain a preset answer to the preset question. Based on any two preset keywords in the preset answer, and using a preset second shortest search path, if the path between the two preset keywords in the graph structure data is less than or equal to the second shortest search path, then construct a relationship between the two preset keywords to build a preset keyword relationship network based on the relationship between each pair of preset keywords. The preset keywords include required keywords, additional keywords, and overriding keywords.

[0182] A preset keyword relationship network is constructed for the relationship and compared with the entity relationship network in the text to determine the degree of matching between the answer text and the preset answer.

[0183] Optionally, when the first determining module 305 compares the preset keyword relationship network constructed by the relationship with the entity relationship network within the text to determine the degree of matching between the answer text and the preset answer, it is specifically used for:

[0184] The keyword recall rate is obtained by statistically analyzing the coverage rate of the answer text to the preset keywords.

[0185] Based on any two required keywords in the preset answer, and based on the preset third shortest search path, if the path between the two required keywords in the graph structure data is less than or equal to the third shortest search path, then the relationship between the two required keywords is constructed, so as to construct a required keyword relationship network based on the relationship between each pair of required keywords.

[0186] Calculate the first matching degree between the required keyword relationship network and the in-text entity relationship network;

[0187] Based on any two additional keywords in the preset answer, and based on the preset third shortest search path, if the path between the two additional keywords in the graph structure data is less than or equal to the third shortest search path, then the relationship between the two additional keywords is constructed, so as to construct an additional keyword relationship network based on the relationship between each pair of additional keywords.

[0188] Calculate the second matching degree between the additional keyword relationship network and the text-based entity relationship network;

[0189] Based on any two overriding keywords in the preset answer, and based on the preset third shortest search path, if the path between the two overriding keywords in the graph structure data is less than or equal to the third shortest search path, then the relationship between the two overriding keywords is constructed, so as to construct an overriding keyword relationship network based on the relationship between each pair of overriding keywords.

[0190] Calculate the third matching degree between the overriding keyword relationship network and the text-based entity relationship network.

[0191] Optionally, when the second determining module 306 determines the comprehensive ability of the question-answering system to process the preset question based on the degree of matching between the answer text and the graph structure data, and the degree of matching with the preset answer, it is specifically used for:

[0192] The coverage of the preset question by the answer text is calculated based on the keyword recall rate, the first matching degree, the total number of entities and relations contained in the answer text, and the total number of logical relationships between multiple relations contained in the answer text.

[0193] The reliability of the response text is calculated based on the first total number, the regression error rate, and the total number of entities and relations contained in the response text.

[0194] The ratio of the first matching degree to the total number of necessary keyword logics is determined as the logical coverage rate of the answer text;

[0195] The ratio of the second matching degree to the preset upper limit of the number of additional keywords is determined as the additional component of the answer text;

[0196] The score for the number of out-of-weight words in the answer text is calculated based on the third matching degree, the total number of entities and the total number of relations contained in the answer text.

[0197] Calculate the ratio of the first total number to the total number of entities and the total number of relations contained in the answer text, and determine the difference between 1 and the ratio as the illusion quantity score of the answer text;

[0198] The comprehensive ability of the question-answering system in processing the preset question is evaluated using the coverage of the answer text to the preset question, the logical coverage of the answer text, the additional component, the score of the number of overweight words, and the score of the number of illusions.

[0199] Figure 4A schematic diagram of an electronic device provided in this application embodiment includes: a processor 401, a memory 402, and a bus 403. The memory 402 stores machine-readable instructions executable by the processor 401. When the electronic device runs the above-described information processing method, the processor 401 and the memory 402 communicate through the bus 403. The processor 401 executes the machine-readable instructions to perform the steps of the method described in Embodiment 1.

[0200] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps described in Embodiment 1.

[0201] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, electronic devices, and computer-readable storage media described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0202] In the several embodiments provided in this application, it should be understood that the disclosed methods, apparatuses, electronic devices, and computer-readable storage media can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or modules may be electrical, mechanical, or other forms.

[0203] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0204] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0205] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0206] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of protection of the claims.

Claims

1. A verification method based on graph structure data, characterized in that, The verification method based on the graph structure data is used for verifying a to-be-verified intelligent question and answer system, and the to-be-verified intelligent question and answer system is used for generating an answer text for a preset question according to the input preset question; the method comprises the following steps: Based on the graph structure data, target substantive words and target attribute words of each target substantive word are extracted from the answer text; wherein the graph structure data contains knowledge information of multiple fields; the target substantive word is a substantive word common to the graph structure data and the answer text; the target attribute word is an attribute word common to the graph structure data and the answer text; For any two target words, based on a preset first shortest search path, if the path between the two target words in the graph structure data is less than or equal to the first shortest search path, the relationship between the two target words is constructed, so as to construct a regression entity relationship network according to the relationship between each two target words; wherein any two target words include any two target substantive words, any one target substantive word and any one target attribute word; The target substantive words, the target attribute words of each target substantive word and the answer text are input into a semantic extraction model to obtain a text internal entity relationship network containing the relationship between any two target words; The regression entity relationship network and the text internal entity relationship network are compared to determine the matching degree between the answer text and the graph structure data; A preset answer to the preset question is obtained, and the matching degree between the answer text and the preset answer is determined; Based on the matching degree between the answer text and the graph structure data and the matching degree between the answer text and the preset answer, the comprehensive ability of the question and answer system in processing the preset question is determined.

2. The method of claim 1, wherein, The target substantive words and the target attribute words of each target substantive word are extracted from the answer text based on the graph structure data, comprising: The answer text is processed to obtain a word list, and meaningless words in the word list are removed to obtain a substantive word list; For each substantive word in the substantive word list, it is inquired whether the graph structure data entity list contains the substantive word, and when the graph structure data entity list contains the substantive word, the substantive word is saved as a target substantive word in a text entity list; wherein the graph structure data entity list contains each entity in the graph structure data; For each target substantive word in the text entity list, the target attribute word of the target substantive word is searched in the graph structure data, and if the target attribute word appears in the word list, the target substantive word and the target attribute word are stored in a text entity dictionary.

3. The method of claim 1, wherein, The regression entity relationship network and the text internal entity relationship network are compared to determine the matching degree between the answer text and the graph structure data, comprising: counting a first total number of the first target substantive words and the first relations included in the entity relation net in the text, wherein the first target substantive words are target substantive words not appearing in the regression entity relation net, and the first relations are relations not appearing in the regression entity relation net; for any two target words in the entity relation net in the text, if the relation between the two target words in the entity relation net in the text is different from the relation between the two target words in the regression entity relation net, the relation between the two target words in the entity relation net in the text is determined as a matching error relation, and a regression error rate is obtained by counting a probability of the matching error relation in the entity relation net in the text.

4. The method of claim 3, wherein, The method for determining the matching degree between the answer text and the preset answer for the preset question comprises: obtaining the preset answer for the preset question, and constructing a relation between any two preset key words in the preset answer based on a preset second shortest search path, if a path between the two preset key words in the graph structure data is less than or equal to the second shortest search path, to construct a preset key word relation net according to the relation between each two preset key words, wherein the preset key words comprise mandatory key words, additional key words and over-authorized key words; comparing the preset key word relation net constructed based on the relation with the entity relation net in the text to determine the matching degree between the answer text and the preset answer.

5. The method of claim 4, wherein, The method for determining the matching degree between the answer text and the preset answer for the preset question comprises: counting a key word recall rate of the answer text on the preset key words; constructing a relation between any two mandatory key words in the preset answer based on a preset third shortest search path, if a path between the two mandatory key words in the graph structure data is less than or equal to the third shortest search path, to construct a mandatory key word relation net according to the relation between each two mandatory key words; calculating a first matching degree between the mandatory key word relation net and the entity relation net in the text; constructing a relation between any two additional key words in the preset answer based on a preset third shortest search path, if a path between the two additional key words in the graph structure data is less than or equal to the third shortest search path, to construct an additional key word relation net according to the relation between each two additional key words; calculating a second matching degree between the additional key word relation net and the entity relation net in the text; and calculating a second matching degree between the additional key word relation net and the entity relation net in the text. According to any two over-privilege keywords in the preset answer, a third shortest search path is preset, and if a path between the two over-privilege keywords in the graph structure data is less than or equal to the third shortest search path, a relationship between the two over-privilege keywords is constructed, so as to construct an over-privilege keyword relationship network according to the relationship between each two over-privilege keywords; A third matching degree between the over-privilege keyword relationship network and the entity relationship network in the text is calculated.

6. The method of claim 5, wherein, The matching degrees between the answer text and the graph structure data and between the answer text and the preset answer are used to determine the comprehensive ability of the question and answer system in processing the preset question, including: According to the keyword recall rate, the first matching degree, the total number of entities and the total number of relationships included in the answer text, and the total number of logics between the multiple relationships included in the answer text, a coverage of the answer text on the preset question is calculated; According to the first total number, the regression error rate, and the total number of entities and the total number of relationships included in the answer text, a reliability of the answer text is calculated; A ratio of the first matching degree to the total number of must-keyword logics is determined as a logic coverage of the answer text; A ratio of the second matching degree to a preset upper limit number of the additional keywords is determined as an additional score of the answer text; According to the third matching degree, the total number of entities and the total number of relationships included in the answer text, an over-privilege keyword score of the answer text is calculated; A ratio of the first total number to the total number of entities and the total number of relationships included in the answer text is calculated, and a difference between 1 and the ratio is determined as an illusion score of the answer text; The coverage of the answer text on the preset question, the logic coverage of the answer text, the additional score, the over-privilege keyword score, and the illusion score are used to evaluate the comprehensive ability of the question and answer system in processing the preset question.

7. A verification apparatus based on graph structure data, characterized by, The verification device based on graph structure data is used to verify a to-be-verified intelligent question and answer system, and the to-be-verified intelligent question and answer system is used to generate an answer text for a preset question according to the input preset question. The device comprises: An extraction module is configured to extract target substantive words and target attribute words of each target substantive word from the answer text based on graph structure data; the graph structure data comprises knowledge information of multiple fields; the target substantive words are substantive words common to the graph structure data and the answer text; and the target attribute words are attribute words common to the graph structure data and the answer text; A construction module is configured to, for any two target words, construct a relationship of the two target words based on a preset first shortest search path if a path between the two target words in the graph structure data is less than or equal to the first shortest search path, so as to construct a regression entity relationship network according to the relationship of each two target words; and any two target words include any two target substantive words, any one target substantive word, and any one target attribute word. The input module is configured to input each target substantive word and the target attribute word of each target substantive word and the answer text into a semantic extraction model to obtain an intra-text entity relationship network containing a relationship between any two target words; The comparison module is configured to compare the regression entity relationship network and the intra-text entity relationship network to determine a matching degree between the answer text and the graph structure data; The first determination module is configured to obtain a preset answer to the preset question and determine a matching degree between the answer text and the preset answer; The second determination module is configured to determine a comprehensive capability of the question and answer system in processing the preset question based on the matching degree between the answer text and the graph structure data and the matching degree between the answer text and the preset answer.

8. The apparatus of claim 7, wherein, When the extraction module is used to extract target substantive words and target attribute words of each target substantive word from the answer text based on the graph structure data, the extraction module is specifically configured to: perform word segmentation on the answer text to obtain a word list and remove meaningless words in the word list to obtain a substantive word list; for each substantive word in the substantive word list, query whether the graph structure data entity list contains the substantive word, and when the graph structure data entity list contains the substantive word, save the substantive word as a target substantive word to a text entity list, wherein the graph structure data entity list contains each entity in the graph structure data; for each target substantive word in the text entity list, search for a target attribute word of the target substantive word in the graph structure data, and if the target attribute word appears in the word list, store the target substantive word and the target attribute word in a text entity dictionary.

9. An electronic device, comprising: comprise: a processor, a memory, and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, the machine readable instructions are executed by the processor to perform the steps of the method of any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, which is executed by the processor to perform the steps of the method of any one of claims 1 to 6.

Images