A knowledge intelligent question-answering system based on a knowledge graph

Abstract

The application relates to the technical field of artificial intelligence, in particular to a knowledge intelligent question-answering system based on knowledge graph driving, which comprises a knowledge graph construction module, an information point extraction module, an information analysis module, a correlation coefficient calculation module and a heuristic sorting module. The knowledge graph construction module extracts entities and the relationship between entities from a domain knowledge graph data set, traverses neighbor nodes of each entity node in a breadth-first manner, integrates remote nodes with a question distance value of 2 hops and above into a core knowledge graph, and obtains a domain knowledge graph containing extended correlation nodes. The information point extraction module performs double filtering and judgment of associated sentences and extended sentences on the first answer of the evaluated model, counts the number of effective associated information points and records the corresponding node total set. The application realizes quantitative evaluation and model optimization of the heuristic ability of the knowledge question-answering model, and effectively solves the problems of single heuristic evaluation dimension and the lack of horizontal comparison between models in the prior art.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a knowledge graph-driven intelligent question-answering system. Background Technology

[0002] A knowledge-based question-answering system is an intelligent information processing system capable of understanding, retrieving, reasoning about, and generating answers to user-posed questions. Its core objective is to achieve automated mapping from natural language questions to precise knowledge-based answers. Existing knowledge-based question-answering systems typically rely on large language models, generating answers after semantically understanding user questions. Knowledge graph-driven intelligent question-answering systems, on the other hand, use knowledge graphs as the core knowledge organization carrier and combine natural language processing, semantic understanding, graph relation reasoning, and large language model generation technologies to construct intelligent question-answering systems.

[0003] However, in highly specialized fields such as hydrology and water conservancy, existing knowledge-based question-and-answer systems generally suffer from insufficient depth of professional content in their answers and weak ability to extend and connect information. The extended questions that the system generates after answering a question highly overlap with the answers to the original questions, failing to provide effective answers to the extended questions from a new knowledge perspective. As a result, users find it difficult to gain further knowledge inspiration after obtaining the initial answer. Summary of the Invention

[0004] Therefore, it is necessary for the present invention to provide a knowledge graph-driven intelligent question answering system to solve at least one of the above-mentioned technical problems.

[0005] To achieve the above objectives, a knowledge graph-driven intelligent question-answering system is provided, the system comprising: The knowledge graph construction module is used to introduce domain knowledge graph datasets into the knowledge question answering system, associate extended information related to the target domain with the knowledge base of the knowledge question answering system, and obtain a domain knowledge graph containing extended associated nodes; The information point extraction module is used to count the number of effective related information points in the first answer of the evaluated model to the main question based on the domain knowledge graph, and to obtain the number of effective information points in the first answer. The information analysis module is used to take the related questions triggered by the first answer as input, and based on the domain knowledge graph, count the number of valid related information points that are repeated between the second answer and the first answer of the evaluated model, and use this count as the number of repeated information points. The correlation coefficient calculation module is used to preset the upper limit of the number of effective correlation information points; the difference between the number of the first effective information points and the number of repeated information points is divided by the upper limit to obtain the heuristic correlation coefficient for this problem; The heuristic ranking module is used to rank all evaluated models according to their heuristic correlation coefficients, identify the model with the smallest heuristic correlation coefficient as the optimal heuristic model, and deploy it to the knowledge question answering system.

[0006] This invention constructs a domain knowledge graph containing extended related nodes, and links multi-source extended information of the target domain to the knowledge base of the knowledge question answering system in a structured manner. This enables the system to retrieve deep knowledge nodes that go beyond the direct related scope of the question itself when answering questions, fundamentally expanding the knowledge coverage that the system can trigger extended queries, and providing sufficient knowledge structure support for subsequent heuristic evaluation.

[0007] During the knowledge graph construction phase, this invention calculates the shortest path hop count between extended related nodes and the core entity of the main question. Nearby nodes with a distance of 1 hop from the question are excluded from the domain knowledge graph, retaining only distant nodes for subsequent evaluation. This design eliminates the interference of shallow nodes naturally covered by the model when answering the main question on the statistics of effective information points. This allows the statistical results of the information point extraction module to more accurately reflect the model's ability to truly access deep knowledge, avoiding the problem of a systematic overestimation of the initial number of effective information points due to the repeated counting of near-distance nodes.

[0008] In the information point identification stage, this invention performs a dual filtering judgment of relevance and extensibility for each sentence in the answer. Only sentences that simultaneously satisfy the conditions of having a knowledge graph path association with the main question and having uncovered neighbor nodes in the knowledge graph are counted as valid related information points. This dual filtering mechanism directly links the validity of information points with their extensibility potential in the knowledge graph, so that the statistical results not only reflect the relevance of the answer content to the question, but also reflect the actual guiding ability of the answer content for subsequent knowledge exploration.

[0009] In the question generation phase, this invention extracts the types of association edges between uncovered neighboring nodes and the main question, and generates extended questions in the priority order of causal, parallel, and attribute types, prioritizing nodes corresponding to causal association edges to generate related questions. This design ensures that the input question for the second response naturally points to the direction with the greatest difference from the knowledge boundary of the first response, proactively increasing the probability of the second response introducing a new perspective from the question generation stage, rather than passively waiting for the model to spontaneously generate knowledge boundary breakthroughs.

[0010] In the duplication determination stage, this invention uses the intersection of knowledge graph node sets instead of traditional textual semantic similarity to measure the degree of duplication between two responses. This removes the interference of expression form from the duplication determination, reflecting only the actual coverage and overlap of the two responses at the knowledge graph node level. This transformation reveals the essential difference between linguistic expression and knowledge coverage: two responses may differ significantly in wording, but may still highly overlap at the knowledge graph node level. Traditional semantic similarity methods cannot identify this deep duplication.

[0011] In the heuristic correlation coefficient calculation stage, this invention introduces normalized depth values ​​and normalized deduction weights, simultaneously incorporating the depth distribution of effective information points and the shallow concentration of repeated information points into the coefficient calculation. This allows the heuristic correlation coefficient to reflect not only the degree of repetition between two responses but also the depth level of the knowledge graph where the repetitive behavior occurs. Shallow repetition is penalized more severely, while deep repetition is penalized less severely, enabling the coefficient to distinguish between two fundamentally different repetitive behaviors: "circling around the main question" and "intersecting at the boundaries of deep knowledge." This allows for a more accurate measurement of the model's true heuristic ability. When repeated information points are highly concentrated in shallow nodes, causing the total amount of shallow repetition to exceed the total information depth, the system generates a negative depth-weighted non-repetition rate by using the opposite of the excess proportion. This places the model at the bottom of the heuristic ranking, ensuring that extremely low-heuristic models do not affect the relative ranking relationship between normal models, and maintaining the overall evaluation results stable and reliable even in extreme cases. Attached Figure Description

[0012] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a schematic diagram of a knowledge graph-driven intelligent question-answering system. Figure 2 This is a schematic diagram of the hierarchical relationship of nodes in a domain knowledge graph according to one embodiment. Figure 3 This is a schematic diagram comparing the coverage of two response nodes in one embodiment. Detailed Implementation

[0013] The technical method of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0014] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor methods and / or microcontroller methods.

[0015] It should be understood that although the terms "first," "second," etc., may be used herein to describe various units, these units should not be limited by these terms. These terms are used merely to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, a first unit may be referred to as a second unit, and similarly, a second unit may be referred to as a first unit. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0016] To achieve the above objectives, please refer to Figures 1 to 3 This invention provides a knowledge graph-driven intelligent question-answering system, see reference. Figure 1 The knowledge graph-driven intelligent question-answering system 100 includes: The knowledge graph construction module 101 is used to introduce a domain knowledge graph dataset into the knowledge question answering system, associate extended information related to the target domain with the knowledge base of the knowledge question answering system, and obtain a domain knowledge graph containing extended associated nodes. The information point extraction module 102 is used to count the number of effective related information points in the first answer of the evaluated model to the main question based on the domain knowledge graph, and to obtain the number of first effective information points. The information analysis module 103 is used to take the related questions caused by the first answer as input, and count the number of valid related information points that are repeated between the second answer and the first answer of the evaluated model based on the domain knowledge graph, and use this count as the number of repeated information points. The correlation coefficient calculation module 104 is used to preset the upper limit of the number of effective correlation information points; the difference between the number of the first effective information points and the number of repeated information points is divided by the upper limit to obtain the heuristic correlation coefficient for the problem; The heuristic ranking module 105 is used to rank all evaluated models according to their heuristic correlation coefficients, determine the model with the smallest heuristic correlation coefficient as the optimal heuristic model, and deploy it to the knowledge question answering system.

[0017] This embodiment uses a knowledge question-and-answer system in the field of watershed water conservancy as an application scenario to quantitatively evaluate the heuristics of two evaluated models (Model A and Model B).

[0018] The knowledge graph construction module extracts entities and their relationships from a watershed water conservancy knowledge graph dataset. Taking water conservancy project allocation as the core theme, it constructs a core knowledge graph with water conservancy project entities as nodes and relationships between entities as edges. Using each entity in the core knowledge graph as a seed node, a breadth-first search is performed on its neighboring nodes. All entity nodes and their associated edges within a path of no more than three hops from the seed node are included in an extended association node set. This extended association node set is then merged into the core knowledge graph, resulting in a domain knowledge graph containing extended association nodes. The domain knowledge graph includes extended association nodes related to water conservancy projects, water allocation, ecological flow, and project impacts, along with their interrelationships.

[0019] The information point extraction module takes the main question "Briefly describe the water conservancy allocation in this watershed" as input and obtains the first responses from Model A and Model B respectively. Sentence-level segmentation is performed on the first responses of both models to obtain their respective sentence sequences. For each sentence in the sentence sequence, it is determined whether its semantics are related to an entity in the domain knowledge graph with a path length of no more than 3 hops in the main question. If so, it is marked as a related sentence. Further, it is determined whether the entities appearing in the related sentences have neighboring nodes in the domain knowledge graph that have not yet appeared in the first response. If so, they are marked as extended sentences. Sentences that simultaneously satisfy both the related sentence and extended sentence markings are counted as valid related information points. Statistics show that Model A has 17 valid first-response information points, and Model B has 24. The knowledge graph nodes corresponding to each valid related information point are merged to obtain the total set of first-response nodes for Model A and Model B, respectively, for use by the information analysis module.

[0020] The information analysis module extracts neighboring nodes from the extended related node sets corresponding to each valid related information point in the first responses of Model A and Model B that did not appear in the first response, obtaining separate sets of uncovered neighboring nodes. Combining the type of the edge connections between each node and the main question in the domain knowledge graph, extended questions centered on the nodes are generated, resulting in a set of related questions. The related question with the shortest path to the main question in the domain knowledge graph is selected as the related question triggered by the first response, and input into both models to obtain the second response. The second response undergoes the same sentence-level segmentation and related / extended sentence labeling as the first response, yielding the valid related information points and their corresponding node sets. The intersection of the node set of the second response and the total node set of the first response is calculated to obtain the node overlap set. Valid related information points in the second response that overlap with the node overlap set are identified as duplicate information points. Statistically, Model A has 6 duplicate information points, and Model B has 9 duplicate information points.

[0021] The correlation coefficient calculation module presets an upper limit of 20 for the number of valid correlation information points. The heuristic correlation coefficients for the two models are calculated by subtracting the number of valid information points from the number of repeated information points and dividing the difference by the upper limit. The heuristic correlation coefficient of model A = (17 - 6) ÷ 20 = 0.55; The heuristic correlation coefficient of model B = (24 - 9) ÷ 20 = 0.75.

[0022] The upper limit of 20 was determined as follows: before the formal evaluation, 5 questions were randomly selected from each of the two models for pre-testing. The difference between the number of valid information points in the first test and the number of repeated information points in each pre-test question was counted. The maximum value in the difference sequence was 18. The smallest integer multiple of 5 greater than 18 was selected to obtain the upper limit of 20.

[0023] The heuristic ranking module sorted Model A and Model B according to their heuristic correlation coefficients from smallest to largest, yielding the following heuristic ranking results: Model A has a heuristic correlation coefficient of 0.55, while Model B has a heuristic correlation coefficient of 0.75. The lower heuristic correlation coefficient of Model A indicates that Model A introduced more new knowledge perspectives in its second response, resulting in lower repetition and superior heuristic power. Therefore, Model A was determined to be the optimal heuristic model and was deployed to the knowledge question-answering system.

[0024] Furthermore, the knowledge graph construction module includes: Extract entities and relationships between entities from the domain knowledge graph dataset, and construct a core knowledge graph with core entities as nodes and relationships between entities as edges; Using each entity in the core knowledge graph as a seed node, a breadth-first traversal is performed on its neighboring nodes. All entity nodes and relation edges whose path length from the seed node does not exceed 3 hops are included in the extended associated node set of the seed node. By incorporating the extended related nodes of all core entities into the core knowledge graph, a domain knowledge graph containing the extended related nodes is obtained.

[0025] This embodiment extracts entities and relationships between entities from a knowledge graph dataset in the field of watershed water conservancy. Using "water allocation" as the core entity, a core knowledge graph is constructed with entities as nodes and relationships between entities as edges. The core knowledge graph contains entity nodes such as "water diversion projects," "reservoirs," "irrigation," and "ecological flow," along with their interrelationships.

[0026] Using each entity in the core knowledge graph as a seed node, a breadth-first traversal is performed to find its related neighbor nodes. All entity nodes and their associated edges that are no more than 3 hops away from the seed node are included in the extended related node set of that seed node. Taking "water diversion project" as an example, its 1-hop neighbor nodes are "water source" and "water receiving area", its 2-hop neighbor nodes are "water quality monitoring" and "water conveyance canal", and its 3-hop neighbor nodes are "pollution prevention" and "ecology along the route". All of these are included in the extended related node set of "water diversion project".

[0027] Furthermore, the specific steps of incorporating the extended associated nodes of all core entities into the core knowledge graph are as follows: Calculate the shortest path hop count between each node in the extended association node set and the core entity of the main problem in the domain knowledge graph to obtain the problem distance value of each extended association node; Mark the extended associated nodes with a problem distance value of 1 hop as near nodes, and mark the extended associated nodes with a problem distance value of 2 hops or more as far nodes, thus obtaining the set of near nodes and the set of far nodes. Only by incorporating the set of distant nodes into the core knowledge graph, a domain knowledge graph containing extended related nodes is obtained.

[0028] In this embodiment, when incorporating the extended associated node set into the core knowledge graph, the shortest path hop count between each node in the extended associated node set and the core entity of the main problem, "water conservancy allocation," is calculated in the domain knowledge graph to obtain the problem distance value of each extended associated node. Among them, the problem distance value of "water source area" and "water receiving area" is 1 hop, and they are marked as near nodes; the problem distance value of "water quality monitoring," "water conveyance canal," "pollution prevention and control," and "ecology along the route" is 2 hops or more, and they are marked as far nodes.

[0029] Since the near nodes are highly semantically related to the core entities of the main question, the evaluated model has naturally covered them when answering the main question. If they are included in the domain knowledge graph, the information point extraction module will overestimate the number of effective information points in the first time, which will interfere with the accuracy of the heuristic association coefficient. Therefore, only the set of far nodes is incorporated into the core knowledge graph to obtain a domain knowledge graph that includes extended association nodes.

[0030] Figure 2 The diagram illustrates the node hierarchy of the domain knowledge graph constructed by the knowledge graph construction module of this invention. Figure 2 In the domain knowledge graph, the top layer is the core node "Water Conservancy Allocation," represented by a bold circle. The second layer consists of one-hop nearby nodes, including "Water Diversion Projects," "Reservoirs," "Irrigation," and "Ecological Flow," represented by gray-filled circles with an "×" symbol. These nodes are excluded from the domain knowledge graph because they are highly semantically related to the main question and are naturally covered by the model in its response. The third layer consists of two-hop distant nodes, including "Water Quality Monitoring," "Water Conveyance Canals," "Water Replenishment Benefits," and "Engineering Scheduling," represented by white circles and included in the domain knowledge graph. The fourth layer consists of three-hop distant nodes, including "Pollution Prevention," "Ecology Along the Route," "Dry Season Plan," and "Scheduling Rules," also represented by white circles and included in the domain knowledge graph. Figure 1 The connections between nodes distinguish three types of associated edges: solid arrows represent causal associated edges, dashed arrows represent parallel associated edges, and dotted-dash arrows represent attribute associated edges.

[0031] Furthermore, the information point extraction module includes: The first answer of the evaluated model to the main question is segmented into sentences to obtain a sentence sequence. Determine whether the semantics of each sentence in the sentence sequence are related to the core entity of the main question in the domain knowledge graph with a path length of no more than 3 hops. If the semantics are related, mark it as a related sentence. Determine whether the entities appearing in the related sentences have neighbor nodes in the domain knowledge graph that have not yet appeared in the first answer; if so, mark them as extended sentences. Sentences that simultaneously satisfy the tags for related sentences and extended sentences are counted as valid related information points, and their total number is used as the first number of valid information points.

[0032] This embodiment takes the main question "Briefly describe the water conservancy allocation in this basin" as input and obtains the first answers from Model A and Model B respectively.

[0033] Sentence-level segmentation was performed on the first response of Model A to obtain a sentence sequence. Taking the sentence "The Yangtze River Diversion Project diverts an average of 3.1 billion cubic meters of water annually to compensate for the downstream water reduction caused by the Central Route Water Diversion Project" as an example, the semantics of this sentence have an entity association with the core entity "water conservancy allocation" in the domain knowledge graph with a path length of no more than 3 hops, and it is marked as an associated sentence. Further, it is determined that the entity "Yangtze River Diversion Project" appearing in this sentence has neighboring nodes such as "project scheduling rules" and "water replenishment benefit assessment" that have not yet appeared in the first response in the domain knowledge graph, which meet the extensibility condition and are marked as extended sentences. This sentence satisfies the dual marking of associated sentence and extended sentence and is included in the effective associated information point. After performing the above judgment on all sentences in the first response of Model A, the number of effective information points in the first response was counted as 17.

[0034] The same operation was performed on the first response of Model B, and the number of valid information points in the first response was found to be 24.

[0035] At the same time, the knowledge graph nodes corresponding to each valid associated information point are merged to obtain the first answer node set of Model A and Model B, respectively, for use by the information analysis module.

[0036] Furthermore, the specific related questions triggered by the first answer in the information analysis module are as follows: Extract the neighboring nodes that did not appear in the first answer from the extended associated node set corresponding to each valid associated information point in the first answer, and obtain the set of uncovered neighboring nodes; Taking the nodes in the set of uncovered neighbor nodes as the main body, and combining the types of their association edges with the main question in the domain knowledge graph, we generate extended questions centered on the nodes, thus obtaining a set of related questions. The related question with the shortest path to the main question in the domain knowledge graph is selected from the set of related questions and used as the related question triggered by the first answer. This related question is then input into the evaluated model to obtain the second answer.

[0037] This embodiment takes the first response of Model A as an example. It extracts the neighbor nodes that have not appeared in the first response from the extended associated node set corresponding to each valid associated information point, and obtains the set of uncovered neighbor nodes, which includes nodes such as "engineering scheduling rules", "water replenishment benefit assessment", "ecological flow standard" and "water quality monitoring indicators".

[0038] By combining the types of edges connecting each node to the core entity "water allocation" in the domain knowledge graph, extended questions centered on the nodes are generated. Specifically, the edge connecting "ecological flow standard" to the main question is causal, generating the extended question "What impact has the water diversion project had on the ecological flow of this basin?"; the edge connecting "water quality monitoring indicators" to the main question is attribute-based, generating the extended question "What are the main water quality monitoring indicators for this basin?"; these extended questions together constitute a set of related questions.

[0039] The question “What impact has the water diversion project had on the ecological flow of the watershed”, which has the shortest path to the main question in the domain knowledge graph, is taken from the set of related questions. This question is then input into model A and model B respectively to obtain the second answers from the two models.

[0040] Furthermore, in the information analysis module, based on the domain knowledge graph, the number of valid related information points that overlap between the second and first responses of the evaluated model is specifically as follows: Sentence-level segmentation and tagging of related and extended sentences are performed on the second response to obtain the effective related information points of the second response; Merge the nodes corresponding to each valid related information point in the first answer to obtain the total set of nodes in the first answer; The intersection of the set of nodes corresponding to the valid associated information points in the second response and the total set of nodes in the first response is calculated to obtain the node overlap set. In the second response, valid related information points that overlap with the node overlapping set are identified as duplicate information points, and the total number of duplicate information points is used as the number of duplicate information points.

[0041] In this embodiment, sentence-level segmentation and related sentence and extended sentence labeling are performed on the second responses of Model A and Model B, respectively, to obtain the effective related information points of the second responses and their corresponding node sets.

[0042] Taking Model A as an example, the set of nodes corresponding to the effective related information points in the second response includes nodes such as "ecological flow standard", "Yangtze River to Han River Water Diversion Project", "water transfer volume", and "water quality monitoring indicators". Merging the nodes corresponding to each effective related information point in the first response of Model A yields the total set of nodes for the first response, which includes nodes such as "Yangtze River to Han River Water Diversion Project", "water transfer volume", "reservoir scheduling", and "irrigation water".

[0043] The intersection of the node set in the second response and the total node set in the first response is calculated to obtain the node overlap set, which includes the nodes "Yangtze River Diversion Project" and "Water Diversion Volume". Valid related information points in the second response that overlap with the node overlap set are identified as duplicate information points. The number of duplicate information points in model A is calculated to be 6.

[0044] Performing the same operation on model B, the number of duplicate information points in model B was found to be 9.

[0045] Furthermore, the correlation coefficient calculation module includes: Preset an upper limit on the number of valid associated information points; Extract the shortest path hop count in the domain knowledge graph between each valid related information point in the first answer and the core entity of the main question. Divide the shortest path hop count by the maximum hop count of breadth-first traversal (3) to obtain the normalized depth value of that information point. The total information depth is the sum of the normalized depth values ​​of all valid related information points in the first response.

[0046] In this embodiment, the shortest path hop count between each valid associated information point in the first response of Model A and the core entity "Water Conservancy Allocation" in the domain knowledge graph is extracted. The shortest path hop count is divided by the maximum hop count of 3 to obtain the normalized depth value of each information point. Taking the "Yangtze River Diversion Project" node as an example, its shortest path hop count is 1, and its normalized depth value is 1÷3≈0.33; taking the "Ecological Flow Standard" node as an example, its shortest path hop count is 2, and its normalized depth value is 2÷3≈0.67; taking the "Water Replenishment Benefit Assessment" node as an example, its shortest path hop count is 3, and its normalized depth value is 3÷3=1.00. The normalized depth values ​​of all 17 valid associated information points in Model A are summed to obtain the total information depth of Model A as 10.33.

[0047] Furthermore, the correlation coefficient calculation module also includes: Extract the shortest path hop count between each information point in the overlapping node set and the core entity of the main problem in the domain knowledge graph, and divide 3 by the shortest path hop count to obtain the normalized deduction weight of the overlapping information point. The sum of the normalized weights of all duplicate information points is used as the total amount of duplicate shallowing. This embodiment extracts the shortest path hop count between the corresponding node of each duplicate information point in the node overlap set and the core entity of the main problem, "Water Conservancy Allocation," in the domain knowledge graph. Dividing 3 by the shortest path hop count yields the normalized deduction weight for each duplicate information point. Taking the "Yangtze River Diversion Project" node as an example, its shortest path hop count is 1, and its normalized deduction weight is 3 ÷ 1 = 3.00; taking the "Water Flow" node as an example, its shortest path hop count is 2, and its normalized deduction weight is 3 ÷ 2 = 1.50. Summing up the normalized deduction weights of all 6 duplicate information points in Model A yields a total shallowing weight of 10.50 for Model A.

[0048] Divide the total amount of shallow repetition by the total amount of information depth to get the depth-weighted repetition rate. Subtract the depth-weighted repetition rate from 1 to get the depth-weighted non-repetition rate. Divide the number of valid information points in the first round by the upper limit to obtain the information point density value; use the product of the depth-weighted non-repetition rate and the information point density value as the heuristic correlation coefficient.

[0049] In this embodiment, the total amount of repeated shallowing, 10.50, is divided by the total amount of information depth, 10.33, to obtain a depth-weighted repetition rate of 1.02. Subtracting the depth-weighted repetition rate from 1, we obtain a depth-weighted non-repetition rate of -0.02.

[0050] The negative value of the depth-weighted non-repetition rate indicates that the repetitive information points in model A are concentrated in shallow nodes, causing the total amount of shallow repetition to exceed the total information depth. The corresponding adaptive adjustment mechanism adjusts the upper limit to a multiple of 5 (the smallest integer multiple of the difference) and recalculates, ultimately obtaining the heuristic correlation coefficient of model A.

[0051] Perform the same calculation process on model B, substitute the heuristic correlation coefficients of the two models into the heuristic ranking module, sort them in ascending order of heuristic correlation coefficients, determine the optimal heuristic model, and deploy it to the knowledge question answering system.

[0052] Importantly, when the total amount of repeated shallowing exceeds the total information depth, the following steps are also included: Calculate the difference between the total amount of repeated shallowing and the total amount of information depth to obtain the dimensional excess value; Divide the excess value by the total information depth to obtain the excess ratio; Using the excess ratio as the extreme repetition penalty coefficient of the model on this problem, the depth-weighted non-repetition rate is set to the negative of the excess ratio, resulting in a negative depth-weighted non-repetition rate. When the heuristic ranking module sorts all models, models with negative depth-weighted non-repetition rates are ranked after all models with positive depth-weighted non-repetition rates. The smaller the negative value, the further back in the ranking. This ensures that models in extreme repetition cases are at the bottom of the heuristic ranking and do not affect the relative ranking relationship between normal models.

[0053] Of particular importance, the upper limit of the number of preset valid correlation information points in the correlation coefficient calculation module includes: For each model being evaluated, at least 5 questions are randomly selected for pre-testing. The number of valid information points in the first test and the number of duplicate information points for each pre-test question are counted by the information point extraction module and the information analysis module, respectively. Calculate the difference between the number of valid information points in the first test and the number of repeated information points for each pretest question to obtain the difference sequence; Take the maximum value in the difference sequence, and select the smallest integer multiple of 5 that is greater than the maximum value to obtain the upper limit value.

[0054] Before the formal evaluation, this embodiment randomly selects 5 questions from both Model A and Model B for pre-testing. The information point extraction module and the information analysis module respectively count the number of valid information points in the first test and the number of repeated information points for each pre-test question, and calculate the difference between the two. The difference sequence is as follows: The difference sequence for model A is: 11, 8, 13, 9, 10; The difference sequence for model B is: 14, 12, 15, 11, 13.

[0055] The maximum value of the difference sequence between the two models is taken as 15. The smallest integer multiple of 5 that is greater than 15 is selected to obtain the upper limit value of 20. This value is then substituted into the correlation coefficient calculation module as the normalization denominator to ensure that the heuristic correlation coefficients of model A and model B are horizontally comparable under the same normalization benchmark.

[0056] Furthermore, the heuristic sorting module includes: All evaluated models are sorted in ascending order of heuristic correlation coefficient to obtain a heuristic ranking. Extract the effective associated information points corresponding to the models with the largest heuristic association coefficient in the heuristic level ranking, which are judged as duplicate information points, obtain the out-degree of their corresponding nodes in the domain knowledge graph, and obtain the out-degree value of each duplicate node. Nodes whose out-degree value is lower than the average out-degree value of all duplicate nodes are designated as nodes to be expanded. Send a supplementary instruction to the knowledge graph construction module to add new related edges and neighbor nodes to the domain knowledge graph, starting from the node to be expanded, to obtain the updated enhanced knowledge graph. The information point extraction module of the enhanced knowledge graph is re-executed to the heuristic ranking module. The model with the smallest heuristic correlation coefficient is determined as the optimal heuristic model. This process continues until the heuristic correlation coefficient of the optimal heuristic model in the heuristic ranking no longer decreases significantly. The optimal heuristic model is then deployed to the knowledge question answering system.

[0057] In this embodiment, Model A and Model B are sorted from smallest to largest according to their heuristic correlation coefficients, and the heuristic ranking results are as follows: Model A has a heuristic correlation coefficient of 0.55, and Model B has a heuristic correlation coefficient of 0.75. Model B has the largest heuristic correlation coefficient and the worst heuristic performance.

[0058] Nine valid related information points identified as duplicate information points in the information analysis module of Model B were extracted. The out-degree of their corresponding nodes in the domain knowledge graph was obtained, and the out-degree value of each duplicate node was calculated. Taking the "Yangtze River Diversion Project" node as an example, its out-degree value is 8; the "Water Transfer Quantity" node as an example, its out-degree value is 3; and the "Reservoir Scheduling" node as an example, its out-degree value is 2. The average out-degree value of all duplicate nodes is calculated to be 4.5. Nodes with out-degree values ​​lower than 4.5, such as "Water Transfer Quantity" and "Reservoir Scheduling", are marked as nodes to be expanded.

[0059] Send supplementary instructions to the knowledge graph construction module, starting with "water transfer volume" and "reservoir scheduling", to add new related edges and neighbor nodes to the domain knowledge graph, including nodes such as "annual average water transfer volume statistics", "scheduling rules", and "dry season scheduling plan" and their related edges, to obtain an enhanced knowledge graph.

[0060] To enhance the knowledge graph, the information point extraction module is re-executed into the heuristic ranking module, and the heuristic correlation coefficient between model A and model B is recalculated. If the heuristic correlation coefficient of model A no longer decreases significantly, then model A is determined to be the optimal heuristic model, and model A is deployed to the knowledge question answering system.

[0061] Figure 3 This paper illustrates the specific implementation process of the present invention for heuristic evaluation of model A and model B. Figure 3In the table, the left column shows the evaluation results of Model A, and the right column shows the evaluation results of Model B. Each column is divided into three areas: the first response coverage nodes, the node overlap set, and the second response coverage nodes. In the first response coverage node area, Model A covers 6 effective related information points: "Yangtze River Diversion Project (1 jump)," "Water Transfer Volume (2 jumps)," "Reservoir Scheduling (2 jumps)," "Irrigation Water (2 jumps)," "Ecological Flow Standard (2 jumps)," and "Water Replenishment Benefit Assessment (3 jumps)," with a first effective information point count of 17. Model B covers 6 effective related information points: "Yangtze River Diversion Project (1 jump)," "Water Transfer Volume (2 jumps)," "Reservoir Scheduling (2 jumps)," "Ecological Flow Standard (2 jumps)," "Water Replenishment Benefit Assessment (3 jumps)," and "Project Scheduling (3 jumps)," with a first effective information point count of 24. The overlapping node sets are marked with a gray dashed box. Model A's overlapping node set includes "Yangtze River Diversion Project (1 hop)" and "Water Transfer Volume (2 hops)," while Model B's overlapping node set includes "Yangtze River Diversion Project (1 hop)," "Water Transfer Volume (2 hops)," and "Reservoir Scheduling (2 hops)." In the second response's covered node area, gray dashed circle nodes represent duplicate information points, and white solid circle nodes represent newly added information points. Model A has 6 duplicate information points, and Model B has 9. Model A has a total information depth of 10.33, a total amount of duplicate shallowing of 4.50, a depth-weighted non-duplication rate of 0.564, and a heuristic correlation coefficient of 0.55. Model B has a total information depth of 14.67, a total amount of duplicate shallowing of 8.00, a depth-weighted non-duplication rate of 0.455, and a heuristic correlation coefficient of 0.75. The heuristic correlation coefficient of model A is less than that of model B. The heuristic ranking module determines that model A is the optimal heuristic model and deploys model A to the knowledge question answering system.

[0062] Therefore, the embodiments should be considered as exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of the equivalents of the application are intended to be included within the invention.

[0063] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features of the invention herein.

Claims

1. A knowledge graph-driven intelligent question-answering system, characterized in that, Includes the following modules: The knowledge graph construction module is used to introduce domain knowledge graph datasets into the knowledge question answering system, associate extended information related to the target domain with the knowledge base of the knowledge question answering system, and obtain a domain knowledge graph containing extended associated nodes; The information point extraction module is used to count the number of effective related information points in the first answer of the evaluated model to the main question based on the domain knowledge graph, and to obtain the number of effective information points in the first answer. The information analysis module is used to take the related questions triggered by the first answer as input, and based on the domain knowledge graph, count the number of valid related information points that are repeated between the second answer and the first answer of the evaluated model, and use this count as the number of repeated information points. The correlation coefficient calculation module is used to preset the upper limit of the number of effective correlation information points; the difference between the number of the first effective information points and the number of repeated information points is divided by the upper limit to obtain the heuristic correlation coefficient for this problem; The heuristic ranking module is used to rank all evaluated models according to their heuristic correlation coefficients, identify the model with the smallest heuristic correlation coefficient as the optimal heuristic model, and deploy it to the knowledge question answering system.

2. The knowledge graph-driven intelligent question-answering system according to claim 1, characterized in that, The knowledge graph construction module includes: Extract entities and relationships between entities from the domain knowledge graph dataset, and construct a core knowledge graph with core entities as nodes and relationships between entities as edges; Using each entity in the core knowledge graph as a seed node, a breadth-first traversal is performed on its neighboring nodes. All entity nodes and relation edges whose path length from the seed node does not exceed 3 hops are included in the extended associated node set of the seed node. By incorporating the extended related nodes of all core entities into the core knowledge graph, a domain knowledge graph containing the extended related nodes is obtained.

3. The knowledge graph-driven intelligent question-answering system according to claim 2, characterized in that, The specific steps involved in incorporating the extended associated nodes of all core entities into the core knowledge graph are as follows: Calculate the shortest path hop count between each node in the extended association node set and the core entity of the main problem in the domain knowledge graph to obtain the problem distance value of each extended association node; Mark the extended associated nodes with a problem distance value of 1 hop as near nodes, and mark the extended associated nodes with a problem distance value of 2 hops or more as far nodes, thus obtaining the set of near nodes and the set of far nodes. Only by incorporating the set of distant nodes into the core knowledge graph, a domain knowledge graph containing extended related nodes is obtained.

4. The knowledge graph-driven intelligent question-answering system according to claim 3, characterized in that, The information point extraction module includes: The first answer of the evaluated model to the main question is segmented into sentences to obtain a sentence sequence. Determine whether the semantics of each sentence in the sentence sequence are related to the core entity of the main question in the domain knowledge graph with a path length of no more than 3 hops. If the semantics are related, mark them as related sentences. Determine whether the entities appearing in the related sentences have neighbor nodes in the domain knowledge graph that have not yet appeared in the first answer; if so, mark them as extended sentences. Sentences that simultaneously satisfy the tags for related sentences and extended sentences are counted as valid related information points, and their total number is used as the first number of valid information points.

5. The knowledge graph-driven intelligent question-answering system according to claim 4, characterized in that, The specific related questions triggered by the first answer in the information analysis module are as follows: Extract the neighboring nodes that did not appear in the first answer from the extended associated node set corresponding to each valid associated information point in the first answer, and obtain the set of uncovered neighboring nodes; Taking the nodes in the set of uncovered neighbor nodes as the main body, and combining the types of their association edges with the main question in the domain knowledge graph, we generate extended questions centered on the nodes, thus obtaining a set of related questions. The question with the shortest path to the main question in the domain knowledge graph from the set of related questions is taken as the related question triggered by the first answer, and input into the evaluated model to obtain the second answer.

6. The knowledge graph-driven intelligent question-answering system according to claim 5, characterized in that, In the information analysis module, based on the domain knowledge graph, the number of valid relevance information points that overlap between the second and first responses of the evaluated model is specifically as follows: Sentence-level segmentation and tagging of related and extended sentences are performed on the second response to obtain the effective related information points of the second response; Merge the nodes corresponding to each valid related information point in the first answer to obtain the total set of nodes in the first answer; The intersection of the set of nodes corresponding to the valid associated information points in the second response and the total set of nodes in the first response is calculated to obtain the node overlap set. In the second response, valid related information points that overlap with the node's overlapping set are identified as duplicate information points, and the total number of duplicate information points is used as the number of duplicate information points.

7. The knowledge graph-driven intelligent question-answering system according to claim 6, characterized in that, The correlation coefficient calculation module includes: Preset an upper limit on the number of valid associated information points; Extract the shortest path hop count in the domain knowledge graph between each valid related information point in the first answer and the core entity of the main question. Divide the shortest path hop count by the maximum hop count of breadth-first traversal (3) to obtain the normalized depth value of that information point. The total information depth is the sum of the normalized depth values ​​of all valid related information points in the first response.

8. The knowledge graph-driven intelligent question-answering system according to claim 7, characterized in that, The correlation coefficient calculation module also includes: Extract the shortest path hop count between each information point in the overlapping node set and the core entity of the main problem in the domain knowledge graph, and divide 3 by the shortest path hop count to obtain the normalized deduction weight of the overlapping information point. The sum of the normalized weights of all duplicate information points is used as the total amount of duplicate shallowing. Divide the total amount of shallow repetition by the total amount of information depth to get the depth-weighted repetition rate. Subtract the depth-weighted repetition rate from 1 to get the depth-weighted non-repetition rate. Divide the number of valid information points in the first round by the upper limit to obtain the information point density value; use the product of the depth-weighted non-repetition rate and the information point density value as the heuristic correlation coefficient.

9. The knowledge graph-driven intelligent question-answering system according to claim 8, characterized in that, The upper limit of the number of valid correlation information points preset in the correlation coefficient calculation module includes: For each model being evaluated, at least 5 questions are randomly selected for pre-testing. The number of valid information points in the first test and the number of duplicate information points for each pre-test question are counted by the information point extraction module and the information analysis module, respectively. Calculate the difference between the number of valid information points in the first test and the number of repeated information points for each pre-test question to obtain the difference sequence; Take the maximum value in the difference sequence, and select the smallest integer multiple of 5 that is greater than the maximum value to obtain the upper limit value.

10. The knowledge graph-driven intelligent question-answering system according to claim 9, characterized in that, The heuristic sorting module includes: All evaluated models are sorted in ascending order of heuristic correlation coefficient to obtain a heuristic ranking. Extract the effective associated information points corresponding to the models with the largest heuristic association coefficient in the heuristic level ranking, which are judged as duplicate information points, obtain the out-degree of their corresponding nodes in the domain knowledge graph, and obtain the out-degree value of each duplicate node. Nodes whose out-degree value is lower than the average out-degree value of all duplicate nodes are designated as nodes to be expanded. Send a supplementary instruction to the knowledge graph construction module to add new related edges and neighbor nodes to the domain knowledge graph, starting from the node to be expanded, to obtain the updated enhanced knowledge graph. The information point extraction module of the enhanced knowledge graph is re-executed to the heuristic ranking module. The model with the smallest heuristic correlation coefficient is determined as the optimal heuristic model. This process continues until the heuristic correlation coefficient of the optimal heuristic model in the heuristic ranking no longer decreases significantly. The optimal heuristic model is then deployed to the knowledge question answering system.

Images