Knowledge question and answer logic form determination method and device, equipment and storage medium

Abstract

The invention discloses a knowledge question and answer logic form determination method and device, equipment and a storage medium. The method comprises the steps that an initial case library is screened according to a target question, an initial case set is obtained, the initial case set comprises at least one case, and the relation similarity between the question included in each case and the target question is larger than a similarity threshold value; each case in the initial case set further comprises a logic form corresponding to the question; performing entity replacement on a logic form in each case in an initial case set according to an entity corresponding to the target question to obtain a first logic form set; generating a target logic form set according to the logic forms in the first logic form set, wherein the correlation scores between the logic forms and the target question are greater than a score threshold value; and inputting the target logic form set and the target question into a generation model to obtain a target logic form corresponding to the target question, so that the execution rate of the logic form can be improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computer technology, and in particular, to a method, device, equipment and storage medium for determining a logical form of a knowledge question and answer. Background Art

[0002] With the development of deep learning technology, question-answering systems have developed rapidly. They use deep learning models to understand the questions and texts given by users, establish more accurate semantic representations, and thus improve the accuracy and quality of answers. Current question-answering systems can be divided into structured question-answering systems and unstructured question-answering systems according to the form of data. Unstructured question-answering systems mainly deal with unstructured data, such as text, images, etc. This type of question-answering system uses natural language processing, semantic understanding, information retrieval and other technologies to understand and analyze users' natural language questions, and then obtain answers from a large amount of unstructured data. Knowledge Base Question Answering (KBQA) is a structured question-answering system that stores structured data. Compared with unstructured data question-answering systems, knowledge base question-answering systems have the following advantages:

[0003] 1. High accuracy: The knowledge base question answering system is based on the existing knowledge base, so the accuracy of the answers is often higher. Compared with the text question answering system, the knowledge base question answering system can provide more accurate answers by matching questions with entries in the existing knowledge base.

[0004] 2. Wide applicability: The knowledge base question-answering system can not only be used to quickly answer questions in specific fields, but also to answer common questions, such as customer service consultation.

[0005] 3. Easy to reason: Due to the characteristics of structured data in the knowledge base, it is easier to reason about relationships in the knowledge base based on natural language questions.

[0006] 4. Strong scalability: The knowledge base question and answer system can increase the quantity and quality of answers by continuously updating the knowledge base, and can also enhance the system's functionality and coverage by adding question classifications and knowledge base classifications.

[0007] With the continuous research and development of knowledge base question answering tasks, more and more work focuses on understanding and answering complex questions.

[0008] The performance of the knowledge base question answering model mainly depends on the following factors: First, whether the model can truly understand the semantic information of complex questions and then query the knowledge base to obtain the target answer, which is the focus of current knowledge base question answering research. Secondly, whether there is knowledge in the knowledge base to answer the question, which is the premise for the system to answer the question correctly. Among them, a common type of method is the semantic parse (SP-based) method, which converts natural language questions into logical forms through semantic parsing, and directly queries the knowledge base to obtain answers by executing the logical form. First, semantic parsing is used to understand the semantics of natural language questions, and the questions are converted into logical forms with the same semantics. Then, the query engine is used to query the generated logical forms to obtain the final results. The above process corresponds to the two main stages of semantic parsing and query execution. At the same time, the semantic parsing module is limited by the design and parsing capabilities of the logical form, which is the key point for improving the performance of the semantic parsing method.

[0009] The current mainstream paradigm of semantic parsing-based methods is the enumeration-sorting logical form method. In the process of enumerating logical forms, entities are extracted according to the semantics of the problem, and then the paths in the knowledge base are queried according to the entities, and the units in the paths are filled into the pre-defined templates. This type of work requires pre-definition of possible logical form templates. Due to the limitations of storage and computational costs, it is impossible to cover all possible situations, so the types and number of candidate logical forms obtained are limited. At the same time, there are some problems from which entities cannot be identified, resulting in the inability to enumerate logical forms based on entities. Summary of the invention

[0010] The embodiments of the present invention provide a method, apparatus, device and storage medium for determining the logical form of a knowledge question and answer, so as to enrich the types of candidate logical forms and modify the expanded logical forms to ensure the execution rate of the logical forms.

[0011] According to one aspect of the present invention, a method for determining a knowledge question and answer logic form is provided, comprising:

[0012] The initial case library is screened according to the target problem to obtain an initial case set, wherein the initial case set includes: at least one case, the relationship similarity between the problem included in each case and the target problem is greater than a similarity threshold, and each case in the initial case set also includes: a logical form corresponding to the problem;

[0013] Perform entity replacement on the logical form in each case in the initial case set according to the entity corresponding to the target problem to obtain a first logical form set;

[0014] Generate a target logical form set according to the logical forms in the first logical form set whose relevance scores to the target problem are greater than a score threshold;

[0015] The target logical form set and the target problem are input into a generation model to obtain a target logical form corresponding to the target problem.

[0016] According to another aspect of the present invention, a device for determining a logical form of a knowledge question and answer is provided, the device for determining a logical form of a knowledge question and answer comprises:

[0017] An initial case set determination module is used to screen the initial case library according to the target problem to obtain an initial case set, wherein the initial case set includes: at least one case, the relationship similarity between the problem included in each case and the target problem is greater than a similarity threshold, and each case in the initial case set also includes: a logical form corresponding to the problem;

[0018] A first logic form set determination module, configured to perform entity replacement on the logic form in each case in the initial case set according to the entity corresponding to the target problem, to obtain a first logic form set;

[0019] a target logical form set generating module, configured to generate a target logical form set according to the logical forms in the first logical form set whose relevance scores with the target problem are greater than a score threshold;

[0020] The target logical form determination module is used to input the target logical form set and the target problem into the generation model to obtain the target logical form corresponding to the target problem.

[0021] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0022] at least one processor; and

[0023] a memory communicatively connected to the at least one processor; wherein,

[0024] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for determining the logical form of knowledge questions and answers described in any embodiment of the present invention.

[0025] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for determining the logical form of a knowledge question and answer as described in any embodiment of the present invention when executed.

[0026] The embodiment of the present invention obtains an initial case set by screening an initial case library according to a target problem, wherein the initial case set includes: at least one case, and the relationship similarity between the problem included in each case and the target problem is greater than a similarity threshold, and each case in the initial case set also includes: a logical form corresponding to the problem; performing entity replacement on the logical form in each case in the initial case set according to the entity corresponding to the target problem to obtain a first logical form set; generating a target logical form set according to the logical forms in the first logical form set whose correlation scores with the target problem are greater than a score threshold; inputting the target logical form set and the target problem into a generation model to obtain a target logical form corresponding to the target problem, which can enrich the types of candidate logical forms while modifying the expanded logical forms to ensure the execution rate of the logical forms.

[0027] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0029] Figure 1 is a flow chart of a method for determining a knowledge question and answer logic form in an embodiment of the present invention;

[0030] Figure 2 It is a structural schematic diagram of a knowledge question and answer logic form determination device in an embodiment of the present invention;

[0031] Figure 3 It is a structural schematic diagram of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0034] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0035] Embodiment 1

[0036] Figure 1 A flowchart of a method for determining the logical form of a knowledge question and answer provided in an embodiment of the present invention. This embodiment is applicable to the case of determining the logical form of a knowledge question and answer. The method can be executed by a device for determining the logical form of a knowledge question and answer in an embodiment of the present invention. The device can be implemented in software and / or hardware. Figure 1 As shown, the method specifically comprises the following steps:

[0037] S110, screening the initial case library according to the target problem to obtain an initial case set.

[0038] The initial case set includes: at least one case, the relationship similarity between the question included in each case and the target question is greater than a similarity threshold, and each case in the initial case set also includes: a logical form corresponding to the question.

[0039] Specifically, the initial case library is screened according to the target problem to obtain the initial case set in the following manner: all the problems in the initial case library are entity-masked to obtain the first case library; all the problems in the first case library are input into the encoder in sequence to obtain the representation features corresponding to the problems in the cases; the relationship similarity between the representation features corresponding to the problems in the cases and the target problems is obtained; the cases whose relationship similarity between the problems in the cases in the case library and the target problems is greater than a similarity threshold are determined as target cases; and the initial case set is generated according to the target cases in the initial case library.

[0040] S120, performing entity replacement on the logical form in each case in the initial case set according to the entity corresponding to the target problem to obtain a first logical form set.

[0041] Specifically, the method of performing entity replacement on the logical form in each case in the initial case set according to the entity corresponding to the target problem to obtain the first logical form set may be: obtaining the number of entities corresponding to the logical form in each case in the initial case set and the number of entities corresponding to the target problem; performing entity replacement on the logical form in each case in the initial case set according to the number of entities corresponding to the logical form in each case in the initial case set and the number of entities corresponding to the target problem to obtain the first logical form set.

[0042] S130: Generate a target logical form set according to the logical forms in the first logical form set whose relevance scores to the target question are greater than a score threshold.

[0043] Specifically, the method of generating the target logical form set according to the logical forms in the first logical form set whose relevance scores with the target question are greater than the score threshold may be: obtaining the logical forms in the first logical form set whose relevance scores with the target question are greater than the score threshold, and generating the target logical form set according to the logical forms in the first logical form set whose relevance scores with the target question are greater than the score threshold. The method of generating the target logical form set according to the logical forms in the first logical form set whose relevance scores with the target question are greater than the score threshold may also be: determining a candidate logical form set according to the entities and knowledge graphs corresponding to the target question; determining a second logical form set according to the first logical form set and the candidate logical form set; and generating the target logical form set according to the logical forms in the second logical form set whose relevance scores with the target question are greater than the score threshold.

[0044] S140, inputting the target logical form set and the target problem into a generation model to obtain a target logical form corresponding to the target problem.

[0045] Specifically, the target logical form set and the target problem are input into the generation model to obtain the target logical form corresponding to the target problem by pre-acquiring a trained generation model, splicing the target logical form set and the target problem, and inputting the spliced ​​target logical form set and the target problem into the generation model to obtain the target logical form corresponding to the target problem.

[0046] Optionally, generating a target logical form set according to logical forms in the first logical form set whose relevance scores to the target problem are greater than a score threshold includes:

[0047] Determine a set of candidate logical forms according to the entities and knowledge graphs corresponding to the target problem;

[0048] Determine a second logical form set according to the first logical form set and the candidate logical form set;

[0049] A target logical form set is generated according to the logical forms in the second logical form set whose relevance scores to the target question are greater than a score threshold.

[0050] Specifically, the method of determining the candidate logical form set according to the entity and knowledge graph corresponding to the target question may be: performing entity recognition on the target question to obtain the entity corresponding to the target question, and determining the candidate logical form set according to the entity and knowledge graph corresponding to the target question. For example, it may be that entity recognition is performed by determining whether the n-gram phrase corresponding to the span is an entity, obtaining the entity corresponding to the target question, and determining the candidate logical form set according to the entity and knowledge graph corresponding to the target question.

[0051] Specifically, the method of determining the second logical form set according to the first logical form set and the candidate logical form set may be: concatenating the first logical form set and the candidate logical form set to obtain the second logical form set.

[0052] Specifically, the method for generating a target logical form set based on the logical forms in the second logical form set whose correlation scores with the target problem are greater than a score threshold can be: obtaining the correlation scores between the logical forms in the second logical form set and the target problem, sorting the logical forms in the second logical form set in order of the correlation scores between the logical forms and the target problem from large to small, and generating a target logical form set based on the first K logical forms, where K is a positive integer greater than 2.

[0053] The embodiment of the present invention obtains the corresponding logical form through enumeration and case retrieval by analyzing the natural language question, and then performs entity modification on the logical form obtained by case retrieval, so as to ensure the execution rate of the introduced logical form. Finally, the enumerated logical form is expanded, and the modified logical form and the enumerated logical form are input as candidates into the sorter, and the final generator outputs the predicted logical form. The knowledge question and answer logical form determination method provided by the embodiment of the present invention includes the following steps:

[0054] Step 1: Use the existing entity linking method to identify entities from the target question, and use predefined rules to enumerate the logical forms of the associated edges within the preset hop count in the knowledge graph and the attributes of the associated entities within the preset hop count of the target entity in the knowledge graph to obtain a set of candidate logical forms.

[0055] Step 2: Select all cases in the training set as the case library, and then retrieve the initial set of cases related to the target problem from the case library.

[0056] Step 3: Perform entity replacement on the logical form in each case in the initial case set to avoid introducing entity noise in subsequent steps. Then perform relation modification on the logical form that has completed entity replacement to improve the execution rate of the logical form.

[0057] Step 4: Combine the first logical form set and the candidate logical form set into a second logical form set, and sort the logical forms in the second logical form set.

[0058] Step 5: Take the logical form most relevant to the target question as a candidate, concatenate the target question and the candidate logical form as the input of the generation model, and pass it into the generation model to obtain the target logical form.

[0059] Optionally, determining a set of candidate logical forms according to the entity and knowledge graph corresponding to the target question includes:

[0060] If the entity corresponding to the target question exists in the knowledge graph, the entity corresponding to the target question is mapped to the entity in the knowledge graph to obtain a set of candidate entities;

[0061] Disambiguating entities in the candidate entity set to obtain a target entity;

[0062] Logical form enumeration is performed based on the associated edges of the target entity within a preset number of hops in the knowledge graph and the attributes of the associated entities of the target entity within a preset number of hops in the knowledge graph to obtain a set of candidate logical forms.

[0063] Specifically, the knowledge graph is queried according to the entity mention corresponding to the target question. If the entity mention corresponding to the target question exists in the knowledge graph, it is determined that the entity corresponding to the target question exists in the knowledge graph.

[0064] The preset number of hops may be a pre-set number of hops.

[0065] Specifically, the entities in the candidate entity set are disambiguated to obtain the target entity by sorting the entities in the candidate entity set according to the scores corresponding to the entities, and selecting the entity with the highest score as the target entity.

[0066] In an embodiment of the present invention, first, named entity recognition technology is used to find entity mentions in the target question. Entity mentions are tag sequences in the target question, usually names or aliases corresponding to entities in the target question. The embodiment of the present invention searches all possible sentence spans based on the span method, and performs entity recognition by judging whether the n-gram phrase corresponding to the span is an entity, thereby covering the situation where nested entities may appear. At the same time, compared with the previous recognition method, the span-based method can improve the recall rate of entity mentions. Next, entity linking is divided into two parts: the first part is to map the entity in the target question to the entity in the knowledge graph by querying the dictionary to obtain a candidate entity set. Specifically, the FACC1 dictionary is used to perform entity retrieval and retrieve the candidate entity set; the second part is to disambiguate the entities in the candidate entity set, sort the entities in the candidate entity set, and select the entity with the highest score as the target entity. Finally, logical form enumeration is performed according to the attributes of the associated edges within the preset number of hops of the target entity in the knowledge graph and the associated entities within the preset number of hops of the target entity in the knowledge graph to obtain a candidate logical form set.

[0067] Optionally, the initial case library is screened according to the target problem to obtain an initial case set, including:

[0068] Perform entity masking on all questions in the initial case library to obtain the first case library;

[0069] Input all questions in the first case library into the encoder in sequence to obtain the representation features corresponding to the questions in the case;

[0070] Obtain the relationship similarity between the representation features corresponding to the problem in the case and the target problem;

[0071] Determine as a target case a case in which the relationship similarity between the problem in the case in the case library and the target problem is greater than a similarity threshold;

[0072] An initial case set is generated according to the target case in the initial case library.

[0073] It should be noted that the probability of entity masking for questions in the training stage and the detection stage is different, and the entity masking probability corresponding to the training stage is greater than the entity masking probability corresponding to the detection stage.

[0074] Specifically, all the questions in the first case library are sequentially input into the encoder to obtain the representation features corresponding to the questions in the cases. The method can be: the questions in all the cases in the first case library are sequentially input into the encoder to obtain the representation features corresponding to the questions in all the cases in the first case library.

[0075] In a specific example, all cases in the training set are selected as the case library, and an initial set of cases similar to the target problem is retrieved from the case library. The cases include: Problem q i The S-expression s corresponding to the problem i , the case is represented by (q i ,s i ). The questions in the retrieved cases have a high degree of relational similarity with the target questions, rather than entity similarity. Therefore, in order to reduce the impact of similar entities on the retrieval results, the mention recognition module is used to extract the entities in the questions. During training, the mention recognition module with probability p is used. mask The [BLANK] symbol shields the entity.

[0076] Similarly, the similarity between questions is measured by remote supervision, and the overlap of knowledge base relations between S-expressions is determined as the weight of similarity. Specifically, the weight is obtained by calculating the F1 score between the sets of relations existing in the corresponding S-expressions.

[0077] Optionally, generating a target logical form set according to logical forms in the second logical form set whose relevance scores to the target question are greater than a score threshold includes:

[0078] Inputting the second logical form set and the target question into a ranking model to obtain a correlation score between each logical form in the second logical form set and the target question;

[0079] A target logical form set is generated according to the logical forms in the second logical form set whose relevance scores to the target question are greater than a score threshold.

[0080] Among them, the sorting model can be a BERT-base model.

[0081] In a specific example, the candidate logical form set obtained by enumeration and the first logical form set obtained by retrieval and replacement are merged to obtain the second logical form set, thereby expanding the number and types of candidate logical forms obtained by the original enumeration. The logical forms in the second logical form set are sorted by the BERT-base model. In the test phase, the concatenated target question and the second logical form set are passed into the sorting model to obtain the correlation score between the two, and finally the top k logical forms with the highest scores are selected as the screened logical forms, that is, the target logical form set.

[0082] The target question and the target logical form set are concatenated as the input part of the generative model, and then passed into the generative model to obtain the final target logical form. The generative model is a generator based on the pre-trained model T5. In the test phase, the input part no longer requires positive and negative samples. Use execution to enhance the reasoning procedure. First, use beam search to decode the top-k logical forms, and then execute each logical form until a logical form that produces a valid (non-empty) answer is found. If none of the top k logical forms is valid, the highest-ranked candidate logical form obtained using the sorter is returned as the final logical form. This approach ensures that the final logical form is executable. This reasoning mode ensures that a valid logical form is found for each question, ensuring the execution rate of knowledge base queries.

[0083] Optionally, performing entity replacement on the logical form in each case in the initial case set according to the entity corresponding to the target problem to obtain a first logical form set includes:

[0084] Obtain the number of entities corresponding to the logical form in each case in the initial case set and the number of entities corresponding to the target question;

[0085] According to the number of entities corresponding to the logical form in each case in the initial case set and the number of entities corresponding to the target question, the logical form in each case in the initial case set is replaced with entities to obtain a first logical form set.

[0086] Specifically, the method for obtaining the number of entities corresponding to the logical form in each case in the initial case set and the number of entities corresponding to the target problem can be: obtaining the number of entities corresponding to the target problem through the entity linking module, performing regular item matching on the logical form through a pre-defined regular item template, and obtaining the number of entities corresponding to the logical form in each case in the initial case set.

[0087] It should be noted that the initial case set LF sim Compared with the real logical form, due to the masking strategy set, the similarities between the two are more concentrated in the relationship part, while the entity part is different. In order to avoid introducing noise from different entities in the subsequent process, the following strategy is used for LF sim Replace the logical form in the entity to complete the modification of the logical form. sim Take one of the logical forms to be replaced as an example, firstly, the entity E corresponding to the target question is obtained through the entity linking module. link The regular term matching of the logical form is performed through the pre-defined regular term template, so as to find the entity set E contained in the logical form find . Then according to E find The corresponding number of entities and Elink The corresponding number of entities is used to replace the logical form in each case in the initial case set to obtain a first logical form set.

[0088] E link and E find The domains they belong to in the knowledge base may be different, resulting in the relationship connected to the entity being similar in semantics, but the specific expression of the relationship in the knowledge base may be different. Therefore, in the logical form after entity replacement, the entity and relationship are not connected in the knowledge base, making the logical form unable to execute. Therefore, on the basis of entity replacement, the relationship of the logical form after entity replacement is modified to improve the execution rate of the logical form. The relationship modification strategy is: if the relationship contained in the logical form exists in the relationship path connecting the entity in the knowledge base, no modification is required;

[0089] If the relationship contained in the logical form does not exist in the relationship path connecting the entity in the knowledge base, similar relationships are generally semantically similar. Therefore, by querying the knowledge base to obtain all the relationships connected to the entity, Roberta-base is used to encode the candidate relationships and the relationships in the logical form, and the corresponding cosine similarities are calculated respectively, and then the relationship with the highest similarity to the relationship in the logical form is selected as the replacement relationship. Finally, each logical form obtained after replacement is converted into SPARQL for query in the knowledge base, and the candidate logical forms that cannot be executed to obtain results are discarded.

[0090] Optionally, the logical form in each case in the initial case set is replaced by entities according to the number of entities corresponding to the logical form in each case in the initial case set and the number of entities corresponding to the target question, to obtain a first logical form set, including:

[0091] Perform entity replacement on the logical form in each case in the initial case set based on the following rules to obtain the first logical form set:

[0092] If the number of entities corresponding to the logical form in each case in the initial case set is equal to the number of entities corresponding to the target question, then replace the entities corresponding to the logical form in each case in the initial case set with the entities corresponding to the target question in sequence;

[0093] If the number of entities corresponding to the logical form in each case in the initial case set is greater than the number of entities corresponding to the target question, then the entities corresponding to the logical form in each case in the initial case set are replaced with the entities corresponding to the target question in sequence, and the entities in the logical form in each case in the initial case set that have not been replaced are replaced with any entity corresponding to the target question;

[0094] If the number of entities corresponding to the logical form in each case in the initial case set is less than the number of entities corresponding to the target problem, the logical form is copied, and the entities corresponding to the logical form and the entities corresponding to the copied logical form are replaced with the entities corresponding to the target problem in sequence.

[0095] Optionally, if the number of entities corresponding to the logical form in each case in the initial case set is greater than the number of entities corresponding to the target problem, the entities corresponding to the logical form in each case in the initial case set are replaced with the entities corresponding to the target problem in sequence, and the entities in the logical form in each case in the initial case set that have not been replaced are replaced with the entities corresponding to the target problem. For example, if the number of entities corresponding to the logical form in the case is 2 (X, Y) and the number of entities corresponding to the target problem is 1 (A), then the entity X and the entity Y in the logical form are replaced with the entity A in the target problem.

[0096] Specifically, the method of copying the entities corresponding to the logical form and replacing the entities corresponding to the logical form and the entities corresponding to the copied logical form with the entities corresponding to the target problem in sequence can be: determining the number of copies according to the difference between the number of entities corresponding to the target problem and the number of entities corresponding to the logical form in each case in the initial case set and the number of entities corresponding to the logical form in the case, copying the logical form according to the number of copies, and replacing the entities corresponding to the logical form and the entities corresponding to the copied logical form with the entities corresponding to the target problem in sequence.

[0097] It should be noted that when determining the number of replications, it is necessary to consider the difference between the number of entities corresponding to the target problem and the number of entities corresponding to the logical form in each case in the initial case set, and also the number of entities corresponding to the logical form in the case. For example, if the difference is 4 and the number of entities corresponding to the logical form is 2, then the number of replications is 2.

[0098] For example, if the number of entities corresponding to the target problem is 3 (A, B, C), and the number of entities corresponding to the logical form in each case in the initial case set is 1 (X), then two logical forms are copied, and the entity X corresponding to the logical form is replaced with A, the entity X corresponding to the first copied logical form is replaced with B, and the entity X corresponding to the second copied logical form is replaced with C.

[0099] Generally, the number of entities corresponding to the logical form in each case in the initial case set and the number of entities corresponding to the target problem are 1 or 2. In a specific example, if the number of entities corresponding to the logical form in the case is 1 and the number of entities corresponding to the target problem is 1, the entities in the logical form are replaced with the entities in the target problem; if the number of entities corresponding to the logical form in the case is 2 and the number of entities corresponding to the target problem is 1, the entities in the logical form are replaced with the entities in the target problem; if the number of entities corresponding to the logical form in the case is 1 and the number of entities corresponding to the target problem is 2 (the entities corresponding to the target problem include: A and B), the logical form is copied, the entity corresponding to the original logical form is replaced with an entity (A) in the target problem, and the entity in the copied logical form is replaced with another entity (B) in the target problem; if the number of entities corresponding to the logical form in the case is 2 and the number of entities corresponding to the target problem is 2, the entities in the logical form are replaced with the entities in the target problem in the order of the entities in the logical form (if the entities in the logical form are entity X and entity Y from left to right, and the entities in the target problem are entity A and entity B from left to right, entity X is replaced with entity A, and entity Y is replaced with entity B).

[0100] In a specific example, the embodiment of the present invention uses Freebase as a knowledge base. Freebase is a common knowledge base in question-answering tasks, and its storage format is RDF format. The embodiment of the present invention uses the Virtuoso engine to store data and execute queries on the knowledge base. The knowledge base question-answering model is evaluated on the GrailQA and WebQSP datasets.

[0101] The evaluation indicators are EM and F1 value. EM represents the ratio of predicted logical form to true logical form in semantic similarity. TP is the number of correct positive predictions, FP is the number of wrong negative predictions, TN is the number of correct negative predictions, and FN is the number of wrong positive predictions. All values ​​are obtained using the official evaluation script, with F1 used as the main evaluation metric and EM value as a reference metric.

[0102] The method provided by the embodiment of the present invention is compared with the existing question-answering model, and the specific results are shown in Table 1. The experimental results show that the method provided by the embodiment of the present invention exceeds the performance of the existing question-answering model, proving the effectiveness of the method provided by the embodiment of the present invention:

[0103] Table 1

[0104]

[0105]

[0106] Among them, the bold content in the table is the best experimental result.

[0107] The embodiment of the present invention introduces a new entity linking method to improve the accuracy of entity recognition. The span-based method can improve the recall rate of entities. Entity recognition is the basis of entity linking and also determines the performance of the knowledge base question-answering model. Only by accurately identifying entities from questions can the performance of subsequent modules be guaranteed. The embodiment of the present invention uses the idea of ​​case retrieval to expand the candidate logical forms obtained by the enumerator, while ensuring the execution rate of the introduced logical forms, which can effectively improve the recall rate of the logical forms, thereby improving the performance of the model.

[0108] The technical solution of this embodiment is to obtain an initial case set by screening the initial case library according to the target problem, wherein the initial case set includes: at least one case, and the relationship similarity between the problem included in each case and the target problem is greater than a similarity threshold, and each case in the initial case set also includes: a logical form corresponding to the problem; the logical form in each case in the initial case set is replaced with an entity according to the entity corresponding to the target problem to obtain a first logical form set; a target logical form set is generated according to the logical forms in the first logical form set whose correlation scores with the target problem are greater than a score threshold; the target logical form set and the target problem are input into a generation model to obtain a target logical form corresponding to the target problem, which can enrich the types of candidate logical forms while modifying the expanded logical forms to ensure the execution rate of the logical forms.

[0109] Embodiment 2

[0110] Figure 2 This is a schematic diagram of the structure of a knowledge question and answer logic form determination device provided by an embodiment of the present invention. This embodiment can be applied to the situation of determining the logical form of knowledge questions and answers. The device can be implemented in software and / or hardware. The device can be integrated in any device that provides the function of determining the logical form of knowledge questions and answers, such as Figure 2 As shown, the knowledge question and answer logical form determination device specifically includes: an initial case set determination module 210, a first logical form set determination module 220, a target logical form set generation module 230 and a target logical form determination module 240.

[0111] The initial case set determination module is used to screen the initial case library according to the target problem to obtain an initial case set, wherein the initial case set includes: at least one case, the relationship similarity between the problem included in each case and the target problem is greater than the similarity threshold, and each case in the initial case set also includes: the logical form corresponding to the problem;

[0112] A first logic form set determination module, configured to perform entity replacement on the logic form in each case in the initial case set according to the entity corresponding to the target problem, to obtain a first logic form set;

[0113] a target logical form set generating module, configured to generate a target logical form set according to the logical forms in the first logical form set whose relevance scores with the target problem are greater than a score threshold;

[0114] The target logical form determination module is used to input the target logical form set and the target problem into the generation model to obtain the target logical form corresponding to the target problem.

[0115] Optionally, the target logical form set generation module is specifically used for:

[0116] Determine a set of candidate logical forms according to the entities and knowledge graphs corresponding to the target problem;

[0117] Determine a second logical form set according to the first logical form set and the candidate logical form set;

[0118] A target logical form set is generated according to the logical forms in the second logical form set whose relevance scores to the target question are greater than a score threshold.

[0119] Optionally, the target logical form set generation module is specifically used for:

[0120] If the entity corresponding to the target question exists in the knowledge graph, the entity corresponding to the target question is mapped to the entity in the knowledge graph to obtain a set of candidate entities;

[0121] Disambiguating entities in the candidate entity set to obtain a target entity;

[0122] Logical form enumeration is performed based on the associated edges of the target entity within a preset number of hops in the knowledge graph and the attributes of the associated entities of the target entity within a preset number of hops in the knowledge graph to obtain a set of candidate logical forms.

[0123] Optionally, the initial case set determination module is specifically used for:

[0124] Perform entity masking on all questions in the initial case library to obtain the first case library;

[0125] Input all questions in the first case library into the encoder in sequence to obtain the representation features corresponding to the questions in the case;

[0126] Obtain the relationship similarity between the representation features corresponding to the problem in the case and the target problem;

[0127] Determine as a target case a case in which the relationship similarity between the problem in the case in the case library and the target problem is greater than a similarity threshold;

[0128] An initial case set is generated according to the target case in the initial case library.

[0129] Optionally, the target logical form set generation module is specifically used for:

[0130] Inputting the second logical form set and the target question into a ranking model to obtain a correlation score between each logical form in the second logical form set and the target question;

[0131] A target logical form set is generated according to the logical forms in the second logical form set whose relevance scores to the target question are greater than a score threshold.

[0132] Optionally, the first logical form set determination module is specifically used for:

[0133] Obtain the number of entities corresponding to the logical form in each case in the initial case set and the number of entities corresponding to the target question;

[0134] According to the number of entities corresponding to the logical form in each case in the initial case set and the number of entities corresponding to the target question, the logical form in each case in the initial case set is replaced with entities to obtain a first logical form set.

[0135] Optionally, the first logical form set determination module is specifically used for:

[0136] Perform entity replacement on the logical form in each case in the initial case set based on the following rules to obtain the first logical form set:

[0137] If the number of entities corresponding to the logical form in each case in the initial case set is equal to the number of entities corresponding to the target question, then replace the entities corresponding to the logical form in each case in the initial case set with the entities corresponding to the target question in sequence;

[0138] If the number of entities corresponding to the logical form in each case in the initial case set is greater than the number of entities corresponding to the target question, then the entities corresponding to the logical form in each case in the initial case set are replaced with the entities corresponding to the target question in sequence, and the entities in the logical form in each case in the initial case set that have not been replaced are replaced with any entity corresponding to the target question;

[0139] If the number of entities corresponding to the logical form in each case in the initial case set is less than the number of entities corresponding to the target problem, the logical form is copied, and the entities corresponding to the logical form and the entities corresponding to the copied logical form are replaced with the entities corresponding to the target problem in sequence.

[0140] The above-mentioned product can execute the method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0141] Embodiment 3

[0142] Figure 3 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0143] like Figure 3 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0144] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0145] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the knowledge question and answer logic form determination method.

[0146] In some embodiments, the knowledge question and answer logic form determination method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the knowledge question and answer logic form determination method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the knowledge question and answer logic form determination method in any other appropriate manner (for example, by means of firmware).

[0147] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0148] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0149] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0150] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0151] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0152] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0153] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0154] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for determining the logical form of knowledge question answering, characterized in that: include: The initial case library is screened according to the target problem to obtain an initial case set, wherein the initial case set includes: at least one case, the relationship similarity between the problem included in each case and the target problem is greater than a similarity threshold, and each case in the initial case set also includes: a logical form corresponding to the problem; Perform entity replacement on the logical form in each case in the initial case set according to the entity corresponding to the target problem to obtain a first logical form set; Generate a target logical form set according to the logical forms in the first logical form set whose relevance scores to the target problem are greater than a score threshold; The target logical form set and the target problem are input into a generation model to obtain a target logical form corresponding to the target problem.

2. The method according to claim 1, characterized in that: Generating a target logical form set according to the logical forms in the first logical form set whose relevance scores to the target problem are greater than a score threshold comprises: Determine a set of candidate logical forms according to the entities and knowledge graphs corresponding to the target problem; Determine a second logical form set according to the first logical form set and the candidate logical form set; A target logical form set is generated according to the logical forms in the second logical form set whose relevance scores to the target question are greater than a score threshold.

3. The method according to claim 2, characterized in that Determine a set of candidate logical forms according to the entities and knowledge graphs corresponding to the target problem, including: If the entity corresponding to the target question exists in the knowledge graph, the entity corresponding to the target question is mapped to the entity in the knowledge graph to obtain a set of candidate entities; Disambiguating entities in the candidate entity set to obtain a target entity; Logical form enumeration is performed based on the associated edges of the target entity within a preset number of hops in the knowledge graph and the attributes of the associated entities of the target entity within a preset number of hops in the knowledge graph to obtain a set of candidate logical forms.

4. The method according to claim 1, characterized in that: The initial case library is screened according to the target problem to obtain an initial case set, including: Perform entity masking on all questions in the initial case library to obtain the first case library; Input all questions in the first case library into the encoder in sequence to obtain the representation features corresponding to the questions in the case; Obtain the relationship similarity between the representation features corresponding to the problem in the case and the target problem; Determine as a target case a case in which the relationship similarity between the problem in the case in the case library and the target problem is greater than a similarity threshold; An initial case set is generated according to the target case in the initial case library.

5. The method according to claim 2, characterized in that: Generating a target logical form set according to the logical forms in the second logical form set whose relevance scores to the target problem are greater than a score threshold comprises: Inputting the second logical form set and the target question into a ranking model to obtain a correlation score between each logical form in the second logical form set and the target question; A target logical form set is generated according to the logical forms in the second logical form set whose relevance scores to the target question are greater than a score threshold.

6. The method according to claim 1, characterized in that Perform entity replacement on the logical form in each case in the initial case set according to the entity corresponding to the target problem to obtain a first logical form set, including: Obtain the number of entities corresponding to the logical form in each case in the initial case set and the number of entities corresponding to the target question; According to the number of entities corresponding to the logical form in each case in the initial case set and the number of entities corresponding to the target question, the logical form in each case in the initial case set is replaced with entities to obtain a first logical form set.

7. The method according to claim 6, characterized in that According to the number of entities corresponding to the logical form in each case in the initial case set and the number of entities corresponding to the target question, the logical form in each case in the initial case set is replaced by an entity to obtain a first logical form set, including: The logical form in each case in the initial case set is replaced by entities based on the following rules to obtain the first logical form set: If the number of entities corresponding to the logical form in each case in the initial case set is equal to the number of entities corresponding to the target question, then replace the entities corresponding to the logical form in each case in the initial case set with the entities corresponding to the target question in sequence; If the number of entities corresponding to the logical form in each case in the initial case set is greater than the number of entities corresponding to the target question, then the entities corresponding to the logical form in each case in the initial case set are replaced with the entities corresponding to the target question in sequence, and the entities in the logical form in each case in the initial case set that have not been replaced are replaced with any entity corresponding to the target question; If the number of entities corresponding to the logical form in each case in the initial case set is less than the number of entities corresponding to the target problem, the logical form is copied, and the entities corresponding to the logical form and the entities corresponding to the copied logical form are replaced with the entities corresponding to the target problem in sequence.

8. A device for determining the logical form of knowledge question and answer, characterized in that: include: An initial case set determination module is used to screen the initial case library according to the target problem to obtain an initial case set, wherein the initial case set includes: at least one case, the relationship similarity between the problem included in each case and the target problem is greater than a similarity threshold, and each case in the initial case set also includes: a logical form corresponding to the problem; A first logic form set determination module, configured to perform entity replacement on the logic form in each case in the initial case set according to the entity corresponding to the target problem, to obtain a first logic form set; a target logical form set generating module, configured to generate a target logical form set according to the logical forms in the first logical form set whose relevance scores with the target problem are greater than a score threshold; The target logical form determination module is used to input the target logical form set and the target problem into the generation model to obtain the target logical form corresponding to the target problem.

9. The device according to claim 8, characterized in that The target logical form set generation module is specifically used for: Determine a set of candidate logical forms according to the entities and knowledge graphs corresponding to the target problem; Determine a second logical form set according to the first logical form set and the candidate logical form set; A target logical form set is generated according to the logical forms in the second logical form set whose relevance scores to the target question are greater than a score threshold.

10. The device according to claim 9, characterized in that The target logical form set generation module is specifically used for: If the entity corresponding to the target question exists in the knowledge graph, the entity corresponding to the target question is mapped to the entity in the knowledge graph to obtain a set of candidate entities; Disambiguating entities in the candidate entity set to obtain a target entity; Logical form enumeration is performed based on the associated edges of the target entity within a preset number of hops in the knowledge graph and the attributes of the associated entities of the target entity within a preset number of hops in the knowledge graph to obtain a set of candidate logical forms.

11. The device according to claim 8, characterized in that The initial case set determination module is specifically used for: Perform entity masking on all questions in the initial case library to obtain the first case library; Input all questions in the first case library into the encoder in sequence to obtain the representation features corresponding to the questions in the case; Obtain the relationship similarity between the representation features corresponding to the problem in the case and the target problem; Determine as a target case a case in which the relationship similarity between the problem in the case in the case library and the target problem is greater than a similarity threshold; An initial case set is generated according to the target case in the initial case library.

12. The device according to claim 9, characterized in that The target logical form set generation module is specifically used for: Inputting the second logical form set and the target question into a ranking model to obtain a correlation score between each logical form in the second logical form set and the target question; A target logical form set is generated according to the logical forms in the second logical form set whose relevance scores to the target question are greater than a score threshold.

13. The device according to claim 8, characterized in that The first logical form set determination module is specifically used for: Obtain the number of entities corresponding to the logical form in each case in the initial case set and the number of entities corresponding to the target question; According to the number of entities corresponding to the logical form in each case in the initial case set and the number of entities corresponding to the target question, the logical form in each case in the initial case set is replaced with entities to obtain a first logical form set.

14. The device according to claim 13, characterized in that The first logical form set determination module is specifically used for: The logical form in each case in the initial case set is replaced by entities based on the following rules to obtain the first logical form set: If the number of entities corresponding to the logical form in each case in the initial case set is equal to the number of entities corresponding to the target question, then replace the entities corresponding to the logical form in each case in the initial case set with the entities corresponding to the target question in sequence; If the number of entities corresponding to the logical form in each case in the initial case set is greater than the number of entities corresponding to the target question, then the entities corresponding to the logical form in each case in the initial case set are replaced with the entities corresponding to the target question in sequence, and the entities in the logical form in each case in the initial case set that have not been replaced are replaced with any entity corresponding to the target question; If the number of entities corresponding to the logical form in each case in the initial case set is less than the number of entities corresponding to the target problem, the logical form is copied, and the entities corresponding to the logical form and the entities corresponding to the copied logical form are replaced with the entities corresponding to the target problem in sequence.

15. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for determining the logical form of knowledge question and answer according to any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the knowledge question and answer logic form determination method described in any one of claims 1-7 when executed.

Images