A knowledge fusion-based reasoning question answering method

Abstract

The application discloses a kind of inference question and answer methods based on knowledge fusion, wherein the method includes: obtaining key entity and non-key entity in the problem text to be processed, and determining the first explanation text information corresponding to key entity and the second explanation text information corresponding to non-key entity;Determine the background knowledge vector to be used corresponding to the problem text to be processed based on key entity, non-key entity, first explanation text information and second text information;Determine the problem vector to be used corresponding to the problem text to be processed, and obtain target question vector based on the problem vector to be used and the background knowledge vector to be used;Determine at least one candidate answer entity corresponding to target question vector, and obtain target answer entity according to the answer evaluation attribute corresponding to each candidate answer entity, to determine the target answer corresponding to the problem text to be processed based on target answer entity.The effect of more accurate determination of the answer information corresponding to the input question is obtained.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine reading comprehension, and particularly relates to a reasoning question answering method based on knowledge fusion. BACKGROUND

[0002] With the rapid development of artificial intelligence, automatic question answering system is one of the visions of strong artificial intelligence.

[0003] At present, the automatic question answering system can be divided into two stages in general. The first stage is to match the question and answer from the pre-constructed knowledge base, and the second stage focuses on the computer's real understanding of the meaning of the question and the enhancement of the question answering reasoning ability by adding open knowledge. However, the question answering system in the first stage has a small application range and is high in accuracy and breadth of the question answering database, and the question answering system in the second stage is not accurate enough in understanding the question, resulting in an inaccurate answer.

[0004] In order to solve the above problems, the reasoning question answering method needs to be improved. SUMMARY

[0005] The present application provides a reasoning question answering method based on knowledge fusion to solve the problem that the answer is not accurate enough when the question answering system analyzes the input question because the background knowledge associated with the input question is ignored.

[0006] In a first aspect, an embodiment of the present application provides a reasoning question answering method based on knowledge fusion, comprising:

[0007] obtaining key entities and non-key entities in a to-be-processed question text, and determining first explanation text information corresponding to the key entities and second explanation text information corresponding to the non-key entities; wherein the key entities are matched with entities in a pre-set entity database;

[0008] determining a to-be-used background knowledge vector corresponding to the to-be-processed question text based on the key entities, the non-key entities, the first explanation text information and the second text information;

[0009] determining a to-be-used question vector corresponding to the to-be-processed question text, and obtaining a target question vector based on the to-be-used question vector and the to-be-used background knowledge vector;

[0010] determining at least one candidate answer entity corresponding to the target question vector, and obtaining a target answer entity according to answer evaluation attributes corresponding to each candidate answer entity, to determine a target answer corresponding to the to-be-processed question text based on the target answer entity.

[0011] The technical scheme of the embodiment acquires the key entity and the non-key entity in the to-be-processed question text, and determines the first explanation text information corresponding to the key entity and the second explanation text information corresponding to the non-key entity. In order to construct the background knowledge information corresponding to the to-be-processed question text, the entity recognition technology is used to extract the key entity and the non-key entity from the to-be-processed question text, and the first explanation text information corresponding to the key entity and the second explanation text information corresponding to the non-key entity are acquired from the explanation text database. Further, based on the key entity, the non-key entity, the first explanation text information and the second text information, the to-be-used background knowledge vector corresponding to the to-be-processed question text is determined, the to-be-matched triple is constructed based on the key entity and the non-key entity, the target triple is determined from the to-be-matched triple, and the target triple, the key entity, the non-key entity, the first explanation text information and the second text information are respectively subjected to vectorization processing to obtain corresponding vectors. According to the vectors and corresponding weights, the to-be-used background knowledge vector corresponding to the to-be-processed question text is obtained. In addition, the to-be-used question vector corresponding to the to-be-processed question text is determined, and the target question vector is obtained based on the to-be-used question vector and the to-be-used background knowledge vector. The to-be-processed question text is subjected to vector encoding based on the vector processing model to obtain the to-be-used question vector, and the to-be-used question vector and the to-be-used background knowledge vector are spliced to obtain the target question vector. At least one candidate answer entity corresponding to the target question vector is determined, and the target answer entity is obtained according to the answer evaluation attribute corresponding to each candidate answer entity, so as to determine the target answer corresponding to the to-be-processed question text based on the target answer entity. The target question vector is analyzed and processed by the pre-trained model, at least one candidate answer entity corresponding to the target question vector is determined from the answer entity set, each candidate answer entity is evaluated based on the answer evaluation model to obtain a corresponding answer evaluation attribute, and the candidate answer entity corresponding to the highest answer evaluation attribute is taken as the target answer entity, so as to determine the target answer corresponding to the to-be-processed question text based on the target answer entity. The problem that the answer obtained by the question and answer system when analyzing the input question is not accurate enough due to the neglect of the background knowledge associated with the input question is solved, and the effect of more accurately determining the answer information corresponding to the input question is obtained.

[0012] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the application, nor is it used to limit the scope of the application. Other features of the application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0013] In order to make the technical solutions in the embodiments of the present application clearer, the drawings needed in the embodiment description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0014] Figure 1 is a flow chart of a knowledge fusion-based reasoning question answering method according to an embodiment of the present application;

[0015] Figure 2 is a flow chart of a knowledge fusion-based reasoning question answering method according to an embodiment of the present application;

[0016] Figure 3 is a schematic diagram of determining a background knowledge vector to be used according to an ALBERT model according to an embodiment of the present application;

[0017] Figure 4 is a schematic diagram of determining a question vector to be used according to a GPT-2 model according to an embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to make those skilled in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort should be within the scope of protection of the present application.

[0019] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0020] Before the technical solutions are described, the application scenarios of the technical solutions will be briefly introduced so as to make the technical solutions more clearly understood.

[0021] Automatic question answering system is very important for the development of artificial intelligence, and the current automatic question answering system is generally divided into two categories: one is the question answering system based on knowledge base, which generally converts the problem input by the user into a database query instruction by using entity recognition technology combined with syntax analysis, and then queries in the corresponding database to output the answer. However, this kind of question answering method is very dependent on the establishment of the question answering database, and needs to establish a complete question answering database according to the limited field and application direction, so the precision and rigidity of the question answering database directly affect the performance of the question answering system, and the question answering system is usually applied to the professional limited field, and the application range is not extensive, if the problem input by the user exceeds the field limit, the question answering system will not be able to give an accurate answer. The second is the common sense reasoning question answering system, which generally performs semantic understanding on the problem and adds open knowledge to enhance the reasoning ability of the question answering system, but such question answering system usually lacks consideration of background knowledge.

[0022] Based on this, the technical scheme provides a reasoning question answering method based on knowledge fusion, which can be applied to an open knowledge-based question answering system. This kind of question answering system mainly focuses on the understanding of the problem itself, and does not act on the limited field. This kind of question answering system mainly includes three stages of problem analysis, information retrieval and answer extraction. In the problem analysis stage, the problem needs to be classified and the key information in the problem needs to be extracted. These key information is generally the entity information in the problem, which is usually completed by combining entity recognition technology. The information retrieval stage aims to construct the corresponding evidence for the problem. In the machine reading comprehension question answering task, a piece of background knowledge evidence is given to better understand the problem by associating the background knowledge with the problem in the answer extraction stage, so as to obtain the best matching answer from the answer knowledge base.

[0023] Embodiment one

[0024] Figure 1 A flowchart of a reasoning question answering method based on knowledge fusion is provided for the embodiment one of the present application. The embodiment can be applied to the case of adding background knowledge associated with the input question text to determine the most matching answer to the input question text based on the analysis of the input question text in the question answering system.

[0025] As Figure 1 shown, the method comprises:

[0026] S110, acquiring key entities and non-key entities in the problem text to be processed, and determining first explanation text information corresponding to the key entities and second explanation text information corresponding to the non-key entities.

[0027] The to-be-processed question text can be a question text input by a user in an open knowledge-based question answering system. For example, the to-be-processed question text can be "What movies are made by user A?" In the technical solution, the information in the to-be-processed question text is divided into key entities and non-key entities. A key entity is an entity that matches an entity in a pre-set entity database, and a non-key entity is an entity that does not match an entity in the pre-set entity database. For example, the pre-set entity database includes entity 1, entity 2, and entity 3, and the to-be-processed question text includes entity 1 and entity 4. Entity 1 is a key entity, and entity 4 is a non-key entity. That is, if an entity in the to-be-processed question text can be queried in the pre-set entity database, the entity can be regarded as a key entity, otherwise, the entity can be regarded as a non-key entity.

[0028] The first explanation text information can be understood as background knowledge information associated with the key entity, and the second explanation text information can be understood as background knowledge information associated with the non-key entity. For example, the first explanation text information corresponding to the key entity is taken as an example. The key entity is "chrysanthemum", and the corresponding first explanation text information can include plant properties, species, color characteristics, shape characteristics, odor characteristics, and related efficacy of chrysanthemum.

[0029] Specifically, the technical solution can be applied to an open knowledge-based question answering system. In actual application, based on the editing control provided by the question answering system, a to-be-processed question text can be input, and entities in the to-be-processed question text can be extracted. The entities are classified to obtain key entities and non-key entities in the to-be-processed question text. At the same time, in order to enable the question answering system to better understand the to-be-processed question text, first explanation text information corresponding to the key entity and second explanation text information corresponding to the non-key entity also need to be obtained.

[0030] Optionally, obtaining the key entities and the non-key entities in the to-be-processed question text includes: extracting at least one to-be-determined entity from the to-be-processed question text based on an entity recognition technology; determining whether a current entity matches an entity in a pre-set entity database for each to-be-determined entity; if yes, determining that the current entity is a key entity; and if no, determining that the current entity is a non-key entity.

[0031] The entity recognition technology is an information extraction technology, which can obtain entity data such as a person name and a place name from text data. In the technical solution, the entity recognition technology is used to extract the to-be-determined entity in the to-be-processed question text. For example, the to-be-processed question text is "user A likes to watch movie A", and based on the entity recognition technology, the to-be-processed question text is identified and extracted, and the to-be-determined entity is obtained as "user A" and "movie A". The entity database can be a self-defined entity database or an existing entity database. For example, the entity database can be a database provided by ConceptNet.

[0032] In actual application, at least one to-be-determined entity is extracted from the to-be-processed question text based on the entity recognition technology. Taking one of the to-be-determined entities as a current entity as an example, it is determined whether the current entity exists in the pre-set entity database. If the current entity exists, the current entity is determined as a key entity. Otherwise, if the current entity does not exist, the current entity is determined as a non-key entity.

[0033] Optionally, the first explanation text information corresponding to the key entity and the second explanation text information corresponding to the non-key entity are determined, including: determining the first explanation text information corresponding to the key entity and the second explanation text information corresponding to the non-key entity from the explanation text database based on the keyword detection technology.

[0034] The keyword detection technology can be understood as a technology for detecting keywords, and the explanation text database can be understood as a database for storing background knowledge corresponding to each entity. The explanation text database includes explanation text information associated with at least one entity.

[0035] Specifically, after the key entity and the non-key entity are determined, in order to obtain the first explanation text information corresponding to the key entity and the second explanation text information corresponding to the non-key entity, the key entity and the non-key entity are used as keywords, and it is determined whether the explanation text information associated with the keywords exists in the explanation text database based on the keyword detection technology. That is, in the technical solution, the key entity and the non-key entity in the to-be-processed question text can be used as keywords for detection.

[0036] S120, based on the key entity, the non-key entity, the first explanation text information and the second text information, determining a to-be-used background knowledge vector corresponding to the to-be-processed question text.

[0037] The to-be-used background knowledge vector can be used to represent the background knowledge associated with the to-be-processed question text.

[0038] In the technical solution, in order to make the question and answer system better understand the text meaning of the problem text to be processed, the first explanation text information corresponding to the key entity and the second explanation text information corresponding to the non-key entity are obtained. On this basis, the key entity, the non-key entity, the first explanation text information and the second text information are vectorized to obtain the corresponding to-be-used background knowledge vector, so that the question and answer system supplements the background knowledge of the problem text to be processed based on the to-be-used background knowledge vector.

[0039] In actual application, based on the key entity, the non-key entity, the first explanation text information and the second text information, the to-be-used background knowledge vector corresponding to the problem text to be processed is determined, including: based on at least one non-key entity and key entity, at least one to-be-matched triple is obtained; in the triple database, each to-be-matched triple is matched respectively, and the successfully matched to-be-matched triple is determined as a target triple; based on the target triple, the key entity, the non-key entity, the first explanation text information and the second text information, the to-be-used background knowledge vector corresponding to the problem text to be processed is determined.

[0040] In the technical solution, in order to make the question and answer system better understand the text meaning of the problem text to be processed, the first explanation text information corresponding to the key entity and the second explanation text information corresponding to the non-key entity are obtained. On this basis, the key entity, the non-key entity, the first explanation text information and the second text information are vectorized to obtain the corresponding to-be-used background knowledge vector, so that the question and answer system supplements the background knowledge of the problem text to be processed based on the to-be-used background knowledge vector.

[0041] Specifically, at least one key entity and at least one non-key entity are extracted from the problem text to be processed, for each key entity, taking one key entity as a current key entity as an example, at least one non-key entity associated with the current key entity is selected, and a corresponding to-be-matched triple is formed based on each non-key entity and the current key entity. Further, it is inquired whether there is a triple consistent with the to-be-matched triple in the pre-constructed triple database, if there is, the to-be-matched triple is determined as a target triple.

[0042] Further, after obtaining the target triple, based on the target triple, the key entity, the non-key entity, the first explanation text information and the second explanation text information, the to-be-used background knowledge vector corresponding to the problem text to be processed is obtained.

[0043] It should be noted that when matching each to-be-matched triple, the corresponding triple may not be included in the triple database, that is, the target triple that best represents the logical relationship between the current key entity and the non-key entity cannot be determined from the at least one to-be-matched triple. Based on this, if each to-be-matched triple does not exist in the triple database, a triple evaluation attribute corresponding to each to-be-matched triple is determined; by comparing each triple evaluation attribute, the to-be-matched triple corresponding to the highest triple evaluation attribute is determined as the target triple; and based on the target triple, the key entity, the non-key entity, the first explanation text information, and the second text information, a to-be-used background knowledge vector corresponding to the to-be-processed problem text is determined.

[0044] In order to determine the target triple that best represents the logical relationship between the current key entity and the non-key entity from each to-be-matched triple, each to-be-matched triple is evaluated, and the obtained evaluation value is taken as the corresponding triple evaluation attribute.

[0045] Specifically, the number of to-be-determined triples corresponding to the triples in the triple database containing the current key entity is determined, and the total number of triples in the triple database is determined, and based on the ratio of the total number of triples and the number of to-be-determined triples, a to-be-used weight corresponding to the to-be-matched triple corresponding to the current key entity is obtained. At the same time, according to the number of triples of the same triple category as the to-be-matched triple appearing in the triple database, the number of to-be-determined is obtained, and according to the ratio of the number of to-be-determined and the total number of triples, a to-be-determined weight corresponding to the to-be-matched triple containing the current key entity is obtained, so as to obtain the corresponding triple evaluation attribute based on the product of the to-be-used weight and the to-be-determined weight. It can be understood that based on a similar method, the triple evaluation attribute corresponding to each to-be-matched triple can be determined.

[0046] After obtaining the triple evaluation attribute corresponding to each to-be-matched triple, each triple evaluation attribute is compared to obtain the highest triple evaluation attribute, and the to-be-matched triple corresponding to the highest triple evaluation attribute is taken as the target triple. Further, based on the target triple, the key entity, the non-key entity, the first explanation text information, and the second text information, a to-be-used background knowledge vector corresponding to the to-be-processed problem text is determined.

[0047] Optionally, based on the target triple, the key entity, the non-key entity, the first explanation text information and the second text information, a to-be-used background knowledge vector corresponding to the to-be-processed question text is determined, including: respectively performing vector encoding on the target triple, the key entity, the non-key entity, the first explanation text information and the second text information to obtain a first vector, a second vector, a second vector, a third vector, a fourth vector and a fifth vector; based on the vectors and corresponding weights, a to-be-fused vector is determined; and the to-be-fused vectors are fused to obtain the to-be-used background knowledge vector.

[0048] In the technical solution, when the target triple, the key entity, the non-key entity, the first explanation text information and the second text information are respectively vector encoded, a byte pair encoding (BPE) mode can be used.

[0049] It can be understood that in the embedding control of BPE, all words are identified by fixed vectors. Since multiple sources and format background knowledge are introduced, and different background knowledge has different influences on the question and answer system, in order to distinguish the sources of the expanded knowledge, corresponding end embedding offsets can be added to the background knowledge, so that after vector encoding based on the BPE encoding mode, corresponding vectors are obtained. That is, based on the BPE encoding mode, a first vector corresponding to the target triple, a second vector corresponding to the key entity, a third vector corresponding to the non-key entity, a fourth vector corresponding to the first explanation text information, and a fifth vector corresponding to the second explanation text are obtained. Further, since different vectors have different influences on the question and answer system, weights corresponding to the vectors can be determined, so that based on the vectors and corresponding weights, a to-be-fused vector is obtained.

[0050] Specifically, based on the product of the first vector and the corresponding first weight, a first to-be-fused vector is obtained; based on the product of the second vector and the corresponding second weight, a second to-be-fused vector is obtained; based on the product of the third vector and the corresponding third weight, a third to-be-fused vector is obtained; based on the product of the fourth vector and the corresponding fourth weight, a fourth to-be-fused vector is obtained; and based on the product of the fifth vector and the corresponding fifth weight, a fifth to-be-fused vector is obtained. Further, the to-be-fused vectors are spliced to obtain the to-be-used background knowledge vector.

[0051] It should be noted that in the technical solution, the splicing order of the to-be-fused splicing vectors is: {the first to-be-spliced vector; the second to-be-spliced vector; the fourth to-be-spliced vector; the third to-be-spliced vector; the first to-be-spliced vector}. In other words, the technical solution is spliced in the order of the to-be-fused vectors corresponding to {the target triple; the key entity; the first explanation text information; the non-key entity; the second explanation text information}.

[0052] S130, determine a to-be-used question vector corresponding to the to-be-processed question text, and obtain a target question vector based on the to-be-used question vector and the to-be-used background knowledge vector.

[0053] The to-be-used question vector can be understood as a vector obtained by directly performing vector processing on the to-be-processed question text. The target question vector can be understood as a vector corresponding to the to-be-processed question text and associated background knowledge information.

[0054] Optionally, determining the to-be-used question vector corresponding to the to-be-processed question text, and obtaining the target question vector based on the to-be-used question vector and the to-be-used background knowledge vector, comprises: performing vector processing on the to-be-processed question text based on a vector processing model to obtain the to-be-used question vector; and splicing the to-be-used question vector and the to-be-used background knowledge vector to obtain the target question vector.

[0055] The vector processing model can be a GPT-2 pre-training model.

[0056] In actual application, the to-be-processed question text can be vector encoded based on the GPT-2 pre-training model to obtain the to-be-used question vector. Further, in order to enhance the analysis of the to-be-used question vector carried by the question and answer system, the to-be-used background knowledge vector is spliced based on the to-be-used question vector to obtain the target question vector.

[0057] The advantage of such a setting is that the target question vector not only includes the question semantic information corresponding to the to-be-processed question text, but also includes the background knowledge information associated with the to-be-processed question text. On this basis, when the question and answer system analyzes and processes the target question vector, it can more accurately understand the actual meaning expressed by the to-be-processed question text, so as to determine the best answer corresponding to the to-be-processed question text.

[0058] S140, determine at least one candidate answer entity corresponding to the target question vector, and obtain a target answer entity according to the answer evaluation attribute corresponding to each candidate answer entity, to determine a target answer corresponding to the to-be-processed question text based on the target answer entity.

[0059] In the technical solution, at least one candidate answer entity corresponding to the target problem vector can be determined according to the target problem vector. The candidate answer entity can be an entity determined according to a keyword in the candidate answer. The answer evaluation attribute can be used to represent the matching degree between the candidate answer entity and the target problem vector. The higher the answer evaluation attribute, the higher the matching degree between the candidate answer entity and the target problem vector. The lower the answer evaluation attribute, the lower the matching degree between the candidate answer entity and the target problem vector. The target answer entity can be understood as the candidate answer entity with the highest matching degree with the target problem. The target answer can be understood as the answer text information determined based on the target answer entity.

[0060] In actual application, the target problem vector can be input into the ALBERT model to perform vector analysis on the target problem vector based on the model, and at least one candidate answer entity corresponding to the target problem vector can be obtained. Further, the candidate answer entity with the highest matching degree with the target problem vector can be found from the candidate answer entities. Each candidate answer entity can be evaluated to obtain a corresponding answer evaluation attribute, and the target answer entity can be determined based on the answer evaluation attributes to determine the target answer corresponding to the problem text to be processed based on the target answer entity.

[0061] Optionally, at least one candidate answer entity corresponding to the target problem vector is determined, and the target answer entity is obtained according to the answer evaluation attributes corresponding to the candidate answer entities, including: determining at least one candidate answer entity corresponding to the target problem vector based on a pre-set answer entity set; performing entity evaluation on each candidate answer entity based on an answer evaluation model to obtain a corresponding answer evaluation attribute; and determining the candidate answer entity corresponding to the highest answer evaluation attribute as the target answer entity based on the answer evaluation attributes.

[0062] In the answer entity set, at least one answer entity is included, for example, the corresponding answer entity set can be constructed according to the answer entity in the concept network. The answer evaluation model can be understood as a model for evaluating the matching degree between each candidate answer entity and the target problem vector. Based on the answer evaluation model, the answer evaluation attribute corresponding to each candidate answer entity can be obtained.

[0063] Specifically, the target problem vector can be input into the ALBERT model to perform vector analysis on the target problem vector based on the model, and at least one candidate answer entity with a higher matching degree with the target problem vector can be obtained in the answer entity set. For example, the number of candidate answer entities can be pre-set, or a corresponding number of candidate answer entities can be determined as the candidate answer entities corresponding to the target problem vector from a large number of candidate answer entities according to a pre-set proportion.

[0064] Further, based on the answer evaluation model, each candidate answer entity is evaluated to obtain a corresponding answer evaluation attribute, and the candidate answer entity corresponding to the highest answer evaluation attribute is taken as the target answer entity.

[0065] The technical scheme of the embodiment acquires the key entity and the non-key entity in the to-be-processed question text, and determines the first explanation text information corresponding to the key entity and the second explanation text information corresponding to the non-key entity. In order to construct the background knowledge information corresponding to the to-be-processed question text, the entity recognition technology is used to extract the key entity and the non-key entity from the to-be-processed question text, and the first explanation text information corresponding to the key entity and the second explanation text information corresponding to the non-key entity are acquired from the explanation text database. Further, based on the key entity, the non-key entity, the first explanation text information and the second text information, the to-be-used background knowledge vector corresponding to the to-be-processed question text is determined, the to-be-matched triple is constructed based on the key entity and the non-key entity, the target triple is determined from the to-be-matched triple, and the target triple, the key entity, the non-key entity, the first explanation text information and the second text information are respectively vectorized to obtain corresponding vectors. According to each vector and the corresponding weight, the to-be-used background knowledge vector corresponding to the to-be-processed question text is obtained. In addition, the to-be-used question vector corresponding to the to-be-processed question text is determined, and the target question vector is obtained based on the to-be-used question vector and the to-be-used background knowledge vector. The vector processing model is used to perform vector coding on the to-be-processed question text to obtain the to-be-used question vector, and the to-be-used question vector and the to-be-used background knowledge vector are spliced to obtain the target question vector. At least one candidate answer entity corresponding to the target question vector is determined, and the target answer entity is obtained according to the answer evaluation attribute corresponding to each candidate answer entity, so as to determine the target answer corresponding to the to-be-processed question text based on the target answer entity. The target question vector is analyzed and processed by the pre-trained model, at least one candidate answer entity corresponding to the target question vector is determined from the answer entity set, each candidate answer entity is evaluated based on the answer evaluation model to obtain a corresponding answer evaluation attribute, and the candidate answer entity corresponding to the highest answer evaluation attribute is taken as the target answer entity, so as to determine the target answer corresponding to the to-be-processed question text based on the target answer entity. The problem that the answer obtained by the question and answer system when analyzing the input question is not accurate enough due to the neglect of the background knowledge associated with the input question is solved, and the effect of more accurately determining the answer information corresponding to the input question is achieved.

[0066] Embodiment two

[0067] In a specific example, as Figure 2As shown, the question text (i.e., the to-be-processed question text) is input in the editing control provided by the open knowledge-based question answering system, and in order to make the to-be-processed question text more standardized, the to-be-processed question text is preprocessed before entity recognition, such as removing hyphens, quotes or various special characters in the to-be-processed question text. Next, key entities and non-key entities are extracted from the to-be-processed question text based on entity recognition technology. Specifically, a pre-trained model BERT+ conditional random field CRF framework is used for sequence labeling of the question. In order to make the recognition result more accurate, an entity dictionary is constructed, all entities in the knowledge graph concept net (i.e., the entity database) are extracted to constitute the entity dictionary, the entities in the sequence labeling result are semantically matched with the entities in the entity dictionary, the successfully matched entities are marked as key entities, and other recognized entities are marked as ordinary entities (i.e., non-key entities). Taking the to-be-processed question text "A revolving door is convenient for two direction travel, but it also serves as a security measure at a what?" as an example, the entity recognition entities include "revolving door" and "security measure", wherein "revolving door" can be successfully matched in the concept net, so the entity can be determined as a key entity, and "security measure" is not successfully matched, so it is marked as an ordinary entity (i.e., a non-key entity).

[0068] Further, the background knowledge is automatically constructed in the open corpus according to the key entity. In order to make the background knowledge meet the common sense requirement, the structured knowledge graph concept net constructed by artificial and the Wikipedia dictionary containing extensive explanatory knowledge can be selected as the expanded corpus.

[0069] Specifically, the to-be-matched triple corresponding to the key entity is constructed. Starting from the key entity, a triple method based on relationship weight is used to select the triple most associated with the to-be-processed question text. The triple set in the concept net is denoted as C, e q represents the key entity in the question, e c represents the ordinary entity in the question, and the specific process is as follows:

[0070] 1) If there is an edge relationship r, such that (e q , r, e c ) ∈ C, the triple (e q , r, e c) As a result. That is, the to-be-matched triple is matched in the pre-set triple database, and if the to-be-matched triple is contained in the triple database, the to-be-matched triple is determined as the target triple.

[0071] 2) If the to-be-matched triple is not contained in the triple database, all to-be-matched triples containing e q are reselected, the total number is N, and a score s j is calculated for the jth triple, which is obtained by multiplying the weight w j of the triple in the concept network (i.e., the weight to be used) by the weight of the current relationship (i.e., the weight to be determined) based on the current relationship. The calculation of the relationship weight is shown in the following formula:

[0072]

[0073] wherein r j represents the relationship type of the jth triple, represents the number of times of occurrence of the relationship r q in the N triples containing e j (i.e., the number of to-be-determined triples), and N represents the number of triples containing e q in the triple database.

[0074] It can be understood that, the smaller the s j is and the higher the w j is, the stronger the background knowledge and common sense implied by the triple and the lower the noise are. The w j may represent the confidence corresponding to the to-be-matched triple.

[0075] Based on the following formula, the triple evaluation attribute corresponding to the to-be-matched triple is determined:

[0076]

[0077] wherein s j represents the triple evaluation attribute, w j represents the weight to be used, and w j represents the weight to be determined.

[0078] Based on a similar method, the triple evaluation attribute corresponding to each to-be-matched triple can be obtained, and the to-be-matched triple corresponding to the highest triple evaluation attribute is taken as the target triple.

[0079] Meanwhile, for the unstructured dictionary, a request header is directly constructed for each key entity or non-key entity to call the API of the Wiki dictionary, the access rate is limited to 5 times per second, the explanation text information corresponding to the entity is obtained, and then the key-value pair is saved in the form of a json file. That is, based on the keyword detection technology, the first explanation text information corresponding to the key entity and the second explanation text information corresponding to the non-key entity are determined.

[0080] Further, in order to add the corresponding background knowledge information on the basis of the problem text to be processed, a text sequence is constructed based on the target triple, the key entity, the first explanation text information, the non-key entity and the second explanation text information, and the text sequence is subjected to vectorization processing to obtain a corresponding vector.

[0081] Specifically, a [CLS] marker is inserted in the head according to the standard input format of the pre-training model, which can represent the semantic information of the entire sequence in the pre-training model, then the triple extracted from the concept net is inserted, a [SEP] separator is inserted, and then the key entity, the first explanation text information, the non-key entity and the second explanation text information are inserted in turn and separated by the [SEP] marker. After the text sequence is constructed, each text sub-sequence in the text sequence is subjected to vector encoding based on the BPE encoding method in the ALBERT model to obtain a first vector corresponding to the target triple, a second vector corresponding to the key entity, a third vector corresponding to the non-key entity, a fourth vector corresponding to the first explanation text information, and a fifth vector corresponding to the second explanation text. Further, a segment embedding offset vector is added to each vector, so that the ALBERT model obtains a first to-be-fused vector based on the product of the first vector and the corresponding first weight, obtains a second to-be-fused vector based on the product of the second vector and the corresponding second weight, obtains a third to-be-fused vector based on the product of the third vector and the corresponding third weight, obtains a fourth to-be-fused vector based on the product of the fourth vector and the corresponding fourth weight, and obtains a fifth to-be-fused vector based on the product of the fifth vector and the corresponding fifth weight, and the fused vectors are spliced to obtain a to-be-used background knowledge vector.

[0082] Specifically, as shown in Figure 3 , using the [CLS] marker to correspond to the semantic information will lose a lot of background knowledge features, so feature fusion needs to be performed on the feature vector matrix to simultaneously retain the knowledge graph triple information, the key entity dictionary explanation information and the ordinary entity dictionary explanation information. It is assumed that the text embedding representation of the final hidden layer of the ALBERT model is X=(x0, x1,..., x m ), where x i ∈R dd is the dimension of the word embedding vector. By fitting the attention mechanism of human thinking characteristics, we can analyze the focus of the text vector. Adding an attention mechanism before the ALBERT output vector and linear transformation to perform weighted summation of different knowledge features can effectively improve the efficiency of knowledge utilization. The specific method is shown in the following formula.

[0083] First, we introduce a randomly initialized parameter vector u, u∈R. d All embedding vectors x i After multiplying and performing a linear transformation, the result is fed into the normalized function softmax to obtain the probability, which serves as the attention weight α for each embedding vector. i Taking key entities as an example:

[0084] α i =softmax(u T x i )

[0085] Where, α i The second weight represents the key entity, u represents the initialization vector, and x represents the second weight. i This represents the second vector corresponding to the key entity, and softmax represents the normalization function.

[0086] Then, the attention weight value of each vector is used to perform feature fusion, which is represented by a weighted summation of the embedded feature vector matrix.

[0087]

[0088] Here, vector g represents the final fused background knowledge vector to be used, and α i x represents the second weight corresponding to the key entity. i Let m represent the second vector corresponding to the key entity, m represent the number of vectors, and i represent the i-th vector.

[0089] Simultaneously, the text of the question to be processed is vectorized based on the GPT-2 pre-trained model to obtain the question vector to be used. Furthermore, to enhance the analysis of the information carried in the question vector to be used by the question answering system, a background knowledge vector to be used is concatenated with the question vector to obtain the target question vector. Specifically, as follows... Figure 4 As shown, [START] and markers are first added to the beginning and end of the question. The one-way GPT-2 can fit human reading and comprehension habits and use the vector q corresponding to the marker to generate the answer to the question. Here, q represents the question vector to be used. At the same time, incorporating background knowledge can effectively improve the model's question-answering ability and improve the quality and accuracy of the answer. Based on the question vector to be used, the background knowledge vector to be used is concatenated to obtain the target question vector v = [q; l].

[0090] Furthermore, the target question vector can be input into the ALBERT model to perform vector analysis on the target question vector, thereby obtaining at least one candidate answer entity from the answer entity set that has a high degree of matching with the target question vector. For example, the number of candidate answer entities can be preset, or a corresponding number of candidate answer entities can be selected from a large number of candidate answer entities according to a preset ratio as candidate answer entities corresponding to the target question vector.

[0091] Furthermore, based on the answer evaluation model, each candidate answer entity is evaluated to obtain the corresponding answer evaluation attributes, and the candidate answer entity corresponding to the highest answer evaluation attribute is taken as the target answer entity.

[0092] Specifically, the target question vector v is mapped to the set of answer entities, where w is a randomly initialized linear transformation parameter matrix w∈R. d*k d is the dimension of the language model feature vector, which can be set to 768 in the ALBERT model, k is the number of candidate answers, and b is a bias vector.

[0093] s = v T w+b

[0094] Where s represents the target problem vector, v represents the target problem vector, w represents the linear transformation parameter matrix, and b represents the bias vector.

[0095] To effectively evaluate answer entities, scores need to be normalized so that the feature values ​​of each answer entity's score are compressed to the range (0, 1), and the sum of the feature values ​​of all answer entities is 1. This feature value represents the probability that each answer entity is the correct answer. A commonly used normalization function is softmax, an exponential normalization function that calculates the final probability y of each entity's output for the question. i The calculation is shown in the following formula.

[0096]

[0097] Among them, y i Let represent the probability that the i-th candidate answer is the correct answer. This represents the score corresponding to the i-th candidate answer entity. This represents the sum of scores corresponding to the n candidate answer entities.

[0098] Let the final evaluation probability of the answer be denoted as Y = {y1, y2, ..., y...} n} Generate sample label vectors based on the answers provided for each question in the training and validation sets of the commonsense reasoning dataset. If the answer to the question is the second entity in the answer entity set, then the generated label vector To train the model, the present technical solution uses a cross-entropy loss function to evaluate the difference between the label sample and the prediction result, and the definition of the cross-entropy loss function is shown in the following formula:

[0099]

[0100] where y j is the probability that the jth answer is the correct answer, is 1 or 0, depending on whether the jth answer is the correct answer.

[0101] Minimizing the cross-entropy loss function between the label sample and the prediction result is the goal of the present model training. Finally, the backpropagation error adjusts the parameters in the pre-trained model GPT-2, ALBERT, feature fusion module and answer reasoning model, and completes the training of the entire reasoning question and answer model. In the final answer prediction, the answer corresponding to the answer entity with the maximum probability (i.e., the target answer entity) is directly input as the answer to the question (i.e., the target answer).

[0102] The technical scheme of the embodiment acquires the key entity and the non-key entity in the to-be-processed question text, and determines the first explanation text information corresponding to the key entity and the second explanation text information corresponding to the non-key entity. In order to construct the background knowledge information corresponding to the to-be-processed question text, the entity recognition technology is used to extract the key entity and the non-key entity from the to-be-processed question text, and the first explanation text information corresponding to the key entity and the second explanation text information corresponding to the non-key entity are acquired from the explanation text database. Further, based on the key entity, the non-key entity, the first explanation text information and the second text information, the to-be-used background knowledge vector corresponding to the to-be-processed question text is determined, the to-be-matched triple is constructed based on the key entity and the non-key entity, the target triple is determined from the to-be-matched triple, and the target triple, the key entity, the non-key entity, the first explanation text information and the second text information are respectively subjected to vectorization processing to obtain corresponding vectors. According to the vectors and corresponding weights, the to-be-used background knowledge vector corresponding to the to-be-processed question text is obtained. In addition, the to-be-used question vector corresponding to the to-be-processed question text is determined, and the target question vector is obtained based on the to-be-used question vector and the to-be-used background knowledge vector. The to-be-processed question text is subjected to vector encoding based on the vector processing model to obtain the to-be-used question vector, and the to-be-used question vector and the to-be-used background knowledge vector are spliced to obtain the target question vector. At least one candidate answer entity corresponding to the target question vector is determined, and the target answer entity is obtained according to the answer evaluation attribute corresponding to each candidate answer entity, so as to determine the target answer corresponding to the to-be-processed question text based on the target answer entity. The target question vector is analyzed and processed by the pre-trained model, at least one candidate answer entity corresponding to the target question vector is determined from the answer entity set, each candidate answer entity is evaluated based on the answer evaluation model to obtain a corresponding answer evaluation attribute, and the candidate answer entity corresponding to the highest answer evaluation attribute is taken as the target answer entity, so as to determine the target answer corresponding to the to-be-processed question text based on the target answer entity. The problem that the answer obtained by the question and answer system when analyzing the input question is not accurate enough due to the neglect of the background knowledge associated with the input question is solved, and the effect of more accurately determining the answer information corresponding to the input question is achieved.

[0103] It should be understood that the steps can be reordered, added, or deleted using the various forms of flowcharts shown above. For example, the steps described in the present application can be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical scheme of the present application can be achieved, which is not limited herein.

[0104] The above detailed description does not limit the scope of the application. Various modifications, combinations, sub-combinations and alternatives can be made to the detailed embodiment within the scope of the application. Any modification, equivalent replacement and improvement made without departing from the spirit and principle of the application shall fall within the scope of the application.

Claims

1. A knowledge fusion-based inference question answering method, characterized in that, The method comprises the following steps: obtaining key entities and non-key entities in a to-be-processed question text, and determining first explanation text information corresponding to the key entities and second explanation text information corresponding to the non-key entities; wherein the key entities are matched with entities in a pre-set entity database; determining a to-be-used background knowledge vector corresponding to the to-be-processed question text based on the key entities, the non-key entities, the first explanation text information, and the second explanation text information; determining a to-be-used question vector corresponding to the to-be-processed question text, and obtaining a target question vector based on the to-be-used question vector and the to-be-used background knowledge vector; determining at least one candidate answer entity corresponding to the target question vector, and obtaining a target answer entity according to answer evaluation attributes corresponding to each candidate answer entity, so as to determine a target answer corresponding to the to-be-processed question text based on the target answer entity; the step of determining a to-be-used background knowledge vector corresponding to the to-be-processed question text based on the key entities, the non-key entities, the first explanation text information, and the second explanation text information comprises the following steps: obtaining at least one to-be-matched triple based on at least one non-key entity and the key entities; performing triple matching on each to-be-matched triple in a triple database, and determining a successfully-matched to-be-matched triple as a target triple; determining a to-be-used background knowledge vector corresponding to the to-be-processed question text based on the target triple, the key entities, the non-key entities, the first explanation text information, and the second explanation text information.

2. The method of claim 1, wherein, the step of obtaining key entities and non-key entities in a to-be-processed question text comprises the following steps: extracting at least one to-be-determined entity from the to-be-processed question text based on an entity recognition technology; determining whether a current entity is matched with an entity in a pre-set entity database for each to-be-determined entity; if yes, determining that the current entity is a key entity; if no, determining that the current entity is a non-key entity.

3. The method of claim 1, wherein, the step of determining first explanation text information associated with the key entities and second explanation text information corresponding to the non-key entities comprises the following steps: determining first explanation text information corresponding to the key entities and second explanation text information corresponding to the non-key entities from an explanation text database according to a keyword detection technology; wherein the explanation text database comprises explanation text information associated with at least one entity.

4. The method of claim 1, wherein, the method further comprises the following steps: if each to-be-matched triple does not exist in the triple database, determining triple evaluation attributes corresponding to each to-be-matched triple; comparing each triple evaluation attribute, and determining a to-be-matched triple corresponding to the highest triple evaluation attribute as a target triple; determining a to-be-used background knowledge vector corresponding to the to-be-processed question text based on the target triple, the key entities, the non-key entities, the first explanation text information, and the second explanation text information.

5. The method according to claim 1 or 4, characterized in that, The determining, based on the target triple, the key entity, the non-key entity, the first explanation text information and the second explanation text information, of a to-be-used background knowledge vector corresponding to the to-be-processed question text comprises: vector encoding is performed on the target triple, the key entity, the non-key entity, the first explanation text information and the second explanation text information respectively to obtain a first vector, a second vector, a second vector, a third vector, a fourth vector and a fifth vector; based on the vectors and corresponding weights, a to-be-fused vector is determined; the to-be-fused vectors are fused to obtain the to-be-used background knowledge vector.

6. The method of claim 1, wherein, The determining, based on the target triple, the key entity, the non-key entity, the first explanation text information and the second explanation text information, of a to-be-used background knowledge vector corresponding to the to-be-processed question text comprises: based on a vector processing model, the to-be-processed question text is vectorized to obtain a to-be-used question vector; the to-be-used question vector and the to-be-used background knowledge vector are spliced to obtain a target question vector.

7. The method of claim 1, wherein, The determining, based on the target triple, the key entity, the non-key entity, the first explanation text information and the second explanation text information, of a to-be-used background knowledge vector corresponding to the to-be-processed question text comprises: based on a vector processing model, the to-be-processed question text is vectorized to obtain a to-be-used question vector; the to-be-used question vector and the to-be-used background knowledge vector are spliced to obtain a target question vector. The determining, based on the target triple, the key entity, the non-key entity, the first explanation text information and the second explanation text information, of a to-be-used background knowledge vector corresponding to the to-be-processed question text comprises: based on a vector processing model, the to-be-processed question text is vectorized to obtain a to-be-used question vector; the to-be-used question vector and the to-be-used background knowledge vector are spliced to obtain a target question vector. The determining, based on the target triple, the key entity, the non-key entity, the first explanation text information and the second explanation text information, of a to-be-used background knowledge vector corresponding to the to-be-processed question text comprises: based on a vector processing model, the to-be-processed question text is vectorized to obtain a to-be-used question vector; the to-be-used question vector and the to-be-used background knowledge vector are spliced to obtain a target question vector.

Images