Answer text generation method and device and storage medium

Abstract

The invention provides an answer text generation method and device and a storage medium, relates to the technical field of artificial intelligence, and can automatically perform entity representation and semantic representation matching according to a question text and generate an answer text with relatively high accuracy according to a matching result. The method comprises the steps of obtaining entity representation and semantic representation in a target question text; determining first text information according to the entity representation, and determining second text information according to the semantic representation; and generating an answer text of the target question text according to the first text information and the second text information.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device and storage medium for generating an answer text. Background Art

[0002] With the development of artificial intelligence, people have higher and higher requirements for the convenience of knowledge acquisition and information interaction. As an important intelligent application, the knowledge base question answering system aims to help users quickly and accurately obtain the required knowledge and answers from the massive knowledge base.

[0003] At present, knowledge base question and answer requires additional reliance on manpower or pre-labeling to extract entities and entity relationships. In small and medium-sized enterprise departments and personal question and answer assistant scenarios, there is a lack of manpower or pre-made templates for entity and entity relationship extraction, which makes it difficult to promote application scenarios. Summary of the invention

[0004] The present application provides a method, device and storage medium for generating an answer text, which can automatically match entity representation and semantic representation according to the question text, and generate an answer text according to the matching results.

[0005] In order to achieve the above objectives, this application adopts the following technical solutions:

[0006] In a first aspect, the present application provides a method for generating an answer text, the method comprising:

[0007] Obtain entity representation and semantic representation in the target question text; determine first text information according to the entity representation, and determine second text information according to the semantic representation; generate an answer text for the target question text according to the first text information and the second text information.

[0008] According to the above-mentioned technical means, the present application can match the first text information and the second text information related to the target question text according to the entity representation and the semantic representation corresponding to the target question text, and generate an answer text based on the first text information and the second text information. The matching of the two representations can make the generated answer text more accurate.

[0009] In a possible implementation, a target node representation among multiple node representations in a preset hypergraph and text information corresponding to the target node representation are determined, and the similarity between the target node representation and the entity representation is greater than a first preset threshold; the preset hypergraph is constructed based on multiple text information; and the text information corresponding to the target node representation is used as the first text information.

[0010] According to the above-mentioned technical means, the present application can determine a target node representation that is similar to the entity representation based on the entity representation, and use the text information corresponding to the target node representation as the first text information for the subsequent generation of the answer text. The accuracy of the generated answer text is improved through the deep understanding of the preset knowledge base text contained in the target node representation.

[0011] In one possible implementation, the similarity between the semantic representation of each text in a plurality of texts in a preset knowledge base and the semantic representation of a target question text is determined; the target text in the plurality of texts is used as the second text information; and the similarity between the semantic representation of the target text and the semantic representation of the target question text is greater than a second preset threshold.

[0012] According to the above-mentioned technical means, the present application can match similar texts in a preset knowledge base according to semantic representation, and use the text as the second text information for subsequent generation of answer text, providing a second method for obtaining the data basis for generating answer text, and discovering deep connections through semantic representation, thereby improving the accuracy of the answer text.

[0013] In a possible implementation, semantic feature information in a knowledge base graph is obtained, and the semantic feature information includes semantic space features and entity attribute features; the knowledge base graph is constructed based on multiple texts of a preset knowledge base; a preset hypergraph is constructed based on the semantic feature information; the preset hypergraph includes multiple hyperedges, and the hyperedges include multiple nodes with association relationships.

[0014] According to the above-mentioned technical means, the present application can construct a preset hypergraph by constructing multiple hyperedges, and use the preset hypergraph to understand the deeper relationships of the knowledge in the preset knowledge base, deepen the mining of the relationships between multiple nodes, so that subsequent deep models can have a more comprehensive understanding of the knowledge base.

[0015] In one possible implementation, attribute information of multiple texts of a preset knowledge base is obtained, the attribute information including entity information in the text, relationships between entities, entity attributes, and text paragraph information; based on the attribute information, a knowledge base graph is constructed, the knowledge base graph includes multiple nodes and association relationships between nodes; the multiple nodes include entities, sentences, paragraphs, and documents, and the association relationships include relationships between entities and text paragraph information corresponding to the nodes.

[0016] According to the above technical means, the present application can construct a knowledge base graph by presetting the attribute information of the knowledge base text, and can clearly present the knowledge structure of the knowledge base text in natural language form in a visual way, which is convenient for mining potential knowledge associations and improving the efficiency of knowledge utilization.

[0017] In one possible implementation, a preset hypergraph is sampled to obtain multiple samples; a deep model is constructed based on the preset hypergraph and the multiple samples, and the deep model includes node representations corresponding to multiple nodes of the preset hypergraph.

[0018] According to the above-mentioned technical means, the present application can construct and train an optimized deep model based on a hypergraph and samples, thereby providing a way to obtain node representation, and the node representation has a deeper understanding of the relationship between multiple nodes in the hypergraph, so that when faced with complex question texts, the text information corresponding to nodes with higher similarity can still be matched.

[0019] In a possible implementation, a meta-path is set according to the node type in a preset hypergraph, and based on the meta-path, a graph walk is performed on the preset hypergraph to obtain a walk sequence; the walk sequence includes multiple first samples; the first sample includes nodes with an associated relationship; based on a heterogeneous negative sampling method, negative sampling is performed on the preset hypergraph to obtain multiple second samples; the second sample includes nodes with no associated relationship; wherein the multiple samples include multiple first samples and multiple second samples.

[0020] According to the above-mentioned technical means, the present application can obtain multiple samples, and multiple models include multiple first samples and multiple second samples. There are multiple nodes associated with the first samples and multiple nodes not associated with the second samples. In the subsequent deep model optimization, the first samples can be used to enable the deep model to have a deeper understanding of the relationship between the multiple nodes, and the second samples can be used to improve the generalization ability of the deep model.

[0021] In a second aspect, the present application provides an answer text generating device, which includes: a communication unit and a processing unit.

[0022] A communication unit, used to obtain entity representation and semantic representation in the target question text;

[0023] A processing unit, configured to determine the first text information according to the entity representation, and determine the second text information according to the semantic representation;

[0024] The processing unit is further used to generate an answer text of the target question text according to the first text information and the second text information.

[0025] In one possible implementation, the processing unit is also used to determine a target node representation among multiple node representations in a preset hypergraph and text information corresponding to the target node representation, and the similarity between the target node representation and the entity representation is greater than a first preset threshold; the preset hypergraph is constructed based on multiple text information; and the text information corresponding to the target node representation is used as the first text information.

[0026] In one possible implementation, the processing unit is also used to determine the similarity between the semantic representation of each text in multiple texts of a preset knowledge base and the semantic representation of the target question text; use the target text in the multiple texts as the second text information; the similarity between the semantic representation of the target text and the semantic representation of the target question text is greater than a second preset threshold.

[0027] In one possible implementation, the processing unit is also used to obtain semantic feature information in the knowledge base graph, and the semantic feature information includes semantic space features and entity attribute features; the knowledge base graph is constructed based on multiple texts of a preset knowledge base; the processing unit is also used to construct a preset hypergraph based on the semantic feature information; the preset hypergraph includes multiple hyperedges, and the hyperedges include multiple nodes with associated relationships.

[0028] In one possible implementation, the processing unit is further used to obtain attribute information of multiple texts of a preset knowledge base, the attribute information including entity information in the text, relationships between entities, entity attributes, and text paragraph information; the processing unit is further used to construct a knowledge base graph based on the attribute information, the knowledge base graph including multiple nodes and association relationships between nodes; the multiple nodes include entities, sentences, paragraphs, and documents, and the association relationships include relationships between entities and text paragraph information corresponding to the nodes.

[0029] In one possible implementation, the processing unit is also used to sample a preset hypergraph to obtain multiple samples; the processing unit is also used to build a deep model based on the preset hypergraph and the multiple samples, and the deep model includes node representations corresponding to multiple nodes of the preset hypergraph.

[0030] In one possible implementation, the processing unit is further used to set a meta-path according to the node type in a preset hypergraph, and based on the meta-path, perform a graph walk on the preset hypergraph to obtain a walk sequence; the walk sequence includes multiple first samples; the first sample includes nodes with an associated relationship; the processing unit is further used to perform negative sampling on the preset hypergraph based on a heterogeneous negative sampling method to obtain multiple second samples; the second sample includes nodes with no associated relationship; wherein the multiple samples include multiple first samples and multiple second samples.

[0031] In a third aspect, the present application provides an electronic device comprising: a processor and a communication interface; the communication interface and the processor are coupled, and the processor is used to run a computer program or instructions to implement the answer text generation method described in the first aspect and any possible implementation method of the first aspect.

[0032] In a fourth aspect, the present application provides a computer-readable storage medium, which stores instructions. When the instructions are executed on a terminal, the terminal executes the answer text generation method described in the first aspect and any possible implementation of the first aspect.

[0033] In a fifth aspect, the present application provides a computer program product comprising instructions, which, when executed on an electronic device, enables the electronic device to execute the answer text generation method as described in the first aspect and any possible implementation of the first aspect.

[0034] In a sixth aspect, the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor is used to run a computer program or instructions to implement the answer text generation method described in the first aspect and any possible implementation method of the first aspect.

[0035] Specifically, the chip provided in the present application also includes a memory for storing computer programs or instructions. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A process of generating an answer text provided in an embodiment of the present application Figure 1 ;

[0037] Figure 2 A process of generating an answer text provided in an embodiment of the present application Figure 2 ;

[0038] Figure 3 A process of generating an answer text provided in an embodiment of the present application Figure 3 ;

[0039] Figure 4 A process of generating an answer text provided in an embodiment of the present application Figure 4 ;

[0040] Figure 5 A schematic diagram of a method for generating an answer text provided in an embodiment of the present application;

[0041] Figure 6 A schematic diagram of the structure of an answer text generating device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] The answer text generation method, device and storage medium provided in the embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0043] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0044] The terms "first" and "second" and the like in the specification and drawings of this application are used to distinguish different objects, or to distinguish different processing of the same object, rather than to describe a specific order of objects.

[0045] In addition, the terms "including" and "having" and any variations thereof mentioned in the description of the present application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0046] It should be noted that, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0047] In the description of the present application, unless otherwise specified, “plurality” means two or more.

[0048] At present, knowledge base question and answer requires additional reliance on manpower or pre-labeling to extract entities and entity relationships. In small and medium-sized enterprise departments and personal question and answer assistant scenarios, there is a lack of manpower or pre-made templates for entity and entity relationship extraction, which makes it difficult to promote application scenarios.

[0049] In view of this, an embodiment of the present application provides an answer text generation method, which is used to obtain first text information and second text information through entity retrieval and semantic retrieval, and generate an answer text for the target question text based on the first text information and the second text information, so that the answer text is more accurate and comprehensive.

[0050] The answer text generation method provided by the embodiment of the present application is described below in conjunction with the accompanying drawings. Among them, the actions, terms, etc. involved in the various embodiments of the present application can refer to each other without limitation. The message name or parameter name in the message exchanged between the various devices in the embodiment of the present application is only an example, and other names can also be used in the specific implementation without limitation. The actions involved in the various embodiments of the present application are only an example, and other names can also be used in the specific implementation, such as: "included in" in the embodiment of the present application can also be replaced by "carried on" or "carried in", etc.

[0051] It should be noted that the execution subject of the present application may be a terminal device, or a server or a device (such as a chip) in the server. The following description will be made taking a server as an example.

[0052] like Figure 1 As shown, the embodiment of the present application proposes a method for generating an answer text, which includes: S101 to S103.

[0053] S101. Obtain entity representation and semantic representation in the target question text.

[0054] The target question text is the text information of the question raised by the user or system based on the knowledge base text. Entity refers to things, concepts, objects, etc. that have specific meanings and can be clearly identified. Entity representation means expressing and presenting entities in a certain form so that systems such as computers can understand, process and analyze them. Semantic representation is to present the semantic information contained in language units in the form of symbols, vectors, etc., for example, symbolic representation, vector representation, distributed semantic representation and semantic role labeling.

[0055] In one example, entity representation is obtained based on a preset hypergraph and deep model constructed based on multiple texts of a preset knowledge base. Based on an artificial intelligence model, a semantic representation of a target question text and a semantic representation of a knowledge base text are obtained.

[0056] The preset knowledge base includes multiple documents, each document includes multiple paragraphs, and each paragraph includes multiple sentences. The content of the preset knowledge base includes text used to generate answer text corresponding to the target question text, and the knowledge base text is the basic data for building the preset hypergraph and deep model. The hypergraph is a graph that reflects the logical relationship between sentences, paragraphs, and entities in the knowledge base text in natural speech format. The deep model has an effective feature representation and is easy to capture long-distance dependencies. The process of building a deep model based on a preset hypergraph and multiple samples for optimizing the deep model is referenced in Figure 2 The corresponding embodiments are not described in detail here.

[0057] S102: Determine first text information according to the entity representation, and determine second text information according to the semantic representation.

[0058] The first text information is the text information corresponding to the node to which the node representation similar to the entity representation belongs. The second text information is the text information corresponding to the semantic representation similar to the semantic representation of the target question text in the knowledge base.

[0059] In one example, a target node representation and text information corresponding to the target node representation are determined from among multiple node representations in a preset hypergraph, and the text information corresponding to the target node representation is used as the first text information. The similarity between the semantic representation of each text in multiple texts in a preset knowledge base and the semantic representation of the target question text is determined, and the target text in the multiple texts is used as the second text information.

[0060] Among them, the similarity between the target node representation and the entity representation is greater than the first preset threshold. The similarity between the semantic representation of the target text and the semantic representation of the target question text is greater than the second preset threshold. The similarity can be calculated by a variety of vector retrieval algorithms, for example, the Faiss similarity retrieval algorithm, the fast library for approximate nearest neighbors (FLANN), etc. The preset hypergraph is constructed based on multiple text information. Specifically, the preset hypergraph is constructed according to the knowledge base graph, and the knowledge base graph is constructed according to multiple text information of the preset knowledge base. The first preset threshold and the second preset threshold can be adjusted according to demand, which will not be elaborated here.

[0061] In a possible implementation, a node ID corresponding to the target node representation is queried, and text information corresponding to the node is obtained according to the node ID. The text information includes entities, attributes, text paragraphs, and the corresponding documents.

[0062] S103: Generate an answer text of the target question text according to the first text information and the second text information.

[0063] In one example, the first text information and the second text information are preprocessed to obtain preprocessed first text information and second text information. A prompt is formulated based on the preprocessed first text information and second text information, and the preprocessed first text information and second text information and the prompt are input into an artificial intelligence model in natural language to obtain an answer text for answering the target question text.

[0064] Preprocessing includes relevance screening, removal of duplicate content, fusion and sorting. Prompt words are used to guide the AI ​​model to generate answer text that meets the requirements of the question and inform the model of the direction and topic of the answer, so that the AI ​​model can accurately generate the corresponding answer text.

[0065] According to the above-mentioned technical means, the present application can match the first text information and the second text information related to the target question text according to the entity representation and the semantic representation corresponding to the target question text, and generate an answer text according to the first text information and the second text information. The first text information and the second text information can complement each other in the process of generating the answer text, thereby making the answer text more accurate, and by retrieving text information with higher similarity through the entity representation and the semantic representation, the generated answer text can provide a deeper answer to the question.

[0066] In another embodiment (Example 2), as Figure 2 As shown, a method for generating answer text provided by the present application is used to construct a preset hypergraph and a deep model based on the knowledge base text. The method includes S201-S203.

[0067] S201. Obtain semantic feature information in the knowledge base graph.

[0068] The knowledge base graph is constructed based on multiple texts of the preset knowledge base. The semantic feature information includes semantic space features and entity attribute features.

[0069] In one example, attribute information of multiple texts in a preset knowledge base is obtained, a knowledge base graph is constructed based on the attribute information, and semantic feature information in the knowledge base graph is obtained through an artificial intelligence model.

[0070] Among them, information extraction operations include entity extraction operations, entity relationship extraction operations, entity attribute extraction operations, and text paragraph segmentation operations. Attribute information includes entity information in the text, relationships between entities, entity attributes, and text paragraph information. The knowledge base graph includes multiple nodes and associations between nodes. Multiple nodes include entities, sentences, paragraphs, and documents, and associations include relationships between entities and text paragraph information corresponding to nodes.

[0071] In a possible implementation, before obtaining the semantic feature information in the knowledge base graph, a knowledge base graph is constructed based on multiple texts of a preset knowledge base. The knowledge base text is obtained, and information extraction prompts for entity extraction operations, entity relationship extraction operations, entity attribute extraction operations, and text paragraph segmentation are written based on the knowledge base text, and a target output format is formulated in the information extraction prompts to facilitate data parsing and storage. The prompts are input into an artificial intelligence model to extract information from the preset knowledge base, and attribute information of multiple texts of the preset knowledge base is obtained.

[0072] Based on the attribute information of multiple texts of the above-mentioned preset knowledge base, the entities, sentences, paragraphs, and documents in the attribute information are obtained as nodes in the graph, and for each node, the unique discrete hash value of the node is calculated based on the text information corresponding to the node as the unique ID of the node. The node attributes corresponding to multiple nodes are obtained, a node list is constructed, and the edges between entity type nodes are constructed based on the entity relationships between multiple nodes contained in the node list. According to the inclusion relationship between entities, sentences, paragraphs, and documents, heterogeneous node edges are constructed, thereby completing the construction of the knowledge base graph.

[0073] S202: Construct a preset hypergraph according to semantic feature information.

[0074] The preset hypergraph includes multiple hyperedges, and the hyperedges include multiple nodes with association relationships.

[0075] In one example, distance neighbor hyperedges and L1-hypergraph hyperedges are constructed based on semantic space feature information. Attribute hyperedges are constructed based on entity attribute feature information. The preset hypergraph includes distance neighbor hyperedges, L1-hypergraph hyperedges, and attribute hyperedges. For details, the construction of the preset hypergraph can refer to Figure 3 The corresponding embodiments are not described in detail here.

[0076] Furthermore, a deep model is constructed based on the preset hypergraph. The deep model contains node representations corresponding to multiple nodes of the preset hypergraph. For details, the construction process of the deep model can be referred to Figure 4 The corresponding embodiments are not described in detail here.

[0077] According to the above-mentioned technical means, the present application can construct a preset hypergraph by constructing multiple hyperedges, and use the preset hypergraph to understand the deeper relationships of the knowledge in the preset knowledge base, deepen the mining of the relationships between multiple nodes, so that subsequent deep models can have a more comprehensive understanding of the knowledge base.

[0078] In another embodiment (Example 3), as Figure 3 As shown, an answer text generation method is provided for an embodiment of the present application, which is used to construct a hypergraph according to a knowledge base graph. The method includes S301 to S303.

[0079] S301. Construct distance neighbor hyperedges and L1-hypergraph hyperedges according to semantic space feature information.

[0080] In one example, nodes in the knowledge base graph are used as nodes in a preset hypergraph, and distances between multiple nodes in the preset hypergraph are calculated. Distance neighbor hyperedges and L1-hypergraph hyperedges are constructed based on the distances between the multiple nodes.

[0081] In one application scenario, the text information corresponding to the nodes in the knowledge base graph is obtained in the knowledge base text, and the text information is vectorized by the artificial intelligence model to obtain the vectorized representation of the text information corresponding to the node. The distances between multiple nodes are calculated by the cosine similarity formula. The cosine similarity formula satisfies Formula 1.

[0082]

[0083] Among them, A and B are the vectorized representations of the text information corresponding to two different nodes. The closer cosθ is to 1, the closer the distance between two different nodes is. The artificial intelligence model can be the ERNIE model or other artificial intelligence models with text information vectorization function.

[0084] Further, based on the distance between nodes, one or more nearest neighbor nodes of each node are determined, and the node and the nearest one or more neighbor nodes form a distance neighbor hyperedge. After obtaining the distance neighbor hyperedge, the weight or confidence of the distance neighbor hyperedge is calculated according to a preset formula. The preset formula satisfies Formula 2.

[0085]

[0086] Among them, w(e) is the weight of hyperedge e, u and v are the nodes in hyperedge e, σ is the median distance between every two nodes in the hyperedge, d represents the distance between two nodes, and X represents the vector representation of the text information corresponding to the node.

[0087] In another application scenario, the L1 norm is used to perform sparse coding to model the relationship between nodes, and the relationship between nodes is further improved by reconstruction, and the coefficient value with the best reconstruction effect is calculated by a preset formula. The preset formula is Formula 3.

[0088]

[0089] Among them, argmin represents the value of each independent variable when the formula takes the minimum value as a whole. In the case where the formula takes the minimum value as a whole, z i The value of z i is the correlation coefficient learned by the i-th nearest neighbor node. c ) is the semantic feature vector representation of the central node v, and B is the semantic feature vector representation of one or more nearest neighbor nodes.

[0090] Based on the above process, in the samples of the nearest neighbor nodes, the nodes whose coefficients are not 0 constitute the L1-hypergraph hyperedges.

[0091] S302: Construct attribute hyperedges according to entity attribute feature information.

[0092] Among them, entity attribute feature information refers to information used to describe the various characteristics and properties of the entity itself. For example, the knowledge base text is "He went to the supermarket to buy apples", "Apples are 5 yuan per catty", "He bought 2 catties", and the entity attribute feature information of the entity "apple" includes the price attribute (5 yuan per catty) and the weight attribute (the weight purchased is 2 catties). Attribute hyperedges are edges that connect multiple related entities based on the attribute characteristics of the entities, and are used to reflect the connection between entities at the specific attribute level.

[0093] In one example, based on the fact that entities are associated with each other due to common attributes and / or due to the fact that an entity's attributes are associated with other entities, multiple entities that are associated are connected to obtain an attribute hyperedge. The weight of the attribute hyperedge is calculated by taking the average value of the heat kernel weights of the edge within the group to complete the construction of the attribute hyperedge. The weight of the attribute hyperedge satisfies Formula 4.

[0094]

[0095] Where w(e) represents the weight of hyperedge e. δ(e) represents the hyperparameter contained in hyperedge e, and exp represents the exponential function. For example, exp(x+1) represents e x+1 .

[0096] In one application scenario, the knowledge base text is "He went to the supermarket to buy apples", "Apples are 5 yuan per pound", and "He bought 2 pounds".

[0097] Among them, entities may include: he, supermarket, apple. The entity attributes corresponding to the entity "he" include behavior attributes and location-related attributes. The behavior attributes include the behavior of going to the supermarket and the behavior of buying apples. The location-related attribute is that the activity venue involves the supermarket. The entity attributes corresponding to the entity "supermarket" include functional attributes and location attributes. The functional attribute is selling goods, and the location attribute is a fixed business location. The entity attributes corresponding to the entity "apple" include price attributes, weight attributes, and product attributes. The price attribute is 5 yuan per catty, the weight attribute is that the weight of this transaction is 2 catties, and the product attribute is that the product has the circulation.

[0098] Based on the entity attributes of the above entities and multiple entities, multiple attribute hyperedges are constructed. Multiple attribute hyperedges include purchase behavior hyperedges, location association hyperedges, and sales relationship hyperedges. The purchase behavior hyperedge includes "he" and "apple". The attribute basis is "he"'s purchase behavior and "apple"'s commodity attributes. The two attributes are associated through the purchase behavior to form a hyperedge. The location association hyperedge includes "he" and "supermarket". The attribute basis is "he"'s location association attribute and "supermarket"'s location attribute. The two attributes are associated through "he"'s behavior of going to "supermarket" to form a hyperedge. The sales relationship hyperedge includes "supermarket" and "apple". The attribute basis is that the functional attributes of "supermarket" and the commodity attributes of "apple" are associated to form a hyperedge.

[0099] S303: Construct a preset hypergraph based on the distance neighbor hyperedge, the L1-hypergraph hyperedge, and the attribute hyperedge.

[0100] In one example, the distance neighbor hyperedge, L1-hypergraph hyperedge, attribute hyperedge, and nodes contained in multiple hyperedges together constitute a preset hypergraph. Let G = (V, E) represent the hypergraph G, and the matrix H corresponding to the hypergraph G is represented by |V|*|E|. In the matrix H, rows represent different nodes and columns represent different hyperedges. Among them, the values ​​of the elements in the matrix satisfy Formula 5.

[0101] h(v i ,e j )=1ifv i ∈e j ,otherwiseh(v i ,e j )=0Formula 5

[0102] Among them, h(v i ,e j ) indicates that the position in the matrix H is (v i ,e j ), if the node v i Belongs to the hyperedge e j , then h(v i ,e j )=1, otherwise h(v i ,e j )=0.

[0103] According to the above technical means, the present application can construct three types of hyperedges: distance neighbor hyperedges, L1-hypergraph hyperedges and attribute hyperedges to complete the construction of the hypergraph. A hypergraph contains multiple different types of hyperedges, which makes the preset hypergraph have a more comprehensive understanding of the knowledge base text and increases the understanding at multiple levels and angles.

[0104] In another embodiment, Figure 4As shown, an answer text generation method is provided for an embodiment of the present application, which is used to construct a deep model based on a preset hypergraph and obtain node representations corresponding to multiple nodes in the preset hypergraph. The method includes: S401 to S402.

[0105] S401: Sampling a preset hypergraph to obtain multiple samples.

[0106] The sampling includes positive sampling and negative sampling, and the multiple samples include multiple first samples and multiple second samples. The first samples obtained by positive sampling are nodes with associations in the preset hypergraph. The second samples obtained by negative sampling are samples without associations in the preset hypergraph.

[0107] In one example, a meta-path is set according to the node type in a preset hypergraph, and a graph walk is performed on the preset hypergraph based on the meta-path to obtain a walk sequence.

[0108] The walking sequence includes multiple first samples. The first samples can also be called positive samples. In the walking sequence, every two adjacent nodes form a positive sample pair, and multiple positive sample pairs form a positive sample.

[0109] In one application scenario, a meta-path is an ordered combination of node types defined in a graph structure based on a specific semantic relationship, for example, The V table represents the node type, R represents the relationship between two nodes, and the subscripts 1, 2…t, t+1…l represent the order. V1 is the starting node type, V2 is the next node type of V1, and the relationship between V1 and V2 is R1. And so on. The complete meta-path is from V1 to V l .

[0110] Furthermore, in the process of graph walking based on the meta-path, the starting node of the graph walking is determined according to the starting node type of the meta-path. The node of the next node type specified by the meta-path is determined among the multiple nodes connected to the starting node, and the transition probability of each node is calculated, that is, the probability of reaching the node in the next step. The graph is walked according to the transition probability of each node to obtain a walking sequence. The transition probability satisfies Formula 6.

[0111]

[0112] in, represents the current node, v i+1 Indicates the next step node, represents the degree of the node, E represents the hyperedge, Representation Node The number of context nodes of node type t that exist, and φ represents the value of the mapping of node types.

[0113] Based on the transition probability of formula 6 above, the concept of hypergraph is introduced, and formula 6 is multiplied by formula 7 to obtain the meta-path walk transition probability based on the hypergraph. According to the meta-path walk transition probability based on the hypergraph, a random graph walk is performed on the preset hypergraph to obtain a walk sequence.

[0114]

[0115] Among them, p(v i ,v j ) is the assignment formula of the matrix p established according to the preset hypergraph, v i represents the row of the matrix, v j represents the columns of the matrix and E represents the hyperedge.

[0116] In a possible implementation, multiple groups of basic meta-paths are set, which are shorter than conventional meta-paths, for example, ESE and ERE, where E represents an entity type node, S represents a sentence type node, and R represents a relationship between two nodes. Multiple basic meta-paths are combined to obtain a longer meta-path, for example, (ESE)-(ERE), which means that two entities E appear in the same sentence S, and there is a relationship R between the two entities E.

[0117] In another example, based on the heterogeneous negative sampling method, negative sampling is performed on the preset hypergraph to obtain multiple second samples.

[0118] The second sample can also be called a negative sample. Compared with conventional negative sampling, heterogeneous negative sampling considers heterogeneous features such as different types of nodes and relationships for sampling.

[0119] In an application scenario, negative sampling is performed based on a heterogeneous negative sampling method to obtain negative sample pairs. Multiple negative sample pairs constitute negative samples. Before negative sampling, a preset formula, i.e., a negative sampling objective function, is optimized by an optimization function to obtain an optimized negative sampling objective function. According to the optimized negative sampling objective function, negative sampling is performed to obtain negative sample pairs, and multiple negative sample pairs constitute negative samples. The optimization formula satisfies Formula 8.

[0120]

[0121] Where p is the node c t Perform softmax normalization on nodes of the same type to get node c t The probability distribution of all corresponding node types. Based on this probability distribution, the negative sampling objective function can be optimized so that the negative sampling objective function can sample negative samples that meet the requirements. t |v;θ) means that given the node v and the function parameter θ, the node c t The probability distribution of the node type.t is the set of multiple nodes of type t in the entire preset hypergraph, u t Indicates V t Except for node c t For other nodes except , X represents the semantic feature vector representation of the corresponding node, and e is a natural constant.

[0122] The optimized negative sampling objective function satisfies Formula 9.

[0123]

[0124] Among them, X is the semantic feature vector representation corresponding to the node. X is input into the negative sampling objective function, and the output value of the objective function is obtained. According to the output value, the other node in the negative sample pair that matches the input node can be screened and determined. E is the mathematical expectation, and σ represents the degree of the hyperedge to which the node belongs, that is, the number of nodes contained in the hyperedge.

[0125] S402: Build a deep model based on a preset hypergraph and multiple samples.

[0126] Among them, the deep model built based on the preset hypergraph integrates the hypergraph structure into the deep learning framework. The hypergraph can represent the characteristics of complex multivariate relationships, so that the model can deeply learn the attributes and relationship information contained in nodes, edges, and hyperedges.

[0127] In one example, a deep model is constructed based on a preset hypergraph, and feature vectors in the hidden layer of the deep model are extracted as inputs for contrastive learning. Based on positive and negative samples, the loss value between the positive and negative samples is calculated using a contrastive learning loss function, and the deep model is optimized based on the loss value.

[0128] Among them, the contrastive learning loss function can be a cross entropy loss function, a noise contrastive estimation loss (NCE LOSS), or an information noise contrastive estimation loss (INFONCE LOSS) function, which is used to calculate the loss between positive and negative samples to achieve deep model optimization.

[0129] In another example, Figure 5As shown, the deep model built based on the preset hypergraph includes entity embeddings, high-dimensional channels (HG Channel), low-dimensional channels (LG Channel), document embedding, semantic similarity and structure learning (SS Learning), probability prediction, and soft label probability distribution (softmax). The high-dimensional channel contains two hypergraph convolutional layers (HGConv Layer) and a soft attention layer, and the low-dimensional channel contains an average pooling layer (Average Pooling) and two graph convolutional layers (GConv Layer). Multiple hypergraph convolutional layers are also called hypergraph convolution channels, which are used for the propagation and update of entity embedding and document embedding. The definition of the hypergraph convolutional layer satisfies Formula 10.

[0130]

[0131] Wherein, X represents a semantic feature representation in vector form. For details, refer to the definition formula of the hypergraph convolution layer in the prior art, which will not be described in detail here.

[0132] Furthermore, based on the deep model constructed above, the knowledge base text (documents) is constructed into a knowledge base graph, and a hypergraph construction is performed, and the hypergraph is transformed, and multiple nodes in the hyperedges are connected in pairs to obtain a transformed hypergraph. The hyperedges in the hypergraph are used as nodes to construct a line graph. The node attribute features, node relationship features and other information contained in the hypergraph are input into the high-dimensional channel of the deep model, and the node attribute features, node relationship features and other information contained in the line graph are input into the low-dimensional channel of the deep model. After the low-dimensional channel and the high-dimensional channel preliminarily organize the node attribute features, node relationship features and other information contained in the hypergraph and the line graph, feature information in vector form is formed and output.

[0133] The knowledge base text includes three sentences (S1, S2, S3), S1 includes nodes v1, v2, v3, S2 includes nodes v4, v5, v6, S3 includes nodes v7, v8, v9, and the hypergraph constructed based on the knowledge base text includes three hyperedges (e1, e2, e3). The line graph is constructed based on the hyperedges in the hypergraph, and each hyperedge is a node of the line graph. If every two hyperedges contain the same node, there is an edge between the nodes represented by the two hyperedges, reflecting the relationship between multiple hyperedges in the hypergraph.

[0134] The document embedding workflow is based on the feature information in the form of vectors output by high-dimensional channels and low-dimensional channels. The feature information of high-dimensional channels and the feature information of low-dimensional channels are spliced ​​to obtain a more comprehensive feature representation, namely SS Learning. For example, the vector dimension of the high-dimensional channel is 100, the vector dimension of the low-dimensional channel is 50, and the dimension of the vector after splicing is 150.

[0135] After the feature information in the form of vector output based on the high-dimensional channel passes through the document embedding process, a low-dimensional vector representation that integrates semantics and key features is obtained. The low-dimensional vector representation and the features in the form of vector output by embedding are concatenated and / or weighted summed to form a comprehensive feature vector. Based on the comprehensive feature vector, node type probability prediction can be performed, and the type probability prediction results are processed according to the softmax function to obtain the soft label probability distribution, that is, V1 (0.02), V2 (0.05), V3 (0.34), ... V n (0.01), V1 to V n That is, different node types.

[0136] In one possible implementation, sequence features are not involved in the above operation process, while there are entities, sentences, paragraphs, and documents in multiple node types. There is sequentiality between entities, sentences, paragraphs, and documents. In order to enable the deep model to understand information such as the relationship features between multiple nodes, a trainable position encoding matrix is ​​introduced, and the trainable position encoding matrix satisfies Formula 11.

[0137] P γ =[p1,p2,p3,…,p m Formula 11

[0138] Among them, γ, 1, 2, ..., m are the indices of the position encoding matrix, P represents a document type node, and p is the entity, sentence, and paragraph type node contained in the document type node.

[0139] Through the trainable position encoding matrix, the feature information represented by the vector output by the document embedding process of the entity, sentence, and paragraph type nodes can be spliced ​​with the feature information represented by the vector output by the document embedding process of the nodes of the corresponding document type, and the splicing process satisfies Formula 12.

[0140]

[0141] Among them, α t It is used to calculate the importance of nodes based on documents and the entities, sentences, and paragraph nodes contained in the documents, θ h It is used to perform weighted summation of the semantic feature vectors of nodes according to the importance of the nodes to obtain SS Learning. is the semantic feature vector corresponding to the node of the document type, It is the semantic feature vector of the tth node among the multiple entities, sentences, and paragraphs contained in the document type node. W represents the weight, and c is a preset constant. By performing weighted summation through formula 12, effective features can be screened and relatively unimportant features in the document can be eliminated, thereby achieving the effect of noise reduction and error reduction.

[0142] According to the above-mentioned technical means, the present application can obtain multiple samples, and multiple models include multiple first samples and multiple second samples. There are multiple nodes associated with the first samples and multiple nodes not associated with the second samples. In the subsequent deep model optimization, the deep model can have a deeper understanding of the relationship between the multiple nodes, thereby improving the generalization ability of the deep model. At the same time, it provides node representation for entity retrieval, which facilitates the generation of answer text.

[0143] It is understandable that the above-mentioned answer text generation method can be implemented by an answer text generation device. In order to realize the above-mentioned functions, the answer text generation device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the modules and algorithm steps of each example described in the embodiments disclosed herein, the embodiments disclosed in this application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments disclosed in this application.

[0144] The embodiments disclosed in the present application can divide the functional modules of the answer text generation device generated by the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiments disclosed in the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0145] Figure 6 This is a schematic diagram of the structure of an answer text generation device provided in an embodiment of the present application. Figure 6 As shown, the answer text generating device 60 can be used to perform Figure 1-Figure 5 The answer text generation method shown in the figure. The answer text generation device 60 includes: a communication unit 601 and a processing unit 602.

[0146] The communication unit 601 is used to obtain entity representation and semantic representation in the target question text;

[0147] A processing unit 602 is used to determine the first text information according to the entity representation, and determine the second text information according to the semantic representation;

[0148] The processing unit 602 is further configured to generate an answer text to the target question text according to the first text information and the second text information.

[0149] In one possible implementation, the processing unit 602 is also used to determine a target node representation among multiple node representations in a preset hypergraph and text information corresponding to the target node representation, and the similarity between the target node representation and the entity representation is greater than a first preset threshold; the preset hypergraph is constructed based on multiple text information; and the text information corresponding to the target node representation is used as the first text information.

[0150] In one possible implementation, the processing unit 602 is also used to determine the similarity between the semantic representation of each text in the multiple texts of the preset knowledge base and the semantic representation of the target question text; the target text in the multiple texts is used as the second text information; the similarity between the semantic representation of the target text and the semantic representation of the target question text is greater than a second preset threshold.

[0151] In one possible implementation, the processing unit 602 is also used to obtain semantic feature information in the knowledge base graph, and the semantic feature information includes semantic space features and entity attribute features; the knowledge base graph is constructed based on multiple texts of a preset knowledge base; the processing unit 602 is also used to construct a preset hypergraph based on the semantic feature information; the preset hypergraph includes multiple hyperedges, and the hyperedges include multiple nodes with associated relationships.

[0152] In one possible implementation, the processing unit 602 is also used to obtain attribute information of multiple texts of a preset knowledge base, the attribute information including entity information in the text, relationships between entities, entity attributes, and text paragraph information; the processing unit 602 is also used to construct a knowledge base graph based on the attribute information, the knowledge base graph including multiple nodes and association relationships between nodes; the multiple nodes include entities, sentences, paragraphs, and documents, and the association relationships include relationships between entities and text paragraph information corresponding to the nodes.

[0153] In one possible implementation, processing unit 602 is also used to sample a preset hypergraph to obtain multiple samples; processing unit 602 is also used to build a deep model based on the preset hypergraph and multiple samples, and the deep model includes node representations corresponding to multiple nodes of the preset hypergraph.

[0154] In one possible implementation, the processing unit 602 is further used to set a meta-path according to the node type in the preset hypergraph, and based on the meta-path, perform a graph walk on the preset hypergraph to obtain a walk sequence; the walk sequence includes multiple first samples; the first sample includes nodes with an associated relationship; the processing unit 602 is further used to perform negative sampling on the preset hypergraph based on a heterogeneous negative sampling method to obtain multiple second samples; the second sample includes nodes with no associated relationship; wherein the multiple samples include multiple first samples and multiple second samples.

[0155] Through the description of the above implementation methods, technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0156] The present disclosure also provides a computer-readable storage medium having instructions stored thereon. When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the answer text generation method provided by the above-mentioned embodiment of the present disclosure.

[0157] The embodiment of the present disclosure also provides a computer program product containing instructions, which, when executed on an electronic device, enables the electronic device to execute the answer text generation method provided by the embodiment of the present disclosure.

[0158] Among them, the computer readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a register, a hard disk, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above, or any other form of computer readable storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application-specific integrated circuit (ASIC). In the embodiments of the present application, a computer-readable storage medium may be any tangible medium that contains or stores a program, which may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0159] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for generating an answer text, characterized in that: The method comprises: Obtain entity representation and semantic representation in the target question text; Determine the first text information according to the entity representation, and determine the second text information according to the semantic representation; An answer text of the target question text is generated according to the first text information and the second text information.

2. The method according to claim 1, characterized in that The determining the first text information according to the entity representation includes: Determine a target node representation among multiple node representations in a preset hypergraph and text information corresponding to the target node representation, wherein the similarity between the target node representation and the entity representation is greater than a first preset threshold; the preset hypergraph is constructed based on multiple text information; The text information corresponding to the target node representation is used as the first text information.

3. The method according to claim 1, characterized in that The determining the second text information according to the semantic representation of the target question text includes: Determine the similarity between the semantic representation of each text in the plurality of texts in the preset knowledge base and the semantic representation of the target question text; A target text among the multiple texts is used as the second text information; and a similarity between a semantic representation of the target text and a semantic representation of the target question text is greater than a second preset threshold.

4. The method according to claim 2, characterized in that: The method further comprises: Acquire semantic feature information in a knowledge base graph, the semantic feature information including semantic space features and entity attribute features; the knowledge base graph is constructed based on multiple texts of the preset knowledge base; The preset hypergraph is constructed according to the semantic feature information; the preset hypergraph includes a plurality of hyperedges, and the hyperedges include a plurality of nodes with association relationships.

5. The method according to claim 4, characterized in that The method further comprises: Acquire attribute information of multiple texts in the preset knowledge base, wherein the attribute information includes entity information in the text, relationships between entities, entity attributes, and text paragraph information; According to the attribute information, a knowledge base graph is constructed, wherein the knowledge base graph includes a plurality of nodes and association relationships between the nodes; the plurality of nodes include entities, sentences, paragraphs, and documents, and the association relationships include relationships between entities and text paragraph information corresponding to the nodes.

6. The method according to claim 4, characterized in that The method further comprises: Sampling the preset hypergraph to obtain multiple samples; The deep model is constructed based on the preset hypergraph and multiple samples, and the deep model includes node representations corresponding to multiple nodes of the preset hypergraph.

7. The method according to claim 6, characterized in that The sampling of the preset hypergraph to obtain a plurality of samples includes: According to the node type in the preset hypergraph, a meta-path is set, and based on the meta-path, a graph walk is performed on the preset hypergraph to obtain a walk sequence; the walk sequence includes a plurality of first samples; the first samples include nodes having an associated relationship; Based on the heterogeneous negative sampling method, negative sampling is performed on the preset hypergraph to obtain a plurality of second samples; the second samples include nodes that have no association relationship; The multiple samples include the multiple first samples and the multiple second samples.

8. An answer text generating device, characterized in that: The device comprises: a communication unit and a processing unit; The communication unit is used to obtain entity representation and semantic representation in the target question text; The processing unit is used to determine the first text information according to the entity representation, and determine the second text information according to the semantic representation; The processing unit is further configured to generate an answer text for the target question text according to the first text information and the second text information.

9. An electronic device, characterized in that: include: A memory and a processor; the memory and the processor are coupled; The memory is used to store instructions executable by the processor; When the processor executes the instructions, it performs the answer text generation method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the answer text generation method as described in any one of claims 1-7.

11. A computer program product, characterized in that The computer program product comprises computer program instructions, which, when executed by a processor, implement the answer text generation method as described in any one of claims 1 to 7.

Images