Embedding representation model training method, question answering method and device based on knowledge graph

Abstract

The present disclosure provides a training method for an embedded representation model, which can be applied to the financial field and the field of artificial intelligence technology. The training method for the embedded representation model includes: obtaining an initial knowledge graph; preprocessing the initial knowledge graph to generate a sample knowledge graph; obtaining triples from the sample knowledge graph, wherein the triples include a head entity, a relationship, and a tail entity neighborhood; using the triples to train the embedded representation model to be trained to obtain a trained embedded representation model. The present disclosure also provides a question-answering method based on a knowledge graph, a training device for an embedded representation model, a question-answering device based on a knowledge graph, a device, a storage medium, and a program product.

Description

Technical Field

[0001] The present disclosure relates to the fields of finance and artificial intelligence technology, and more specifically to an embedded representation model training method, a question-answering method based on a knowledge graph, a device, equipment, medium and program product. Background Art

[0002] The knowledge graph effectively processes, handles and integrates the data of complex documents, converts them into simple and clear "entity, relationship, entity" triplets, and finally aggregates a large amount of knowledge to achieve rapid response and reasoning of knowledge.

[0003] However, with the continuous growth and change of knowledge data, in order to better utilize the knowledge graph, it is necessary to embed the entities and relationships in the knowledge graph.

[0004] In the process of implementing the concept of the present disclosure, the inventors discovered that the method of embedding knowledge graphs in related technologies is not accurate enough, resulting in that the knowledge graphs after embedding cannot be well utilized. Summary of the invention

[0005] In view of the above problems, the present disclosure provides a training method for an embedded representation model, a question-answering method based on a knowledge graph, a training device for an embedded representation model, a question-answering device based on a knowledge graph, a device, a storage medium and a program product.

[0006] According to a first aspect of the present disclosure, a method for training an embedding representation model is provided, comprising:

[0007] Obtain the initial knowledge graph;

[0008] Preprocessing the initial knowledge graph to generate a sample knowledge graph, wherein the sample knowledge graph includes a head entity, a relationship, and a tail entity, wherein the head entity, the relationship, and the tail entity in the sample knowledge graph are represented by random vectors;

[0009] Acquire a triple from the sample knowledge graph, wherein the triple includes a head entity, a relationship, and a tail entity neighborhood, wherein the tail entity neighborhood includes a set consisting of multiple tail entities having the same relationship as the head entity;

[0010] The embedded representation model to be trained is trained using the triples to obtain a trained embedded representation model, wherein the embedded representations of the head entity, the relationship, and the neighborhood of the tail entity in the embedded representation triples output by the trained embedded representation model meet preset conditions.

[0011] According to an embodiment of the present disclosure, the step of training the embedding representation model to be trained by using the triples to obtain the trained embedding representation model includes:

[0012] Inputting the triple into the embedding representation model to be trained, and outputting a predicted triple, wherein the embedding representation in the predicted triple is obtained by adjusting the random variables in the triple by the embedding representation model;

[0013] Verifying the predicted triplet using the objective function to generate a verification result;

[0014] When the verification result satisfies the preset condition, the trained embedding representation model is output.

[0015] According to an embodiment of the present disclosure, the preset conditions include:

[0016] A difference between an embedding representation of any tail entity among the plurality of tail entities in the neighborhood of the tail entity and a first embedding representation is less than a preset threshold, wherein the first embedding representation is obtained by adding an embedding representation of the head entity to an embedding representation of the relationship.

[0017] According to an embodiment of the present disclosure, obtaining an initial knowledge graph includes:

[0018] Acquire an initial knowledge set;

[0019] Performing a first keyword filtering process on the initial knowledge set to generate a first knowledge set;

[0020] Extracting second keywords from the first knowledge set to generate a second knowledge set;

[0021] The initial knowledge graph is generated according to the second knowledge set, wherein the head entity, relationship and tail entity in the initial knowledge graph are represented by first preset characters, wherein the first preset characters include Chinese characters or letters.

[0022] According to an embodiment of the present disclosure, the head entity, the relationship, and the tail entity in the initial knowledge graph are represented by first preset characters, wherein the first preset characters include Chinese characters or letters;

[0023] The preprocessing of the initial knowledge graph to generate a sample knowledge graph includes:

[0024] The head entity, the relationship, and the tail entity in the initial knowledge graph are randomly initialized to convert the first preset character into a random vector to generate the sample knowledge graph.

[0025] A second aspect of the present disclosure provides a question-answering method based on a knowledge graph, comprising:

[0026] Obtain a first knowledge graph;

[0027] Inputting the first knowledge graph into an embedding representation model, and outputting a first embedding representation knowledge graph, wherein the embedding representation model is trained by the embedding representation model training method provided by an embodiment of the present disclosure;

[0028] Obtaining description information of the question to be answered, wherein the description information of the question to be answered includes a head entity and a relationship;

[0029] The description information of the question to be answered is input into the first embedding representation knowledge graph, and an answer corresponding to the description information of the question to be answered is output, wherein the answer includes a target entity determined according to the head entity and the relationship.

[0030] According to an embodiment of the present disclosure, inputting the description information of the question to be answered into the first embedding representation knowledge graph, and outputting an answer corresponding to the description information of the question to be answered includes:

[0031] Determining an embedded representation of the head entity and an embedded representation of the relationship according to the description information of the question to be answered;

[0032] Obtaining a second embedding representation according to the embedding representation of the head entity and the embedding representation of the relationship;

[0033] Calculate multiple similarity distances between the second embedding representation and the embedding representations of multiple entities in the first embedding representation knowledge graph;

[0034] Determine a minimum similarity distance from the multiple similarity distances;

[0035] An entity corresponding to the minimum similarity distance is determined as the target entity.

[0036] According to an embodiment of the present disclosure, the calculating of multiple similarity distances between the second embedding representation and the embedding representations of multiple entities in the first embedding representation knowledge graph includes:

[0037] Calculate multiple Euclidean distances between the second embedding representation and the embedding representations of multiple entities in the first embedding representation knowledge graph according to the Euclidean distance calculation method; or

[0038] According to the cosine distance calculation method, multiple cosine distances between the second embedding representation and the embedding representations of multiple entities in the first embedding representation knowledge graph are calculated.

[0039] A third aspect of the present disclosure provides a training device for an embedded representation model, comprising:

[0040] The first acquisition module is used to acquire an initial knowledge graph;

[0041] A preprocessing module, configured to preprocess the initial knowledge graph to generate a sample knowledge graph, wherein the sample knowledge graph includes a head entity, a relationship, and a tail entity, wherein the head entity, the relationship, and the tail entity in the sample knowledge graph are represented by random vectors; and

[0042] A second acquisition module is used to acquire a triple from the sample knowledge graph, wherein the triple includes a head entity, a relationship, and a tail entity neighborhood, wherein the tail entity neighborhood includes a set consisting of multiple tail entities having the same relationship with the head entity;

[0043] A training module is used to train the embedding representation model to be trained using the triples to obtain a trained embedding representation model, wherein the embedded representations of the head entity, the relationship, and the neighborhood of the tail entity in the embedding representation triples output by the trained embedding representation model meet preset conditions.

[0044] A fourth aspect of the present disclosure provides a question-answering device based on a knowledge graph, comprising:

[0045] A third acquisition module is used to acquire the first knowledge graph;

[0046] An input module, used to input the first knowledge graph into an embedding representation model, and output a first embedding representation knowledge graph, wherein the embedding representation model is trained by the embedding representation model training method provided by an embodiment of the present disclosure;

[0047] A fourth acquisition module is used to acquire description information of the question to be answered, wherein the description information of the question to be answered includes a head entity and a relationship;

[0048] An output module is used to input the description information of the question to be answered into the first embedding representation knowledge graph, and output an answer corresponding to the description information of the question to be answered, wherein the answer includes a target entity determined according to the head entity and the relationship.

[0049] The fifth aspect of the present disclosure provides an electronic device, comprising: one or more processors; a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above-mentioned training method of the embedded representation model and the question-answering method based on the knowledge graph.

[0050] The sixth aspect of the present disclosure also provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the above-mentioned training method of the embedded representation model and the question-answering method based on the knowledge graph.

[0051] The seventh aspect of the present disclosure also provides a computer program product, including a computer program, which, when executed by a processor, implements the above-mentioned training method of the embedded representation model and the question-answering method based on the knowledge graph. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0053] Figure 1 Schematically illustrates an application scenario diagram of a training method for an embedded representation model, a question-answering method based on a knowledge graph, a training device for an embedded representation model, and a question-answering device based on a knowledge graph according to an embodiment of the present disclosure;

[0054] Figure 2 A flowchart of a method for training an embedding representation model according to an embodiment of the present disclosure is schematically shown;

[0055] Figure 3 A flowchart schematically illustrates a method of training an embedding representation model to be trained using triples to obtain a trained embedding representation model according to an embodiment of the present disclosure.

[0056] Figure 4 A flowchart for obtaining an initial knowledge graph according to an embodiment of the present disclosure is schematically shown;

[0057] Figure 5 A flowchart of a question-answering method based on a knowledge graph according to an embodiment of the present disclosure is schematically shown;

[0058] Figure 6 A flowchart schematically illustrates inputting description information of a question to be answered into a first embedding representation knowledge graph and outputting an answer corresponding to the description information of the question to be answered according to an embodiment of the present disclosure;

[0059] Figure 7 The structure block diagram of the training device of the embedded representation model according to the embodiment of the present disclosure is schematically shown;

[0060] Figure 8 A structural block diagram of a question-answering device based on a knowledge graph according to an embodiment of the present disclosure is schematically shown; and

[0061] Fig. 9 A block diagram of an electronic device suitable for implementing a training method for an embedded representation model and a question-answering method based on a knowledge graph according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0062] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0063] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0064] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0065] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0066] The present disclosure provides a training method for an embedded representation model, which can be applied to the financial field and the field of artificial intelligence technology. The training method for the embedded representation model includes: obtaining an initial knowledge graph; preprocessing the initial knowledge graph to generate a sample knowledge graph; obtaining triples from the sample knowledge graph, wherein the triples include a head entity, a relationship, and a tail entity neighborhood; using the triples to train the embedded representation model to be trained to obtain a trained embedded representation model. The present disclosure also provides a question-answering method based on a knowledge graph, a training device for an embedded representation model, a question-answering device based on a knowledge graph, a device, a storage medium, and a program product.

[0067] It should be noted that the methods and devices determined in the embodiments of the present disclosure can be applied to the financial field and the field of artificial intelligence technology, and can also be used in any field other than the financial field and the field of artificial intelligence technology. The application fields of the methods and devices determined in the embodiments of the present disclosure are not limited.

[0068] Figure 1The application scenario diagram schematically shows a training method for an embedded representation model, a question-answering method based on a knowledge graph, a training device for an embedded representation model, and a question-answering device based on a knowledge graph according to an embodiment of the present disclosure.

[0069] like Figure 1 As shown, the application scenario 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0070] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only examples).

[0071] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0072] The server 105 may be a server that provides various services, such as a background management server (only an example) that provides support for websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0073] It should be noted that the training method of the embedded representation model and the question-answering method based on the knowledge graph provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the training device of the embedded representation model and the question-answering device based on the knowledge graph provided in the embodiment of the present disclosure can generally be set in the server 105. The training method of the embedded representation model and the question-answering method based on the knowledge graph provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the training device of the embedded representation model and the question-answering device based on the knowledge graph provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105.

[0074] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0075] The following will be based on Figure 1 The scene described by Figure 2 to Figure 4 The training method of the embedding representation model of the disclosed embodiment is described in detail.

[0076] Figure 2 A flowchart of a method for training an embedding representation model according to an embodiment of the present disclosure is schematically shown.

[0077] like Figure 2 As shown, the training method of the embedding representation model of this embodiment includes operations S201 to S204.

[0078] In operation S201, an initial knowledge graph is obtained.

[0079] In operation S202, the initial knowledge graph is preprocessed to generate a sample knowledge graph, wherein the sample knowledge graph includes a head entity, a relationship, and a tail entity, wherein the head entity, the relationship, and the tail entity in the sample knowledge graph are represented by random vectors.

[0080] According to an embodiment of the present disclosure, the head entity, relationship and tail entity in the sample knowledge graph can all be represented by an M-dimensional random vector, where M is a positive integer, for example, M can be 1, 2 or 3.

[0081] According to an embodiment of the present disclosure, a random vector with a dimension that is too low may result in insufficient training accuracy of a model trained using this vector; while a random vector with a dimension that is too high may result in difficulty in using this vector to fit a model that meets the requirements. Therefore, the dimension of the random vector can be flexibly set by technicians in this field according to actual application requirements.

[0082] In operation S203, triples are obtained from the sample knowledge graph, wherein the triples include a head entity, a relationship, and a tail entity neighborhood, wherein the tail entity neighborhood includes a set consisting of multiple tail entities having the same relationship with the head entity.

[0083] According to an embodiment of the present disclosure, for example, in a sample knowledge graph, there is a head entity "Xiao Ming", a relationship "like", and multiple tail entities: "basketball", "football" and "table tennis", wherein the multiple tail entities are all connected to the head entity "Xiao Ming" through the relationship "like", thus, the multiple tail entities: "entity basketball", "football" and "table tennis" together constitute the tail entity neighborhood of the head entity "Xiao Ming".

[0084] In operation S204, the embedding representation model to be trained is trained using the triples to obtain a trained embedding representation model, wherein the embedded representations of the head entity, the relationship, and the neighborhood of the tail entity in the embedding representation triples output by the trained embedding representation model meet preset conditions.

[0085] According to an embodiment of the present disclosure, the embedding representation model to be trained can be constructed based on a deep learning network, such as a recurrent neural network (RNN) or a convolutional neural network (CNN), but is not limited to this. The embedding representation model to be trained can also be constructed based on a machine learning model.

[0086] In the embodiments of the present disclosure, an embedding representation model is trained using the neighborhood of the tail entity associated with the head entity, which fully takes into account the network structure characteristics of the knowledge graph. Therefore, when the knowledge graph is embedded in the trained embedding representation model, the correlation between entities in the knowledge graph can be comprehensively considered, thereby solving the technical problem of insufficient accuracy of the embedding representation method existing in the related technology and achieving the technical effect of improving the accuracy of the embedding representation. Therefore, when the knowledge graph represented by the embedding is used for relationship prediction, the accuracy of the relationship prediction can be improved.

[0087] Figure 3 The flowchart schematically shows how to train an embedding representation model to be trained using triples to obtain a trained embedding representation model according to an embodiment of the present disclosure.

[0088] like Figure 3 As shown, this embodiment uses triples to train the embedding representation model to be trained, and obtains the trained embedding representation model, including operations S301 to S303.

[0089] In operation S301, a triple is input into an embedding representation model to be trained, and a predicted triple is output, wherein the embedding representation in the predicted triple is obtained by adjusting the random variables in the triple by the embedding representation model.

[0090] In operation S302, the predicted triples are verified using the objective function to generate a verification result.

[0091] In operation S303, when the verification result meets the preset conditions, the trained embedding representation model is output.

[0092] According to an embodiment of the present disclosure, in operation S204 and / or operation S303, the preset conditions include: the difference between the embedding representation of any tail entity among multiple tail entities in the tail entity neighborhood and the first embedding representation is less than a preset threshold, wherein the first embedding representation is obtained by adding the embedding representation of the head entity and the embedding representation of the relationship.

[0093] According to an embodiment of the present disclosure, since the head entity, relationship and tail entity in a triple are represented by random vectors, they cannot fully and accurately reflect the correlation between the head entity, relationship and tail entity. Therefore, it is necessary to use an embedded representation model to adjust the random vectors of the head entity, relationship and tail entity in the triple.

[0094] According to an embodiment of the present disclosure, for example, in a triplet, the random vector of the head entity may be [0.32, 0.54, 0.78, 0.45], the random vector of the relationship may be [0.12, 0.14, 0.68, 0.85], and the random vector of a tail entity in the neighborhood of the tail entity may be [0.47, 0.85, 0.99, 0.90].

[0095] According to an embodiment of the present disclosure, a triple is input into the embedding representation model to be trained, and the embedding representation of the predicted head entity in the output predicted triple can be [0.12, 0.23, 0.45, 0.61], the embedding representation of the predicted relationship can be [0.21, 0.33, 0.34, 0.12], and the embedding representation of a predicted tail entity in the neighborhood of the predicted tail entity can be [0.32, 0.57, 0.82, 0.70], that is, the embedding representation model adjusts the random vectors of the head entity, relationship and tail entity in the triple.

[0096] It should be noted that the above specific examples are only used to help those skilled in the art understand the present disclosure, and are not intended to make any improper limitations on the present disclosure.

[0097] According to an embodiment of the present disclosure, the objective function can be expressed by the following formula (1).

[0098]

[0099] Among them, E represents the entity set in the sample knowledge graph, h represents the embedded representation of the predicted head entity, t represents the embedded representation of the predicted tail entity, r represents the embedded representation of the predicted relationship, and L(h) represents the tail entity neighborhood of the head entity h.

[0100] According to an embodiment of the present disclosure, when the verification result does not meet the preset conditions, another triplet can be obtained from the sample knowledge graph, and the embedded representation model can be optimized and trained using the triplet until the predicted triplet output by the embedded representation model satisfies the verification and the verification result generated meets the preset conditions.

[0101] Figure 4 A flowchart for obtaining an initial knowledge graph according to an embodiment of the present disclosure is schematically shown.

[0102] like Figure 4 As shown, the acquisition of the initial knowledge graph in this embodiment includes operations S401 to S404.

[0103] In operation S401, an initial knowledge set is obtained.

[0104] According to the embodiments of the present disclosure, the knowledge corresponding to the book can be obtained by performing OCR recognition on the book, but it is not limited to this. For example, the initial knowledge set can also be obtained by crawling knowledge websites in a certain field using a crawler.

[0105] According to an embodiment of the present disclosure, the knowledge in the initial knowledge set may include any one or more of characters, words, phrases, sentences or pictures.

[0106] According to the embodiments of the present disclosure, the knowledge in the initial knowledge set is generally messy and includes interference information. Therefore, it is necessary to filter the knowledge in the initial knowledge set.

[0107] In operation S402, a first keyword filtering process is performed on the initial knowledge set to generate a first knowledge set.

[0108] According to an embodiment of the present disclosure, the first keyword may include, for example, sensitive words, adjectives, articles, quantifiers, and other words without actual meaning.

[0109] In operation S403, second keywords are extracted from the first knowledge set to generate a second knowledge set.

[0110] According to an embodiment of the present disclosure, the second keyword may include, for example, nouns, verbs, and adverbs.

[0111] In operation S404, an initial knowledge graph is generated according to the second knowledge set, wherein the head entity, the relationship, and the tail entity in the initial knowledge graph are represented by first preset characters, wherein the first preset characters include Chinese characters or letters.

[0112] According to an embodiment of the present disclosure, the nouns in the second knowledge set may be used as entities and the adverbs as relationships to construct an initial knowledge graph.

[0113] According to an embodiment of the present disclosure, the head entity, relationship, and tail entity in the initial knowledge graph are represented by first preset characters, wherein the first preset characters include Chinese characters or letters.

[0114] According to an embodiment of the present disclosure, since the head entity, relationship and tail entity in the initial knowledge graph generated according to the second knowledge set are represented by Chinese characters or letters, the triples obtained from the initial knowledge graph cannot be directly input as training samples into the embedding representation model to be trained to train the model, and thus, the initial knowledge graph needs to be processed.

[0115] According to an embodiment of the present disclosure, the above operation S202 includes the following operations:

[0116] The head entity, relationship, and tail entity in the initial knowledge graph are randomly initialized to convert the first preset character into a random vector to generate a sample knowledge graph.

[0117] Based on the training method of the above-mentioned embedding representation model, the present disclosure also provides a question-answering method based on knowledge graph. Figure 5 The method is described in detail.

[0118] Figure 5 A flowchart of a knowledge graph-based question-answering method according to an embodiment of the present disclosure is schematically shown.

[0119] like Figure 5 As shown, the knowledge graph-based question-answering method of this embodiment includes operations S501 to S504.

[0120] In operation S501, a first knowledge graph is obtained.

[0121] In operation S502, the first knowledge graph is input into an embedding representation model, and a first embedding representation knowledge graph is output, wherein the embedding representation model is trained by the embedding representation model training method provided by an embodiment of the present disclosure.

[0122] According to an embodiment of the present disclosure, the first knowledge graph may be a first knowledge graph generated based on a knowledge set.

[0123] According to an embodiment of the present disclosure, the head entity, relationship and tail entity in the first knowledge graph can be represented by Chinese characters or letters.

[0124] According to an embodiment of the present disclosure, by processing the first knowledge graph with an embedding representation model, a first embedding representation model is obtained, so that Chinese characters or letters in the first knowledge graph can be converted into corresponding embedding representations, and the embedding representations of the head entity, relationship and tail entity in the first knowledge graph satisfy that the difference between the embedding representation of any tail entity among multiple tail entities in the neighborhood of the tail entity and the first embedding representation is less than a preset threshold.

[0125] In operation S503, description information of the question to be answered is obtained, wherein the description information of the question to be answered includes a head entity and a relationship.

[0126] According to an embodiment of the present disclosure, the description information of the question to be answered may include, for example, "What sport does Xiao Ming like?" In the description information of the question to be answered, "Xiao Ming" may be the head entity, and "like" may be a relationship.

[0127] In operation S504, the description information of the question to be answered is input into the first embedding representation knowledge graph, and an answer corresponding to the description information of the question to be answered is output, wherein the answer includes a target entity determined according to the head entity and the relationship.

[0128] Figure 6 A flowchart schematically illustrates inputting description information of a question to be answered into a first embedded representation knowledge graph and outputting an answer corresponding to the description information of the question to be answered according to an embodiment of the present disclosure.

[0129] like Figure 6 As shown, this embodiment inputs the description information of the question to be answered into the first embedded representation knowledge graph, and outputs the answer corresponding to the description information of the question to be answered, including operations S601 to S605.

[0130] In operation S601, an embedded representation of a head entity and an embedded representation of a relationship are determined according to description information of a question to be answered.

[0131] According to an embodiment of the present disclosure, after obtaining the description information of the question to be answered, based on the head entities and relationships included in the description information of the question to be answered, an embedded representation corresponding to the head entities and relationships included in the description information of the question to be answered is determined from the first embedded representation knowledge graph.

[0132] In operation S602, a second embedding representation is obtained according to the embedding representation of the head entity and the embedding representation of the relationship.

[0133] According to an embodiment of the present disclosure, for example, the second embedding representation may be obtained by summing the embedding representation of the head entity and the embedding representation of the relationship.

[0134] In operation S603, multiple similarity distances between the second embedding representation and the embedding representations of multiple entities in the first embedding representation knowledge graph are calculated.

[0135] In operation S604, a minimum similarity distance value is determined from among the plurality of similarity distances.

[0136] In operation S605 , an entity corresponding to the minimum similarity distance is determined as a target entity.

[0137] According to the embodiments of the present disclosure, since the knowledge of the real world is constantly increasing and updating, although the knowledge graph can also be updated and modified, it is still unable to fully represent the knowledge of the real world. That is to say, due to the updating of the real world knowledge, some problems in the real world will not be answered in the knowledge graph.

[0138] According to an embodiment of the present disclosure, by determining the entity corresponding to the minimum similarity distance as the target entity, the connection that does not exist in the first embedding representation knowledge graph can be predicted to obtain the answer to the question to be answered.

[0139] According to an embodiment of the present disclosure, operation S603 includes:

[0140] Calculate multiple Euclidean distances between the second embedding representation and the embedding representations of multiple entities in the first embedding representation knowledge graph respectively according to the Euclidean distance calculation method; or

[0141] According to the cosine distance calculation method, multiple cosine distances between the second embedding representation and the embedding representations of multiple entities in the first embedding representation knowledge graph are calculated.

[0142] Based on the above-mentioned training method of the embedded representation model, the present disclosure also provides a training device for the embedded representation model. Figure 7 The device is described in detail.

[0143] Figure 7 The structural block diagram of the training device of the embedded representation model according to the embodiment of the present disclosure is schematically shown.

[0144] like Figure 7 As shown, the training device 700 for the embedded representation model of this embodiment includes a first acquisition module 701, a preprocessing module 702, a second acquisition module 703 and a training module 704.

[0145] The first acquisition module 701 is used to acquire an initial knowledge graph. In one embodiment, the first acquisition module 701 can be used to perform the operation S201 described above, which will not be described in detail here.

[0146] The preprocessing module 702 is used to preprocess the initial knowledge graph to generate a sample knowledge graph, wherein the sample knowledge graph includes a head entity, a relationship, and a tail entity, wherein the head entity, the relationship, and the tail entity in the sample knowledge graph are represented by random vectors. In one embodiment, the preprocessing module 702 can be used to perform the operation S202 described above, which will not be repeated here.

[0147] The second acquisition module 703 is used to obtain triples from the sample knowledge graph, wherein the triples include a head entity, a relationship, and a tail entity neighborhood, wherein the tail entity neighborhood includes a set of multiple tail entities having the same relationship as the head entity. In one embodiment, the second acquisition module 730 can be used to perform the operation S203 described above, which will not be repeated here.

[0148] The training module 704 is used to train the embedding representation model to be trained using the triples to obtain a trained embedding representation model, wherein the embedding representations of the head entity, the relationship, and the neighborhood of the tail entity in the embedding representation triples output by the trained embedding representation model meet preset conditions. In one embodiment, the training module 704 can be used to perform the operation S204 described above, which will not be repeated here.

[0149] According to an embodiment of the present disclosure, the training module 704 includes an input unit, a verification unit, and an output unit.

[0150] The input unit is used to input the triplet into the embedding representation model to be trained, and output the predicted triplet, wherein the embedding representation in the predicted triplet is obtained by adjusting the random variables in the triplet by the embedding representation model.

[0151] The verification unit is used to verify the predicted triples using the objective function and generate a verification result.

[0152] The output unit is used to output the trained embedding representation model when the verification result meets the preset conditions.

[0153] According to an embodiment of the present disclosure, the first acquisition module 701 includes a first acquisition unit, a filtering unit, a keyword extraction unit and a generation unit.

[0154] The first acquisition unit is used to acquire an initial knowledge set.

[0155] The filtering unit is used to perform a first keyword filtering process on the knowledge set to generate a first knowledge set.

[0156] The keyword extraction unit is used to extract second keywords from the first knowledge set to generate a second knowledge set.

[0157] A generating unit is used to generate an initial knowledge graph according to a second knowledge set, wherein the head entity, the relationship and the tail entity in the initial knowledge graph are represented by a first preset character, wherein the first preset character includes Chinese characters or letters.

[0158] According to an embodiment of the present disclosure, the head entity, relationship, and tail entity in the initial knowledge graph are represented by first preset characters, wherein the first preset characters include Chinese characters or letters.

[0159] According to an embodiment of the present disclosure, the pre-processing module 702 includes a random initialization unit.

[0160] The random initialization unit is used to randomly initialize the head entity, relationship and tail entity in the initial knowledge graph to convert the first preset character into a random vector to generate a sample knowledge graph.

[0161] Based on the above-mentioned question-answering method based on knowledge graph, the present disclosure also provides a question-answering device based on knowledge graph. Figure 8 The device is described in detail.

[0162] Figure 8 The structural block diagram of the question-answering device based on the knowledge graph according to an embodiment of the present disclosure is schematically shown.

[0163] like Figure 8 As shown, the knowledge graph-based question-answering device 800 of this embodiment includes a third acquisition module 801, an input module 802, a fourth acquisition module 803 and an output module 804.

[0164] The third acquisition module 801 is used to acquire the first knowledge graph. In one embodiment, the third acquisition module 801 can be used to perform the operation S501 described above, which will not be described in detail here.

[0165] The input module 802 is used to input the first knowledge graph into the embedding representation model and output the first embedding representation knowledge graph, wherein the embedding representation model is trained by the embedding representation model training method provided by the embodiment of the present disclosure. In one embodiment, the input module 802 can be used to perform the operation S502 described above, which will not be repeated here.

[0166] The fourth acquisition module 803 is used to acquire description information of the question to be answered, wherein the description information of the question to be answered includes a head entity and a relationship. In one embodiment, the fourth acquisition module 830 can be used to perform the operation S503 described above, which will not be described in detail here.

[0167] The output module 804 is used to input the description information of the question to be answered into the first embedding representation knowledge graph, and output an answer corresponding to the description information of the question to be answered, wherein the answer includes a target entity determined according to the head entity and the relationship. In one embodiment, the output module 804 can be used to perform the operation S504 described above, which will not be repeated here.

[0168] According to an embodiment of the present disclosure, the output module 804 includes a first determining unit, a second determining unit, a calculating unit, a third determining unit, and a fourth determining unit.

[0169] The first determining unit is used to determine the embedding representation of the head entity and the embedding representation of the relationship according to the description information of the question to be answered.

[0170] The second determining unit is used to obtain a second embedding representation according to the embedding representation of the head entity and the embedding representation of the relationship.

[0171] A calculation unit is used to calculate multiple similarity distances between the second embedding representation and the embedding representations of multiple entities in the first embedding representation knowledge graph.

[0172] The third determining unit is used to determine a minimum similarity distance from multiple similarity distances.

[0173] The fourth determining unit is used to determine the entity corresponding to the minimum similarity distance as the target entity.

[0174] According to an embodiment of the present disclosure, the computing unit includes a first computing subunit and / or a second computing subunit.

[0175] The first calculation subunit is used to calculate multiple Euclidean distances between the second embedding representation and the embedding representations of multiple entities in the first embedding representation knowledge graph according to the Euclidean distance calculation method.

[0176] The second calculation subunit is used to calculate multiple cosine distances between the second embedding representation and the embedding representations of multiple entities in the first embedding representation knowledge graph according to the cosine distance calculation method.

[0177] According to an embodiment of the present disclosure, any multiple modules among the first acquisition module 701, the preprocessing module 702, the second acquisition module 703, the training module 704, the third acquisition module 801, the input module 802, the fourth acquisition module 803 and the output module 804 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the first acquisition module 701, the preprocessing module 702, the second acquisition module 703, the training module 704, the third acquisition module 801, the input module 802, the fourth acquisition module 803 and the output module 804 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware and firmware or by an appropriate combination of any of them. Alternatively, at least one of the first acquisition module 701, the preprocessing module 702, the second acquisition module 703, the training module 704, the third acquisition module 801, the input module 802, the fourth acquisition module 803 and the output module 804 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be performed.

[0178] Fig. 9 A block diagram of an electronic device suitable for implementing a training method for an embedded representation model and a question-answering method based on a knowledge graph according to an embodiment of the present disclosure is schematically shown.

[0179] like Fig. 9 As shown, the electronic device 900 according to an embodiment of the present disclosure includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 into a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include an onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0180] In RAM 903, various programs and data required for the operation of electronic device 900 are stored. Processor 901, ROM 902 and RAM 903 are connected to each other via bus 904. Processor 901 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 902 and / or RAM 903. It should be noted that the program can also be stored in one or more memories other than ROM 902 and RAM 903. Processor 901 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0181] According to an embodiment of the present disclosure, the electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to the bus 904. The electronic device 900 may further include one or more of the following components connected to the I / O interface 905: an input portion 906 including a keyboard, a mouse, etc.; an output portion 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 908 including a hard disk, etc.; and a communication portion 909 including a network interface card such as a LAN card, a modem, etc. The communication portion 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage portion 908 as needed.

[0182] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0183] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus, or a device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 902 and / or RAM 903 described above and / or one or more memories other than ROM 902 and RAM 903.

[0184] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the training method of the embedded representation model and the question-answering method based on the knowledge graph provided in the embodiment of the present disclosure.

[0185] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 901. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0186] In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and downloaded and installed through the communication part 909, and / or installed from a removable medium 911. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0187] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0188] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0189] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0190] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0191] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A method for training an embedding representation model, comprising: Obtain the initial knowledge graph; Preprocessing the initial knowledge graph to generate a sample knowledge graph, wherein the sample knowledge graph includes a head entity, a relationship, and a tail entity, wherein the head entity, the relationship, and the tail entity in the sample knowledge graph are represented by random vectors; Acquire a triple from the sample knowledge graph, wherein the triple includes a head entity, a relationship, and a tail entity neighborhood, wherein the tail entity neighborhood includes a set consisting of multiple tail entities having the same relationship as the head entity; Inputting the triple into an embedding representation model to be trained, and outputting a predicted triple, wherein the embedding representation in the predicted triple is obtained by adjusting the random variables in the triple by the embedding representation model, and the predicted triple includes a predicted head entity, a predicted tail entity, and a predicted relationship; The predicted triplet is verified using the objective function to generate a verification result, wherein the objective function F is represented by the following formula (1): (1); Wherein, E represents the entity set in the sample knowledge graph, h represents the embedded representation of the predicted head entity, t represents the embedded representation of the predicted tail entity, and r represents the embedded representation of the predicted relationship. Represents the tail entity neighborhood of the head entity h; When the verification result does not meet the preset conditions, obtaining a substitute triple from the sample knowledge graph, and optimizing and training the embedding representation model based on the substitute triple; In the case where the verification result satisfies the preset condition, outputting the trained embedding representation model, wherein the embedded representation of the head entity, the relationship, and the neighborhood of the tail entity in the embedded representation triplet output by the trained embedding representation model satisfies the preset condition; The preset conditions include: A difference between an embedding representation of any tail entity among the plurality of tail entities in the neighborhood of the tail entity and a first embedding representation is less than a preset threshold, wherein the first embedding representation is obtained by adding an embedding representation of the head entity to an embedding representation of the relationship.

2. The method according to claim 1, wherein: The obtaining of the initial knowledge graph comprises: Acquire an initial knowledge set; Performing a first keyword filtering process on the initial knowledge set to generate a first knowledge set; Extracting second keywords from the first knowledge set to generate a second knowledge set; The initial knowledge graph is generated according to the second knowledge set, wherein the head entity, relationship and tail entity in the initial knowledge graph are represented by first preset characters, wherein the first preset characters include Chinese characters or letters.

3. The method according to claim 1, wherein: The head entity, the relationship and the tail entity in the initial knowledge graph are represented by first preset characters, wherein the first preset characters include Chinese characters or letters; The preprocessing of the initial knowledge graph to generate a sample knowledge graph includes: The head entity, the relationship, and the tail entity in the initial knowledge graph are randomly initialized to convert the first preset character into a random vector to generate the sample knowledge graph.

4. A question-answering method based on a knowledge graph, comprising: Obtain a first knowledge graph; Inputting the first knowledge graph into an embedding representation model, and outputting a first embedding representation knowledge graph, wherein the embedding representation model is trained by the embedding representation model training method according to any one of claims 1 to 3; Obtaining description information of the question to be answered, wherein the description information of the question to be answered includes a head entity and a relationship; The description information of the question to be answered is input into the first embedding representation knowledge graph, and an answer corresponding to the description information of the question to be answered is output, wherein the answer includes a target entity determined according to the head entity and the relationship.

5. The method according to claim 4, wherein: The step of inputting the description information of the question to be answered into the first embedding representation knowledge graph and outputting an answer corresponding to the description information of the question to be answered comprises: Determining an embedded representation of the head entity and an embedded representation of the relationship according to the description information of the question to be answered; Obtaining a second embedding representation according to the embedding representation of the head entity and the embedding representation of the relationship; Calculate multiple similarity distances between the second embedding representation and the embedding representations of multiple entities in the first embedding representation knowledge graph; Determine a minimum similarity distance from the multiple similarity distances; An entity corresponding to the minimum similarity distance is determined as the target entity.

6. According to the method of claim 5, the calculating of multiple similarity distances between the second embedding representation and the embedding representations of multiple entities in the first embedding representation knowledge graph comprises: Calculate multiple Euclidean distances between the second embedding representation and the embedding representations of multiple entities in the first embedding representation knowledge graph according to the Euclidean distance calculation method; or According to the cosine distance calculation method, multiple cosine distances between the second embedding representation and the embedding representations of multiple entities in the first embedding representation knowledge graph are calculated.

7. A training device for an embedded representation model, comprising: The first acquisition module is used to acquire an initial knowledge graph; A preprocessing module, configured to preprocess the initial knowledge graph to generate a sample knowledge graph, wherein the sample knowledge graph includes a head entity, a relationship, and a tail entity, wherein the head entity, the relationship, and the tail entity in the sample knowledge graph are represented by random vectors; and A second acquisition module is used to acquire a triple from the sample knowledge graph, wherein the triple includes a head entity, a relationship, and a tail entity neighborhood, wherein the tail entity neighborhood includes a set consisting of multiple tail entities having the same relationship with the head entity; A training module is used to input the triple into the embedding representation model to be trained, and output a predicted triple, wherein the embedding representation in the predicted triple is obtained by adjusting the random variables in the triple by the embedding representation model, and the predicted triple includes a predicted head entity, a predicted tail entity, and a predicted relationship; and verify the predicted triple using an objective function to generate a verification result, wherein the objective function F is represented by the following formula (1): (1); where E represents the entity set in the sample knowledge graph, h represents the embedded representation of the predicted head entity, t represents the embedded representation of the predicted tail entity, and r represents the embedded representation of the predicted relationship. The method further comprises: obtaining a replacement triplet from the sample knowledge graph and optimizing the embedding representation model based on the replacement triplet when the verification result does not meet the preset conditions, outputting the trained embedding representation model, wherein the embedding representation of the head entity, the relationship and the tail entity neighborhood in the embedding representation triplet output by the trained embedding representation model meets the preset conditions; the preset conditions include: A difference between an embedding representation of any tail entity among the plurality of tail entities in the neighborhood of the tail entity and a first embedding representation is less than a preset threshold, wherein the first embedding representation is obtained by adding an embedding representation of the head entity to an embedding representation of the relationship.

8. A question-answering device based on a knowledge graph, comprising: A third acquisition module is used to acquire the first knowledge graph; An input module, used to input the first knowledge graph into an embedding representation model, and output a first embedding representation knowledge graph, wherein the embedding representation model is trained by the embedding representation model training method according to any one of claims 1 to 3; A fourth acquisition module is used to acquire description information of the question to be answered, wherein the description information of the question to be answered includes a head entity and a relationship; An output module is used to input the description information of the question to be answered into the first embedding representation knowledge graph, and output an answer corresponding to the description information of the question to be answered, wherein the answer includes a target entity determined according to the head entity and the relationship.

9. An electronic device, comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors execute the method according to any one of claims 1 to 6.

10. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method according to any one of claims 1 to 6.

11. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images