Relationship extraction method, device, equipment, storage medium and computer program product

Abstract

The present application discloses a relationship extraction method, apparatus, device, storage medium and computer program product, the method comprising: when predicting the i-th predicted character in an entity relationship sequence corresponding to a text to be processed, determining the representation information required for generating the i-th predicted character; based on the representation information required for generating the i-th predicted character, determining the generation probability of each reference character in a preset vocabulary as the i-th predicted character; each character to be processed belongs to the preset vocabulary; based on the similarity between the representation information required for generating the i-th predicted character and the representation information of each character to be processed in the text to be processed, determining the probability of each character to be processed as a pointer to the i-th predicted character; according to the generation probability of each reference character as the i-th predicted character and the probability of each character to be processed as a pointer to the i-th predicted character, determining the i-th predicted character from the preset vocabulary and adding it to the entity relationship sequence corresponding to the text to be processed.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a relationship extraction method, apparatus, device, storage medium and computer program product. Background Art

[0002] Relation extraction refers to mining the relationship between entities contained in a text, and is a hot research issue in the field of natural language processing (NLP). For example, there is a text "User A, born in region a on September 27, 1961", which contains three entities: "User A", "September 27, 1961" and "Region a". It can be seen that there is a "birth time" relationship between the entities "User A" and "September 27, 1961", and there is a "birth place" relationship between the entities "User A" and "Region a". Therefore, relationship extraction aims to mine the two entity relationship sequences (User A, birth time, September 27, 1961) and (User A, birth place, region a) from the text. How to improve the accuracy of mining the relationship between the entities contained in the text is the research focus in the field of relationship extraction. Summary of the invention

[0003] The embodiments of the present application provide a relationship extraction method, apparatus, device, storage medium and computer program product, which can improve the accuracy of mining the relationship between entities contained in a text.

[0004] On the one hand, an embodiment of the present application provides a relationship extraction method, including:

[0005] Obtaining representation information of each character to be processed in the text to be processed;

[0006] When predicting the i-th predicted character in the entity relationship sequence corresponding to the to-be-processed text, determining the representation information required to generate the i-th predicted character; i is a positive integer;

[0007] Based on the representation information required to generate the i-th predicted character, determining the generation probability of each reference character in a preset vocabulary as the i-th predicted character; each character to be processed in the to-be-processed text belongs to the preset vocabulary;

[0008] Determining the probability of each of the characters to be processed being a pointer to the i-th predicted character based on the similarity between the representation information required to generate the i-th predicted character and the representation information of each of the characters to be processed;

[0009] According to the generation probability of each reference character as the i-th predicted character and the probability of each to-be-processed character as a pointer to the i-th predicted character, the i-th predicted character is determined from the preset vocabulary, and the i-th predicted character is added to the entity relationship sequence corresponding to the to-be-processed text.

[0010] On the one hand, an embodiment of the present application provides a relationship extraction device, including:

[0011] A representation information determination unit, used to obtain the representation information of each character to be processed in the text to be processed;

[0012] The representation information determination unit is further used to determine the representation information required to generate the i-th predicted character when predicting the i-th predicted character in the entity relationship sequence corresponding to the to-be-processed text; i is a positive integer;

[0013] A probability determination unit, configured to determine the generation probability of each reference character in a preset vocabulary as the i-th predicted character based on the representation information required to generate the i-th predicted character; each character to be processed in the to-be-processed text belongs to the preset vocabulary;

[0014] The probability determination unit is further used to determine the probability that each to-be-processed character serves as a pointer to the i-th predicted character based on the similarity between the representation information required to generate the i-th predicted character and the representation information of each to-be-processed character;

[0015] A character determination unit is used to determine the i-th predicted character from the preset vocabulary according to the generation probability of each reference character as the i-th predicted character and the probability of each to-be-processed character as a pointer to the i-th predicted character, and add the i-th predicted character to the entity relationship sequence corresponding to the to-be-processed text.

[0016] On the one hand, an embodiment of the present application provides a relationship extraction device, characterized in that the relationship extraction device includes an input interface and an output interface, and further includes:

[0017] a processor adapted to implement one or more instructions; and,

[0018] A computer storage medium storing one or more instructions, wherein the one or more instructions are suitable for being loaded by the processor and executing the above-mentioned relation extraction method.

[0019] On the one hand, an embodiment of the present application provides a computer storage medium, characterized in that computer program instructions are stored in the computer storage medium, and when the computer program instructions are executed by a processor, they are used to execute the above-mentioned relationship extraction method.

[0020] On the one hand, an embodiment of the present application provides a computer program product, which includes a computer program stored in a computer storage medium; a processor of a relationship extraction device reads the computer program from the computer storage medium, and the processor executes the computer program, so that the relationship extraction device performs the above-mentioned relationship extraction method.

[0021] In an embodiment of the present application, in the process of predicting the entity relationship sequence corresponding to the text to be processed, the representation information of each character to be processed in the text to be processed can be obtained; when predicting the i-th predicted character in the entity relationship sequence corresponding to the text to be processed, the representation information required to generate the i-th predicted character is determined; based on the representation information required to generate the i-th predicted character, the generation probability of each reference character in the preset vocabulary as the i-th predicted character is determined; based on the similarity between the representation information required to generate the i-th predicted character and the representation information of each character to be processed, the probability of each character to be processed as a pointer to the i-th predicted character is determined; based on the generation probability of each reference character as the i-th predicted character and the probability of each character to be processed as a pointer to the i-th predicted character, the i-th predicted character is determined from the preset vocabulary, and the i-th predicted character is added to the entity relationship sequence corresponding to the text to be processed until the entity relationship sequence corresponding to the text to be processed is predicted; wherein, i is a positive integer, and each character to be processed in the text to be processed belongs to the preset vocabulary. When predicting the i-th predicted character in the entity relationship sequence corresponding to the text to be processed, the probability of generating the i-th predicted character by referring to each reference character in the determined preset vocabulary can be combined with the probability of each character to be processed in the text to be processed being a pointer to the i-th predicted character, thereby increasing the probability that the generated i-th predicted character is the character to be processed in the text to be processed. In other words, the probability of entities or relationships not included in the text to be processed appearing in the predicted entity relationship sequence corresponding to the text to be processed can be reduced, thereby improving the accuracy of mining the relationships between entities contained in the text. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 It is a schematic diagram of predicting an entity relationship sequence corresponding to a text to be processed based on a relationship extraction model provided in an embodiment of the present application;

[0024] Figure 2 It is a flowchart of a relationship extraction method provided in an embodiment of the present application;

[0025] Figure 3 is a schematic diagram of determining the i-th predicted character based on generation probability and pointer probability provided by an embodiment of the present application;

[0026] Figure 4 is a flowchart of another relationship extraction method provided in an embodiment of the present application;

[0027] Figure 5 is a flowchart of another relationship extraction method provided in an embodiment of the present application;

[0028] Figure 6 is a structural schematic diagram of a relationship extraction device provided in an embodiment of the present application;

[0029] Figure 7 It is a structural schematic diagram of a relationship extraction device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0031] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0032] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology (Computer Vision, CV), speech processing technology, natural language processing technology, and machine learning (Machine Learning, ML) / deep learning (Deep Learning, DL) and other major directions.

[0033] Natural language processing mainly studies various theories and methods that can achieve effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field will involve natural language, that is, the language people use in daily life, so it is closely related to the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question answering, knowledge graph, relationship extraction and other technologies. The present application provides a relationship extraction scheme based on the relationship extraction technology in the field of natural language processing, which can improve the accuracy of mining the relationship between entities contained in the text from the text; the relationship extraction scheme indicates: when predicting the i-th predicted character in the entity relationship sequence corresponding to the text to be processed, the generation probability of each reference character in the preset vocabulary as the i-th predicted character can be determined based on the representation information required to generate the i-th predicted character; and based on the similarity between the representation information required to generate the i-th predicted character and the representation information of each character to be processed in the obtained text to be processed, the probability of each character to be processed as a pointer to the i-th predicted character is determined; then, according to the generation probability of each reference character as the i-th predicted character and the probability of each character to be processed as the pointer to the i-th predicted character, the i-th predicted character is determined from the preset vocabulary, and the i-th predicted character is added to the entity relationship sequence corresponding to the text to be processed until the entity relationship sequence corresponding to the text to be processed is predicted; wherein i is a positive integer, and each character to be processed in the text to be processed belongs to the preset vocabulary. Among them, the entity relationship sequence corresponding to the text to be processed includes at least two entities and at least one entity relationship. For example, if the text to be processed is "Du Fu's courtesy name Zimei, and his pseudonym Shaoling Ye Lao", the entity relationship sequence corresponding to the text to be processed can be: (Du Fu, courtesy name, Zimei)(Du Fu, pseudonym, Shaoling Ye Lao), then the entities included in the entity relationship sequence corresponding to the text to be processed are: Du Fu, Zimei, Shaoling Ye Lao, and the entity relationships included in the entity relationship sequence corresponding to the text to be processed are: courtesy name, pseudonym; the entity relationship "custom name" indicates the relationship between the entities "Du Fu" and "Zimei", and the entity relationship "pseudonym" indicates the relationship between the entities "Du Fu" and "Shaoling Ye Lao".

[0034] The above-mentioned relation extraction scheme can be used to enrich the knowledge of the knowledge graph in the knowledge base, that is, the entity relationship in the text can be mined based on the above-mentioned relation extraction scheme, and the entity relationship sequence corresponding to the text can be predicted, and then the knowledge graph can be supplemented based on the predicted entity relationship sequence corresponding to the text. Knowledge graphs are widely used in many fields; for example, in the search field, users may search for knowledge-based questions such as "What is Li Bai's name?" and "How high is Mount Everest?"; these two questions can be parsed into "(Li Bai, name,?)" and "(Mount Everest, altitude,?)", and then the knowledge graph in the knowledge base can be queried based on this to obtain the corresponding answers; for example, in the recommendation field, the knowledge in the knowledge graph can be combined with the recommendation model to achieve better content recommendations for users. Due to the existence of massive texts on the Internet, rich knowledge is contained, and the relation extraction scheme proposed in this application can well mine the entity relationships in the text, that is, it can well mine the knowledge in the text, so it can well supplement the knowledge in the knowledge graph.

[0035] In a specific implementation, the relationship extraction scheme proposed in the present application can be executed by a relationship extraction device, which can be a terminal device or a server; the terminal device here may include but is not limited to: computers, smart phones, tablets, laptops, intelligent voice interaction devices, smart home appliances, vehicle terminals, smart wearable devices, etc.; the server here can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), as well as big data and artificial intelligence platforms.

[0036] The relation extraction scheme proposed in the present application can be implemented by calling a relation extraction model, which can be a seq2seq model in a neural network. The relation extraction model can include an encoding module, a decoding module, and a character prediction module. The encoding module can be used to perform relation feature encoding processing on the text to be processed to obtain a representation vector of each character to be processed in the text to be processed. The decoding module can cooperate with the character prediction module to predict the entity relationship sequence corresponding to the text to be processed based on the representation vector of each character to be processed and the decoding reference character. Wherein, when predicting the i-th predicted character in the entity relationship sequence corresponding to the text to be processed, when i=1, the decoding reference character is a special decoding character, and when i>1, the decoding reference character is the i-1-th predicted character; the special decoding character can be used to prompt the decoding module to start cooperating with the character prediction module to predict the entity relationship sequence corresponding to the text to be processed. The special decoding character can be set according to specific needs. For example, the special decoding character can be set to "B". That is to say, the first predicted character in the entity relationship sequence corresponding to the text to be processed is predicted based on the representation vectors of each character to be processed and the special decoding character, the second predicted character in the entity relationship sequence corresponding to the text to be processed is predicted based on the representation vectors of each character to be processed and the first predicted character, the third predicted character in the entity relationship sequence corresponding to the text to be processed is predicted based on the representation vectors of each character to be processed and the second predicted character, and so on, until the entity relationship sequence corresponding to the text to be processed is obtained.

[0037] Optionally, the encoding module can be an encoder based on a transformer structure (i.e., encoder), and the decoding module can be a decoder based on a transformer structure (i.e., decoder); the encoding module can also be a GRU (Gated Recurrent Unit) model in a Long Short-Term Memory (LSTM) network or a variant model of a Long Short-Term Memory network, etc.; this application will be described later with the encoding module being an encoder based on a transformer structure and the decoding module being a decoder based on a transformer structure. Figure 1, is a schematic diagram of an entity relationship sequence corresponding to a text to be processed based on a relationship extraction model provided in an embodiment of the present application. Assuming that the text to be processed is "Du Fu, whose courtesy name is Zimei and whose pseudonym is Shaoling Ye Lao", the encoding module in the relationship extraction model can be called to perform relationship feature encoding processing on the text to be processed to obtain a representation vector of each character to be processed in the text to be processed; the decoding module and the character prediction module in the relationship extraction model can be called to predict the entity relationship sequence corresponding to the text to be processed based on the representation vector of each character to be processed and the decoded reference character; specifically, the first predicted character can be predicted based on the representation vector of each character to be processed and the special decoded character, the second predicted character can be predicted based on the representation vector of each character to be processed and the first predicted character, and the third predicted character can be predicted based on the representation vector of each character to be processed and the second predicted character, until the entity relationship sequence corresponding to the text to be processed is obtained, which can be: (Du Fu, courtesy name, Zimei)(Du Fu, pseudonym, Shaoling Ye Lao).

[0038] It is particularly important to note that in the specific implementation of this application, user-related data is involved. For example, when the text to be processed is text generated by the user, when the embodiment of this application is applied to a specific product or technology, the user's permission or consent must be obtained, and the collection, use and processing of the relevant data must comply with local laws, regulations and standards.

[0039] Based on the above relationship extraction scheme, the present application embodiment provides a relationship extraction method. Figure 2 , which is a flow chart of a relationship extraction method provided in an embodiment of the present application. Figure 2 The relationship extraction method shown can be executed by a relationship extraction device, and specifically can be executed by calling a relationship extraction model. Figure 2 The relationship extraction method shown may include the following steps:

[0040] S201, obtaining representation information of each character to be processed in the text to be processed.

[0041] In one embodiment, the text to be processed may include multiple characters to be processed, and the representation information of each character to be processed may include a representation vector of each character to be processed; the representation vector of each character to be processed is obtained by calling the encoding module in the relationship extraction model to perform relationship feature encoding on the text to be processed; that is, the relationship extraction device can call the encoding module in the relationship extraction model to perform relationship feature encoding on the text to be processed to obtain the representation vector of each character to be processed in the text to be processed; the representation vector of a character to be processed may contain the relationship features between the character to be processed and other characters to be processed in the text to be processed, and the representation vector of each character to be processed may contain the entity relationship features of the text to be processed.

[0042] S202, when predicting the i-th predicted character in the entity relationship sequence corresponding to the text to be processed, determining the representation information required to generate the i-th predicted character.

[0043] In one embodiment, the representation information required to generate the i-th predicted character may include the representation vector required to generate the i-th predicted character; the representation vector required to generate the i-th predicted character is obtained by calling the decoding module in the relationship extraction model to decode the representation vectors of each character to be processed and the decoding reference character; wherein, i is a positive integer, when i=1, the decoding reference character is a special decoding character, and when i>1, the decoding reference character is the i-1-th predicted character. That is to say, the relationship extraction device can call the decoding module in the relationship extraction model to decode the representation vectors of each character to be processed and the decoding reference characters to obtain the representation vector required to generate the i-th predicted character; specifically, the relationship extraction device can call the decoding module in the relationship extraction model to decode the representation vectors of each character to be processed and the special decoding characters to obtain the representation vector required to generate the first predicted character, and then generate the first predicted character; the decoding module in the relationship extraction model can be called to decode the representation vectors of each character to be processed and the first predicted character to obtain the representation vector required to generate the second predicted character, and then generate the second predicted character; the decoding module in the relationship extraction model can be called to decode the representation vectors of each character to be processed and the second predicted character to obtain the representation vector required to generate the third predicted character, and then generate the third predicted character, and so on.

[0044] S203: Based on the representation information required to generate the i-th predicted character, determine the generation probability of each reference character in the preset vocabulary as the i-th predicted character.

[0045] In one embodiment, the preset vocabulary may include multiple reference characters. The preset vocabulary may be a vocabulary used when training the relation extraction model. The reference characters in the preset vocabulary may include at least characters in the sample used when training the relation extraction model and each character to be processed in the text to be processed; that is, each character to be processed in the text to be processed belongs to the preset vocabulary. The preset vocabulary may be constructed by a technician or may be a vocabulary commonly used in existing relation extraction tasks.

[0046] In one embodiment, the relationship extraction device can call the character prediction module in the relationship extraction model, and determine the generation probability of each reference character in the preset vocabulary as the i-th predicted character based on the representation information required to generate the i-th predicted character; in a specific implementation, when the representation information required to generate the i-th predicted character includes the representation vector required to generate the i-th predicted character, the representation vector required to generate the i-th predicted character can be processed by the fully connected layer in the character prediction module to obtain the feature value distribution pointing to each reference character in the preset vocabulary, and then the feature value distribution pointing to each reference character in the preset vocabulary can be normalized by a normalization layer in the character prediction module to obtain the generation probability of each reference character in the preset vocabulary as the i-th predicted character; wherein the normalization layer can be a softmax activation function layer.

[0047] Among them, when the representation vector required to generate the i-th predicted character is processed by the fully connected layer in the character prediction module to obtain the feature value distribution pointing to each reference character in the preset vocabulary, the model parameters of the fully connected layer in the character prediction module can be processed with the representation vector required to generate the i-th predicted character to obtain the feature value distribution pointing to each reference character in the preset vocabulary; if y is used i represents the representation vector required to generate the i-th predicted character. The number of reference characters in the preset vocabulary is N. The distribution of feature values ​​pointing to each reference character in the preset vocabulary can be given by the following formula 1.1:

[0048] Logits1 i =W 1 y i (1.1)

[0049] Among them, Logits1 i Logits1 represents the distribution of feature values ​​pointing to each reference character in the preset vocabulary. i ∈R N , W 1 are the model parameters of the fully connected layer in the character prediction module.

[0050] Taking the probability of obtaining the nth reference character among the N reference characters in the preset vocabulary as the i-th predicted character as an example, the ratio between the index value of the feature value of the nth reference character among the N reference characters in the preset vocabulary, which is based on the obtained feature value distribution pointing to each reference character in the preset vocabulary, and the sum of the index values ​​of the feature values ​​of each reference character in the N reference characters pointing to the preset vocabulary, can be used as the probability of the nth reference character being the i-th predicted character; the probability of the nth reference character being the i-th predicted character can be given by the following formula 1.2:

[0051]

[0052] Among them, Prob1 in Indicates the probability of generating the nth reference character as the ith predicted character, Logits1 in Logits1 represents the feature value of the nth reference character in the N reference characters in the preset vocabulary, in the obtained feature value distribution of each reference character in the preset vocabulary. ip It represents the feature value of the pth reference character among the N reference characters in the preset vocabulary in the obtained feature value distribution pointing to each reference character in the preset vocabulary.

[0053] S204: Determine the probability of each character to be processed being a pointer to the i-th predicted character based on the similarity between the representation information required to generate the i-th predicted character and the representation information of each character to be processed.

[0054] In one embodiment, the relationship extraction device can call the character prediction module in the relationship extraction model, and determine the probability of each character to be processed being a pointer to the i-th predicted character based on the similarity between the representation information required to generate the i-th predicted character and the representation information of each character to be processed; in a specific implementation, when the representation information of each character to be processed includes the representation vector of each character to be processed, and the representation information required to generate the i-th predicted character includes the representation vector required to generate the i-th predicted character, the representation vector of each character to be processed and the representation vector required to generate the i-th predicted character can be inner-producted to obtain the similarity distribution between the representation vector required to generate the i-th predicted character and the representation vector of each character to be processed; the similarity distribution is normalized to obtain the probability of each character to be processed being a pointer to the i-th predicted character. Among them, the normalization of the similarity distribution to obtain the probability of each character to be processed being a pointer to the i-th predicted character can be performed by a normalization layer in the character prediction module; the normalization layer can be a softmax activation function layer. If y is used i represents the representation vector required to generate the i-th predicted character, and the number of reference characters in the preset vocabulary is N; if the number of characters to be processed in the to-be-processed text is M, M is less than or equal to N, use x m Represents the characterization vector of the mth character to be processed among the M characters to be processed of the text to be processed, and X represents the matrix composed of the characterization vectors of each character to be processed among the M characters to be processed, X∈R M*d , where d can be the dimension of the representation vector of any character to be processed, then the similarity distribution can be given by the following formula 2.1:

[0055] Logits2 i =Xy i (2.1)

[0056] Among them, Logits2 i Represents the similarity distribution between the representation vector required to generate the i-th predicted character and the representation vectors of each character to be processed, Logits2 i ∈R M .

[0057] Taking the probability of the mth character to be processed among the M characters to be processed of the text to be processed as the pointer of the ith predicted character as an example, the ratio of the index value of the similarity between the representation vector required for generating the ith predicted character and the representation vector of the mth character to be processed in the similarity distribution between the representation vector required for generating the ith predicted character and the representation vectors of each character to be processed to the sum of the index values ​​of the similarity between the representation vector required for generating the ith predicted character and the representation vectors of each character to be processed can be used as the pointer probability of the mth character to be processed as the ith predicted character; the probability of the mth character to be processed as the pointer of the ith predicted character can be given by the following formula 2.2:

[0058]

[0059] Among them, Prob2 im Indicates the probability that the mth character to be processed is the pointer of the ith predicted character, Logits2 im In the similarity distribution between the representation vector required to generate the i-th predicted character and the representation vector of each character to be processed, the similarity between the representation vector required to generate the i-th predicted character and the representation vector of the m-th character to be processed is represented, Logits2 iq In the similarity distribution between the representation vector required to generate the ith predicted character and the representation vectors of each character to be processed, it represents the similarity between the representation vector required to generate the ith predicted character and the representation vector of the qth character to be processed.

[0060] S205, according to the generation probability of each reference character as the i-th predicted character and the pointer probability of each to-be-processed character as the i-th predicted character, determine the i-th predicted character from the preset vocabulary, and add the i-th predicted character to the entity relationship sequence corresponding to the to-be-processed text.

[0061] In one embodiment, the relationship extraction device can call the character prediction module in the relationship extraction model, and determine the i-th predicted character from the preset vocabulary according to the generation probability of each reference character as the i-th predicted character and the pointer probability of each to-be-processed character as the i-th predicted character. Specifically, it may include: obtaining the generation probability weight and the pointer probability weight; multiplying the generation probability weight with the generation probability of each reference character as the i-th predicted character to obtain the weighted generation probability corresponding to each reference character, and multiplying the pointer probability weight with the pointer probability of each to-be-processed character as the i-th predicted character to obtain the weighted pointer probability corresponding to each to-be-processed character; adding the weighted generation probability corresponding to each reference character and the weighted pointer probability corresponding to each to-be-processed character to obtain the prediction probability of each reference character as the i-th predicted character, and determining the reference character pointed to by the prediction probability that meets the prediction probability condition in the preset vocabulary as the i-th predicted character; wherein the generation probability weight is determined based on the representation information required to generate the i-th predicted character, and the sum of the generation probability weight and the pointer probability weight is 1.

[0062] Among them, the model parameters used to calculate the generation probability weight in the relationship extraction model can be processed with the representation vector required to generate the i-th predicted character by inner product, so as to obtain the generation probability weight required to generate the i-th predicted character; the generation probability weight can be given by the following formula 3.1:

[0063] λ i =W 2 y i (3.1)

[0064] Among them, λ i represents the generation probability weight required to generate the i-th predicted character, W 2 is the model parameter used to calculate the generation probability weight in the relation extraction model; since the generation probability weight and the pointer probability weight add up to 1, the pointer probability weight required to generate the i-th predicted character is expressed as 1-λ i .

[0065] The prediction probability of each reference character as the i-th predicted character can be given by the following formula 3.2:

[0066] Prob i =λ i Prob1 i +(1-λ i )Prob2 i (3.2)

[0067] Among them, Prob i Represents the prediction probability of each reference character as the i-th predicted character, Prob1i represents the probability of each reference character being the i-th predicted character, λ i Prob1 i Represents the weighted generation probability corresponding to each reference character, Prob2 i represents the probability of each character to be processed as the pointer of the i-th predicted character, (1-λ i )Prob2 i Indicates the weighted pointer probability corresponding to each character to be processed.

[0068] Further, if each reference character in the preset vocabulary is divided into each to-be-processed character in the to-be-processed text and other characters except each to-be-processed character, that is, each reference character in the preset vocabulary includes each to-be-processed character and other characters; the weighted generation probability corresponding to each reference character includes the weighted generation probability corresponding to each to-be-processed character and the weighted generation probability corresponding to other characters; then, the weighted generation probability corresponding to each reference character is added to the weighted pointer probability corresponding to each to-be-processed character to obtain the prediction probability of each reference character as the i-th predicted character, which may include: summing the weighted generation probability corresponding to each to-be-processed character and the weighted pointer probability corresponding to each to-be-processed character to obtain the prediction probability of each to-be-processed character as the i-th predicted character; and determining the weighted generation probability corresponding to other characters as the prediction probability of other characters as the i-th predicted character. For example, if the preset word list includes N reference characters, N is 8, and the first reference character to the eighth reference character are: character 1, character 2, character 3, character 4, character 5, character 6, character 7 and character 8, the weighted generation probability corresponding to each reference character is: λ i Prob1 i (That is, respectively: λ i Prob1 i1 , i Prob1 i2 , i Prob1 i3 , i Prob1 i4 , i Prob1 i5 , i Prob1 i6 , i Prob1 i7 , i Prob1 i8 ); If the text to be processed includes M characters to be processed, M is 5, the first to be processed characters to the fifth to be processed characters are: character 1, character 2, character 5, character 6, character 8, and the weighted pointer probability corresponding to each character to be processed is (1-λi )Prob2 i (That is, respectively: (1-λ i )Prob2 i1 , (1-λ i )Prob2 i2 , (1-λ i )Prob2 i3 , (1-λ i )Prob2 i4 , (1-λ i )Prob2 i5 ); then the prediction probabilities of each reference character as the i-th predicted character are: i Prob1 i1 +(1-λ i )Prob2 i1 , i Prob1 i2 +(1-λ i )Prob2 i2 , i Prob1 i3 , i Prob1 i4 , i Prob1 i5 +(1-λ i )Prob2 i3 , i Prob1 i6 +(1-λ i )Prob2 i4 , i Prob1 i7 , i Prob1 i8 +(1-λ i )Prob2 i5 . Further, if the weighted generation probabilities corresponding to the reference characters are: p1 1 、p1 2 、p1 3 、p1 4 、p1 5 、p1 6 、p1 7 、p1 8 , if the weighted pointer probabilities corresponding to the characters to be processed are: p2 1 、p2 2 、p2 3 、p2 4 、p2 5 , then the prediction probabilities of each reference character as the i-th predicted character are: p1 1 +p21 , p1 2 + p2 2 , p1 3 , p1 4 , p1 5 + p2 3 , p1 6 + p2 4 , p1 7 , p1 8 + p2 5 .

[0069] Further, when the relationship extraction device calls the character prediction module in the relationship extraction model and determines the reference character pointed to by the prediction probability that meets the prediction probability condition in the preset word list as the i-th predicted character, the prediction probability condition may indicate the maximum prediction probability; that is, the reference character pointed to by the maximum prediction probability in the preset word list can be determined as the i-th predicted character. For example, if the preset word list includes the above 8 reference characters, the text to be processed includes the above 5 characters to be processed, and the maximum prediction probability is p1 5 + p2 3 , then the character 5 pointed to by the maximum prediction probability is determined as the i-th predicted character.

[0070] See Figure 3 , which is a schematic diagram of determining the i-th predicted character based on the generation probability and the pointer probability provided by the embodiment of the present application. The relationship extraction model can be a pointer generation network; assume that the reference characters included in the preset word list are: (, ), ,, Du, Zhou, Fu, Pu, Ming, Zi, Hao, Zi, Zai, Mei, Chou, Shao, Duo, Ling, Ling, Ye, Lao; assume that the text to be processed is "Du Fu's courtesy name is Zimei, and his hao is Shaoling Yelao"; assume that the 1st predicted character in the entity relationship sequence corresponding to the text to be processed obtained by prediction is "(, and the 2nd predicted character is "Du". Taking i = 3, that is, predicting the 3rd predicted character as an example, if the representation information of each character to be processed includes the representation vector of each character to be processed, and the representation information required to generate the i-th predicted character includes the representation vector required to generate the i-th predicted character; then the relationship extraction device can call the encoding module in the relationship extraction model to perform relationship feature encoding processing on the text to be processed to obtain the representation vectors of each character to be processed in the text to be processed; call the decoding module in the relationship extraction model to perform decoding processing on the representation vectors of each character to be processed and the 2nd predicted character "Du" to obtain the representation vector required to generate the 3rd predicted character.

[0071] Call the character prediction module, and based on the representation vector required to generate the 3rd predicted character, determine the generation probability of each reference character in the preset word list as the 3rd predicted character, where Figure 3The generation probability of each reference character in the preset vocabulary shown as the third predicted character is only an exemplary description, and is not necessarily the actual generation probability obtained by the relationship extraction model provided by the embodiments of the present application; based on the similarity between the representation vector required to generate the third predicted character and the representation vectors of each character to be processed, determine the pointer probability of each character to be processed as the third predicted character, where Figure 3 The pointer probability of each character to be processed shown as the third predicted character is only an exemplary description, and is not necessarily the actual pointer probability obtained by the relationship extraction model provided by the embodiments of the present application.

[0072] Obtain the generation probability weight and the pointer probability weight; perform a multiplication operation on the generation probability weight and the generation probability of each reference character as the third predicted character respectively to obtain the weighted generation probability corresponding to each reference character, and perform a multiplication operation on the pointer probability weight and the pointer probability of each character to be processed as the third predicted character respectively to obtain the weighted pointer probability corresponding to each character to be processed; add the weighted generation probability corresponding to each reference character and the weighted pointer probability corresponding to each character to be processed correspondingly to obtain the prediction probability of each reference character as the third predicted character. Further, when determining the reference character pointed to by the prediction probability that meets the prediction probability condition in the preset vocabulary as the third predicted character, when the prediction probability condition indicates the maximum prediction probability, determine the reference character "Fu" pointed to by the maximum prediction probability in the preset vocabulary as the third predicted character.

[0073] In one embodiment, the obtained entity relationship sequence corresponding to the text to be processed can be added to the knowledge base to enrich the knowledge in the knowledge base; if the knowledge in the knowledge base is represented by a knowledge graph, then each entity included in the entity relationship sequence corresponding to the text to be processed can be used as a node of the knowledge graph in the knowledge base; use any entity relationship included in the entity relationship sequence corresponding to the text to be processed as an edge between the two nodes indicated by any entity relationship; to enrich the knowledge graph based on the entity relationship sequence corresponding to the text to be processed.

[0074] In an embodiment of the present application, in the process of predicting the entity relationship sequence corresponding to the text to be processed, the representation information of each character to be processed in the text to be processed can be obtained; when predicting the i-th predicted character in the entity relationship sequence corresponding to the text to be processed, the representation information required to generate the i-th predicted character is determined; based on the representation information required to generate the i-th predicted character, the generation probability of each reference character in the preset vocabulary as the i-th predicted character is determined; based on the similarity between the representation information required to generate the i-th predicted character and the representation information of each character to be processed, the probability of each character to be processed as a pointer to the i-th predicted character is determined; based on the generation probability of each reference character as the i-th predicted character and the probability of each character to be processed as a pointer to the i-th predicted character, the i-th predicted character is determined from the preset vocabulary, and the i-th predicted character is added to the entity relationship sequence corresponding to the text to be processed until the entity relationship sequence corresponding to the text to be processed is predicted; wherein, i is a positive integer, and each character to be processed in the text to be processed belongs to the preset vocabulary. When predicting the i-th predicted character in the entity relationship sequence corresponding to the text to be processed, the probability of generating the i-th predicted character by referring to each reference character in the determined preset vocabulary can be combined with the probability of each character to be processed in the text to be processed being a pointer to the i-th predicted character, thereby increasing the probability that the generated i-th predicted character is the character to be processed in the text to be processed. In other words, the probability of entities or relationships not included in the text to be processed appearing in the predicted entity relationship sequence corresponding to the text to be processed can be reduced, thereby improving the accuracy of mining the relationships between entities contained in the text.

[0075] The above-mentioned relationship extraction method can be implemented by calling a relationship extraction model, which may include an encoding module, a decoding module, and a character prediction module; the relationship extraction model may be obtained by training based on training samples, and the training samples may include sample texts and entity relationship label sequences corresponding to the sample texts. The entity relationship label sequences corresponding to the sample texts may be obtained by annotating the sample texts, for example, based on manual annotation. Figure 4 , which is a flow chart of another relationship extraction method provided in an embodiment of the present application. The relationship extraction method includes a training process of a relationship extraction model. The relationship extraction method can be executed by a relationship extraction device. The training process of the relationship extraction model can also be executed by any electronic device that can train the relationship extraction model. The embodiment of the present application will be described later with reference to the relationship extraction device. Figure 4 The relationship extraction method shown may include the following steps:

[0076] S401, calling the encoding module in the relation extraction model to perform relation feature encoding processing on the sample text to obtain the representation vector of each sample character in the sample text.

[0077] Among them, step S401 is similar to the process of calling the encoding module in the relationship extraction model in the above step S201 to perform relationship feature encoding processing on the text to be processed to obtain the representation vector of each character to be processed in the text to be processed, which will not be repeated here.

[0078] S402, calling the decoding module in the relationship extraction model, decoding the representation vector of each sample character and the decoding reference character during training, and obtaining the representation vector required to generate the jth predicted character during training.

[0079] Wherein, j is a positive integer, when j=1, the decoding reference character during training is a special decoding character, and when j>1, the decoding reference character during training is the j-1th predicted character during training. Step S402 is similar to the above step S202, in which the decoding module in the relation extraction model is called to decode the representation vectors of each character to be processed and the decoding reference character to obtain the representation vector required for generating the i-th predicted character, and will not be repeated here.

[0080] S403, performing entity relationship prediction processing based on generating the representation vector required for predicting the j-th character during training, and obtaining an entity relationship prediction sequence corresponding to the sample text.

[0081] In one embodiment, the relationship extraction device can call the character prediction module in the relationship extraction model, perform entity relationship prediction processing based on the representation vector required to generate the j-th training prediction character, and obtain the entity relationship prediction sequence corresponding to the sample text, which can specifically include: determining the training generation probability of each reference character in the preset vocabulary as the j-th training prediction character based on the representation vector required to generate the j-th training prediction character; determining the training pointer probability of each sample character as the j-th training prediction character based on the similarity between the representation vector required to generate the j-th training prediction character and the representation vector of each sample character; determining the j-th training prediction character from the preset vocabulary according to the training generation probability of each reference character as the j-th training prediction character and the training pointer probability of each sample character as the j-th training prediction character, and adding the j-th training prediction character to the entity relationship prediction sequence corresponding to the sample text until the entity relationship prediction sequence corresponding to the sample text is obtained; wherein each sample character in the sample text belongs to the preset vocabulary.

[0082] Among them, based on the representation vector required to generate the j-th predicted character during training, determining the training generation probability of each reference character in the preset vocabulary as the j-th predicted character during training is similar to the process of the above-mentioned step S203. The representation vector required to generate the j-th predicted character during training can be processed by the fully connected layer in the character prediction module to obtain the training feature value distribution pointing to each reference character in the preset vocabulary. Then, the training feature value distribution pointing to each reference character in the preset vocabulary can be normalized by a normalization layer in the character prediction module to obtain the training generation probability of each reference character in the preset vocabulary as the j-th predicted character during training. Based on the similarity between the representation vector required to generate the jth training prediction character and the representation vectors of each sample character, determining the probability of each sample character being a training pointer for the jth training prediction character is similar to the above step S204, and the representation vector of each sample character can be inner-producted with the representation vector required to generate the jth training prediction character to obtain the training similarity distribution between the representation vector required to generate the jth training prediction character and the representation vectors of each sample character, and the training similarity distribution is normalized to obtain the training pointer probability of each sample character in the sample text being the jth training prediction character; wherein, normalizing the training similarity distribution to obtain the training pointer probability of each sample character in the sample text being the jth training prediction character can be performed by a normalization layer in the character prediction module, and the normalization layer can be a softmax activation function layer.

[0083] Further, the relationship extraction device determines the jth training prediction character from the preset vocabulary according to the training generation probability of each reference character as the jth training prediction character and the training pointer probability of each sample character as the jth training prediction character, which is similar to the above process of determining the ith prediction character, and may include: obtaining the training generation probability weight and the training pointer probability weight; multiplying the training generation probability weight with the training generation probability of each reference character as the jth training prediction character to obtain the training weighted generation probability corresponding to each reference character, and multiplying the training pointer probability weight with each sample character to obtain the training weighted generation probability corresponding to each reference character, and multiplying the training pointer probability weight with each sample character to obtain the training weighted generation probability corresponding to each reference character. The training pointer probabilities of the characters as the jth training prediction characters are multiplied to obtain the training weighted pointer probabilities corresponding to each sample character; the training weighted generation probabilities corresponding to each reference character are added to the training weighted pointer probabilities corresponding to each sample character to obtain the training prediction probabilities of each reference character as the jth training prediction character, and the reference characters that meet the training preset conditions in the preset vocabulary are determined as the jth training prediction characters; wherein the training generation probability weight is determined based on the representation vector required to generate the jth training prediction character, and the training generation probability weight and the training pointer probability weight are added to 1. wherein the training preset condition may indicate: the reference character in the preset vocabulary that is the same as the jth entity relationship label character in the entity relationship label sequence corresponding to the sample text is determined as the jth training prediction character.

[0084] S404, adding the entity relationship label sequence and the entity relationship prediction sequence to the training reference information, and training the relationship extraction model based on the training reference information.

[0085] In one embodiment, the relationship extraction device trains the relationship extraction model based on the training reference information, which may include: determining the loss value of the first loss function based on the difference between the entity relationship label sequence and the entity relationship prediction sequence; and adjusting the model parameters in the relationship extraction model in the direction of reducing the loss value of the first loss function.

[0086] In one embodiment, if the entity relationship prediction sequence corresponding to the sample text includes J training prediction characters, the loss value corresponding to each training prediction character can be calculated first, and then the loss value corresponding to each training prediction character can be summed to obtain the loss value of the first loss function; the loss value of the first loss function can be given by the following formula 4.1:

[0087]

[0088] Among them, Loss1 represents the loss value of the first loss function, P1′ jrepresents the training prediction probability of the target reference character as the jth training prediction character, where the target reference character refers to: the reference character in the preset vocabulary that is the same as the i-th entity relationship label character in the entity relationship label sequence corresponding to the sample text, -logP1′ j Indicates the loss value corresponding to the jth predicted character during training. For example, if the preset vocabulary includes N reference characters, N is 8, and the first reference character to the eighth reference character are: character 1, character 2, character 3, character 4, character 5, character 6, character 7 and character 8 respectively; if Prob′ is used j represents the training prediction probability of each reference character as the jth training prediction character. The training prediction probability of each reference character as the jth training prediction character can be expressed as: Prob′ j1 , Prob′ j2 , Prob′ j3 , Prob′ j4 , Prob′ j5 , Prob′ j6 , Prob′ j6 , Prob′ j8 ; If the jth entity relationship label character in the entity relationship label sequence corresponding to the sample text is character 3, then the target reference character is character 3, P1′ j That is Prob′ j3 .

[0089] In a specific implementation, the training prediction probability of each reference character as the jth training prediction character can be given by the following formula 4.2:

[0090] Prob′ j =λ′ j Prob1′ j +(1-λ′ j )Prob2′ j (4.2)

[0091] Among them, λ′ j represents the training generation probability weight required to generate the jth training prediction character, (1-λ′ j ) represents the training pointer probability weight required to generate the jth training prediction character, Prob1′j represents the training generation probability of each reference character as the jth training prediction character, Prob2′ j represents the training time pointer probability of each sample character as the jth training time prediction character.

[0092] Among them, when obtaining the training generation probability of the nth reference character among the reference characters as the jth training prediction character, the ratio between the index value of the training feature value of the nth reference character among the N reference characters pointing to the preset vocabulary in the obtained training feature value distribution pointing to each reference character in the preset vocabulary and the sum of the index values ​​of the training feature values ​​of each reference character among the N reference characters pointing to the preset vocabulary can be used as the training generation probability of the nth reference character as the jth training prediction character; the training generation probability of the nth reference character among the reference characters as the jth training prediction character can be given by the following formula 4.3:

[0093]

[0094] Among them, Prob1′ jn Logits1′ represents the probability of the nth reference character being used as the jth training prediction character during training. jn In the distribution of training feature values ​​pointing to each reference character in the preset vocabulary, the training feature value of the nth reference character among the N reference characters in the preset vocabulary is represented, Logits1′ jp In the obtained training feature value distribution pointing to each reference character in the preset vocabulary, the training feature value of the pth reference character among the N reference characters in the preset vocabulary is represented; the obtained training feature value distribution pointing to each reference character in the preset vocabulary can be expressed as Logits1′ j It means that when the representation vector required to generate the j-th predicted character during training is processed by the fully connected layer in the character prediction module to obtain the distribution of training feature values ​​pointing to each reference character in the preset vocabulary, the model parameters of the fully connected layer in the character prediction module during training can be processed with the representation vector required to generate the j-th predicted character during training to obtain the distribution of training feature values ​​pointing to each reference character in the preset vocabulary; specifically, it is given by the following formula 4.4:

[0095] Logits1′ j =W′ 1 y′ j (4.4)

[0096] Among them, Logits1′ j ∈R N , W′ 1 are the model parameters of the fully connected layer in the character prediction module during training.

[0097] Further, if the sample text includes H sample characters; when obtaining the hth sample character among the sample characters as the training pointer probability of the jth predicted character during training, the ratio of the index value of the training similarity between the representation vector required for generating the jth predicted character during training and the representation vector of each sample character in the training similarity distribution between the representation vector required for generating the jth predicted character during training and the representation vector of each sample character, and the sum of the index values ​​of the training similarity between the representation vector required for generating the jth predicted character during training and the representation vector of each sample character, can be used as the training pointer probability of the hth sample character among the sample characters as the jth predicted character during training; the training pointer probability of the hth sample character among the sample characters as the jth predicted character during training can be given by the following formula 4.5:

[0098]

[0099] Among them, Prob2′ jh Logits2′ represents the probability of the hth sample character being the training pointer of the jth training prediction character. jh In the distribution of similarity between the representation vector required to generate the jth training prediction character and the representation vector of each sample character, the similarity between the representation vector required to generate the jth training prediction character and the representation vector of the hth sample character is generated during training. Logits2′ jt In the training similarity distribution between the representation vector required to generate the jth training prediction character and the representation vector of each sample character, the training similarity between the representation vector required to generate the jth training prediction character and the representation vector of the tth sample character is represented; the training similarity distribution between the representation vector required to generate the jth training prediction character and the representation vector of each sample character can be expressed by Logits2′ j It is expressed as follows; specifically given by the following formula 4.6:

[0100] Logits2′ j =X′y′ j (4.6)

[0101] Among them, X′ represents the matrix composed of the representation vectors of each sample character in the H sample characters of the sample text, X′∈R H*d , where d can be the dimension of the representation vector of any sample character, y′ j Represents the representation vector required to generate the jth character prediction during training, Logits2′ j ∈R H .

[0102] When generating the training-time generation probability weight required for the j-th training-time prediction character, the model parameters used to calculate the training-time generation probability weight when training the relationship extraction model can be inner-producted with the representation vector required to generate the j-th training-time prediction character to obtain the training-time generation probability weight required to generate the j-th training-time prediction character; the training-time generation probability weight required to generate the j-th training-time prediction character can be given by the following formula 4.7:

[0103] λ′ j =W′ 2 y′ j (4.7)

[0104] Among them, W′ 2 When training a relationship extraction model, the model parameters used to calculate the probability weights generated during training.

[0105] S405, calling the encoding module in the relationship extraction model to obtain the representation information of each character to be processed in the text to be processed.

[0106] S406, calling the decoding module in the relationship extraction model, when predicting the i-th predicted character in the entity relationship sequence corresponding to the text to be processed, determining the representation information required to generate the i-th predicted character.

[0107] S407, calling the character prediction module in the relationship extraction model, based on the representation information required to generate the i-th predicted character, determines the generation probability of each reference character in the preset vocabulary as the i-th predicted character.

[0108] S408, calling the character prediction module in the relationship extraction model, and determining the probability of each to-be-processed character being a pointer to the i-th predicted character based on the similarity between the representation information required to generate the i-th predicted character and the representation information of each to-be-processed character.

[0109] S409, calling the character prediction module in the relationship extraction model, determining the i-th predicted character from the preset vocabulary according to the generation probability of each reference character as the i-th predicted character and the pointer probability of each character to be processed as the i-th predicted character, and adding the i-th predicted character to the entity relationship sequence corresponding to the text to be processed.

[0110] Among them, steps S405 to S409 are consistent with the above-mentioned steps S201 to S205, and are not repeated here.

[0111] In an embodiment of the present application, when training the relationship extraction model, the loss value of the first loss function can be determined based on the difference between the entity relationship label sequence corresponding to the sample text and the entity relationship prediction sequence corresponding to the sample text predicted by the relationship extraction model; and the model parameters in the relationship extraction model are adjusted in the direction of reducing the loss value of the first loss function; so that the trained relationship extraction model can process the text to be processed, predict the entity relationship sequence corresponding to the text to be processed, and the trained relationship extraction model has high accuracy in mining the relationship between entities contained in the text.

[0112] Based on the above relationship extraction method, the present application embodiment provides another relationship extraction method. Figure 5 , which is a flow chart of another relationship extraction method provided in an embodiment of the present application. Figure 5 The relationship extraction method shown includes a training process of a relationship extraction model, in which an entity labeling task is added during the training process of the relationship extraction model, so as to improve the relationship extraction model's attention to the entity information in the text to be processed when predicting the entity relationship sequence corresponding to the text to be processed, thereby reducing the probability of erroneous entities not included in the text to be processed appearing in the predicted entity relationship sequence corresponding to the text to be processed; the added entity labeling task is performed by an entity prediction module added to the relationship extraction model; that is, the relationship extraction model includes an encoding module, a decoding module, a character prediction module, and an entity prediction module during the training process. The relationship extraction model can be obtained by training based on training samples, and the training samples can include sample text, an entity relationship label sequence corresponding to the sample text, and an entity label sequence corresponding to the sample text. The entity relationship label sequence corresponding to the sample text can be obtained by labeling the sample text, and the entity label sequence corresponding to the sample text can be obtained by entity labeling the sample text, for example, it can be obtained based on manual labeling. Optionally, the sample text may be labeled with entities using the BIEOS tag commonly used in sequence labeling. In the BIEOS tag, B represents the beginning of an entity, I represents the middle of an entity, E represents the end of an entity, S represents a single-word entity, and O represents others. For example, if the sample text is "Li Bai, also known as Taibai and Qinglian Jushi, a great poet in the Tang Dynasty", the entity label sequence corresponding to the sample text may be "BEOSBEOSBIIEOBEOOOBE". Figure 5 The relationship extraction method shown can be executed by a relationship extraction device. The training process of the relationship extraction model in the relationship extraction method can also be executed by any electronic device that can train the relationship extraction model. The embodiments of the present application will be explained later with reference to the relationship extraction device. Figure 5 The relationship extraction method shown may include the following steps:

[0113] S501, calling the encoding module in the relation extraction model to perform relation feature encoding processing on the sample text to obtain the representation vector of each sample character in the sample text.

[0114] S502, calling the decoding module in the relationship extraction model, decoding the representation vector of each sample character and the decoding reference character during training, and obtaining the representation vector required to generate the jth predicted character during training.

[0115] S503, performing entity relationship prediction processing based on generating the representation vector required for predicting the j-th character during training, and obtaining an entity relationship prediction sequence corresponding to the sample text.

[0116] Among them, step S501 to step S503 are consistent with the above-mentioned step S401 to step S403, and will not be repeated here.

[0117] S504, performing entity prediction processing on the representation vector of each sample character to obtain an entity prediction sequence corresponding to the sample text.

[0118] In one embodiment, the relationship extraction device can call the entity prediction module in the relationship extraction model, perform entity prediction processing on the representation vector of each sample character, and obtain the entity prediction sequence corresponding to the sample text. If the sample text includes H sample characters, the number of reference entity labels included in the entity label sequence corresponding to the sample text is H, and the number of predicted entity labels included in the entity prediction sequence corresponding to the sample text is H. Taking the hth predicted entity label in the entity prediction sequence corresponding to the sample text as an example, the relationship extraction device can call the entity prediction module, perform entity prediction processing on the representation vector of the hth sample character, and obtain the hth predicted entity label in the entity prediction sequence corresponding to the sample text. In a specific implementation, the representation vector of the hth sample character can be processed by the fully connected layer in the entity prediction module to obtain the feature value distribution pointing to each preset entity label; then, the feature value distribution pointing to each preset entity label can be normalized by a normalization layer in the entity prediction module to obtain the entity prediction probability of each preset entity label as the hth predicted entity label; and the preset entity label that meets the preset training conditions among the preset entity labels is determined as the hth predicted entity label. Among them, the normalization layer can be a softmax activation function layer; when the entity label sequence corresponding to the sample text is annotated based on the BIEOS label, the preset entity labels can be: B, I, E, O, S respectively; the preset training condition indicates: among the preset entity labels, the preset entity label that is the same as the hth reference entity label in the entity label sequence corresponding to the sample text is determined as the hth predicted entity label.

[0119] If x′ is usedh Represents the representation vector of the h-th sample character, and the number of preset entity labels is K; when the representation vector of the h-th sample character is processed by the fully connected layer in the entity prediction module to obtain the feature value distribution pointing to each preset entity label, the training model parameters of the fully connected layer in the entity prediction module can be processed with the representation vector of the h-th sample character to obtain the feature value distribution pointing to each preset entity label; the feature value distribution pointing to each preset entity label can be given by the following formula 5.1:

[0120] Logits3′ h =W′ 3 x′ h (5.1)

[0121] Among them, Logits3′ h Represents the distribution of feature values ​​pointing to each preset entity label, Logits3′ h ∈R K , W′ 3 is the training model parameter of the fully connected layer in the entity prediction module, W′ 3 ∈R K*d .

[0122] Taking the entity prediction probability of obtaining the kth preset entity tag among the preset entity tags as the hth predicted entity tag as an example, the ratio between the index value of the feature value pointing to the kth preset entity tag among the preset entity tags based on the feature value distribution pointing to each preset entity tag and the sum of the index values ​​of the feature values ​​pointing to each preset entity tag can be used as the entity prediction probability of the kth preset entity tag as the hth predicted entity tag; the entity prediction probability of the kth preset entity tag as the hth predicted entity tag can be given by the following formula 5.2:

[0123]

[0124] Among them, Prob3′ hk Logits3′ represents the entity prediction probability of the kth preset entity label as the hth predicted entity label. hk Logits3′ represents the feature value of the kth preset entity tag in the feature value distribution pointing to each preset entity tag. hr Indicates the feature value pointing to the rth preset entity tag among the preset entity tags in the feature value distribution pointing to each preset entity tag.

[0125] S505, training the relationship extraction model based on the training reference information, wherein the training reference information includes an entity relationship label sequence corresponding to the sample text, an entity relationship prediction sequence corresponding to the sample text, an entity label sequence corresponding to the sample text, and an entity prediction sequence corresponding to the sample text.

[0126] In a specific implementation, the relationship extraction device trains the relationship extraction model based on the training reference information, which may include: determining the loss value of the first loss function based on the difference between the entity relationship label sequence and the entity relationship prediction sequence; determining the loss value of the second loss function based on the difference between the entity label sequence and the entity prediction sequence; determining the loss value of the target loss function based on the loss value of the first loss function and the loss value of the second loss function; and adjusting the model parameters in the relationship extraction model in the direction of reducing the loss value of the target loss function.

[0127] In one embodiment, based on the difference between the entity relationship label sequence and the entity relationship prediction sequence, the loss value of the first loss function can be determined by the above formula 4.1; based on the difference between the entity label sequence and the entity prediction sequence, when determining the loss value of the second loss function, the loss value corresponding to each predicted entity label can be calculated first, and then the loss value corresponding to each predicted entity label can be summed to obtain the loss value of the second loss function; the loss value of the second loss function can be given by the following formula 6.1:

[0128]

[0129] Among them, Loss2 represents the loss value of the second loss function, P2′ h represents the entity prediction probability of the target preset entity tag as the h-th predicted entity tag, where the target preset entity tag refers to: among the preset entity tags, the preset entity tag that is the same as the h-th reference entity tag in the entity tag sequence corresponding to the sample text; -logP2′ h Represents the loss value corresponding to the h-th predicted entity label. For example, if the preset entity labels are: B, I, E, O, S; if Prob3′ is used hk It represents the entity prediction probability of the kth preset entity label as the hth predicted entity label. The entity prediction probability of each preset entity label as the hth predicted entity label can be expressed as: Prob3′ h1 , Prob3′ h2 , Prob3′ h3 , Prob3′ h4 , Prob3′ h5 ; If the hth reference entity tag in the entity tag sequence corresponding to the sample text is E, then the target preset entity tag is E, P2′ hProb3′ h3 .

[0130] Furthermore, the loss value of the target loss function can be obtained by summing the loss value of the first loss function and the loss value of the second loss function; the loss value of the target loss function can be given by the following formula 6.2:

[0131] Loss = Loss1 + Loss2 (6.2)

[0132] In one embodiment, before the relationship extraction model is trained based on the training sample, the relationship extraction model can be obtained by training based on the pre-training sample; that is, the relationship extraction model can be pre-trained based on the pre-training sample, and then the pre-trained relationship extraction model can be trained and fine-tuned based on the training sample, so as to obtain a relationship extraction model that can predict the entity relationship sequence corresponding to the text to be processed. Among them, the process of pre-training the relationship extraction model based on the pre-training sample is similar to the process of training the relationship extraction model based on the training sample, which will not be repeated here. The pre-training sample includes the pre-training sample text and the entity relationship label sequence corresponding to the pre-training sample text, and the entity relationship label sequence corresponding to the pre-training sample text is the preset entity relationship sequence in the knowledge base. That is to say, when constructing the pre-training sample, the text to be annotated can be obtained, and the preset entity relationship sequence in the knowledge base can be obtained; the preset entity relationship sequence includes the preset entity and the preset entity relationship; if the text to be annotated includes the preset entity and the preset entity relationship, the text to be annotated is used as the pre-training text, and the preset entity relationship sequence is used as the entity relationship label sequence corresponding to the pre-training text to construct the pre-training sample. For example, if the text to be annotated is "Bai Juyi, courtesy name Letian, also known as Xiangshan Jushi, a great realist poet in the Tang Dynasty, with representative works "Song of Everlasting Regret" and "Pipa Song", there is a preset entity relationship sequence "(Bai Juyi, courtesy name, Letian)" in the knowledge base. Since the preset entities included in the preset entity relationship sequence are "Bai Juyi" and "Letian", respectively, and the preset entity relationship included in the preset entity relationship sequence is "character", the text to be annotated includes the preset entity and the preset entity relationship, then the text to be annotated can be used as a pre-training text, and the preset entity relationship sequence can be used as the entity relationship label sequence corresponding to the pre-training text to construct a pre-training sample. The pre-training samples constructed by this method are used to pre-train the relationship extraction model, which can solve the problem of insufficient labeled samples (i.e., training samples).

[0133] S506, calling the encoding module in the relationship extraction model to obtain the representation information of each character to be processed in the text to be processed.

[0134] S507, calling the decoding module in the relationship extraction model, when predicting the i-th predicted character in the entity relationship sequence corresponding to the text to be processed, determining the representation information required to generate the i-th predicted character.

[0135] S508, calling the character prediction module in the relationship extraction model, based on the representation information required to generate the i-th predicted character, determines the generation probability of each reference character in the preset vocabulary as the i-th predicted character.

[0136] S509, calling the character prediction module in the relationship extraction model, and determining the probability of each to-be-processed character being a pointer to the i-th predicted character based on the similarity between the representation information required to generate the i-th predicted character and the representation information of each to-be-processed character.

[0137] S510, calling the character prediction module in the relationship extraction model, determining the i-th predicted character from the preset vocabulary according to the generation probability of each reference character as the i-th predicted character and the pointer probability of each to-be-processed character as the i-th predicted character, and adding the i-th predicted character to the entity relationship sequence corresponding to the to-be-processed text.

[0138] Among them, step S506 to step S510 are consistent with the above-mentioned step S201 to step S205, and will not be repeated here.

[0139] In the embodiment of the present application, an entity labeling task is added in the training process of the relation extraction model. The loss value of the target loss function can be determined based on the difference between the entity relationship label sequence corresponding to the sample text and the entity relationship prediction sequence corresponding to the sample text predicted by the relation extraction model, and the difference between the entity label sequence corresponding to the sample text and the entity prediction sequence corresponding to the sample text predicted by the relation extraction model; and the model parameters in the relation extraction model are adjusted in the direction of reducing the loss value of the target loss function. When the trained relation extraction model predicts the entity relationship sequence corresponding to the to-be-processed text, it can increase the attention paid to the entity information in the to-be-processed text, thereby reducing the probability of erroneous entities not included in the to-be-processed text appearing in the predicted entity relationship sequence corresponding to the to-be-processed text.

[0140] Based on the above-mentioned embodiments related to the relationship extraction method, the present application embodiment provides a relationship extraction device. Figure 6 , is a structural diagram of a relationship extraction device provided in an embodiment of the present application, and the relationship extraction device may include a representation information determination unit 601, a probability determination unit 602, and a character determination unit 603. Figure 6 The relationship extraction device shown can be used to perform the following operations:

[0141] The characterization information determining unit 601 is used to obtain the characterization information of each character to be processed in the text to be processed;

[0142] The representation information determination unit 601 is further used to determine the representation information required to generate the i-th predicted character when predicting the i-th predicted character in the entity relationship sequence corresponding to the to-be-processed text; i is a positive integer;

[0143] A probability determination unit 602 is used to determine the generation probability of each reference character in a preset vocabulary as the i-th predicted character based on the representation information required to generate the i-th predicted character; each character to be processed in the to-be-processed text belongs to the preset vocabulary;

[0144] The probability determination unit 602 is further configured to determine the probability that each to-be-processed character serves as a pointer to the i-th predicted character based on the similarity between the representation information required to generate the i-th predicted character and the representation information of each to-be-processed character;

[0145] The character determination unit 603 is used to determine the i-th predicted character from the preset vocabulary according to the generation probability of each reference character as the i-th predicted character and the probability of each to-be-processed character as a pointer to the i-th predicted character, and add the i-th predicted character to the entity relationship sequence corresponding to the to-be-processed text.

[0146] In one embodiment, when the character determination unit 603 determines the i-th predicted character from the preset word list according to the generation probability of each reference character as the i-th predicted character and the probability of each to-be-processed character as a pointer to the i-th predicted character, the character determination unit 603 specifically performs the following operations:

[0147] Obtaining a generation probability weight and a pointer probability weight; the generation probability weight is determined based on the representation information required to generate the i-th predicted character, and the sum of the generation probability weight and the pointer probability weight is 1;

[0148] The generation probability weight is multiplied by the generation probability of each reference character as the i-th predicted character to obtain the weighted generation probability corresponding to each reference character, and the pointer probability weight is multiplied by the pointer probability of each to-be-processed character as the i-th predicted character to obtain the weighted pointer probability corresponding to each to-be-processed character;

[0149] The weighted generation probabilities corresponding to the reference characters and the weighted pointer probabilities corresponding to the characters to be processed are added together to obtain the prediction probabilities of the reference characters as the i-th predicted characters, and the reference character pointed to by the prediction probabilities that meet the prediction probability conditions in the preset vocabulary is determined as the i-th predicted character.

[0150] In one embodiment, the reference characters include the characters to be processed and other characters; the weighted generation probabilities corresponding to the reference characters include the weighted generation probabilities corresponding to the characters to be processed and the weighted generation probabilities corresponding to the other characters;

[0151] When the character determination unit 603 adds the weighted generation probabilities corresponding to the reference characters and the weighted pointer probabilities corresponding to the characters to be processed to obtain the prediction probabilities of the reference characters as the i-th predicted character, the following operations are specifically performed:

[0152] The weighted generation probability corresponding to each of the characters to be processed and the weighted pointer probability corresponding to each of the characters to be processed are summed up to obtain the predicted probability of each of the characters to be processed being the i-th predicted character;

[0153] The weighted generation probabilities corresponding to the other characters are determined as the predicted probabilities of the other characters being the i-th predicted character.

[0154] In one embodiment, the representation information of each character to be processed includes a representation vector of each character to be processed, and the representation information required to generate the i-th predicted character includes the representation vector required to generate the i-th predicted character; the representation vector of each character to be processed is obtained by calling the encoding module in the relational extraction model to perform relational feature encoding processing on the text to be processed; the relational extraction model also includes a decoding module, and the representation vector required to generate the i-th predicted character is obtained by calling the decoding module in the relational extraction model to perform decoding processing on the representation vectors of each character to be processed and a decoding reference character, wherein, when i=1, the decoding reference character is a special decoding character, and when i>1, the decoding reference character is the i-1th predicted character.

[0155] In one embodiment, the representation information of each to-be-processed character includes a representation vector of each to-be-processed character, and the representation information required to generate the i-th predicted character includes a representation vector required to generate the i-th predicted character;

[0156] When the probability determination unit 602 determines the probability of each to-be-processed character being a pointer to the i-th predicted character based on the similarity between the representation information required to generate the i-th predicted character and the representation information of each to-be-processed character, the following operations are specifically performed:

[0157] Performing inner product processing on the representation vectors of the characters to be processed and the representation vector required to generate the i-th predicted character, to obtain a similarity distribution between the representation vector required to generate the i-th predicted character and the representation vectors of the characters to be processed;

[0158] The similarity distribution is normalized to obtain the probability that each of the characters to be processed serves as a pointer to the i-th predicted character.

[0159] In one embodiment, the relationship extraction device further includes a model training unit 604, wherein the relationship extraction model is obtained based on training samples, wherein the training samples include sample texts and entity relationship label sequences corresponding to the sample texts; when the relationship extraction model is trained based on the training samples:

[0160] The representation information determination unit 601 is further used to call the encoding module in the relationship extraction model to perform relationship feature encoding processing on the sample text to obtain the representation vector of each sample character in the sample text;

[0161] The representation information determination unit 601 is further used to call the decoding module in the relationship extraction model to decode the representation vectors of the sample characters and the reference characters for decoding during training to obtain the representation vectors required for generating the j-th predicted character during training; wherein j is a positive integer, when j=1, the reference character for decoding during training is the special decoding character, and when j>1, the reference character for decoding during training is the j-1-th predicted character during training;

[0162] The representation information determination unit 601, the probability determination unit 602 and the character determination unit 603 are used in cooperation to perform entity relationship prediction processing based on the representation vector required for generating the j-th predicted character during training, so as to obtain an entity relationship prediction sequence corresponding to the sample text;

[0163] The model training unit 604 is used to add the entity relationship label sequence and the entity relationship prediction sequence to the training reference information, and train the relationship extraction model based on the training reference information.

[0164] In one embodiment, the training reference information further includes: an entity label sequence corresponding to the sample text and an entity prediction sequence corresponding to the sample text; the entity prediction sequence is obtained by calling an entity prediction module in the relationship extraction model to perform entity prediction processing on the representation vectors of each sample character;

[0165] When the model training unit 604 trains the relationship extraction model based on the training reference information, the following operations are specifically performed:

[0166] Determining a loss value of a first loss function based on a difference between the entity relationship label sequence and the entity relationship prediction sequence;

[0167] Determining a loss value of a second loss function based on a difference between the entity label sequence and the entity prediction sequence;

[0168] Determine a loss value of a target loss function based on the loss value of the first loss function and the loss value of the second loss function;

[0169] The model parameters in the relationship extraction model are adjusted in a direction of reducing the loss value of the objective loss function.

[0170] In one embodiment, before the relationship extraction model is trained based on the training sample, the relationship extraction model is obtained based on pre-training sample training; the pre-training sample includes pre-training sample text and an entity relationship label sequence corresponding to the pre-training sample text, and the entity relationship label sequence corresponding to the pre-training sample text is a preset entity relationship sequence in the knowledge base.

[0171] In one embodiment, the relationship extraction device further includes a sample acquisition unit 605, and the sample acquisition unit 605 is used to:

[0172] Acquire the text to be annotated, and acquire a preset entity relationship sequence in the knowledge base; the preset entity relationship sequence includes preset entities and preset entity relationships;

[0173] If the text to be annotated includes the preset entity and the preset entity relationship, the text to be annotated is used as the pre-training text, and the preset entity relationship sequence is used as the entity relationship label sequence corresponding to the pre-training text to construct the pre-training sample.

[0174] In one embodiment, the entity relationship sequence corresponding to the text to be processed includes at least two entities and at least one entity relationship;

[0175] The relationship extraction device further includes a knowledge graph construction unit 606, and the knowledge graph construction unit 606 is used to:

[0176] Each entity included in the entity relationship sequence corresponding to the text to be processed is used as a node of the knowledge graph in the knowledge base;

[0177] Any entity relationship included in the entity relationship sequence corresponding to the text to be processed is used as an edge between two nodes indicated by any entity relationship; so as to enrich the knowledge graph based on the entity relationship sequence corresponding to the text to be processed.

[0178] According to one embodiment of the present application, Figure 2 , Figure 4 as well as Figure 5 The various steps involved in the relationship extraction method shown can be Figure 6 The relationship extraction device shown in FIG. 1 is executed by each unit in the relationship extraction device. For example, Figure 2 Steps S201 to S202 shown in FIG. Figure 6 The representation information determination unit 601 in the relationship extraction device shown in the figure is executed, Figure 2 Steps S203 to S204 shown in FIG. Figure 6 The probability determination unit 602 in the relationship extraction device shown in the figure is executed. Figure 2 Step S205 shown can be performed by Figure 6 The character determination unit 603 in the relationship extraction device shown is executed. Figure 4 Steps S401 to S402 shown in FIG. Figure 6 The representation information determination unit 601 in the relationship extraction device shown in the figure is executed, Figure 4 Step S403 shown can be performed by Figure 6 The representation information determination unit 601, the probability determination unit 602 and the character determination unit 603 in the relationship extraction device shown in the figure are executed in cooperation. Figure 4 Step S404 shown can be performed by Figure 6 The model training unit 604 in the relationship extraction device shown is executed; Figure 4 Steps S405 to S406 shown in FIG. Figure 6 The representation information determination unit 601 in the relationship extraction device shown in the figure is executed, Figure 4 Steps S407 to S408 shown in FIG. Figure 6 The probability determination unit 602 in the relationship extraction device shown in the figure is executed. Figure 4 Step S409 shown can be performed by Figure 6 The character determination unit 603 in the relationship extraction device shown is executed. Figure 5 Steps S501 to S502 shown in FIG. Figure 6 The representation information determination unit 601 in the relationship extraction device shown in the figure is executed, Figure 5 Step S503 shown can be performed by Figure 6 The representation information determination unit 601, the probability determination unit 602 and the character determination unit 603 in the relationship extraction device shown in the figure are executed in cooperation. Figure 5 Steps S504 to S505 shown in FIG. Figure 6 The model training unit 604 in the relationship extraction device shown is executed; Figure 5 Steps S506 to S507 shown in FIG. Figure 6 The representation information determination unit 601 in the relationship extraction device shown in the figure is executed, Figure 5 Steps S508 to S509 shown in FIG. Figure 6 The probability determination unit 602 in the relationship extraction device shown in the figure is executed. Figure 5 Step S510 shown can be performed by Figure 6 The character determination unit 603 in the relationship extraction device shown is executed.

[0179] According to another embodiment of the present application, Figure 6 The various units in the relationship extraction device shown can be separately or completely combined into one or several other units to form, or one (some) of the units can be further divided into multiple functionally smaller units to form, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In practical applications, the functions of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. For example, the functions implemented by the above-mentioned units can be implemented by one processing unit. In other embodiments of the present application, the relationship extraction device based on logical function division can also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.

[0180] According to another embodiment of the present application, the program can be executed by running on a general computing device such as a computer including a central processing unit (CPU), a random access memory medium (RAM), a read-only memory medium (ROM), and other processing elements and storage elements. Figure 2 , Figure 4 as well as Figure 5 A computer program (including program code) for each step involved in the corresponding method shown in FIG. Figure 6 The relationship extraction device shown in and the relationship extraction method of the embodiment of the present application are implemented. The computer program can be recorded on, for example, a computer-readable storage medium, and loaded into the above-mentioned computing device through the computer-readable storage medium and run therein.

[0181] In an embodiment of the present application, in the process of predicting the entity relationship sequence corresponding to the text to be processed, the representation information of each character to be processed in the text to be processed can be obtained; when predicting the i-th predicted character in the entity relationship sequence corresponding to the text to be processed, the representation information required to generate the i-th predicted character is determined; based on the representation information required to generate the i-th predicted character, the generation probability of each reference character in the preset vocabulary as the i-th predicted character is determined; based on the similarity between the representation information required to generate the i-th predicted character and the representation information of each character to be processed, the probability of each character to be processed as a pointer to the i-th predicted character is determined; based on the generation probability of each reference character as the i-th predicted character and the probability of each character to be processed as a pointer to the i-th predicted character, the i-th predicted character is determined from the preset vocabulary, and the i-th predicted character is added to the entity relationship sequence corresponding to the text to be processed until the entity relationship sequence corresponding to the text to be processed is predicted; wherein, i is a positive integer, and each character to be processed in the text to be processed belongs to the preset vocabulary. When predicting the i-th predicted character in the entity relationship sequence corresponding to the text to be processed, the probability of generating the i-th predicted character by referring to each reference character in the determined preset vocabulary can be combined with the probability of each character to be processed in the text to be processed being a pointer to the i-th predicted character, thereby increasing the probability that the generated i-th predicted character is the character to be processed in the text to be processed. In other words, the probability of entities or relationships not included in the text to be processed appearing in the predicted entity relationship sequence corresponding to the text to be processed can be reduced, thereby improving the accuracy of mining the relationships between entities contained in the text.

[0182] Based on the above-mentioned related embodiments of the relationship extraction method and the embodiment of the relationship extraction device, the present application also provides a relationship extraction device. Figure 7 , which is a structural diagram of a relationship extraction device provided in an embodiment of the present application. Figure 7 The relationship extraction device shown may include at least a processor 701, an input interface 702, an output interface 703, and a computer storage medium 704. The processor 701, the input interface 702, the output interface 703, and the computer storage medium 704 may be connected via a bus or other means.

[0183] The computer storage medium 704 can be stored in the memory of the relation extraction device. The computer storage medium 704 is used to store a computer program. The computer program includes program instructions. The processor 701 is used to execute the program instructions stored in the computer storage medium 704. The processor 701 (or CPU (Central Processing Unit)) is the computing core and control core of the relation extraction device, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions to implement the above-mentioned relation extraction method process or corresponding functions.

[0184] The embodiment of the present application also provides a computer storage medium (Memory), which is a memory device in a relational extraction device for storing programs and data. It is understandable that the computer storage medium here can include both the built-in storage medium in the terminal and the extended storage medium supported by the terminal. The computer storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor 701 are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer storage medium here can be a high-speed random access memory (RAM) memory, or a non-volatile memory (non-volatile memory), such as at least one disk storage; optionally, it can also be at least one computer storage medium located away from the aforementioned processor.

[0185] In one embodiment, the processor 701 may load and execute one or more instructions stored in a computer storage medium to implement the above-mentioned Figure 2 , Figure 4 as well as Figure 5 In the corresponding steps of the method in the embodiment of the method for extracting the relationship, in a specific implementation, one or more instructions in the computer storage medium are loaded by the processor 701 and the following steps are executed:

[0186] Obtaining representation information of each character to be processed in the text to be processed;

[0187] When predicting the i-th predicted character in the entity relationship sequence corresponding to the to-be-processed text, determining the representation information required to generate the i-th predicted character; i is a positive integer;

[0188] Based on the representation information required to generate the i-th predicted character, determining the generation probability of each reference character in a preset vocabulary as the i-th predicted character; each character to be processed in the to-be-processed text belongs to the preset vocabulary;

[0189] Determining the probability of each of the characters to be processed being a pointer to the i-th predicted character based on the similarity between the representation information required to generate the i-th predicted character and the representation information of each of the characters to be processed;

[0190] According to the generation probability of each reference character as the i-th predicted character and the probability of each to-be-processed character as a pointer to the i-th predicted character, the i-th predicted character is determined from the preset vocabulary, and the i-th predicted character is added to the entity relationship sequence corresponding to the to-be-processed text.

[0191] In one embodiment, the processor 701 performs the following operations when determining the i-th predicted character from the preset word list according to the generation probability of each reference character as the i-th predicted character and the probability of each to-be-processed character as a pointer to the i-th predicted character:

[0192] Obtaining a generation probability weight and a pointer probability weight; the generation probability weight is determined based on the representation information required to generate the i-th predicted character, and the sum of the generation probability weight and the pointer probability weight is 1;

[0193] The generation probability weight is multiplied by the generation probability of each reference character as the i-th predicted character to obtain the weighted generation probability corresponding to each reference character, and the pointer probability weight is multiplied by the pointer probability of each to-be-processed character as the i-th predicted character to obtain the weighted pointer probability corresponding to each to-be-processed character;

[0194] The weighted generation probabilities corresponding to the reference characters and the weighted pointer probabilities corresponding to the characters to be processed are added together to obtain the prediction probabilities of the reference characters as the i-th predicted characters, and the reference character pointed to by the prediction probabilities that meet the prediction probability conditions in the preset vocabulary is determined as the i-th predicted character.

[0195] In one embodiment, the reference characters include the characters to be processed and other characters; the weighted generation probabilities corresponding to the reference characters include the weighted generation probabilities corresponding to the characters to be processed and the weighted generation probabilities corresponding to the other characters;

[0196] When the processor 701 adds the weighted generation probabilities corresponding to the reference characters to the weighted pointer probabilities corresponding to the characters to be processed to obtain the prediction probabilities of the reference characters as the i-th predicted character, the processor 701 specifically performs the following operations:

[0197] The weighted generation probability corresponding to each of the characters to be processed and the weighted pointer probability corresponding to each of the characters to be processed are summed up to obtain the predicted probability of each of the characters to be processed being the i-th predicted character;

[0198] The weighted generation probabilities corresponding to the other characters are determined as the predicted probabilities of the other characters being the i-th predicted character.

[0199] In one embodiment, the representation information of each character to be processed includes a representation vector of each character to be processed, and the representation information required to generate the i-th predicted character includes the representation vector required to generate the i-th predicted character; the representation vector of each character to be processed is obtained by calling the encoding module in the relational extraction model to perform relational feature encoding processing on the text to be processed; the relational extraction model also includes a decoding module, and the representation vector required to generate the i-th predicted character is obtained by calling the decoding module in the relational extraction model to perform decoding processing on the representation vectors of each character to be processed and a decoding reference character, wherein, when i=1, the decoding reference character is a special decoding character, and when i>1, the decoding reference character is the i-1th predicted character.

[0200] In one embodiment, the representation information of each to-be-processed character includes a representation vector of each to-be-processed character, and the representation information required to generate the i-th predicted character includes a representation vector required to generate the i-th predicted character;

[0201] When the processor 701 determines the probability of each to-be-processed character being a pointer of the i-th predicted character based on the similarity between the representation information required to generate the i-th predicted character and the representation information of each to-be-processed character, the processor 701 specifically performs the following operations:

[0202] Performing inner product processing on the representation vectors of the characters to be processed and the representation vector required to generate the i-th predicted character, to obtain a similarity distribution between the representation vector required to generate the i-th predicted character and the representation vectors of the characters to be processed;

[0203] The similarity distribution is normalized to obtain the probability that each of the characters to be processed serves as a pointer to the i-th predicted character.

[0204] In one embodiment, the relationship extraction model is obtained based on training samples, and the training samples include sample texts and entity relationship label sequences corresponding to the sample texts; when the processor 701 trains the relationship extraction model based on the training samples, the processor 701 specifically performs the following operations:

[0205] Calling the encoding module in the relation extraction model to perform relation feature encoding processing on the sample text to obtain a representation vector of each sample character in the sample text;

[0206] Calling the decoding module in the relationship extraction model to decode the representation vectors of the sample characters and the reference characters for decoding during training, and obtaining the representation vector required for generating the j-th predicted character during training; wherein j is a positive integer, when j=1, the reference character for decoding during training is the special decoding character, and when j>1, the reference character for decoding during training is the j-1-th predicted character during training;

[0207] Performing entity relationship prediction processing based on the representation vector required for generating the j-th predicted character during training to obtain an entity relationship prediction sequence corresponding to the sample text;

[0208] The entity relationship label sequence and the entity relationship prediction sequence are added to the training reference information, and the relationship extraction model is trained based on the training reference information.

[0209] In one embodiment, the training reference information further includes: an entity label sequence corresponding to the sample text and an entity prediction sequence corresponding to the sample text; the entity prediction sequence is obtained by calling an entity prediction module in the relationship extraction model to perform entity prediction processing on the representation vectors of each sample character;

[0210] When the processor 701 trains the relationship extraction model based on the training reference information, the processor 701 specifically performs the following operations:

[0211] Determining a loss value of a first loss function based on a difference between the entity relationship label sequence and the entity relationship prediction sequence;

[0212] Determining a loss value of a second loss function based on a difference between the entity label sequence and the entity prediction sequence;

[0213] Determine a loss value of a target loss function based on the loss value of the first loss function and the loss value of the second loss function;

[0214] The model parameters in the relationship extraction model are adjusted in a direction of reducing the loss value of the objective loss function.

[0215] In one embodiment, before the relationship extraction model is trained based on the training sample, the relationship extraction model is obtained based on pre-training sample training; the pre-training sample includes pre-training sample text and an entity relationship label sequence corresponding to the pre-training sample text, and the entity relationship label sequence corresponding to the pre-training sample text is a preset entity relationship sequence in the knowledge base.

[0216] In one embodiment, the processor 701 is further configured to:

[0217] Acquire the text to be annotated, and acquire a preset entity relationship sequence in the knowledge base; the preset entity relationship sequence includes preset entities and preset entity relationships;

[0218] If the text to be annotated includes the preset entity and the preset entity relationship, the text to be annotated is used as the pre-training text, and the preset entity relationship sequence is used as the entity relationship label sequence corresponding to the pre-training text to construct the pre-training sample.

[0219] In one embodiment, the entity relationship sequence corresponding to the text to be processed includes at least two entities and at least one entity relationship;

[0220] The processor 701 is further configured to:

[0221] Each entity included in the entity relationship sequence corresponding to the text to be processed is used as a node of the knowledge graph in the knowledge base;

[0222] Any entity relationship included in the entity relationship sequence corresponding to the text to be processed is used as an edge between two nodes indicated by any entity relationship; so as to enrich the knowledge graph based on the entity relationship sequence corresponding to the text to be processed.

[0223] The embodiment of the present application provides a computer program product, which includes a computer program stored in a computer storage medium; a processor of a relationship extraction device reads the computer program from the computer storage medium, and the processor executes the computer program, so that the relationship extraction device performs the above-mentioned Figure 2 , Figure 4 as well as Figure 5 The method embodiment shown in the figure. The computer-readable storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0224] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A relationship extraction method, characterized in that: include: Acquire representation information of each character to be processed in the text to be processed, wherein the representation information of each character to be processed includes a representation vector of each character to be processed; the representation vector of each character to be processed is obtained by calling an encoding module in a relation extraction model to perform relation feature encoding processing on the text to be processed; When predicting the i-th predicted character in the entity relationship sequence corresponding to the text to be processed, determining the representation information required to generate the i-th predicted character; i is a positive integer; the representation information required to generate the i-th predicted character includes generating a representation vector required for the i-th predicted character; the relationship extraction model also includes a decoding module, and the representation vector required to generate the i-th predicted character is obtained by calling the decoding module in the relationship extraction model to decode the representation vectors of each character to be processed and the decoding reference character; When i=1, the decoding reference character is a special decoding character, and when i>1, the decoding reference character is the i-1th predicted character; Based on the representation information required to generate the i-th predicted character, determining the generation probability of each reference character in a preset vocabulary as the i-th predicted character; Each character to be processed in the text to be processed belongs to the preset vocabulary; Determining the probability of each of the characters to be processed being a pointer to the i-th predicted character based on the similarity between the representation information required to generate the i-th predicted character and the representation information of each of the characters to be processed; According to the generation probability of each reference character as the i-th predicted character and the probability of each to-be-processed character as a pointer to the i-th predicted character, the i-th predicted character is determined from the preset vocabulary, and the i-th predicted character is added to the entity relationship sequence corresponding to the to-be-processed text.

2. The method according to claim 1, characterized in that The step of determining the i-th predicted character from the preset word list according to the generation probability of each reference character as the i-th predicted character and the probability of each to-be-processed character as a pointer to the i-th predicted character comprises: Obtaining a generation probability weight and a pointer probability weight; the generation probability weight is determined based on the representation information required to generate the i-th predicted character, and the sum of the generation probability weight and the pointer probability weight is 1; The generation probability weight is multiplied by the generation probability of each reference character as the i-th predicted character to obtain the weighted generation probability corresponding to each reference character, and the pointer probability weight is multiplied by the pointer probability of each to-be-processed character as the i-th predicted character to obtain the weighted pointer probability corresponding to each to-be-processed character; The weighted generation probabilities corresponding to the reference characters and the weighted pointer probabilities corresponding to the characters to be processed are added together to obtain the prediction probabilities of the reference characters as the i-th predicted characters, and the reference character pointed to by the prediction probabilities that meet the prediction probability conditions in the preset vocabulary is determined as the i-th predicted character.

3. The method according to claim 2, characterized in that The reference characters include the characters to be processed and other characters; the weighted generation probabilities corresponding to the reference characters include the weighted generation probabilities corresponding to the characters to be processed and the weighted generation probabilities corresponding to the other characters; The step of adding the weighted generation probabilities corresponding to the reference characters to the weighted pointer probabilities corresponding to the characters to be processed to obtain the prediction probabilities of the reference characters as the i-th predicted characters includes: The weighted generation probability corresponding to each of the characters to be processed and the weighted pointer probability corresponding to each of the characters to be processed are summed up to obtain the predicted probability of each of the characters to be processed being the i-th predicted character; The weighted generation probabilities corresponding to the other characters are determined as the predicted probabilities of the other characters being the i-th predicted character.

4. The method according to claim 1, characterized in that The determining, based on the similarity between the representation information required to generate the i-th predicted character and the representation information of each to-be-processed character, the probability that each to-be-processed character serves as a pointer to the i-th predicted character comprises: Performing inner product processing on the representation vectors of the characters to be processed and the representation vector required to generate the i-th predicted character, to obtain a similarity distribution between the representation vector required to generate the i-th predicted character and the representation vectors of the characters to be processed; The similarity distribution is normalized to obtain the probability that each of the characters to be processed serves as a pointer to the i-th predicted character.

5. The method according to claim 1, characterized in that The relationship extraction model is obtained based on training samples, wherein the training samples include sample texts and entity relationship label sequences corresponding to the sample texts; Training the relationship extraction model based on the training sample includes: Calling the encoding module in the relation extraction model to perform relation feature encoding processing on the sample text to obtain a representation vector of each sample character in the sample text; Calling the decoding module in the relationship extraction model to decode the representation vectors of the sample characters and the reference characters for decoding during training, to obtain the representation vector required for generating the j-th predicted character during training; wherein j is a positive integer, when j=1, the reference character for decoding during training is the special decoding character, and when j>1, the reference character for decoding during training is the j-1-th predicted character during training; Performing entity relationship prediction processing based on the representation vector required for generating the j-th predicted character during training to obtain an entity relationship prediction sequence corresponding to the sample text; The entity relationship label sequence and the entity relationship prediction sequence are added to the training reference information, and the relationship extraction model is trained based on the training reference information.

6. The method according to claim 5, characterized in that The training reference information also includes: an entity label sequence corresponding to the sample text and an entity prediction sequence corresponding to the sample text; the entity prediction sequence is obtained by calling the entity prediction module in the relationship extraction model to perform entity prediction processing on the representation vectors of each sample character; The training of the relationship extraction model based on the training reference information includes: Determining a loss value of a first loss function based on a difference between the entity relationship label sequence and the entity relationship prediction sequence; Determining a loss value of a second loss function based on a difference between the entity label sequence and the entity prediction sequence; Determine a loss value of a target loss function based on the loss value of the first loss function and the loss value of the second loss function; The model parameters in the relationship extraction model are adjusted in a direction of reducing the loss value of the objective loss function.

7. The method according to claim 5 or 6, characterized in that Before training the relationship extraction model based on the training sample, the relationship extraction model is obtained based on pre-training sample training; the pre-training sample includes pre-training sample text and an entity relationship label sequence corresponding to the pre-training sample text, and the entity relationship label sequence corresponding to the pre-training sample text is a preset entity relationship sequence in the knowledge base.

8. The method according to claim 7, characterized in that The method further comprises: Acquire the text to be annotated, and acquire a preset entity relationship sequence in the knowledge base; the preset entity relationship sequence includes preset entities and preset entity relationships; If the text to be annotated includes the preset entity and the preset entity relationship, the text to be annotated is used as the pre-training text, and the preset entity relationship sequence is used as the entity relationship label sequence corresponding to the pre-training text to construct the pre-training sample.

9. The method according to claim 1, characterized in that The entity relationship sequence corresponding to the text to be processed includes at least two entities and at least one entity relationship; The method further comprises: Each entity included in the entity relationship sequence corresponding to the text to be processed is used as a node of the knowledge graph in the knowledge base; Any entity relationship included in the entity relationship sequence corresponding to the text to be processed is used as an edge between two nodes indicated by any entity relationship; so as to enrich the knowledge graph based on the entity relationship sequence corresponding to the text to be processed.

10. A relationship extraction device, characterized in that: include: A representation information determination unit is used to obtain the representation information of each character to be processed in the text to be processed, wherein the representation information of each character to be processed includes a representation vector of each character to be processed; the representation vector of each character to be processed is obtained by calling the encoding module in the relation extraction model to perform relation feature encoding processing on the text to be processed; The representation information determination unit is further used to determine the representation information required to generate the i-th predicted character when predicting the i-th predicted character in the entity relationship sequence corresponding to the to-be-processed text; i is a positive integer; the representation information required to generate the i-th predicted character includes generating the representation vector required for the i-th predicted character; the relation extraction model also includes a decoding module, and the representation vector required to generate the i-th predicted character is obtained by calling the decoding module in the relation extraction model to decode the representation vectors of each to-be-processed character and the decoding reference character; When i=1, the decoding reference character is a special decoding character, and when i>1, the decoding reference character is the i-1th predicted character; A probability determination unit, configured to determine the generation probability of each reference character in a preset vocabulary as the i-th predicted character based on the representation information required to generate the i-th predicted character; each character to be processed in the to-be-processed text belongs to the preset vocabulary; The probability determination unit is further used to determine the probability that each to-be-processed character serves as a pointer to the i-th predicted character based on the similarity between the representation information required to generate the i-th predicted character and the representation information of each to-be-processed character; A character determination unit is used to determine the i-th predicted character from the preset vocabulary according to the generation probability of each reference character as the i-th predicted character and the probability of each to-be-processed character as a pointer to the i-th predicted character, and add the i-th predicted character to the entity relationship sequence corresponding to the to-be-processed text.

11. A relationship extraction device, characterized in that: The relationship extraction device includes an input interface and an output interface, and also includes: a processor adapted to implement one or more instructions; and, A computer storage medium storing one or more instructions, wherein the one or more instructions are suitable for being loaded by the processor and executing the relationship extraction method as described in any one of claims 1 to 9.

12. A computer storage medium, characterized in that: The computer storage medium stores computer program instructions, and when the computer program instructions are executed by the processor, they are used to execute the relationship extraction method according to any one of claims 1 to 9.

13. A computer program product, characterized in that The computer program product includes a computer program, which, when executed by a processor, is used to load and execute the relationship extraction method according to any one of claims 1 to 9.

Images