An information extraction method and device

Abstract

Embodiments of the present application provide an information extraction method and device, applied in the field of artificial intelligence technology, comprising: obtaining a to-be-recognized text, and constructing a word vector sequence of the to-be-recognized text; inputting the word vector sequence into a relation classification layer to obtain a target relation type contained in the to-be-recognized text; for each target relation type, determining at least one entity contained in the to-be-recognized text under the target relation type based on a type feature vector corresponding to the target relation type and the word vector sequence; and constructing a triple information corresponding to the target relation type based on the target relation type and the at least one entity. Through the relation classification layer, at least one target relation type contained in the to-be-recognized text is extracted, and the entity contained in the to-be-recognized text under the target relation type is determined, thereby solving the problem that a relation model cannot be constructed for an incomplete sentence structure, and avoiding errors generated by an entity extraction model from being passed to a relation extraction model, so as to improve information extraction accuracy.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an information extraction method and apparatus. Background Technology

[0002] In recent years, the continuous development of natural language processing technology has promoted the application and popularization of machine understanding in various practical scenarios, such as search engines, automatic question answering, and intelligent assistants. Short texts are widely present in these scenarios. Users interact with the system through natural language, and the queries they submit contain information about the query subject and other related information. Effectively extracting this information is the foundation for achieving machine understanding. Therefore, short text understanding is a crucial and challenging task in realizing machine-user interaction.

[0003] Under relevant technologies, text fragments are first extracted directly from the text as entities. These extracted entities are then paired to form entity pairs, and finally, the relationship between the two entities is determined, forming subject-object-relation triples. However, due to the limited word count, simple sentence structure, and colloquial nature of short texts, relation triples may lack a subject or object, making it impossible to determine the relation type through enumerating entity pairs and performing multi-classification. Furthermore, errors generated by the entity extraction model are passed to the relation extraction model, leading to error accumulation and further affecting the accuracy of information extraction. Summary of the Invention

[0004] This application provides an information extraction method and apparatus for extracting entity information and relational information from text fragments.

[0005] In a first aspect, embodiments of this application provide an information extraction method, including:

[0006] Obtain the text to be recognized and construct a sequence of word vectors for the text to be recognized;

[0007] The word vector sequence is input into the relation classification layer to obtain at least one target relation type contained in the text to be identified;

[0008] For each target relation type, based on the type feature vector corresponding to the target relation type and the word vector sequence, determine at least one entity contained in the text to be identified under the target relation type;

[0009] Based on the target relation type and the at least one entity, construct the triplet information corresponding to the target relation type.

[0010] The system extracts at least one target relation type from the text to be identified through a relation classification layer. Then, for each target relation type, it determines at least one entity contained in the text under the target relation type based on the type feature vector and word vector sequence corresponding to the target relation type. This avoids constructing entity pairs before relation extraction, solves the problem of not being able to construct relation model input when the sentence structure is incomplete, and also avoids the error generated by the entity extraction model from being passed to the relation extraction model, thereby improving the accuracy of information extraction.

[0011] Optionally, constructing the word vector sequence of the text to be identified includes:

[0012] The text to be identified is divided into segments at the character level to obtain the target text sequence;

[0013] The target text sequence is encoded using a word vector encoding layer to obtain the word vector sequence.

[0014] Optionally, inputting the word vector sequence into the relation classification layer to obtain at least one target relation type contained in the text to be identified includes:

[0015] The word vector sequence is input into the relation classification layer to obtain the probability that the text to be identified contains each type of reference relation.

[0016] Reference relationship types with a probability greater than a preset threshold are taken as the target relationship types contained in the text to be identified.

[0017] Optionally, before determining at least one entity contained in the text to be identified under the target relation type based on the type feature vector corresponding to the target relation type and the word vector sequence, the method further includes:

[0018] Obtain the type feature vector corresponding to the target relationship type from the preset relationship embedding matrix.

[0019] Optionally, determining at least one entity contained in the text to be identified under the target relation type based on the type feature vector corresponding to the target relation type and the word vector sequence includes:

[0020] The type feature vector and the word vector sequence are fused to obtain a fused feature vector sequence;

[0021] The fused feature vector sequence is input into the entity information extraction layer to obtain the entity type label for each character in the text to be identified;

[0022] Based on the entity type label of each obtained character, determine at least one entity contained in the text to be identified under the target relation type.

[0023] Optionally, the entity information extraction layer includes a first entity extraction module and a second entity extraction module;

[0024] The step of inputting the fused feature vector sequence into the entity information extraction layer to obtain the entity type label for each character in the text to be identified includes:

[0025] The fused feature vector sequence is input into the first entity extraction module to obtain the entity type label when each character in the text to be identified corresponds to the subject entity;

[0026] The fused feature vector sequence is input into the second entity extraction module to obtain the entity type label when each character in the text to be identified corresponds to an object entity.

[0027] Optionally, determining at least one entity contained in the text to be identified under the target relation type based on the obtained entity type label of each character includes:

[0028] Based on the entity type label obtained for each character, determine at least one entity start position and corresponding entity end position in the text to be identified;

[0029] Based on the starting position of the at least one entity and the corresponding ending position of the entity, at least one entity contained in the text to be identified is determined.

[0030] Optionally, determining the at least one entity contained in the text to be identified based on the at least one entity start position and the corresponding entity end position includes:

[0031] The text segment between the start position and the corresponding end position of each entity in the text to be identified is taken as an entity in the text to be identified.

[0032] Optionally, the at least one entity is a subject entity or an object entity;

[0033] The step of constructing triplet information corresponding to the target relation type based on the target relation type and the at least one entity includes:

[0034] Based on the target relation type, the preset empty character, and the subject entity, construct the triplet information corresponding to the target relation type; or...

[0035] Based on the target relation type, the preset empty character, and the object entity, construct the triplet information corresponding to the target relation type.

[0036] Optionally, the at least one entity includes a subject entity and an object entity;

[0037] The step of constructing triplet information corresponding to the target relation type based on the target relation type and the at least one entity includes:

[0038] Based on the target relation type, the subject entity, and the object entity, construct the triplet information corresponding to the target relation type.

[0039] Secondly, embodiments of this application provide an information extraction device, comprising:

[0040] The acquisition module is used to acquire the text to be recognized and construct a word vector sequence of the text to be recognized;

[0041] The processing module is used to input the word vector sequence into the relation classification layer to obtain at least one target relation type contained in the text to be identified;

[0042] The processing module is further configured to, for each target relation type, determine at least one entity contained in the text to be identified under the target relation type based on the type feature vector corresponding to the target relation type and the word vector sequence;

[0043] A construction module is used to construct triplet information corresponding to the target relationship type based on the target relationship type and the at least one entity.

[0044] Optionally, the acquisition module is specifically used for:

[0045] The text to be identified is divided into segments at the character level to obtain the target text sequence;

[0046] The target text sequence is encoded using a word vector encoding layer to obtain the word vector sequence.

[0047] Optionally, the processing module is specifically used for:

[0048] The word vector sequence is input into the relation classification layer to obtain the probability that the text to be identified contains each type of reference relation.

[0049] Reference relationship types with a probability greater than a preset threshold are taken as the target relationship types contained in the text to be identified.

[0050] Optionally, the acquisition module is specifically used for:

[0051] Obtain the type feature vector corresponding to the target relationship type from the preset relationship embedding matrix.

[0052] Optionally, the processing module is specifically used for:

[0053] The type feature vector and the word vector sequence are fused to obtain a fused feature vector sequence;

[0054] The fused feature vector sequence is input into the entity information extraction layer to obtain the entity type label for each character in the text to be identified;

[0055] Based on the entity type label of each obtained character, determine at least one entity contained in the text to be identified under the target relation type.

[0056] Optionally, the processing module is specifically used for:

[0057] The entity information extraction layer includes a first entity extraction module and a second entity extraction module;

[0058] The step of inputting the fused feature vector sequence into the entity information extraction layer to obtain the entity type label for each character in the text to be identified includes:

[0059] The fused feature vector sequence is input into the first entity extraction module to obtain the entity type label when each character in the text to be identified corresponds to the subject entity;

[0060] The fused feature vector sequence is input into the second entity extraction module to obtain the entity type label when each character in the text to be identified corresponds to an object entity.

[0061] Optionally, the processing module is specifically used for:

[0062] Based on the entity type label obtained for each character, determine at least one entity start position and corresponding entity end position in the text to be identified;

[0063] Based on the starting position of the at least one entity and the corresponding ending position of the entity, at least one entity contained in the text to be identified is determined.

[0064] Optionally, the processing module is specifically used for:

[0065] The text segment between the start position and the corresponding end position of each entity in the text to be identified is taken as an entity in the text to be identified.

[0066] Optionally, the building module is specifically used for:

[0067] The step of constructing triplet information corresponding to the target relation type based on the target relation type and the at least one entity includes:

[0068] Based on the target relation type, the preset empty character, and the subject entity, construct the triplet information corresponding to the target relation type; or...

[0069] Based on the target relation type, the preset empty character, and the object entity, construct the triplet information corresponding to the target relation type.

[0070] Optionally, the building module is specifically used for:

[0071] The at least one entity includes a subject entity and an object entity;

[0072] The step of constructing triplet information corresponding to the target relation type based on the target relation type and the at least one entity includes:

[0073] Based on the target relation type, the subject entity, and the object entity, construct the triplet information corresponding to the target relation type.

[0074] Thirdly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor performs any of the information extraction methods described in the first aspect above.

[0075] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform any of the information extraction methods described in the first aspect.

[0076] This application extracts at least one target relation type from the text to be identified through a relation classification layer. Then, for each target relation type, based on the type feature vector and word vector sequence corresponding to the target relation type, it determines at least one entity contained in the text to be identified under the target relation type. This avoids constructing entity pairs before relation extraction, solves the problem of not being able to construct relation model input when the sentence structure is incomplete, and also avoids the error generated by the entity extraction model from being passed to the relation extraction model, thereby improving the accuracy of information extraction. Attached Figure Description

[0077] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0078] Figure 1 A schematic diagram of an information extraction system architecture provided in an embodiment of the present invention;

[0079] Figure 2 A flowchart illustrating an information extraction method provided in an embodiment of the present invention;

[0080] Figure 3 This is a schematic diagram of a process for information extraction using an entity and relation extraction model, provided as an embodiment of the present invention.

[0081] Figure 4 This is a schematic diagram of the logic of an information extraction method provided in an embodiment of the present invention;

[0082] Figure 5 This is a schematic diagram of the structure of an information extraction device provided in an embodiment of the present invention;

[0083] Figure 6 This is a schematic diagram of the structure of a computing device provided in an embodiment of the present invention. Detailed Implementation

[0084] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0085] See Figure 1 This is a system architecture diagram used in an embodiment of this application. The system architecture includes a terminal device 101 and a server 102. The terminal device 101 is used to collect short text input by the user. The terminal device 101 can be a smartphone, tablet computer, laptop computer, desktop computer, smart home appliance, etc., but is not limited to these.

[0086] Server 102 receives short text sent by terminal device 101 and, based on an entity and relation extraction model, identifies and extracts entity and relation information from the received short text. The entity and relation extraction model consists of four parts: a word vector encoding layer, a relation classification layer, a fusion layer, and an entity information extraction layer. Server 102 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0087] Terminal device 101 and server 102 can be connected directly or indirectly via wired or wireless communication, and this application does not impose any restrictions. This system architecture can be applied to various machine understanding scenarios such as search engines, automatic question answering, and intelligent assistants on various platforms to realize interaction between machines and users.

[0088] Based on the system architecture shown in Figure 1 Figure 2 An exemplary flow of an information extraction method is shown, which is executed by the server and includes the following steps:

[0089] Step S201: Obtain the text to be recognized and construct the word vector sequence of the text to be recognized.

[0090] Specifically, after the server obtains the text to be recognized, it transforms the text to construct the corresponding word vector sequence.

[0091] In some embodiments, the text to be recognized is divided into segments at the character level to obtain a target text sequence. Then, a word vector encoding layer is used to encode the target text sequence to obtain a word vector sequence.

[0092] Specifically, the word vector encoding layer can be a BERT model, ALBERT, RoBERTa, ELECTRA3, or SpanBERT, etc. The acquired text to be recognized is divided into characters, and special symbols "[CLS]" and "[SEP]" are added to the beginning and end of each character, respectively. Assuming the length of the text to be recognized is n, the target text sequence W = w is obtained. cls ,w1,…,w n ,w sep The target text sequence is input into the BERT model, and after encoding by the BERT model, the word vector sequence H = h is obtained. cls ,h1,h2,…,h n ,h sep The BERT model is a language representation model that consists of two steps: pre-training and fine-tuning. In pre-training, the model is trained on different pre-training tasks based on unlabeled data, thus incorporating a large amount of common-sense knowledge. In fine-tuning, the model is first initialized based on the parameters obtained from pre-training, and then all parameters are fine-tuned using labeled data from downstream specific tasks.

[0093] For example, if the user inputs the text to be recognized as "Zhang San's friend is?", it is divided into characters to obtain "Zhang, San, of, friend, friend, is,?". Adding special markers to the beginning and end of the text results in the target text sequence "[CLS], Zhang, San, of, friend, friend, is,?, [SEP]". The target text sequence is then input into the BERT model to calculate the word vector (with dimension h) corresponding to each character, thereby constructing the word vector sequence.

[0094] In this embodiment, a word vector encoder is used to train two tasks together, which can capture the dependencies between tasks and improve the model's semantic understanding ability. This joint approach also avoids error propagation, prevents erroneous results from the previous step from affecting subsequent steps, and improves the overall task accuracy.

[0095] Step S202: Input the word vector sequence into the relation classification layer to obtain at least one target relation type contained in the text to be identified.

[0096] Specifically, word vectors are extracted sequentially starting from the first position in the word vector sequence. These extracted word vectors are then input into a relation classification layer. The relation classification layer processes each word vector in the word vector sequence to identify at least one target relation type in the text to be identified. This at least one target relation type constitutes the target relation type set R. x The set of all relation types is represented as R = {r}. i For example, the relationship types can include father, mother, colleague, friend, son, daughter, etc., resulting in the target set of relationship types.

[0097] The relation classification layer is constructed from a fully connected feedforward neural network, which contains two fully connected layers (FC) and an activation function (σ).

[0098] In some embodiments, a word vector sequence is input into a relation classification layer to obtain the probability that the text to be identified contains each reference relation type; reference relation types with a probability greater than a preset threshold are taken as the target relation types contained in the text to be identified.

[0099] Specifically, the word vector sequence is input sequentially into the relation classification layer to calculate the probability of each relation type r in the text to be identified. The probability of each relation type is represented as p. r (r|x). If the calculated probability of the relation type is greater than the preset threshold, then the relation type is taken as the target relation type. The specific calculation process of the probability of each relation is shown in formula (1):

[0100] p r (r|x)=Sigmoid(FC(h cls ))…………(1)

[0101] For example, with a preset threshold of 0.5, the word vector sequence of the text to be identified 1 is input into the relation classification layer. The probability of the "father" relation is calculated to be 0.3, the probability of the "friend" relation is 0.7, the probability of the "colleague" relation is 0.6, and the probability of the "son" relation is 0.4. The relation types with a probability greater than 0.5 are taken as the target relation types, and the final target relation type set of the text to be identified 1 is {friend, colleague}.

[0102] Step S203: For each target relation type, based on the type feature vector and word vector sequence corresponding to the target relation type, determine at least one entity contained in the text to be identified under the target relation type.

[0103] Specifically, each target relation type is represented by a corresponding type feature vector. Based on the type feature vector and word vector sequence corresponding to each target relation type, the entities in the text to be identified under that target relation are determined. The entities can be names of people, organizations, addresses, times, etc.

[0104] In some embodiments, before determining at least one entity contained in the text to be identified under the target relation type based on the type feature vector and word vector sequence corresponding to the target relation type, the method further includes: obtaining the type feature vector corresponding to the target relation type from a preset relation embedding matrix.

[0105] Specifically, from the pre-defined relation embedding matrix H R In step S202, obtain the embedding representation corresponding to each target relation type. Embedded representation This is a set of type feature vectors representing the relation type. The relation embedding matrix H R This is obtained from the initial settings. For example, setting a 3x1 relational embedding matrix H. R This includes three types of relational embedding representations, each of which is a one-dimensional vector.

[0106] Step S204: Based on the target relation type and at least one entity, construct the triplet information corresponding to the target relation type.

[0107] Specifically, the triple information is composed of the target relation type and the extracted entities. The triple information includes a subject, a predicate, and an object, with the specific structure of (subject, relation, object), thus connecting entities under the same semantic relation.

[0108] For example, if the target relation type is {friend, colleague}, and the two entities extracted are {Zhang San} and {Li Si}, then the final triples obtained are (Zhang San, friend, Li Si) and (Zhang San, colleague, Li Si).

[0109] In this embodiment, at least one target relation type is first extracted from the text to be identified through a relation classification layer. Then, for each target relation type, based on the type feature vector and word vector sequence corresponding to the target relation type, at least one entity contained in the text to be identified under the target relation type is determined. This avoids constructing entity pairs before relation extraction, solves the problem of not being able to construct relation model input when the sentence structure is incomplete, and also avoids the error generated by the entity extraction model being passed to the relation extraction model, thereby improving the accuracy of information extraction.

[0110] In step S203 above, based on the type feature vector and word vector sequence corresponding to the target relation type, at least one entity contained in the text to be identified under the target relation type is determined, including:

[0111] The type feature vector and word vector sequence are fused to obtain a fused feature vector sequence; the fused feature vector sequence is input into the entity information extraction layer to obtain the entity type label of each character in the text to be identified; based on the obtained entity type label of each character, at least one entity contained in the text to be identified under the target relation type is determined.

[0112] Specifically, the embedding representation of each target relation type The input is normalized in the conditional layer, which is the embedding representation of each target relation type. and word vector sequence H = h cls h1, h2, ..., h n h sep The input is fused into a relation type label mixing layer to obtain a fused feature vector sequence. Conditional Layer Normalization (CLN) is an improvement on Layer Normalization (LN) and is the most commonly used normalization method in neural networks. Under condition c, for word vector h... i The normalization calculation method is shown in the following formulas (2) to (4):

[0113]

[0114] γ c =W γ c+b γ …………(3)

[0115] β c=W β c+b β …………(4)

[0116] in, and Word vectors The mean and variance of the elements, and These are the gain and bias under condition c, used to scale and shift the standardized word vectors to incorporate conditional information, thereby controlling the model calculation results to a certain extent and achieving different normalization for different data.

[0117] A multi-label binary classification model is used to determine the relation types in the input text and output a set of relation types, supporting the extraction of multiple relation types. A CLN (Content-Based Network) is employed to fuse relation types and word vector representations, performing conditional probability modeling for entity extraction under specified relation types. This allows the introduction of prior information about relation types, improving the model's entity extraction capability for specific types.

[0118] Then fuse the feature vector sequence The input is fed into the entity information extraction layer, which calculates the probability p of the entity type for each character, thus obtaining the entity type label for each character. Let t represent the entity type label, t = {t...} i |i = 1, ..., N t Based on the entity type label of each character, at least one entity under the target relation type can be identified. This entity information extraction layer is constructed from two sets of entity extraction modules of fully connected feedforward neural networks, where t∈T, and T is the set of all entity type labels, such as {non-entity, address, name, time, organization, etc.}.

[0119] In some embodiments, the entity information extraction layer includes a first entity extraction module and a second entity extraction module; inputting the fused feature vector sequence into the entity information extraction layer to obtain the entity type label of each character in the text to be identified includes: inputting the fused feature vector sequence into the first entity extraction module to obtain the entity type label of each character in the text to be identified when it corresponds to a subject entity; and inputting the fused feature vector sequence into the second entity extraction module to obtain the entity type label of each character in the text to be identified when it corresponds to an object entity.

[0120] Specifically, the entity information extraction layer consists of two entity modules, enabling separate extraction of the subject and object. The first entity extraction module is for subject entities, and the second entity extraction module is for object entities. When the fused feature vector sequence is input into the first entity extraction module, it identifies the subject entities in the fused feature vector sequence and determines the entity type of each character in the subject entities. Similarly, when the fused feature vector sequence is input into the second entity extraction module, it identifies the object entities in the fused feature vector sequence and determines the entity type of each character in the object entities.

[0121] By using two entity extraction modules to extract the subject and object from the text to be identified, the limitation of needing to construct entity pairs to complete relation extraction is broken, and the problem of being unable to extract relation types when the sentence structure is incomplete is solved.

[0122] In some embodiments, based on the entity type label of each obtained character, at least one entity start position and corresponding entity end position in the text to be identified are determined; based on at least one entity start position and corresponding entity end position, at least one entity contained in the text to be identified is determined.

[0123] Specifically, the fused feature vector sequence After inputting into the entity information extraction layer, the probability of the entity type of each character is calculated. If the probability of the entity type of the character is greater than the preset threshold, the entity type is used as the entity type label of the character. The specific calculation process is shown in formula (1).

[0124] Under the same entity type, calculate the probability of each character being the starting position and the probability of it being the ending position of an entity. The specific calculation process is shown in formula (1), where the starting position is represented by s and the ending position by e. If the probability of a character being the starting position of an entity is greater than a preset threshold, then the character is used as the starting position of the entity. The process for determining the ending position is the same as above. The feature vector sequence will then be fused. After the input is fed into the entity information extraction layer, the entity location and type are output simultaneously. Entities are divided into subject entities (object, abbreviated as o) and object entities (subject, abbreviated as s). The input text x is calculated in the target relation type r. x The following are the four sets of conditional probabilities:

[0125] (1) Calculate the starting position and entity type probability p of a certain character as the subject entity. o (t s |x,r x This involves determining whether a certain character is the beginning of a subject entity and identifying the entity type.

[0126] (2) Calculate the termination position and entity type probability p of a certain character as the subject entity. o (t e |x,r x This involves determining whether a certain character marks the end of a subject entity and identifying the entity type.

[0127] (3) Calculate the starting position and entity type probability p of a certain character as an object entity. s (t s |x,r x This involves determining whether a certain character is the beginning of an object entity and identifying the entity type.

[0128] (4) Calculate the termination position and entity type probability p of a certain character as an object entity. s (t e |x,r x This involves determining whether a certain character marks the end of an object entity and identifying the entity type.

[0129] Based on the start and end positions of the characters and the entity type, the start and end positions of at least one entity in the text to be identified are determined. Then, based on the entity type label of each character within the entity, the category of that entity is determined, thus completing the entity extraction. The text to be identified may contain only one entity or multiple entities.

[0130] The entity extraction module consists of a multi-classifier that determines the location and type of an entity fragment in the text by inferring the start and end positions and types of the entity. Compared to the commonly used sequence labeling model CRF, it does not require sentence-level annotation decoding (which is usually solved by beam search), thus speeding up the decoding process. It is computationally efficient and fast, and can also solve the problem of overlapping entities.

[0131] In some embodiments, the text segment between the start position and the corresponding end position of each entity in the text to be identified is regarded as an entity in the text to be identified.

[0132] Specifically, if the text to be identified contains multiple entities, then multiple words will be judged as the starting position of the entity and multiple words will be judged as the ending position. Then, the segment between the ending position word that is closest to the starting position word will be regarded as an entity in the text to be identified.

[0133] Specifically, when the text to be recognized is input into the first entity extraction module, the extracted entity will be used as a subject entity in the text to be recognized. When the text to be recognized is input into the second entity extraction module, the extracted entity will be used as an object entity in the text to be recognized.

[0134] Taking a 7-character text to be recognized as an example, the set of all entity type labels is {non-entity, address, name, time, organization}, with corresponding numbers {0, 1, 2, 3, 4}. After each character of the text is processed by the entity information extraction layer, the subject entity starts at the first character, with the entity type label being "name," resulting in the entity type label sequence 0, 2, 0, 0, 0, 0, 0, 0. The subject entity ends at the second character, with the entity type label also being "name," resulting in the entity type label sequence 0, 0, 2, 0, 0, 0, 0, 0, 0. Therefore, the final subject entity is composed of the first and second characters of the text to be recognized.

[0135] In some embodiments, triplet information corresponding to the target relation type is constructed based on the target relation type, a preset null character, and a subject entity; or, triplet information corresponding to the target relation type is constructed based on the target relation type, a preset null character, and an object entity.

[0136] Specifically, when the extracted entity is one or more subject entities, the relation types in the target relation type are traversed in turn. That is, the corresponding subject entity is extracted under each target relation type, and the missing object entity is replaced with a preset empty character to complete the ternary extraction of all relations. The resulting triple is (subject, relation, ?).

[0137] When the extracted entity is one or more object entities, the relation types in the target relation type are traversed sequentially. That is, the corresponding object entity is extracted under each target relation type, and the missing subject entity is replaced with a preset empty character to complete the ternary extraction of all relations. The resulting triplet is (?, relation, object).

[0138] In some embodiments, at least one entity includes a subject entity and an object entity; based on the target relation type, the subject entity, and the object entity, triple information corresponding to the target relation type is constructed.

[0139] Specifically, when the extracted entities are one or more subject and object entities, the relation types in the target relation type are traversed sequentially. That is, the corresponding subject and object entities are extracted under each target relation type to complete the triple extraction of all relations. The resulting triples can be (subject1, relation1, object1), (subject1, relation2, object1), (subject3, relation2, object3), (subject4, relation4, object4), etc.

[0140] For example, such as Figure 3 As shown, the entity and relation extraction model includes a BERT word vector encoding layer, a multi-label relation classification layer, a relation type label hybrid layer, and an entity location / type extraction layer.

[0141] In the training stage of the entity and relation extraction model, the binary cross entropy (BCE) is used to calculate the relation classification loss L R and the multi-class cross entropy (CE) is used to calculate the entity extraction loss L E . The calculation formula is as follows:

[0142] L R = ∑ r∈R BCE(p r (r|x), y r )…………(5)

[0143]

[0144] where y r ∈ {0, 1}, y os , y oe , y ss , The calculation of binary cross entropy is as follows:

[0145] BCE(p, y) = -(y log p + (1 - y) log(1 - p))…………(7)

[0146] The calculation of multi-class cross entropy is as follows:

[0147] CE(p, y) = -y log p y …………(8)

[0148] The final loss sums the two with weights and jointly trains, and the parameters of the entity and relation extraction model are optimized by the gradient descent method.

[0149] L = L R + λL E …………(9)

[0150] After the training is completed, the input text "Who is Zhang San's friend?" is input into the entity and relation extraction model. The input text is divided by grains and special markers are added at the beginning and end to obtain the text sequence "[CLS], Zhang, San, 's, friend, is,?, [SEP]". This text sequence is input into the BERT word vector encoding layer to obtain a word vector encoding sequence; the word vector encoding sequence is input into the multi-label relation classification layer to obtain at least one relation type contained in the input text, and then input into the relation type label mixing layer to fuse the relation type represented by the relation type feature vector and the word vector encoding sequence to obtain a word vector encoding representation of the mixed relation type label; finally, the word vector encoding representation of the mixed relation type label is input into the entity position / type extraction layer to determine and extract the positions of the subject and object.

[0151] See Figure 4 The present application provides a logic diagram for an information extraction method. This method can extract entity information and relationship information even when the input text is incomplete. Specifically, it includes the following steps:

[0152] Step S401, Begin.

[0153] Step S402: Construct a sequence of word vectors for the input text.

[0154] Specifically, the input text is segmented, and a word vector encoding layer is used to encode the input text to obtain a word vector sequence.

[0155] Step S403, classification of multi-label relationship types.

[0156] Specifically, the word vector sequence is input into the relation classification layer to obtain at least one target relation type contained in the input text.

[0157] Step S404: Determine whether the traversal relationship type has ended.

[0158] Specifically, it determines whether all relation types within the target relation type have been traversed, i.e., whether the corresponding subject and object entities have been extracted for each target relation type. If the traversal is complete, all relation triples are output; otherwise, the step of merging relation types and word vector sequences continues.

[0159] Step S405: Merge the relation type with the word vector sequence.

[0160] Specifically, the relation type represented by the type feature vector and the word vector sequence are fused to obtain a fused feature vector sequence; the fused feature vector sequence is input into the entity information extraction layer to calculate the probability of the entity type of each character, and the entity type label of each character in the text to be identified is obtained. Based on the entity type label of each character, at least one entity under the target relation type can be determined.

[0161] Step S406: Calculate the probability of the subject / object entity.

[0162] Specifically, after inputting the fused feature vector sequence into the entity information extraction layer, the probability of the entity type of each character is calculated. If the probability of the entity type of the character is greater than a preset threshold, the entity type is used as the entity type label of the character, and the probability of each character as a subject / object entity under the target relation is determined.

[0163] Step S407: Determine the start / end position and entity type of the entity, and construct triples.

[0164] Specifically, based on the start and end positions of the characters and the entity type, the start and end positions of at least one entity in the text to be identified are determined, and the category of the entity is determined based on the entity type label of each character in the entity, thus completing the entity extraction.

[0165] Step S408: Output all triples.

[0166] Specifically, iterate through all relation types in the target relation type and output the triples of these relations.

[0167] Step S409, End.

[0168] The subject and object are extracted separately through two sets of entity extraction modules, which can directly determine the role of the entity in the relation triple (i.e., whether it is the subject or the object). This avoids building entity pairs before relation extraction and solves the problem of not being able to build relation model input when the sentence structure is incomplete. It is more suitable for entity and relation extraction of short texts.

[0169] Based on the same technical concept, this application provides a schematic diagram of an information extraction device, such as... Figure 5 As shown, the device 500 includes:

[0170] The acquisition module 501 is used to acquire the text to be identified and construct a word vector sequence of the text to be identified;

[0171] Processing module 502 is used to input the word vector sequence into the relation classification layer to obtain at least one target relation type contained in the text to be identified;

[0172] The processing module 502 is further configured to, for each target relation type, determine at least one entity contained in the text to be identified under the target relation type based on the type feature vector corresponding to the target relation type and the word vector sequence;

[0173] The construction module 503 is used to construct triplet information corresponding to the target relationship type based on the target relationship type and the at least one entity.

[0174] Optionally, the acquisition module 501 is specifically used for:

[0175] The text to be identified is divided into segments at the character level to obtain the target text sequence;

[0176] The target text sequence is encoded using a word vector encoding layer to obtain the word vector sequence.

[0177] Optionally, the processing module 502 is specifically used for:

[0178] The word vector sequence is input into the relation classification layer to obtain the probability that the text to be identified contains each type of reference relation.

[0179] Reference relationship types with a probability greater than a preset threshold are taken as the target relationship types contained in the text to be identified.

[0180] Optionally, the acquisition module 501 is specifically used for:

[0181] Obtain the type feature vector corresponding to the target relationship type from the preset relationship embedding matrix.

[0182] Optionally, the processing module 502 is specifically used for:

[0183] The type feature vector and the word vector sequence are fused to obtain a fused feature vector sequence;

[0184] The fused feature vector sequence is input into the entity information extraction layer to obtain the entity type label for each character in the text to be identified;

[0185] Based on the entity type label of each obtained character, determine at least one entity contained in the text to be identified under the target relation type.

[0186] Optionally, the processing module 502 is specifically used for:

[0187] The entity information extraction layer includes a first entity extraction module and a second entity extraction module;

[0188] The step of inputting the fused feature vector sequence into the entity information extraction layer to obtain the entity type label for each character in the text to be identified includes:

[0189] The fused feature vector sequence is input into the first entity extraction module to obtain the entity type label when each character in the text to be identified corresponds to the subject entity;

[0190] The fused feature vector sequence is input into the second entity extraction module to obtain the entity type label when each character in the text to be identified corresponds to an object entity.

[0191] Optionally, the processing module 502 is specifically used for:

[0192] Based on the entity type label obtained for each character, determine at least one entity start position and corresponding entity end position in the text to be identified;

[0193] Based on the starting position of the at least one entity and the corresponding ending position of the entity, at least one entity contained in the text to be identified is determined.

[0194] Optionally, the processing module 502 is specifically used for:

[0195] The text segment between the start position and the corresponding end position of each entity in the text to be identified is taken as an entity in the text to be identified.

[0196] Optionally, the construction module 503 is specifically used for:

[0197] The step of constructing triplet information corresponding to the target relation type based on the target relation type and the at least one entity includes:

[0198] Based on the target relation type, the preset empty character, and the subject entity, construct the triplet information corresponding to the target relation type; or...

[0199] Based on the target relation type, the preset empty character, and the object entity, construct the triplet information corresponding to the target relation type.

[0200] Optionally, the construction module 503 is specifically used for:

[0201] The at least one entity includes a subject entity and an object entity;

[0202] The step of constructing triplet information corresponding to the target relation type based on the target relation type and the at least one entity includes:

[0203] Based on the target relation type, the subject entity, and the object entity, construct the triplet information corresponding to the target relation type.

[0204] The subject and object are extracted separately through two sets of entity extraction modules, which can directly determine the role of the entity in the relation triple (i.e., whether it is the subject or the object), avoiding the construction of entity pairs before relation extraction. This solves the problem of not being able to construct a relation model input when the sentence structure is incomplete, and is suitable for entity and relation extraction from short texts. It effectively extracts entity and relation information, which helps to achieve machine understanding.

[0205] Based on the same technical concept, embodiments of this application provide a computer device, which can be... Figure 1 The terminal device 101 and / or server 102 shown are, for example Figure 6 As shown, it includes at least one processor 601 and a memory 602 connected to at least one processor. In this embodiment, the specific connection medium between the processor 601 and the memory 602 is not limited. Figure 6 Taking the connection between the processor 601 and the memory 602 via a bus as an example, the bus can be divided into address bus, data bus, control bus, etc.

[0206] In this embodiment of the application, the memory 602 stores instructions that can be executed by at least one processor 601. By executing the instructions stored in the memory 602, at least one processor 601 can perform the steps of the above-described information extraction method.

[0207] The processor 601 is the control center of the computer device. It can connect to various parts of the computer device using various interfaces and lines. By running or executing instructions stored in the memory 602 and calling data stored in the memory 602, it can display images at specific frame moments. Optionally, the processor 601 may include one or more processing units. The processor 601 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 601. In some embodiments, the processor 601 and the memory 602 may be implemented on the same chip; in some embodiments, they may also be implemented on separate chips.

[0208] Processor 601 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0209] Memory 602, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 602 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 602 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer device, but is not limited thereto. In the embodiments of this application, memory 602 may also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0210] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the above-described information extraction method.

[0211] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0212] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0213] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0214] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0215] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0216] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. An information extraction method, characterized in that, include: Obtain the text to be recognized and construct a sequence of word vectors for the text to be recognized; The word vector sequence is input into the relation classification layer to obtain at least one target relation type contained in the text to be identified; For each target relation type, based on the type feature vector corresponding to the target relation type and the word vector sequence, at least one entity contained in the text to be identified under the target relation type is determined, and the at least one entity is a subject entity or an object entity; The entity is obtained in the following ways: The text segment between the start position and the corresponding end position of each entity in the text to be identified is regarded as an entity in the text to be identified; When the text to be recognized is input into the first entity extraction module, the extracted entity will be used as a subject entity in the text to be recognized. When the text to be recognized is input into the second entity extraction module, the extracted entity will be used as an object entity in the text to be recognized. Based on the target relation type, the preset empty character, and the subject entity, construct the triplet information corresponding to the target relation type; or, Based on the target relation type, the preset empty character, and the object entity, construct the triplet information corresponding to the target relation type.

2. The method according to claim 1, characterized in that, The construction of the word vector sequence of the text to be identified includes: The text to be identified is divided into segments at the character level to obtain the target text sequence; The target text sequence is encoded using a word vector encoding layer to obtain the word vector sequence.

3. The method according to claim 1, characterized in that, The step of inputting the word vector sequence into a relation classification layer to obtain at least one target relation type contained in the text to be identified includes: The word vector sequence is input into the relation classification layer to obtain the probability that the text to be identified contains each type of reference relation. Reference relationship types with a probability greater than a preset threshold are taken as the target relationship types contained in the text to be identified.

4. The method according to claim 1, characterized in that, Before determining at least one entity contained in the text to be identified under the target relation type based on the type feature vector corresponding to the target relation type and the word vector sequence, the method further includes: Obtain the type feature vector corresponding to the target relationship type from the preset relationship embedding matrix.

5. The method according to claim 1, characterized in that, The step of determining at least one entity contained in the text to be identified under the target relation type based on the type feature vector corresponding to the target relation type and the word vector sequence includes: The type feature vector and the word vector sequence are fused to obtain a fused feature vector sequence; The fused feature vector sequence is input into the entity information extraction layer to obtain the entity type label for each character in the text to be identified; Based on the entity type label of each obtained character, determine at least one entity contained in the text to be identified under the target relation type.

6. The method according to claim 5, characterized in that, The entity information extraction layer includes a first entity extraction module and a second entity extraction module; The step of inputting the fused feature vector sequence into the entity information extraction layer to obtain the entity type label for each character in the text to be identified includes: The fused feature vector sequence is input into the first entity extraction module to obtain the entity type label when each character in the text to be identified corresponds to the subject entity; The fused feature vector sequence is input into the second entity extraction module to obtain the entity type label when each character in the text to be identified corresponds to an object entity.

7. The method according to claim 5, characterized in that, The step of determining at least one entity contained in the text to be identified under the target relation type based on the obtained entity type label of each character includes: Based on the entity type label obtained for each character, determine at least one entity start position and corresponding entity end position in the text to be identified; Based on the starting position of the at least one entity and the corresponding ending position of the entity, at least one entity contained in the text to be identified is determined.

8. The method according to claim 7, characterized in that, The step of determining at least one entity contained in the text to be identified based on the start position and corresponding end position of the at least one entity includes: The text segment between the start position and the corresponding end position of each entity in the text to be identified is taken as an entity in the text to be identified.

9. The method according to any one of claims 1 to 8, characterized in that, The at least one entity includes a subject entity and an object entity; The step of constructing triplet information corresponding to the target relation type based on the target relation type and the at least one entity includes: Based on the target relation type, the subject entity, and the object entity, construct the triplet information corresponding to the target relation type.

10. An information extraction device, characterized in that, include: The acquisition module is used to acquire the text to be recognized and construct a word vector sequence of the text to be recognized; The processing module is used to input the word vector sequence into the relation classification layer to obtain at least one target relation type contained in the text to be identified; The processing module is further configured to, for each target relation type, determine, based on the type feature vector corresponding to the target relation type and the word vector sequence, at least one entity contained in the text to be identified under the target relation type, wherein the at least one entity is a subject entity or an object entity; The construction module is used to take the text segment between the start position and the corresponding end position of each entity in the text to be identified as an entity in the text to be identified; When the text to be recognized is input into the first entity extraction module, the extracted entity will be used as a subject entity in the text to be recognized. When the text to be recognized is input into the second entity extraction module, the extracted entity will be used as an object entity in the text to be recognized. Based on the target relation type, the preset empty character, and the subject entity, construct the triplet information corresponding to the target relation type; or, Based on the target relation type, the preset empty character, and the object entity, construct the triplet information corresponding to the target relation type.

11. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, It stores a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the method according to any one of claims 1 to 9.

Images