Model training-based relation extraction method and device, electronic equipment and medium

Abstract

This invention relates to the field of natural language processing, disclosing a method, apparatus, electronic device, and storage medium for relation extraction based on model training. The method includes: inserting preset characters before and after entity pairs in training text and text to be extracted, obtaining target training text and target text to be extracted; encoding entity pairs in the target training text using a preset model, and concatenating the encoded vectors to obtain entity pair concatenation vectors; calculating the loss value of the entity pair concatenation vectors; adjusting the parameters of the preset model according to the loss value until the loss value meets a preset loss threshold, obtaining a trained preset model; using the trained preset model to extract target entity concatenation vectors from the target text to be extracted; calculating the similarity between the target entity concatenation vectors and preset sample representation vectors, and using the entity relation of the preset sample representation vector with the highest similarity as the entity relation of the text to be extracted. This invention can improve the efficiency of triple entity relation extraction.

Description

Technical Field

[0001] This invention relates to the field of natural language processing, and more particularly to a method, apparatus, electronic device, and readable storage medium for relation extraction based on model training. Background Technology

[0002] With the development of technology, knowledge graphs are being applied to more and more fields. The most important aspect of building a knowledge graph is constructing triples. A triple is a set of the form (s, p, o), where s is an entity, o can be an entity or an attribute value, and p is the relationship between two entities or an attribute of an entity. For example, from the statement "The beneficiary of the personal safety insurance is Xiaoming", we can extract the triple (personal safety insurance, beneficiary, Xiaoming).

[0003] Currently, most common triple relation extraction methods first extract entities and then determine the relationship between pairs of entities based on some publicly available labeled data. However, since the relationships between entities in different domains are different, a large amount of relation extraction data needs to be labeled when building a vertical domain knowledge graph, which increases the workload of triple relation extraction and reduces the efficiency of relation extraction. Summary of the Invention

[0004] This invention provides a method, apparatus, electronic device, and readable storage medium for relation extraction based on model training, with the aim of improving the efficiency of triple entity relation extraction.

[0005] To achieve the above objectives, the present invention provides a relation extraction method based on model training, the method comprising:

[0006] The training text and the text to be extracted are obtained. The training text entity pairs and the text to be extracted are identified in the training text and the text to be extracted, respectively. A preset character is inserted before and after the training text entity pair to obtain the target training text. The preset character is inserted before and after the text to be extracted entity pair to obtain the target text to be extracted.

[0007] The training text entity pairs in the target training text are encoded using a preset semantic representation model to obtain the first entity encoding vector and the second entity encoding vector corresponding to the training text entity pairs in the target training text.

[0008] The first entity encoding vector and the second entity encoding vector are concatenated to obtain an entity pair concatenation vector, and the loss value of the entity pair concatenation vector is calculated.

[0009] The parameters of the semantic representation model are adjusted according to the loss value, and the process of encoding the training text entity pairs in the target training text using the preset semantic representation model to obtain the first entity encoding vector and the second entity encoding vector corresponding to the training text entity pairs in the target training text is repeated until the loss value meets the preset loss threshold, and the trained semantic representation model is obtained.

[0010] Using the trained semantic representation model, extract the target entity concatenation vector of the target text to be extracted;

[0011] The similarity between the target entity concatenation vector and the preset sample representation vector is calculated using cosine similarity, and the entity relationship of the preset sample representation vector with the highest similarity is taken as the entity relationship of the text to be extracted.

[0012] Optionally, the step of encoding the training text entity pairs in the target training text using a preset semantic representation model to obtain the first entity encoding vector and the second entity encoding vector corresponding to the training text entity pairs in the target training text includes:

[0013] The entity pairs in the target training text are encoded character by character using a pre-defined semantic representation model to obtain a sequence of character vectors.

[0014] Extract the feature vectors of entity pairs from the target training text to obtain a first word vector sequence and a second word vector sequence;

[0015] Based on the number of characters in the word vector sequence, the first word vector sequence and the second word vector sequence are expanded respectively to obtain a first aligned word vector sequence and a second aligned word vector sequence that are aligned with the word vector sequence;

[0016] The first aligned word vector sequence and the second aligned word vector sequence are cross-multiplied with a preset transformation matrix to obtain a first target word vector sequence and a second target word vector sequence with the same dimension as the word vector sequence.

[0017] The first target word vector sequence and the second target word vector sequence are added to the corresponding character vector sequences respectively to obtain the first character word vector sequence and the second character word vector sequence;

[0018] For each character in the entity pair in the target training text, perform position index encoding to obtain the text vector position encoding;

[0019] The first word vector sequence and the second word vector sequence are added to the corresponding text vector position codes to obtain a first text concatenation vector and a second text concatenation vector. The first text concatenation vector and the second text concatenation vector are then encoded using the encoding layer in the preset semantic representation model to obtain a first entity encoding vector and a second entity encoding vector.

[0020] Optionally, calculating the loss value of the entity pair concatenated vector includes:

[0021] The loss value of the entity pair concatenation vector is calculated using the following loss function. :

[0022]

[0023] in, This represents a concatenated vector of any pair of entities. Represents the transpose matrix. Indicates and The corresponding concatenated vectors of entities with the same relation. Indicates and The corresponding concatenated vector of non-identical relational entity pairs, express Entity pair sequence number, express The number of corresponding entity pairs.

[0024] Optionally, the step of identifying the training text entity pairs and the text entity pairs to be extracted in the training text and the text to be extracted respectively includes:

[0025] The training text and the text to be extracted are encoded using a preset encoding model to obtain the training text vector and the text to be extracted vector, respectively.

[0026] The weights of the training text vector and the text vector to be extracted are calculated using a preset self-attention mechanism to obtain weighted training text vectors and weighted text vectors to be extracted.

[0027] The entity positions in the weighted training text vector and the weighted text vector to be extracted are predicted using a pre-defined semi-pointer semi-label prediction model, respectively, to obtain training text entity pairs and text entity pairs to be extracted.

[0028] Optionally, the step of using a preset semi-pointer, semi-label prediction model to predict the entity positions in the weighted training text vector and the weighted text vector to be extracted, respectively, to obtain training text entity pairs and text entity pairs to be extracted, includes:

[0029] Using the start and end pointers in the preset half-pointer half-label prediction model, the start and end positions of the corresponding candidate entity pairs are selected from the weighted training text vector and the weighted text vector to be extracted, respectively.

[0030] Based on the first and last positions of the candidate entity pair, position encoding is performed on the candidate entity pair to obtain the position encoding vector of the candidate entity pair;

[0031] The candidate entity pair position encoding vector is multiplied by the corresponding training text weight vector and the text weight vector to be extracted, respectively, to obtain the training text weighted sum vector and the text weighted sum vector to be extracted.

[0032] Target training weighted summation vectors and target extraction weighted summation vectors with values ​​not less than preset values ​​are selected from the weighted summation vectors of the training text and the weighted summation vectors of the text to be extracted, respectively. The first and last positions of the candidate entity pairs corresponding to the target training weighted summation vectors are used as the first and last positions of the entity pairs of the training text to obtain training text entity pairs. The first and last positions of the candidate entity pairs corresponding to the target extraction weighted summation vectors are used as the first and last positions of the entity pairs of the text to be extracted to obtain text entity pairs.

[0033] Optionally, the step of calculating the similarity between the target entity concatenation vector and the preset sample representation vector using cosine similarity includes:

[0034] The similarity between the concatenated vector of the target entity and the preset sample representation vector is calculated using the following formula. :

[0035]

[0036] in, This represents the concatenated vector of the target entity. This represents the preset sample representation vector. This indicates the size of the target entity concatenation vector. This indicates the size of the preset sample representation vector.

[0037] Optionally, before calculating the loss value of the entity pair concatenated vector, the method includes:

[0038] Obtain a preset number of entity pair concatenation vectors;

[0039] Cluster the entity pair concatenation vectors to obtain entity pair concatenation vectors with the same relationship and entity pair concatenation vectors with different relationships.

[0040] To address the above problems, the present invention also provides a relation extraction device based on model training, the device comprising:

[0041] The entity pair recognition module is used to acquire training text and text to be extracted, respectively identify the training text entity pairs and the text to be extracted in the training text and the text to be extracted, and insert preset characters before and after the training text entity pairs to obtain the target training text, and insert preset characters before and after the text to be extracted entity pairs to obtain the target text to be extracted.

[0042] The model training module is used to encode the training text entity pairs in the target training text using a preset semantic representation model, to obtain the first entity encoding vector and the second entity encoding vector corresponding to the training text entity pairs in the target training text, to concatenate the first entity encoding vector and the second entity encoding vector to obtain the entity pair concatenation vector, to calculate the loss value of the entity pair concatenation vector, to adjust the parameters of the semantic representation model according to the loss value, and to return to the step of encoding the training text entity pairs in the target training text using the preset semantic representation model to obtain the first entity encoding vector and the second entity encoding vector corresponding to the training text entity pairs in the target training text, until the loss value meets the preset loss threshold, and the trained semantic representation model is obtained.

[0043] The entity relation extraction module is used to extract the target entity concatenation vector of the target text to be extracted using the trained semantic representation model, calculate the similarity between the target entity concatenation vector and the preset sample representation vector using cosine similarity, and take the entity relation of the preset sample representation vector with the highest similarity as the entity relation of the text to be extracted.

[0044] To address the above problems, the present invention also provides an electronic device, the electronic device comprising:

[0045] Memory, storing at least one computer program; and

[0046] The processor executes the computer program stored in the memory to implement the model training-based relation extraction method described above.

[0047] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing at least one computer program, which is executed by a processor in an electronic device to implement the model-trained relation extraction method described above.

[0048] This invention, through obtaining training text and text to be extracted, identifies the training text entity pairs and the text to be extracted in the training text and the text to be extracted, respectively. Preset characters are inserted before and after the training text entity pairs to obtain the target training text. Similarly, preset characters are inserted before and after the text to be extracted entity pairs to obtain the target text to be extracted. The positions of the entity pairs are marked, ensuring accurate encoding of the entity pairs during model training, thus obtaining the corresponding entity pair encoding vectors. Next, a preset semantic representation model is trained using the target training text to obtain the trained semantic representation model. By concatenating the encoding vectors of the entity pairs in the target training text, the entity pairs in the target training text are encoded. The more precise relationships between entity pairs lead to more accurate semantic representations of entity relationships, more precise classifications, and more clustered categories. This reduces the amount of labeled data required, decreases the workload of labeling data in triplet relation extraction, and improves the efficiency of triplet relation extraction. Finally, the target text to be extracted is input into the trained semantic representation model to obtain the target entity concatenation vector. The similarity between the target entity concatenation vector and the preset sample representation vector is calculated using cosine similarity. The entity relationship of the sample representation vector with the highest similarity is taken as the entity relationship of the text to be extracted, ensuring the accuracy of triplet entity relationships in the text to be extracted without the need for extensive relation labeling, thereby improving the efficiency of extracting triplet entity relationships from the text to be extracted. Therefore, the relation extraction method, apparatus, device, and storage medium based on model training provided by this invention can improve the efficiency of triplet entity relation extraction. Attached Figure Description

[0049] Figure 1 This is a flowchart illustrating a relation extraction method based on model training provided in an embodiment of the present invention.

[0050] Figures 2 to 3 This is a detailed flowchart of one step in a relation extraction method based on model training provided in an embodiment of the present invention.

[0051] Figure 4 This is a schematic diagram of a model-trained relation extraction device according to an embodiment of the present invention;

[0052] Figure 5 A schematic diagram of the internal structure of an electronic device that implements a model-trained relation extraction method according to an embodiment of the present invention;

[0053] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0054] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0055] This invention provides a relation extraction method based on model training. The execution entity of the model-trained relation extraction method includes, but is not limited to, at least one of electronic devices, such as servers and terminals, that can be configured to execute the method provided in this application. In other words, the model-trained relation extraction method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server can include an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0056] Reference Figure 1 The flowchart shown is a model-trained relation extraction method according to an embodiment of the present invention. In this embodiment, the model-trained relation extraction method includes:

[0057] S1. Obtain training text and text to be extracted, identify training text entity pairs and text entity pairs to be extracted in the training text and the text to be extracted respectively, and insert preset characters before and after the training text entity pairs to obtain target training text, and insert preset characters before and after the text entity pairs to be extracted to obtain target text to be extracted.

[0058] In this embodiment of the invention, the training text can be text data for model training with confirmed triples. The text to be extracted can be text data extracted from text triples that have not been determined. The entity pair can be a set of head and tail entities in the text. For example, in the text "The beneficiary of personal safety insurance is Xiaoming", the head entity is "personal safety insurance" and the tail entity is "Xiaoming". Therefore, the entity pair of this text can be represented as (personal safety insurance, Xiaoming). The preset character can be a special character that can be recognized by the model. For example, in the text "The headquarters of XX Group is in Shenzhen", the entity pair can be "XX Group" and "Shenzhen". Therefore, after inserting the preset character, the text becomes [E1]XX Group[\E1]'s headquarters is in [E2]Shenzhen[\E2].

[0059] In an optional embodiment of the present invention, target text is obtained by querying network text data, and then the target text is filtered to obtain training text, ensuring the randomness of the training text and thus making the training results of the model more accurate. Further, text statements are selected as the text to be extracted from a locally stored knowledge graph text database, wherein the knowledge graph text database may store text data used to construct the knowledge graph.

[0060] This invention, through recognizing the training text entity pairs and the text to be extracted in the training text and the text to be extracted respectively, determines the position of the entity pairs in the training text and the text to be extracted, thereby facilitating the insertion of special characters into the entity pairs.

[0061] Furthermore, as an optional embodiment of the present invention, reference is made to... Figure 2 As shown, the step of identifying the training text entity pairs and the text to be extracted in the training text and the text to be extracted, respectively, includes:

[0062] S11. Encode the training text and the text to be extracted using a preset encoding model to obtain the training text vector and the text to be extracted vector.

[0063] S12. Calculate the weights of the training text vector and the text vector to be extracted using a preset self-attention mechanism to obtain weighted training text vector and weighted text vector to be extracted.

[0064] S13. Using a preset semi-pointer semi-label prediction model, the entity positions in the weighted training text vector and the weighted text vector to be extracted are predicted respectively, to obtain training text entity pairs and text entity pairs to be extracted.

[0065] In this embodiment of the invention, the encoding model can be a bidirectional encoder based on Transformer or a neural network model based on deep learning. The self-attention mechanism has the ability to determine which part of the input to focus on, allocate limited information processing resources to the important parts, and weight the important parts. The semi-pointer, semi-label prediction model can be a CNN model built on a semi-pointer-semi-label structure, mainly used to predict the start and end positions of target words.

[0066] In an optional embodiment of the present invention, firstly, text features of the training text and the text to be extracted are extracted respectively, and word-word hybrid encoding is performed on the text features to obtain a word vector sequence. Further, position index encoding is performed on each character in the training text and the text to be extracted respectively to obtain a text vector position encoding. Finally, the word vector sequence and the text vector position encoding are added together to obtain a text concatenation vector. The text concatenation vector is then encoded using the encoding module in the preset semantic representation model to obtain the training text vector and the text vector to be extracted.

[0067] In another optional embodiment of the present invention, the general location of the entity can first be determined based on the entity's own attributes, and the general location can be weighted using a normalized exponential function in a preset self-attention mechanism to obtain the corresponding training text weight vector and the text weight vector to be extracted. Here, determining the general location of the entity based on its own attributes means determining the position of the entity in the training text and the text to be extracted based on the entity's noun, subject, and object attributes. For example, the policyholder of pension insurance is an elderly person, where "is" is the predicate, so the elderly person after "is" can be one of the entities.

[0068] Furthermore, in an optional embodiment of the present invention, reference is made to... Figure 3 As shown, the step of using a preset semi-pointer, semi-label prediction model to predict the entity positions in the training text weight vector and the text weight vector to be extracted, respectively, to obtain training text entity pairs and text entity pairs to be extracted, includes:

[0069] S14. Using the start and end pointers in the preset half-pointer half-label prediction model, the start and end positions of the corresponding candidate entity pairs are selected from the weighted training text vector and the weighted text vector to be extracted, respectively.

[0070] S15. Based on the first and last positions of the candidate entity pair, perform position encoding on the candidate entity pair to obtain the candidate entity pair position encoding vector;

[0071] S16. Perform dot product between the candidate entity pair position encoding vector and the corresponding training text weight vector and the text weight vector to be extracted, respectively, to obtain the training text weighted sum vector and the text weighted sum vector to be extracted.

[0072] S17. Select target training weighted summation vectors and target extraction weighted summation vectors that are greater than preset values ​​from the training text weighted summation vector and the text to be extracted weighted summation vector, respectively. Take the first and last positions of the candidate entity pairs corresponding to the target training weighted summation vector as the first and last positions of the entity pairs of the training text to obtain training text entity pairs. Take the first and last positions of the candidate entity pairs corresponding to the target extraction weighted summation vector as the first and last positions of the entity pairs of the text to be extracted to obtain text entity pairs.

[0073] In this embodiment of the invention, the start pointer and the end pointer can be memory addresses of the beginning and end positions of an entity.

[0074] Furthermore, in an optional embodiment of the present invention, special characters that can be recognized by the semantic representation model are set, and the feature characters are inserted before and after the successfully recognized entities to obtain the target training text and the target text to be extracted. This makes it easier for the semantic representation model to find the entity pair positions during encoding, thereby improving the model encoding efficiency.

[0075] S2. Encode the training text entity pairs in the target training text using a preset semantic representation model to obtain the first entity encoding vector and the second entity encoding vector corresponding to the training text entity pairs in the target training text.

[0076] In this embodiment of the invention, the semantic representation model may be a Bert (Bidirectional Encoder Representation from Transformers) model, wherein the Bert model is a bidirectional encoder representation based on Transformers.

[0077] This invention utilizes a preset semantic representation model to encode training text entity pairs in the target training text, obtaining a first entity encoding vector and a second entity encoding vector corresponding to the training text entity pairs in the target training text. This ensures that the target training text can be further processed based on the position of the entity pairs, thereby improving the keyword extraction capability of the semantic representation model for the target training text and increasing the accuracy of triple relation extraction.

[0078] Further, as an optional embodiment of the present invention, the step of encoding the training text entity pairs in the target training text using a preset semantic representation model to obtain the first entity encoding vector and the second entity encoding vector corresponding to the training text entity pairs in the target training text includes:

[0079] The entity pairs in the target training text are encoded character by character using a pre-defined semantic representation model to obtain a sequence of character vectors.

[0080] Extract the feature vectors of entity pairs from the target training text to obtain a first word vector sequence and a second word vector sequence;

[0081] Based on the number of characters in the word vector sequence, the first word vector sequence and the second word vector sequence are expanded respectively to obtain a first aligned word vector sequence and a second aligned word vector sequence that are aligned with the word vector sequence;

[0082] The first aligned word vector sequence and the second aligned word vector sequence are cross-multiplied with a preset transformation matrix to obtain a first target word vector sequence and a second target word vector sequence with the same dimension as the word vector sequence.

[0083] The first target word vector sequence and the second target word vector sequence are added to the corresponding character vector sequences respectively to obtain the first character word vector sequence and the second character word vector sequence;

[0084] For each character in the entity pair in the target training text, perform position index encoding to obtain the text vector position encoding;

[0085] The first word vector sequence and the second word vector sequence are added to the corresponding text vector position codes to obtain a first text concatenation vector and a second text concatenation vector. The first text concatenation vector and the second text concatenation vector are then encoded using the encoding layer in the preset semantic representation model to obtain a first entity encoding vector and a second entity encoding vector.

[0086] In this embodiment of the invention, the preset text dimensionality reduction layer is mainly used to reduce the dimensionality of text to obtain a shorter character vector sequence. The preset transformation matrix can be a manually set matrix that changes according to the dimensionality of the character vector sequence.

[0087] In an optional embodiment of the present invention, since simply using word vectors can easily lead to entity recognition errors, by performing word-word hybrid dimensionality reduction processing on entity pairs in the target training text to obtain a word vector sequence, it helps to improve the accuracy and comprehensive coverage of the final insurance domain triple extraction of the insurance target text.

[0088] S3. Concatenate the first entity encoding vector and the second entity encoding vector to obtain the entity pair concatenation vector, and calculate the loss value of the entity pair concatenation vector.

[0089] In an optional embodiment of the present invention, the first entity encoding vector and the second entity encoding vector are concatenated by a vector concatenation method to obtain an entity pair concatenation vector, thereby tightly connecting the first entity and the second entity in the training text, making the relationship between the entity pairs more prominent and improving the accuracy of model training.

[0090] This invention improves the accuracy of the model by calculating the loss value of the entity pair concatenation vector and determining whether the preset semantic representation model has been trained.

[0091] Further, as an optional embodiment of the present invention, before calculating the loss value of the entity pair concatenation vector, the method includes:

[0092] Obtain a preset number of entity pair concatenation vectors;

[0093] Cluster the entity pair concatenation vectors to obtain entity pair concatenation vectors with the same relationship and entity pair concatenation vectors with different relationships.

[0094] In this embodiment of the invention, in order to ensure that the semantic representation model after training has a good representation of the semantic relationship between entity pairs, it is necessary to ensure that there are positive and negative relation entity pairs during model training. The positive and negative relation entity pairs can be entity pairs corresponding to the concatenation vector of entity pairs with the same relation and the concatenation vector of entity pairs with different relation.

[0095] Further, as an optional embodiment of the present invention, calculating the loss value of the entity pair concatenation vector includes:

[0096] The loss value of the entity pair concatenation vector is calculated using the following loss function. :

[0097]

[0098] in, This represents a concatenated vector of any pair of entities. Represents the transpose matrix. Indicates and The corresponding concatenated vectors of entities with the same relation. Indicates and The corresponding concatenated vector of non-identical relational entity pairs, express Entity pair sequence number, express The number of corresponding entity pairs.

[0099] S4. Adjust the parameters of the semantic representation model according to the loss value, and return to the step of encoding the training text entity pairs in the target training text using the preset semantic representation model to obtain the first entity encoding vector and the second entity encoding vector corresponding to the training text entity pairs in the target training text, until the loss value meets the preset loss threshold, and obtain the trained semantic representation model.

[0100] In this embodiment of the invention, the parameter can be a parameter that affects the encoding process of the semantic representation model. The loss threshold can be the maximum loss value allowed by the semantic representation model.

[0101] In an optional embodiment of the present invention, when the loss value is greater than a preset loss threshold, it is determined that the training effect of the semantic representation model has not reached the preset goal, which has a significant impact on the accuracy of triplet relation extraction. When the loss value is not greater than the preset loss threshold, it is determined that the training effect of the semantic representation model has reached the preset goal, thereby improving the accuracy of determining triplet relations.

[0102] Furthermore, in an optional embodiment of the present invention, in order to ensure the universality of the semantic representation model, that is, to have a good vector concatenation effect on texts other than training texts, and to ensure that the semantic representation model does not overfit during training, the training text can be divided into training group texts and test group texts. The semantic representation model is used to train the training group texts, and then the test group texts are input into the trained semantic representation model to further verify the universality of the trained semantic representation model.

[0103] S5. Using the trained semantic representation model, extract the target entity concatenation vector of the target text to be extracted.

[0104] In an optional embodiment of the present invention, the entity pairs in the target text to be extracted are first encoded using the trained semantic representation model to obtain the third entity encoding vector and the fourth entity encoding vector corresponding to the entity pair. Finally, the third entity encoding vector and the fourth entity encoding vector are concatenated to obtain the target entity pair concatenation vector.

[0105] Furthermore, in an optional embodiment of the present invention, the step of encoding entity pairs in the target text to be extracted using the trained semantic representation model to obtain the third entity encoding vector and the fourth entity encoding vector corresponding to the entity pairs is similar to the step of encoding training text entity pairs in the target training text using a preset semantic representation model to obtain the first entity encoding vector and the second entity encoding vector corresponding to the training text entity pairs in the target training text, and therefore will not be described in detail. The step of concatenating the third entity encoding vector and the fourth entity encoding vector to obtain the target entity pair concatenation vector is similar to the step of concatenating the first entity encoding vector and the second entity encoding vector to obtain the entity pair concatenation vector, and therefore will not be described in detail.

[0106] S6. Calculate the similarity between the target entity concatenation vector and the preset sample representation vector using cosine similarity, and take the entity relationship of the preset sample representation vector with the highest similarity as the entity relationship of the text to be extracted.

[0107] In this embodiment of the invention, the preset sample representation vector can be an entity concatenation vector in the sample text.

[0108] In this embodiment of the invention, the similarity between the target entity concatenation vector and the preset sample representation vector is calculated using cosine similarity to determine the similarity between the target entity concatenation vector and the preset sample representation vector.

[0109] The similarity of the preset sample representation vectors is set, and the entity relationship of the preset sample representation vector with the highest similarity is selected as the entity relationship of the text to be extracted, thereby completing the extraction of triplet relationships and improving the extraction efficiency of triplet relationships.

[0110] Further, as an optional embodiment of the present invention, the step of calculating the similarity between the target entity concatenation vector and the preset sample representation vector using cosine similarity includes:

[0111] The similarity between the concatenated vector of the target entity and the preset sample representation vector is calculated using the following formula. :

[0112]

[0113] in, This represents the concatenated vector of the target entity. This represents the preset sample representation vector. This indicates the size of the target entity concatenation vector. This indicates the size of the preset sample representation vector.

[0114] This invention, through obtaining training text and text to be extracted, identifies the training text entity pairs and the text to be extracted in the training text and the text to be extracted, respectively. Preset characters are inserted before and after the training text entity pairs to obtain the target training text. Similarly, preset characters are inserted before and after the text to be extracted entity pairs to obtain the target text to be extracted. The positions of the entity pairs are marked, ensuring accurate encoding of the entity pairs during model training, thus obtaining the corresponding entity pair encoding vectors. Next, a preset semantic representation model is trained using the target training text to obtain the trained semantic representation model. By concatenating the encoding vectors of the entity pairs in the target training text, the target text is obtained. The more precise relationships between entity pairs in the training text result in more accurate semantic representations of entity relationships, more precise classification, and more clustered categories. This reduces the amount of labeled data required, decreases the workload of labeling data in triplet relation extraction, and improves the efficiency of triplet relation extraction. Finally, the target text to be extracted is input into the trained semantic representation model to obtain a target entity concatenation vector. The similarity between the target entity concatenation vector and a preset sample representation vector is calculated using cosine similarity. The entity relationship of the sample representation vector with the highest similarity is taken as the entity relationship of the text to be extracted, ensuring the accuracy of triplet entity relationships in the text to be extracted and improving the efficiency of extracting triplet entity relationships from the text. Therefore, the relation extraction method based on model training provided by this invention can improve the efficiency of triplet entity relation extraction.

[0115] like Figure 4 The diagram shown is a functional block diagram of the relation extraction device based on model training according to the present invention.

[0116] The model-trained relation extraction device 100 of the present invention can be installed in an electronic device. Depending on the functions implemented, the model-trained relation extraction device 100 may include an entity pair recognition module 101, a model training module 102, and an entity relation extraction module 103. The module mentioned in the present invention may also be referred to as a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and are stored in the memory of the electronic device.

[0117] In this embodiment, the functions of each module / unit are as follows:

[0118] The entity pair recognition module 101 is used to acquire training text and text to be extracted, respectively recognize the training text entity pairs and the text to be extracted in the training text and the text to be extracted, and insert preset characters before and after the training text entity pairs to obtain target training text, and insert preset characters before and after the text to be extracted entity pairs to obtain target text to be extracted.

[0119] The model training module 102 is used to encode the training text entity pairs in the target training text using a preset semantic representation model, to obtain the first entity encoding vector and the second entity encoding vector corresponding to the training text entity pairs in the target training text, to concatenate the first entity encoding vector and the second entity encoding vector to obtain the entity pair concatenation vector, to calculate the loss value of the entity pair concatenation vector, to adjust the parameters of the semantic representation model according to the loss value, and to return to the step of using the preset semantic representation model to encode the training text entity pairs in the target training text to obtain the first entity encoding vector and the second entity encoding vector corresponding to the training text entity pairs in the target training text, until the loss value meets the preset loss threshold, and the trained semantic representation model is obtained.

[0120] The entity relationship extraction module 103 is used to extract the target entity concatenation vector of the target text to be extracted using the trained semantic representation model, calculate the similarity between the target entity concatenation vector and the preset sample representation vector using cosine similarity, and take the entity relationship of the preset sample representation vector with the highest similarity as the entity relationship of the text to be extracted.

[0121] like Figure 5 The diagram shown is a schematic representation of the electronic device that implements the relation extraction method based on model training according to the present invention.

[0122] The electronic device may include a processor 10, a memory 11, a communication bus 12 and a communication interface 13, and may also include a computer program stored in the memory 11 and capable of running on the processor 10, such as a relation extraction program based on model training.

[0123] The memory 11 includes at least one type of readable storage medium, such as flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device, such as a plug-in portable hard drive, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc. Furthermore, the memory 11 can include both internal and external storage units of the electronic device. The memory 11 can be used not only to store application software and various types of data installed on the electronic device, such as code for a relation extraction program based on model training, but also to temporarily store data that has been output or will be output.

[0124] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., relation extraction programs based on model training) and calls data stored in the memory 11 to perform various functions of the electronic device and process data.

[0125] The communication bus 12 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The communication bus 12 is configured to enable communication between the memory 11 and at least one processor 10, etc. For ease of illustration, only one thick line is used in the figure, but this does not indicate that there is only one bus or one type of bus.

[0126] Figure 5 Only electronic devices with components are shown; those skilled in the art will understand that... Figure 5The structure shown does not constitute a limitation on the electronic device and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0127] For example, although not shown, the electronic device may also include a power supply (such as a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0128] Optionally, the communication interface 13 may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), which is typically used to establish communication connections between the electronic device and other electronic devices.

[0129] Optionally, the communication interface 13 may further include a user interface, which may be a display, an input unit (such as a keyboard), or a standard wired or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device and to display a visual user interface.

[0130] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0131] The relation extraction program based on model training stored in the memory 11 of the electronic device is a combination of multiple computer programs. When run in the processor 10, it can achieve the following:

[0132] The training text and the text to be extracted are obtained. The training text entity pairs and the text to be extracted are identified in the training text and the text to be extracted, respectively. A preset character is inserted before and after the training text entity pair to obtain the target training text. The preset character is inserted before and after the text to be extracted entity pair to obtain the target text to be extracted.

[0133] The training text entity pairs in the target training text are encoded using a preset semantic representation model to obtain the first entity encoding vector and the second entity encoding vector corresponding to the training text entity pairs in the target training text.

[0134] The first entity encoding vector and the second entity encoding vector are concatenated to obtain an entity pair concatenation vector, and the loss value of the entity pair concatenation vector is calculated.

[0135] The parameters of the semantic representation model are adjusted according to the loss value, and the process of encoding the training text entity pairs in the target training text using the preset semantic representation model to obtain the first entity encoding vector and the second entity encoding vector corresponding to the training text entity pairs in the target training text is repeated until the loss value meets the preset loss threshold, and the trained semantic representation model is obtained.

[0136] Using the trained semantic representation model, extract the target entity concatenation vector of the target text to be extracted;

[0137] The similarity between the target entity concatenation vector and the preset sample representation vector is calculated using cosine similarity, and the entity relationship of the preset sample representation vector with the highest similarity is taken as the entity relationship of the text to be extracted.

[0138] Specifically, the processor 10's implementation method of the above-mentioned computer program can be found in [reference needed]. Figure 1 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0139] Furthermore, if the modules / units integrated into the electronic device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable medium can be non-volatile or volatile. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0140] Embodiments of the present invention may also provide a computer-readable storage medium storing a computer program, which, when executed by a processor of an electronic device, can perform the following:

[0141] The training text and the text to be extracted are obtained. The training text entity pairs and the text to be extracted are identified in the training text and the text to be extracted, respectively. A preset character is inserted before and after the training text entity pair to obtain the target training text. The preset character is inserted before and after the text to be extracted entity pair to obtain the target text to be extracted.

[0142] The training text entity pairs in the target training text are encoded using a preset semantic representation model to obtain the first entity encoding vector and the second entity encoding vector corresponding to the training text entity pairs in the target training text.

[0143] The first entity encoding vector and the second entity encoding vector are concatenated to obtain an entity pair concatenation vector, and the loss value of the entity pair concatenation vector is calculated.

[0144] The parameters of the semantic representation model are adjusted according to the loss value, and the process of encoding the training text entity pairs in the target training text using the preset semantic representation model to obtain the first entity encoding vector and the second entity encoding vector corresponding to the training text entity pairs in the target training text is repeated until the loss value meets the preset loss threshold, and the trained semantic representation model is obtained.

[0145] Using the trained semantic representation model, extract the target entity concatenation vector of the target text to be extracted;

[0146] The similarity between the target entity concatenation vector and the preset sample representation vector is calculated using cosine similarity, and the entity relationship of the preset sample representation vector with the highest similarity is taken as the entity relationship of the text to be extracted.

[0147] Furthermore, the computer's usable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store the operating system, applications required for at least one function, etc.; and the data storage area may store data created based on the use of blockchain nodes, etc.

[0148] In the several embodiments provided by this invention, it should be understood that the disclosed electronic devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0149] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0150] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0151] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0152] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0153] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0154] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. The term "second class" is used to indicate names and does not indicate any specific order.

[0155] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A model-based training method for relation extraction, characterized in that, The method includes: The training text and the text to be extracted are obtained. The training text entity pairs and the text to be extracted are identified in the training text and the text to be extracted, respectively. A preset character is inserted before and after the training text entity pair to obtain the target training text. The preset character is inserted before and after the text to be extracted entity pair to obtain the target text to be extracted. The training text entity pairs in the target training text are encoded using a preset semantic representation model to obtain the first entity encoding vector and the second entity encoding vector corresponding to the training text entity pairs in the target training text. The first entity encoding vector and the second entity encoding vector are concatenated to obtain an entity pair concatenation vector, and the loss value of the entity pair concatenation vector is calculated. The parameters of the semantic representation model are adjusted according to the loss value, and the process of encoding the training text entity pairs in the target training text using the preset semantic representation model to obtain the first entity encoding vector and the second entity encoding vector corresponding to the training text entity pairs in the target training text is repeated until the loss value meets the preset loss threshold, and the trained semantic representation model is obtained. Using the trained semantic representation model, extract the target entity concatenation vector of the target text to be extracted; The similarity between the target entity concatenation vector and the preset sample representation vector is calculated using cosine similarity, and the entity relationship of the preset sample representation vector with the highest similarity is taken as the entity relationship of the text to be extracted. The step of identifying training text entity pairs and extractable text entity pairs in the training text and the text to be extracted, respectively, includes: encoding the training text and the text to be extracted using a preset encoding model to obtain training text vectors and extractable text vectors; calculating the weights of the training text vectors and the text to be extracted using a preset self-attention mechanism to obtain weighted training text vectors and weighted extractable text vectors; and predicting the entity positions in the weighted training text vectors and weighted extractable text vectors using a preset semi-pointer semi-label prediction model to obtain training text entity pairs and extractable text entity pairs. 2.The model training-based relation extraction method of claim 1, wherein, The step of encoding the training text entity pairs in the target training text using a preset semantic representation model to obtain the first entity encoding vector and the second entity encoding vector corresponding to the training text entity pairs in the target training text includes: The entity pairs in the target training text are encoded character by character using a pre-defined semantic representation model to obtain a sequence of character vectors. Extract the feature vectors of entity pairs from the target training text to obtain a first word vector sequence and a second word vector sequence; Based on the number of characters in the word vector sequence, the first word vector sequence and the second word vector sequence are expanded respectively to obtain a first aligned word vector sequence and a second aligned word vector sequence that are aligned with the word vector sequence; The first aligned word vector sequence and the second aligned word vector sequence are cross-multiplied with a preset transformation matrix to obtain a first target word vector sequence and a second target word vector sequence with the same dimension as the word vector sequence. The first target word vector sequence and the second target word vector sequence are added to the corresponding character vector sequences respectively to obtain the first character word vector sequence and the second character word vector sequence; For each character in the entity pair in the target training text, perform position index encoding to obtain the text vector position encoding; The first word vector sequence and the second word vector sequence are added to the corresponding text vector position codes to obtain a first text concatenation vector and a second text concatenation vector. The first text concatenation vector and the second text concatenation vector are then encoded using the encoding layer in the preset semantic representation model to obtain a first entity encoding vector and a second entity encoding vector.

3. The relation extraction method based on model training as described in claim 1, characterized in that, The calculation of the loss value of the entity pair concatenated vector includes: The loss value of the entity pair concatenation vector is calculated using the following loss function. : in, This represents a concatenated vector of any entity pair. Represents the transpose matrix. Indicates and The corresponding concatenated vectors of entities with the same relation. Indicates and The corresponding concatenated vector of non-identical entity pairs. express Entity pair sequence number, express The number of corresponding entity pairs.

4. The relation extraction method based on model training as described in claim 1, characterized in that, The step involves using a pre-defined semi-pointer, semi-label prediction model to predict entity positions in both the weighted training text vector and the weighted text vector to be extracted, thereby obtaining training text entity pairs and text entity pairs to be extracted. This includes: Using the start and end pointers in the preset half-pointer half-label prediction model, the start and end positions of the corresponding candidate entity pairs are selected from the weighted training text vector and the weighted text vector to be extracted, respectively. Based on the first and last positions of the candidate entity pair, position encoding is performed on the candidate entity pair to obtain the position encoding vector of the candidate entity pair; The candidate entity pair position encoding vector is multiplied by the corresponding training text weight vector and the text weight vector to be extracted, respectively, to obtain the training text weighted sum vector and the text weighted sum vector to be extracted. Target training weighted summation vectors and target extraction weighted summation vectors with values ​​not less than preset values ​​are selected from the weighted summation vectors of the training text and the weighted summation vectors of the text to be extracted, respectively. The first and last positions of the candidate entity pairs corresponding to the target training weighted summation vectors are used as the first and last positions of the entity pairs of the training text to obtain training text entity pairs. The first and last positions of the candidate entity pairs corresponding to the target extraction weighted summation vectors are used as the first and last positions of the entity pairs of the text to be extracted to obtain text entity pairs.

5. The relation extraction method based on model training as described in claim 1, characterized in that, The calculation of the similarity between the target entity concatenation vector and the preset sample representation vector using cosine similarity includes: The similarity between the concatenated vector of the target entity and the preset sample representation vector is calculated using the following formula. : in, This represents the concatenated vector of the target entity. This represents the preset sample representation vector. This indicates the size of the target entity concatenation vector. This indicates the size of the preset sample representation vector.

6. The relation extraction method based on model training as described in claim 1, characterized in that, Before calculating the loss value of the entity pair concatenation vector, the method includes: Obtain a preset number of entity pair concatenation vectors; Cluster the entity pair concatenation vectors to obtain entity pair concatenation vectors with the same relationship and entity pair concatenation vectors with different relationships.

7. A relation extraction device based on model training, used to implement the relation extraction method based on model training as described in any one of claims 1 to 6, characterized in that, The device includes: The entity pair recognition module is used to acquire training text and text to be extracted, respectively identify the training text entity pairs and the text to be extracted in the training text and the text to be extracted, and insert preset characters before and after the training text entity pairs to obtain the target training text, and insert preset characters before and after the text to be extracted entity pairs to obtain the target text to be extracted. The model training module is used to encode the training text entity pairs in the target training text using a preset semantic representation model, to obtain the first entity encoding vector and the second entity encoding vector corresponding to the training text entity pairs in the target training text, to concatenate the first entity encoding vector and the second entity encoding vector to obtain the entity pair concatenation vector, to calculate the loss value of the entity pair concatenation vector, to adjust the parameters of the semantic representation model according to the loss value, and to return to the step of encoding the training text entity pairs in the target training text using the preset semantic representation model to obtain the first entity encoding vector and the second entity encoding vector corresponding to the training text entity pairs in the target training text, until the loss value meets the preset loss threshold, and the trained semantic representation model is obtained. The entity relation extraction module is used to extract the target entity concatenation vector of the target text to be extracted using the trained semantic representation model, calculate the similarity between the target entity concatenation vector and the preset sample representation vector using cosine similarity, and take the entity relation of the preset sample representation vector with the highest similarity as the entity relation of the text to be extracted.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores computer program instructions that can be executed by the at least one processor to enable the at least one processor to perform the model-trained relation extraction method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the relation extraction method based on model training as described in any one of claims 1 to 6.

Images