A method for entity recognition and relation extraction

Abstract

The present invention discloses an entity recognition and relationship extraction method in the technical field of natural language processing. The method firstly captures rich contextual information of entities in texts through a suspended tag strategy in entity recognition and relationship extraction models, including context and semantic information around the entities, so as to provide richer features for entity recognition and relationship extraction, thereby improving the accuracy of entity recognition and relationship extraction. Then, the connection between spans and suspended tags and between suspended tags is fully utilized through a suspended tag matrix, so as to provide complete data support for downstream classification tasks, thereby improving the accuracy of the model.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to an entity recognition and relationship extraction method. Background Art

[0002] With the development of science and technology and the improvement of living standards, a large amount of text data continues to emerge, leaving people with massive and complex information. How to mine valuable information from this data has become an important challenge facing today's society. Named entity recognition and relationship extraction technology can help automatically extract entities and relationships between entities from unstructured text data, and help convert a large amount of text information into structured knowledge. This structured knowledge has a wide range of important applications in many fields such as information extraction, intelligent search, semantic understanding, risk management, and medical health, providing strong support for the realization of intelligent text understanding and knowledge mining.

[0003] The pre-trained model BERT (Bidirectional Encoder Representations from Transformers) learns deep language representations of text data through large-scale unsupervised pre-training, which enables it to achieve remarkable results in various downstream tasks. This has also attracted researchers to widely apply BERT to entity recognition and relationship extraction tasks. In recent years, suspended tags are used to obtain the span information on both sides of the entity to construct the input text sequence, and the semantic information of the entity is obtained through the pre-trained model BERT. Finally, the suspended tags and entity information are connected in series for entity recognition and relationship extraction. This method has achieved good evaluation index scores in the task of relationship extraction, but ignores the connectivity between suspended tags. Summary of the invention

[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and to provide an entity recognition and relationship extraction method. The entity recognition and relationship extraction method obtains entity boundary information through floating tags, makes full use of the connectivity between floating tags, and performs overall modeling of text data and floating tags in two tasks by generating matrices in data processing strategy. This can improve the problem of ignoring the connectivity of floating tags in the prior art, thereby ensuring both the accuracy of entity recognition results and the consistency of relationship extraction.

[0005] To achieve the above object, the present invention is implemented by adopting the following technical solutions:

[0006] In a first aspect, the present invention provides an entity recognition and relationship extraction method, comprising:

[0007] Acquire text data to be processed, and segment the text data to be processed into sentences to obtain segmented text data;

[0008] The segmented text data is input into the pre-trained entity recognition model to obtain entity position triples ( B,e,E ) , where B represents the starting position of the entity, e represents the type of the entity, and E represents the ending position of the entity;

[0009] The segmented text data and entity position triples ( B,e,E ) Input the pre-trained relation extraction model to obtain entity relation triples ( S,r,O ) , where S represents the subject, r represents the relationship type, and O represents the object.

[0010] Further, the entity recognition model includes an entity recognition input sequence generator, an entity recognition encoder, an entity recognition suspended tag matrix generator, and an entity recognition decoder;

[0011] The segmented text data is input into a pre-trained entity recognition model to obtain entity position triples ( B,e,E ) ,include:

[0012] Based on the segmented text data, use the entity recognition input sequence generator to construct an entity recognition input sequence with suspended tags;

[0013] According to the entity recognition input sequence, using an entity recognition encoder to generate an entity recognition semantic feature sequence including context information;

[0014] According to the entity recognition semantic feature sequence including context information, using an entity recognition suspended mark matrix generator to generate a feature-fused entity recognition three-dimensional matrix;

[0015] According to the entity recognition three-dimensional matrix of the feature fusion, an entity recognition decoder is used to generate entity triples (B, e, E) in the text data.

[0016] Furthermore, the entity recognition input sequence generator is used to construct an entity recognition input sequence with a suspended mark according to the segmented text data, including:

[0017] Set the maximum length N of the sentence text data sequence, the maximum span length L of the entity, and the upper limit K of the number of each subset of the entity suspension mark set;

[0018] The text data sequence is represented as X = {x1, x2, ..., xN}, where x1,x2,…,x N Corresponding to each word in the text data in turn; the candidate entity span pair set is represented as S(X) = {(x1,x1),…,(x1,x L ),…,(x N ,x N )}, then the entity suspension tag set corresponding to S(X) can be expressed as T(O) = {([O1],[ / O1]),…,([O1],[ / O L ]),…,([O N ],[ / O N ])}, where [O i ], i∈(1,K) indicates the floating mark at the beginning of the entity span, [ / O i ], i∈(1,K) represents the suspension mark at the end of the entity span;

[0019] Divide T(O) into multiple subsets, the number of each subset does not exceed K, and then construct the text data sequence X and the divided subsets into an entity recognition input sequence with suspended tags The specific construction method is as follows:

[0020]

[0021] in, Input sequence for entity recognition with hover markers.

[0022] Further, the entity recognition encoder includes a pre-trained model BERT;

[0023] The BERT adopts a 12-layer stacked network structure, each of which includes a self-attention mechanism unit, a residual layer normalization unit, a feedforward network unit, and a residual layer normalization unit;

[0024] and / or,

[0025] The entity recognition suspended label matrix generator includes a full connection layer, a Dropout random inactivation layer and a ReLU activation layer;

[0026] The entity recognition matrix M generated by the entity recognition suspended tag matrix generator is expressed as:

[0027]

[0028] Where E is the representation of all entity types; (·) is the ReLU activation function; drop(·) is the drop strategy; W is a trainable weight parameter, g is a trainable bias parameter; [;] is a concatenation operation; S is the entity feature; O is the feature of the suspended marker;

[0029] and / or,

[0030] The entity recognition decoder includes a full connection layer and a Softmax normalization layer;

[0031] The expression of the Softmax layer is as follows:

[0032]

[0033] In the formula, y i represents the probability value at each position of the matrix, O i represents the generated value at each position of the matrix, ∑ k exp(O k ) represents the sum of the generated values ​​at all positions after passing through the exponential function exp(·).

[0034] Furthermore, the relation extraction model includes a relation extraction input sequence generator, a relation extraction encoder, a relation extraction suspension tag matrix generator and a relation extraction decoder;

[0035] The segmented text data and entity position triples (B, e, E) are input into a pre-trained relation extraction model to obtain entity relation triples (S, r, O), including:

[0036] According to the segmented text data and the entity position triple (B, e, E), a relation extraction input sequence generator is used to generate a relation extraction input sequence with suspended marks;

[0037] According to the relation extraction input sequence, using a relation extraction encoder to generate a relation extraction semantic feature sequence including context information;

[0038] According to the relation extraction semantic feature sequence including context information, a relation extraction suspended label matrix generator is used to generate a feature-fused relation extraction three-dimensional matrix;

[0039] According to the relation extraction three-dimensional matrix of the feature fusion, a relation extraction decoder generator is used to generate relation triplets (S, r, O( in the text data.

[0040] Furthermore, according to the segmented text data and the entity position triple (B, e, E), a relation extraction input sequence generator is used to generate a relation extraction input sequence with a suspension mark, wherein the relation extraction input sequence generator adopts a subject-oriented suspension mark integration strategy to construct an input sequence, including:

[0041] According to the entity position triple (B, e, E) and the segmented text data, an entity is selected as the subject span, denoted as S i=(a,b),i∈(1,m), where m represents the number of entities extracted by the entity extraction model, and then other entities are taken as object spans, represented as (c1,d1),(c2,d2),…,(c m ,d m ), the corresponding suspended mark span is expressed as ([Obj1],[ / Obj1]),([Obj2],[ / Obj2]),…,([Obj m ],[ / Obj m ]);

[0042] Insert the subject start suspension marker [S] and the subject end suspension marker [ / S] before and after the subject span, respectively, to obtain the subject after the suspension markers are inserted;

[0043] The suspended marker [Obj i ] and the suspended marker at the end of the object [ / Obj i ] is applied to each object span to obtain the candidate object after the suspended marker is inserted;

[0044] The subject after the suspended marker is inserted and the candidate object after the suspended marker is inserted are integrated into one instance for modeling. Represents the sequence after modeling. The specific modeling form is as follows:

[0045]

[0046] In the formula, Indicates the constructed relation extraction input sequence, [S] indicates the suspended marker at the beginning of the subject, [ / S] indicates the suspended marker at the end of the subject, [Obj i ], i∈(1,m) indicates the suspended marker of the beginning of the object, [ / Obj i ], i∈(1,m) represents the suspended marker at the end of the object.

[0047] Further, the relation extraction encoder includes a pre-trained model BERT;

[0048] The BERT adopts a 12-layer stacked network structure, each of which includes a self-attention mechanism unit, a residual layer normalization unit, a feedforward network unit, and a residual layer normalization unit;

[0049] and / or,

[0050] The relation extraction suspended label matrix generator comprises a full connection layer, a Dropout layer and a ReLU activation layer;

[0051] The expression of the relation extraction matrix Y generated by the relation extraction suspension mark matrix generator is as follows:

[0052]

[0053] Where F is the representation of all relation types; (·) is the ReLU activation function; drop(·) is the drop strategy; W is a trainable weight parameter; g is a trainable bias parameter; [;] is a concatenation operation; Sub represents the subject feature; Obj represents the object feature;

[0054] and / or,

[0055] The relation extraction decoder includes a full connection layer and a Softmax normalization layer.

[0056] Furthermore, the training method of the entity recognition model includes:

[0057] Obtaining a training set, wherein the training set includes original text data, a corresponding entity start position, a corresponding entity end position, and an entity type;

[0058] The original text data is divided into sentences, and then the divided text data is input into the pre-built entity recognition model to extract entity position triples;

[0059] According to the extracted entity position triples and the formal entity position triples of the corresponding text data, the loss function of the entity recognition model is calculated, and the expression is as follows:

[0060]

[0061] in, represents the loss of the entity recognition model; represents the entity position triple extracted by the entity recognition model; y represents the formal entity position triple; Q represents the length of the segmented text data; u represents the number of labels of the entity type corresponding to the text data; p represents the probability that the predicted value is equal to the true value;

[0062] Iteratively update the entity recognition model based on the gradient descent method, and use the model with the minimum loss function of the entity recognition model as the trained entity recognition model to obtain the trained entity recognition model;

[0063] and / or,

[0064] The training method of the relationship extraction model includes:

[0065] Obtain a training set, the training set comprising original text data and entity position triples output by the pre-trained entity recognition model ( B,e,E ) ;

[0066] The training set is divided into sentences, and then the divided text data is input into a pre-built relation extraction model to extract entity relation triples;

[0067] According to the extracted entity relationship triples and the formal entity relationship triples of the corresponding text data, the loss function of the relationship extraction model is calculated, and the expression is as follows:

[0068]

[0069] in, represents the loss of the relation extraction model; Q represents the length of the segmented text data; v represents the number of labels of the relation type corresponding to the text data; p represents the probability that the predicted value is equal to the true value; represents the entity relationship triple extracted by the relation extraction model, and y represents the formal entity relationship triple;

[0070] The relationship extraction model is iteratively updated and trained based on the gradient descent method, and the model with the smallest loss function of the relationship extraction model is used as the trained entity recognition model to obtain the trained relationship extraction model.

[0071] In a second aspect, the present invention provides a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the steps of the entity recognition and relationship extraction method.

[0072] In a third aspect, the present invention provides a computer device / equipment / system, comprising:

[0073] Memory, for storing computer programs / instructions;

[0074] A processor is used to execute the computer program / instructions to implement the steps of the entity recognition and relationship extraction method.

[0075] Compared with the prior art, the present invention has the following beneficial effects:

[0076] The entity recognition and relationship extraction method provided by the present invention first captures rich contextual information of entities in text through a suspended tag strategy in the entity recognition and relationship extraction model, including the context and semantic information around the entity, which can provide richer features for entity recognition and relationship extraction, thereby improving the accuracy of entity recognition and relationship extraction. Then, the connection between spans and suspended tags and between suspended tags is fully utilized through a suspended tag matrix to provide complete data support for downstream classification tasks, thereby improving the accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1It is a flowchart of an entity recognition and relationship extraction method based on a suspended tag matrix provided by an embodiment of the present invention;

[0078] Figure 2 It is a structural diagram of an entity recognition and relationship extraction model provided by an embodiment of the present invention;

[0079] Figure 3 is a schematic diagram of a neighbor-oriented floating mark integration strategy provided by an embodiment of the present invention;

[0080] Figure 4 It is a schematic diagram of the data processing flow of the pre-training model BERT provided by an embodiment of the present invention;

[0081] Figure 5 is a schematic diagram of a subject-oriented suspension mark integration strategy provided by an embodiment of the present invention;

[0082] Figure 6 It is a schematic diagram of a strategy for constructing a suspended marker matrix for entity recognition provided by an embodiment of the present invention;

[0083] Figure 7 It is a schematic diagram of a strategy for constructing a suspended tag matrix for relationship extraction provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0084] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0085] Example 1

[0086] This embodiment provides an entity recognition and relationship extraction method, such as Figure 1 and Figure 2 As shown, the method includes:

[0087] Obtaining text data to be processed, and segmenting the text data into sentences;

[0088] The divided text data is input into the trained entity recognition model to obtain ( B,e,E ) The text data and entity data obtained by the entity recognition model are then input into the trained relation extraction model to obtain ( S,r,O ) Triples;

[0089] Among them, the entity recognition model includes:

[0090] The entity recognition input sequence generator adopts a neighbor-oriented suspended tag integration strategy, which takes advantage of the parallelism of suspended tags, packs a series of related spans into a training instance, and obtains the boundary information of entities in text data through tags;

[0091] The entity recognition encoder is used to input the constructed input sequence into the pre-trained BERT model to obtain semantic features including context information;

[0092] An entity recognition suspended mark matrix generator is used to generate a feature-fused entity recognition three-dimensional matrix based on the semantic features including context information;

[0093] An entity recognition decoder is used to generate entity triplets in text data based on the entity recognition three-dimensional matrix fused with the features ( B,e,E ) .

[0094] The relation extraction model includes:

[0095] The relation extraction input sequence generator adopts a subject-oriented suspension tag integration strategy, uses the data obtained by the entity recognition model and the original text data, and performs separate modeling based on each subject. It uses the flexibility of suspension tags to obtain the boundary information of the subject and object in the text data through tags;

[0096] A relation extraction encoder, configured to generate semantic features including context information based on the relation extraction input sequence;

[0097] A relation extraction suspended label matrix generator, used to generate a feature-fused relation extraction three-dimensional matrix according to the semantic features including context information;

[0098] A relation extraction decoder is used to generate relation triplets in text data based on the relation extraction three-dimensional matrix of the feature fusion ( S,r,O ) .

[0099] The technical concept of the present invention is as follows: based on an entity recognition input sequence generator, entities in a text are identified and their boundaries are accurately located; based on an entity recognition encoder, semantic features including contextual information are obtained; based on an entity recognition suspended mark matrix generator, features that fuse entity boundaries and entity types are obtained, and then entity recognition results are obtained through an entity recognition decoder; based on a relationship extraction input sequence generator, subjects and objects in a text are identified and their boundaries are accurately located; based on a relationship extraction encoder, semantic features of text data and suspended marks are obtained; a relationship extraction suspended mark matrix generator is used to obtain fused features of subjects, suspended marks, and relationship types, and finally, relationship extraction results are obtained through a relationship extraction decoder.

[0100] like Figure 3 As shown in Figure 1, the neighbor-oriented suspended token integration strategy in the entity recognition input sequence generator first sorts the suspended token pairs by taking the position of the start token as the first keyword and the position of the end token as the second keyword. After that, they are divided into groups of size not exceeding K, so that adjacent spans are clustered into the same group.

[0101] like Figure 4 As shown, the pre-trained model BERT extracts semantic features from the constructed sequence.

[0102] like Figure 5 As shown in the figure, the formal entity matrix (ESE) of text data under a single entity type. The design idea of ​​ESE is to first construct a two-dimensional matrix based on each entity type separately and mark the data in the matrix. The marking strategy in entity recognition uses the following four types of marks: (1) ES-OS, the two positions referred to by this label are the entity start position and the floating mark start position under a specific entity type. (2) ES-OE, the two positions referred to by this label are the entity end position and the floating mark start position under a specific entity type. (3) EE-OE, the two positions referred to by this label are the entity end position and the floating mark end position under a specific entity type. (4) “-”, all positions except the above three positions are marked as “-”. Finally, a three-dimensional matrix is ​​constructed based on each entity type.

[0103] like Figure 6 As shown in FIG. 1 , the subject-oriented suspension marker integration strategy in the relation extraction input sequence generator is based on separate modeling of each subject, and special markers are inserted before and after the subject and object respectively.

[0104] like Figure 7 As shown in the figure, the formal relation matrix (RSE) of text data under a single relation type. The design idea of ​​RSE is to first construct a two-dimensional matrix based on each relation type and mark the data in the matrix. The marking strategy in relation extraction uses the following four types of marks: (1) SS-OS, the two positions referred to by this label are the start position of the subject and the start position of the floating mark under a certain relation type. (2) SS-OE, the two positions referred to by this label are the end position of the subject and the start position of the floating mark under a certain relation type. (3) SE-OE, the two positions referred to by this label are the end position of the subject and the end position of the floating mark under a certain relation type. (4) “-”, all positions except the above three positions are marked as “-”. Finally, they are integrated into a three-dimensional matrix according to different relation types;

[0105] The training method of the entity recognition and relationship extraction model in this embodiment includes the following steps:

[0106] 1. Obtaining the training set includes the following steps:

[0107] From the public relation extraction dataset, e.g.

[0108] AutomaticContentExtraction2004(ACE2004),

[0109] Automatic Content Extraction 2005 (ACE2005), obtain the training set.

[0110] Second, input the text data in the training set into the entity recognition input sequence generator to generate a new sequence including the suspended mark, as follows:

[0111] Set the maximum length N of the sentence text data sequence, the maximum span length L of the entity, and the upper limit K of the number of each subset of the entity suspension mark set;

[0112] The text data sequence is represented as X = {x1, x2, ..., x N}, where x1,x2,…,x N Corresponding to each word in the text data in turn; the candidate entity span pair set is represented as S(X) = {(x1,x1),…,(x1,x L ),…,(x N ,x N )}, then the entity suspension tag set corresponding to S(X) can be expressed as T(O) = {([O1],[ / O1]),…,([O1],[ / O L ]),…,([O N ],[ / O N ])}, where [O i ], i∈(1,K) indicates the floating mark at the beginning of the entity span, [ / O i ], i∈(1,K) represents the suspension mark at the end of the entity span;

[0113] Divide T(O) into multiple subsets, the number of each subset does not exceed K, and then construct the text data sequence X and the divided subsets into an entity recognition input sequence with suspended tags The specific construction method is as follows:

[0114]

[0115] in, Input sequence for entity recognition with hover markers.

[0116] 3. Input the obtained new sequence into the entity recognition encoder, which is composed of the pre-trained model BERT, so that the suspended marker can obtain the boundary features of the entity.

[0117] 4. Input the data feature sequence into the entity recognition suspended mark matrix generator to construct a three-dimensional matrix M N×m×N Get the score of each position in the matrix. The specific formula is as follows:

[0118]

[0119] Where E is the representation of all entity types; (·) is the ReLU activation function; drop(·) is the drop strategy; W is a trainable weight parameter, g is a trainable bias parameter; [;] is a concatenation operation; S is the entity feature; O is the feature of the suspended marker;

[0120] 5. Input the three-dimensional matrix M into the entity recognition decoder, first pass it through a fully connected layer to obtain a score vector v based on the entity start and end positions of each entity type, and then input the obtained score vector into the softmax function to obtain the final evaluation value:

[0121]

[0122] When training the entity recognition model, the cross entropy method is used to calculate the loss of entity recognition and update the model parameters. The specific calculation formula is as follows:

[0123]

[0124] Among them, y is the real entity label in the ESE matrix, θ represents the training parameters of the model, and the training is carried out by minimizing To update the model parameters, finally through the minimum The model obtains entity recognition results.

[0125] 6. After the entity recognition model training is completed, the training set and the corresponding entity recognition model results are input into the relation extraction input sequence generator. The generator selects an entity from the entity recognition results as the subject span S i =(a,b),i∈(1,m), where m represents the number of entities extracted by the entity extraction model, and then other entities are taken as object spans, represented as (c1,d1),(c2,d2),…,(c m ,d m ), the corresponding suspended mark span is expressed as ([Obj1],[ / Obj1]),([Obj2],[ / Obj2]),…,(Obj m ],[ / Obj m]); insert the subject start suspension mark [S] and the subject end suspension mark [ / S] before and after the subject span respectively, and obtain the subject after the suspension mark is inserted;

[0126] The suspended marker [Obj i ] and the suspended marker at the end of the object [ / Obj i ] is applied to each object span to obtain the candidate object after the suspended marker is inserted;

[0127] The subject after the suspended marker is inserted and the candidate object after the suspended marker is inserted are integrated into one instance for modeling. Represents the sequence after modeling. The specific modeling form is as follows:

[0128]

[0129] 7. Input the modeled new sequence into the relation extraction encoder to obtain the data feature sequence, and let the suspended marker obtain the boundary features of the subject and object.

[0130] 8. Input the data feature sequence into the relation extraction suspended mark matrix generator to construct a three-dimensional matrix Y N×l×N , and then input the labeled model into the classifier to get the score of the matrix Y. The specific formula is as follows:

[0131]

[0132] Where F is the representation of all relation types; (·) is the ReLU activation function; drop(·) is the drop strategy; W is a trainable weight parameter; g is a trainable bias parameter; [;] is a concatenation operation; Sub represents the subject feature; Obj represents the object feature;

[0133] 10. Input the three-dimensional matrix Y into the relation extraction decoder, first pass through a fully connected layer to obtain a score vector v based on the entity start and end positions of each entity type, and then input the obtained score vector into the softmax function to obtain the final evaluation value:

[0134]

[0135] When training the relationship extraction model, the cross entropy method is used to calculate the loss of relationship extraction and update the model parameters. The specific calculation formula is as follows:

[0136]

[0137] Among them, y is the real relationship label in the RSE matrix, θ represents the training parameters of the model, and the training is carried out by minimizing to update the model parameters.

[0138] Example 2

[0139] Based on the same inventive concept as Example 1, this embodiment provides a computer-readable storage medium on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the steps of the entity recognition and relationship extraction method described in Example 1 are implemented.

[0140] Example 3

[0141] Based on the same inventive concept as Embodiment 1, this embodiment provides a computer device / equipment / system, including:

[0142] Memory, for storing computer programs / instructions;

[0143] A processor is used to execute the computer program / instructions to implement the steps of the entity recognition and relationship extraction method as described in Example 1.

[0144] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0145] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0146] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0147] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0148] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

Claims

1. A method for entity recognition and relationship extraction, characterized in that: include: Acquire text data to be processed, and segment the text data to be processed into sentences to obtain segmented text data; Input the segmented text data into a pre-trained entity recognition model to obtain an entity position triple (B, e, E), where B represents the starting position of the entity, e represents the type of the entity, and E represents the ending position of the entity; Input the segmented text data and entity position triples (B, e, E) into a pre-trained relation extraction model to obtain entity relation triples (S, r, O), where S represents the subject, r represents the relation type, and O represents the object; The entity recognition model includes an entity recognition input sequence generator, an entity recognition encoder, an entity recognition suspended mark matrix generator and an entity recognition decoder; The segmented text data is input into a pre-trained entity recognition model to obtain an entity position triple (B, e, E), including: Based on the segmented text data, use the entity recognition input sequence generator to construct an entity recognition input sequence with suspended tags; According to the entity recognition input sequence, using an entity recognition encoder to generate an entity recognition semantic feature sequence including context information; According to the entity recognition semantic feature sequence including context information, using an entity recognition suspended mark matrix generator to generate a feature-fused entity recognition three-dimensional matrix; According to the entity recognition three-dimensional matrix of the feature fusion, an entity recognition decoder is used to generate entity triples (B, e, E) in the text data; The entity recognition input sequence generator is used to construct an entity recognition input sequence with a suspended mark according to the segmented text data, including: Set the maximum length N of the sentence text data sequence, the maximum span length L of the entity, and the upper limit K of the number of each subset of the entity suspension mark set; The text data sequence is represented as X = {x1, x2, ..., x N }, where x1,x2,…,x N Corresponding to each word in the text data in turn; the candidate entity span pair set is represented as S(X) = {(x1,x1),…,(x1,x L ),…,(x N ,x N )}, then the entity suspension tag set corresponding to S(X) can be expressed as T(O) = {([O1],[ / O1]),…,([O1],[ / O L ]),…,([O N ],[ / O N ])}, where [O i ], i∈(1,K) indicates the floating mark at the beginning of the entity span, [ / O i ], i∈(1,K) represents the suspension mark at the end of the entity span; Divide T(O) into multiple subsets, the number of each subset does not exceed K, and then construct the text data sequence X and the divided subsets into an entity recognition input sequence with suspended tags The specific construction method is as follows: in, Input sequence for entity recognition with hover markers.

2. The entity recognition and relationship extraction method according to claim 1, characterized in that: The entity recognition encoder includes a pre-trained model BERT; The BERT adopts a 12-layer stacked network structure, each of which includes a self-attention mechanism unit, a residual layer normalization unit, a feedforward network unit, and a residual layer normalization unit; and / or, The entity recognition suspended label matrix generator includes a full connection layer, a Dropout random inactivation layer and a ReLU activation layer; The entity recognition matrix M generated by the entity recognition suspended tag matrix generator is expressed as: Where E is the representation of all entity types; (·) is the ReLU activation function; drop(·) is the drop strategy; W is a trainable weight parameter; g is a trainable bias parameter; [;] is a concatenation operation; S is the entity feature; O is the feature of the suspended tag; and / or, The entity recognition decoder includes a full connection layer and a Softmax normalization layer; The expression of the Softmax layer is as follows: In the formula, y i represents the probability value at each position of the matrix, O i represents the generated value at each position of the matrix, ∑ k exp(O k ) represents the sum of the generated values ​​at all positions after passing through the exponential function exp(·).

3. The entity recognition and relationship extraction method according to claim 1, characterized in that: The relation extraction model includes a relation extraction input sequence generator, a relation extraction encoder, a relation extraction suspension tag matrix generator and a relation extraction decoder; The segmented text data and entity position triples (B, e, E) are input into a pre-trained relation extraction model to obtain entity relation triples (S, r, O), including: According to the segmented text data and the entity position triple (B, e, E), a relation extraction input sequence generator is used to generate a relation extraction input sequence with suspended marks; According to the relation extraction input sequence, using a relation extraction encoder to generate a relation extraction semantic feature sequence including context information; According to the relation extraction semantic feature sequence including context information, a relation extraction suspended label matrix generator is used to generate a feature-fused relation extraction three-dimensional matrix; According to the relation extraction three-dimensional matrix of the feature fusion, a relation extraction decoder generator is used to generate relation triplets (S, r, O) in the text data.

4. The entity recognition and relationship extraction method according to claim 3, characterized in that: According to the segmented text data and the entity position triple (B, e, E), a relation extraction input sequence generator is used to generate a relation extraction input sequence with a suspension mark, wherein the relation extraction input sequence generator adopts a subject-oriented suspension mark integration strategy to construct an input sequence, including: According to the entity position triple (B, e, E) and the segmented text data, an entity is selected as the subject span, denoted as S i =(a,b), i∈(1,m), m represents the number of entities mentioned in the entity extraction model, and then other entities are taken as object spans, represented as (c1,d1),(c2,d2),…,(c m ,d m ), the corresponding suspended mark span is expressed as ([Obj1],[ / Obj1]),([Obj2],[ / Obj2]),…,([Obj m ],[ / Obj m ]); Insert the subject start suspension marker [S] and the subject end suspension marker [ / S] before and after the subject span, respectively, to obtain the subject after the suspension markers are inserted; The suspended marker [Obj i ] and the suspended marker at the end of the object [ / Obj i ] is applied to each object span to obtain the candidate object after the suspended marker is inserted; The subject after the suspended marker is inserted and the candidate object after the suspended marker is inserted are integrated into one instance for modeling. Represents the sequence after modeling. The specific modeling form is as follows: In the formula, Indicates the constructed relation extraction input sequence, [S] indicates the suspended marker at the beginning of the subject, [ / S] indicates the suspended marker at the end of the subject, [Obj i ], i∈(1,m) indicates the suspended marker of the beginning of the object, [ / Obj i ], i∈(1,m) represents the suspended marker at the end of the object.

5. The entity recognition and relationship extraction method according to claim 3, characterized in that: The relation extraction encoder includes a pre-trained model BERT; The BERT adopts a 12-layer stacked network structure, each of which includes a self-attention mechanism unit, a residual layer normalization unit, a feedforward network unit, and a residual layer normalization unit; and / or, The relation extraction suspended label matrix generator comprises a full connection layer, a Dropout layer and a ReLU activation layer; The expression of the relation extraction matrix Y generated by the relation extraction suspension mark matrix generator is as follows: Where F is the representation of all relation types; (·) is the ReLU activation function; drop(·) is the drop strategy; W is a trainable weight parameter; g is a trainable bias parameter; [;] is a concatenation operation; Sub represents the subject feature; Obj indicates object features; and / or, The relation extraction decoder includes a full connection layer and a Softmax normalization layer.

6. The entity recognition and relationship extraction method according to claim 1, characterized in that: The training method of the entity recognition model includes: Obtaining a training set, wherein the training set includes original text data, a corresponding entity start position, a corresponding entity end position, and an entity type; The original text data is divided into sentences, and then the divided text data is input into the pre-built entity recognition model to extract entity position triples; According to the extracted entity position triples and the formal entity position triples of the corresponding text data, the loss function of the entity recognition model is calculated, and the expression is as follows: in, represents the loss of the entity recognition model; represents the entity position triple extracted by the entity recognition model; y represents the formal entity position triple; Q represents the length of the segmented text data; u represents the number of labels of the entity type corresponding to the text data; p represents the probability that the predicted value is equal to the true value; Iteratively update the entity recognition model based on the gradient descent method, and use the model with the minimum loss function of the entity recognition model as the trained entity recognition model to obtain the trained entity recognition model; and / or, The training method of the relationship extraction model includes: Obtain a training set, wherein the training set includes original text data and entity position triples (B, e, E) output by the pre-trained entity recognition model; The training set is divided into sentences, and then the divided text data is input into a pre-built relation extraction model to extract entity relation triples; According to the extracted entity relationship triples and the formal entity relationship triples of the corresponding text data, the loss function of the relationship extraction model is calculated, and the expression is as follows: in, represents the loss of the relation extraction model; Q represents the length of the segmented text data; v represents the number of labels of the relation type corresponding to the text data; p represents the probability that the predicted value is equal to the true value; represents the entity relationship triple extracted by the relation extraction model, and y represents the formal entity relationship triple; The relationship extraction model is iteratively updated and trained based on the gradient descent method, and the model with the smallest loss function of the relationship extraction model is used as the trained entity recognition model to obtain the trained relationship extraction model.

7. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instruction is executed by a processor, the steps of the entity recognition and relationship extraction method described in any one of claims 1-6 are implemented.

8. A computer device / equipment / system, characterized in that: include: Memory, for storing computer programs / instructions; A processor, configured to execute the computer program / instructions to implement the steps of the entity recognition and relationship extraction method according to any one of claims 1 to 6.

Images