Extensible text information joint extraction method and device based on prompt learning

Abstract

The application discloses an extensible text information joint extraction method and device based on prompt learning, belongs to the technical field of natural language processing text information extraction, and comprises the following steps: constructing a unified representation framework of text information extraction annotation information; constructing a unified prompt template based on an original data set and an extended data set; constructing and training a text information joint extraction model based on prompt learning; and performing joint information extraction on input text by using the trained model. The application solves the technical bottleneck caused by data expansion or cross-domain, and improves the accuracy and robustness of joint information extraction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing Chinese text information extraction, more specifically, it relates to a scalable text information joint extraction method and device based on prompt learning. BACKGROUND

[0002] In recent years, with the popularity of Internet and mobile terminal information dissemination software, extracting necessary text information from massive text data can better manage text resources, and can assist in downstream services such as information recommendation, knowledge graph construction, and text material collection. However, the current text information extraction method has various problems, among which the more prominent ones are:

[0003] (1) cannot jointly extract different types of text information (such as entities, relationships, events, etc.);

[0004] (2) when facing data with different text information structure definitions, especially when cross-domain data or new text information structures appear, unified information extraction cannot be performed.

[0005] The above two problems greatly limit the performance of information extraction in actual application scenarios. SUMMARY

[0006] The purpose of the present application is to overcome the shortcomings of the prior art, provide a scalable text information joint extraction method and device based on prompt learning, solve the technical bottleneck brought by data expansion or cross-domain, and improve the accuracy and robustness of joint information extraction.

[0007] The purpose of the present application is achieved by the following scheme:

[0008] A scalable text information joint extraction method based on prompt learning, comprising the following steps:

[0009] A, constructing a unified representation framework of text information extraction annotation information;

[0010] B, constructing a unified prompt template based on the original data set and the expanded data set;

[0011] C, constructing and training a text information joint extraction model based on prompt learning;

[0012] D, using the trained model to perform joint information extraction on the input text.

[0013] Further, in step A, the construction of the unified representation framework of text information extraction annotation information includes the following sub-steps:

[0014] A1. For the named entity recognition task, the annotation information representation framework is constructed as: (Entity_type:span_e), where Entity_type represents the name of a selected entity type, and span_e represents the text fragment in the text that belongs to the entity type Entity_type;

[0015] A2. For the entity relation extraction task, the annotation information representation framework is constructed as: (Entity_type:span_e:(Relation_type_i:span_r_i)), where Relation_type_i represents the name of the i-th entity relation type associated with the entity type Entity_type, and span_r_i represents the tail entity text fragment corresponding to span_e under Relation_type_i;

[0016] A3. For the event extraction task, the annotation information representation framework is constructed as: (Event_type:span_v:(Role_type_i:span_o_i)), where Event_type represents the name of a selected event type, span_v represents the text fragment of the event trigger word belonging to the Event_type event type, Role_type_i represents the name of the i-th event role associated with the Event_type event, and span_o_i represents the text fragment of the event role corresponding to span_v under Role_type_i;

[0017] A4. Integrate named entity recognition, entity relationship extraction, and event extraction annotation information representation framework to form a unified representation framework for text information extraction and annotation information.

[0018] Furthermore, in step B, the construction of a unified prompt template based on the original dataset and the extended dataset includes the following sub-steps:

[0019] B1. Define the original dataset as D. ori ={(S i L i ), i = 1, ..., N ori}, where S i L represents the text of the i-th data item in the original dataset. i N represents the annotation information of the i-th data in the original dataset. ori This indicates the number of original data samples;

[0020] B2. Define the extended dataset as D. ext ={(S j L j ), j = 1, ..., N ext}, where Sj denotes the text of the j-th data of the extended dataset, L j denotes the annotation information of the j-th data of the extended dataset, N ext denotes the number of extended data samples;

[0021] B3. Fuse the original dataset and the extended dataset, and construct a unified prompt template for each piece of annotation information.

[0022] Further, in step C, the constructing and training of the prompt learning text information joint extraction model includes the following sub-steps:

[0023] C1. Set an encoder based on the Transformer network architecture, and the output dimension is d E , denoted as E model ;

[0024] C2. Construct a fully connected layer network module for the starting position of the text segment, denoted as FC start , and construct a fully connected layer network module for the ending position of the text segment, denoted as FC end , the input dimensions of FC start and FC end are both d E , and the output dimensions are both 1;

[0025] C3. For a specified input triple (S k , P k , L k ), perform prompting and text splicing according to P k S k , and the length of the spliced text is denoted as d k ;

[0026] C4. Perform null character completion on the spliced text, i.e., add a meaningless null character sequence Q k after the spliced text, so that the character length of P k S k Q k reaches d * = max(d k , k = 1,..., K);

[0027] C5. Construct a PkS k Q k mask vector T k = (t1,..., t d* ), where,

[0028]

[0029] C6. Use E model to PkS kQ k Encoding is performed, and the encoded feature matrix is denoted as

[0030] C7. Based on L k , the start position and the end position in S k , a reference result vector GT k_start is constructed. wherein GT k_start is a vector of 0s except that L k is 1 at the start position in S k , and GT k_endt is a vector of 0s except that L k is 1 at the end position in S k .

[0031] C8. A maximum number of iterations EP is set, and a current number of completed iterations ep is recorded.

[0032] C9. A minimum information entropy threshold value ρ = exp[ep / (EP+1)] acceptable to the model at the current number of iterations is calculated. -1 ;

[0033] C10. F k is input into FC start and FC end respectively, and sigmoid is used for activation to obtain prediction vectors P start and P end respectively.

[0034] C11. The information entropy of P start and P end is calculated respectively, denoted as θ start and θ end .

[0035] C12. If θ start < ρ and θ end < ρ, cross-entropy loss is calculated based on P start and P end and GT k_start and GT k_end respectively, and the model is trained using back propagation; otherwise, the current data is skipped.

[0036] C13. Steps C3 to C12 are performed for all K groups of data, and after completing a round of training of the K groups of data, the value of ep is increased by 1.

[0037] C14. C13 is repeatedly performed until ep = EP, and a trained text information joint extraction model is denoted as Model.

[0038] Further, in step D, the joint information extraction on the input text using the trained model comprises the following sub-steps:

[0039] D1. Set an output threshold value δ ∈ (0, 1);

[0040] D2. For the input text Text, traverse all entity category names {Entity_type(q), q = 1,..., Q} contained in the trained data, Q being the number of all entity category names; concatenate Entity_type(q) and Text and input them into Model to obtain the named entity recognition extraction prediction vector under Entity_type(q), with P start >δ as the starting point, the nearest P end >δ after this position as the ending point, and the text segment between the two positions as the predicted entity text segment of Entity_type(q);

[0041] D3. For the input text Text and the entity text segment predicted in step D2, traverse the entity relationship type names {Relation_type(g), g = 1,..., G} associated with the entity type to which the predicted entity text segment belongs, G being the number of all entity relationship types associated with the entity type to which the predicted entity text segment belongs; concatenate 'Entity_span's Relation_type(g)' and Text and input them into Model to obtain the tail entity extraction prediction vector associated with Entity_span under Relation_type(g), with P dtart >δ as the starting point, the nearest P end >δ after this position as the ending point, and the text segment between the two positions as the predicted tail entity text segment of Entity_span under Relation_type(g);

[0042] D4. For the input text Text, traverse all event category names {Event_type(h), h = 1,..., H} contained in the trained data, H being the number of all event category names; concatenate Event_type(h) and Text and input them into Model to obtain the event trigger word extraction prediction vector under Event_type(h), with P dtart >δ as the starting point, the nearest P end >δ after this position as the ending point, and the text segment between the two positions as the predicted event trigger word text segment of Event_type(h);

[0043] D5. Traverse the event role name {Role_type(w), w = 1,..., W} associated with the event type to which the predicted event trigger text fragment Event_span belongs, W is the number of all event roles associated with the event type to which Event_span belongs; concatenate 'Event_span's Role_type(w)' and Text, and input it into Model to obtain the event role extraction prediction vector associated with Event_span under Role_type(w), and P dtart >δ is the starting point, and the position of the nearest P end >δ after this position is the end point, and the text fragment between the two positions is taken as the event role text fragment of Event_span under Role_type(w);

[0044] D6. The prediction results of steps D2 to D5 are fused according to the unified representation framework constructed in step A as the joint extraction results of the text information.

[0045] Further, in step B3, the following sub-steps are included:

[0046] B3-1. Construct the named entity recognition task prompt template as follows:

[0047] {

[0048] 'Text': S

[0049] 'Prompt': Entity_type

[0050] 'Result': span_e

[0051] }

[0052] Wherein, S is the text in the original data set or the expanded data set, Entity_type is the name of a selected entity type in the annotation information of the data set to which S belongs, and span_e is the entity text fragment of S belonging to Entity_type;

[0053] B3-2. Construct the entity relationship extraction task prompt template as follows:

[0054] {

[0055] 'Text': S

[0056] 'Prompt': Relation_type of span_e

[0057] 'Result': span_r

[0058] }

[0059] wherein S is the text in the original dataset or the extended dataset, span_e is the text segment in S as the head entity of a certain relation type, Relation_type is the name of a selected entity relation type in the annotation information of the dataset to which S belongs, and span_r is the tail entity text segment in S that has the selected relation Relation_type with span_e;

[0060] B3-3. The entity relation extraction task prompt template is constructed as follows:

[0061] {

[0062] ‘Text’: S

[0063] ‘Prompt’: Event_type

[0064] ‘Result’: span_v

[0065] }

[0066] {

[0067] ‘Text’: S

[0068] ‘Prompt’: Role_type of span_v

[0069] ‘Result’: span_o

[0070] }

[0071] wherein S is the text in the original dataset or the extended dataset, Event_type is the name of a specific event type in the annotation information of the dataset to which S belongs, span_v is the text segment in S as the trigger word of a certain event type, Role_type is the name of a specific event role in the annotation information of the dataset to which S belongs, and span_o is the text segment in S that is annotated as the Role_type of the Event_type event;

[0072] B3-4. All the annotation information in the fused data is reconstructed according to steps B3-1 to B3-3, and all entity type names, relation type names, event type names, and event role names are randomly sampled to construct negative sample data with the text segments in the annotation data, i.e., samples that cannot query the corresponding ‘results’; the dataset after fusion, reconstruction, and addition of negative samples is denoted as D mix = {(S k , P k , L k ), k = 1, …, K}, wherein S k represents the text, P k represents the prompt, and L kThe result is represented, K is the number of all three data sets of the data set.

[0073] Further, in step B3-1, if there is no entity text segment belonging to Entity_type type in S, span_e is an empty string.

[0074] Further, in step B3-2, if there is no tail entity text segment in S with a specific relationship Relation_type with span_e, span_r is an empty string.

[0075] Further, in step B3-3, the two'result' fields are empty strings when the corresponding text segments are not found.

[0076] An extensible text information joint extraction device based on prompt learning, based on any one of the methods described above, further comprising:

[0077] A first construction module for constructing a unified representation framework of text information extraction annotation information;

[0078] A second construction module for constructing a unified prompt template based on the original data set and the extended data set;

[0079] A third construction module for constructing and training a text information joint extraction model based on prompt learning;

[0080] An extraction module for using the trained model to perform joint information extraction on input text.

[0081] The beneficial effects of the present application include:

[0082] The present application is based on the theory of prompt learning, and a unified text information representation architecture is constructed. Different types of information extraction tasks can be integrated into the unified framework, and different definitions of data sets can be jointly trained to solve the technical bottleneck caused by data expansion or cross-domain. On this basis, the information entropy filtering method is used to avoid the negative impact of noise data on the model, and to a certain extent, the data overfitting is alleviated, and the accuracy and robustness of joint information extraction are improved. BRIEF DESCRIPTION OF DRAWINGS

[0083] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the embodiment or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0084] Figure 1A flowchart of an extensible text information joint extraction method based on prompt learning according to an embodiment of the present application. DETAILED DESCRIPTION

[0085] All features disclosed in the specification or in any document incorporated by reference can be combined and / or substituted for one another in any manner, unless the features are mutually exclusive.

[0086] As shown in Figure 1 The embodiment proposes an extensible text information joint extraction method based on prompt learning, which comprises the following steps:

[0087] A. Constructing a unified representation framework of text information extraction annotation information;

[0088] B. Constructing a unified prompt template based on the original data set and the extended data set;

[0089] C. Constructing and training a text information joint extraction model based on prompt learning;

[0090] D. Using the trained model to perform joint information extraction on the input text.

[0091] In some embodiments, step A of constructing a unified representation framework of text information extraction annotation information comprises the following sub-steps:

[0092] A1. For the named entity recognition task, the annotation information representation framework is constructed as: (Entity_type:span_e), wherein Entity_type represents a specific entity type name, and span_e represents a text segment in the text belonging to the Entity_type entity type;

[0093] A2. For the entity relationship extraction task, the annotation information representation framework is constructed as: (Entity_type:span_e:(Relation_type_i:span_r_i)), wherein Relation_type_i represents the i-th entity relationship type name associated with the Entity_type entity type, and span_r_i represents the tail entity text segment corresponding to span_e under Relation_type_i;

[0094] A3. For the event extraction task, the annotation information representation framework is constructed as: (Event_type:span_v:(Role_type_i:span_o_i)), wherein, Event_type represents a specific event type name, span_v represents a text segment of an event trigger belonging to the event type of Event_type in the text, Role_type_i represents the i-th event role name associated with the event of the type of Event_type, and span_o_i represents a text segment of the event role corresponding to span_v under Role_type_i.

[0095] A4. The annotation information representation frameworks of named entity recognition, entity relationship extraction and event extraction are fused to form a unified representation framework of text information extraction annotation information, and an example form is as follows:

[0096]

[0097]

[0098] In some embodiments, step B of constructing a unified prompt template based on the original data set and the extended data set comprises the following sub-steps:

[0099] B1. Define the original data set as D ori ={(S i ,L i ),i=1,...,N ori} wherein S i represents the text of the i-th data of the original data set, L i represents the annotation information (organized according to the unified representation framework of step A) of the i-th data of the original data set, and N ori represents the number of original data samples.

[0100] B2. Define the extended data set as D ext ={(S j ,L j ),j=1,...,N ext} wherein S j represents the text of the j-th data of the extended data set, L j represents the annotation information (organized according to the unified representation framework of step A) of the j-th data of the extended data set, and N ext represents the number of extended data samples.

[0101] B3. Fuse the original data set and the extended data set, and construct a unified prompt template for each piece of annotation information.

[0102] In some embodiments, step B3 comprises the following sub-steps:

[0103] B3-1. Construct the named entity recognition task prompt template as follows:

[0104] {

[0105] ‘Text’: S

[0106] ‘Prompt’: Entity_type

[0107] ‘Result’: span_e

[0108] }

[0109] where S is the text in the original dataset or the extended dataset, Entity_type is the name of a specific entity type in the annotation information of the dataset described by S, and span_e is the entity text segment of S belonging to Entity_type (in particular, if there is no entity text segment of S belonging to Entity_type, then span_e is an empty string);

[0110] B3-2. Construct the entity relationship extraction task prompt template as follows:

[0111] {

[0112] ‘Text’: S

[0113] ‘Prompt’: Relation_type of span_e

[0114] ‘Result’: span_r

[0115] }

[0116] where S is the text in the original dataset or the extended dataset, span_e is the text segment in S as the head entity of a certain relationship type, Relation_type is the name of a specific entity relationship type in the annotation information of the dataset described by S, and span_r is the tail entity text segment in S that has a specific relationship Relation_type with span_e (in particular, if there is no tail entity text segment in S that has a specific relationship Relation_type with span_e, then span_r is an empty string);

[0117] B3-3. Construct the entity relationship extraction task prompt template as follows:

[0118] {

[0119] ‘Text’: S

[0120] ‘Prompt’: Event_type

[0121] ‘Result’: span_v

[0122] }

[0123] {

[0124] ‘Text’: S

[0125] ‘Prompt’: Role_type of span_v

[0126] ‘Result’: span_o

[0127] }

[0128] wherein S is a text in the original dataset or the extended dataset, Event_type is a specific event type name in the annotation information of the dataset to which S belongs, span_v is a text segment in S that is a trigger word of a certain event type, Role_type is a specific event role name in the annotation information of the dataset to which S belongs, and span_o is a text segment in S that is annotated as the Role_type role of the Event_type event (in particular, the two ‘Result’ fields are also allowed to be empty strings when the corresponding text segments are not queried).

[0129] B3-4. Reconstruct the data in all the annotation information in the fused data according to steps B3-1 to B3-3, and randomly sample and construct negative sample data in all entity type names, relationship type names, event type names, and event role names from the text segments in the annotated data, i.e., samples that cannot query the corresponding ‘Result’; the dataset after fusion, reconstruction, and addition of negative samples is denoted as D mix = {(S k , P k , L k ), k = 1,..., K}, wherein S k represents text, P k represents prompt, L k represents result, and K is the number of all triple data groups in the dataset.

[0130] In some embodiments, step C of constructing and training a joint extraction model based on prompt learning of text information includes the following sub-steps:

[0131] C1. Set an encoder based on a Transformer network architecture, with an output dimension of d E , denoted as E model ;

[0132] C2. Construct a fully connected layer network module for the starting position of a text segment, denoted as FC start , and construct a fully connected layer network module for the ending position of a text segment, denoted as FC end , FC start , and FC endThe input dimensions of are d E , and the output dimensions are both 1.

[0133] C3. For a specified input triple data (S k , P k , L k ), prompt and text splicing are performed according to P k S k , and the length of the spliced text is denoted as d k .

[0134] C4. The spliced text is subjected to null character completion, i.e., meaningless null character sequence Q k is added after the spliced text, so that the character length of PkS k Q k reaches d * = max(d k , k = 1,..., K).

[0135] C5. The P k S k Q k mask vector T k = (t1,..., t d* ) is constructed, where

[0136]

[0137] C6. E model is used to encode P k S k Q k , and the encoded feature matrix is denoted as

[0138] C7. Based on the start position and the end position of L k in S k , the reference result vector GT k_start is constructed, where, GT k_start is 0 except that the start position of L k in S k is 1, and GT k_endt is 0 except that the end position of L k in S k is 1.

[0139] C8. The maximum number of iterations EP is set, and the current number of completed iterations ep is recorded.

[0140] C9. The minimum information entropy threshold ρ = exp[ep / (EP+1)] acceptable to the model at the current iteration number is calculated. -1 ;

[0141] C10. Calculate the information entropy of P k and P start , respectively, denoted as θ end and θ start , respectively. end ;

[0142] C11. Calculate the information entropy of P start and P end , respectively, denoted as θ start and θ end ;

[0143] C12. If θ start < p and θ end < p, calculate the cross-entropy loss based on P start and P end and GT k_start and GT k_end , respectively, and use backpropagation to train the model; otherwise, skip the current data.

[0144] C13. Perform steps C3 to C12 on all K groups of data. After completing a round of training on K groups of data, increase the value of ep by 1.

[0145] C14. Repeat C13 until ep = EP. The trained text information joint extraction model is denoted as Model.

[0146] In some embodiments, step D uses the trained model to perform joint information extraction on the input text, including the following sub-steps:

[0147] D1. Set the output threshold δ ∈ (0, 1)

[0148] D2. For the input text Text, traverse all entity category names {Entity_type(q), q = 1,..., Q} contained in the trained data, where Q is the number of all entity category names. Concatenate Entity_type(q) and Text and input them into Model to obtain the named entity recognition and extraction prediction vector under Entity_type(q). If P start > δ, take the position as the starting point, and take the position of the nearest P end > δ after this position as the ending point, and take the text segment between the two positions as the predicted entity text segment of Entity_type(q).

[0149] D3. For the input text Text and the predicted entity text span in step D2, traverse the entity relation type names {Relation_type(g), g = 1,..., G} associated with the entity type to which the predicted entity text span (denoted as Entity_span) belongs, G being the number of all entity relation types associated with the entity type to which Entity_span belongs; concatenate 'Relation_type(g) of Entity_span' and Text and input the result into Model to obtain the tail entity extraction prediction vector associated with Entity_span under Relation_type(g), denoted as P start >δ as the starting point, the nearest P end >δ as the ending point, and the text segment between the two positions as the predicted tail entity text segment of Entity_span under Relation_type(g);

[0150] D4. For the input text Text, traverse all event category names {Event_type(h), h = 1,..., H} contained in the training data, H being the number of all event category names; concatenate Event_type(h) and Text and input the result into Model to obtain the event trigger word extraction prediction vector under Event_type(h), denoted as P start >δ as the starting point, the nearest P end >δ as the ending point, and the text segment between the two positions as the predicted event trigger word text segment of Event_type(h);

[0151] D5. For the input text Text and the predicted event trigger word text span in step D4, traverse the event role names {Role_type(w), w = 1,..., W} associated with the event type to which the predicted event trigger word text span (denoted as Event_span) belongs, W being the number of all event roles associated with the event type to which Event_span belongs; concatenate 'Role_type(w) of Event_span' and Text and input the result into Model to obtain the event role extraction prediction vector associated with Event_span under Role_type(w), denoted as P start >δ as the starting point, the nearest P end >δ as the ending point, and the text segment between the two positions as the event role text segment of Event_span under Role_type(w);

[0152] D6. The unified representation framework constructed according to step A fuses the prediction results of steps D2 to D5 as the joint extraction results of the text information.

[0153] Up to now, the text information such as named entities, entity relationships and events can be jointly extracted from the given text according to steps A to D. The present application constructs a unified text information representation framework, different types of information extraction tasks can be fused into the unified framework, and different data sets defined in different ways can be jointly trained to solve the technical bottleneck caused by data expansion or cross-domain. On this basis, the information entropy filtering method is used to avoid the negative impact of noise data on the model, and to a certain extent, alleviate the data overfitting, and improve the accuracy and robustness of joint information extraction.

[0154] It should be noted that within the scope of protection defined in the claims of the present application, the following embodiments can be combined and / or extended, replaced by any logical manner from the above specific embodiments, such as disclosed technical principles, disclosed technical features or implied disclosed technical features.

[0155] Embodiment 1

[0156] A scalable text information joint extraction method based on prompt learning includes the following steps:

[0157] A. Construct a unified representation framework of text information extraction annotation information;

[0158] B. Construct a unified prompt template based on the original data set and the expanded data set;

[0159] C. Construct and train a text information joint extraction model based on prompt learning;

[0160] D. Use the trained model to perform joint information extraction on the input text.

[0161] Embodiment 2

[0162] On the basis of embodiment 1, in step A, the unified representation framework of text information extraction annotation information includes the following sub-steps:

[0163] A1. For the named entity recognition task, the annotation information representation framework is constructed as follows: (Entity_type:span_e), wherein Entity_type represents the name of a selected entity type, and span_e represents a text segment in the text that belongs to the Entity_type type entity.

[0164] A2. For the entity relation extraction task, the annotation information representation framework is constructed as: (Entity_type:span_e:(Relation_type_i:span_r_i)), wherein Relation_type_i represents the i-th entity relation type name associated with the Entity_type entity type, and span_r_i represents the tail entity text segment corresponding to span_e under Relation_type_i;

[0165] A3. For the event extraction task, the annotation information representation framework is constructed as: (Event_type:span_v:(Role_type_i:span_o_i)), wherein Event_type represents a selected event type name, span_v represents an event trigger text segment in the text belonging to the Event_type event type, Role_type_i represents the i-th event role name associated with the Event_type type event, and span_o_i represents the event role text segment corresponding to span_v under Role_type_i;

[0166] A4. The annotation information representation frameworks of named entity recognition, entity relation extraction, and event extraction are fused to form a unified representation framework of text information extraction annotation information, and an example form is as follows:

[0167]

[0168]

[0169] Embodiment 3

[0170] On the basis of embodiment 1, in step B, the constructing of the unified prompt template based on the original data set and the extended data set comprises the following sub-steps:

[0171] B1. Define the original data set as D ori ={(S i ,L i ),i=1,...,N ori} wherein S i represents the text of the i-th data of the original data set, L i represents the annotation information of the i-th data of the original data set, and N ori represents the number of original data samples;

[0172] B2. Define the extended data set as D ext ={(S j ,L j ),j=1,...,N ext} wherein S jtext representing the jth data of the extended data set, L j annotation information representing the jth data of the extended data set, N ext representing the number of extended data samples;

[0173] B3. Fuse the original data set and the extended data set, and construct a unified prompt template for each piece of annotation information.

[0174] Embodiment 4

[0175] On the basis of Embodiment 1, in step C, the constructing and training of the prompt learning text information joint extraction model includes the following sub-steps:

[0176] C1. Set an encoder based on the Transformer network architecture, with an output dimension of d E , denoted as E model ;

[0177] C2. Construct a fully connected layer network module for the starting position of the text segment, denoted as FC start , and construct a fully connected layer network module for the ending position of the text segment, denoted as FC end , FC start and FC end have input dimensions of d E , and output dimensions of 1;

[0178] C3. For a specified input triple (S k , P k , L k ), perform prompting and text splicing according to P k S k , and the length of the spliced text is denoted as d k ;

[0179] C4. Perform null character completion on the spliced text, i.e., add a meaningless null character sequence Q k to the spliced text, so that the character length of PkS k Q k reaches d * = max(d k , k = 1,..., K);

[0180] C5. Construct a P k S k Q k mask vector wherein,

[0181]

[0182] C6. Use E model to process P k Sk Q k Encoding is performed, and the encoded feature matrix is denoted as

[0183] C7. Based on L k The starting position and the ending position in S k , the reference result vector GT k_start , wherein GT k_start L k The starting position in S k is 1, and the rest are 0, GT k_endt L k The ending position in S k is 1, and the rest are 0;

[0184] C8. Set the maximum number of iterations EP, and record the current number of completed iterations as ep;

[0185] C9. Calculate the minimum information entropy threshold ρ of the model that can be accepted at the current iteration number, ρ = exp[ep / (EP+1)] -1 ;

[0186] C10. Input F k into FC start and FC end respectively, and use sigmoid for activation to obtain the prediction vectors P start and P end respectively;

[0187] C11. Calculate the information entropy of P start and P end respectively, denoted as θ start and θ end ;

[0188] C12. If θ start < ρ and θ end < ρ, calculate the cross-entropy loss based on P start and P end and GT k_start and GT k_end respectively, and use back propagation for model training, otherwise, skip the current data;

[0189] C13. Perform steps C3 to C12 on all K groups of data, and after completing a round of training of the K groups of data, increase the value of ep by 1;

[0190] C14. Repeat C13 until ep = EP, and obtain the trained text information joint extraction model denoted as Model.

[0191] Example 5

[0192] On the basis of Embodiment 1, in Step D, the joint information extraction on the input text using the trained model comprises the following sub-steps:

[0193] D1. Set an output threshold δ ∈ (0, 1);

[0194] D2. For the input text Text, traverse all entity category names {Entity_type(q), q = 1,..., Q} contained in the trained data, Q being the number of all entity category names; concatenate Entity_type(q) and Text and input Model to obtain the named entity recognition extraction prediction vector under Entity_type(q), with P start >δ as the starting point, the nearest P end >δ as the ending point, and the text segment between the two positions as the predicted entity text segment of Entity_type(q);

[0195] D3. For the input text Text and the entity text segment predicted in Step D2, traverse the entity relationship type names {Relation_type(g), g = 1,..., G} associated with the entity type to which the predicted entity text segment belongs, G being the number of all entity relationship types associated with the entity type to which the predicted entity text segment belongs; concatenate 'Entity_span’s Relation_type(g)' and Text and input Model to obtain the tail entity extraction prediction vector associated with Entity_span under Relation_type(g), with P start >δ as the starting point, the nearest P end >δ as the ending point, and the text segment between the two positions as the predicted tail entity text segment of Entity_span under Relation_type(g);

[0196] D4. For the input text Text, traverse all event category names {Event_type(h), h = 1,..., H} contained in the trained data, H being the number of all event category names; concatenate Event_type(h) and Text and input Model to obtain the event trigger word extraction prediction vector under Event_type(h), with P start >δ as the starting point, the nearest P end >δ as the ending point, and the text segment between the two positions as the predicted event trigger word text segment of Event_type(h);

[0197] D5. For the input text Text and the event trigger word text fragments predicted in step D4, iterate through the event role names associated with the event type of Event_span, denoted as {Role_type(w), w = 1, ..., W}, where W is the number of all event roles associated with the event type of Event_span; concatenate 'Role_type(w)' of Event_span with Text and input it into Model to obtain the prediction vector of the event roles associated with Event_span under Role_type(w), denoted as P. start The position of δ is the starting point, and the nearest P after this position is... end The position of >δ is the termination point, and the text fragment between the two positions is taken as the event role text fragment under Role_type(w) of Event_span;

[0198] D6. The prediction results from steps D2 to D5 are integrated into the unified representation framework constructed in step A as the result of joint extraction of text information.

[0199] Example 6

[0200] Based on Example 3, step B3 includes the following sub-steps:

[0201] B3-1. The following is a template for constructing a named entity recognition task:

[0202] {

[0203] 'Text':S

[0204] 'Hint': Entity_type

[0205] 'Result': span_e

[0206] }

[0207] Where S is the text in the original dataset or extended dataset, Entity_type is the name of an entity type selected in the annotation information of the dataset to which S belongs, and span_e is the text fragment of an entity in S that belongs to the Entity_type type;

[0208] B3-2. The task prompt template for constructing entity relationship extraction is as follows:

[0209] {

[0210] 'Text':S

[0211] 'Hint': Relation_type of span_e

[0212] 'Result':span_r

[0213] }

[0214] Where S is the text in the original dataset or extended dataset, span_e is the text fragment in S that is the head entity of a certain relation type, Relation_type is the name of an entity relation type selected in the annotation information of the dataset to which S belongs, and span_r is the text fragment of the tail entity in S that has a selected relation Relation_type with span_e.

[0215] B3-3. The task prompt template for constructing entity relationship extraction is as follows:

[0216] {

[0217] 'Text':S

[0218] 'Prompt': Event_type

[0219] 'Result':span_v

[0220] }

[0221] {

[0222] 'Text':S

[0223] 'Hint': Role_type of span_v

[0224] 'Result':span_o

[0225] }

[0226] Where S is the text in the original dataset or extended dataset, Event_type is a specific event type name in the annotation information of the dataset to which S belongs, span_v is a text fragment in S that is a trigger word for a certain event type, Role_type is a specific event role name in the annotation information of the dataset to which S belongs, and span_o is a text fragment in S that is labeled as the Role_type event.

[0227] B3-4. Reconstruct the data for all labeled information in the fused data according to steps B3-1 to B3-3, and randomly sample text fragments from all entity type names, relation type names, event type names, and event role names to construct negative sample data, i.e., samples for which no corresponding 'result' can be found; the dataset after fusion, reconstruction, and addition of negative samples is denoted as D. mix ={(S k ,P k ,L k ),k=1,...,K}, where S k P represents text. k Indicates a prompt, Lk The result is represented by K, which is the number of all three data sets in the dataset.

[0228] Example 7

[0229] Based on Example 6, in step B3-1, if there is no entity text fragment of type Entity_type in S, then span_e is set to an empty string.

[0230] Example 8

[0231] Based on Example 6, in step B3-2, if there is no tail entity text fragment in S that has a specific relationship Relation_type with span_e, then span_r is set to an empty string.

[0232] Example 9

[0233] Based on Example 6, in step B3-3, the two 'result' fields are set to empty strings when no corresponding text fragment is found.

[0234] Example 10

[0235] A scalable text information joint extraction device based on cue learning, based on the method described in any one of Embodiments 1 to 9, further includes:

[0236] The first building module is used to construct a unified representation framework for extracting and labeling text information.

[0237] The second building module is used to build a unified prompt template based on the original dataset and the extended dataset;

[0238] The third building module is used to build and train a joint text information extraction model based on prompting learning;

[0239] The extraction module is used to extract joint information from the input text using the trained model.

[0240] The units described in the embodiments of the present invention can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0241] According to one aspect of the present invention, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations described above.

[0242] In another aspect, embodiments of the present invention also provide a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods described in the above embodiments.

[0243] All parts not covered in this invention are the same as or can be implemented using existing technologies.

[0244] The above technical solution is only one embodiment of the present invention. For those skilled in the art, based on the application methods and principles disclosed in the present invention, it is easy to make various types of improvements or modifications, and not limited to the methods described in the above specific embodiments of the present invention. Therefore, the methods described above are only preferred and are not restrictive.

[0245] In addition to the examples above, other embodiments may be obtained by those skilled in the art based on the above disclosure or by making modifications using knowledge or technology in related fields. The features of each embodiment may be interchanged or replaced. Modifications and changes made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A scalable text information joint extraction method based on cue learning, characterized in that, Includes the following steps: A. Construct a unified representation framework for extracting and labeling text information; B. Construct a unified prompt template based on the original dataset and the extended dataset; C. Construct and train a joint text information extraction model based on prompting learning; D. Use the trained model to perform joint information extraction on the input text; In step C, the construction and training of the cue-based text information joint extraction model includes the following sub-steps: C1. Configure an encoder based on a Transformer network architecture, with an output dimension of... , recorded as ; C2. Construct a fully connected layer network module targeting the start position of the text segment, denoted as... Construct a fully connected layer network module targeting the end position of a text segment, denoted as . , and The input dimensions are all The output dimension is 1. C3. For specified input triplet data ,according to Perform prompts and text concatenation; the length of the concatenated text is recorded as follows: ;in, Represents text, This indicates a prompt. Indicates the result; C4. Add null characters to the concatenated text, that is, add a sequence of meaningless null characters after the concatenated text. , making The character length reaches ; C5. Construction Mask vector ,in, C6. Use right Encode the feature matrix, and the encoded feature matrix is ​​denoted as... ; C7. Based on exist The baseline result vector is constructed from the start and end positions in the vector. ,in, remove exist Except for the initial position which is 1, all other positions are 0. remove exist Except for the terminating position which is 1, all other values ​​are 0; C8. Set the maximum number of iterations Record the current number of iterations completed. ; C9. Calculate the lowest acceptable information entropy threshold for the model at the current iteration number. ; C10. Will Enter them separately and And sigmoid is used for activation to obtain the prediction vectors respectively. and ; C11. Calculate separately and Information entropy, denoted as and ; C12. If and Based on and respectively with and Calculate the cross-entropy loss and use backpropagation to train the model; otherwise, skip the current data. C13. For all Perform steps C3 to C12 on the group data to complete one round. After training with the dataset, let The value increases by 1; The number of all three data sets in the dataset; C14. Repeat C13 until... So far, the trained text information joint extraction model is denoted as... .

2. The scalable text information joint extraction method based on cue learning according to claim 1, characterized in that, Step A, the construction of a unified representation framework for extracting and labeling text information, includes the following sub-steps: A1. For the named entity recognition task, the following framework for representing labeled information is constructed: ,in, Indicates the name of a selected entity type. Indicates that the text belongs to Text fragments of type entities; A2. For the entity relation extraction task, the following annotation information representation framework is constructed: ,in, Indicates and The first entity type association Entity relationship type name, express exist The corresponding tail entity text fragment; A3. For the event extraction task, the framework for representing labeled information is constructed as follows: ,in, Indicates the name of the selected event type. Indicates that the text belongs to Event trigger word text fragments of event types, Indicates and The type of event associated with the first Event character name, express exist The corresponding event character text fragment; A4. Integrate named entity recognition, entity relationship extraction, and event extraction annotation information representation framework to form a unified representation framework for text information extraction and annotation information.

3. The scalable text information joint extraction method based on cue learning according to claim 1, characterized in that, Step B, which involves constructing a unified prompt template based on the original dataset and the extended dataset, includes the following sub-steps: B1. Define the original dataset as... ,in, Represents the original dataset's first... The text of the data, Represents the original dataset's first... The annotation information of each data item This indicates the number of original data samples; B2. Define the extended dataset as follows ,in, Indicates the extended dataset number The text of the data, Indicates the extended dataset number The annotation information of each data item Indicates the number of extended data samples; B3. Integrate the original dataset and the extended dataset, and build a unified prompt template for each annotation.

4. The scalable text information joint extraction method based on cue learning according to claim 1, characterized in that, Step D, which involves using the trained model to extract joint information from the input text, includes the following sub-steps: D1. Set the output threshold ; D2. For the input text Iterate through all entity category names contained in the trained data. , The number of all entity category names; and Input after splicing ,get Named entity recognition extracts prediction vectors to... Starting from that position, the nearest [location] is... The position is taken as the endpoint, and the text segment between the two positions is used as... Predicted entity text fragments; D3. For the input text The entity text fragments predicted in step D2 are traversed and predicted as follows: The name of the entity relationship type associated with the entity type , for The number of all entity relationship types associated with the given entity type; Will' of 'and Input after splicing ,get The following and The associated tail entity is extracted to predict the vector, in order to Starting from that position, the nearest [location] is... The position is taken as the endpoint, and the text segment between the two positions is used as... exist The predicted tail entity text fragment; D4. For the input text Iterate through all event category names contained in the trained data. , The number of all event category names; and Input after splicing ,get Extract prediction vectors from event trigger words, in order to Starting from that position, the nearest [location] is... The position is taken as the endpoint, and the text segment between the two positions is used as... The text fragment of the predicted event trigger word; D5. For the input text The event trigger word text fragments predicted in step D4 are traversed and predicted, denoted as... Event role name associated with the event type , for The number of all event roles associated with the event type; of 'and Input after splicing ,get The following and The associated event roles are used to extract prediction vectors, in order to Starting from that position, the nearest [location] is... The position is taken as the endpoint, and the text segment between the two positions is used as... exist The following is a text fragment about the event characters; D6. The prediction results from steps D2 to D5 are integrated into the unified representation framework constructed in step A as the result of joint extraction of text information.

5. A scalable text information joint extraction device based on cue learning, characterized in that, Based on the method according to any one of claims 1 to 4, it further includes: The first building module is used to construct a unified representation framework for extracting and labeling text information. The second building module is used to build a unified prompt template based on the original dataset and the extended dataset; The third building module is used to build and train a joint text information extraction model based on prompting learning; The extraction module is used to extract joint information from the input text using the trained model.

Images