Text temporal extraction method and device, electronic equipment and storage medium

Abstract

The application provides a text time sequence extraction method and device, electronic equipment and a storage medium. The method comprises the following steps: obtaining a target instance in a text to be extracted for a time sequence relationship; dividing the target instance into a plurality of instance segments; encoding a word vector into a first hidden representation of the instance segment; calculating a position weight of the instance segment based on entity position information; weighting the first hidden representation of the instance segment by using the position weight to obtain a second hidden representation of the instance segment; calculating a correlation attention parameter of the instance segment based on the second hidden representation of the instance segment; combining the correlation attention parameter of the instance segment into a target hidden representation of the target instance; and performing time sequence extraction on the target instance in the text based on the target hidden representation to obtain a time sequence relationship of the target instance in the text. The application obtains the hidden representation of the instance through the correlation attention network, realizes more differentiated and associated instance representation, and effectively captures the entity time sequence relationship of the text.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a method, apparatus, electronic device, and storage medium for extracting text temporal sequence. Background Technology

[0002] Temporal relation extraction is a key task in natural language processing, aiming to automatically identify and extract the temporal order relationships between entities from text. It is crucial for a deeper understanding of event progression within text, constructing accurate timelines, and establishing causal relationships for inference. In recent years, electronic health records have become widely used, and extracting information for secondary use from the text of electronic health records has attracted user attention. Because temporal relation extraction can construct timelines and determine the relationships between medical events and temporal expressions, it is considered a necessary process for the secondary use of electronic health records.

[0003] Currently, electronic health records contain highly structured tables and lists, but also a large amount of unstructured text such as doctors' descriptions, medical history records, and chief complaints. Unstructured text is often fragmented, lacking continuity between paragraphs, and contains a large number of entities and entity pairs within the same text. This makes it difficult to distinguish instances and identify context-related information for specific entity pairs, increasing the complexity of temporal relation extraction. Summary of the Invention

[0004] In view of this, the present invention aims to propose a text temporal extraction method, apparatus, electronic device and storage medium to solve the problem that it is difficult to distinguish and identify text instances and entities based on contextual information, which leads to the complexity of text temporal relationship extraction, and improves the level of text classification and recognition and the ability to extract temporal relationships.

[0005] According to a first aspect of the present invention, a text temporal extraction method is provided, the method comprising:

[0006] Obtain target instances from the text from which temporal relationships are to be extracted; the target instances include entities and entity pairs.

[0007] The target instance is divided into multiple instance segments; each instance segment includes word vectors and entity location information.

[0008] Encode the word vectors into a first hidden representation of the instance fragment;

[0009] The position weight of the instance fragment is calculated based on the entity position information;

[0010] The first hidden representation of the instance fragment is weighted using the positional weights to obtain the second hidden representation of the instance fragment;

[0011] The association attention parameters of the instance fragment are calculated from the second hidden representation of the instance fragment;

[0012] The associated attention parameters of the instance fragments are combined to form the target hidden representation of the target instance;

[0013] Based on the target hiding representation, the target instances in the text are extracted temporally to obtain the temporal relationship of the target instances in the text.

[0014] Optionally, the step of dividing the target instance into multiple instance segments; the instance segments include word vectors and entity location information, including:

[0015] Based on the positions of entities and entity pairs within the target instance, the target instance is divided into multiple instance fragments;

[0016] Identify the entities and their positions within the instance fragment to obtain the entity position information of the instance fragment;

[0017] The instance fragment is transformed into a vector to obtain the word vector corresponding to the instance fragment.

[0018] Optionally, encoding the word vector into a first hidden representation of the instance fragment includes:

[0019] The word vectors corresponding to the instance fragments are integrated into a multi-dimensional vector;

[0020] The multidimensional vector is encoded to obtain the forward hidden state and the reverse hidden state of the instance fragment;

[0021] The forward hidden state and the reverse hidden state are concatenated to obtain the first hidden representation of the instance fragment.

[0022] Optionally, calculating the position weight of the instance fragment based on the entity position information includes:

[0023] Determine the segment length of the instance segment and the entity location information within the instance segment;

[0024] The position weight of the instance segment is obtained by performing nonlinear calculations on the entity position information and the segment length.

[0025] Optionally, the position weight of the instance fragment is obtained by performing a nonlinear calculation on the entity position information and the fragment length. The position weight of the instance fragment is obtained by the following formula:

[0026]

[0027]

[0028] in, and Let be the position weights of the instance fragment corresponding to the i-th position, and tanh be a non-linear function. To perform upper limit operations, Len L Len C and Len R These are the fragment lengths of the left instance fragment, the middle instance fragment, and the right instance fragment, respectively.

[0029] Optionally, the instance fragment includes a left instance fragment, a middle instance fragment, and a right instance fragment, and the second hidden representation of the instance fragment is used to calculate the associated attention parameters of the instance fragment, including:

[0030] The second hidden representation of the intermediate instance fragment is pooled to obtain the overall representation of the intermediate instance fragment and the importance weight of the intermediate instance fragment.

[0031] The importance weights of the left and right instance segments are obtained by calculating the overall representation of the intermediate instance segment;

[0032] Using the second hidden representation and the importance weight, the association attention parameters of the instance segment are calculated; the association attention parameters of the instance segment are obtained by the following formula:

[0033]

[0034] Where, r C r L r R These are the attention parameters for the middle instance fragment, the left instance fragment, and the right instance fragment, respectively. These are the importance weights for the middle instance fragment, the left instance fragment, and the right instance fragment, respectively. These are the second hidden representations of the i-th word in the middle instance fragment, left instance fragment, and right instance fragment, respectively.

[0035] Optionally, the associative attention parameters of the instance fragments are combined to form the target hidden representation of the target instance, which is obtained by the following formula:

[0036]

[0037] y = softmax(W o ·r)

[0038] Where r is the associative attention combination of instance fragments, and sigmoid is a non-linear activation function. The weight matrices r represent the weights of the middle instance fragment, the left instance fragment, and the right instance fragment, respectively. C r L r R These are the attention parameters for the middle instance fragment, left instance fragment, and right instance fragment, respectively. softmax is a non-linear function, and W... o The output weight matrix is ​​y, where y is the target hidden representation of the target instance.

[0039] According to a second aspect of the present invention, a text temporal extraction apparatus is provided, the apparatus comprising:

[0040] The instance acquisition module is used to acquire target instances of the time sequence relationship to be extracted from the text; the target instances include entities and entity pairs.

[0041] An instance segmentation module is used to divide the target instance into multiple instance segments; the instance segments include word vectors and entity location information.

[0042] Vector encoding module, used to encode the word vectors into a first hidden representation of the instance fragment;

[0043] The weight determination module is used to calculate the position weight of the instance fragment based on the entity position information.

[0044] The weighted processing module is used to weight the first hidden representation of the instance fragment using the position weights to obtain the second hidden representation of the instance fragment;

[0045] The associated attention parameter module is used to calculate the associated attention parameters of the instance fragment from the second hidden representation of the instance fragment;

[0046] The parameter processing module is used to combine the associated attention parameters of the instance fragments into the target hidden representation of the target instance;

[0047] The temporal extraction module is used to extract the target instances in the text temporally based on the target hiding representation, so as to obtain the temporal relationship of the target instances in the text.

[0048] According to another aspect of the present invention, an electronic device is also provided, comprising:

[0049] processor;

[0050] Memory used to store the processor's executable instructions;

[0051] The processor is configured to execute the instructions to implement the text temporal extraction method described above.

[0052] According to another aspect of the present invention, a readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the text timing extraction method as described above.

[0053] The text temporal extraction method provided in this invention obtains target instances of the temporal relationship to be extracted from the text, divides the target instances into multiple instance segments, encodes word vectors into first hidden representations of the instance segments, calculates positional weights of the instance segments based on entity position information, weights the first hidden representations of the instance segments using positional weights to obtain second hidden representations of the instance segments, calculates association attention parameters of the instance segments using the second hidden representations of the instance segments, combines the association attention parameters of the instance segments into target hidden representations of the target instances, and performs temporal extraction of target instances in the text based on the target hidden representations to obtain the temporal relationship of the target instances in the text. This invention divides target instances into multiple instance segments, encodes word vectors of the instance segments to obtain hidden representations representing the dependencies between instance segments, uses positional weights to weight the hidden representations, emphasizes important information around entities, and obtains hidden representations that reflect the relationships between instance segments through an association attention network. This allows for better handling of instance segments at different positions, improves the understanding of instance segments, obtains more differentiated and related instance representations, further enhances text classification and recognition levels and temporal relationship extraction capabilities, and effectively captures the temporal relationship of entities in the text.

[0054] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0055] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0056] Figure 1 This is a flowchart of the steps of a text temporal extraction method provided in an embodiment of the present invention;

[0057] Figure 2 yes Figure 1 A flowchart of step 102 in the text temporal extraction method provided in this embodiment of the invention;

[0058] Figure 3 yes Figure 1A flowchart of step 103 in the text temporal extraction method provided in this embodiment of the invention;

[0059] Figure 4 yes Figure 1 A flowchart of step 104 in the text temporal extraction method provided in this embodiment of the invention;

[0060] Figure 5 yes Figure 1 A flowchart of step 106 in the text temporal extraction method provided in this embodiment of the invention;

[0061] Figure 6 This is a schematic diagram of a scenario for the text temporal extraction method provided in an embodiment of the present invention;

[0062] Figure 7 This is a schematic diagram illustrating the instance division of the text temporal extraction method provided in this embodiment of the invention;

[0063] Figure 8 This is a schematic diagram of the structure of a text temporal extraction device provided in an embodiment of the present invention;

[0064] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the various embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details are presented in the various embodiments of the present invention to facilitate a better understanding of this application. However, the technical solutions claimed in this application can be implemented even without these technical details and various changes and modifications based on the following embodiments. The division of the various embodiments below is for ease of description and should not constitute any limitation on the specific implementation of the present invention. The various embodiments can be combined with and referenced by each other without contradiction.

[0066] Reference Figure 1 The flowchart illustrates the steps of a text temporal extraction method provided in an embodiment of the present invention. The method may include:

[0067] Step 101: Obtain the target instance of the temporal relationship to be extracted from the text; the target instance includes entities and entity pairs.

[0068] In this embodiment of the invention, since unstructured text in electronic health records is characterized by strong paragraph structure and a large number of entities, in order to solve the problem that the large number of entities and entity pairs in the same text makes it difficult to distinguish instances and identify the context-related information of specific entity pairs, resulting in complex extraction of text temporal relationships, this embodiment divides each instance into a left segment, a middle segment, and a right segment according to the position of the text entities. The three segments are encoded separately using a bidirectional long short-term memory network. The proposed associative attention mechanism is used to strengthen the important information hidden in each segment, reconstruct the association between segments, and obtain more differentiated and related segment representations, thereby improving the ability to extract temporal relationships from electronic health record text.

[0069] It should be noted that this embodiment constructs an associative attention network and uses it to extract temporal relationships from text. Specifically, it obtains target instances of the temporal relationships to be extracted from the text. Target instances include entities and entity pairs. In the task of extracting temporal relationships from text, target instances can be long sentences or short paragraphs in the text. Target instances include entities (such as medical events, time expressions, etc.) and entity pairs (i.e., the relationship between two entities). The selection of target instances is determined according to the actual text content and requirements. This embodiment does not impose specific limitations on this.

[0070] Step 102: Divide the target instance into multiple instance fragments; instance fragments include word vectors and entity location information.

[0071] In this embodiment of the invention, the target instance is divided into multiple instance segments according to the position of entities and entity pairs. Instance segments include left instance segments, middle instance segments, and right instance segments. Specifically, this embodiment first sets a maximum length for the target instance; instances exceeding this length do not participate in the temporal relation extraction task. Instances are divided into long instances and short instances based on half of the maximum length. For any given instance, it is preprocessed to be divided into left, middle, and right parts according to the position of the entities, and both entities are included in the middle segment.

[0072] Specifically, the target instance is divided into three parts (left, middle, and right) according to the entity's position. Based on the division result, the entity position information of the instance fragment is obtained. The entity position information includes the start and end positions of the entity. Word vectors map character-type words or characters / words in the instance fragment into numerical vectors that can be recognized by computers. It should be noted that, in order to contain as much semantic and syntactic information as possible between the vectors, this embodiment uses the BERT model to perform vector transformation on the instance fragment, converting each word in the instance fragment into a continuous vector. This helps to combine the contextual information of the instance fragment with the temporal relationship, making it easier to understand the temporal relationship between text events.

[0073] Step 103: Encode the word vectors into the first hidden representation of the instance fragment.

[0074] In this embodiment of the invention, a Long Short-Term Memory (LSTM) network is used to encode word vectors into a first hidden representation of an instance segment. The first hidden representation refers to the intermediate layer output obtained after a series of calculations in the neural network. The intermediate layer output is usually a high-dimensional numerical vector that can capture the complex features and semantic information of the input data. LSTM is a type of recurrent neural network used to solve the gradient vanishing and gradient exploding problems during the training process of long sequences, and it performs better in longer sequences. In this embodiment of the invention, Bi-LSTM is used to encode each instance segment. The preprocessed word vectors are sequentially fed into Bi-LSTM. The word vectors of each instance segment are input twice, forward and backward, to obtain two embedding codes. These two embedding codes are then concatenated to represent the word, thereby obtaining the hidden representation of each segment.

[0075] Step 104: Calculate the position weight of the instance fragment based on the entity position information.

[0076] In this embodiment of the invention, the target instance is divided into multiple instance fragments according to the location of the entities. The number and location of entities included in the instance fragments are different. Assuming that the context of the electronic health record has the same characteristics, words closer to the entities are more meaningful. Therefore, this embodiment calculates the position weight of the instance fragments based on the entity position information and uses the position weight to emphasize the hidden representation in the instance fragments. Specifically, the position weight can be calculated by the ratio of the relative distance between the words and entities in the instance fragment to the fragment length, which will not be elaborated here.

[0077] Step 105: The first hidden representation of the instance fragment is weighted using position weights to obtain the second hidden representation of the instance fragment.

[0078] Step 106: Calculate the association attention parameters of the instance fragment for the second hidden representation of the instance fragment.

[0079] This invention employs an attention mechanism to emphasize relevant information to aggregate entities and similar instances, making instance representations easier to distinguish and highlighting important information hidden within instances. Specifically, based on the weighted weighting of the first hidden representation of an instance fragment to obtain a second hidden representation, the association attention parameters for the middle instance fragment and the left and right instance fragments are calculated respectively. The association attention parameters are characterized by importance weights and weighted sums, where importance weights represent the importance of different parts of the input data and can determine which parts should be focused on when processing the input data. By multiplying the input data by the corresponding importance weights and performing a weighted sum calculation, a weighted representation can be obtained. This representation, i.e., the association attention parameters, can better capture the important information in the input data.

[0080] Specifically, since the intermediate instance fragment usually contains the richest temporal information between two entities, it is regarded as the key segment. First, the association attention parameters of the intermediate instance fragment are calculated, and then the association attention parameters of the left and right instance fragments are calculated based on the association attention parameters of the intermediate instance fragment. This will not be elaborated on here.

[0081] Step 107: Combine the associated attention parameters of the instance fragments into the target hidden representation of the target instance.

[0082] Step 108: Extract the target instances in the text temporally based on the target hiding representation to obtain the temporal relationship of the target instances in the text.

[0083] In this embodiment of the invention, the target instances in the text are temporally extracted based on the target hiding representation to obtain the temporal relationship of the target instances in the text. Specifically, the target hiding representation is provided to the output layer to predict the temporal relationship between entity pairs. The target hiding representation contains the semantic information of the instance fragments and the associated attention parameters. The target hiding representation is provided to the output layer, which is usually a fully connected layer. The temporal relationship between entity pairs is predicted based on the output of the output layer. There are usually many types of temporal relationships, such as: event A occurs before event B, event A occurs after event B, event A and event B occur simultaneously, and event A and event B overlap. The temporal relationship of the target instances is integrated to obtain the temporal relationship of the events in the text.

[0084] The text temporal extraction method provided in this invention obtains target instances of the temporal relationship to be extracted from the text, divides the target instances into multiple instance segments, encodes word vectors into first hidden representations of the instance segments, calculates positional weights of the instance segments based on entity position information, weights the first hidden representations of the instance segments using positional weights to obtain second hidden representations of the instance segments, calculates association attention parameters of the instance segments using the second hidden representations of the instance segments, combines the association attention parameters of the instance segments into target hidden representations of the target instances, and performs temporal extraction of target instances in the text based on the target hidden representations to obtain the temporal relationship of the target instances in the text. This invention divides target instances into multiple instance segments, encodes word vectors of the instance segments to obtain hidden representations representing the dependencies between instance segments, uses positional weights to weight the hidden representations, emphasizes important information around entities, and obtains hidden representations that reflect the relationships between instance segments through an association attention network. This allows for better handling of instance segments at different positions, improves the understanding of instance segments, obtains more differentiated and related instance representations, further enhances text classification and recognition levels and temporal relationship extraction capabilities, and effectively captures the temporal relationship of entities in the text.

[0085] Furthermore, refer to Figure 2 , showed Figure 1 The flowchart of step 102 in the provided text temporal extraction method is basically the same as the text temporal extraction method provided in the first embodiment of the present invention. Step 102 may include:

[0086] Step 201: Divide the target instance into multiple instance fragments based on the positions of entities and entity pairs in the target instance.

[0087] In this embodiment of the invention, the entire target instance is divided into multiple instance segments according to the position of entities and entity pairs in the target instance. Each instance segment contains one or more entities and their corresponding relationships. It should be noted that in this embodiment, the maximum length of the target instance to be processed can be set first. Instances exceeding this length will not participate in the processing. The instance is divided into long instances and short instances according to half of the maximum length to obtain the target instance. This ensures that the instances participating in training have a reasonable length and avoids that excessively long instances will have a negative impact on model training. Through preprocessing, the target instance is divided into three parts, left, middle and right, according to the position of the entities, and both entities are included in the middle instance segment.

[0088] For example, refer to Figure 7 This illustration shows an instance partitioning diagram of the text temporal extraction method provided in this embodiment of the invention. Taking a target instance "She does not have her walker here today and is using the wheelchair primarily." as an example, assuming entity 1 is "walker" and entity 2 is "wheelchair", the processed instance can be partitioned as follows: left part: She does not have her; middle part: walker here today and is using the; and right part: wheelchair primarily. Alternatively, assuming entity 1 is "today" and entity 2 is "using", the processed instance can be partitioned as follows: left part: She does not have her walker here; middle part: today and is using; and right part: the wheelchair primarily.

[0089] Step 202: Determine the entities and their positions in the instance fragment to obtain the entity position information of the instance fragment.

[0090] In this embodiment, after the target instance is divided into multiple instance segments, the entities and their positions in the instance segments will change as the instance segments are divided according to the positions of the entities and entity pairs in the target instance. By dividing the target instance into three parts (left, middle, and right) according to the positions of the entities, and including both entities in the middle instance segment, in order to capture the relationship between entities, the entities and their positions in the instance segments are determined and recorded to obtain the entity position information of the instance segments. The entity position information includes the start position and end position of the entity.

[0091] Step 203: Perform vector transformation on the instance fragment to obtain the word vector corresponding to the instance fragment.

[0092] Specifically, the instance fragments are transformed into vectors to obtain the word vectors corresponding to the instance fragments. Each instance fragment can be input into a pre-trained vector model to obtain the word vectors corresponding to the instance fragments. Word vectors are usually high-dimensional numerical vectors that can capture the semantic information in the text.

[0093] It should be noted that the word vectors corresponding to the instance fragments are obtained through pre-trained vector models. In this embodiment, BERT (Bidirectional Encoder Representations from Transformers) is used as the pre-trained vector model. The BERT model is a deep pre-trained model built using bidirectional Transformers. The key in the Transformer module is the multi-head attention mechanism, which inputs the instance fragments into the pre-trained vector model and outputs the word vectors corresponding to the instance fragments. Specifically, the BERT model is used to pre-train the instance fragments to obtain a word embedding lookup table, which converts each word in the instance fragment into a continuous word vector.

[0094] This invention, through dividing a target instance into multiple instance segments and performing vector transformation on the instance segments to obtain word vectors corresponding to the instance segments, can better handle instance segments at different positions and improve the ability to understand instance segments.

[0095] Furthermore, refer to Figure 3 , showed Figure 1 The flowchart of step 103 in the provided text temporal extraction method is basically the same as the text temporal extraction method provided in the first embodiment of the present invention. Step 103 may include:

[0096] Step 301: Integrate the word vectors corresponding to the instance fragments into a multi-dimensional vector.

[0097] Step 302: Encode the multidimensional vector to obtain the forward hidden state and the reverse hidden state of the instance fragment.

[0098] Step 303: Concatenate the forward hidden state and the reverse hidden state to obtain the first hidden representation of the instance fragment.

[0099] In this embodiment of the invention, the word vectors corresponding to the instance fragments are input into a Long Short-Term Memory (Bi-LSTM) network, and the word vectors corresponding to the instance fragments are integrated into a multi-dimensional vector. Bi-LSTM is a commonly used sequence model that can capture long-term dependencies in sequence data. By encoding the multi-dimensional vectors through Bi-LSTM, the forward hidden state and the reverse hidden state of the word vectors can be obtained. The forward hidden state and the reverse hidden state are concatenated to obtain the first hidden representation of the instance fragment. The concatenation process usually involves connecting the forward hidden state and the reverse hidden state element by element. The hidden representation refers to the intermediate layer output obtained after a series of calculations in the neural network. The intermediate layer output is usually a high-dimensional numerical vector that can capture the complex features and semantic information of the input data.

[0100] The embodiments of the present invention obtain a hidden representation for characterizing the dependency relationship of instance fragments by encoding the word vectors of instance fragments, resulting in a more differentiated and related instance representation, which helps to capture the temporal relationship of entities in the text.

[0101] Furthermore, refer to Figure 4 , showed Figure 1 The flowchart of step 104 in the provided text temporal extraction method is basically the same as the text temporal extraction method provided in the first embodiment of the present invention. Step 104 may include:

[0102] Step 401: Determine the fragment length of the instance fragment and the entity location information within the instance fragment.

[0103] Step 402: Perform nonlinear calculations on the entity location information and fragment length to obtain the position weight of the instance fragment.

[0104] In this embodiment of the invention, the segment length of the instance fragment and the entity position information within the instance fragment are determined. The entity position information includes the start and end positions of the entity. A nonlinear calculation is performed on the entity position information and the segment length to obtain the position weight of the instance fragment. Specifically, the position weight is calculated based on the relative distance between the instance fragment context and the entity. The position weight is typically a numerical value representing the influence of the distance between a word at a specified position in the instance fragment and the entity on the word's importance. It should be noted that this embodiment uses a nonlinear function to calculate the entity position information and the segment length to generate the position weight of the instance fragment. Specifically, the nonlinear function tanh is used to smoothly calculate the ratio of the relative distance between the word and the entity to the segment length.

[0105] The embodiments of the present invention employ position-weighted hidden representation, which emphasizes important information around the entity, enabling better processing of instance fragments at different locations, improving the understanding of instance fragments, and obtaining more differentiated and related instance representations.

[0106] Specifically, the position weight of the instance fragment is obtained by performing nonlinear calculations on the entity location information and the fragment length. The position weight of the instance fragment is obtained by the following formula:

[0107]

[0108] in, and Let be the position weights of the instance fragment corresponding to the i-th position, and tanh be a non-linear function. To perform upper limit operations, Len L Len C and Len R These are the fragment lengths of the left instance fragment, the middle instance fragment, and the right instance fragment, respectively.

[0109] It should be noted that in this embodiment, the position weight of the left instance fragment (right fragment) is calculated by the ratio of the relative distance between the word and the first entity to the fragment length. Therefore, the farther a word is from the entity, the lower its weight. To avoid assigning zero weights to the first word of the left instance fragment and the last word of the right fragment, this embodiment adds one to the fragment length. Since the middle instance fragment hides important information and may be very short, the word positions in the middle instance fragment are treated equally, and the position weights of all words in the middle instance fragment are smoothed out.

[0110] Furthermore, refer to Figure 5 , showed Figure 1 A flowchart of step 106 in a text temporal extraction method is provided. This method is basically the same as the text temporal extraction method provided in the first embodiment of the present invention. Step 106 may include:

[0111] Step 501: Perform pooling calculation on the second hidden representation of the intermediate instance fragment to obtain the overall representation of the intermediate instance fragment and the importance weight of the intermediate instance fragment.

[0112] In this embodiment, since intermediate instance fragments typically contain the richest temporal information between two entities, they are considered key segments. The second hidden representation of the intermediate instance fragment is pooled to obtain the overall representation of the intermediate instance fragment and its importance weight. Specifically, max pooling (·) is first used to obtain the overall representation of the intermediate instance fragment. Overall expression of intermediate instance fragments It is calculated using the following formula:

[0113]

[0114] in, These are the hidden representations of the start and end positions in the intermediate instance fragment, respectively.

[0115] It should be noted that the importance weight of intermediate instance fragments is calculated using the following formula:

[0116]

[0117] in, W represents the importance weight of intermediate instance fragments. C The weight matrix pre-trained for intermediate instance segments. This represents the hidden representation of the i-th word in the intermediate instance segment.

[0118] Step 502: Calculate the overall representation of the intermediate instance fragment to obtain the importance weights of the left and right instance fragments.

[0119] In this embodiment of the invention, the left and right instance segments are associated with the middle instance segment respectively, and a scoring function formula is defined. The hidden representation of each word in the left part and the overall representation in the middle part are used as inputs to the function to calculate the importance weight of the i-th word in the left and right instance segments. In this embodiment, the softmax(·) function can be used to calculate the regularized importance weight of the instance segment.

[0120] Specifically, using the hidden representation of each word in the left part and the overall representation in the center part as input to the function, the importance weight of the i-th word in the left instance segment is calculated:

[0121]

[0122] in, The weight matrix pre-trained for the left instance segment. Let be the hidden representation of the i-th word in the left instance segment. Let be the overall representation of the intermediate instance fragment, and tanh be a nonlinear function.

[0123] Specifically, in this embodiment, the importance weights of the fragment after regularization are calculated using the softmax(·) function:

[0124]

[0125] in, Let represent the importance weight of the i-th word in the left instance segment.

[0126] It should be noted that the calculation process for the importance weight of the i-th word in the right instance segment is the same as the calculation process for the importance weight of the left instance segment. Specifically:

[0127]

[0128] in, The weight matrix pre-trained for the right instance segment. Let be the hidden representation of the i-th word in the right instance segment. Let tanh be the overall representation of the intermediate instance fragment, and let tanh be a nonlinear function. Let represent the importance weight of the i-th word in the right instance segment.

[0129] Step 503: Using the second hidden representation and importance weights, calculate the association attention parameters of the instance fragment.

[0130] The associated attention parameters of the instance fragment are obtained using the following formula:

[0131]

[0132] Where, r C r L r R These are the attention parameters for the middle instance fragment, the left instance fragment, and the right instance fragment, respectively. These are the importance weights for the middle instance fragment, the left instance fragment, and the right instance fragment, respectively. These are the second hidden representations of the i-th word in the middle instance fragment, left instance fragment, and right instance fragment, respectively.

[0133] This invention employs an associative attention network to perform associative calculations on the hidden representations of instance fragments, thereby obtaining hidden representations that reflect the relationships between instance fragments. This further yields more differentiated and associated instance representations, improving text classification and recognition capabilities as well as the ability to extract temporal relationships.

[0134] Specifically, to avoid information redundancy and control information fusion, an information control gate is used at the output layer to control r. L r C and r R The parameters of the instance fragments are combined to form the final hidden representation r of the entire target instance. The associated attention parameters of the instance fragments are combined to form the target hidden representation of the target instance. The target hidden representation of the target instance is obtained by the following formula:

[0135]

[0136] y = softmax(W o ·r)

[0137] Where r is the associative attention combination of instance fragments, and sigmoid is a non-linear activation function. These are importance weight matrices for the middle instance fragment, left instance fragment, and right instance fragment, respectively, r. C r L r R These are the attention parameters for the middle instance fragment, left instance fragment, and right instance fragment, respectively. softmax is a non-linear function, and W... o The output weight matrix is ​​y, where y is the target hidden representation of the target instance.

[0138] To facilitate a thorough understanding of the above-described text temporal extraction method by those skilled in the art, please refer to... Figure 6 , Figure 6 This is a schematic diagram of a text temporal extraction method provided by an embodiment of the present invention, illustrating the overall process of text temporal extraction provided by the embodiment of the present invention. Specifically, it includes: constructing an association attention network, subdividing the target instance of the temporal relationship to be extracted in the text into short and long parts, the two parts having the same network structure, which will not be described in detail here; dividing each target instance into a left instance fragment, a middle instance fragment, and a right instance fragment according to the position of the entity pair; encoding these three fragments separately using a bidirectional long short-term memory network; using the association attention mechanism of the association attention network to strengthen the important information hidden in each instance fragment and reconstruct the association between the fragments; through the association attention mechanism, more differentiated and related instance fragment representations can be obtained; finally, the association attention parameters of the three instance fragments are combined and connected as a hidden representation of the entire target instance, and provided to the output layer to extract the temporal relationship between entity pairs in the text instance.

[0139] Reference Figure 8 The diagram illustrates a structural schematic of a text temporal extraction device according to an embodiment of the present invention. The device includes:

[0140] The instance acquisition module 601 is used to acquire target instances of the time sequence relationship to be extracted from the text; the target instances include entities and entity pairs.

[0141] The instance segmentation module 602 is used to divide the target instance into multiple instance segments; the instance segments include word vectors and entity location information.

[0142] Vector encoding module 603 is used to encode the word vectors into a first hidden representation of the instance fragment;

[0143] The weight determination module 604 is used to calculate the position weight of the instance fragment based on the entity position information.

[0144] Weighted processing module 605 is used to weight the first hidden representation of the instance fragment using the position weights to obtain the second hidden representation of the instance fragment;

[0145] The associated attention parameter module 606 is used to calculate the associated attention parameters of the instance fragment from the second hidden representation of the instance fragment;

[0146] The parameter processing module 607 is used to combine the associated attention parameters of the instance fragment into the target hidden representation of the target instance;

[0147] The temporal extraction module 608 is used to extract the target instance in the text temporally based on the target hiding representation to obtain the temporal relationship of the target instance in the text.

[0148] Furthermore, the instance partitioning module 602 includes:

[0149] The segmentation submodule is used to divide the target instance into multiple instance fragments based on the position of entities and entity pairs in the target instance;

[0150] The first determining submodule is used to determine the entities and their positions in the instance fragment, thereby obtaining the entity position information of the instance fragment;

[0151] The first processing submodule is used to perform vector transformation on the instance fragment to obtain the word vector corresponding to the instance fragment.

[0152] Furthermore, the vector encoding module 603 includes:

[0153] The second processing submodule is used to integrate the word vectors corresponding to the instance fragments into multi-dimensional vectors; and to encode the multi-dimensional vectors to obtain the forward hidden state and the reverse hidden state of the instance fragments.

[0154] The third processing submodule is used to concatenate the forward hidden state and the reverse hidden state to obtain the first hidden representation of the instance fragment.

[0155] Furthermore, the weight determination module 604 includes:

[0156] The second determining submodule is used to determine the segment length of the instance segment and the entity position information in the instance segment;

[0157] The first calculation submodule is used to perform nonlinear calculations on the entity location information and the segment length to obtain the position weight of the instance segment.

[0158] Furthermore, the first calculation submodule is specifically configured such that the position weight of the instance fragment is obtained using the following formula:

[0159]

[0160] in, and Let be the position weights of the instance fragment corresponding to the i-th position, and tanh be a non-linear function. To perform upper limit operations, Len L Len C and Len R These are the fragment lengths of the left instance fragment, the middle instance fragment, and the right instance fragment, respectively.

[0161] Furthermore, the instance fragment includes a left instance fragment, a middle instance fragment, and a right instance fragment, and the associated attention parameter module 606 includes:

[0162] The pooling submodule is used to perform pooling calculation on the second hidden representation of the intermediate instance fragment to obtain the overall representation of the intermediate instance fragment and the importance weight of the intermediate instance fragment.

[0163] The weighting submodule is used to calculate the overall representation of the intermediate instance fragment to obtain the importance weights of the left instance fragment and the right instance fragment;

[0164] The second calculation submodule is used to calculate the association attention parameters of the instance fragment using the second hidden representation and the importance weight; the association attention parameters of the instance fragment are obtained by the following formula:

[0165]

[0166] Where, r C r L r R These are the attention parameters for the middle instance fragment, the left instance fragment, and the right instance fragment, respectively. These are the importance weights for the middle instance fragment, the left instance fragment, and the right instance fragment, respectively. These are the second hidden representations of the i-th word in the middle instance fragment, left instance fragment, and right instance fragment, respectively.

[0167] Furthermore, the parameter processing module 607 is specifically configured such that the target hidden representation of the target instance is obtained through the following formula:

[0168]

[0169] y = softmax(W o ·r)

[0170] Where r is the associative attention combination of instance fragments, and sigmoid is a non-linear activation function. The weight matrices r represent the weights of the middle instance fragment, the left instance fragment, and the right instance fragment, respectively. C r L r R These are the attention parameters for the middle instance fragment, left instance fragment, and right instance fragment, respectively. softmax is a non-linear function, and W... o The output weight matrix is ​​y, where y is the target hidden representation of the target instance.

[0171] The text temporal extraction device provided in this invention obtains target instances of temporal relationships to be extracted from text, divides the target instances into multiple instance segments, encodes word vectors into first hidden representations of instance segments, calculates positional weights of instance segments based on entity position information, weights the first hidden representations of instance segments using positional weights to obtain second hidden representations of instance segments, calculates association attention parameters of instance segments based on the second hidden representations of instance segments, combines the association attention parameters of instance segments into target hidden representations of target instances, and performs temporal extraction of target instances in text based on target hidden representations to obtain temporal relationships of target instances in text. This invention divides target instances into multiple instance segments, encodes word vectors of instance segments to obtain hidden representations representing dependencies between instance segments, uses positional weights to weight hidden representations to emphasize important information surrounding entities, and obtains hidden representations that reflect relationships between instance segments through an association attention network. This allows for better handling of instance segments at different positions, improves the understanding of instance segments, obtains more differentiated and related instance representations, further enhances text classification and recognition levels and temporal relationship extraction capabilities, and effectively captures the temporal relationships of entities in text.

[0172] Reference Figure 9 The present invention also provides an electronic device, such as... Figure 9 As shown, it includes a processor 701, a communication interface 702, a memory 703, and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704.

[0173] Memory 703 is used to store computer programs;

[0174] When processor 701 executes a program stored in memory 703, it performs the following steps:

[0175] Obtain target instances from the text from which temporal relationships are to be extracted; the target instances include entities and entity pairs.

[0176] The target instance is divided into multiple instance segments; each instance segment includes word vectors and entity location information.

[0177] Encode the word vectors into a first hidden representation of the instance fragment;

[0178] The position weight of the instance fragment is calculated based on the entity position information;

[0179] The first hidden representation of the instance fragment is weighted using the positional weights to obtain the second hidden representation of the instance fragment;

[0180] The association attention parameters of the instance fragment are calculated from the second hidden representation of the instance fragment;

[0181] The associated attention parameters of the instance fragments are combined to form the target hidden representation of the target instance;

[0182] Based on the target hiding representation, the target instances in the text are extracted temporally to obtain the temporal relationship of the target instances in the text.

[0183] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0184] The communication interface is used for communication between the aforementioned terminal and other devices.

[0185] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0186] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0187] In another embodiment of the present invention, a computer-readable storage medium is also provided, which stores instructions that, when executed on a computer, cause the computer to perform any of the text timing extraction methods described in the above embodiments.

[0188] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0189] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0190] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0191] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for extracting temporal sequence data from text, characterized in that, The method includes: Obtain target instances from the text from which temporal relationships are to be extracted; the target instances include entities and entity pairs. The target instance is divided into multiple instance segments; each instance segment includes word vectors and entity location information. Encode the word vectors into a first hidden representation of the instance fragment; The position weight of the instance fragment is calculated based on the entity position information; The first hidden representation of the instance fragment is weighted using the positional weights to obtain the second hidden representation of the instance fragment; The association attention parameters of the instance segment are calculated by computing the second hidden representation of the instance segment, including: pooling the second hidden representation of the intermediate instance segment to obtain the overall representation and importance weight of the intermediate instance segment; calculating the importance weights of the left and right instance segments based on the overall representation of the intermediate instance segment; and calculating the association attention parameters of the instance segment using the second hidden representation and the importance weights. The instance segment includes a left instance segment, a intermediate instance segment, and a right instance segment, and the association attention parameters of the instance segment are obtained by the following formula: in, , , These are the attention parameters for the middle instance fragment, the left instance fragment, and the right instance fragment, respectively. , , These are the importance weights for the middle instance fragment, the left instance fragment, and the right instance fragment, respectively. These are the middle instance fragment, the left instance fragment, and the right instance fragment, respectively. The second hidden representation of each word; The associated attention parameters of the instance fragments are combined to form the target hidden representation of the target instance, which is obtained by the following formula: in, It is a non-linear activation function. , , These are the weight matrices for the middle instance fragment, the left instance fragment, and the right instance fragment, respectively. , , These are the attention parameters for the middle instance fragment, the left instance fragment, and the right instance fragment, respectively. It is a nonlinear function. To output the weight matrix, Target hiding representation of a target instance; Based on the target hiding representation, the target instances in the text are extracted temporally to obtain the temporal relationship of the target instances in the text.

2. The method according to claim 1, characterized in that, The target instance is divided into multiple instance segments; each instance segment includes word vectors and entity location information, including: Based on the positions of entities and entity pairs within the target instance, the target instance is divided into multiple instance fragments; Identify the entities and their positions within the instance fragment to obtain the entity position information of the instance fragment; The instance fragment is transformed into a vector to obtain the word vector corresponding to the instance fragment.

3. The method according to claim 1, characterized in that, The process of encoding the word vectors into a first hidden representation of the instance fragment includes: The word vectors corresponding to the instance fragments are integrated into a multi-dimensional vector; The multidimensional vector is encoded to obtain the forward hidden state and the reverse hidden state of the instance fragment; The forward hidden state and the reverse hidden state are concatenated to obtain the first hidden representation of the instance fragment.

4. The method according to claim 1, characterized in that, The step of calculating the position weight of the instance fragment based on the entity position information includes: Determine the segment length of the instance segment and the entity location information within the instance segment; The position weight of the instance segment is obtained by performing nonlinear calculations on the entity position information and the segment length.

5. The method according to claim 4, characterized in that, The position weight of the instance fragment is obtained by performing a nonlinear calculation on the entity location information and the fragment length. The position weight of the instance fragment is obtained by the following formula: in, , and The first The position weight of the instance fragment corresponding to each position. It is a nonlinear function. To perform the upper limit operation, , and These are the fragment lengths of the left instance fragment, the middle instance fragment, and the right instance fragment, respectively.

6. A text temporal extraction device, characterized in that, The device includes: The instance acquisition module is used to acquire target instances of the time sequence relationship to be extracted from the text; the target instances include entities and entity pairs. An instance segmentation module is used to divide the target instance into multiple instance segments; the instance segments include word vectors and entity location information. Vector encoding module, used to encode the word vectors into a first hidden representation of the instance fragment; The weight determination module is used to calculate the position weight of the instance fragment based on the entity position information. The weighted processing module is used to weight the first hidden representation of the instance fragment using the position weights to obtain the second hidden representation of the instance fragment; The association attention parameter module is used to calculate the association attention parameters of the instance fragment based on the second hidden representation of the instance fragment. This includes: pooling the second hidden representation of the intermediate instance fragment to obtain the overall representation and importance weight of the intermediate instance fragment; calculating the overall representation of the intermediate instance fragment to obtain the importance weights of the left and right instance fragments; and using the second hidden representation and the importance weights, calculating the association attention parameters of the instance fragment. The instance fragment includes a left instance fragment, a intermediate instance fragment, and a right instance fragment. The association attention parameters of the instance fragment are obtained using the following formula: in, , , These are the attention parameters for the middle instance fragment, the left instance fragment, and the right instance fragment, respectively. , , These are the importance weights for the middle instance fragment, the left instance fragment, and the right instance fragment, respectively. These are the middle instance fragment, the left instance fragment, and the right instance fragment, respectively. The second hidden representation of each word; The parameter processing module is used to combine the associated attention parameters of the instance fragments into a target hidden representation of the target instance, which is obtained by the following formula: in, It is a non-linear activation function. , , These are the weight matrices for the middle instance fragment, the left instance fragment, and the right instance fragment, respectively. , , These are the attention parameters for the middle instance fragment, the left instance fragment, and the right instance fragment, respectively. It is a nonlinear function. To output the weight matrix, Target hiding representation of a target instance; The temporal extraction module is used to extract the target instances in the text temporally based on the target hiding representation, so as to obtain the temporal relationship of the target instances in the text.

7. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute the instructions to implement the text temporal extraction method as described in any one of claims 1 to 5.

8. A readable storage medium, characterized in that, A computer program is stored on the readable storage medium, which, when executed by a processor, implements the text temporal extraction method as described in any one of claims 1 to 5.

Images