Financial event extraction method based on small samples

Abstract

The present invention discloses a method for extracting financial events based on small samples. The method determines financial events and the elements to be extracted, including trigger words and several event arguments, based on actual needs. An event element extraction model is constructed, comprising an additional data encoding module, an element classifier, a sentence reconstruction module, a reconstructed sentence encoder, a relationship classifier, and an event decoding module. The model is trained using training samples. When a sentence needs to be used for financial event extraction, the corresponding word sequence is input into the trained event element extraction model. The model predicts the elements contained in the sentence and the element pairs with trigger-word-event argument relationships. The most frequently occurring element is selected as the trigger word, and the remaining elements are selected as event arguments to construct the financial event. The present invention combines the task of extracting financial events with small samples, enabling the event extraction model to achieve performance close to that achieved using sufficient training samples, even with a small number of training samples.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence natural language processing, and more specifically, relates to a financial event extraction method based on small samples. Background Art

[0002] Although natural language processing has gradually matured in the field of financial event extraction in recent years, it often faces the dilemma of insufficient training samples in practical applications. Therefore, how to use limited and small training samples to train a high-performance event extraction model has become an important research topic. Automatic extraction of event elements can generate the following values: (1) reduce the model's dependence on data and reduce the workload of researchers in collecting data; (2) save the expenses incurred by collecting and processing data; (3) using small sample technology to train the model can save model training time and hardware costs, and at the same time can complete model training and put it into production faster, accelerating the overall deployment of the system.

[0003] Small-sample financial event extraction essentially involves leveraging the limited amount of available data, combining it with external data or increasing model parameters to improve performance when training data is insufficient. This approach aims to achieve performance comparable to that achieved with sufficient training samples. Currently, the mainstream development of small-sample technology can be categorized into the following three categories:

[0004] The first is small sample technology based on additional training data. This type of technology primarily enhances model performance during pre-training or fine-tuning of language models. The main approach is to train or fine-tune the model with task-related data before training it on data from downstream tasks, thereby improving the model's ability to encode text in that field. For example, a large amount of unlabeled financial-related text can be used to train a pre-trained model. Although this is not direct event element extraction data, data from the same financial field can provide the model with more domain knowledge. However, collecting and processing additional pre-training data adapted for specific downstream tasks first consumes a significant amount of manpower and time. Secondly, to achieve better results, it may be necessary to redesign the pre-training or fine-tuning training objectives to better enable the model to learn knowledge related to downstream tasks. This work may require extensive experimentation to arrive at the optimal experimental setup.

[0005] The second is a small-sample technique based on prompt learning. This is generally used for feature extraction given known event types. Prompts related to the event are manually summarized or the model is allowed to learn them. This allows the model to leverage the knowledge learned during pre-training through these prompts to complete the extraction task. The theoretical basis for this approach is that the extraction paradigm of traditional downstream tasks differs significantly from that of pre-training tasks, resulting in underutilization of the knowledge learned by the pre-trained model. However, prompts can enhance the model's utilization of this knowledge to a certain extent. However, prompt design for prompt learning is a major challenge. This is typically achieved through manual analysis and summary of training data patterns, which also requires significant human resources. Different prompts can significantly impact task performance. Furthermore, most prompt learning techniques are primarily suitable for generative models, which are slow to train and infer. In practice, training and prediction take longer, making generative models less efficient than extractive models.

[0006] The third approach is to improve the performance of small sample tasks by increasing the model size and parameters. Increasing the model size allows the model to learn the corpus more thoroughly. This also means that the model learns the pre-training corpus more thoroughly, learning more common knowledge across different tasks than a small model, allowing it to better complete small sample tasks. Increasing the model parameters is a simple approach, but the obvious disadvantage is that it requires extremely high additional costs, including data and training costs. Furthermore, increasing the model size may require consideration of underlying hardware interaction issues, which is also a difficulty in training large models. Summary of the Invention

[0007] The purpose of the present invention is to overcome the shortcomings of the existing technology and provide a financial event extraction method based on small samples, which combines the financial event extraction task with small samples, so that the event extraction model can use a small amount of training samples to obtain performance close to that of training with sufficient training samples.

[0008] In order to achieve the above-mentioned object of the invention, the financial event extraction method based on small samples of the present invention includes the following steps:

[0009] S1: Determine the financial events according to actual needs and determine the elements to be extracted, including trigger words and several event arguments;

[0010] Collect several sentences of the determined financial events and preprocess the collected sentences using a preset preprocessing method; extract the corresponding word sequence from the preprocessed sentences, and record the word sequence of the sentence as S = {x1, x2, ..., x N}, x i Represents the i-th word in the sentence, i = 1, 2, ..., N, N represents the number of words in the sentence;

[0011] For each sentence's word sequence, we use the BIO tag sequence tagging method to annotate the trigger words and event arguments. We then pair the trigger words with each event argument. The word sequence is then used as input, and the trigger word-event argument pair is used as the corresponding event label to form a training sample.

[0012] S2: Construct an event feature extraction model, including an additional data encoding module, a feature classifier, a sentence reconstruction module, a reconstructed sentence encoder, a relation classifier, and an event decoding module, where:

[0013] The additional data encoding module is used to encode the word sequence S={x1,x2,…,x N}, the preset element data is integrated into the fusion matrix F and sent to the element classifier. The specific method of fusing the statement and element data is as follows:

[0014] For sentences that need event extraction, its word sequence S={x1,x2,…,x N Each word in} is encoded separately to obtain its hidden layer representation Where d represents the dimension of the preset hidden layer representation, and the hidden layer representation of each word is constructed as a row vector to obtain the hidden layer representation matrix of the sentence

[0015] Let the number of factors involved in a financial event be M, and encode the M factors separately to obtain their hidden layer representations The hidden layer representation of each element is constructed as a row vector to obtain the hidden layer representation matrix of the element Then copy the hidden layer representation matrix N times to construct a three-dimensional element matrix

[0016] The following formula is used to fuse the element matrix with the hidden layer representation matrix of the sentence to obtain the fusion matrix

[0017] F=E′DX T

[0018] in, is a learnable parameter matrix, and the superscript T indicates transpose;

[0019] The feature classifier is used to slice the fusion matrix F according to the third dimension to obtain N matrices The matrix f i Classify the i-th word in the input sentence as a feature, predict whether the word is an element of a financial event, and send the word classification result to the sentence reconstruction module;

[0020] The sentence reconstruction module is used to reconstruct the sentence according to the word classification results. The specific method is: according to the word classification results, the word sequence S={x1,x2,…,x N}, the number of elements screened out is K, and the kth element is p k , then combine the K elements in pairs to obtain K(K-1) element pairs (p k ,p k′ ), k, k′=1,2,…,K and k′≠k; each element pair (p k ,p k′ ) and word sequence S={x1,x2,…,x N} splicing to obtain the word sequence S k,k′ ={p k ,p k′ ,x1,x2,…,x N} and output to the reconstructed sentence encoder;

[0021] The reconstruction sentence encoder is used to reconstruct the K(K-1) word sequences S k,k′ Encode and get the corresponding hidden layer representation R k,k′ and sent to the relation classifier;

[0022] The relational classifier is used to represent the K(K-1) hidden layers R k,k′ Classify and get the corresponding element pairs (p k ,p k′ ) is a trigger word-event argument relationship, and sends the classification result to the event decoding module;

[0023] The event decoding module is used to complete event decoding based on the classification results of feature pairs. The specific method is as follows: from the feature pairs determined to be trigger word-event argument relationships, the same feature is selected as the trigger word, and the other features are used as event arguments to form a financial event and output it;

[0024] S3: Take the word sequence in the training sample of step S1 as input and the corresponding event label as the expected output to train the event element extraction model;

[0025] S4: When it is necessary to extract financial events from a certain sentence, obtain the word sequence S′ of the sentence and input it into the event element extraction model trained in step S3 to predict the elements contained in the sentence and the element pairs with trigger word-event argument relationships. From the element pairs that are determined to have trigger word-event argument relationships, select the element with the highest frequency of occurrence as the trigger word and the other elements as event arguments to constitute a financial event.

[0026] The present invention is based on a small sample financial event extraction method. It determines financial events according to actual needs and determines the elements that need to be extracted, including trigger words and several event arguments. Then, it collects several training samples and constructs an event element extraction model including an additional data encoding module, an element classifier, a sentence reconstruction module, a reconstructed sentence encoder, a relationship classifier and an event decoding module. The event element extraction model is trained using the training samples. When it is necessary to extract financial events from a certain sentence, the word sequence of the sentence is obtained and input into the trained event element extraction model. The elements contained in the sentence and the element pairs with the trigger word-event argument relationship are predicted. From the element pairs determined to be the trigger word-event argument relationship, the element with the highest frequency of occurrence is screened out as the trigger word and the other elements as the event arguments to constitute a financial event.

[0027] The present invention reasonably divides the event extraction task so that the extractive model and the event extraction task can be adapted to each other. At the same time, a prompt learning paradigm for adapting the extractive model is proposed, thereby reducing the sample quantity requirement of the event element extraction model and achieving high-accuracy event element extraction even in small sample conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is a flowchart of a specific implementation of the financial event factor extraction system based on a small sample size of the present invention;

[0029] Figure 2 It is a structural diagram of the event element extraction model in the present invention;

[0030] Figure 3 This is an example diagram of the results of element identification and relationship classification in this embodiment. DETAILED DESCRIPTION

[0031] The following describes the specific embodiments of the present invention in conjunction with the accompanying drawings so that those skilled in the art can better understand the present invention. It should be noted that in the following description, when detailed descriptions of known functions and designs may dilute the main content of the present invention, such descriptions will be omitted here.

[0032] Example

[0033] Figure 1 This is a flowchart of a specific implementation of the financial event factor extraction system based on a small sample of the present invention. Figure 1 As shown, the specific steps of the financial event factor extraction method based on a small sample of the present invention include:

[0034] S101: Get training samples:

[0035] Determine financial events based on actual needs and determine the elements that need to be extracted, including trigger words and several event arguments.

[0036] Collect several statements of the identified financial events and pre-process the collected statements using a preset pre-processing method, such as data cleaning. Extract the corresponding word (token) sequence from the pre-processed statements and record the word (token) sequence of the statement as S = {x1, x2, ..., x N}, x i Represents the i-th word in the sentence, i = 1, 2, ..., N, and N represents the number of words in the sentence.

[0037] For each sentence's word sequence, the BIO tag sequence tagging method is used to annotate the trigger words and event arguments. The trigger words are paired with each event argument. The word sequence is then used as input, and the trigger word-event argument pair is used as the corresponding event label to form a training sample.

[0038] BIO tag sequence annotation is a joint annotation method that labels each element as "XB," "XI," or "O." "XB" indicates that the element's segment belongs to type X and is at the beginning of the segment; "XI" indicates that the element's segment belongs to type X and is in the middle of the segment; and "O" indicates that it does not belong to any type. For example, in the case of a pledge event, the trigger word "pledge" needs to be extracted; the following event arguments need to be extracted: disclosure time, pledger, pledged company, pledged assets, number of pledged stocks / shares, pledged shareholding ratio, pledged shareholding ratio, and event time. After BIO tag sequence annotation, for example, "Pledgor-B" indicates that the current term is the beginning of a pledger-type entity, while "Pledgor-I" indicates that the current term is not the beginning of a pledger-type entity.

[0039] S102: Build event element extraction model:

[0040] This invention transforms the event extraction task into an "entity recognition" + "relationship recognition" approach. First, the trigger word (trigger) is treated as an element for prediction, just like the event argument. Furthermore, to correctly combine the trigger word with the corresponding event argument, a dimension of information is required to link the trigger word and event argument. This invention constructs a "TA" (Trigger-Argument) relationship between the trigger word and all other event arguments in an event, indicating the connection between each trigger word and event argument. For example, the original prediction target is a pledge event: {pledge (trigger word), company, shareholding ratio of the pledged item}. This invention breaks the event down into an entity recognition task, requiring identification of the following entities: {pledge, company, shareholding ratio of the pledged item}; and a relationship recognition task, requiring identification of the following relations: {(TA, pledge, company), (TA, pledge, shareholding ratio of the pledged item)}. Thus, the event element extraction task is transformed into an entity recognition task plus a relationship extraction task.

[0041] Figure 2 This is the structural diagram of the event element extraction model in the present invention. Figure 2 As shown, the small sample event feature extraction model of the present invention includes an additional data encoding module, a feature classifier, a sentence reconstruction module, a reconstructed sentence encoder, a relationship classifier and an event decoding module. Each module is described in detail below.

[0042] The additional data encoding module is used to encode the word sequence S={x1,x2,…,x N}, the preset element data is integrated into the fusion matrix F and sent to the element classifier. The specific method of fusing the statement and element data is as follows:

[0043] In feature extraction, each feature type can be characterized by a corresponding natural language text (ie, description). In the present invention, the representation vector of the description is used as additional information to assist the extraction model in encoding the contextual hidden layer representation.

[0044] For sentences that need event extraction, its word sequence S={x1,x2,…,x N Each word in} is encoded separately to obtain its hidden layer representation Where d represents the dimension of the preset hidden layer representation, and the hidden layer representation of each word is constructed as a row vector to obtain the hidden layer representation matrix of the sentence

[0045] Let the number of factors involved in a financial event be M, and encode the M factors separately to obtain their hidden layer representations The hidden layer representation of each element is constructed as a row vector to obtain the hidden layer representation matrix of the element Then copy the hidden layer representation matrix N times to construct a three-dimensional element matrix

[0046] The following formula is used to fuse the element matrix with the hidden layer representation matrix of the sentence to obtain the fusion matrix

[0047] F=E′DX T

[0048] in, is a learnable parameter matrix, and the superscript T indicates transpose.

[0049] The feature classifier is used to slice the fusion matrix F according to the third dimension to obtain N matrices The matrix f i The i-th word in the input sentence is used as a feature to classify the i-th word, predict whether the word is an element of a financial event, and send the word classification result to the sentence reconstruction module.

[0050] The sentence reconstruction module is used to reconstruct sentences based on the word classification results. The specific method is as follows:

[0051] According to the word classification results, the word sequence S={x1,x2,…,x N}, the number of elements screened out is K, and the kth element is p k , then combine the K elements in pairs to obtain K(K-1) element pairs (p k ,p k′ ), k, k′=1,2,…,K and k′≠k. k ,p k′ ) and word sequence S={x1,x2,…,x N} splicing to obtain the word sequence S k,k′ ={p k ,p k′ ,x1,x2,…,x N} and output to the reconstructed sentence encoder.

[0052] The reconstruction sentence encoder is used to reconstruct the K(K-1) word sequences S k,k′ Encode and get the corresponding hidden layer representation R k,k′ And sent to the relation classifier.

[0053] The relational classifier is used to represent the K(K-1) hidden layers R k,k′ Classify and get the corresponding element pairs (p k ,p k′ ) is a trigger word-event argument relationship, and the classification result is output.

[0054] Figure 3 This is an example diagram of the results of element identification and relationship classification in this embodiment. Figure 3 As shown in the figure, if there is a trigger word-event argument relationship between two elements, then the element is considered to belong to the event argument indicated by the trigger word. The trigger word-event argument relationship must be a relationship between an event argument and a trigger word. Depending on the data processing method, the trigger word can be placed before or after ( Figure 3 In the example above, the trigger word comes after the trigger word, which is a one-way relationship. Therefore, according to different trigger words, the event arguments related to them can be decoded, and the event extraction task is finally completed.

[0055] S103: Training event feature extraction model:

[0056] The word sequence in the training sample of step S101 is used as input and the corresponding event label is used as the expected output to train the event element extraction model.

[0057] In the present invention, since there are two classifiers, namely the feature classifier and the relationship classifier, loss functions are designed for the two classifiers during the event feature extraction model training process. The loss function calculation formula of the feature classifier is:

[0058]

[0059] Among them, N ent Indicates the number of words in the training sample word sequence, y n Indicates whether the nth word is the true label of the element. When y n =1 means the nth word is an element. n =0 means the nth word is not a factor, Indicates whether the nth word obtained by the feature classifier is a feature label.

[0060] The loss function calculation formula of the relation classifier is:

[0061]

[0062] Among them, N rel Indicates the number of feature pairs composed of the features obtained by the feature classifier in the training sample, z n′ Indicates whether the n′th element pair has a true label of a trigger word-event argument relationship. When z n′ =1 indicates that the n′th element pair has a trigger word-event argument relationship. n′ =0 means that there is no trigger word-event argument relationship for the n′th element pair. Indicates the true label of whether the n′th element pair obtained by the relationship classifier has a trigger word-event argument relationship.

[0063] When training the event feature extraction model, the above two loss functions are used to alternately train the event feature extraction model, thereby training the feature classification task and the relationship classification task respectively.

[0064] S104: Financial event extraction:

[0065] When it is necessary to extract financial events from a certain sentence, the word sequence S′ of the sentence is obtained and input into the event element extraction model trained in step S103 to predict the elements contained in the sentence and the element pairs with trigger word-event argument relationships. From the element pairs that are determined to have trigger word-event argument relationships, the element with the highest frequency of occurrence is selected as the trigger word and the other elements as event arguments to constitute a financial event.

[0066] In order to improve the accuracy of extracting financial event requirements, before using the event element extraction model to extract event elements from the word sequence S′, the word sequence of the W samples with the highest similarity to the word sequence can be screened from the training samples. The value of W is determined according to the actual situation. The event element extraction model is fine-tuned using W samples, and then the word sequence S′ is input into the fine-tuned event element extraction model for event element extraction. When calculating the similarity of the word sequence, the word sequence S′ and the word sequence in the training sample can be encoded separately to obtain the hidden layer representation, and then the Euclidean distance of the hidden layer representation can be calculated. The smaller the distance, the greater the similarity. It can be seen that this method completes the retrieval of the training samples through the nearest neighbor algorithm, which is simple to implement and has good retrieval effect.

[0067] In actual applications, the financial events obtained by the event decoding module may have unpredicted or incomplete predictions. Therefore, the constructed financial events can also be supplemented. The specific method is as follows:

[0068] The elements in the element pairs that have not been determined to have a trigger word-event argument relationship are combined into a set of pending elements. A check is performed to determine whether the trigger word for the financial event exists in this set. If so, the trigger word and the detected event argument are combined to form a supplementary trigger word-event argument pair. Otherwise, no action is taken. The constructed financial event is completed using all the supplementary trigger word-event argument pairs obtained.

[0069] Although the above describes the illustrative specific embodiments of the present invention to facilitate understanding of the present invention by those skilled in the art, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations using the concepts of the present invention are protected.

Claims

1. A financial event extraction method based on small samples, characterized in that: The following steps are involved: S1: Determine the financial events according to actual needs and determine the elements to be extracted, including trigger words and several event arguments; Collect several sentences of the determined financial events and preprocess the collected sentences using a preset preprocessing method; extract the corresponding word sequence from the preprocessed sentences, and record the word sequence of the sentence as S = {x1, x2, ..., x N }, x i Represents the i-th word in the sentence, i = 1, 2, ..., N, N represents the number of words in the sentence; For each sentence's word sequence, we use the BIO tag sequence tagging method to annotate the trigger words and event arguments. We then pair the trigger words with each event argument. The word sequence is then used as input, and the trigger word-event argument pair is used as the corresponding event label to form a training sample. S2: Construct an event feature extraction model, including an additional data encoding module, a feature classifier, a sentence reconstruction module, a reconstructed sentence encoder, a relation classifier, and an event decoding module, where: The additional data encoding module is used to encode the word sequence S={x1,x2,…,x N }, the preset element data is integrated into the fusion matrix F and sent to the element classifier. The specific method of fusing the statement and element data is as follows: For sentences that need event extraction, its word sequence S={x1,x2,…,x N Each word in} is encoded separately to obtain its hidden layer representation Where d represents the dimension of the preset hidden layer representation, and the hidden layer representation of each word is constructed as a row vector to obtain the hidden layer representation matrix of the sentence Let the number of factors involved in a financial event be M, and encode the M factors separately to obtain their hidden layer representations The hidden layer representation of each element is constructed as a row vector to obtain the hidden layer representation matrix of the element Then copy the hidden layer representation matrix N times to construct a three-dimensional element matrix The following formula is used to fuse the element matrix with the hidden layer representation matrix of the sentence to obtain the fusion matrix F=E′DX T in, is a learnable parameter matrix, and the superscript T indicates transpose; The feature classifier is used to slice the fusion matrix F according to the third dimension to obtain N matrices The matrix f i Classify the i-th word in the input sentence as a feature, predict whether the word is an element of a financial event, and send the word classification result to the sentence reconstruction module; The sentence reconstruction module is used to reconstruct the sentence according to the word classification results. The specific method is: according to the word classification results, the word sequence S={x1,x2,…,x N }, the number of elements screened out is K, and the kth element is p k , then combine the K elements in pairs to obtain K(K-1) element pairs (p k ,p k′ ), k, k′=1,2,…,K and k′≠k; each element pair (p k ,p k′ ) and word sequence S={x1,x2,…,x N } splicing to obtain the word sequence S k,k′ ={p k ,p k′ ,x1,x2,…,x N } and output to the reconstructed sentence encoder; The reconstruction sentence encoder is used to reconstruct the K(K-1) word sequences S k,k′ Encode and get the corresponding hidden layer representation R k,k′ and sent to the relation classifier; The relational classifier is used to represent the K(K-1) hidden layers R k,k′ Classify and get the corresponding element pairs (p k ,p k′ ) is a trigger word-event argument relationship, and sends the classification result to the event decoding module; The event decoding module is used to complete event decoding based on the classification results of feature pairs. The specific method is as follows: from the feature pairs determined to be trigger word-event argument relationships, the same feature is selected as the trigger word, and the other features are used as event arguments to form a financial event and output it; S3: Take the word sequence in the training sample of step S1 as input and the corresponding event label as the expected output to train the event element extraction model; S4: When it is necessary to extract financial events from a certain sentence, obtain the word sequence S′ of the sentence and input it into the event element extraction model trained in step S3 to predict the elements contained in the sentence and the element pairs with trigger word-event argument relationships. From the element pairs that are determined to have trigger word-event argument relationships, select the element with the highest frequency of occurrence as the trigger word and the other elements as event arguments to constitute a financial event.

2. The financial event extraction method according to claim 1, characterized in that: The loss function used in the event element extraction model training in step S3 is as follows: The loss function calculation formula of the feature classifier is: Among them, N ent Indicates the number of words in the training sample word sequence, y n Indicates whether the nth word is the true label of the element. When y n =1 means the nth word is an element. n =0 means the nth word is not a factor, Indicates whether the nth word obtained by the feature classifier is a feature label; The loss function calculation formula of the relation classifier is: Among them, N rel Indicates the number of feature pairs composed of the features obtained by the feature classifier in the training sample, z n′ Indicates whether the n′th element pair has a true label of a trigger word-event argument relationship. When z n′ =1 indicates that the n′th element pair has a trigger word-event argument relationship. n′ =0 means that there is no trigger word-event argument relationship for the n′th element pair. Indicates the true label of whether the n′th element pair obtained by the relationship classifier has a trigger word-event argument relationship; When training the event feature extraction model, the above two loss functions are used to alternately train the event feature extraction model, thereby training the feature classification task and the relationship classification task respectively.

3. The financial event extraction method according to claim 1, characterized in that: In step S4, before extracting event elements from the word sequence S′ using the event element extraction model, the word sequences of W samples with the highest similarity to the word sequence are selected from the training samples, where the value of W is determined according to the actual situation; W samples are used to fine-tune the event element extraction model, and then the word sequence S′ is input into the fine-tuned event element extraction module to extract event elements.

4. The financial event extraction method according to claim 3, characterized in that: The similarity of the word sequence is calculated by the following method: the word sequence S′ and the word sequence in the training sample are respectively encoded to obtain hidden layer representations, and then the Euclidean distance of the hidden layer representations is calculated. The smaller the distance, the greater the similarity.

5. The financial event extraction method according to claim 1, characterized in that: The step S4 also includes completing the constructed financial event, and the specific method is as follows: The elements in the element pairs that are not determined to have a trigger word-event argument relationship are used to form a pending element set, and it is determined whether there is a trigger word preset for the financial event in the pending element set. If so, the trigger word and the detected event argument are used to form a supplementary trigger word-event argument pair, otherwise no operation is performed; the constructed financial event is completed based on all the obtained supplementary trigger word-event argument pairs.

Images