A method for extracting and classifying financial text relations by integrating regular expressions

Abstract

The present invention discloses a method for extracting and classifying financial text relationships by integrating regular expressions. The method encodes the financial text, identifies the relationship entity pairs contained in the text, and marks them; matches the financial text with each regular expression in turn, converts the matching results into regular expression labels, and finally integrates the labels with the marking results, and extracts and classifies the relationships between entity pairs through a deep neural network. The present invention realizes end-to-end relationship extraction and classification of financial texts that integrate regular expression information, with a clear logical structure and distinct levels, without the need for manual labeling, saving labor costs, improving work efficiency, high accuracy, and strong practicality.

Description

Technical Field

[0001] The present invention belongs to the intersection of natural language processing and finance, and particularly relates to a financial text relationship extraction and classification method integrating regular expressions. Background Art

[0002] The innovation and development of technologies such as natural language processing have greatly improved the scientific and technological productivity of the financial industry. Automatically extracting and classifying the relationships between major financial events from texts such as financial news and announcements can effectively save the labor costs of practitioners and significantly improve work efficiency.

[0003] Currently, the known text relationship extraction and classification technologies in the financial field have the following shortcomings:

[0004] 1) Existing methods for extracting relationships from financial texts require manual annotation of the financial texts before classifying the relationships. However, in actual scenarios, the original financial texts are unannotated, which greatly limits the application of these methods in practice.

[0005] 2) Existing methods for relationship extraction in the financial field based on deep neural networks cannot effectively utilize rule information that conforms to human logic, and cannot capture some important features in financial texts, which affects the accuracy of relationship extraction. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention proposes a financial text relationship extraction and classification method integrating regular expressions. The method of the present invention relies on logical prior knowledge and integrates regular expression information to improve the accuracy of financial text relationship classification.

[0007] The technical solution adopted by the present invention to solve the technical problem is: a financial text relationship extraction and classification method integrating regular expressions, which is performed in the financial text relationship extraction process according to the following steps:

[0008] (1) Encode the financial text in the financial text library, convert each word in the financial text into a word vector, annotate each word vector through a deep neural network, identify the relationship entity pairs contained in the financial text, mark the relationship entity pairs, and obtain the annotation results;

[0009] (2) Match the financial text with each regular expression in the regular expression database in turn, convert the matching results into regular expression labels of the financial text, input the relationship entity pair labeling results obtained in step (1) into a bidirectional long short-term memory network to extract features;

[0010] (3) The regular expression label obtained in step (2) is concatenated with the output feature of the bidirectional long short-term memory network obtained in step (2), and the financial text relationship classification is completed through the deep neural network. The final financial text relationship extraction result is obtained by querying the relationship type corresponding to the relationship type index C.

[0011] Furthermore, the specific process of step (1) is as follows:

[0012] (1.1) Take a financial text S from the financial text library, segment it, and obtain the financial text represented in words:

[0013]

[0014] Among them, s represents the classified words of financial text, and n is the total number of words;

[0015] (1.2) Use the Embedding mapping method to map the word-classified financial text S into a high-dimensional word vector set E:

[0016]

[0017] The high-dimensional word vector set ,in Represents the word vector corresponding to each word s in the financial text;

[0018] (1.3) The word vector set is passed through a deep neural network consisting of BERT, a fully connected neural network FC1, and a conditional random field CRF to obtain the entity pair annotation result L:

[0019] .

[0020] Furthermore, the specific process of step (2) is as follows:

[0021] (2.1) Match the financial text with the regular expressions in the regular expression library in sequence. After a successful match, convert the match result into a vector consisting of 0 and 1, which is the regular expression label RElabel;

[0022] (2.2) The regular expression label is used as the input of a fully connected neural network, and after passing through the fully connected neural network, it is converted into a regular expression feature, and the regular expression feature is integrated into the deep neural network:

[0023]

[0024] Among them, REfeature represents the obtained regular expression feature, FC2 represents the fully connected neural network in this module used to convert regular expression labels into regular expression features;

[0025] (2.3) The annotation result L obtained in step (1.3) is input into the bidirectional long short-term memory network to obtain the features extracted from the annotation result :

[0026]

[0027] in is the output of the bidirectional long short-term memory network.

[0028] Furthermore, the specific process of step (3) is as follows:

[0029] (3.1) Concatenate the bidirectional long short-term memory network output features and regular expression features to obtain the overall features :

[0030] ;

[0031] (3.2) The overall characteristics Send it to the attention layer and get the output of the attention layer , the specific calculation process is as follows:

[0032]

[0033] Among them, tanh is the tanh activation function, softmax is the softmax function, is a trainable parameter vector, represents its transpose, T is the length of the text;

[0034] (3.3) The output of the attention layer After inputting the fully connected classification network and performing the softmax operation, the classification probability of each relationship type can be obtained. The category corresponding to the maximum probability is taken as the classification result. The specific operation process is as follows:

[0035]

[0036]

[0037] Among them, FC3 represents a fully connected classification network. represents the classification probability of each relationship type, m is the number of relationship categories, argmax represents the function of the relationship type index corresponding to the maximum value, and C represents the obtained relationship type index value;

[0038] (3.4) By querying the relationship type corresponding to the relationship type index value C, the financial relationship extraction result is obtained.

[0039] The technical concept of the present invention is as follows: encoding the text in the financial text library, converting each word in the text into a word vector, and then using the encoded features to annotate each word vector using a deep neural network, identifying the relationship entity pairs contained in the text, and marking the entities; matching the financial text with each regular expression in the regular expression library in turn, converting the matching result into a regular expression label of the financial text, and combining the label with the result of relationship entity extraction, and classifying the relationship between the two relationship entities through a deep neural network.

[0040] The beneficial effects of the present invention are mainly manifested in: 1. Relation entity pairs and relationship categories can be obtained without manual labeling. This end-to-end relationship extraction form is highly practical and has high application value; 2. The integration of regular expression information can capture rule features, improve extraction and classification performance, and meet the requirements for high accuracy of relationship extraction and classification in application scenarios; 3. Successfully identify and label entity pairs to form a large number of labeled financial texts, which provides valuable data resources for other relationship extraction systems that require labeled texts or other natural language processing technologies used for financial texts. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a schematic diagram of relational entity extraction proposed by the present invention;

[0042] Figure 2 This is a schematic diagram of the relationship classification proposed by the present invention. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present invention are described clearly and completely below in conjunction with the accompanying drawings of the embodiments of the present invention.

[0044] The present invention proposes a financial text relationship extraction and classification method integrating regular expressions. Figure 1 This is a schematic diagram of the relational entity extraction proposed in the present invention. Figure 2 This is a schematic diagram of the relationship classification proposed by the present invention, which specifically includes the following implementation steps:

[0045] (1) Encoding the financial text in the financial text library, converting each word in the financial text into a word vector, annotating each word vector through a deep neural network, identifying the relationship entity pairs contained in the financial text, and marking each relationship entity pair; the financial text stored in the financial text library is a sentence containing a certain financial relationship.

[0046] It includes the following sub-steps:

[0047] (1.1) Take a financial text from the financial text library, segment it into words, and obtain the financial text S represented in words:

[0048] 。

[0049] Among them, s represents the segmented words, and n is the total number of words. For example, let , after word segmentation, it becomes , then n = 9, and each segmented word corresponds to 。

[0050] (1.2) Use the Embedding mapping method to map the segmented financial text S into a high-dimensional word vector set E:

[0051] 。

[0052] The said high-dimensional word vector set , in this example, n = 9, where represents the word vector corresponding to each word s in the financial text.

[0053] (1.3) Pass the word vector set through a deep neural network composed of an encoder BERT (Bidirectional Encoder Representations from Transformers, BERT) composed of a bidirectional Transformer shown in Figure 1 , a fully connected neural network (Fully Connected neural network, FC), and a conditional random field (Conditional Random Field, CRF) to obtain the entity pair annotation result L:

[0054] 。

[0055] Among them, is the annotation result described represents a non-relationship entity, such as "yesterday", "acquisition", "thirty percent", "of", and "equity". represents relationship entity 1, represents relationship entity 2, and together form a relationship entity pair. Each relationship entity consists of one or more words. For example, entity 1 is composed of the words "company" and "A", that is = "Company A", and entity 2 is composed of the words "company" and "B", that is = "Company B".

[0056] (2) Matching the financial text with each regular expression in a regular expression database in turn, wherein the regular expression database includes regular expressions that can match the financial text, converting the matching result into a regular expression label of the financial text, and combining the regular expression label with the relationship entity pair labeling result obtained in step (1), and extracting and classifying the relationship between the two relationship entities through a deep neural network.

[0057] It includes the following sub-steps:

[0058] (2.1) The financial text S is matched with the regular expressions in the regular expression library in sequence. After the match is successful, the matching result can be converted into a vector consisting of 0 and 1, which is the regular expression label RElabel. Optionally, 10 common financial entity relationships are given here, as shown in Table 1, corresponding to symbols C1-C10 respectively:

[0059] Table 1: Common financial entity relationship table

[0060] C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 Staking Shareholders increase their holdings Shareholder reduction Company listing Company bankruptcy Corporate Acquisition Corporate Finance Executive Changes Being interviewed Profit and Loss

[0061] Since writing regular expressions for each relationship type is a tedious and time-consuming task, and the written regular expressions may not be applicable to new corpora and have poor generalization ability, there is no need to use the matching results of regular expressions for accurate classification. Only the matching results of regular expressions are used to obtain some information that is helpful for the relationship classification task. Specifically, a regular expression in the regular expression database can match more than one relationship type. For example, the financial text S in this example can be matched by the regular expression C2_C6, and the definition of C2_C6 is: the relationship type that can be matched by this regular expression is likely to be a C2 or C6 relationship type. Of course, it is also possible that neither of these two relationship types belongs to the same. As mentioned earlier, we do not need to use regular expressions to accurately classify text relationship types. We only need to let it provide some prior rule features. The final classification task is still handed over to the deep neural network in the module.

[0062] After being matched by the regular expression C1_C6, it is converted into a vector with 1s in the corresponding positions of C2 and C6 and 0s in the other positions as the regular expression label RElabel corresponding to the text S.

[0063] (2.2) The regular expression label is used as the input of a fully connected neural network, and the fully connected neural network FC2 is as follows: Figure 2 As shown, the regular expression feature REfeature is obtained after being converted through the fully connected neural network FC2, and the regular expression feature is integrated into the deep neural network:

[0064]

[0065] Among them, REfeature represents the obtained regular expression feature, and FC2 represents the fully connected neural network in this module used to convert regular expression labels into regular expression features.

[0066] (2.4) The annotation result L obtained in step (1.3) is input as follows Figure 2 In the bidirectional long short-term memory network BiLSTM in , the features extracted from the annotation results are obtained :

[0067]

[0068] in, It is the output feature of the bidirectional long short-term memory network BiLSTM.

[0069] (3) The regular expression label obtained in step (2) is concatenated with the output feature of the bidirectional long short-term memory network BiLSTM obtained in step (2), and the financial text relationship classification is completed through the deep neural network. The final financial text relationship extraction result is obtained by querying the relationship type corresponding to the relationship type index C.

[0070] The specific steps include:

[0071] (3.1) Output features of the bidirectional long short-term memory network BiLSTM Combined with the regular expression feature REfeature to obtain the overall feature :

[0072] .

[0073] (3.2) The overall characteristics Send in Figure 2 The attention layer in gets the output of the attention layer , the specific calculation process is as follows:

[0074]

[0075] Among them, tanh is the tanh activation function, softmax is the softmax function, is a trainable parameter vector, Represents its transpose.

[0076] (3.3) The output of the attention layer Send in Figure 2The fully connected classification network FC3 in the example can be used to obtain the classification probability of each relationship type after softmax operation. The category corresponding to the maximum probability is taken as the classification result. The specific operation process is as follows:

[0077]

[0078]

[0079] in represents the classification probability of each relationship type, m is the number of relationship categories, in this case m is 10, argmax represents the function of the relationship type index corresponding to the maximum value, and C represents the obtained relationship type index value.

[0080] (3.4) By querying the relationship type corresponding to the relationship type index C, the final financial relationship extraction result is obtained. The probability is the largest, that is, C=6, so the relationship entity pair in the final financial text S is "Company A" and "Company B", and the financial relationship type is C6, that is, "Company Acquisition". So far, the relationship extraction and classification of the financial text S is realized through the financial text relationship extraction and classification method integrating regular expressions proposed by the present invention.

[0081] The above embodiments are used to illustrate the present invention rather than to limit the present invention. Any modification and change made to the present invention within the spirit of the present invention and the protection scope of the claims shall fall within the protection scope of the present invention.

Claims

1. A financial text relationship extraction and classification method integrating regular expressions, characterized in that: The following steps are involved: (1) Encode the financial text in the financial text library, convert each word in the financial text into a word vector, annotate each word vector through a deep neural network, identify the relationship entity pairs contained in the financial text, mark the relationship entity pairs, and obtain the annotation results; (2) Matching the financial text with each regular expression in the regular expression database in sequence, converting the matching result into a regular expression label of the financial text, inputting the labeling result obtained in step (1) into a bidirectional long short-term memory network, and the bidirectional long short-term memory network outputs features; Specifically, it includes: taking the regular expression label as the input of a fully connected neural network, converting it into a regular expression feature after passing through the fully connected neural network, and integrating the regular expression feature into a deep neural network. The expression is as follows: ； Among them, REfeature represents the obtained regular expression feature, FC2 represents the fully connected neural network used to convert regular expression labels into regular expression features, and RElabel represents the regular expression label; Input the labeling result L into the bidirectional long short-term memory network to obtain the features extracted from the labeling result : ； in is the output of the bidirectional long short-term memory network; (3) Concatenate the regular expression label obtained in step (2) with the output feature of the bidirectional long short-term memory network. The expression is as follows: ； In the formula, Indicates overall characteristics: The overall characteristics Send it to the attention layer and get the output of the attention layer , the specific calculation process is as follows: ； Among them, tanh is the tanh activation function, softmax is the softmax function, is a trainable parameter vector, represents its transpose; The output of the attention layer After inputting the fully connected classification network and performing the softmax operation, the classification probability of each relationship type can be obtained. The category corresponding to the maximum probability is taken as the classification result. The specific operation process is as follows: ； ； Among them, FC3 represents a fully connected classification network. represents the classification probability of each relationship type, m is the number of relationship categories, argmax represents the function of the relationship type index corresponding to the maximum value, and C represents the obtained relationship type index value; By querying the relationship type corresponding to the relationship type index value C, the financial relationship extraction result is obtained.

2. The method for extracting and classifying financial text relations by integrating regular expressions as claimed in claim 1, characterized in that: The specific process of step (1) is as follows: (1.1) Take a financial text from the financial text library, segment it into words, and obtain the financial text S represented in words: ； Among them, s represents the classified words of financial text, and n is the total number of words; (1.2) Use the Embedding mapping method to map the word-classified financial text S into a high-dimensional word vector set E: ； The high-dimensional word vector set ,in Represents the word vector corresponding to each word s in the financial text; (1.3) The word vector set is passed through a deep neural network consisting of BERT, a fully connected neural network FC1, and a conditional random field CRF to obtain the entity pair annotation result L: 。 3. A financial text relationship extraction and classification method integrating regular expressions as described in claim 2, characterized in that: The process of matching the financial text with each regular expression in the regular expression database in sequence and converting the matching result into a regular expression label of the financial text includes: The financial text S is matched with the regular expressions in the regular expression library in sequence. After a successful match, the matching result is converted into a vector consisting of 0 and 1, which is the regular expression label RElabel.

Images