Financial text recognition method, device, computer equipment and storage medium

Abstract

The present application relates to a financial text recognition method, device, computer device, storage medium, and computer program product, which can be used in the field of artificial intelligence technology. The method includes: obtaining a financial text to be recognized and an entity set of the financial text to be recognized; determining a target vector of the financial text to be recognized and a target vector of each entity in the entity set through a pre-trained financial text recognition model; respectively inputting the target vector of each entity and the target vector of the financial text to be recognized into a first key entity recognition model to obtain an entity label for each entity; the entity label is used to characterize whether the corresponding entity is a key entity; the first key entity recognition model is obtained by retraining the financial text recognition model; screening out the entities corresponding to the key entity labels from the entity set as the key entities of the financial text to be recognized. By using this method, the recognition accuracy of key entities in financial texts can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular, to a financial text recognition method, device, computer device, storage medium, and computer program product. Background Art

[0002] With the transformation from traditional finance to Internet finance, the financial information in the network is becoming more and more complex, and more and more portal websites and social media will report financial news and views. Identifying key entities in financial texts on the Internet is beneficial to timely grasp the key entities of financial texts.

[0003] In traditional technologies, generally, a deep neural network model is used to process entity recognition tasks; however, in financial texts, due to a wide variety of specific financial entities in the financial field, such as accounts, credit cards, network points, card types, etc. Among these specific financial entities, many entities are redundant or irrelevant to the theme of the financial text. Through the deep neural network model, entities in the financial text can be recognized, but not all of these entities are key entities, and manual screening is required, resulting in a low accuracy rate for identifying key entities in financial texts. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide a financial text recognition method, device, computer device, computer-readable storage medium, and computer program product that can improve the accuracy rate of identifying key entities in financial texts.

[0005] In a first aspect, the present application provides a financial text recognition method. The method includes:

[0006] Obtain a financial text to be recognized and an entity set corresponding to the financial text to be recognized;

[0007] Determine a target vector of the financial text to be recognized and a target vector of each entity in the entity set through a pre-trained financial text recognition model;

[0008] Input the target vector of each entity and the target vector of the financial text to be recognized into a pre-trained first key entity recognition model respectively, and obtain an entity label corresponding to each entity; the entity label is used to represent whether the corresponding entity is a key entity; the first key entity recognition model is obtained by retraining the pre-trained financial text recognition model;

[0009] Select, from the entity set, an entity whose corresponding entity label is a key entity label as the key entity of the financial text to be recognized.

[0010] In one embodiment, the pre-trained financial text recognition model is trained in the following manner:

[0011] Obtain sample texts and the actual target vectors corresponding to the sample texts; the sample texts include general texts and financial texts, and the actual target vectors are composed of the combined vectors of each word in the sample texts, and the combined vectors are composed of the word vectors, text vectors, position vectors, and task vectors of the corresponding words;

[0012] Input the sample texts into the financial text recognition model to be trained, and obtain the predicted target vectors corresponding to the sample texts;

[0013] Train the financial text recognition model to be trained according to the difference between the actual target vector and the predicted target vector until the first preset end condition is satisfied, and use the trained financial text recognition model that satisfies the first preset end condition as the pre-trained financial text recognition model.

[0014] In one embodiment, the step of respectively inputting the target vectors of each entity and the target vector of the financial text to be recognized into the pre-trained first key entity recognition model to obtain the entity label corresponding to each entity includes:

[0015] Combine the target vectors of each entity and the target vector of the financial text to be recognized respectively to obtain the combined target vector of each entity;

[0016] Input the combined target vectors of each entity into the pre-trained first key entity recognition model respectively to obtain the key entity probabilities of each entity; the key entity probabilities are used to represent the probabilities that the corresponding entities belong to key entities;

[0017] Determine the entity label corresponding to each entity according to the key entity probability of each entity.

[0018] In one embodiment, the pre-trained first key entity recognition model is trained in the following manner:

[0019] Obtain sample financial texts, the sample entities of the sample financial texts, and the actual entity labels corresponding to the sample entities;

[0020] Determine the target vector of the sample financial text and the target vector of the sample entity through the pre-trained financial text recognition model;

[0021] Input the target vector of the sample financial text and the target vector of the sample entity into the pre-trained financial text recognition model to obtain the predicted entity label corresponding to the sample entity;

[0022] Retrain the pre-trained financial text recognition model according to the difference between the actual entity label and the predicted entity label until a second preset end condition is met, and use the trained financial text recognition model that meets the second preset end condition as the pre-trained first key entity recognition model.

[0023] In one embodiment, the method further includes:

[0024] Obtain the changed topic label of the financial text to be recognized; the changed topic label is obtained by changing the topic label of the financial text to be recognized;

[0025] Determine the target vector of the changed topic label through the pre-trained financial text recognition model;

[0026] Input the target vector of the financial text to be recognized and the target vector of the changed topic label into a pre-trained second key entity recognition model to obtain the key entity position in the financial text to be recognized; the second key entity recognition model is obtained by retraining the pre-trained financial text recognition model;

[0027] Determine the key entity corresponding to the key entity position in the financial text to be recognized as the target key entity corresponding to the topic label of the financial text to be recognized.

[0028] In one embodiment, the pre-trained second key entity recognition model is trained in the following manner:

[0029] Obtain a sample financial text, the sample changed topic label of the sample financial text, and the actual key entity position in the sample financial text;

[0030] Determine the target vector of the sample financial text and the target vector of the sample changed topic label through the pre-trained financial text recognition model;

[0031] Input the target vector of the sample financial text and the target vector of the sample changed topic label into the pre-trained financial text recognition model to obtain the predicted key entity position in the sample financial text;

[0032] Retrain the pre-trained financial text recognition model according to the difference between the actual key entity position and the predicted key entity position until a third preset end condition is met, and use the trained financial text recognition model that meets the third preset end condition as the pre-trained second key entity recognition model.

[0033] In one embodiment, the method further includes:

[0034] Input the financial text to be recognized into a pre-trained text sentiment recognition model to obtain the sentiment label of the financial text to be recognized; the text sentiment recognition model is obtained by retraining the pre-trained financial text recognition model;

[0035] Use the key entity, the target key entity, and the sentiment label of the financial text to be recognized as the recognition result of the financial text to be recognized.

[0036] In one embodiment, the pre-trained text sentiment recognition model is trained in the following manner:

[0037] Obtain sample financial texts and the actual sentiment labels of the sample financial texts;

[0038] Input the sample financial texts into a pre-trained financial text recognition model to obtain the predicted sentiment labels of the sample financial texts;

[0039] According to the difference between the actual sentiment label and the predicted sentiment label, retrain the pre-trained financial text recognition model until a fourth preset end condition is met, and use the trained financial text recognition model that meets the fourth preset end condition as the pre-trained text sentiment recognition model.

[0040] In a second aspect, the present application also provides a financial text recognition device. The device includes:

[0041] An entity acquisition module, configured to acquire a financial text to be recognized and the entity set corresponding to the financial text to be recognized;

[0042] A vector determination module, configured to determine the target vector of the financial text to be recognized and the target vector of each entity in the entity set through a pre-trained financial text recognition model;

[0043] A label determination module, configured to input the target vector of each entity and the target vector of the financial text to be recognized into a pre-trained first key entity recognition model respectively to obtain the entity label corresponding to each entity; the entity label is used to represent whether the corresponding entity is a key entity; the first key entity recognition model is obtained by retraining the pre-trained financial text recognition model;

[0044] An entity screening module, configured to screen out the entities corresponding to the key entity labels from the entity set as the key entities of the financial text to be recognized.

[0045] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0046] Obtain a financial text to be recognized and an entity set corresponding to the financial text to be recognized;

[0047] Determine a target vector of the financial text to be recognized and a target vector of each entity in the entity set through a pre-trained financial text recognition model;

[0048] Input the target vector of each entity and the target vector of the financial text to be recognized into a pre-trained first key entity recognition model respectively, to obtain an entity label corresponding to each entity; the entity label is used to represent whether the corresponding entity is a key entity; the first key entity recognition model is obtained by retraining the pre-trained financial text recognition model;

[0049] Screen out entities corresponding to the entity label being a key entity label from the entity set as the key entities of the financial text to be recognized.

[0050] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, and when the computer program is executed by a processor, the following steps are implemented:

[0051] Obtain a financial text to be recognized and an entity set corresponding to the financial text to be recognized;

[0052] Determine a target vector of the financial text to be recognized and a target vector of each entity in the entity set through a pre-trained financial text recognition model;

[0053] Input the target vector of each entity and the target vector of the financial text to be recognized into a pre-trained first key entity recognition model respectively, to obtain an entity label corresponding to each entity; the entity label is used to represent whether the corresponding entity is a key entity; the first key entity recognition model is obtained by retraining the pre-trained financial text recognition model;

[0054] Screen out entities corresponding to the entity label being a key entity label from the entity set as the key entities of the financial text to be recognized.

[0055] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0056] Obtain the financial text to be recognized and the entity set corresponding to the financial text to be recognized;

[0057] Determine the target vector of the financial text to be recognized and the target vector of each entity in the entity set through a pre-trained financial text recognition model;

[0058] Input the target vector of each entity and the target vector of the financial text to be recognized into a pre-trained first key entity recognition model respectively, to obtain the entity label corresponding to each entity; the entity label is used to characterize whether the corresponding entity is a key entity; the first key entity recognition model is obtained by retraining the pre-trained financial text recognition model;

[0059] Select, from the entity set, the entities whose corresponding entity labels are key entity labels as the key entities of the financial text to be recognized.

[0060] The above financial text recognition method, device, computer device, storage medium and computer program product obtain the financial text to be recognized and the entity set corresponding to the financial text to be recognized; then determine the target vector of the financial text to be recognized and the target vector of each entity in the entity set through a pre-trained financial text recognition model; then input the target vector of each entity and the target vector of the financial text to be recognized into a pre-trained first key entity recognition model respectively, to obtain the entity label corresponding to each entity; the entity label is used to characterize whether the corresponding entity is a key entity; the first key entity recognition model is obtained by retraining the pre-trained financial text recognition model; finally, select, from the entity set, the entities whose corresponding entity labels are key entity labels as the key entities of the financial text to be recognized. In this way, through the first key entity recognition model trained based on the pre-trained financial text recognition model, determine the entity label corresponding to each entity in the entity set of the financial text to be recognized, and select, from the entity set, the entities whose corresponding entity labels are key entity labels as the key entities of the financial text to be recognized, achieving the purpose of analyzing whether each entity in the entity set is a key entity through the first key entity recognition model, which is beneficial to improving the accuracy of key entity recognition of financial texts; at the same time, there is no need to manually screen the recognized entities, thus avoiding the defect that errors are likely to occur in the manual screening process, resulting in a low accuracy of key entity recognition of financial texts, and further improving the accuracy of key entity recognition of financial texts. Brief Description of the Drawings

[0061] Figure 1 It is a schematic flow chart of the financial text recognition method in an embodiment;

[0062] Figure 2Schematic diagram of the training steps of a financial text recognition model in an embodiment;

[0063] Figure 3 Schematic diagram of a pre-trained language model in the financial field based on multi-task self-supervised learning in an embodiment;

[0064] Figure 4 Schematic diagram of a coarse-grained key entity recognition model in an embodiment;

[0065] Figure 5 Schematic diagram of a fine-grained key entity recognition model in an embodiment;

[0066] Figure 6 Schematic diagram of a financial text sentiment tendency analysis model in an embodiment;

[0067] Figure 7 Schematic diagram of the process of a financial text recognition method in another embodiment;

[0068] Figure 8 Structural block diagram of a financial text recognition device in an embodiment;

[0069] Figure 9 Internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0070] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0071] In one embodiment, as Figure 1 shown, a financial text recognition method is provided. In this embodiment, it is exemplified that the method is applied to a server. It can be understood that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0072] Step S101, obtain the financial text to be recognized and the entity set corresponding to the financial text to be recognized.

[0073] Among them, the financial text to be recognized refers to the financial text for which key entities need to be recognized; the financial text refers to the text in the financial field, such as the mortgage rate is gradually decreasing, the exchange rate has dropped too much, etc.

[0074] Among them, the entity set refers to a list of entities, and each entity in the financial text to be recognized is included in the list of entities. An entity refers to an entity with a specific meaning in the text to be recognized, mainly including personal names, place names, organization names, proper nouns, etc., such as accounts, credit cards, outlets, card types, etc.

[0075] Specifically, in response to the selection operation of the financial text, the terminal obtains the selected financial text as the financial text to be recognized, generates a financial text recognition request according to the financial text to be recognized, and sends the financial text recognition request to the corresponding server. The server parses the received financial text recognition request to obtain the financial text to be recognized; inputs the financial text to be recognized into the deep neural network model, and performs entity recognition on the financial text to be recognized through the deep neural network model to obtain each entity in the financial text to be recognized; combines each entity in the financial text to be recognized to obtain the entity set corresponding to the financial text to be recognized.

[0076] For example, the server inputs the financial text to be recognized into a deep neural network model based on LSTM (Long Short-Term Memory) + CRF (Conditional Random Field), and through the deep neural network model based on LSTM + CRF, performs entity recognition on the financial text to be recognized to obtain each entity in the financial text to be recognized, such as entity A, entity B, entity C, and combines these entities to obtain the entity set corresponding to the financial text to be recognized (entity A, entity B, entity C).

[0077] Step S102: Determine the target vector of the financial text to be recognized and the target vector of each entity in the entity set through a pre-trained financial text recognition model.

[0078] Among them, the pre-trained financial text recognition model refers to a pre-trained language model in the financial field, specifically a pre-trained language model in the financial field based on multi-task self-supervised learning. In the pre-training stage, the corpora in the general field and the corpora in the financial field are pre-trained simultaneously, and the domain knowledge and semantic information of the vocabulary in the financial field are transferred to the model, so that a large number of pre-trained word vectors that can represent the vocabulary in the financial field can be obtained through this model. In the actual scenario, the pre-trained financial text recognition model refers to a multi-layer Transformer encoder based on the self-attention mechanism.

[0079] It should be noted that traditional pre-trained language models (such as the BERT model) are pre-trained on a large-scale unlabeled dataset in the general domain based on Wikipedia, enabling the word vectors in the pre-training process to capture the context semantic information of sentences, thereby obtaining a set of word vectors containing context semantics. However, the present application is directed to tasks in the financial field, and there are significant differences between the vocabulary in financial texts and that in the general domain. For example, words such as "exchange rate", "amount", and "bank" in financial texts appear frequently. Therefore, simply pre-training the BERT model in the general domain will not result in many pre-trained word vectors that can represent financial domain vocabulary, and thus it is impossible to better utilize the word vectors to complete specific classification tasks in the financial field (such as key entity recognition tasks, sentiment analysis tasks) during the fine-tuning stage.

[0080] Among them, the target vector of the financial text to be recognized is composed of the combined vectors of each character in the financial text to be recognized, and the target vector of each entity is composed of the combined vectors of each character in the entity. The combined vector of each character is composed of the word vector, text vector, and position vector corresponding to the character. Among them, the word vector refers to the vector represented by each character, such as E [房] 、E [贷] 、E [利] 、E [率] ; the text vector is used to distinguish in which sentence the character is located. The text vectors of each character in the first sentence are represented in the form of E A , and the text vectors of each character in the second sentence are represented in the form of E B ; the position vector is used to mark the position information of each character (including special symbols) in the sentence. "[CLS]" is represented by E0, "house" is represented by E1, "loan" is represented by E2, "interest" is represented by E3, "rate" is represented by E4, etc.

[0081] Specifically, the server inputs the financial text to be recognized and an entity of the financial text to be recognized into the pre-trained financial text recognition model at the same time. The financial text recognition model performs word segmentation on the financial text to be recognized to obtain each character (including special symbols) in the financial text to be recognized; obtains the word vector, text vector, and position vector of each character, and combines the word vector, text vector, and position vector of each character to obtain the combined vector of each character; combines the combined vectors of each character again to obtain the target vector of the financial text to be recognized. At the same time, the financial text recognition model performs word segmentation on each entity to obtain each character (including special symbols) in each entity; obtains the word vector, text vector, and position vector of each character, and combines the word vector, text vector, and position vector of each character to obtain the combined vector of each character; combines the combined vectors of each character again to obtain the target vector of each entity.

[0082] For example, if the financial text to be recognized is "After national regulation, the mortgage rate has gradually decreased", first, the financial text to be recognized is tokenized, and two special symbols, [CLS] and [SEP], are introduced. [CLS] is placed at the beginning of the text, and [SEP] is placed after each sentence to play the role of separating sentences, thus obtaining the representation form of the financial text to be recognized: [CLS] After national regulation [SEP] The mortgage rate has gradually decreased [SEP]; then the target vector corresponding to the financial text to be recognized is (E [CLS] +E A +E0, E [经] +E A +E1, E [过] +E A +E2, E [国] +E A +E3, E [家] +E A +E4, E [调] +E A +E5, E [控] +E A +E6, E [SEP] +E A +E7, E [房] +E B +E8, E [贷] +E B +E9, E [利] +E B +E 10 , E [率] +E B +E 11 , E [逐] +E B +E 12 , E [渐] +E B +E 13 , E [下] +E B +E 14 , E [降] +E B +E 15 , E [SEP] +E B +E 16 ).

[0083] Step S103: Respectively input the target vector of each entity and the target vector of the financial text to be recognized into the pre-trained first key entity recognition model to obtain the entity label corresponding to each entity; the entity label is used to characterize whether the corresponding entity is a key entity; the first key entity recognition model is obtained by retraining the pre-trained financial text recognition model.

[0084] Among them, the first key entity recognition model refers to a model used to recognize key entities in the financial text to be recognized, which is obtained by retraining a pre-trained financial text recognition model, avoiding the defects that the labeled corpus in the financial field is scarce, the manual labeling of the corpus is time-consuming and costly, and the lack of labeled corpus for model training will lead to insufficient model training, causing overfitting, and ultimately resulting in poor model performance.

[0085] Among them, the entity label corresponding to each entity is used to characterize whether the entity is a key entity; for example, if the entity label of the entity is a key entity label, it means that the entity is a key entity; if the entity label of the entity is a non-key entity label, it means that the entity is a non-key entity.

[0086] Specifically, after the server combines the target vector of each entity with the target vector of the financial text to be recognized, it inputs them into the pre-trained first key entity recognition model to obtain the key entity probability corresponding to each entity; if the key entity probability of the entity is greater than the preset probability, it is confirmed that the entity label of the entity is a key entity label; if the key entity probability of the entity is less than or equal to the preset probability, it is confirmed that the entity label of the entity is a non-key entity label.

[0087] Step S104: From the entity set, screen out the entities whose corresponding entity labels are key entity labels as the key entities of the financial text to be recognized.

[0088] Among them, the key entity of the financial text to be recognized refers to the entity with a relatively high similarity between the entity set and the financial text to be recognized.

[0089] Specifically, the server screens out the entities whose corresponding entity labels are key entity labels from the entity set and uses the entity as the key entity of the financial text to be recognized; combines the key entities of the financial text to be recognized to obtain the key entity set of the financial text to be recognized, such as a key entity list.

[0090] For example, the entity set includes entity A, entity B, entity C, entity D, entity E, and entity F, and the entity labels corresponding to entity A, entity B, and entity C are key entity labels, and the entity labels corresponding to entity D, entity E, and entity F are non-key entity labels, indicating that the key entities of the financial text to be recognized are entity A, entity B, and entity C.

[0091] In the above financial text recognition method, a financial text to be recognized and a corresponding entity set of the financial text to be recognized are obtained; then, through a pre-trained financial text recognition model, a target vector of the financial text to be recognized and a target vector of each entity in the entity set are determined; then, the target vector of each entity and the target vector of the financial text to be recognized are respectively input into a pre-trained first key entity recognition model to obtain an entity label corresponding to each entity; the entity label is used to represent whether the corresponding entity is a key entity; the first key entity recognition model is obtained by retraining the pre-trained financial text recognition model; finally, from the entity set, entities corresponding to the entity labels that are key entity labels are screened out as the key entities of the financial text to be recognized. In this way, through the first key entity recognition model trained based on the pre-trained financial text recognition model, the entity label corresponding to each entity in the entity set of the financial text to be recognized is determined, and entities corresponding to the entity labels that are key entity labels are screened out from the entity set as the key entities of the financial text to be recognized, achieving the purpose of analyzing whether each entity in the entity set is a key entity through the first key entity recognition model, which is beneficial to improving the accuracy of key entity recognition in financial texts; at the same time, there is no need to manually screen the recognized entities, thus avoiding the defect that errors are likely to occur in the manual screening process, resulting in a low accuracy of key entity recognition in financial texts, and further improving the accuracy of key entity recognition in financial texts.

[0092] In one embodiment, as Figure 2 shown, the pre-trained financial text recognition model is trained through the following steps:

[0093] Step S201, obtain a sample text and an actual target vector corresponding to the sample text; the sample text includes a general text and a financial text, and the actual target vector is composed of combined vectors of each character in the sample text, and the combined vector is composed of a character vector, a text vector, a position vector, and a task vector corresponding to the character.

[0094] Among them, the general text refers to the text in the general domain corpus, such as referring to Figure 3 , the general text refers to the text in Wikipedia, the text in Baidu Encyclopedia. The financial text refers to the text in the financial domain corpus, such as referring to Figure 3 , the financial text refers to the text on financial news websites, the text about finance on Sina Weibo, etc.

[0095] Among them, the combined vector of each word = word vector + text vector + position vector + task vector. The task vector is used to represent the characteristics of different tasks. Since six self-supervised pre-training tasks are constructed in this application, six different task vectors are initialized respectively, and the task vectors are represented by vector 0 to vector 5 respectively. For example, vector 0 represents the pre-training task of masked word prediction, vector 1 represents the pre-training task of word capitalization form prediction, vector 2 represents the pre-training task of keyword prediction, vector 3 represents the pre-training task of sentence order prediction, vector 4 represents the pre-training task of sentence distance prediction, and vector 5 represents the pre-training task of sentence semantic relationship prediction.

[0096] Among them, for the pre-training task of masked word prediction: The task uses the same masked word prediction rule as in the BERT model, randomly masking 15% of the vocabulary in the text. 80% of these vocabulary are replaced with the special [mask] symbol, 10% of the probability is replaced by a random vocabulary, and 10% of the probability remains unchanged. The training model predicts the real vocabulary corresponding to the position of the masked word, forcing the model to predict the vocabulary by connecting the context, so that the masked word has more context information.

[0097] Among them, for the pre-training task of word capitalization form prediction: In financial texts, words in capitalization form usually have special semantic meanings in sentences, such as ATM (Automated Teller Machine), CDR (Cumulative Default Rate), and EL (Expected Loss), etc. In order to make the word vectors obtained by pre-training be better used in entity recognition tasks, it is necessary to judge whether the words in the sentence are in capitalization form.

[0098] Among them, for the pre-training task of keyword prediction: Words that appear multiple times in financial texts are usually common financial vocabulary, and this vocabulary is likely to be related to the theme of the financial text. The training model identifies the financial vocabulary that appears multiple times in the text and uses it as the keyword representing the theme of the text, so that the model can capture the theme of the financial text.

[0099] Among them, for the pre-training task of sentence order prediction: In order to learn the relationship between sentences in the text, a text is divided into n sentences, and these n sentences are scrambled. Then, the situation where the sentences are arranged in the original order is used as the training target, and the training model reorders the scrambled sentences to complete the multi-classification task of the text.

[0100] Among them, for the pre-training task of sentence distance prediction: In order to capture the distance information between sentences in multiple texts, different from the BERT model that only predicts whether two sentences in a single text are adjacent, this application defines the distance information between two sentences as a three-classification task: "00" represents that two sentences are in the same text and adjacent; "01" represents that two sentences are in the same text but not adjacent; "11" represents that two sentences are in different texts.

[0101] Among them, for the pre-training task of sentence semantic relationship prediction: in order to capture the semantic relationships between sentences, this application constructs a question-and-answer dataset based on general-domain texts and financial-domain texts, because in the question-and-answer dataset, questions corresponding to the same answer tend to be semantically close or even identical.

[0102] Step S202: Input the sample text into the financial text recognition model to be trained, and obtain the predicted target vector corresponding to the sample text.

[0103] Among them, the financial text recognition model to be trained consists of multiple Transformer encoders, specifically including an input part, a multi-head attention mechanism part, and a feed-forward neural network part; the multi-head attention mechanism part is assembled by a set of multiple self-attention mechanisms, mainly using self-attention to learn the relationships within the sentence; the feed-forward neural network part refers to a fully connected network; the input part refers to taking the sum of the word vector, text vector, and position vector of each character as the model input.

[0104] Step S203: According to the difference between the actual target vector and the predicted target vector, train the financial text recognition model to be trained until the first preset end condition is met, and use the trained financial text recognition model that meets the first preset end condition as the pre-trained financial text recognition model.

[0105] Among them, the first preset end condition refers to that the number of model training times reaches the first preset number of times, the first loss value is less than the first preset threshold, etc.

[0106] Specifically, the server obtains general text and financial text as sample texts to be trained; performs word segmentation on the sample texts, and introduces two special symbols, [CLS] and [SEP]. [CLS] is placed at the beginning of the text, and [SEP] is placed after each sentence to play a role in separating sentences, so as to obtain each character (including special symbols) in the sample text; obtains the word vector, text vector, position vector, and task vector of each character through the fasttext tool, and combines the word vector, text vector, position vector, and task vector of each character to obtain the combined vector of each character; combines the combined vectors of each character again to obtain the actual target vector of the sample text. Next, the server inputs the sample text into the financial text recognition model to be trained, and outputs the predicted target vector corresponding to the sample text through the financial text recognition model to be trained; obtains the first loss value according to the difference between the actual target vector and the predicted target vector; adjusts the model parameters of the financial text recognition model to be trained according to the first loss value, and retrains the financial text recognition model after the model parameters are adjusted until the first loss value obtained through the trained financial text recognition model is less than the first preset threshold, then uses the trained financial text recognition model as the pre-trained financial text recognition model.

[0107] For example, referring to Figure 3 , first initialize the words in the general domain corpus and the financial domain corpus as word vectors, and then input the word vectors and their corresponding position vectors, text vectors, and task vectors into the multi-layer Transformer encoder based on the self-attention mechanism at the same time. The self-attention mechanism can help the word vectors obtain the global context semantic information of the sentence. At the same time, six self-supervised pre-training tasks are constructed to learn different levels of knowledge in the corpus. These two self-supervised pre-training tasks are respectively the masked word prediction pre-training task, the word capitalization form prediction pre-training task, the keyword prediction pre-training task, the sentence order prediction pre-training task, the sentence distance prediction pre-training task, and the sentence semantic relationship prediction pre-training task.

[0108] Furthermore, referring to Figure 3, first, each sentence in the general domain corpus and the financial domain corpus is tokenized. Then, each character (including special symbols) is randomly initialized into a character vector, its corresponding text vector, position vector, and task vector using the fasttext tool. The character vector, text vector, position vector, and task vector of each character are simultaneously input into a multi-layer Transformer encoder based on the self-attention mechanism. The self-attention mechanism can help the character vector obtain the global context semantic information of the sentence. Then, six pre-training tasks are used to gradually improve the parameters in this multi-layer Transformer encoder based on the self-attention mechanism, so that the text semantic representation output by the model can depict the essence of the language and facilitate subsequent fine-tuning for specific NLP (Neuro-Linguistic Programming) tasks.

[0109] Take the masked word prediction pre-training task as an example: 15% of the words in a sentence need to be predicted. Among these words to be predicted, 80% are replaced with the special [mask] character, 10% are replaced with a random word, and 10% remain unchanged with a certain probability. For example, the sentence "The subprime mortgage crisis in Country A has exposed the defects in the financial market and the loopholes in financial supervision" becomes "The sub[mask] mortgage crisis in Country A has exposed the defects in the financial citizens and the loopholes in financial supervision". In this sentence, "prime" is replaced with [mask], "market" is replaced with "citizens", and "supervision" remains unchanged. Then, each word in "The sub[mask] mortgage crisis in Country A has exposed the defects in the financial citizens and the loopholes in financial supervision" is initialized as a vector (the sum of word vectors, text vectors, position vectors, and task vectors) using the fasttext tool. This vector is input into a multi-layer Transformer encoder based on the self-attention mechanism, and through this encoder, the 15% of the original real words in this sentence are predicted. Since the original sentence "The subprime mortgage crisis in Country A has exposed the defects in the financial market and the loopholes in financial supervision" can also be represented as a vector using the fasttext tool, the original sentence vector is used as the objective function. Through multiple iterations of training, the sentence vector output from the multi-layer Transformer encoder is made to be infinitely close to the original sentence vector, thereby continuously optimizing the model parameters of the multi-layer Transformer encoder, forcing the model to rely more on context information to predict words, and endowing the model with a certain error correction ability. In this training process, no artificial supervision signal is given to the model. The supervision signal comes from the content of the data itself, and the model gives itself the supervision signal, so it is a self-supervised training method. Similarly, for the capitalized word prediction pre-training task, the capitalized words in the sentence are replaced with [mask], and after being transformed into vectors, they are input into the Transformer encoder. The sentence vector before transformation is used as the objective function, and through multiple iterations of training, the parameters of the Transformer encoder are continuously optimized. For the keyword prediction pre-training task, the words that appear more frequently (threshold is 3 times) in the financial sentence are calculated, and it is considered that these words are likely to be related to the theme described by the sentence. Therefore, these words are replaced with [mask], and after being transformed into vectors, they are input into the Transformer encoder for training. For the sentence order prediction pre-training task, the order of the sentences in a text is randomly scrambled. The sentences are represented as vectors and input into the Transformer encoder. The situation where the sentences are arranged completely in the original order is used as the objective function, and through iterative training, the output result of the encoder is made to continuously approach the objective function. For the sentence distance prediction pre-training task, two sentences are randomly selected from different documents. After these two sentences are transformed into vectors, they are input into the encoder. The objective function is the correct position information (adjacent or not adjacent) of these two sentences. Through continuous training of the encoder, the encoder can capture the position information between the two sentences.The pre-training task of sentence semantic relationship prediction is based on a question-answering dataset. Questions corresponding to the same answer are often semantically close or even the same. For example, the semantics of the three questions "How to understand the risk structure of interest rates", "What are the risk structures of interest rates", and "What is the risk structure of interest rates" are the same, and their corresponding answers are also the same. Therefore, a question-answering dataset based on financial domain texts is constructed in a way that one answer corresponds to multiple questions. Randomly select a question sentence and an answer sentence, start with the [CLS] symbol, and separate the question and the answer with the [SEP] symbol; for example, [CLS] What are the risk structures of interest rates [SEP] Industrial and Commercial Bank of China [SEP]; Input it into the Transformer encoder for training, and the objective function is the correct answer corresponding to the question. Optimize the parameters of the encoder so that the encoder can learn the knowledge of the correct answer corresponding to the question.

[0110] It should be noted that these six pre-training tasks adopt the method of multi-task continuous training, that is, whenever a new task appears, the continuous multi-task learning method will first use the parameters learned previously to initialize the model, and then train the newly introduced task and the original tasks simultaneously. For example, first perform the pre-training task of masked word prediction to train the Transformer encoder. After training for N rounds, keep the parameters of the trained Transformer encoder unchanged, and perform the training of the pre-training task of word capitalization form prediction. Use the Transformer encoder as a shared encoder to train these two tasks in parallel, and so on until the six pre-training tasks are carried out simultaneously. The Transformer encoder obtained in this way will not forget the knowledge learned before, and at the same time ensures the training efficiency.

[0111] It should be noted that the purpose of training the pre-trained language model in the financial domain is to obtain the word vector representation after fusing context information; the dimensions of the input and output of the pre-trained language model are exactly the same. It can be understood that after each word in a financial sentence of the input has fused context information, it is then output in the form of the sentence as it is.

[0112] In this embodiment, by using general texts and financial texts, a financial text recognition model is trained, which is beneficial to subsequently obtaining a large number of pre-trained word vectors that can represent financial domain vocabulary through the financial text recognition model, thereby improving the determination accuracy of the word vectors of financial texts.

[0113] In one embodiment, in the above step S103, the target vector of each entity and the target vector of the financial text to be recognized are respectively input into a pre-trained first key entity recognition model to obtain the entity label corresponding to each entity, which specifically includes the following content: The target vector of each entity and the target vector of the financial text to be recognized are combined to obtain the combined target vector of each entity; the combined target vector of each entity is respectively input into a pre-trained first key entity recognition model to obtain the key entity probability of each entity; the key entity probability is used to represent the probability that the corresponding entity belongs to a key entity; according to the key entity probability of each entity, the entity label corresponding to each entity is determined.

[0114] Among them, the pre-trained first key entity recognition model refers to a coarse-grained key entity recognition model. The purpose of coarse-grained key entity recognition is to screen out one or more key entities from the existing entity set that can reflect the text theme of the financial text.

[0115] Specifically, the server combines the target vector of each entity and the target vector of the financial text to be recognized to obtain the combined target vector of each entity. For example Figure 4 in ([CLS], Tok1……[SEP]……Tok M' ); the combined target vector of each entity is respectively input into a pre-trained first key entity recognition model, and the first key entity recognition model recognizes the combined target vector of each entity to obtain the key entity probability of each entity. For example Figure 4 the output vector C of [CLS] in; if the key entity probability of the entity is greater than the preset probability, it is confirmed that the entity label of the entity is a key entity label; if the key entity probability of the entity is less than or equal to the preset probability, it is confirmed that the entity label of the entity is a non-key entity label, so as to obtain the entity label corresponding to each entity.

[0116] Illustrate by way of example, refer to Figure 4 , the server first uses the trained LSTM+CRF model to perform entity recognition on the input financial text, and forms a list of all recognized entities. In the fine-tuning stage of the model, the word vectors pre-trained by the above-mentioned financial domain pre-trained language model based on multi-task self-supervised learning are used to initialize both the financial text and each entity in the entity list and input them into the model. The key entity recognition task is transformed into a sentence matching task, and the model compares the similarity between the input entity and the financial text (that is, judges whether the currently input entity is a key entity of the financial text through the output vector of [CLS]), so as to screen out the entities with higher similarity to the financial text as key entities, and form a list of key entities with them.

[0117] In this embodiment, by inputting the target vectors of each entity and the target vector of the financial text to be recognized into the pre-trained first key entity recognition model respectively, the entity label corresponding to each entity is obtained, and then the key entity of the financial text to be recognized is determined through the entity label, without manual screening, thereby improving the recognition accuracy of the key entity of the financial text.

[0118] In one embodiment, the pre-trained first key entity recognition model is trained in the following manner: Obtain the sample financial text, the sample entity of the sample financial text, and the actual entity label corresponding to the sample entity; Determine the target vector of the sample financial text and the target vector of the sample entity through the pre-trained financial text recognition model; Input the target vector of the sample financial text and the target vector of the sample entity into the pre-trained financial text recognition model to obtain the predicted entity label corresponding to the sample entity; Retrain the pre-trained financial text recognition model according to the difference between the actual entity label and the predicted entity label until the second preset end condition is satisfied, and use the trained financial text recognition model that satisfies the second preset end condition as the pre-trained first key entity recognition model.

[0119] Among them, the second preset end condition refers to that the number of model training times reaches the second preset number of times, the second loss value is less than the second preset threshold, etc.

[0120] Specifically, the server first obtains the sample financial text, the sample entity of the sample financial text, and the actual entity label corresponding to the sample entity; Then input the sample financial text and the sample entity into the pre-trained financial text recognition model, and perform vector initialization processing on the sample financial text and the sample entity through the pre-trained financial text recognition model to obtain the target vector of the sample financial text and the target vector of the sample entity; Input the target vector of the sample financial text and the target vector of the sample entity into the pre-trained financial text recognition model at the same time to obtain the predicted entity label corresponding to the sample entity, such as Figure 4 the output vector C of [CLS] in. Obtain the second loss value according to the difference between the actual entity label and the predicted entity label; Adjust the model parameters of the pre-trained financial text recognition model according to the second loss value to obtain the financial text recognition model with adjusted model parameters; Retrain the financial text recognition model with adjusted model parameters until the second loss value obtained according to the trained financial text recognition model is less than the second preset threshold, then use the trained financial text recognition model as the pre-trained first key entity recognition model.

[0121] For example, refer to Figure 4, after the pre-trained pre-trained language model in the financial field, determine the word vectors corresponding to the financial text and the word vectors corresponding to each entity in the entity list, and then respectively use the word vectors corresponding to the financial text and the word vectors corresponding to each entity in the entity list in the form of Figure 4 (i.e., [CLS] financial text [SEP] entity), and input it into the pre-trained pre-trained language model in the financial field again for fine-tuning, and use the output vector C to determine whether the entity is a key entity label. Specifically, in the fine-tuning stage of the model, a financial sentence vector and an entity vector are separated by [SEP] and input into the pre-trained language model at the same time to obtain the output vector C calculated by the model. If the entity is a key entity label, train the model to approach 1 through multiple iterations. If the entity is a non-key entity label, train the model to approach 0 through multiple iterations. Further, after fine-tuning, a financial text and an entity are input into the fine-tuned pre-trained language model in the financial field in the form of Figure 4 at the same time to obtain the value of vector C. If the value of C is higher than the preset threshold, it is considered that the entity has a high similarity with the financial text, indicating that the entity is a key entity.

[0122] In this embodiment, by retraining the pre-trained financial text recognition model, a first key entity recognition model is obtained, which overcomes the problem of scarce labeled corpus in the key entity recognition task; at the same time, the first key entity recognition model obtained through training is beneficial to accurately identify the key entities of the financial text to be recognized.

[0123] In one embodiment, the financial text recognition method provided by this application further includes the step of determining the target key entity corresponding to the topic label of the financial text to be recognized, which specifically includes the following content: obtaining the changed topic label of the financial text to be recognized; the changed topic label is obtained by changing the topic label of the financial text to be recognized; through the pre-trained financial text recognition model, determine the target vector of the changed topic label; input the target vector of the financial text to be recognized and the target vector of the changed topic label into the pre-trained second key entity recognition model to obtain the key entity position in the financial text to be recognized; the second key entity recognition model is obtained by retraining the pre-trained financial text recognition model; determine the key entity corresponding to the key entity position in the financial text to be recognized as the target key entity corresponding to the topic label of the financial text to be recognized.

[0124] Among them, the topic label of the financial text to be recognized refers to the label that can reflect the topic information of the financial text to be recognized, such as fraud, bankruptcy, loss, etc. The changed topic label refers to changing the topic label into a question in the machine reading comprehension task, such as which financial institution has fraudulent behavior?

[0125] It should be noted that the machine reading comprehension task is to given a passage of text and a question, and use a model to find the answer corresponding to the question in the text. Assuming the answer is in the text, the goal of the reading comprehension task is to obtain the starting position and the ending position of the answer, and the characters between the starting position and the ending position are the answer.

[0126] Among them, the pre-trained second key entity recognition model refers to a fine-grained key entity recognition model, such as Figure 5 ; the purpose of fine-grained key entity recognition is to find a key entity in the financial text that is most relevant to the existing topic label of the text.

[0127] Among them, the key entity position includes the starting position and the ending position of the key entity. The key entity corresponding to the key entity position refers to the characters between the starting position and the ending position of the key entity.

[0128] Among them, the target key entity refers to the key entity in the financial text to be recognized that is most relevant to the topic label of the financial text to be recognized.

[0129] Specifically, the server identifies the topic label of the financial text to be recognized through a topic model, and performs a change process on the topic label of the financial text to be recognized to obtain the changed topic label of the financial text to be recognized; inputs the financial text to be recognized and the changed topic label into the pre-trained financial text recognition model at the same time, and performs vector initialization processing on the financial text to be recognized and the changed topic label through the pre-trained financial text recognition model to obtain the target vector of the financial text to be recognized and the target vector of the changed topic label; combines the target vector of the financial text to be recognized and the target vector of the changed topic label, and inputs the combined vector, such as Figure 5 in ([CLS], Tok1……[SEP]……Tok M' ) into the pre-trained second key entity recognition model to obtain the starting position and the ending position of the key entity in the financial text to be recognized; determines the characters between the starting position and the ending position of the key entity in the financial text to be recognized, such as Figure 5 the answer in, as the target key entity corresponding to the topic label of the financial text to be recognized.

[0130] Illustrate with an example, refer to Figure 5, the fine-grained key entity recognition task is transformed into a machine reading comprehension task. Each financial text is regarded as a passage in the machine reading comprehension task (MRC), and its corresponding topic label is changed to a question in the machine reading comprehension task. For example, which financial institution has fraudulent behavior? In the fine-tuning stage of the model, the word vectors pre-trained by the above-mentioned financial domain pre-trained language model based on multi-task self-supervised learning are used. At the same time, the financial text and its changed topic label are initialized and input into the model. The model is trained to predict the answer corresponding to the question from the financial text, and the answer is regarded as the key entity most relevant to the topic label.

[0131] In this embodiment, by inputting the target vector of the financial text to be recognized and the target vector of the changed topic label into the pre-trained second key entity recognition model, the key entity position in the financial text to be recognized is obtained, and the key entity corresponding to the key entity position in the financial text to be recognized is determined as the target key entity corresponding to the topic label of the financial text to be recognized, thus achieving the purpose of determining the target key entity corresponding to the topic label of the financial text to be recognized.

[0132] In one embodiment, the pre-trained second key entity recognition model is trained in the following manner: obtaining a sample financial text, the sample changed topic label of the sample financial text, and the actual key entity position in the sample financial text; determining the target vector of the sample financial text and the target vector of the sample changed topic label through the pre-trained financial text recognition model; inputting the target vector of the sample financial text and the target vector of the sample changed topic label into the pre-trained financial text recognition model to obtain the predicted key entity position in the sample financial text; re-training the pre-trained financial text recognition model according to the difference between the actual key entity position and the predicted key entity position until the third preset end condition is met, and taking the trained financial text recognition model that meets the third preset end condition as the pre-trained second key entity recognition model.

[0133] Among them, the third preset end condition refers to that the number of model training times reaches the third preset number of times, the third loss value is less than the third preset threshold, etc.

[0134] Specifically, the server first obtains the sample financial text, the topic label of the sample financial text, and the actual key entity positions in the sample financial text, and performs a change process on the topic label of the sample financial text to obtain the sample changed topic label of the sample financial text; then inputs the sample financial text and the sample changed topic label into the pre-trained financial text recognition model, and performs vector initialization processing on the sample financial text and the sample changed topic label through the pre-trained financial text recognition model to obtain the target vector of the sample financial text and the target vector of the sample changed topic label; inputs the target vector of the sample financial text and the target vector of the sample changed topic label into the pre-trained financial text recognition model at the same time to obtain the predicted key entity positions in the sample financial text; obtains the third loss value according to the difference between the actual key entity positions and the predicted key entity positions; adjusts the model parameters of the pre-trained financial text recognition model according to the third loss value to obtain the financial text recognition model with adjusted model parameters; retrains the financial text recognition model with adjusted model parameters until the third loss value obtained according to the trained financial text recognition model is less than the third preset threshold, and then uses the trained financial text recognition model as the pre-trained second key entity recognition model.

[0135] For example, referring to Figure 5 , the server splits a topic label question and a financial text with the [SEP] symbol into the format of "[CLS] topic label question [SEP] financial text", and then uses the pre-trained pre-trained language model in the financial field to initialize the sentence in this format into a vector, and inputs this vector into the pre-trained pre-trained language model in the financial field again. Using the labels of the start position and end position of the manually annotated answer as the objective function, the language model is iteratively trained multiple times to make the output of the model continuously approach the objective function. After completing multiple trainings, a fine-tuned pre-trained language model in the financial field is obtained. At this time, a topic label question and a financial text are input into the fine-tuned model, and the model can better predict the position of the answer in the text (i.e., the start position and end position of the answer). For example, the topic label question: "Which company does the inventor work for?"; Financial text: "Both Bank A and Bank B are large state-owned banks." The labels of the manually annotated answer (i.e., Bank B) are the start vector label [0,0,0,1,0,0,0,0,0,0,0,0,0] and the end vector label [0,0,0,0,1,0,0,0,0,0,0,0,0]. After the financial text is converted into a vector and calculated by the language model, the dimensions and quantities of the output vectors will not change, such as Figure 5, assign the same weight to each word vector of the financial text output by the model, that is, perform a dot product of each word vector with this weight, and then use the softmax activation function to generate a probability for all word vectors. The word with the highest probability is used as the word at the starting position of the answer; the probability calculation of the word at the ending position of the answer is the same. Suppose the starting position vector obtained after the dot product and softmax activation of the model's first output is [0.7, 0.2, 0.1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], and the ending position vector is [0, 0.9, 0, 0.1, 0, 0, 0, 0, 0, 0, 0, 0, 0], indicating that the answer predicted by the model for the first time is "Jia Bank", rather than the correct answer "Yi Bank". At this time, use the cross-entropy loss function to calculate the loss between the label predicted by the model and the true manually labeled label, reduce the loss, and optimize the parameters of the language model through backpropagation, so that the prediction ability of the model is continuously improved, and finally a fine-grained key entity recognition model is obtained.

[0136] In this embodiment, by retraining the pre-trained financial text recognition model, a second key entity recognition model is obtained, which overcomes the problem of scarce labeled corpus in the key entity recognition task; at the same time, the second key entity recognition model obtained through training is conducive to accurately identifying the target key entity corresponding to the topic label of the financial text to be recognized.

[0137] In one embodiment, the financial text recognition method provided by this application further includes the step of determining the sentiment label of the financial text to be recognized, which specifically includes the following content: input the financial text to be recognized into the pre-trained text sentiment recognition model to obtain the sentiment label of the financial text to be recognized; the text sentiment recognition model is obtained by retraining the pre-trained financial text recognition model; the key entity, target key entity, and sentiment label of the financial text to be recognized are used as the recognition result of the financial text to be recognized.

[0138] Among them, the pre-trained financial text recognition model refers to a financial text sentiment tendency analysis model that can predict the sentiment label of the financial text to be recognized.

[0139] Among them, the sentiment label includes positive (active) and negative (negative).

[0140] Specifically, the server inputs the financial text to be recognized into the pre-trained text sentiment recognition model, performs sentiment prediction on the financial text to be recognized through the text sentiment recognition model, obtains the sentiment label of the financial text to be recognized, such as Figure 6 the C vector in; finally, the key entity, target key entity, and sentiment label of the financial text to be recognized are combined to obtain the comprehensive recognition result of the financial text to be recognized.

[0141] For example, referring to Figure 6 the server inputs the financial text into the financial text sentiment tendency analysis model to obtain the sentiment tendency of the financial text. For example, Figure 6 the C vector in

[0142] In this embodiment, by inputting the financial text to be recognized into the pre-trained text sentiment recognition model to obtain the sentiment label of the financial text to be recognized, the purpose of accurately recognizing the sentiment label of the financial text is achieved, thereby improving the recognition accuracy of the sentiment label of the financial text. At the same time, taking the key entity, target key entity, and sentiment label of the financial text to be recognized as the recognition result of the financial text to be recognized is beneficial to improving the richness of the recognition result of the financial text.

[0143] In one embodiment, the pre-trained text sentiment recognition model is trained in the following manner: obtaining the sample financial text and the actual sentiment label of the sample financial text; inputting the sample financial text into the pre-trained financial text recognition model to obtain the predicted sentiment label of the sample financial text; re-training the pre-trained financial text recognition model according to the difference between the actual sentiment label and the predicted sentiment label until the fourth preset end condition is met, and taking the trained financial text recognition model that meets the fourth preset end condition as the pre-trained text sentiment recognition model.

[0144] Among them, the fourth preset end condition refers to that the number of model training times reaches the fourth preset number, the fourth loss value is less than the fourth preset threshold, etc.

[0145] Specifically, the server obtains the sample financial text and the actual sentiment label of the sample financial text; inputs the sample financial text into the pre-trained financial text recognition model, and performs sentiment prediction on the sample financial text through the pre-trained financial text recognition model to obtain the predicted sentiment label of the sample financial text; obtains the fourth loss value according to the difference between the actual sentiment label and the predicted sentiment label; adjusts the model parameters of the pre-trained financial text recognition model according to the fourth loss value to obtain the financial text recognition model with adjusted model parameters; re-train the financial text recognition model with adjusted model parameters until the fourth loss value obtained according to the trained financial text recognition model is less than the fourth preset threshold, then take the trained financial text recognition model as the pre-trained text sentiment recognition model.

[0146] For example, referring to Figure 6, after the pre-training of the above-mentioned pre-trained language model in the financial field based on multi-task self-supervised learning is completed, it enters the fine-tuning stage of the model. The pre-trained word vectors are used to initialize the financial text input into the model, and the last layer of the model outputs the sentiment tendency of the financial text. The cross-entropy loss function is used in the fine-tuning stage of the model, and the method of ensemble learning is used. The model is trained with different random seeds, and then the 5 models with the highest scores are selected to predict the results, and the final prediction result is obtained by voting.

[0147] For example, referring to Figure 6 , after obtaining the pre-trained language model in the financial field, in order to judge the sentiment tendency of the financial text, the financial text is split into sentence forms and input into the model in sequence, and the [CLS] symbol is added in front of the sentence. For example: [CLS] The exchange rate has dropped too much. The language model after the pre-training is used to convert this sentence into a vector form, such as Figure 6 shown, the output vector of [CLS] - C can be used to judge the sentiment tendency of the sentence, positive is 1, negative is 0. The output vector of [CLS] can be used as the semantic representation of the whole sentence for text classification. It can be understood that: compared with other words and phrases in the sentence, the symbol [CLS] without obvious semantic information will fuse the semantic information of each word and phrase in the sentence more "fairly". Specifically, the server will first fine-tune the model. Fine-tuning is a supervised learning process, that is, input a financial sentence with positive sentiment, the model will output a C vector, and the model is iteratively trained N times to make C continuously approach the 1 vector; on the contrary, input a financial sentence with negative sentiment, and train the model to make C continuously approach the 0 vector. Then, the financial sentence whose sentiment tendency needs to be predicted is input into the fine-tuned model, and the sentiment label of the financial sentence predicted by the model can be obtained.

[0148] It should be noted that the cross-entropy loss function is one of the most common loss functions in using neural networks to process classification problems. The present application aims to solve the problem of sentiment tendency classification (classification task). Therefore, when using this loss function, the closer the predicted output is to the true sample label, the smaller the loss function L; otherwise, L is larger. Ensemble learning can be used for classification problem integration. For the training set data, in the present application, by training a number of individual learners and through a certain combination strategy, a strong learner can be finally formed to achieve the purpose of drawing on the strengths of all. The present application first randomly samples the training set. Through 5 random samplings, 5 sampling sets can be obtained (the present application uses random sampling, that is, first randomly collect a sample and put it into the sampling set, and then put the sample back, which means that the sample may still be sampled in the next sampling). Based on these 5 sampling sets, 5 weak learners can be trained. In the present application, after inputting a financial sentence, use these 5 weak learners to predict the sentiment tendency of the sentence respectively. Using the relative majority voting strategy, that is, the minority obeys the majority. Among the prediction results of these 5 weak learners, the category with the largest number is the final classification category. If more than one category obtains the highest number of votes, randomly select one as the final category. For example, if 4 are positive and 1 is negative, it is determined that the sentiment tendency of the financial sentence is positive.

[0149] It should be noted that Figure 6 The E in [CLS] and symbols such as E1 and T1 are all vectors. E [CLS] is the vector initialized by the [CLS] symbol. E1 to E N are the vectors initialized by the financial sentence input into the model. The sentence consists of N characters. C is E [CLS] the vector output after being calculated by the pre-trained language model. T1 to T N are respectively the vectors output after being calculated by the pre-trained language model for E1 to E N T and E are in one-to-one correspondence.

[0150] In this embodiment, by retraining the pre-trained financial text recognition model, a text sentiment recognition model is obtained, which overcomes the problem of scarce labeled corpus in the task of financial text sentiment tendency analysis; at the same time, the text sentiment recognition model obtained through training is conducive to accurately identifying the sentiment label of the financial text to be recognized, and solves the problem that the existing BERT model cannot well complete the task of financial text sentiment tendency analysis.

[0151] In one embodiment, as Figure 7 shown, another financial text recognition method is provided. Taking the application of this method to a server as an example for illustration, it includes the following steps:

[0152] Step S701: Obtain a sample text and the actual target vector corresponding to the sample text. The sample text includes a general text and a financial text. The actual target vector is composed of the combined vectors of each character in the sample text, and the combined vector is composed of the character vector, text vector, position vector, and task vector corresponding to the character.

[0153] Step S702: Input the sample text into the financial text recognition model to be trained, and obtain the predicted target vector corresponding to the sample text.

[0154] Step S703: Train the financial text recognition model to be trained according to the difference between the actual target vector and the predicted target vector until the first preset end condition is satisfied, and use the trained financial text recognition model that meets the first preset end condition as the pre-trained financial text recognition model.

[0155] Step S704: Obtain the financial text to be recognized and the entity set corresponding to the financial text to be recognized. Determine the target vector of the financial text to be recognized and the target vectors of each entity in the entity set through the pre-trained financial text recognition model.

[0156] Step S705: Input the target vectors of each entity and the target vector of the financial text to be recognized into the pre-trained first key entity recognition model respectively, and obtain the entity label corresponding to each entity. The entity label is used to represent whether the corresponding entity is a key entity. The first key entity recognition model is obtained by retraining the pre-trained financial text recognition model.

[0157] Step S706: Screen out the entities corresponding to the key entity labels from the entity set as the key entities of the financial text to be recognized.

[0158] Step S707: Obtain the changed theme label of the financial text to be recognized. The changed theme label is obtained by changing the theme label of the financial text to be recognized. Determine the target vector of the changed theme label through the pre-trained financial text recognition model.

[0159] Step S708: Input the target vector of the financial text to be recognized and the target vector of the changed theme label into the pre-trained second key entity recognition model, and obtain the key entity position in the financial text to be recognized. The second key entity recognition model is obtained by retraining the pre-trained financial text recognition model.

[0160] Step S709: Determine the key entity corresponding to the key entity position in the financial text to be recognized as the target key entity corresponding to the theme label of the financial text to be recognized.

[0161] Step S710: Input the financial text to be recognized into a pre-trained text sentiment recognition model to obtain the sentiment label of the financial text to be recognized; the text sentiment recognition model is obtained by retraining a pre-trained financial text recognition model.

[0162] Step S711: Use the key entity, target key entity, and sentiment label of the financial text to be recognized as the recognition result of the financial text to be recognized.

[0163] In the above financial text recognition method, the purpose of accurately identifying the key entity, target key entity, and sentiment label of the financial text to be recognized is achieved; at the same time, using the key entity, target key entity, and sentiment label of the financial text to be recognized as the recognition result of the financial text to be recognized is conducive to improving the richness of the financial text recognition result.

[0164] In one embodiment, to more clearly illustrate the financial text recognition method provided by the embodiments of the present application, the following uses a specific embodiment to specifically describe the financial text recognition method. In one embodiment, aiming at the problems of scarce labeled corpus for the financial text sentiment tendency analysis task and the existing BERT model being unable to complete the financial text sentiment tendency analysis task well, the pre-trained language model - BERT model is transformed, and based on the transfer learning idea, a pre-trained language model in the financial field is constructed using multi-task learning and self-supervised learning. At the same time, aiming at the problem that the LSTM+CRF model, which has good entity recognition effect on financial texts at present, cannot identify the key entities that can reflect the text theme, and considering that the key entity recognition task also faces the problem of scarce labeled corpus, the entity recognition problem is transformed into a sentence matching problem or a machine reading comprehension problem, and a pre-trained language model in the financial field based on multi-task self-supervised learning is proposed to simultaneously process the sentiment tendency analysis task and the key entity recognition task at different granularities. The specific content is as follows:

[0165] I. Model pre-training stage

[0166] In the model pre-training stage, a Chinese general domain dataset crawled and collected from Wikipedia and Baidu Encyclopedia is used, and a Chinese financial domain dataset crawled and collected from financial-related news websites and Weibo is used. The dataset constructed after collection is shown in Table 1. The total vocabulary in the dataset reaches 1.0506 billion, and the total size of the dataset reaches 39GB.

[0167] Table 1 Dataset situation used in the pre-training stage

[0168] Dataset Vocabulary size Occupied space Field of application Wikipedia 1.138 billion 5G General field Baidu Encyclopedia 2.002 billion 8G General field Financial news website 4.296 billion 16G Financial field Sina Weibo 3.070 billion 10G Financial field

[0169] The pre-trained language model in the financial domain proposed in this application is transformed from the BERT model. The parameters of the multi-layer Transformer encoder in the pre-trained language model in the financial domain are the same as those of the Transformer encoder in the BERT model. The pre-trained language model in the financial domain is trained on a server with 64 Nvidia v100 GPU cards, and the Adam optimizer is used to optimize the model parameters. Each batch size (training batch) of the model contains 389265 words, and the learning rate is set to 0.000128. To reduce memory consumption and the pre-training time of the model, the pre-trained language model uses a mixed-precision training strategy for multi-task self-supervised learning, using FP16 (half-precision floating-point format) to store activation functions, gradients, and weights, and using FP32 (single-precision floating-point format) to update weight parameters, so as to reduce the memory required for model training and shorten the training time.

[0170] II. Model Fine-tuning Phase

[0171] Since the financial text key entity recognition module has key entity recognition tasks with two granularities, coarse and fine, datasets are constructed for the two granularity tasks respectively. To ensure that the pre-trained language model has sufficient generalization ability, the general domain and financial domain datasets collected by crawling during the pre-training stage are not used. Instead, 20,000 financial texts are crawled from financial-related websites to construct Dataset 1 for sentiment tendency analysis and coarse-grained key entity recognition tasks; 4,500 financial texts are crawled to construct Dataset 2 for fine-grained key entity recognition tasks. Dataset 1 contains two types of financial texts, positive (active) and negative (passive), and also contains a list of entities identified using the LSTM+CRF model; Dataset 2 only contains negative (passive) financial texts and their corresponding manually defined topic labels. Dataset 1 and Dataset 2 are divided into training sets, test sets, and validation sets using the 10-fold cross-validation method respectively.

[0172] Accuracy is used as an indicator to evaluate the sentiment tendency analysis task. In the validation set, M represents the number of texts with correct sentiment tendency predictions by the model, N represents the total number of texts in the validation set, and M / N×100% represents the accuracy of the model prediction.

[0173] The F1 value is used as an indicator to evaluate key entity recognition, and its definition is as follows:

[0174]

[0175]

[0176]

[0177] In the validation set, FN represents the number of key entities that should be key entities but the model fails to recognize; F P represents the number of non - key entities recognized by the model; T P represents the number of key entities correctly recognized by the model.

[0178] In the experiment of sentiment tendency analysis of financial texts, SVM (Support Vector Machine), NBM (Naive Bayes Model), Multi - Bi - LSTM (Multi - Channel Bidirectional Long Short - Term Memory Network), MC - BiGRU - Capsule (Capsule Model integrating Convolutional Neural Network and Bidirectional GRU Network) and BERT model are set as baseline models, and the sentiment tendency analysis model proposed in this application is compared with the baseline models. The experimental results are shown in Table 2.

[0179] Table 2 Experimental Results of Sentiment Tendency Analysis of Financial Texts

[0180] Model name Accuracy rate SVM 70.32 NBM 73.96 Multi - Bi - LSTM 82.40 MC - BiGRU - Capsule 83.15 BERT 91.64 The method of this application 93.96

[0181] From Table 2, it can be seen that the accuracy of the method in this application is higher than that of the baseline models. Compared with the accuracy of SVM, NBM, Multi - Bi - LSTM and MC - BiGRU - Capsule models, there is a significant improvement. The reason is that the pre - trained language model in the financial domain proposed in this application pre - trains the texts in both the general domain and the financial domain at the same time, enabling the pre - trained word vectors to capture context information, and then accurately representing the semantics of the words in financial texts in the downstream sentiment tendency analysis task, thus improving the accuracy of model prediction.

[0182] In the experiment of coarse - grained key entity recognition in financial texts, since the coarse - grained key entity recognition task is transformed into a sentence matching task, BiLSTM - CoVe - Attn (Bidirectional Recurrent Neural Network Model integrating Attention Mechanism), BiLSTM - ELMO - Attn (Bidirectional Recurrent Neural Network Model based on ELMO Representation integrating Attention Mechanism), PAMN (Segment - based Attention Matching Network), DRCN (Recursive Joint Attention - based Matching Model) and BERT model are set as baseline models, and the sentence matching tasks are completed on Dataset 1 respectively. The coarse - grained key entity recognition model proposed in this application is compared with the baseline models. The experimental results are shown in Table 3.

[0183] Table 3 Experimental Results of Coarse - Grained Key Entity Recognition in Financial Texts

[0184] Model name F1 score BiLSTM - CoVe - Attn 83.08 BiLSTM - ELMO - Attn 81.74 PAMN 86.90 DRCN 89.53 BERT 92.17 The method of this application 96.63

[0185] From Table 3, it can be seen that the F1 values of the method of this application are all higher than those of the baseline model. In the experiment, the BERT baseline model used the general domain and financial domain texts mentioned in the method of this application to complete model pre-training. Furthermore, in the fine-tuning stage, the key entity recognition task was also regarded as a sentence matching task to achieve the recognition of a series of key entities in financial texts. However, the F1 value of the method of this application increased by up to 4.46 compared with the BERT model. The reason is that the method of this application added four self-supervised pre-training tasks compared with the BERT model and extended the pre-training task of "predicting whether two sentences in a single text are adjacent" to the self-supervised pre-training task of "predicting whether two sentences in multiple texts are adjacent", so that the pre-trained word vectors contain various semantic knowledge and information, and finally improved the F1 value of key entity recognition.

[0186] In the experiment of fine-grained key entity recognition in financial texts, since the fine-grained key entity recognition task was transformed into a machine reading comprehension task, BiDAF (Bidirectional Attention Flow based Reading Comprehension Model), CogQA (Cognitive Graph based Question Answering Model), DFGN (Dynamic Entity Graph based on Graph Attention Mechanism for Machine Reading Comprehension), QFE (Joint Optimization Model of Passage Extraction and Reading Comprehension) and BERT model were set as baseline models to complete the reading comprehension task on Dataset 2 respectively. The fine-grained key entity recognition model proposed in this application was compared with the baseline models, and the experimental results are shown in Table 4.

[0187] Table 4 Experimental Results of Fine-grained Key Entity Recognition in Financial Texts

[0188] Model name F1 score BiDAF 61.94 CogQA 63.08 DFGN 72.35 QFE 74.21 BERT 85.70 The method of this application 90.18

[0189] From Table 4, it can be seen that the F1 values of the method of this application are all higher than those of the baseline model. Similar to the experimental analysis results of coarse-grained key entity recognition, in the fine-grained key entity recognition task, the method of this application increased the F1 value by 4.48 compared with the BERT model due to the addition of various self-supervised pre-training tasks.

[0190] The above embodiments can achieve the following technical effects: (1) Aiming at the problem that the existing BERT model cannot perform the financial text sentiment tendency analysis task well, the idea of transfer learning is proposed. During the pre-training stage, the corpora in the general domain and the financial domain are pre-trained simultaneously, and the domain knowledge and semantic information of the words in the financial domain are transferred into the model to obtain a large number of pre-trained word vectors that can represent the words in the financial domain. (2) Aiming at the problem of scarce labeled corpora faced by the financial text sentiment tendency analysis task and the key entity recognition task, a pre-trained language model in the financial domain based on multi-task self-supervised learning is proposed. (3) Aiming at the problem that there is currently no method that can better identify the key entities reflecting the text theme in financial texts, the key entity recognition problem is transformed into a sentence matching problem and a machine reading comprehension problem, and the newly proposed pre-trained language model in the financial domain is used to process the key entity recognition tasks at different granularities.

[0191] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown sequentially according to the arrows, these steps do not necessarily have to be executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least some of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages do not necessarily have to be executed at the same time, but can be executed at different times. The execution order of these steps or stages does not necessarily have to be sequential, but can be executed alternately or alternately with at least a part of other steps or steps or stages in other steps.

[0192] Based on the same inventive concept, an embodiment of the present application also provides a financial text recognition device for implementing the financial text recognition method involved above. The implementation solutions for solving problems provided by this device are similar to the implementation solutions described in the above method. Therefore, the specific limitations in one or more embodiments of the financial text recognition device provided below can refer to the limitations on the financial text recognition method in the above text and will not be repeated here.

[0193] In one embodiment, as Figure 8 shown, a financial text recognition device is provided, including: an entity acquisition module 810, a vector determination module 820, a label determination module 830, and an entity screening module 840, where:

[0194] The entity acquisition module 810 is used to acquire the financial text to be recognized and the entity set corresponding to the financial text to be recognized.

[0195] A vector determination module 820, configured to determine a target vector of the financial text to be recognized and a target vector of each entity in the entity set through a pre-trained financial text recognition model.

[0196] A label determination module 830, configured to input the target vector of each entity and the target vector of the financial text to be recognized into a pre-trained first key entity recognition model respectively, to obtain an entity label corresponding to each entity; the entity label is used to characterize whether the corresponding entity is a key entity; the first key entity recognition model is obtained by retraining the pre-trained financial text recognition model.

[0197] An entity screening module 840, configured to screen out the entities whose corresponding entity labels are key entity labels from the entity set as the key entities of the financial text to be recognized.

[0198] In one embodiment, the financial text recognition device further includes a first training module, configured to obtain a sample text and an actual target vector corresponding to the sample text; the sample text includes a general text and a financial text, and the actual target vector is composed of combined vectors of each word in the sample text, and the combined vector is composed of a word vector, a text vector, a position vector, and a task vector corresponding to the corresponding word; input the sample text into the financial text recognition model to be trained to obtain a predicted target vector corresponding to the sample text; train the financial text recognition model to be trained according to the difference between the actual target vector and the predicted target vector until a first preset end condition is met, and use the trained financial text recognition model that meets the first preset end condition as the pre-trained financial text recognition model.

[0199] In one embodiment, the label determination module 830 is further configured to combine the target vector of each entity and the target vector of the financial text to be recognized respectively to obtain a combined target vector of each entity; input the combined target vector of each entity into the pre-trained first key entity recognition model respectively to obtain a key entity probability of each entity; the key entity probability is used to represent the probability that the corresponding entity belongs to a key entity; determine the entity label corresponding to each entity according to the key entity probability of each entity.

[0200] In one embodiment, the financial text recognition device further includes a second training module, configured to obtain a sample financial text, sample entities of the sample financial text, and actual entity labels corresponding to the sample entities; determine a target vector of the sample financial text and a target vector of the sample entities through a pre-trained financial text recognition model; input the target vector of the sample financial text and the target vector of the sample entities into the pre-trained financial text recognition model to obtain predicted entity labels corresponding to the sample entities; re-train the pre-trained financial text recognition model according to the difference between the actual entity labels and the predicted entity labels until a second preset end condition is met, and use the trained financial text recognition model that meets the second preset end condition as the pre-trained first key entity recognition model.

[0201] In one embodiment, the financial text recognition device further includes an entity determination module, configured to obtain a changed theme label of the financial text to be recognized; the changed theme label is obtained by changing the theme label of the financial text to be recognized; determine a target vector of the changed theme label through a pre-trained financial text recognition model; input the target vector of the financial text to be recognized and the target vector of the changed theme label into a pre-trained second key entity recognition model to obtain the key entity positions in the financial text to be recognized; the second key entity recognition model is obtained by re-training the pre-trained financial text recognition model; determine the key entities corresponding to the key entity positions in the financial text to be recognized as the target key entities corresponding to the theme label of the financial text to be recognized.

[0202] In one embodiment, the financial text recognition device further includes a third training module, configured to obtain a sample financial text, sample changed theme labels of the sample financial text, and actual key entity positions in the sample financial text; determine a target vector of the sample financial text and a target vector of the sample changed theme labels through a pre-trained financial text recognition model; input the target vector of the sample financial text and the target vector of the sample changed theme labels into the pre-trained financial text recognition model to obtain predicted key entity positions in the sample financial text; re-train the pre-trained financial text recognition model according to the difference between the actual key entity positions and the predicted key entity positions until a third preset end condition is met, and use the trained financial text recognition model that meets the third preset end condition as the pre-trained second key entity recognition model.

[0203] In one embodiment, the financial text recognition device further includes an emotion recognition module, which is configured to input the financial text to be recognized into a pre-trained text emotion recognition model to obtain the emotion label of the financial text to be recognized; the text emotion recognition model is obtained by retraining the pre-trained financial text recognition model; the key entity, target key entity, and emotion label of the financial text to be recognized are used as the recognition result of the financial text to be recognized.

[0204] In one embodiment, the financial text recognition device further includes a fourth training module, which is configured to obtain a sample financial text and the actual emotion label of the sample financial text; input the sample financial text into the pre-trained financial text recognition model to obtain the predicted emotion label of the sample financial text; retrain the pre-trained financial text recognition model according to the difference between the actual emotion label and the predicted emotion label until the fourth preset end condition is met, and use the trained financial text recognition model that meets the fourth preset end condition as the pre-trained text emotion recognition model.

[0205] Each module in the above financial text recognition device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above respective modules.

[0206] In one embodiment, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as Figure 9 shown. The computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as the key entities of the financial text to be recognized. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a financial text recognition method.

[0207] Those skilled in the art can understand that Figure 9 the structure shown in

[0208] In one embodiment, a computer device is further provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the foregoing method embodiments are implemented.

[0209] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the foregoing method embodiments are implemented.

[0210] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the foregoing method embodiments are implemented.

[0211] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data that have been authorized by the user or fully authorized by all parties.

[0212] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0213] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0214] The above-described embodiments merely represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A financial text recognition method, characterized in that, The method includes: Obtaining the financial text to be recognized and the entity set corresponding to the financial text to be recognized; Determining the target vector of the financial text to be recognized and the target vector of each entity in the entity set through a pre-trained financial text recognition model; Inputting the target vector of each entity and the target vector of the financial text to be recognized into a pre-trained first key entity recognition model respectively, to obtain the entity label corresponding to each entity; the entity label is used to represent whether the corresponding entity is a key entity; the first key entity recognition model is obtained by retraining the pre-trained financial text recognition model; Screening out the entities with the corresponding entity label being the key entity label from the entity set as the key entities of the financial text to be recognized; The method further includes: Obtaining the changed theme label of the financial text to be recognized; the changed theme label is obtained by changing the theme label of the financial text to be recognized; Determining the target vector of the changed theme label through the pre-trained financial text recognition model; Inputting the target vector of the financial text to be recognized and the target vector of the changed theme label into a pre-trained second key entity recognition model, to obtain the key entity position in the financial text to be recognized; the second key entity recognition model is obtained by retraining the pre-trained financial text recognition model; Determining the key entity corresponding to the key entity position in the financial text to be recognized as the target key entity corresponding to the theme label of the financial text to be recognized; Inputting the financial text to be recognized into a pre-trained text sentiment recognition model, to obtain the sentiment label of the financial text to be recognized; the text sentiment recognition model is obtained by retraining the pre-trained financial text recognition model; Taking the key entity, the target key entity and the sentiment label of the financial text to be recognized as the recognition result of the financial text to be recognized.

2. The method according to claim 1, wherein The pre-trained financial text recognition model is trained through the following method: Obtaining the sample text and the actual target vector corresponding to the sample text; the sample text includes general text and financial text, and the actual target vector is composed of the combined vectors of each word in the sample text, and the combined vector is composed of the word vector, the text vector, the position vector and the task vector corresponding to the corresponding word; Inputting the sample text into the financial text recognition model to be trained, to obtain the predicted target vector corresponding to the sample text; Training the financial text recognition model to be trained according to the difference between the actual target vector and the predicted target vector until a first preset end condition is satisfied, and taking the trained financial text recognition model that satisfies the first preset end condition as the pre-trained financial text recognition model.

3. The method according to claim 1, wherein The step of inputting the target vector of each entity and the target vector of the financial text to be recognized into a pre-trained first key entity recognition model respectively, to obtain the entity label corresponding to each entity includes: Combine the target vector of each entity and the target vector of the financial text to be recognized respectively to obtain the combined target vector of each entity; Input the combined target vector of each entity into a pre-trained first key entity recognition model respectively to obtain the key entity probability of each entity; the key entity probability is used to represent the probability that the corresponding entity belongs to a key entity; Determine the entity label corresponding to each entity according to the key entity probability of each entity.

4. The method according to claim 3, wherein The pre-trained first key entity recognition model is trained in the following manner: Obtain a sample financial text, the sample entity of the sample financial text, and the actual entity label corresponding to the sample entity; Determine the target vector of the sample financial text and the target vector of the sample entity through the pre-trained financial text recognition model; Input the target vector of the sample financial text and the target vector of the sample entity into the pre-trained financial text recognition model to obtain the predicted entity label corresponding to the sample entity; Retrain the pre-trained financial text recognition model according to the difference between the actual entity label and the predicted entity label until a second preset end condition is met, and use the trained financial text recognition model that meets the second preset end condition as the pre-trained first key entity recognition model.

5. The method according to claim 1, characterized in that, The pre-trained second key entity recognition model is trained in the following manner: Obtain a sample financial text, the sample change topic label of the sample financial text, and the actual key entity position in the sample financial text; Determine the target vector of the sample financial text and the target vector of the sample change topic label through the pre-trained financial text recognition model; Input the target vector of the sample financial text and the target vector of the sample change topic label into the pre-trained financial text recognition model to obtain the predicted key entity position in the sample financial text; Retrain the pre-trained financial text recognition model according to the difference between the actual key entity position and the predicted key entity position until a third preset end condition is met, and use the trained financial text recognition model that meets the third preset end condition as the pre-trained second key entity recognition model.

6. The method according to claim 1, characterized in that The pre-trained text sentiment recognition model is trained in the following manner: Obtain a sample financial text and the actual sentiment label of the sample financial text; Input the sample financial text into the pre-trained financial text recognition model to obtain the predicted sentiment label of the sample financial text; Retrain the pre-trained financial text recognition model according to the difference between the actual sentiment label and the predicted sentiment label until a fourth preset end condition is met, and use the trained financial text recognition model that meets the fourth preset end condition as the pre-trained text sentiment recognition model.

7. A financial text recognition device, characterized in that, The device includes: An entity acquisition module, configured to acquire a financial text to be recognized and an entity set corresponding to the financial text to be recognized; A vector determination module, configured to determine a target vector of the financial text to be recognized and a target vector of each entity in the entity set through a pre-trained financial text recognition model; A label determination module, configured to input the target vector of each entity and the target vector of the financial text to be recognized into a pre-trained first key entity recognition model respectively, to obtain an entity label corresponding to each entity; the entity label is used to characterize whether the corresponding entity is a key entity; the first key entity recognition model is obtained by retraining the pre-trained financial text recognition model; An entity screening module, configured to screen out entities with corresponding entity labels being key entity labels from the entity set as the key entities of the financial text to be recognized; The apparatus further includes: an entity determination module and an emotion recognition module, The entity determination module is specifically configured to: obtain a changed topic label of the financial text to be recognized; the changed topic label is obtained by changing the topic label of the financial text to be recognized; determine a target vector of the changed topic label through the pre-trained financial text recognition model; input the target vector of the financial text to be recognized and the target vector of the changed topic label into a pre-trained second key entity recognition model, to obtain a key entity position in the financial text to be recognized; the second key entity recognition model is obtained by retraining the pre-trained financial text recognition model; determine the key entity corresponding to the key entity position in the financial text to be recognized as the target key entity corresponding to the topic label of the financial text to be recognized; The emotion recognition module is specifically configured to: input the financial text to be recognized into a pre-trained text emotion recognition model, to obtain an emotion label of the financial text to be recognized; the text emotion recognition model is obtained by retraining the pre-trained financial text recognition model; use the key entity, the target key entity and the emotion label of the financial text to be recognized as the recognition result of the financial text to be recognized.

8. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images