A financial text processing method, device and equipment and storage medium

Abstract

A financial text processing method and device, computer equipment and a storage medium are disclosed to solve the problem of low efficiency in entity and relationship extraction of financial text recognition. The method comprises: replacing numerical information related to a to-be-processed task in the financial text with a numerical mask to obtain a masked text; constructing a task question of the numerical mask according to the to-be-processed task; splicing the masked text, the task question and a task prompt vector to obtain a target text; and predicting a target answer of the task question through a language model according to the target text.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and in particular to a financial text processing method and device, computer equipment and a storage medium. BACKGROUND

[0002] With the development of the financial industry, the demand for information in the financial field is increasing. Information text in the financial field helps relevant personnel obtain effective information such as funds and development trends of the relevant industry. However, the increasing amount of information in the financial field is increasingly chaotic, increasing the difficulty for relevant personnel to obtain effective information. Therefore, it is increasingly important to quickly analyze and process financial text, intelligently extract effective information from massive financial text, and help relevant personnel make high-quality decisions.

[0003] When extracting effective information from financial text, not only entity information needs to be identified, but also the corresponding relationship between entities needs to be extracted, wherein the entity information includes but is not limited to numerical information, time information, company information, etc. Due to the rich entity information in financial text, especially the numerous numerical and time information, various types of information interact closely, so financial text has obvious domain characteristics compared to ordinary text. In the prior art, when performing entity recognition and relationship extraction on ordinary text, the identified entities are usually combined two by two to extract the corresponding relationship between entities. Using this method to perform entity recognition and relationship extraction on financial text will result in low efficiency of information extraction due to the lack of consideration of the domain characteristics of financial text. SUMMARY

[0004] The embodiments of the present application provide a financial text processing method, device, computer equipment and storage medium to solve the problem of low efficiency of entity recognition and relationship extraction of financial text.

[0005] In a first aspect, the present application provides a financial text processing method, comprising:

[0006] replacing numerical information related to a to-be-processed task in the financial text with a numerical mask to obtain a masked text;

[0007] constructing a task question of the numerical mask according to the to-be-processed task;

[0008] concatenating the masked text, the task question and a task prompt vector to obtain a target text;

[0009] predicting a target answer of the task question through a language model according to the target text.

[0010] In one possible design, before concatenating the masked text, the task question and the task prompt vector, the method further comprises:

[0011] splicing the masked text, the task question and the initial task prompt vector to obtain a training text;

[0012] inputting the training text into a language model to predict an initial answer of the task question;

[0013] iteratively adjusting matrix parameters of the initial task prompt vector according to a difference between the expected answer and the initial answer until the initial answer conforms to the expected answer to obtain a task prompt vector.

[0014] In a possible design, before the replacing the numerical information related to the task to be processed in the financial text with a numerical mask to obtain the masked text, the method further includes:

[0015] identifying time information in the financial text;

[0016] judging whether the time information is complete;

[0017] if not, completing the time information.

[0018] In a possible design, the task prompt vector includes a position prompt vector, a first task prompt vector, a second task prompt vector and a third task prompt vector, and the splicing the masked text, the task question and the initial task prompt vector to obtain the training text includes:

[0019] inserting the position prompt vector before the numerical mask of the masked text to obtain a first spliced text;

[0020] inserting the first task prompt vector before the task question to obtain a second spliced text;

[0021] inserting the second task prompt vector after the second spliced text to obtain a third spliced text;

[0022] inserting the first spliced text after the third spliced text to obtain a fourth spliced text;

[0023] inserting the third task prompt vector after the fourth spliced text to obtain the target text.

[0024] In a possible design, the predicting the target answer of the task question according to the target text through the language model includes:

[0025] inputting the target text into the language model to output a first sequence value of a target answer language sequence of the task question;

[0026] Screening a preset fragment area in a context of the first sequence value in the target text, obtaining a target fragment area with the largest probability, and taking a probability corresponding to the target fragment area as a fragment prediction probability;

[0027] Taking a word corresponding to the target prediction probability as a second sequence value of a prediction result language sequence output by the language model;

[0028] The language model outputs the second sequence value after the first sequence value, and obtains a target answer of the task question.

[0029] In a possible design, the screening of the preset fragment area in the context of the first sequence value in the target text and the obtaining of the target fragment area with the largest probability, and the taking of the probability corresponding to the target fragment area as the fragment prediction probability include:

[0030] The similarity between the first sequence value and each text area in the preset fragment area is calculated respectively, and a similarity score of each text area is obtained;

[0031] The similarity scores of all the text areas in the preset fragment area in the context are added as a region probability of the preset fragment area;

[0032] The largest region probability is screened out as the fragment prediction probability.

[0033] In a possible design, the taking of the word corresponding to the target prediction probability as the second sequence value of the prediction result language sequence output by the language model includes:

[0034] If the target prediction probability is the first prediction probability, the end marker is taken as the second sequence value;

[0035] If the target prediction probability is the second prediction probability, the split marker is taken as the second sequence value;

[0036] If the target prediction probability is the text prediction probability, a text word obtained by the text prediction probability is taken as the second sequence value;

[0037] If the target prediction probability is the vocabulary prediction probability, a vocabulary word obtained by the vocabulary prediction probability is taken as the second sequence value;

[0038] If the target prediction probability is the fragment prediction probability, a start word of the preset fragment area corresponding to the fragment prediction probability is taken as the second sequence value.

[0039] In a third aspect, the present application provides a training device for a financial task prompt vector, comprising:

[0040] a replacing module, configured to replace numerical information related to a to-be-processed task in the financial text with a numerical mask to obtain a masked text;

[0041] a constructing module, configured to construct a task question of the numerical mask according to the to-be-processed task;

[0042] a splicing module, configured to splice the masked text, the task question and an initial task prompt vector to obtain a training text;

[0043] a predicting module, configured to input the training text into a language model to predict an initial answer of the task question;

[0044] an outputting module, configured to iteratively adjust matrix parameters of the initial task prompt vector according to a difference between an expected answer and the initial answer until the initial answer meets the expected answer to obtain a task prompt vector.

[0045] In a fourth aspect, the present application provides a computer device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above methods when executing the computer program.

[0046] In a fifth aspect, the present application provides a computer readable storage medium, which stores a computer program, wherein the computer program is executable on a processor to implement the steps of the above methods.

[0047] The training method, the decoding method, the financial text processing method and device, the computer device and the storage medium can identify numerical information related to a to-be-processed task in a financial text according to the field characteristics of the financial text, wherein the to-be-processed task includes but is not limited to entity recognition and relation extraction. Then, the identified numerical information is replaced with a numerical mask to obtain a masked text. Since this operation can identify all numerical information related to the to-be-processed task in the financial text at one time, it is convenient to perform entity recognition and relation extraction on all numerical information at one time subsequently, which greatly improves the processing efficiency of the financial text. Then, a task question of the numerical mask is constructed to convert entity recognition and relation extraction of the financial text into a text question answering task, without pre-setting the number of relation categories, but directly obtaining the relation category name through a subsequent language model. Since the number of categories is not pre-set, the expansibility of the relation extraction method is improved, so that the relation extraction method can be applied to other tasks and fields. Finally, the masked text, the task question and an initial task prompt vector are spliced to obtain a target text, and a target answer of the task question is predicted through a language model according to the target text, which is also the result of entity recognition and relation extraction of the financial text. Attached Figure Description

[0048] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a schematic diagram of an application environment for each method in one embodiment of the present invention;

[0050] Figure 2 This is a flowchart illustrating a method for training financial task prompt vectors according to an embodiment of the present invention;

[0051] Figure 3 This is a flowchart illustrating a financial text processing method according to an embodiment of the present invention;

[0052] Figure 4 This is a schematic diagram of a training device for financial task prompt vectors in one embodiment of the present invention;

[0053] Figure 5 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] The financial text processing method and training method provided in this invention can be applied to, for example... Figure 1 In this application environment, the client's terminal device communicates with the server on the server side via a network. The terminal device collects financial text and transmits it to the server, which then identifies and processes the financial text. The terminal device can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be a standalone server or a server cluster consisting of multiple servers.

[0056] In one embodiment, such as Figure 3 As shown, a method for training financial task cue vectors is provided, which can be applied to... Figure 1 Taking the server in the server-side as an example, the following steps are included:

[0057] S10: Identify the numerical information related to the to-be-processed task in the financial text.

[0058] The terminal device of the client collects the financial text, and the server identifies the numerical information related to the to-be-processed task in the financial text after the financial text is transmitted to the server. The financial text includes but is not limited to enterprise research reports, financial news, etc., and the way of collecting the financial text includes but is not limited to collecting through a crawler tool, collecting user input through an interface, etc. The to-be-processed task includes but is not limited to entity recognition, relationship extraction, etc. The numerical information includes but is not limited to proportion, price, and year-on-year change. The method of identifying the numerical information includes but is not limited to using a regular expression to process the financial text.

[0059] For example, the terminal device collects the financial text “fourth quarter GDP increased by 2.9%, the whole year GDP increased by 3.0%, which provides sufficient space for the economic rebound in 2023, and the GDP in 2023 is expected to exceed 5.5%”, and identifies the numerical information “2.9%”, “3.0%”, and “5.5%” in the text.

[0060] S20: Replace the numerical information in the financial text with a numerical mask to obtain a masked text.

[0061] After obtaining the numerical information in the financial text, the numerical information is replaced with a numerical mask to obtain a masked text with a numerical mask. The numerical mask refers to a marker of the numerical value to be analyzed, including but not limited to a unique identifier and a type identifier.

[0062] For example, the numerical information “2.9%” in the financial text “fourth quarter GDP increased by 2.9%” is obtained, and the numerical information is replaced with <number>, get the mask text as "four seasons GDP growth <number>".

[0063] S30: constructing a task question about the numerical mask according to the to-be-processed task.

[0064] After obtaining the masked text, a task question about the numerical mask is constructed according to the to-be-processed task. The task question includes, but is not limited to, fill-in-the-blank questions, question-and-answer questions, etc., and the scope of the task question includes, but is not limited to, inquiring about the corresponding type of the masked text, inquiring about the subject information of the masked text, etc.

[0065] For example, the obtained masked text is "four seasons GDP growth <number>", the current to-be-processed task is to identify the relationship type of the numerical mask, and then a fill-in-the-blank question, that is, a task question, is constructed according to the to-be-processed task: <number>The relationship type of the two is __.

[0066] S40: Splice the mask text, task question, and initial task prompt vector to obtain a training text.

[0067] After constructing the task question with numerical mask, the mask text, task question, and initial task prompt vector are spliced to obtain a training text. The method for splicing the training text includes but is not limited to splicing the mask text, task question, and initial task prompt vector according to any position to form the training text. The initial task prompt vector is a randomly initialized hidden vector, and there can be one or more. The number of task prompt vectors is not limited here, and multiple task prompt vectors can be split and spliced at different positions to prompt the language model, so as to adjust the downstream task of the language model, that is, adjust the matrix parameters of each task prompt vector respectively, so that the downstream task adapts to the language model. Finally, through the adjustment of the task prompt vector, the text of the input language model is reformed, and the language model obtains the expected answer to the task question.

[0068] For example, the mask text is A, the task question is B, and the initial task prompt vector has 99 vectors. The initial task prompt vector is divided into three parts, C, D, and E, each part having 33 vectors. The final spliced training text is CBDAE.

[0069] S50: Input the training text into the language model to predict an initial answer to the task question.

[0070] After splicing the training text, the training text is input into a pre-trained language model (Language Model) to predict an initial answer to the task question. The pre-trained language model refers to modeling the sentence probability distribution of the training text, including but not limited to CPM-2 (Large-Scale Cost-Effective Pre-Trained Language Models), GPT-2 (Generative Pre-Training), and the like. The initial answer to the task question refers to the answer of the pre-trained language model to the task question constructed in step S30.

[0071] For example, the training text is input into CPM-2 in this embodiment to obtain the sequence output of CPM-2 at each moment.

[0072] S60: According to the difference between the expected answer and the initial answer, iteratively adjust the matrix parameters of the initial task prompt vector until the initial answer conforms to the expected answer to obtain a task prompt vector.

[0073] Based on the initial answer output by the pre-trained language model, determine whether the answer matches the expected answer. If it does not match the expected answer, adjust the matrix parameters of the initial task prompt vector in the training text. Then, re-input the training text carrying the adjusted task prompt vector into the pre-trained language model until the answer from the pre-trained language model matches the expected answer. At this point, the task prompt vector in the training text is the task prompt vector required in step S60.

[0074] For example, if the expected answer is A and the initial answer is B, then determine whether the initial answer B is the expected answer A. When the initial answer B ≠ the expected answer A, adjust the matrix parameters of the task prompt vector in the training text and re-input it into the language model until the language model outputs A. The task prompt vector at this point is the task prompt vector required in step S60.

[0075] It should be noted that this embodiment, based on the domain characteristics of financial text, identifies numerical information related to the task to be processed within the financial text. This task includes, but is not limited to, entity recognition and relation extraction. Then, the identified numerical information is replaced with a numerical mask to obtain masked text. Since this operation can identify all numerical information related to the task to be processed in the financial text at once, it facilitates subsequent entity recognition and relation extraction based on all numerical information, greatly improving the processing efficiency of financial text. Next, by constructing the task question using the numerical mask, the entity recognition and relation extraction of the financial text are transformed into a text question-and-answer task. There is no need to pre-set the number of relation categories; instead, the relation category names are directly obtained through subsequent language models. Because the number of categories cannot be pre-set, the scalability of the relation extraction method is improved, allowing this method to be applied to other tasks and domains. Finally, the masked text, the task question, and the initial task hint vector are concatenated to obtain the training text. This training text is then input into the language model to predict the initial answer to the task question. Based on the difference between the expected answer and the initial answer, the matrix parameters of the initial task hint vector are iteratively adjusted until the initial answer matches the expected answer, thus deriving the task hint vector. This task hint vector training method is based on cue learning. This method freezes the language model's parameters and completes training only by adjusting the matrix parameters of the task hint vector. Since the size of the task hint vector is much smaller than that of the language model, this task hint vector training method saves training time and improves training efficiency.

[0076] In one embodiment, such as Figure 2 As shown, a financial text processing method is provided, which can be applied to... Figure 1 Taking the server in the server-side as an example, the following steps are included:

[0077] S21: Replacing the numerical information related to the to-be-processed task in the financial text with a numerical mask to obtain a masked text.

[0078] The terminal device of the client collects the financial text, and after the financial text is transmitted to the server, the server identifies the numerical information in the financial text according to the to-be-processed task, replaces it with a numerical mask to obtain a masked text, wherein the financial text includes but is not limited to enterprise research reports, financial news, etc., the way of collecting the financial text includes but is not limited to collecting through a crawler tool, collecting user input through an interface, etc., the to-be-processed task includes but is not limited to entity recognition, relationship extraction, etc., and the numerical information includes but is not limited to proportion, price, and ring ratio. The numerical mask refers to the marker of the numerical value to be analyzed, including but not limited to a unique identifier, a type identifier, etc.

[0079] For example, the terminal device collects the financial text "The common debtor accounts receivable are 30,676.99 million yuan, 46,124.28 million yuan, 46,290.36 million yuan, and 46,945.89 million yuan, accounting for 0.35%, 0.36%, 0.27%, and 0.26% of the total assets, respectively", and identifies the numerical value related to the to-be-processed task in the text as "0.35". Replace these numerical values with <number>. Therefore the masked text is "The respective amounts of the joint debtor receivables are RMB 306,7699, RMB 461,2428, RMB 462,9036 and RMB 469,4589, which account for <number>0.36%, 0.27% and 0.26%.

[0080] S22: constructing a task question of the numerical mask according to the to-be-processed task.

[0081] According to the to-be-processed task, a related task question of the numerical mask is constructed. The task question includes, but is not limited to, fill-in-the-blank questions and essay questions, and the scope of the task question includes, but is not limited to, inquiring about the corresponding type of the mask text and inquiring about the subject information of the mask text.

[0082] In this embodiment, the to-be-processed task is entity recognition and relation extraction, so when constructing the task question, the task question for entity recognition and the task question for relation extraction can be constructed respectively, or the entity recognition and the relation extraction can be constructed into one task question, so as to predict the answer of the task question by the subsequent language model.

[0083] For example, the mask text is "The respective amounts of the common debtor receivables are 30,676.99 million yuan, 46,124.28 million yuan, 46,290.36 million yuan and 46,945.89 million yuan, and the proportions in the total assets are <number>, 0.36%, 0.27% and 0.26%”, at which time the identified entity and relation extractions are structured as one fill-in-the-blank question, i.e., the task question: <number>The corresponding type, time, relationship, proportion field, subject is ___.

[0084] S23: Splice the mask text, task question and task prompt vector to obtain the target text, and the task prompt vector is obtained according to the training method of the financial task prompt vector in the above steps.

[0085] The data of the input language model is modified so that the language model derives the correct task question answer through the input data, that is, the task question, the mask text and the task prompt vector trained by the above financial task prompt vector.

[0086] S24: According to the target text, the target answer of the task question is predicted by the language model.

[0087] Through the input data modified by step S23 and the prediction method of the above language model, the predicted answer of the task question is obtained, which includes but is not limited to the recognition result of entity, the extraction result of relationship, etc.

[0088] For example, the mask text is "The respective amounts of joint debtor receivables are 30,676.99 million yuan, 46,124.28 million yuan, 46,290.36 million yuan and 46,945.89 million yuan, accounting for <number>, 0.36%, 0.27% and 0.26%, the task question being "What is the percentage of the following: 0.36%, 0.27% and 0.26%?" <number>The corresponding type, time, relationship, proportion field, and subject are ___. The masked text, task question, and task prompt vector are spliced and modified, input into the language model, and the final predicted answer of the task question is obtained through the prediction method of the language model <start>Percentage type <sep>2019 <sep>Accounts receivable <sep>Total assets <sep>Co-debtor <end>"wherein <start>To start the marker, <sep>to separate the delimiters, <end>A closing tag. From the predicted answer, the results of entity recognition and relation extraction can be seen.

[0089] It should be noted that in the embodiment, the entity recognition and relation extraction are converted into a text generation task, that is, the answer to the task question is obtained through the language model, so that the set of relation categories does not need to be defined, and the fixed number of relation categories does not need to be set in the model, but the names of the relation categories and the entity types are directly generated, that is, the answer of the language model. When new relation categories and corresponding data are added, the model structure and the original data do not need to be changed, and only the task question needs to be constructed as in the embodiment and input into the language model, and the results of entity recognition and relation extraction can be obtained. The method can be flexibly applied to various models and fields, and has good expansibility.

[0090] In an embodiment, before step S21, that is, before the numerical information of the financial text is replaced by the numerical mask to obtain the masked text, the training method further includes the following steps:

[0091] S11: identifying the time information in the financial text.

[0092] S12: determining whether the time information is complete.

[0093] S13: if not complete, then complete the time information.

[0094] In an embodiment, all time information in the financial text is completed.

[0095] Specifically, in step S11, all time information in the financial text is first identified, and the time information refers to all text information that can represent a time period, including but not limited to year, month, date abbreviation, date abbreviation, a time period, a certain time period, etc. For example: "2019 February 2", "last quarter", "this month", etc.

[0096] Then, in steps S12-S13, it is determined whether all the time information identified in step S11 is complete time information, and the complete time information refers to complete time information that can be directly recognized by a machine, including but not limited to incomplete time entity, time description not containing specific time point, etc. If the time information is not complete, the time information is completed, and the method of completing the time information includes but is not limited to supplementing the missing time, replacing the time description with a specific time point, etc.

[0097] For example, the financial text contains the following content: "2019-2021 and the first quarter of 2022 Cumulative box office is 6116.7 million, 5601.2 million, 5001.4 million, 400 million. The number of imported films in the past three years and one period is 136, 62, 73, and 59 respectively." It is identified that "2019-2021" is the missing time information of the time entity, and "the past three years and one period" is the time description without specific time point. After obtaining the identification result, the time information is completed as in step S13: "2019-2021" is completed as "2019, 2020, 2021", and "the past three years" refers to 2020-2022, so "the past three years and one period" is completed as "2020, 2021, 2022 and the first quarter of 2023".

[0098] It should be noted that the particularity of the financial text results in many time information that is easily missed by machines, such as "the past three years" or "2019-2023", but this information is very important in financial analysis. In order to prevent information from being missed due to ambiguous time information during entity recognition or relationship extraction, the embodiment will complete the missing information through certain rules, effectively solving the technical problem of incomplete entity and relationship extraction and easy omission.

[0099] In one embodiment, before step S41, that is, before inserting the position marker into the numerical mask of the mask text, the training method of the financial task prompt vector further includes recording the subscript position of the numerical information related to the to-be-processed task in the financial text. The subscript position is the position where the position marker needs to be inserted.

[0100] In one embodiment, in step S23, that is, the splicing of the mask text, the task question and the task prompt vector to obtain the target text, specifically includes the following steps:

[0101] S41: Insert the position prompt vector before the numerical mask of the mask text to obtain a first spliced text.

[0102] S42: Insert the first task prompt vector before the task question to obtain a second spliced text.

[0103] S43: Insert the second task prompt vector after the second spliced text to obtain a third spliced text.

[0104] S44: Insert the first spliced text after the third spliced text to obtain a fourth spliced text.

[0105] S45: Insert the third task prompt vector after the fourth spliced text to obtain the target text.

[0106] In one embodiment, the mask text, the task question and the task prompt vector are spliced.

[0107] Specifically, a plurality of task prompt vectors are randomly initialized, including but not limited to a position prompt vector, a first task prompt vector, a second task prompt vector and a third task prompt vector. The position prompt vector is inserted before the numerical mask of the mask text to mark the numerical position to be analyzed by the language model. Then, the first task prompt vector, the task question, the second task prompt vector, the mask text after the insertion of the position prompt vector and the third task prompt vector are spliced in sequence to obtain the training text. The number of each prompt vector in the position prompt vector, the first task prompt vector, the second task prompt vector and the third task prompt vector is not limited. The position prompt vector is mainly used to mark the position of the mask text to facilitate the subsequent language model to accurately and conveniently identify the mask text. The first task prompt vector, the second task prompt vector and the third task prompt vector are all randomly initialized hidden vectors, and there can be one or more, which are used to modify the training text of the input language model, so that the final training text of the input language model can obtain accurate target answers without adjusting the language model itself.

[0108] For example, in the present embodiment, 100 task prompt vectors are randomly initialized, including 33 first task prompt vectors A, 33 second task prompt vectors B, 33 third task prompt vectors C and 1 position prompt vector D. The position prompt vector is inserted before the numerical mask of the mask text E to obtain the first spliced text F. The remaining prompt vectors, the first spliced text F and the task question Q are spliced to finally obtain the training text "AQBFC".

[0109] It should be noted that in the present embodiment, based on prompt learning, a plurality of task prompt vectors are randomly initialized to modify the data of the input model, that is, the task prompt vectors are spliced in the input data to train the downstream task of the language model, so as to obtain the target prediction answer of the task answer by adjusting the matrix parameters of the task prompt vectors. The entire training process does not need to adjust the language model parameters, nor does it need to adjust the language model structure. Since the language model is usually a large-scale model, the task prompt vectors are directly trained instead of the language model, which effectively saves the training time and improves the training efficiency.

[0110] In one embodiment, in step S24, the target answer of the task question is predicted by the language model according to the target text, specifically including the following steps:

[0111] S70: replacing the to-be-predicted word in the to-be-processed text with a mask mark to obtain a mask text.

[0112] The terminal device of the client collects the text to be processed, identifies the word to be predicted in the text to be processed, replaces the word to be predicted with a mask mark, and obtains a masked text. The text to be processed refers to the text in which the word or character to be analyzed is located, including but not limited to industry research reports, news articles, etc. The method of collecting the text to be processed includes but is not limited to collecting through a crawler tool, collecting user input through an interface, etc. The word to be predicted refers to the word or character that needs to be analyzed by a language model, including but not limited to letters, numbers, etc. The mask mark refers to the marker of the word or character to be analyzed, including but not limited to a unique identifier, a type identifier, etc.

[0113] For example, the terminal device collects the text to be processed "I like watching movies", and the word to be predicted is "movie". The word to be predicted in the text to be processed is replaced with a mask mark <noun>, then the resulting masked text is "I like to watch <noun>".

[0114] S80: Add the mask label to the task vocabulary.

[0115] After the mask text is constructed, the mask label is added to the task vocabulary. The task vocabulary refers to the list of vectorized results of all predicted words, including but not limited to predicted words, mask labels of predicted words, and vectors of predicted words.

[0116] S90: Input the mask text into the language model to output a first sequence value of the predicted result language sequence.

[0117] The mask text is input into the language model, which will generate the prediction result of the mask text. The prediction result will be output in the form of a language sequence, including but not limited to characters, numbers, start markers, end markers, and separator markers. The prediction result language sequence includes but is not limited to a first sequence value and a second sequence value. In this embodiment, the first sequence value and the second sequence value are sequentially sorted according to the prediction result language sequence. That is, the first sequence value is the sequence that is output first in the prediction result, and the second sequence value is the sequence that is output later.

[0118] For example, "I like to watch <noun>because <noun>Very __”, the language model would come up with a predicted answer for that text, expecting the predicted answer to be "good" <start>Interesting <end>", so the language model will output the language sequence of the predicted answer in turn, that is, output in turn <start>"have", "interesting" and <end>"four language sequences, in the order of the predicted result language sequences," <start>" represents the current output sequence value of the prediction result, which is the first sequence value, and "has" represents the sequence value that the prediction result will soon output, which is the second sequence value. In this embodiment, in step S90, when the language model outputs " <start>"after, according to <start>The next sequence value is predicted, i.e., steps S100-S160.

[0119] S100: Obtain a probability of the end marker occurring after the first sequence value to obtain a first prediction probability, and obtain a probability of the separation marker occurring after the first sequence value to obtain a second prediction probability.

[0120] Since the prediction result output by the language model must contain the start marker and the end marker, the first sequence value of the prediction result must be the start marker. In addition, since each input text can require the output of multiple predicted answers, the separation marker is used to separate each predicted answer, so that all predicted answers are accurately output at one time.

[0121] Therefore, the probabilities of the end marker and the separation marker occurring after the first sequence value are preset, and in step S100, the probabilities of the end marker and the separation marker occurring after the current first sequence value are obtained according to the preset probabilities, i.e., the first prediction probability and the second prediction probability.

[0122] S110: Predict the probability of each word in the task vocabulary occurring after the first sequence value, and take the maximum probability as the vocabulary prediction probability.

[0123] The probabilities of each word in the task vocabulary occurring after the first sequence value are predicted by the language model output after the first sequence value, a plurality of expected probabilities are obtained, the maximum expected probability is taken as the vocabulary prediction probability, and the word with the maximum expected probability is taken as the vocabulary word.

[0124] S120: Predict the probability of each word in the text occurring after the first sequence value, and take the maximum probability as the text prediction probability.

[0125] The probabilities of each word in the text occurring after the first sequence value are predicted by the language model output after the first sequence value, a plurality of expected probabilities are obtained, the maximum expected probability is taken as the text prediction probability, and the word with the maximum expected probability is taken as the text word.

[0126] S130: Screen a preset fragment area in the context of the first sequence value in the text to obtain a target fragment area with the maximum probability, and take the probability corresponding to the target fragment area as the fragment prediction probability.

[0127] The preset method of the context of the first threshold value includes, but is not limited to, specifying a preset number of segment regions, specifying a preset range of text segment regions, etc. Then, the prediction probability of each segment region is calculated, and the segment region with the maximum prediction probability is selected as the segment prediction probability. The probability calculation method of the preset segment region includes, but is not limited to, similarity calculation with the first sequence value, tightness calculation with the first sequence value, etc.

[0128] For example, the text to be processed is "ABCD", the current language model outputs to the sequence value C, so C is taken as the first sequence value. At this time, the segment regions in the context of the first threshold value are "AB" and "D", wherein the similarity of "AB" and "C" is 30%, and the similarity of "C" and "D" is 50%. Therefore, the segment region "D" is selected as the target segment region, and the prediction probability value 50% of "D" is taken as the segment prediction probability, that is, the segment prediction probability is 50%.

[0129] S140: The maximum probability among the first prediction probability, the second prediction probability, the text prediction probability, the word list prediction probability, and the segment prediction probability is selected as the target prediction probability.

[0130] From steps S100-S130, the maximum probability is selected, that is, the probability value of the maximum probability among the first prediction probability, the second prediction probability, the text prediction probability, the word list prediction probability, and the segment prediction probability is selected as the target prediction probability. The target prediction probability is the probability of the sequence value most likely to appear after the first sequence value.

[0131] For example, the first prediction probability is 2%, the second prediction probability is 9%, the text prediction probability is 14%, the word list prediction probability is 7%, and the segment prediction probability is 6%. At this time, the probability value 14% of the text prediction probability is the maximum value, so the text prediction probability is finally taken as the target prediction probability, that is, 14%.

[0132] S150: The word corresponding to the target prediction probability is taken as the second sequence value of the language model.

[0133] After obtaining the target prediction probability, the word corresponding to the target prediction probability is taken as the second sequence value output by the language model.

[0134] For example, the first prediction probability is 2%, the second prediction probability is 9%, the text word with the maximum prediction probability in the text to be processed is A, the text prediction probability is 14%, the vocabulary word with the maximum prediction probability in the vocabulary is B, the vocabulary prediction probability is 7%, the word corresponding to the region with the maximum prediction probability in the preset segment region is C, and the segment prediction probability is 6%. At this time, since the probability value 14% of the text prediction probability is the maximum value among them, the text prediction probability is taken as the target prediction probability. The word corresponding to the target prediction probability is the text word, that is, A. Therefore, the second sequence value output by the language model is A.

[0135] S160: The language model outputs a second sequence value after the first sequence value until the second sequence value is the end marker, and obtains a prediction result of the language model.

[0136] The language model will output a second sequence value after the first sequence value, and repeat the steps of steps S100-160 with the second sequence value as the first sequence value until the second sequence value is the end marker. At this time, the final result output by the language model is its final prediction result.

[0137] For example, the language model will output <start>" and then the " <start>If the first sequence value is used as the basis for predicting the second sequence value as "yes", then the language model will output the second sequence value after the first sequence value. In other words, the language model will output "yes". <start>"have", and then, predicting the second sequence value as "interesting" given the first sequence value of "have", the language model would output "interesting" after "have", and the result of the current output would be <start>Interesting". This loop continues until the second sequence value is the end marker <end>will result in a prediction of the language model <start>Interesting <end>".

[0138] It should be noted that steps S100-S160 are a prediction method of language model restricted decoding provided by the embodiment, in the prediction method, when extracting a single entity, the remaining fragments of the context of the single entity will be searched, that is, step 130, to realize cross-region extraction of entity fragments, so as to realize the extraction of discontinuous entities in the text to be processed, and effectively solve the extraction omission problem caused by the field characteristics of discontinuous text when extracting entities in financial text.

[0139] In an embodiment, in step S150, that is, the word corresponding to the target prediction probability is taken as the second sequence value of the language model, specifically including the following steps:

[0140] S151: If the target prediction probability is the first prediction probability, the end marker is taken as the second sequence value.

[0141] S152: If the target prediction probability is the second prediction probability, the split marker is taken as the second sequence value.

[0142] S153: If the target prediction probability is the text prediction probability, the text word that derives the text prediction probability is taken as the second sequence value.

[0143] S154: If the target prediction probability is the vocabulary prediction probability, the vocabulary word that derives the vocabulary prediction probability is taken as the second sequence value.

[0144] S155: If the target prediction probability is the fragment prediction probability, the start word of the preset fragment region corresponding to the fragment prediction probability is taken as the second sequence value.

[0145] In an embodiment, the word corresponding to the target prediction probability is selected.

[0146] Specifically, in steps S151-S152, if the target prediction probability is the first prediction probability or the second prediction probability, the corresponding end marker or the split marker is directly selected as the second sequence value.

[0147] In steps S153-S154, if the target prediction probability is the text prediction probability or the vocabulary prediction probability, the text word corresponding to the maximum prediction probability in the text to be processed or the vocabulary word corresponding to the maximum prediction probability in the task vocabulary is found as the second sequence value.

[0148] When the target prediction probability is the segment prediction probability in step S155, the starting word in the corresponding preset segment region, i.e., the first word in the segment region, is taken as the second sequence value. For example, the text to be processed is "we are not the same", it is assumed that the current first sequence value is "not", the preset segment regions in the context are "we" and "the same", the prediction probability of "we" is 10%, and the prediction probability of "the same" is 50%, the starting word in the segment region "the same" is selected as the second sequence value, i.e., "one" is taken as the second sequence value.

[0149] It should be noted that in the embodiment, by calculating the prediction probability, all possibilities of the next sequence value can be completely considered, and the word with the largest possibility is selected as the second sequence value, thereby effectively improving the accuracy of the prediction result of the language model.

[0150] In an embodiment, in step S130, i.e., filtering the preset segment region in the context of the first sequence value, the target segment region with the largest probability is obtained, and the probability corresponding to the target segment region is taken as the segment prediction probability, which specifically includes the following steps:

[0151] S131: The similarity between the first sequence value and each text region in the preset segment region is calculated respectively, and the similarity score of each text region is obtained;

[0152] S132: The similarity scores of all text regions in the preset segment region in the context are added to obtain the region probability of the preset segment region;

[0153] S133: The largest region probability is filtered as the segment prediction probability.

[0154] In one of the embodiments, a method for calculating a segment prediction probability is provided.

[0155] Specifically, in step S131, since the preset segment region includes one or more text regions, the similarity between the first sequence value and each text region is calculated respectively, and the similarity score of each text region is obtained. The method for calculating the similarity includes but is not limited to cosine similarity, Pearson correlation coefficient, etc. In steps S132-133, the calculation results of all text regions in the same preset segment region are added to obtain the region probability of the preset region. The largest region probability is selected as the segment prediction probability.

[0156] For example, the text to be processed is "Ren Yin year, the last month of the twenty-eighth", the first sequence value is "month", and the preset fragment region is the previous region "Ren Yin year" and the next region "twenty-eight". The previous region contains four text regions, "Ren", "Yin", "year", and "Laba". The cosine similarity is used to calculate the similarity between each of the four text regions and the first sequence value "month", and the similarity score of each text region is obtained. The sum of all scores is the region probability of the previous region "Ren Yin year", which is 34%. Using the same method, the region probability of the next region "twenty-eight" is 54%. The maximum region probability of 54% is selected as the fragment prediction probability.

[0157] It should be noted that by calculating the similarity between the first sequence value and the context preset fragment region, the most relevant preset fragment region to the first sequence value can be extracted, and the starting word in the preset fragment region is taken as the next sequence value. On the one hand, this effectively improves the accuracy of the prediction results of the language model. On the other hand, since the region of the first sequence value context is searched and its relevance is calculated, even if the answer to the task question is not continuous in the text to be processed, it can be effectively extracted, solving the problem of information omission caused by discontinuous entity extraction.

[0158] To sum up, the application can extract multiple different types of related entities at a time by replacing numerical information in the financial text with mask marks and constructing different task questions according to the mask marks, thereby effectively improving the efficiency of entity extraction and relationship extraction. On the other hand, the input data of the language model is changed by the trained task prompt vector, so that a large-scale language model does not need to be trained, only a small-scale task prompt vector needs to be trained, and the language model can obtain the answer to the task question, that is, the language model can accurately extract entities and relationships, thereby effectively improving the efficiency of entity recognition and relationship extraction. Then, the embodiment improves the output decoding of the language model and proposes a prediction method of the language model. The prediction method can extract cross-regional entity fragments according to the entity context fragments when extracting a single entity, thereby effectively solving the problem of incomplete entity extraction caused by discontinuous entities in the text. Then, the application completes the time information in the financial text according to certain rules according to the characteristics of the financial field, thereby alleviating the problem of missing entity and relationship extraction caused by incomplete information during entity and relationship extraction. Finally, the application provides a technical solution for converting entity recognition and relationship extraction into a text generation task. Unlike the traditional technical solution, the technical solution does not need to set the relationship category set and the number of relationship categories, but directly asks and answers the relationship and entity through the language model, and constructs a task question to make the model directly obtain the relationship and entity name. Since the relationship category set and the number of relationship categories do not need to be set, the technical solution can be flexibly applied to multiple fields and different models, and has good expandability.

[0159] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the application.

[0160] In an embodiment, a financial text processing device is provided, which corresponds to the financial text processing method in the above embodiment. As shown in the figure, the financial text processing device includes a replacement module 10, a construction module 20, a splicing module 30 and an output module 50. The functions of each module are described in detail as follows: Figure 4

[0161] The replacement module 10 is used to replace the numerical information related to the task to be processed in the financial text with a numerical mask to obtain a mask text;

[0162] The construction module 20 is used to construct a task question of the numerical mask according to the task to be processed;

[0163] The splicing module 30 is used to splice the mask text, the task question and the task prompt vector to obtain a target text;​

[0164] The output module 40 is configured to predict the target answer of the task question according to the target text and through a language model.

[0165] The specific limitations of the financial text processing device can refer to the limitations of the financial text processing method described above, which will not be repeated here. Each module in the above financial text processing device can be realized by software, hardware and their combination.

[0166] The above modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so that the processor calls and executes the operations corresponding to the above modules.

[0167] In one embodiment, a computer device is provided, which can be a server or a terminal, and its internal structure diagram can be as shown in Figure 5 The computer device includes a processor, a memory and a network interface connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database of the computer device is configured to store text data collected by the terminal device and data generated in the above method embodiments. The network interface of the computer device is configured to communicate with external terminals through network connection. The computer program is executed by the processor to implement the methods in the above method embodiments.

[0168] Those skilled in the art can understand that Figure 5 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0169] In one embodiment, a computer device is also provided, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0170] In one embodiment, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by the processor to implement the steps in the above method embodiments.

[0171] In an embodiment, a computer program product or computer program is provided, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in each of the above method embodiments.

[0172] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the computer program can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in each embodiment provided in the present application can include at least one of non-volatile and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory. The volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0173] Each technical feature of the above embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of each technical feature in the above embodiments are not described, however, as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0174] The above embodiments only express several implementation manners of the present application, and the description is specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for those of ordinary skill in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of the patent of the present application should be subject to the appended claims.< / end> < / start> < / end> < / start> < / start> < / start> < / start> < / start> < / start> < / start> < / end> < / start> < / end> < / start> < / noun> < / noun> < / noun> < / noun> < / end> < / sep> < / start> < / end> < / sep> < / sep> < / sep> < / sep> < / start> < / number> < / number> < / number> < / number> < / number> < / number> < / number> < / number> < / number> < / number>

Claims

1. A financial text processing method, characterized in that, include: Replace the numerical information related to the task to be processed in the financial text with a numerical mask to obtain the masked text. Based on the task to be processed, construct the task problem of the numerical mask; The target text is obtained by concatenating the masked text, the task question, and the task hint vector. Based on the target text, the target answer to the task question is predicted using a language model; Before concatenating the masked text, the task question, and the task hint vector, the method further includes: The training text is obtained by concatenating the masked text, the task question, and the initial task cue vector. The training text is input into the language model to predict the initial answer to the task question; Based on the difference between the expected answer and the initial answer, the matrix parameters of the initial task prompt vector are iteratively adjusted until the initial answer matches the expected answer, thus obtaining the task prompt vector.

2. The financial text processing method as described in claim 1, characterized in that, Before replacing the numerical information related to the task to be processed in the financial text with a numerical mask to obtain the masked text, the method further includes: Identify the time information in the financial text; Determine whether the time information is complete; If incomplete, complete the time information.

3. The financial text processing method as described in claim 1, characterized in that, The task hint vector includes a location hint vector, a first task hint vector, a second task hint vector, and a third task hint vector. The target text is obtained by concatenating the masked text, the task question, and the task hint vector, including: The position hint vector is inserted before the numerical mask of the masked text to obtain the first concatenated text; Before the task question, insert the first task prompt vector to obtain the second concatenated text; After the second concatenated text, insert the second task prompt vector to obtain the third concatenated text; After the third concatenated text, the first concatenated text is inserted to obtain the fourth concatenated text; After the fourth concatenated text, the third task prompt vector is inserted to obtain the target text.

4. The financial text processing method as described in claim 1, characterized in that, The step of predicting the target answer to the task question based on the target text using a language model includes: Input the target text into the language model, and output the first sequence value of the target answer language sequence for the task question; Filter the preset fragment regions in the context of the first sequence value in the target text to obtain the target fragment region with the highest probability, and use the probability corresponding to the target fragment region as the fragment prediction probability. The word corresponding to the target prediction probability is used as the second sequence value of the predicted language sequence output by the language model; The language model outputs a second sequence value after the first sequence value to obtain the target answer to the task question.

5. The financial text processing method as described in claim 4, characterized in that, The process of filtering a preset segment region within the context of the first sequence value in the target text to obtain the target segment region with the highest probability, and using the probability corresponding to the target segment region as the segment prediction probability, includes: The similarity between the first sequence value and each text region in the preset segment region is calculated to obtain a similarity score for each text region; The similarity scores of all text regions in the preset segment region in the context are added together to obtain the region probability of the preset segment region; The region with the highest probability is selected as the predicted probability of the segment.

6. The financial text processing method as described in claim 4, characterized in that, The step of using the word corresponding to the target prediction probability as the second sequence value of the predicted language sequence output by the language model includes: If the target prediction probability is the first prediction probability, then the end marker is used as the second sequence value; If the target prediction probability is the second prediction probability, then the segmentation marker is used as the second sequence value; If the target prediction probability is a text prediction probability, then the text word from which the text prediction probability is obtained will be used as the second sequence value; If the target prediction probability is a word list prediction probability, then the word list word of the word list prediction probability is used as the second sequence value. If the target prediction probability is a segment prediction probability, then the starting word of the preset segment region corresponding to the segment prediction probability is taken as the second sequence value.

7. A financial text processing apparatus for implementing the method as described in claim 1, characterized in that, include: The replacement module is used to replace the numerical information related to the task to be processed in the financial text with a numerical mask to obtain the masked text. A construction module is used to construct the task problem of the numerical mask based on the task to be processed; The splicing module is used to splice the mask text, task question, and task hint vector to obtain the target text; The output module is used to predict the target answer to the task question based on the target text using a language model.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

Images