Counterfactual data generation method and device

Abstract

The present invention relates to the field of artificial intelligence technology, and provides a method and device for generating counterfactual data. The method comprises: identifying a first sentence based on a predetermined financial sentiment dictionary to determine each financial sentiment word in the first sentence; determining a second sentence corresponding to each financial sentiment word, wherein the second sentence is a sentence after removing the financial sentiment word corresponding to the second sentence from the first sentence; determining the sentiment polarity influence value of each financial sentiment word based on the first sentiment polarity value of the first sentence and the second sentiment polarity value of each second sentence; and determining the counterfactual data of the first sentence based on the sentiment polarity influence value of each financial sentiment word. The method and device for generating counterfactual data provided by the present invention can accurately generate counterfactual data of text in the financial field and significantly save costs.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and device for generating counterfactual data. Background Art

[0002] Existing technologies often use counterfactual data to overcome the problem that deep neural networks are prone to learning spurious relations on language datasets.

[0003] For example, in the task of sentiment analysis in the financial field, the sentiment polarity of a sentence can be reversed with minimal modification. However, generating high-quality augmented data can be expensive and time-consuming, as it usually requires manual feedback and time-consuming and labor-intensive crowdsourcing work. At the same time, this approach also has the following defects: the quality of manual annotation based on crowdsourcing platforms is difficult to guarantee, which affects the quality of manually generated counterfactual data.

[0004] Therefore, how to efficiently and accurately generate counterfactual data for texts in the financial field is of great significance. Summary of the invention

[0005] The present invention provides a method and device for generating counterfactual data, so as to overcome the defects of the prior art in generating counterfactual data of financial field text, such as high cost, low efficiency and poor precision.

[0006] In a first aspect, the present invention provides a method for generating counterfactual data, comprising:

[0007] Recognize the first sentence based on a predetermined financial sentiment dictionary to determine each financial sentiment word in the first sentence;

[0008] Determine a second sentence corresponding to each financial sentiment word, wherein the second sentence is a sentence obtained by removing the financial sentiment word corresponding to the second sentence from the first sentence;

[0009] Determining the sentiment polarity impact value of each financial sentiment word based on the first sentiment polarity value of the first sentence and the second sentiment polarity value of each second sentence;

[0010] Based on the sentiment polarity influence value of each financial sentiment word, counterfactual data of the first sentence is determined.

[0011] In one embodiment, determining the counterfactual data of the first sentence based on the sentiment polarity influence value of each financial sentiment word includes:

[0012] Based on the descending order of the sentiment polarity influence values ​​of the financial sentiment words, the financial sentiment words in the first sentence are replaced in sequence to obtain a plurality of candidate sentences;

[0013] generating counterfactual data of the first sentence according to a target candidate sentence satisfying a preset condition among the multiple candidate sentences;

[0014] The preset conditions include:

[0015] The sentiment polarity of the target candidate sentence is opposite to the sentiment polarity of the first sentence;

[0016] The similarity between the target candidate sentence and the first sentence is greater than the similarity between any other candidate sentence among the multiple candidate sentences and the first sentence.

[0017] In one embodiment, replacing the financial sentiment words in the first sentence includes:

[0018] Generate a substitute word having opposite sentiment polarity to the financial sentiment word in the first sentence based on a bidirectional autoregressive transformer (BART) model, and replace the financial sentiment word in the first sentence with the substitute word;

[0019] The emotional polarity includes negative emotional polarity and positive emotional polarity.

[0020] In one embodiment, the first sentiment polarity value of the first sentence is determined in the following manner:

[0021] Inputting the first sentence into a financial sentiment classifier to obtain a first sentiment polarity value of the first sentence;

[0022] The second sentiment polarity value of the second sentence is determined as follows:

[0023] Inputting the second sentence into the financial sentiment classifier to obtain a second sentiment polarity value of the second sentence;

[0024] The first emotion polarity value or the second emotion polarity value includes a positive emotion polarity value, a neutral emotion polarity value, and a negative emotion polarity value.

[0025] In one embodiment, determining the sentiment polarity impact value of each financial sentiment word based on the first sentiment polarity value of the first sentence and the second sentiment polarity value of each second sentence includes:

[0026] Determine a first absolute value of the difference between the positive emotion polarity value of the first emotion polarity value and the positive emotion polarity value of the second emotion polarity value, a second absolute value of the difference between the neutral emotion polarity value of the first emotion polarity value and the neutral emotion polarity value of the second emotion polarity value, and a third absolute value of the difference between the negative emotion polarity value of the first emotion polarity value and the negative emotion polarity value of the second emotion polarity value;

[0027] A sentiment polarity influence value of the financial sentiment word corresponding to the second sentence is determined based on the first absolute value, the second absolute value, and the third absolute value.

[0028] In one embodiment, the sentiment polarity of the target candidate sentence is opposite to the sentiment polarity of the first sentence, including any of the following:

[0029] The sentiment polarity of the target candidate sentence is positive sentiment polarity, and the sentiment polarity of the first sentence is negative sentiment polarity;

[0030] The sentiment polarity of the target candidate sentence is negative sentiment polarity, and the sentiment polarity of the first sentence is positive sentiment polarity.

[0031] In one embodiment, the similarity between the candidate sentence and the first sentence is determined in the following manner:

[0032] The similarity between the candidate sentence and the first sentence is determined according to the edit distance scoring indicator MoverScore algorithm.

[0033] In a second aspect, the present invention provides a counterfactual data generating device, comprising:

[0034] A word recognition module, used to recognize the first sentence based on a predetermined financial sentiment dictionary to determine each financial sentiment word in the first sentence;

[0035] A word removal module, used to determine a second sentence corresponding to each financial sentiment word, wherein the second sentence is a sentence obtained by removing the financial sentiment word corresponding to the second sentence from the first sentence;

[0036] An influence value determination module, configured to determine the sentiment polarity influence value of each financial sentiment word based on the first sentiment polarity value of the first sentence and the second sentiment polarity value of each second sentence;

[0037] The data generation module is used to determine the counterfactual data of the first sentence based on the sentiment polarity influence value of each financial sentiment word.

[0038] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the first aspect when executing the program.

[0039] In a fourth aspect, the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in the first aspect.

[0040] In a fifth aspect, the present invention provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0041] The counterfactual data generation method and device provided by the present invention accurately locate each financial sentiment word through the financial sentiment dictionary, accurately evaluate the influence of each financial sentiment word on the sentiment polarity of the first sentence according to the first sentiment polarity value of the first sentence and the second sentiment polarity value of each second sentence, and finally determine the counterfactual data of the first sentence based on the sentiment polarity influence value of each financial sentiment word, so as to accurately generate the counterfactual data of the text in the financial field. In addition, the counterfactual data generation method and device provided by the present invention do not require human participation, so compared with the prior art, it also effectively improves the efficiency of generating counterfactual data for text in the financial field and significantly saves costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0043] Figure 1 It is a flowchart of the method for generating counterfactual data provided by the present invention;

[0044] Figure 2 This is a schematic diagram of the results of sentiment analysis of financial texts using the original dataset;

[0045] Figure 3 A schematic diagram of the results of sentiment analysis of financial texts using counterfactual enhanced data generated by the counterfactual data generation method provided by the present invention;

[0046] Figure 4 is a schematic diagram of the structure of the counterfactual data generating device provided by the present invention;

[0047] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] Figure 1 Schematic diagram of the process of generating counterfactual data provided by the present invention. Figure 1 The present invention provides a method for generating counterfactual data, which may include:

[0050] Step 110: Identify the first sentence based on a predetermined financial sentiment dictionary to determine each financial sentiment word in the first sentence;

[0051] Step 120: determining a second sentence corresponding to each financial sentiment word, wherein the second sentence is a sentence obtained by removing the financial sentiment word corresponding to the second sentence from the first sentence;

[0052] Step 130: determining the sentiment polarity influence value of each financial sentiment word based on the first sentiment polarity value of the first sentence and the second sentiment polarity value of each second sentence;

[0053] Step 140: Determine counterfactual data of the first sentence based on the sentiment polarity influence value of each financial sentiment word.

[0054] It should be noted that the executor of the counterfactual data generation method provided by the present invention can be a computer device, such as a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), etc.

[0055] The financial sentiment dictionary may be a predetermined dictionary that includes the sentiment polarity and sentiment polarity score of each financial sentiment word. Financial sentiment words refer to words commonly used in the financial field that can express sentiment polarity. Sentiment polarity includes positive and negative. An example of a financial sentiment dictionary is shown in the following table:

[0056] Table 1 Examples of financial sentiment dictionaries

[0057] Words positive negative Sentiment Polarity Score good √ × 100 not good × √ 0 profit √ × 95 Loss × √ 5 Bear Market × √ 10 Bull Market √ × 90 increase √ × 80 decline × √ 20 efficient √ × 60 invalid × √ 40 Thanks to √ × 65 Blame × √ 35 …… …… …… ……

[0058] Among them, if the sentiment polarity score of a financial sentiment word is ≥50, it can be determined that the sentiment polarity of the financial sentiment word is positive, and the closer the sentiment polarity score of the financial sentiment word is to 100, the stronger the positive sentiment polarity of the financial sentiment word is; if the sentiment polarity score of the financial sentiment word is <50, it can be determined that the sentiment polarity of the financial sentiment word is negative, and the closer the sentiment polarity score of the financial sentiment word is to 0, the stronger the negative sentiment polarity of the financial sentiment word is.

[0059] The first sentence may be firstly identified to determine each financial sentiment word in the first sentence.

[0060] For example, when the first sentence is "A certain country's economic situation is very good this year, thanks to the country's effective control of the epidemic and the growth of export trade", the first sentence is identified through the financial sentiment dictionary, and the financial sentiment words in the first sentence include "good", "benefited from", "effective", "growth", etc.

[0061] After determining the financial sentiment words of the first sentence, second sentences corresponding to the financial sentiment words can be obtained by removing one financial sentiment word respectively.

[0062] For example, for the first statement above, the second statements are:

[0063] The second sentence corresponding to “good” 1: “The economic situation of a certain country this year is very (), thanks to the country’s effective control of the epidemic and the growth of export trade”;

[0064] The second sentence corresponding to "benefited from" is: "A certain country's economic situation is very good this year, which is due to the country's effective control of the epidemic and the growth of export trade";

[0065] The second sentence 3 corresponding to “effective” is: “A certain country’s economic situation is very good this year, thanks to the country’s () control of the epidemic and the growth of export trade”;

[0066] The second sentence corresponding to "growth" 4: "A certain country's economic situation is very good this year, thanks to the country's effective control of the epidemic and export trade ()".

[0067] Next, the sentiment polarity influence value of each financial sentiment word may be determined based on the first sentiment polarity value of the first sentence and the second sentiment polarity value of each second sentence.

[0068] Among them, the sentiment polarity value is used to indicate the degree of each sentiment carried by the sentence. Sentiments include positive sentiment, neutral sentiment, and negative sentiment. When the degree of a certain sentiment is greater than the degrees of the other two sentiments, the sentiment polarity of the sentence can be considered to be the sentiment polarity corresponding to the certain sentiment.

[0069] For example, if the sentiment polarity values ​​of the first sentence are 50% positive sentiment, 20% negative sentiment, and 30% central sentiment, then the sentiment polarity of the first sentence can be considered to be positive.

[0070] The sentiment polarity influence value refers to the influence of the financial sentiment word on the sentiment polarity of the first sentence. The larger the sentiment polarity influence value, the greater the influence of the financial sentiment word on the sentiment polarity of the first sentence.

[0071] Specifically, the sentiment polarity influence value of the financial sentiment word "good" can be determined according to the first sentiment polarity value of the first sentence and the second sentiment polarity value of the second sentence 1; the sentiment polarity influence value of the financial sentiment word "benefit from" can be determined according to the first sentiment polarity value of the first sentence and the second sentiment polarity value of the second sentence 2; the sentiment polarity influence value of the financial sentiment word "effective" can be determined according to the first sentiment polarity value of the first sentence and the second sentiment polarity value of the second sentence 3; the sentiment polarity influence value of the financial sentiment word "growth" can be determined according to the first sentiment polarity value of the first sentence and the second sentiment polarity value of the second sentence 4.

[0072] After determining the sentiment polarity influence value of each financial sentiment word, the counterfactual data of the first sentence may be determined according to the sentiment polarity influence value of each financial sentiment word.

[0073] For example, one or more financial sentiment words may be selected from each financial sentiment word in descending order of sentiment polarity impact values ​​to generate counterfactual data for the first sentence. One or more financial sentiment words may also be selected from each financial sentiment word in descending order of sentiment polarity impact values ​​of each financial sentiment word and in combination with other conditions to generate counterfactual data for the first sentence.

[0074] The counterfactual data generation method provided by the present invention accurately locates each financial sentiment word through a financial sentiment dictionary, accurately evaluates the influence of each financial sentiment word on the sentiment polarity of the first sentence according to the first sentiment polarity value of the first sentence and the second sentiment polarity value of each second sentence, and finally determines the counterfactual data of the first sentence based on the sentiment polarity influence value of each financial sentiment word, so as to accurately generate the counterfactual data of the text in the financial field. In addition, the counterfactual data generation method provided by the present invention does not require human participation, so compared with the prior art, it also effectively improves the efficiency of generating counterfactual data for texts in the financial field and significantly saves costs.

[0075] In one embodiment, the first sentiment polarity value of the first sentence is determined as follows:

[0076] Inputting the first sentence into the financial sentiment classifier to obtain a first sentiment polarity value of the first sentence;

[0077] The second sentiment polarity value of the second sentence is determined as follows:

[0078] Inputting the second sentence into the financial sentiment classifier to obtain a second sentiment polarity value of the second sentence;

[0079] The first sentiment polarity value or the second sentiment polarity value includes a positive sentiment polarity value, a neutral sentiment polarity value, and a negative sentiment polarity value.

[0080] It should be noted that the financial sentiment classifier can be a three-category classifier, such as a decision tree classifier, a selector classifier, an evidence classifier, etc.

[0081] The three-class classifier can be trained according to the training sample set to obtain a financial sentiment classifier, wherein the training data set includes sentences containing financial sentiment words and the sentiment polarity corresponding to each sentence.

[0082] After the first sentence is input into the financial sentiment classifier, the financial sentiment classifier will judge the positive sentiment polarity, neutral sentiment polarity and negative sentiment polarity of the first sentence, and determine the positive sentiment polarity value, neutral sentiment polarity value and negative sentiment polarity value.

[0083] The sentiment polarity of the first sentence can be determined according to the maximum value of the positive sentiment polarity value, the neutral sentiment polarity value, and the negative sentiment polarity value of the first sentence. For example, when the positive sentiment polarity value of the first sentence is greater than the neutral sentiment polarity value and the negative sentiment polarity value, the sentiment polarity of the first sentence can be determined to be positive.

[0084] Similarly, after the second sentence is input into the financial sentiment classifier, the financial sentiment classifier will judge the positive sentiment polarity, neutral sentiment polarity and negative sentiment polarity of the second sentence, and determine the positive sentiment polarity value, neutral sentiment polarity value and negative sentiment polarity value.

[0085] The sentiment polarity of the second sentence can be determined according to the maximum value of the positive sentiment polarity value, the neutral sentiment polarity value, and the negative sentiment polarity value of the second sentence. For example, when the negative sentiment polarity value of the second sentence is greater than the neutral sentiment polarity value and the positive sentiment polarity value, it can be determined that the sentiment polarity of the first sentence is negative.

[0086] The counterfactual data generation method provided by the present invention can avoid the phenomenon of inaccurate and inefficient manual judgment by adopting a financial sentiment classifier to judge the sentiment polarity value of a sentence, and effectively improves the judgment accuracy and efficiency of the sentiment polarity value.

[0087] In one embodiment, determining the sentiment polarity impact value of each financial sentiment word based on the first sentiment polarity value of the first sentence and the second sentiment polarity value of each second sentence may include:

[0088] Determine a first absolute value of a difference between a positive emotion polarity value of the first emotion polarity value and a positive emotion polarity value of the second emotion polarity value, a second absolute value of a difference between a neutral emotion polarity value of the first emotion polarity value and a neutral emotion polarity value of the second emotion polarity value, and a third absolute value of a difference between a negative emotion polarity value of the first emotion polarity value and a negative emotion polarity value of the second emotion polarity value;

[0089] A sentiment polarity influence value of the financial sentiment word corresponding to the second sentence is determined based on the first absolute value, the second absolute value, and the third absolute value.

[0090] Let the positive emotion polarity value of the first emotion polarity value be P1, the neutral emotion polarity value be U1, and the negative emotion polarity value be N1; let the positive emotion polarity value of the second emotion polarity value be P2, the neutral emotion polarity value be U2, and the negative emotion polarity value be N2.

[0091] The first absolute value A1 = |P1-P2|, the second absolute value A2 = |U1-U2|, and the third absolute value A3 = |N1-N2|.

[0092] The sentiment polarity influence value I of the financial sentiment word corresponding to the second sentence may be: I=A1+A2+A3.

[0093] Optionally, the distribution of the sentiment polarity impact value of each financial sentiment word may also be calculated according to the FinSCD algorithm:

[0094] The financial sentiment word is represented by w. According to the FinSCD algorithm, the distribution of the sentiment polarity impact value of the financial sentiment word w is for:

[0095]

[0096] Among them, t -α \w represents the second sentence corresponding to the financial sentiment word w; represents the second sentiment polarity value of the second sentence corresponding to the financial sentiment word w; represents the first sentiment polarity value of the first sentence; represents the sampling sequence of the first sentence; sen(w) represents the sentiment polarity score of the financial sentiment word w in the financial sentiment dictionary. If the financial sentiment word w is not in the financial sentiment dictionary, sen(w) = 0; len(t) represents the word length of the financial sentiment word w.

[0097] The counterfactual data generation method provided by the present invention can achieve accurate evaluation of the degree of influence of financial sentiment words on the sentiment polarity of sentences by determining the sentiment polarity influence value of financial sentiment words according to positive sentiment polarity values, neutral sentiment polarity values ​​and negative sentiment polarity values.

[0098] In one embodiment, step 140 may include:

[0099] Based on the descending order of the sentiment polarity influence values ​​of the financial sentiment words, the financial sentiment words in the first sentence are replaced in sequence to obtain multiple candidate sentences;

[0100] Generate counterfactual data of the first sentence according to a target candidate sentence that meets a preset condition among the multiple candidate sentences;

[0101] Among them, the preset conditions may include:

[0102] The sentiment polarity of the target candidate statement is opposite to that of the first statement;

[0103] The similarity between the target candidate statement and the first statement is greater than the similarity between any other candidate statement in the multiple candidate statements and the first statement.

[0104] For example, for the first statement "The economic situation of a certain country is very good this year, which benefits from the country's effective control of the epidemic and the growth of export trade", if the sentiment polarity influence value of the financial sentiment word "good" is 80, the sentiment polarity influence value of the financial sentiment word "benefit from" is 42, the sentiment polarity influence value of the financial sentiment word "effective" is 55, and the sentiment polarity influence value of the financial sentiment word "growth" is 63, then the order of the sentiment polarity influence values of each financial sentiment word from largest to smallest is: "good", "growth", "effective", "benefit from".

[0105] Based on this order, the "good" in the first statement can be replaced first to obtain the first candidate statement;

[0106] Then, on the basis of the first candidate statement, "growth" can be replaced to obtain the second candidate statement;

[0107] Subsequently, on the basis of the second candidate statement, "effective" can be replaced to obtain the third candidate statement;

[0108] Finally, on the basis of the third candidate statement, "benefit from" can be replaced to obtain the fourth candidate statement.

[0109] Optionally, after determining the multiple candidate statements, other processing can be performed on each candidate statement, such as replacing pseudo-features (such as names of people, places, etc.), and the target candidate statement that meets the preset conditions is selected from the processed multiple candidate statements as the counterfactual data of the first statement.

[0110] It should be noted that in order to improve the efficiency of generating counterfactual data, the present invention can perform the step of replacing financial sentiment words a limited number of times to generate candidate statements, and then perform processing such as pseudo-features (such as names of people, places, etc.). By replacing the financial sentiment word with the largest sentiment polarity influence value first, it is possible to avoid the situation of performing pseudo-feature processing before the most accurate candidate statement is generated, thereby ensuring both the efficiency of generating counterfactual data and the accuracy of counterfactual data.

[0111] In one embodiment, replacing the financial sentiment word in the first statement may include:

[0112] Generate a substitute word with opposite sentiment polarity to the financial sentiment word in the first sentence based on a bidirectional autoregressive transformer (BART) model, and replace the financial sentiment word in the first sentence with the substitute word;

[0113] Among them, the emotional polarity includes negative emotional polarity and positive emotional polarity.

[0114] For example, the BART model can first replace the word "good" in the first sentence with a mask according to the order of the sentiment polarity impact values ​​of financial sentiment words from large to small, and obtain "The economic situation of a certain country this year is very good (mask), thanks to the country's effective control of the epidemic and the growth of export trade";

[0115] Afterwards, the BART model will generate a financial sentiment word "bad" with the opposite sentiment polarity to "good", and replace the mask with "bad", and obtain the first candidate sentence "The economic situation of a certain country is very bad this year, thanks to the country's effective control of the epidemic and the growth of export trade."

[0116] Next, the BART model replaces "growth" in the first candidate sentence with mask, and obtains "A certain country's economic situation this year is very bad, thanks to the country's effective control of the epidemic and export trade (mask)"; the BART model generates the financial sentiment word "decline" with the opposite sentiment polarity to "growth", and replaces "decline" with mask, and obtains the second candidate sentence "A certain country's economic situation this year is very bad, thanks to the country's effective control of the epidemic and the decline in export trade".

[0117] By analogy, the BART model will obtain the third candidate sentence "A certain country's economic situation this year is very bad, thanks to the country's ineffective control of the epidemic and the decline in export trade", and the fourth candidate sentence "A certain country's economic situation this year is very bad, thanks to the country's ineffective control of the epidemic and the decline in export trade."

[0118] It should be noted that BART is a denoising autoencoder that maps noisy documents to original documents. It is implemented as a sequence-to-sequence model that contains an encoder that performs bidirectional encoding on noisy text and a left-to-right autoregressive decoder. The BART model is designed to fine-tune the sequence generation task of counterfactual data. Given an original text sequence, it is first converted into a vector using the BART encoder. In the i-th step of the BART decoder, the output character y generated by the vector converted by the encoder is 1:i-1 is used as input to generate a representation. In this process, in order to generate text sequences with increasingly expected quality, the loss function of BART can be the cross entropy between the output of the decoder and the original text.

[0119] Optionally, the present invention can also use generative pre-training (such as T5, GPT-2) and other models to generate candidate sentences in the first sentence.

[0120] The counterfactual data generation method provided by the present invention can ensure the accuracy of the counterfactual data of the first sentence by generating alternative words with opposite sentiment polarity to the financial sentiment words in the first sentence based on the BART model, and replacing the financial sentiment words in the first sentence with the alternative words to obtain each candidate sentence.

[0121] In one embodiment, the sentiment polarity of the target candidate sentence is opposite to the sentiment polarity of the first sentence, which may include any of the following:

[0122] The target candidate sentence has a positive sentiment polarity, and the first sentence has a negative sentiment polarity.

[0123] The sentiment polarity of the target candidate sentence is negative sentiment polarity, and the sentiment polarity of the first sentence is positive sentiment polarity.

[0124] After obtaining multiple candidate sentences, each candidate sentence can be input into the financial sentiment classifier respectively. The financial sentiment classifier will judge the positive sentiment polarity, neutral sentiment polarity and negative sentiment polarity of the candidate sentence, and determine the positive sentiment polarity value, neutral sentiment polarity value and negative sentiment polarity value.

[0125] The sentiment polarity of the candidate sentence can be determined according to the maximum value of the positive sentiment polarity value, the neutral sentiment polarity value, and the negative sentiment polarity value of the candidate sentence. For example, when the positive sentiment polarity value of the candidate sentence is greater than the neutral sentiment polarity value and the negative sentiment polarity value, the sentiment polarity of the candidate sentence can be determined to be positive.

[0126] It should be noted that by screening out candidate sentences with sentiment polarity opposite to that of the first sentence, it can be ensured that the target candidate language finally obtained meets the definition of counterfactual data, thereby ensuring the correctness of the generated counterfactual data.

[0127] In one embodiment, the similarity between the candidate sentence and the first sentence may be determined according to the edit distance scoring indicator MoverScore algorithm.

[0128] The MoverScore algorithm can screen out automatically generated counterfactual data that has greater similarity to human-generated counterfactual data for data augmentation training.

[0129] For example, the first sentence is "A certain country's economic situation this year is very good, thanks to the country's effective control of the epidemic and the growth of export trade", and its two candidate sentences are candidate sentence 1 "A certain country's economic situation this year is very bad, which is attributed to the country's ineffective control of the epidemic and the decline in export trade", and candidate sentence 2 "A certain country's economic situation this year is very bad, which is attributed to the country's ineffective control of the epidemic and the decline in employment rate".

[0130] By calculating the similarity between the first sentence and candidate sentence 1, and the similarity between the first sentence and candidate sentence 2 respectively through the MoverScore algorithm, it can be determined that the similarity between the first sentence and candidate sentence 1 is greater than the similarity between the first sentence and candidate sentence 2. If candidate sentence 2 simultaneously satisfies that the sentiment polarity of candidate sentence 2 is opposite to the sentiment polarity of the first sentence, candidate sentence 2 can be used as the counterfactual data of the first sentence.

[0131] Optionally, the preset conditions may also include:

[0132] The similarity between the target candidate sentence and the first sentence is greater than the similarity threshold.

[0133] The MoverScore algorithm can be used to calculate the similarities of multiple groups of sample data (each group of sample data includes a first statement and counterfactual data corresponding to the first statement), and then determine the average or minimum value of the similarity, and use the average or minimum value as the similarity threshold.

[0134] By setting a similarity threshold to filter out counterfactual data, it can be ensured that using the filtered counterfactual data in data augmentation (combining with the original data set for training) can train a text classifier device with more robust performance.

[0135] It can be understood that in addition to sentiment classification tasks, the counterfactual data generation method provided by the present invention can serve as a good starting point for studying the effectiveness of automatically generated counterfactual augmented data in building robust systems in many other NLP tasks, including natural language inference (NLI), named entity recognition (NER), question answering (QA) systems, and other tasks.

[0136] The following will specifically illustrate the technical effect of the counterfactual data generation method provided by the present invention by testing the experimental results of various models on the financial sentiment analysis (three-classification) task.

[0137] In addition to traditional machine learning models: SVM, LSTM, LSTM (ELMO), the models used for testing also include pre-trained models based on the transformer structure: BERT-base-uncased model, WWM-BERT-Large model, XLNet-Large model and RoBERTa-Large model.

[0138] The accuracy test results of sentiment analysis of financial texts are as follows: Figure 2 and Figure 3 shown. Figure 2 and Figure 3 The results of sentiment analysis of financial text using the original data set and the counterfactual enhanced data generated by the counterfactual data generation method provided by the present invention are shown respectively. Wherein, Orig. represents the original data, and CAD represents the counterfactual data generated by the counterfactual data generation method provided by the present invention.

[0139] By comparing the experimental results, it can be found that the counterfactual data generation method provided by the present invention can improve the accuracy of all models tested on financial sentiment analysis tasks, especially for non-transformer pre-trained models with smaller models. The specific improvements are as follows: the accuracy of SVM is increased from 76% to 81%; the accuracy of LSTM is increased from 81% to 83%; the accuracy of the model combining LSTM and ELMO word vectors is increased from 84% to 89%.

[0140] On the large-scale pre-trained model based on the transformer, the counterfactual data generation method provided by the invention also achieved statistically significant improvements on BERT-base (91%-94% Macro F1) and BERT-large (94%-96% Macro F1).

[0141] Figure 4 is a schematic diagram of the structure of the counterfactual data generating device provided by the present invention, referring to Figure 4 , the counterfactual data generating device provided by the embodiment of the present invention may include:

[0142] A word recognition module 410 is used to recognize the first sentence based on a predetermined financial sentiment dictionary to determine each financial sentiment word in the first sentence;

[0143] A word removal module 420 is used to determine a second sentence corresponding to each financial sentiment word, wherein the second sentence is a sentence obtained by removing the financial sentiment word corresponding to the second sentence from the first sentence;

[0144] An influence value determination module 430, configured to determine the sentiment polarity influence value of each financial sentiment word based on the first sentiment polarity value of the first sentence and the second sentiment polarity value of each second sentence;

[0145] The data generation module 440 is used to determine the counterfactual data of the first sentence based on the sentiment polarity influence value of each financial sentiment word.

[0146] The counterfactual data generation device provided by the present invention accurately locates each financial sentiment word through a financial sentiment dictionary, and then accurately evaluates the influence of each financial sentiment word on the sentiment polarity of the first sentence according to the first sentiment polarity value of the first sentence and the second sentiment polarity value of each second sentence, and finally determines the counterfactual data of the first sentence based on the sentiment polarity influence value of each financial sentiment word, so as to accurately generate the counterfactual data of the text in the financial field. In addition, the counterfactual data generation device provided by the present invention does not require human participation, so compared with the prior art, it also effectively improves the efficiency of generating counterfactual data for texts in the financial field and significantly saves costs.

[0147] In one embodiment, the data generation module 440 is specifically used to:

[0148] Based on the descending order of the sentiment polarity influence values ​​of the financial sentiment words, the financial sentiment words in the first sentence are replaced in sequence to obtain a plurality of candidate sentences;

[0149] generating counterfactual data of the first sentence according to a target candidate sentence satisfying a preset condition among the multiple candidate sentences;

[0150] The preset conditions include:

[0151] The sentiment polarity of the target candidate sentence is opposite to the sentiment polarity of the first sentence;

[0152] The similarity between the target candidate sentence and the first sentence is greater than the similarity between any other candidate sentence among the multiple candidate sentences and the first sentence.

[0153] In one embodiment, the data generation module 440 is specifically used to:

[0154] Generate a substitute word having opposite sentiment polarity to the financial sentiment word in the first sentence based on a bidirectional autoregressive transformer (BART) model, and replace the financial sentiment word in the first sentence with the substitute word;

[0155] The emotional polarity includes negative emotional polarity and positive emotional polarity.

[0156] In one embodiment, the first sentiment polarity value of the first sentence is determined in the following manner:

[0157] Inputting the first sentence into a financial sentiment classifier to obtain a first sentiment polarity value of the first sentence;

[0158] The second sentiment polarity value of the second sentence is determined as follows:

[0159] Inputting the second sentence into the financial sentiment classifier to obtain a second sentiment polarity value of the second sentence;

[0160] The first emotion polarity value or the second emotion polarity value includes a positive emotion polarity value, a neutral emotion polarity value, and a negative emotion polarity value.

[0161] In one embodiment, the impact value determination module 430 is specifically used to:

[0162] Determine a first absolute value of the difference between the positive emotion polarity value of the first emotion polarity value and the positive emotion polarity value of the second emotion polarity value, a second absolute value of the difference between the neutral emotion polarity value of the first emotion polarity value and the neutral emotion polarity value of the second emotion polarity value, and a third absolute value of the difference between the negative emotion polarity value of the first emotion polarity value and the negative emotion polarity value of the second emotion polarity value;

[0163] A sentiment polarity influence value of the financial sentiment word corresponding to the second sentence is determined based on the first absolute value, the second absolute value, and the third absolute value.

[0164] In one embodiment, the sentiment polarity of the target candidate sentence is opposite to the sentiment polarity of the first sentence, including any of the following:

[0165] The sentiment polarity of the target candidate sentence is positive sentiment polarity, and the sentiment polarity of the first sentence is negative sentiment polarity;

[0166] The sentiment polarity of the target candidate sentence is negative sentiment polarity, and the sentiment polarity of the first sentence is positive sentiment polarity.

[0167] In one embodiment, the similarity between the candidate sentence and the first sentence is determined in the following manner:

[0168] The similarity between the candidate sentence and the first sentence is determined according to the edit distance scoring indicator MoverScore algorithm.

[0169] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communication interface 520 and the memory 530 communicate with each other via the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the steps of the counterfactual data generation method, for example including:

[0170] Recognize the first sentence based on a predetermined financial sentiment dictionary to determine each financial sentiment word in the first sentence;

[0171] Determine a second sentence corresponding to each financial sentiment word, wherein the second sentence is a sentence obtained by removing the financial sentiment word corresponding to the second sentence from the first sentence;

[0172] Determining the sentiment polarity impact value of each financial sentiment word based on the first sentiment polarity value of the first sentence and the second sentiment polarity value of each second sentence;

[0173] Based on the sentiment polarity influence value of each financial sentiment word, counterfactual data of the first sentence is determined.

[0174] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0175] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program, the computer program may be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer can perform the steps of the counterfactual data generation method, for example including:

[0176] Recognize the first sentence based on a predetermined financial sentiment dictionary to determine each financial sentiment word in the first sentence;

[0177] Determine a second sentence corresponding to each financial sentiment word, wherein the second sentence is a sentence obtained by removing the financial sentiment word corresponding to the second sentence from the first sentence;

[0178] Determining the sentiment polarity impact value of each financial sentiment word based on the first sentiment polarity value of the first sentence and the second sentiment polarity value of each second sentence;

[0179] Based on the sentiment polarity influence value of each financial sentiment word, counterfactual data of the first sentence is determined.

[0180] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the counterfactual data generation method, for example, including:

[0181] Recognize the first sentence based on a predetermined financial sentiment dictionary to determine each financial sentiment word in the first sentence;

[0182] Determine a second sentence corresponding to each financial sentiment word, wherein the second sentence is a sentence obtained by removing the financial sentiment word corresponding to the second sentence from the first sentence;

[0183] Determining the sentiment polarity impact value of each financial sentiment word based on the first sentiment polarity value of the first sentence and the second sentiment polarity value of each second sentence;

[0184] Based on the sentiment polarity influence value of each financial sentiment word, counterfactual data of the first sentence is determined.

[0185] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0186] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0187] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating counterfactual data, characterized in that: include: Recognize the first sentence based on a predetermined financial sentiment dictionary to determine each financial sentiment word in the first sentence; Determine a second sentence corresponding to each financial sentiment word, wherein the second sentence is a sentence obtained by removing the financial sentiment word corresponding to the second sentence from the first sentence; Determining the sentiment polarity impact value of each financial sentiment word based on the first sentiment polarity value of the first sentence and the second sentiment polarity value of each second sentence; Determining counterfactual data of the first sentence based on the sentiment polarity influence value of each financial sentiment word includes: Based on the descending order of the sentiment polarity influence values ​​of the financial sentiment words, the financial sentiment words in the first sentence are replaced in sequence to obtain a plurality of candidate sentences; generating counterfactual data of the first sentence according to a target candidate sentence satisfying a preset condition among the multiple candidate sentences; The preset conditions include: The sentiment polarity of the target candidate sentence is opposite to the sentiment polarity of the first sentence; The similarity between the target candidate sentence and the first sentence is greater than the similarity between any other candidate sentence among the multiple candidate sentences and the first sentence.

2. The method for generating counterfactual data according to claim 1, characterized in that: The replacing the financial sentiment words in the first sentence includes: Generate a substitute word having opposite sentiment polarity to the financial sentiment word in the first sentence based on a bidirectional autoregressive transformer (BART) model, and replace the financial sentiment word in the first sentence with the substitute word; The emotional polarity includes negative emotional polarity and positive emotional polarity.

3. The method for generating counterfactual data according to claim 1, characterized in that: The first sentiment polarity value of the first sentence is determined as follows: Inputting the first sentence into a financial sentiment classifier to obtain a first sentiment polarity value of the first sentence; The second sentiment polarity value of the second sentence is determined as follows: Inputting the second sentence into the financial sentiment classifier to obtain a second sentiment polarity value of the second sentence; The first emotion polarity value or the second emotion polarity value includes a positive emotion polarity value, a neutral emotion polarity value, and a negative emotion polarity value.

4. The method for generating counterfactual data according to claim 3, characterized in that: The determining the sentiment polarity impact value of each financial sentiment word based on the first sentiment polarity value of the first sentence and the second sentiment polarity value of each second sentence includes: Determine a first absolute value of the difference between the positive emotion polarity value of the first emotion polarity value and the positive emotion polarity value of the second emotion polarity value, a second absolute value of the difference between the neutral emotion polarity value of the first emotion polarity value and the neutral emotion polarity value of the second emotion polarity value, and a third absolute value of the difference between the negative emotion polarity value of the first emotion polarity value and the negative emotion polarity value of the second emotion polarity value; A sentiment polarity influence value of the financial sentiment word corresponding to the second sentence is determined based on the first absolute value, the second absolute value, and the third absolute value.

5. The method for generating counterfactual data according to claim 1, characterized in that: The sentiment polarity of the target candidate sentence is opposite to the sentiment polarity of the first sentence, including any of the following: The sentiment polarity of the target candidate sentence is positive sentiment polarity, and the sentiment polarity of the first sentence is negative sentiment polarity; The sentiment polarity of the target candidate sentence is negative sentiment polarity, and the sentiment polarity of the first sentence is positive sentiment polarity.

6. The method for generating counterfactual data according to claim 1, characterized in that: The similarity between the candidate sentence and the first sentence is determined in the following manner: The similarity between the candidate sentence and the first sentence is determined according to the edit distance scoring indicator MoverScore algorithm.

7. A counterfactual data generating device, characterized in that: include: A word recognition module, used to recognize the first sentence based on a predetermined financial sentiment dictionary to determine each financial sentiment word in the first sentence; A word removal module, used to determine a second sentence corresponding to each financial sentiment word, wherein the second sentence is a sentence obtained by removing the financial sentiment word corresponding to the second sentence from the first sentence; An influence value determination module, configured to determine the sentiment polarity influence value of each financial sentiment word based on the first sentiment polarity value of the first sentence and the second sentiment polarity value of each second sentence; A data generation module, used to determine the counterfactual data of the first sentence based on the sentiment polarity influence value of each financial sentiment word; The data generation module is specifically used for: Based on the descending order of the sentiment polarity influence values ​​of the financial sentiment words, the financial sentiment words in the first sentence are replaced in sequence to obtain a plurality of candidate sentences; generating counterfactual data of the first sentence according to a target candidate sentence satisfying a preset condition among the multiple candidate sentences; The preset conditions include: The sentiment polarity of the target candidate sentence is opposite to the sentiment polarity of the first sentence; The similarity between the target candidate sentence and the first sentence is greater than the similarity between any other candidate sentence among the multiple candidate sentences and the first sentence.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the counterfactual data generation method according to any one of claims 1 to 6 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the counterfactual data generation method according to any one of claims 1 to 6 are implemented.

Images