A data prediction method, device and equipment applied to accounting subjects and a medium

Abstract

The application discloses a data prediction method and device applied to accounting subjects, equipment and medium, relates to the field of artificial intelligence, and comprises the following steps: obtaining to-be-predicted data, cleaning the to-be-predicted data, determining input data based on the format of preset model prediction input data and the cleaned data; inputting the input data into a target model based on Dice Loss for prediction, obtaining a first prediction label and a first prediction probability; inputting the input data into a target model based on a BERT-Attack adversarial training strategy for prediction, obtaining a second prediction label and a second prediction probability; inputting the input data into a target model based on multi-round undersampling for prediction, obtaining a third prediction label and a third prediction probability; and judging whether the prediction labels are the same, and outputting corresponding target prediction labels and target prediction probabilities according to the judgment result. Therefore, the application can effectively solve the problem of uneven distribution of sample labels of a training model.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and in particular to a data prediction method and device applied to accounting subjects, equipment and medium. BACKGROUND

[0002] An accounting subject is a subject classified and accounted for the specific content of accounting elements according to the content of economic business and the requirements of economic management. Due to the habits of filling in by enterprise financial personnel and the difference in financial software used, the subject name and subject code collected in the system of the tax bureau are different, and cannot be effectively accounted for. After the standardization of enterprise subject data, the increase and decrease of each accounting element can be classified and collected, so that the specific classification accounting index is provided. However, due to the uneven distribution of sample labels in the original subject training data, it presents a clear long-tail distribution (some subject categories even have no samples). This uneven distribution of sample categories may cause the model to pay excessive attention to the majority of categories during training, and thus perform poorly on minority categories, ultimately leading to a large difference in model performance between the training set and the test set. Therefore, how to effectively solve the uneven distribution of sample categories is a problem. SUMMARY

[0003] Therefore, the purpose of the present application is to provide a data prediction method and device applied to accounting subjects, equipment and medium, which can improve the current situation of poor model performance caused by uneven sample category distribution based on the accounting subject standardization processing method of uneven sample distribution. The specific scheme is as follows:

[0004] In a first aspect, the present application discloses a data prediction method applied to accounting subjects, comprising:

[0005] Obtaining the to-be-predicted data corresponding to the accounting subject, cleaning the to-be-predicted data, obtaining the cleaned data, and determining the input data based on the pre-set model prediction input data format and the cleaned data;

[0006] Inputting the input data into the pre-trained target model based on Dice Loss for prediction, obtaining the first prediction label and the first prediction probability corresponding to the input data;

[0007] Inputting the input data into the target model based on the pre-trained BERT-Attack adversarial training strategy for prediction, obtaining the second prediction label and the second prediction probability corresponding to the input data;

[0008] Inputting the input data into the target model based on the pre-trained multi-round undersampling for prediction, obtaining the third prediction label and the third prediction probability corresponding to the input data;

[0009] determining whether the prediction labels are same, and outputting a corresponding target prediction label and a target prediction probability according to a result of the determination.

[0010] Optionally, before the obtaining of the to-be-predicted data corresponding to the accounting subject, the method further includes:

[0011] obtaining target data corresponding to the accounting subject, determining original subject names and original subject codes in the target data, and performing cleaning processing on special symbols, punctuation marks and spaces in the original subject names and the original subject codes to obtain corresponding cleaned subject codes and cleaned subject names;

[0012] determining a label value with the highest occurrence frequency in label values corresponding to the cleaned subject codes and the cleaned subject names as a target label value;

[0013] determining a subject type with the highest occurrence frequency in the target data as a target subject type, and determining a subject direction with the highest occurrence frequency in the target data as a target subject direction;

[0014] determining model training data based on the cleaned subject codes, the cleaned subject names, the target subject type and the target subject direction;

[0015] training a preset accounting subject standardization model by using the model training data, and obtaining the target model.

[0016] Optionally, the determining of the model training data based on the cleaned subject codes, the cleaned subject names, the target subject type and the target subject direction includes:

[0017] determining first data on a first preset bit in the cleaned subject codes;

[0018] determining second data on a second preset bit in the cleaned subject names;

[0019] determining the model training data based on a combination of the first data, the second data, the target subject type and the target subject direction.

[0020] Optionally, the method further includes:

[0021] if a first data quantity of the cleaned subject codes is less than a first preset bit quantity corresponding to the first preset bit, determining a first quantity difference between the first data quantity and the first preset bit quantity, combining a plurality of preset characters corresponding to the first quantity difference with the cleaned subject codes, and determining final first data;

[0022] If the second data quantity of the cleaned subject name is less than the second preset bit number corresponding to the second preset bit, a second quantity difference between the second data quantity and the second preset bit number is determined, and a number of preset characters corresponding to the second quantity difference is combined with the cleaned subject name to determine the final second data.

[0023] Optionally, the training of the preset accounting subject standardization model by using the model training data to obtain the target model comprises:

[0024] The preset accounting subject standardization model is trained by using the model training data based on Dice Loss, the highest first validation set accuracy in respective first validation set accuracies corresponding to a number of training rounds is determined, and the target model based on Dice Loss is determined based on a model corresponding to the highest first validation set accuracy.

[0025] The preset accounting subject standardization model is trained by using the model training data based on a BERT-Attack adversarial training strategy, the highest second validation set accuracy in respective second validation set accuracies corresponding to the number of training rounds is determined, and the target model based on the BERT-Attack adversarial training strategy is determined based on a model corresponding to the highest second validation set accuracy.

[0026] The preset accounting subject standardization model is trained by using the model training data based on a multi-round undersampling method, the highest third validation set accuracy in respective third validation set accuracies corresponding to the number of training rounds is determined, and the target model based on the multi-round undersampling is determined based on a model corresponding to the highest third validation set accuracy.

[0027] Optionally, the cleaning of the to-be-predicted data to obtain cleaned data comprises:

[0028] The special symbols, punctuation marks and spaces corresponding to the subject names and subject codes in the to-be-predicted data are cleaned to obtain cleaned data.

[0029] Optionally, the output of the corresponding target prediction label and target prediction probability according to the judgment result comprises:

[0030] If the prediction labels are all the same, the target prediction probability is determined based on the first prediction probability, the second prediction probability and the third prediction probability, and the prediction label and the target prediction probability are outputted.

[0031] If the prediction labels are not the same, a maximum prediction probability is selected from the first prediction probability, the second prediction probability, and the third prediction probability, the maximum prediction probability is determined as the target prediction probability, and the target prediction probability and a target prediction label corresponding to the maximum prediction probability are output.

[0032] In a second aspect, the present application discloses a data prediction device applied to an accounting subject, comprising:

[0033] An input data determination module is configured to acquire to-be-predicted data corresponding to an accounting subject, clean the to-be-predicted data, acquire cleaned data, and determine input data based on a preset model prediction input data format and the cleaned data.

[0034] A first acquisition module is configured to input the input data into a pre-trained target model based on Dice Loss for prediction, and acquire a first prediction label and a first prediction probability corresponding to the input data.

[0035] A second acquisition module is configured to input the input data into the target model based on a BERT-Attack adversarial training strategy for prediction, and acquire a second prediction label and a second prediction probability corresponding to the input data.

[0036] A third acquisition module is configured to input the input data into the target model based on multi-round undersampling for prediction, and acquire a third prediction label and a third prediction probability corresponding to the input data.

[0037] A label and probability output module is configured to determine whether the prediction labels are the same, and output a corresponding target prediction label and a target prediction probability according to a determination result.

[0038] In a third aspect, the present application discloses an electronic device, comprising:

[0039] A memory is configured to save a computer program.

[0040] A processor is configured to execute the computer program to realize steps of the data prediction method applied to an accounting subject disclosed in the foregoing.

[0041] In a fourth aspect, the present application discloses a computer readable storage medium configured to store a computer program; wherein the computer program is executed by a processor to realize steps of the data prediction method applied to an accounting subject disclosed in the foregoing.

[0042] The application firstly obtains to-be-predicted data corresponding to an accounting subject when predicting data of the accounting subject, cleans the to-be-predicted data, obtains cleaned data, and determines input data based on a preset model prediction input data format and the cleaned data; then inputs the input data into a pre-trained target model based on DiceLoss for prediction, obtains a first prediction label and a first prediction probability corresponding to the input data; then inputs the input data into the target model based on the BERT-Attack adversarial training strategy for prediction, obtains a second prediction label and a second prediction probability corresponding to the input data; then inputs the input data into the target model based on multi-round undersampling for prediction, obtains a third prediction label and a third prediction probability corresponding to the input data; finally, judges whether the prediction labels are the same, and outputs a corresponding target prediction label and a target prediction probability according to the judgment result. It can be seen that the application introduces Dice Loss, adversarial training based on BERT-Attack, multi-round undersampling and other methods while using the target base model to train the weights of each model based on the bagging model idea and the minimum sample category loss function, thereby solving the problem of large performance difference between the training set and the test set caused by uneven label distribution. In this way, the final result is weighted predicted according to the accuracy of different category sample predictions of multiple models, and compared with a method to alleviate the uneven distribution of sample categories, the weighted mode of multiple models makes the model have stronger generalization ability. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor based on the provided drawings.

[0044] Figure 1 A flow chart of a data prediction method applied to an accounting subject is disclosed in the present application.

[0045] Figure 2 A standardization flowchart of an accounting subject is disclosed in the present application.

[0046] Figure 3 A model input data format diagram is disclosed in the present application.

[0047] Figure 4 A specific data prediction method applied to an accounting subject is disclosed in the present application.

[0048] Figure 5 A multi-wheel undersampling method disclosed in the present application schematic diagram;

[0049] Figure 6 A data prediction device structure schematic diagram applied to the accounting subject disclosed in the present application;

[0050] Figure 7 A structure diagram of an electronic device disclosed in the present application. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0052] Traditional subject standardization is generally that professional business personnel formulate a number of rules (generally based on business rules) to correspond the original subject name and the original subject code to the standard subject name and the standard subject code. When the original subject name and the original subject code cannot be one-to-one corresponding through these rules, only the subject correspondence can be relied on the professionalism of the business personnel, which seriously depends on the professionalism of the business personnel and has no way to be implemented and popularized on a large scale. At present, with the rapid development of artificial intelligence technology, the accounting subject standardization method based on artificial intelligence algorithm has gradually become possible. However, due to the particularity of the accounting subject, the use rate of some subjects is low, the original training data of these subjects is less, the training sample class distribution is extremely unbalanced, and a long-tail distribution is obviously presented (some classes even have no samples), which leads to a large difference in the performance of the model on the training set and the test set. In order to solve the above technical problems, the present application discloses a data prediction method applied to the accounting subject, which can improve the current situation that the model performance is poor due to the uneven sample class distribution.

[0053] Referring to Figure 1 The embodiment of the present application discloses a data prediction method applied to the accounting subject, which comprises:

[0054] Step S11, obtaining the to-be-predicted data corresponding to the accounting subject, cleaning the to-be-predicted data, obtaining the cleaned data, and determining the input data based on the pre-set model prediction input data format and the cleaned data.

[0055] In the embodiment, before obtaining the to-be-predicted data corresponding to the accounting subject, the target data corresponding to the accounting subject is first obtained, the original subject name and the original subject code in the target data are determined, and the special symbols, punctuation marks and spaces in the original subject name and the original subject code are cleaned to obtain the corresponding cleaned subject code and cleaned subject name; the label value with the highest occurrence frequency in the label values corresponding to the cleaned subject code and the cleaned subject name is determined as a target label value; the subject type with the highest occurrence frequency in the target data is determined as a target subject type, and the subject direction with the highest occurrence frequency in the target data is determined as a target subject direction; the model training data is determined based on the cleaned subject code, the cleaned subject name, the target subject type and the target subject direction; the preset accounting subject standardization model is trained by using the model training data, and the target model is obtained. It should be noted that the subject for classifying and accounting the specific content of accounting elements according to the content of economic business and the requirements of economic management is referred to as an accounting subject. The general classification of the accounting subject table includes six categories: asset class, liability class, common class, owner's equity class, cost class, profit and loss class, and 162 small subjects. The original subject name refers to the subject name filled in by the company (non-standardized subject name). The original subject code refers to the subject code filled in by the company (non-standardized subject code).

[0056] That is, as shown in Figure 2 , first, the original subject name and the original subject code are cleaned, and the special symbols, punctuation marks, spaces and the like appearing therein are processed to ensure the quality of the training data input. At the same time, since some original subject codes and original subject names after cleaning in the training set are consistent, but the corresponding label values are inconsistent. A relatively simple and practical method is adopted in the present application, and the label value with the highest occurrence frequency is selected as the final label. Then, the subject type and the subject direction are corrected, and the same method as selecting the label is adopted to correct the subject type and the subject direction in the training sample. It should be noted that the subject type: according to the accounting principles and practices, the accounting subject can be divided into the following types: asset class, liability class, equity class, cost class, profit and loss class, and other class. The subject direction: the direction of the accounting subject refers to the direction indicated when the account corresponding to each subject is increased or decreased in the accounting equation, which can be divided into the debit side subject and the credit side subject.

[0057] After determining, a first data in a first preset bit of the cleaned subject code is determined; a second data in a second preset bit of the cleaned subject name is determined; the model training data is determined based on a combination of the first data, the second data, the target subject type and the target subject direction. In addition, if a first data quantity of the cleaned subject code is less than a first preset bit quantity corresponding to the first preset bit, a first quantity difference between the first data quantity and the first preset bit quantity is determined, and a plurality of preset characters corresponding to the first quantity difference are combined with the cleaned subject code to determine the final first data; if a second data quantity of the cleaned subject name is less than a second preset bit quantity corresponding to the second preset bit, a second quantity difference between the second data quantity and the second preset bit quantity is determined, and a plurality of preset characters corresponding to the second quantity difference are combined with the cleaned subject name to determine the final second data. Specifically, as shown in Figure 3 the cleaned subject code first six bits (insufficient six bits are supplemented with [PAD]) + the cleaned subject name first nine bits (insufficient nine bits are supplemented with [PAD]) + the corrected subject type + the corrected subject direction are adopted to construct the format of the model input data. The preset accounting subject standardization model is trained by using the model training data to obtain the target model.

[0058] After the target model is trained in advance, the to-be-predicted data corresponding to the accounting subject is obtained, the to-be-predicted data is cleaned to obtain cleaned data. Similarly, the special symbols, punctuation marks and spaces corresponding to the subject name and the subject code in the to-be-predicted data are cleaned to obtain cleaned data. Then, the input data is determined based on the preset model prediction input data format (as shown in Figure 3 ) and the cleaned data.

[0059] In step S12, the input data is input into the pre-trained target model based on Dice Loss to perform prediction, and a first prediction label and a first prediction probability corresponding to the input data are obtained.

[0060] In the process of training the model, the preset accounting subject standardization model is trained based on the model training data based on Dice Loss, the highest first validation set accuracy in the respective first validation set accuracies corresponding to a plurality of rounds of training is determined, and the target model based on Dice Loss is determined based on the model corresponding to the highest first validation set accuracy. That is, in each epoch, the current best model is saved according to the validation_accuracy (validation set accuracy). Then, the input data is input into the target model based on Dice Loss for prediction to obtain the first prediction label and the first prediction probability corresponding to the input data. It should be pointed out that the target model here can be a BERT (Bidirectional Encoder Representations from Transformers, a pre-training language representation method based on the Transformer model) model, and ALBERT (A Lite Bidirectional Encoder Representations from Transformers, a lightweight BERT model), TinyBERT (Distilling BERT for Natural Language Understanding) and the like can also be used.

[0061] Step S13, inputting the input data into the target model based on the pre-trained BERT-Attack adversarial training strategy for prediction to obtain the second prediction label and the second prediction probability corresponding to the input data.

[0062] In the process of training the model, the preset accounting subject standardization model is trained based on the model training data based on the BERT-Attack adversarial training strategy, the highest second validation set accuracy in the respective second validation set accuracies corresponding to a plurality of rounds of training is determined, and the target model based on the BERT-Attack adversarial training strategy is determined based on the model corresponding to the highest second validation set accuracy. That is, in each epoch, the current best model is saved according to the validation_accuracy (validation set accuracy). Then, the input data is input into the target model based on the BERT-Attack adversarial training strategy for prediction to obtain the second prediction label and the second prediction probability corresponding to the input data. It should be pointed out that BERT-Attack is a training strategy in machine learning, which aims to improve the robustness and generalization ability of the model. This method finds out the words with topk scores as vulnerable words through scoring, and then performs subsequent replacement operations on them.

[0063] Step S14, input the input data into the pre-trained target model based on multi-round undersampling for prediction, to obtain a third predicted label and a third prediction probability corresponding to the input data.

[0064] In this embodiment, in the process of training the model, the method based on multi-round undersampling uses the model training data to train the preset accounting subject standardization model, determines the highest third validation set accuracy in the respective third validation set accuracies corresponding to several rounds of training, and determines the target model based on multi-round undersampling based on the model corresponding to the highest third validation set accuracy. That is, in each epoch, the current best model is saved according to the validation_accuracy (validation set accuracy). Then, the input data is input into the target model based on multi-round undersampling for prediction, to obtain a third predicted label and a third prediction probability corresponding to the input data. There are many methods of undersampling, such as single-round random undersampling, stratified sampling, etc. The present application adopts multi-round random sampling to ensure that the training samples of each category in each round are balanced and all samples can participate in training.

[0065] Step S15, determining whether the predicted labels are the same, and outputting the corresponding target predicted label and target prediction probability according to the determination result.

[0066] In this embodiment, it is determined whether the predicted labels are the same. If all the predicted labels are the same, the target prediction probability is determined based on the first prediction probability, the second prediction probability, and the third prediction probability, and the predicted label and the target prediction probability are output. If the predicted labels are not the same, the maximum prediction probability is selected from the first prediction probability, the second prediction probability, and the third prediction probability, the maximum prediction probability is determined as the target prediction probability, and the target prediction probability and the target prediction label corresponding to the maximum prediction probability are output. In other words, according to the predicted label and the prediction probability of the above three models, if the labels are consistent, the prediction probability is accumulated, and if the predicted labels are not consistent, the result is output according to the maximum prediction probability. In this way, the idea of Bagging (Bootstrap aggregating, bootstrap aggregation algorithm) algorithm is used to minimize the loss function of the label sample with low proportion (less than 1%) to train the weight of each model. Bagging is a technique for reducing the generalization error by combining several models. The main idea is to train several different models respectively, and then let all the models vote the output of the test samples. At the same time, the final result is weighted according to the accuracy of the model in predicting different categories of samples. Compared with one method to alleviate the situation of uneven distribution of sample categories, the weighted way of multiple models makes the model have stronger generalization ability

[0067] As can be seen from the above, in the process of predicting the data of the accounting subject, first, the data to be predicted corresponding to the accounting subject is acquired, the data to be predicted is cleaned, the cleaned data is acquired, and the input data is determined based on the format of the preset model prediction input data and the cleaned data; then the input data is input into the pre-trained target model based on Dice Loss for prediction to acquire the first prediction label and the first prediction probability corresponding to the input data; then the input data is input into the target model based on the BERT-Attack adversarial training strategy for prediction to acquire the second prediction label and the second prediction probability corresponding to the input data; then the input data is input into the target model based on multi-round undersampling for prediction to acquire the third prediction label and the third prediction probability corresponding to the input data; finally, it is judged whether each prediction label is the same, and the corresponding target prediction label and target prediction probability are output according to the judgment result. It can be seen that, while using the target base model, the methods of Dice Loss, adversarial training based on BERT-Attack, and multi-round undersampling are introduced to train the weights of each model based on the bagging model idea and the minimum sample class loss function, so as to solve the problem of large performance difference between the training set and the test set caused by the uneven label distribution. In this way, the final result is weighted predicted according to the accuracy of the different class sample predictions of multiple models, and compared with the method of relieving the uneven distribution of sample classes, the weighted mode of multiple models makes the model have stronger generalization ability.

[0068] Based on the above embodiment, the accounting subject standardization model can be trained by using the model training data to acquire the target model. Next, the process of training the accounting subject standardization model by using the model training data to acquire the target model will be described in detail. Referring to FIG. 8, Figure 4 As shown in FIG. 8, the embodiment of the present application discloses a specific data prediction method applied to an accounting subject, which comprises the following steps:

[0069] In step S21, the accounting subject standardization model is trained by using the model training data based on Dice Loss to determine the highest first validation set accuracy rate in the first validation set accuracy rates corresponding to a plurality of rounds of training, and the target model based on Dice Loss is determined based on the model corresponding to the highest first validation set accuracy rate.

[0070] In this embodiment, the BERT training model loss function is modified, and the Dice Loss model is introduced. Taking binary classification as an example, the original loss function is the cross-entropy loss function:

[0071]

[0072] wherein y ij ∈{0,1} is the actual label value of the i-th sample, p ij is the probability value of the i-th sample predicted as j, and N is the number of samples.

[0073] The original loss function has the same weight for each sample on the objective function, and the sample distribution imbalance will weaken the sample data of the less category, so the cross-entropy loss function with weight is generated:

[0074]

[0075] wherein λ i is the corresponding sample weight. From the actual application, it is difficult to determine the value of λ i , especially when there are many categories.

[0076]

[0077] wherein p i0 , p i1 ∈[0,1], y i0 , y i1 ∈{0,1}, and γ is a smoothing coefficient.

[0078] The Dice Loss idea is applied to the training of the accounting subject standardization model, which greatly solves the major defects in the training process of the accounting subject standardization model, i.e. the poor performance of the model caused by the unbalanced distribution of training sample labels. From the actual application effect, the accuracy of the accounting subject standardization can be improved by about 2%.

[0079] Step S22, training the preset accounting subject standardization model based on the BERT-Attack adversarial training strategy using the model training data, determining the highest second validation set accuracy rate in the respective second validation set accuracy rates corresponding to a plurality of rounds of training, and determining the target model based on the BERT-Attack adversarial training strategy based on the model corresponding to the highest second validation set accuracy rate.

[0080] In this embodiment, one means to improve the robustness of the model is to generate enough samples to attack the existing model. Specifically:

[0081] S=[w1, w2,..., w n ];

[0082] S represents a sample, and inputting it into the fine-tuned model can obtain the probability o y(S), for any word w in the sentence i Its importance is as follows:

[0083]

[0084] in, [MASK] replaces the word "wi" with a mask symbol. This retrieves the importance of each word in a sentence. Sort the words in descending order and select the top ε percent of words as the words to be replaced.

[0085] Adversarial training was first applied in the field of Computer Vision (CV). Traditional adversarial training improves the robustness and generalization ability of a model by adding a perturbation to the original input samples. This application utilizes the natural characteristic of BERT's masked language model for predicting masked words through BERT-Attack, applying it to the synonym generation part. Compared with other adversarial training methods, BERT-Attack is much more efficient and also syntactically and semantically correct and consistent.

[0086] Step S23: Using the model training data, train the preset accounting subject standardization model based on the multi-round undersampling method, determine the highest third validation set accuracy among the third validation set accuracy corresponding to each of the several rounds of training, and determine the target model based on the model corresponding to the highest third validation set accuracy.

[0087] In this embodiment, the model is trained through multiple rounds of undersampling on the original training samples. The specific undersampling method is as follows, taking binary classification as an example: Figure 5 As shown, multiple rounds of random sampling ensure that the training samples of each category remain balanced in each round and that all samples can participate in training. Undersampling is common in machine learning model training, but most of it is based on random sampling, which often discards some samples, resulting in the ineffective use of all samples. This application adopts a novel multi-round undersampling mode to train the model, ensuring that all sample data can participate in model training, and maintaining a relatively balanced state of sample data of all categories in each round of training, thus solving the problem of imbalanced sample distribution while ensuring the integrity of training data.

[0088] The classification result of the sample with a high label proportion is accurate enough in steps S21 to S23. Therefore, the training of the model weight in steps S21 to S23 focuses on minimizing the loss function of the label sample with a low actual proportion (a category with a proportion less than 1%). The essence of model training is to minimize the loss function, and when the sample data amount of a certain category is significantly greater than that of other categories, the loss function is affected by the category with a larger sample amount. For example, the sample amount of the label “main business income” is 9500, and the sample amount of the label “non-operating income” is 500. Even if all labels are predicted as “main business income” during the model training process, the accuracy rate can still reach 95%. The loss function of the few-category sample of the model s is as follows:

[0089]

[0090] wherein m is the sample amount, x ij represents the label of sample x i , p(x ij ) represents the actual value of the label of sample x i , and q(x ij ) represents the predicted value of the label of sample x i .

[0091] The total loss function of the model in steps S21 to S23 is as follows:

[0092]

[0093] wherein, β i is the loss function weight of the i-th model.

[0094] In this way, the final result is weighted and predicted according to the accuracy of the prediction of different categories of samples by multiple models, compared with a method for relieving the uneven distribution of sample categories, and the weighted mode of multiple models makes the model have stronger generalization ability.

[0095] Referring to FIG. 1, Figure 6 the embodiment of the present application discloses a data prediction device applied to accounting subjects, comprising:

[0096] An input data determination module 11 is configured to acquire to-be-predicted data corresponding to an accounting subject, clean the to-be-predicted data, acquire cleaned data, and determine input data based on a preset model prediction input data format and the cleaned data;

[0097] A first acquisition module 12 is configured to input the input data into a pre-trained target model based on Dice Loss for prediction, acquire a first predicted label and a first prediction probability corresponding to the input data.

[0098] The second obtaining module 13 is configured to input the input data into the target model based on the BERT-Attack adversarial training strategy which is pre-trained to perform prediction, and obtain a second predicted label and a second predicted probability corresponding to the input data.

[0099] The third obtaining module 14 is configured to input the input data into the target model based on multi-round undersampling which is pre-trained to perform prediction, and obtain a third predicted label and a third predicted probability corresponding to the input data.

[0100] The label and probability output module 15 is configured to determine whether the predicted labels are the same, and output a corresponding target predicted label and a target predicted probability according to the determination result.

[0101] As can be seen from the above, when predicting the data of the accounting subject, the application first obtains the to-be-predicted data corresponding to the accounting subject, cleans the to-be-predicted data, obtains the cleaned data, and determines the input data based on the pre-set model prediction input data format and the cleaned data. Then, the input data is input into the target model based on Dice Loss which is pre-trained to perform prediction, and a first predicted label and a first predicted probability corresponding to the input data are obtained. Then, the input data is input into the target model based on the BERT-Attack adversarial training strategy which is pre-trained to perform prediction, and a second predicted label and a second predicted probability corresponding to the input data are obtained. Then, the input data is input into the target model based on multi-round undersampling which is pre-trained to perform prediction, and a third predicted label and a third predicted probability corresponding to the input data are obtained. Finally, it is determined whether the predicted labels are the same, and a corresponding target predicted label and a target predicted probability are output according to the determination result. It can be seen that the application introduces Dice Loss, adversarial training based on BERT-Attack, multi-round undersampling and other methods while using the target base model to train the weights of each model based on the bagging model idea and the minimum sample class loss function. The problem of large difference between the training set and the test set caused by the uneven distribution of labels is solved. In this way, the final result is weighted predicted according to the accuracy of the different class sample predictions of multiple models, and compared with one method to alleviate the uneven distribution of sample classes, the weighted mode of multiple models makes the model have stronger generalization ability.

[0102] In some specific embodiments, the device can also be used to obtain target data corresponding to the accounting subject, determine original subject names and original subject codes in the target data, and clean special symbols, punctuation marks and spaces in the original subject names and the original subject codes to obtain corresponding cleaned subject codes and cleaned subject names; determine the label value with the highest occurrence frequency in the label values corresponding to the cleaned subject codes and the cleaned subject names as a target label value; determine the subject type with the highest occurrence frequency in the target data as a target subject type, and determine the subject direction with the highest occurrence frequency in the target data as a target subject direction; determine model training data based on the cleaned subject codes, the cleaned subject names, the target subject type and the target subject direction; train a preset accounting subject standardization model using the model training data to obtain the target model.

[0103] In some specific embodiments, the device can be specifically used to determine first data on a first preset bit in the cleaned subject code; determine second data on a second preset bit in the cleaned subject name; determine the model training data based on a combination of the first data, the second data, the target subject type and the target subject direction.

[0104] In some specific embodiments, the device can also be used to, if the number of first data of the cleaned subject code is less than a first preset bit number corresponding to the first preset bit, determine a first quantity difference between the number of first data and the first preset bit number, and combine a number of preset characters corresponding to the first quantity difference with the cleaned subject code to determine final first data; if the number of second data of the cleaned subject name is less than a second preset bit number corresponding to the second preset bit, determine a second quantity difference between the number of second data and the second preset bit number, and combine a number of preset characters corresponding to the second quantity difference with the cleaned subject name to determine final second data.

[0105] In some specific embodiments, the device can be specifically used for training the preset accounting subject standardization model based on the model training data, determining the highest first validation set accuracy in each first validation set accuracy corresponding to a plurality of rounds of training, determining the target model based on Dice Loss based on the model corresponding to the highest first validation set accuracy, training the preset accounting subject standardization model based on the model training data based on the BERT-Attack adversarial training strategy, determining the highest second validation set accuracy in each second validation set accuracy corresponding to a plurality of rounds of training, and determining the target model based on the BERT-Attack adversarial training strategy based on the model corresponding to the highest second validation set accuracy, and training the preset accounting subject standardization model based on the model training data based on the multi-round undersampling method, determining the highest third validation set accuracy in each third validation set accuracy corresponding to a plurality of rounds of training, and determining the target model based on the multi-round undersampling based on the model corresponding to the highest third validation set accuracy.

[0106] In some specific embodiments, the input data determination module 11 can be specifically used for cleaning the special symbols, punctuation marks and spaces corresponding to the subject name and subject code in the to-be-predicted data to obtain cleaned data.

[0107] In some specific embodiments, the label and probability output module 15 can be specifically used for determining the target prediction probability based on the first prediction probability, the second prediction probability and the third prediction probability and outputting the prediction label and the target prediction probability if each of the prediction labels is the same, and selecting the maximum prediction probability from the first prediction probability, the second prediction probability and the third prediction probability, determining the maximum prediction probability as the target prediction probability, and outputting the target prediction probability and the target prediction label corresponding to the maximum prediction probability if each of the prediction labels is different.

[0108] Further, the embodiment of the present application also discloses an electronic device, Figure 7 is a structural diagram of an electronic device 20 according to an exemplary embodiment, and the content in the figure cannot be considered as any limitation on the use range of the present application.

[0109] Figure 7A structural schematic diagram of an electronic device 20 is provided in the embodiments of the present application. The electronic device 20 can specifically include at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25 and a communication bus 26. The memory 22 is configured to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the related steps in the data prediction method for accounting subjects disclosed in any of the foregoing embodiments. In addition, the electronic device 20 in the embodiments of the present application can be specifically an electronic computer.

[0110] In the embodiments of the present application, the power supply 23 is configured to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol followed by the communication interface 24 can be any communication protocol applicable to the technical solution of the present application, which is not limited specifically herein; the input / output interface 25 is configured to obtain external input data or output data to the outside, and the specific interface type can be selected according to the specific application needs, which is not limited specifically herein.

[0111] In addition, the memory 22 as a carrier for resource storage can be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., and the resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage mode can be temporary storage or permanent storage.

[0112] The operating system 221 is configured to manage and control each hardware device on the electronic device 20 and the computer program 222, and can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program for completing the data prediction method for accounting subjects executed by the electronic device 20 disclosed in any of the foregoing embodiments, the computer program 222 can further include computer programs for completing other specific work.

[0113] Further, the present application further discloses a computer readable storage medium for storing a computer program; wherein the computer program is executed by the processor to implement the data prediction method for accounting subjects disclosed in the foregoing embodiments. The specific steps of the method can refer to the corresponding contents disclosed in the foregoing embodiments, which will not be repeated here.

[0114] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts of each embodiment can be referred to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the relevant parts can refer to the method part.

[0115] Those skilled in the art will further appreciate that the units and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various examples have been described herein in terms of their functionality, which has been described generally and symbolically in flow charts. Having thus described the functionality of the examples, a person of ordinary skill in the art will be able to implement such functionality in hardware and / or software, and will recognize that the bounds of the examples are not limited by one approach or the other. The various examples can be realized in a centralized fashion in one computer system or network, or in a distributed fashion where different elements are spread across several computer systems or sub-networks. Any kind of computer system or other apparatus adapted for carrying out the methods described herein is suited. A typical combination of hardware and software could be a general purpose computer system with a computer program that, when being loaded and executed, carries out the methods described herein.

[0116] The steps of a method or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in random access memory (RAM), flash memory, read-only memory (ROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such that the processor can read information from, and write information to, the storage medium. In the alternative, hard disk can be used as a storage medium.

[0117] Finally, it should be noted that the terms "first", "second", and the like, herein do not denote any order, quantity, combination, or importance, but rather are used to distinguish one element from another, and do not imply singular or plural. Moreover, the terms "include", "have", or any other variant thereof are intended to encompass non-exclusive inclusions, such that processes, methods, articles, or apparatuses that comprise a set of elements not expressly listed are also within the scope of the present application. In addition, the articles "a" and "an" are used herein to refer to one or to more than one (i.e., to one or at least one) of the grammatical object of the article. By way of example, "an element" means one element or one or more elements.

[0118] The above detailed description of the technical solutions provided by the present application has been described in detail, and the principles and implementation modes of the present application have been described in the above examples. The above example is only used to help understand the method and core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed; in view of the above, the content of the specification should not be understood as limiting the present application.

Claims

1. A data prediction method applied to accounting subjects, characterized by, The method comprises the following steps: acquiring to-be-predicted data corresponding to an accounting subject, cleaning the to-be-predicted data, acquiring cleaned data, and determining input data based on a preset model prediction input data format and the cleaned data; inputting the input data into a pre-trained first target model based on Dice Loss for prediction, acquiring a first prediction label and a first prediction probability corresponding to the input data; inputting the input data into a pre-trained second target model based on a BERT-Attack adversarial training strategy for prediction, acquiring a second prediction label and a second prediction probability corresponding to the input data; inputting the input data into a pre-trained third target model based on multi-round undersampling for prediction, acquiring a third prediction label and a third prediction probability corresponding to the input data; determining whether the prediction labels are the same, and outputting a corresponding target prediction label and target prediction probability according to the determination result; Before the step of acquiring to-be-predicted data corresponding to an accounting subject, the method further comprises the following steps: acquiring target data corresponding to the accounting subject, determining original subject names and original subject codes in the target data, and cleaning special symbols, punctuation marks and spaces in the original subject names and the original subject codes to obtain corresponding cleaned subject codes and cleaned subject names; determining a label value with the highest occurrence frequency in label values corresponding to the cleaned subject codes and the cleaned subject names as a target label value; determining a subject type with the highest occurrence frequency in the target data as a target subject type, and determining a subject direction with the highest occurrence frequency in the target data as a target subject direction; determining model training data based on the cleaned subject codes, the cleaned subject names, the target subject type and the target subject direction; training a preset accounting subject standardization model using the model training data to obtain a target model; The step of training a preset accounting subject standardization model using the model training data to obtain a target model comprises the following steps: training the preset accounting subject standardization model based on Dice Loss using the model training data, determining a highest first validation set accuracy rate in respective first validation set accuracies corresponding to a plurality of rounds of training, and determining a first target model based on Dice Loss based on a model corresponding to the highest first validation set accuracy rate; training the preset accounting subject standardization model based on a BERT-Attack adversarial training strategy using the model training data, determining a highest second validation set accuracy rate in respective second validation set accuracies corresponding to the plurality of rounds of training, and determining a second target model based on the BERT-Attack adversarial training strategy based on a model corresponding to the highest second validation set accuracy rate; The method based on multi-round under-sampling is used to train the preset accounting subject standardization model based on the model training data, to determine a highest third verification set accuracy rate in respective third verification set accuracies corresponding to a plurality of rounds of training, and to determine a third target model based on multi-round under-sampling based on a model corresponding to the highest third verification set accuracy rate; The outputting of the corresponding target prediction label and target prediction probability according to the judgment result comprises: If the prediction labels are all the same, the first prediction probability, the second prediction probability, and the third prediction probability are accumulated, a corresponding accumulated prediction probability is determined as the target prediction probability, and the prediction label and the target prediction probability are outputted; Otherwise, the maximum prediction probability is selected from the first prediction probability, the second prediction probability, and the third prediction probability, the maximum prediction probability is determined as the target prediction probability, and the target prediction probability and the target prediction label corresponding to the maximum prediction probability are outputted.

2. The data prediction method for accounting subjects according to claim 1, characterized in that, The determination of the model training data based on the cleaned subject code, the cleaned subject name, the target subject type, and the target subject direction comprises: determining first data on a first preset bit in the cleaned subject code; determining second data on a second preset bit in the cleaned subject name; determining the model training data based on a combination of the first data, the second data, the target subject type, and the target subject direction.

3. The data prediction method for accounting accounts according to claim 2, characterized in that, Further comprising: If the number of first data of the cleaned subject code is less than a first preset bit number corresponding to the first preset bit, a first number difference between the number of first data and the first preset bit number is determined, a plurality of preset characters corresponding to the first number difference are combined with the cleaned subject code to determine the final first data; If the number of second data of the cleaned subject name is less than a second preset bit number corresponding to the second preset bit, a second number difference between the number of second data and the second preset bit number is determined, a plurality of preset characters corresponding to the second number difference are combined with the cleaned subject name to determine the final second data.

4. The data prediction method for accounting subjects according to claim 1, characterized in that, The cleaning of the to-be-predicted data to obtain cleaned data comprises: cleaning special symbols, punctuation symbols, and spaces corresponding to subject names and subject codes in the to-be-predicted data to obtain cleaned data.

5. A data prediction device for accounting subjects, characterized in that, Comprise: an input data determination module configured to obtain to-be-predicted data corresponding to accounting subjects, clean the to-be-predicted data to obtain cleaned data, and determine input data based on a preset format of model prediction input data and the cleaned data; a first obtaining module configured to input the input data into a pre-trained first target model based on Dice Loss to predict, and obtain a first prediction label and a first prediction probability corresponding to the input data; The second obtaining module is configured to input the input data into a second target model based on a BERT-Attack adversarial training strategy that is pre-trained to perform prediction, and obtain a second predicted label and a second predicted probability corresponding to the input data; The third obtaining module is configured to input the input data into a third target model based on a multi-round undersampling method that is pre-trained to perform prediction, and obtain a third predicted label and a third predicted probability corresponding to the input data; The label and probability output module is configured to determine whether the predicted labels are the same, and output a target predicted label and a target predicted probability according to a determination result; The device is configured to obtain target data corresponding to the accounting subject, determine an original subject name and an original subject code in the target data, and perform cleaning processing on special symbols, punctuation symbols and spaces in the original subject name and the original subject code to obtain a cleaned subject code and a cleaned subject name; A label value with the highest frequency in label values corresponding to the cleaned subject code and the cleaned subject name is determined as a target label value; A subject type with the highest frequency in the target data is determined as a target subject type, and a subject direction with the highest frequency in the target data is determined as a target subject direction; Model training data is determined based on the cleaned subject code, the cleaned subject name, the target subject type and the target subject direction; A target model is obtained by training a preset accounting subject standardization model using the model training data; The device is configured to train the preset accounting subject standardization model using the model training data based on Dice Loss, determine a highest first validation set accuracy rate in respective first validation set accuracies corresponding to a plurality of rounds of training, determine a first target model based on Dice Loss based on a model corresponding to the highest first validation set accuracy rate, train the preset accounting subject standardization model using the model training data based on a BERT-Attack adversarial training strategy, determine a highest second validation set accuracy rate in respective second validation set accuracies corresponding to the plurality of rounds of training, determine a second target model based on the BERT-Attack adversarial training strategy based on a model corresponding to the highest second validation set accuracy rate, train the preset accounting subject standardization model using the model training data based on a multi-round undersampling method, determine a highest third validation set accuracy rate in respective third validation set accuracies corresponding to the plurality of rounds of training, and determine a third target model based on the multi-round undersampling method based on a model corresponding to the highest third validation set accuracy rate. The label and probability output module is configured to: if the prediction labels are all the same, accumulate the first prediction probability, the second prediction probability, and the third prediction probability, determine a corresponding accumulated prediction probability as the target prediction probability, and output the prediction label and the target prediction probability; or otherwise, select a maximum prediction probability from the first prediction probability, the second prediction probability, and the third prediction probability, determine the maximum prediction probability as the target prediction probability, and output the target prediction probability and a target prediction label corresponding to the maximum prediction probability.

6. An electronic device, comprising: Comprise: a memory for saving a computer program; a processor for executing the computer program to implement the steps of the data prediction method for accounting subjects according to any one of claims 1 to 4.

7. A computer readable storage medium characterized in that, a memory for storing a computer program; wherein the computer program, when executed by a processor, implements the steps of the data prediction method for accounting subjects according to any one of claims 1 to 4.

Images