Intelligent reimbursement classification system based on text classification technology and amount feature two dimensions

Abstract

The invention provides an intelligent reimbursement classification system based on a text classification technology and amount feature two dimensions, and relates to the technical field of machine learning. According to the machine learning method based on text classification, the problem of financial reimbursement automation is solved, invoice content feature information is divided according to a given expense item category through a text classifier to obtain expense item large-category data, and then invoice amount feature information is classified according to the expense item large-category data and the invoice amount feature information through an expense item classifier. The reimbursement fee item classification item of the invoice to be reimbursed is determined, so that the problems that the financial reimbursement efficiency is low, and a large amount of manpower is put into repeated labor are solved; a classification effect evaluation system is designed through a confusion matrix and evaluation indexes, and classification models used by a text classifier and a cost item classifier are obtained; in addition, algorithm defects can be determined and specially improved through experiments, so that the classification accuracy is further improved.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and in particular to an intelligent reimbursement classification system based on the dual dimensions of text classification technology and amount features. Background Art

[0002] In recent years, with the rapid growth of education funding, the financial reimbursement business of colleges and universities has also grown rapidly. Financial reimbursement is an important business in the management of higher education institutions. The existing reimbursement model is inefficient and prone to errors, which takes up a lot of time and energy of teaching and scientific research personnel, and has become a difficult problem that restricts the development of colleges and universities.

[0003] Manual reimbursement usually faces problems such as "prone to errors and time-consuming", wasting a lot of human and financial resources; in order to effectively solve the problem of financial reimbursement, many universities have vigorously developed financial informatization, using the Internet to promote appointment reimbursement, online reimbursement and other methods to try to solve the problem of financial reimbursement, but the current reimbursement model still has obvious problems and shortcomings.

[0004] On the one hand, the financial knowledge required of reimbursement personnel is too high, requiring them to accurately distinguish the financial expense items to which reimbursement business belongs. On the other hand, the work efficiency of financial staff is still low, and they still need to devote a lot of energy to repetitive work.

[0005] Therefore, the present invention provides an intelligent reimbursement classification system based on text classification technology and amount feature dual dimensions to solve the technical problems of low efficiency of existing financial reimbursement and large amount of manpower invested in repetitive work. Summary of the Invention

[0006] The purpose of the present invention is to overcome the shortcomings of the existing technology and provide an intelligent reimbursement classification system based on text classification technology and the dual dimensions of amount features. The machine learning method based on text classification is used to solve the problem of financial reimbursement automation. The text classifier is used to divide the invoice content feature information according to the given expense item category to obtain the expense item category data. The expense item classifier is then used to determine the reimbursement expense item classification items of the invoice to be reimbursed based on the expense item category data and the invoice amount feature information, thereby solving the problem of low financial reimbursement efficiency and a large amount of manpower invested in repetitive work.

[0007] To achieve the above objectives, this application proposes an intelligent reimbursement classification system based on text classification technology and amount characteristics, including: Invoice information acquisition module, used to obtain the original invoice information of the invoice to be reimbursed; An invoice information preprocessing module is used to perform data preprocessing on the original invoice information to obtain corresponding invoice characteristic information; wherein the invoice characteristic information includes invoice content characteristic information and invoice amount characteristic information; An intelligent reimbursement classifier, which is used to classify invoice feature information and obtain corresponding reimbursement expense items; it includes a text classifier and an expense item classifier; A text classifier is used to perform text classification processing on the feature information of the invoice content, divide the feature information of the invoice content into given expense item categories, and obtain the major category data of the expense items; The expense item classifier is used to perform classification processing based on the expense item category data and invoice amount feature information, determine the reimbursement expense item classification of the invoice to be reimbursed, and obtain the corresponding reimbursement expense item.

[0008] As a further solution, the invoice information acquisition module takes a photo or scans and uploads the paper invoice, directly uploads the electronic file for the electronic invoice, and obtains the original invoice information through the optical character recognition module; wherein, the original invoice information includes the bill content and unit price amount.

[0009] As a further solution, the invoice information preprocessing module performs word segmentation on the bill content, deletes non-text information and stop words that have no practical meaning, and obtains the corresponding invoice content feature information by generating word vectors; and obtains the corresponding invoice amount feature information by performing unit normalization on the unit price amount.

[0010] As a further solution, an intelligent reimbursement classifier is obtained through the following steps: Setting the machine learning algorithm used; wherein the machine learning algorithm is set by a classification algorithm; Select training data and set corresponding classification labels; Extract features from the training data to obtain corresponding features and vectorize them; Use the vectorized features and classification labels to train the machine learning algorithm to obtain the corresponding classification model; Input test data and predict the classification label through the classification model; The classification model whose classification label prediction accuracy meets the requirements is set as the corresponding intelligent reimbursement classifier.

[0011] As a further solution, the classification effect of the intelligent reimbursement classifier is evaluated through confusion matrix and evaluation indicators; wherein, the confusion matrix is ​​a two-dimensional matrix composed of test data and classification result data; the evaluation indicators include accuracy evaluation indicator, precision evaluation indicator, recall rate evaluation indicator and F1 value evaluation indicator.

[0012] As a further solution, determine the classification model used by the text classifier through the following steps: Set up the word segmentation algorithm and text segmentation library for the invoice information preprocessing module, and select a text classification experimental model; Conduct classification algorithm accuracy experiments on each text classification experimental model; Set different amounts of training data and conduct classification algorithm accuracy experiments on each text classification experimental model to obtain the impact of the amount of training data on the classification algorithm; According to the classification algorithm accuracy experiment and the impact of the amount of training data on the classification algorithm, the classification algorithm with the best experimental performance is determined as the candidate classification algorithm; Conduct actual effect tests on candidate classification algorithms and obtain classification effect evaluations of candidate classification algorithms; and select test results with unsatisfactory classification effect evaluations; A confusion matrix is ​​generated by comparing the test results with the actual results, and detailed data of all classification errors is obtained based on the confusion matrix. The candidate classification algorithm is optimized based on the detailed data to obtain the classification model used by the text classifier.

[0013] As a further solution, determine the classification model used by the expense item classifier through the following steps: Set the expense item categories and invoice amounts as features of the training data, and select the expense item classification experimental model; Conduct classification algorithm accuracy experiments on each cost item classification experimental model; Set different amounts of training data and conduct classification algorithm accuracy experiments on each cost item classification experimental model to obtain the impact of the amount of training data on the classification algorithm; According to the classification algorithm accuracy experiment and the impact of the amount of training data on the classification algorithm, the classification algorithm with the best experimental performance is determined as the candidate classification algorithm; Perform actual effect tests on the candidate classification algorithms and obtain classification effect evaluations of the candidate classification algorithms; when the classification effect evaluation is expected, set the corresponding candidate classification algorithm as the classification model used by the cost item classifier.

[0014] As a further solution, when the candidate classification algorithm is a decision tree algorithm, an experiment is conducted to investigate the effect of the maximum depth of the decision tree on the accuracy, and the optimal maximum depth is selected for setting.

[0015] As a further solution, a test data set and a training data set for each text classification experimental model or cost item classification experimental model are set through a cross-validation method.

[0016] As a further solution, the classification algorithm includes a linear support vector machine, a decision tree algorithm, a multinomial naive Bayes algorithm and a logistic regression algorithm; wherein, the classification model used by the text classifier is set to a linear support vector machine, and the classification model used by the cost item classifier is set to a decision tree algorithm.

[0017] Compared with related technologies, the intelligent reimbursement classification system based on text classification technology and amount characteristics has the following advantages: 1. This invention uses a machine learning method based on text classification to solve the problem of financial reimbursement automation. A text classifier divides the feature information of invoice content into given expense item categories to obtain expense item category data. An expense item classifier then determines the reimbursement expense item category of the invoice to be reimbursed based on the expense item category data and the feature information of the invoice amount, thereby solving the problems of low financial reimbursement efficiency and the large amount of manual labor involved in repetitive work. 2. This invention designs a classification effect evaluation system using a confusion matrix and evaluation indicators, thereby achieving quantitative evaluation of classification algorithms. Based on this, through experiments on classification algorithm accuracy and the impact of the amount of training data on classification algorithms, we further screen for more effective classification algorithms to improve classification accuracy, and obtain classification models used by text classifiers and expense item classifiers. In addition, through experiments, we can identify algorithm defects and make specific improvements to further improve classification accuracy. 3. The present invention further conducts an experiment on the influence of the maximum depth of the decision tree on the accuracy of the decision tree algorithm, and selects the optimal maximum depth for setting to optimize the effect of the decision tree algorithm; the test data set and training data set of each text classification experimental model or the cost item classification experimental model are set through the cross-validation method to eliminate the influence of the division of the test data set and the training data set on the algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 This is a schematic diagram of the structure of the intelligent reimbursement classification system provided by the present invention; Figure 2 This is a diagram of the machine learning training and prediction process; Figure 3 A schematic diagram of the K-fold cross-validation results of the text classification algorithm provided by the present invention; Figure 4 The effect of the amount of data provided by the present invention on the accuracy of the algorithm is shown in Figure 2. Figure 1 ; Figure 5A schematic diagram of the K-fold cross-validation results of the accuracy of the cost item classification algorithm provided by the present invention; Figure 6 The effect of the amount of data provided by the present invention on the accuracy of the algorithm is shown in Figure 2. Figure 2 ; Figure 7 This is a schematic diagram of the impact of maximum depth on accuracy provided by the present invention.

[0021] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0023] See also Figure 1 The present application embodiment provides an intelligent reimbursement classification system based on text classification technology and amount feature dual dimensions, including: Invoice information acquisition module, used to obtain the original invoice information of the invoice to be reimbursed; An invoice information preprocessing module is used to perform data preprocessing on the original invoice information to obtain corresponding invoice characteristic information; wherein the invoice characteristic information includes invoice content characteristic information and invoice amount characteristic information; An intelligent reimbursement classifier, which is used to classify invoice feature information and obtain corresponding reimbursement expense items; it includes a text classifier and an expense item classifier; A text classifier is used to perform text classification processing on the feature information of the invoice content, divide the feature information of the invoice content into given expense item categories, and obtain the major category data of the expense items; The expense item classifier is used to perform classification processing based on the expense item category data and invoice amount feature information, determine the reimbursement expense item classification of the invoice to be reimbursed, and obtain the corresponding reimbursement expense item.

[0024] It's important to note that financial reimbursement has always been a crucial aspect of university management. Manual reimbursement processes are often prone to errors and time-consuming, wasting significant human and financial resources. Financial reimbursement is a specific and specialized business process that can be abstracted into a multi-class problem.

[0025] This example proposes a machine learning approach based on text classification to solve the problem of automated financial reimbursement. It integrates support vector machines and decision tree algorithms to construct an intelligent reimbursement classifier. The model's effectiveness was validated using a real-world financial reimbursement dataset from a university. Experimental results show that both the accuracy and F1 score of the model exceeded 0.95 during automatic classification.

[0026] From a machine learning perspective, reimbursement can be viewed as a classification problem. Using machine learning algorithms, we can train reimbursement models using training data, enabling automatic classification of new data. In actual university reimbursement, invoice content and amount are two key pieces of information that determine the final expense item.

[0027] To better process this information, we first perform text classification on the invoice content. We then use this classification, along with the invoice amount, as features to categorize expense items, enabling intelligent reimbursement. This reimbursement model can reduce errors caused by information asymmetry, improve the efficiency of financial staff, and free teaching and research staff from the constraints of financial reimbursement, thereby contributing to the advancement of research and teaching in universities.

[0028] Paper invoices are photographed or scanned and uploaded, while electronic invoices are directly uploaded electronically and then the invoice content is captured using optical character recognition (OCR) technology. Taking a general VAT invoice as an example, the main contents of the invoice are shown in Table 1.

[0029] Table 1: Example of invoice contents According to the financial reimbursement policy, the main basis for determining which financial account an invoice should be listed in is the content of the bill and the unit price.

[0030] The preprocessing of bill content mainly includes: word segmentation, deletion of non-text information, deletion of meaningless stop words, etc.

[0031] Then, word vectors are generated, which means that the classified word information is expressed in a vectorized way through a certain method.

[0032] For example, the TF-IDF function is a commonly used word vector representation function. Its basic idea is that if a word appears frequently in a specific document but not frequently in all documents, then the word has a stronger classification and identification ability and should be given a greater weight. The weight calculation formula is: Among them, N represents the number of documents, j represents the feature word, i represents the document, Indicates the frequency of feature word j in document i, Indicates the number of documents containing feature word j, Indicates the weight of feature word j in document i.

[0033] Furthermore, the intelligent reimbursement classifier consists of a text classifier and an expense item classifier. We will train both classifiers. First, we will perform word embedding on the invoice content and train the text classifier. This classifier is used to classify the invoice content into given categories.

[0034] Then, we will use the text classification result data and invoice amount data to train a classifier for reimbursement expense items. The function of this classifier is to classify the invoice into the final reimbursement expense items based on the classification results of the invoice content and the invoice amount.

[0035] Machine learning uses pre-labeled examples as training sets to construct a supervised classifier. The trained classifier can produce a specific output (such as a label) based on a specific input (such as text), where the label is the pre-determined category to which the given text may belong.

[0036] The first step in training a machine learning text classification model is feature extraction: using a method to convert each piece of text into machine-readable information. A common method is the Bag-of-Words (BoW) model. For example, if a dictionary is defined containing the following words: {thief, yes, no, good person, bad person, policeman}, and the text "thief is a bad person" is vectorized, the vectorized representation of the text is {1, 1, 0, 0, 1, 0}. Other common methods include Term Frequency-Inverse Document Frequency (TF-IDF) and N-grams.

[0037] After feature extraction, the training data (the training set contains paired feature sets, i.e., the vector corresponding to each text and the label corresponding to the text) is input into the machine learning algorithm to generate a classification model. After training with enough samples, the classification model can make predictions based on the feature sets converted using the same feature extraction, and can predict the relevant labels when the text is input into the classification model. The training and prediction process is as follows: Figure 2 Machine learning text classification models are generally much more accurate than rule-based systems, especially in complex text classification systems. In addition, machine learning text classifiers are easier to maintain and can be used to label new data at any time to learn new classification tasks.

[0038] Specifically, the intelligent reimbursement classifier is obtained through the following steps: Setting the machine learning algorithm used; wherein the machine learning algorithm is set by a classification algorithm; Select training data and set corresponding classification labels; Extract features from the training data to obtain corresponding features and vectorize them; Use the vectorized features and classification labels to train the machine learning algorithm to obtain the corresponding classification model; Input test data and predict the classification label through the classification model; The classification model whose classification label prediction accuracy meets the requirements is set as the corresponding intelligent reimbursement classifier.

[0039] After classifier training, the performance of both classifiers was tested using test data. The test data and training data were taken from the same sample set and randomly divided into training data and test data according to the proportion. The effectiveness of the classifiers was mainly evaluated using confusion matrices and evaluation indicators.

[0040] A confusion matrix is ​​a two-dimensional matrix consisting of test data and classification results. Each column represents the classification results, and the total number in each column represents the number of cases classified into that category. Each row represents the true classification of the data, and the total number of cases in each row represents the number of test data in that category.

[0041] The confusion matrix has several important concepts: TP(1 Positive): indicates that the actual value is 0 and the predicted value is also 0 FN(0 Negative): indicates that the actual value is 0 and the predicted value is 1 FP(0 Positive): means the actual value is 1 and the predicted value is 0 TN(1 Negative): indicates that the actual value is 1 and the predicted value is also 1 Common evaluation indicators are as follows: Accuracy evaluation index (Accuracy): Indicates the ratio of correctly classified test data to the total test data. The closer the value is to 1, the better the classification effect. The calculation formula is: Precision evaluation index (Precision): It represents the ratio of the number of data correctly classified into a certain category to the total amount of data classified into that category. The closer the value is to 1, the higher the precision of the model. The calculation formula is: Recall: This is the ratio of the number of data correctly classified into a certain category to the total amount of data in that category. The closer the value is to 1, the higher the recall rate of the model. The calculation formula is: F1 value evaluation index: F1 value represents the harmonic mean of precision and recall, representing the comprehensive performance of these two indicators. The larger the F1 value, the stronger the comprehensive classification ability of the model. The calculation formula is: In a specific embodiment, the actual financial reimbursement data of a western university was used as a data set. From the reimbursement vouchers of the school from September 1, 2021 to March 31, 2022, 2,290 invoices and their actual listed expense items were randomly selected according to the proportion of reimbursement expense items. In order to ensure the accuracy of the experiment, there are more than 10 invoice data corresponding to each category of expense items. The experimental data table is shown in Table 2.

[0042] Table 2: Experimental data table The basis for determining the expense items based on invoices is mainly the invoice content, unit price, and total amount. Therefore, the following features are selected for the experimental data: Feature 1: Bill content feature. Based on the text content of the bill, the experimental data was divided into 16 categories. The classification content is shown in Table 3.

[0043] Table 3: Classification of bill content characteristics Feature 2: Unit Price. The experimental data is divided into two categories based on whether the unit price is greater than or equal to 1,000 yuan: no if the unit price is less than 1,000 yuan, and yes if the unit price is greater than or equal to 1,000 yuan. This feature further categorizes the 14th category, "Equipment or Materials," in Feature 1. If the unit price is greater than or equal to 1,000 yuan, it is classified as "Equipment Purchase." If the unit price is less than 1,000 yuan, it moves to the next sub-category.

[0044] Feature 3: Total Amount. The experimental data is divided into two categories based on whether the total amount is greater than or equal to 10,000 yuan. This feature further subdivides the "equipment or materials" category with a unit price of less than 1,000 yuan in Features 1 and 2. If the total amount is greater than or equal to 10,000 yuan, it is classified as "equipment purchase." If the total amount is less than 10,000 yuan, it is classified as "special material expenses."

[0045] The model experiment is generally divided into two parts: the first part is a text classification experiment, in which the content features of the bills in the experimental data are selected for text classification experiments. The purpose is to find a suitable algorithm for the text classifier and analyze the influencing factors of the algorithm; the second part is an expense item classification experiment, in which the result data obtained by text classification and the invoice amount data are used to classify the reimbursement expense items. The purpose is to select a suitable classification algorithm for the expense item classifier and analyze the influencing factors of the algorithm.

[0046] Specifically, the classification model used by the text classifier is determined through the following steps: Set up the word segmentation algorithm and text segmentation library for the invoice information preprocessing module, and select a text classification experimental model; Conduct classification algorithm accuracy experiments on each text classification experimental model; Set different amounts of training data and conduct classification algorithm accuracy experiments on each text classification experimental model to obtain the impact of the amount of training data on the classification algorithm; According to the classification algorithm accuracy experiment and the impact of the amount of training data on the classification algorithm, the classification algorithm with the best experimental performance is determined as the candidate classification algorithm; Conduct actual effect tests on candidate classification algorithms and obtain classification effect evaluations of candidate classification algorithms; and select test results with unsatisfactory classification effect evaluations; A confusion matrix is ​​generated by comparing the test results with the actual results, and detailed data of all classification errors is obtained based on the confusion matrix. The candidate classification algorithm is optimized based on the detailed data to obtain the classification model used by the text classifier.

[0047] In the first part of the text classification experiment, the Jieba Chinese word segmentation database was used for data preprocessing, and the TF-IDF method was used for word vector weighting. Four classification algorithms were selected: linear support vector machine, decision tree, multinomial naive Bayes, and logistic regression. The algorithms were implemented in Python.

[0048] Experiment 1: Comparison of Classification Algorithms’ Accuracy In order to eliminate the impact of the division of the test data set and the training data set on the algorithm, this paper cross-validates the classification algorithm. The method used is the K-fold cross-validation method, with the K value of 5. That is, the collected data set is divided into 5 parts of equal size, and then one of the 5 subsets is used as the test set, and the remaining 4 subsets are used as the training set. The accuracy of the classification algorithm is calculated, and the average of the 5 results is taken as the accuracy of the classification algorithm. The accuracy experimental results of the four algorithms are shown in Table 4 and Figure 3 shown.

[0049] Table 4: Accuracy of four algorithms The experimental results above show that the Linear Support Vector Machine (LinearSVC) algorithm achieved the highest accuracy of the four algorithms, followed by the Logistic Regression algorithm. Both algorithms achieved average accuracy rates exceeding 0.9, demonstrating excellent classification results. The Multinomial Naive Bayes algorithm (MultinomialNB) also achieved an average accuracy of over 0.85, demonstrating good classification results. However, the Decision Tree algorithm performed poorly, with an average accuracy of only 0.43, indicating that this algorithm is not suitable for text classification of invoice content.

[0050] Figure 3 The line graph shows that different test data sets do have a significant impact on the classification accuracy of the algorithm, but the accuracy of the linear support vector machine algorithm (LinearSVC) always remains higher than that of other algorithms.

[0051] Experiment 2: The impact of the amount of training data on the classification algorithm The training data set is randomly selected to form three data sets with the number of 1000, 1500, and 2000, and the accuracy of the classification algorithm is calculated respectively. The experimental results are as follows: Figure 4 shown.

[0052] according to Figure 4 It can be seen that within the range of experimental data, the larger the training data set, the accuracy of most algorithms is positively correlated with the size of the training data set. Only the accuracy of the decision tree algorithm does not change significantly with the size of the training data.

[0053] In addition, according to Figure 4 It can also be seen that within the range of experimental data, when the training data is the same, the accuracy of the linear support vector machine algorithm (LinearSVC) is still the highest.

[0054] Combining the results of Experiment 1 and Experiment 2, the linear support vector machine algorithm can be selected as the classification algorithm of the text classifier (classifier A) in the intelligent reimbursement model.

[0055] Experiment 3: The actual effect of LinearSVC algorithm Based on the results of Experiments 1 and 2, the linear support vector machine was selected as the text classification algorithm for bill content. In order to study the actual effect of this algorithm in classification, Experiment 3 was conducted, and detailed evaluation indicators of the algorithm for each major category of expense items were obtained, as shown in Table 5.

[0056] Table 5: Detailed evaluation metrics of the LinearSVC algorithm The data in Table 5 shows that the LinearSVC algorithm achieved an f1 value of 1 for the five expense categories: "Other postage and telecommunications fees," "Page charges, publishing fees, and other expenses," "Overtime meal expenses," "Activity expenses," and "Mobile communication and network usage fees," indicating the best overall classification results for these five categories. However, for the four categories: "Other transportation expenses (fuel)," "Other," "Equipment or materials," and "Testing and processing fees," the f1 values ​​were all below 0.9, indicating that the algorithm's classification performance in these four categories was less than ideal.

[0057] To find the reason for this unsatisfactory classification, we generated a confusion matrix between the test results and the true results, and obtained detailed data of all classification errors based on the confusion matrix, as shown in Table 6 (for ease of presentation, only part of the data was selected).

[0058] Table 6: Detailed data of classification errors (partial data) By analyzing the data in Table 6, the candidate classification algorithms are optimized to obtain the classification model used by the text classifier.

[0059] Furthermore, the classification model used by the expense item classifier is determined by the following steps: Set the expense item categories and invoice amounts as features of the training data, and select the expense item classification experimental model; Conduct classification algorithm accuracy experiments on each cost item classification experimental model; Set different amounts of training data and conduct classification algorithm accuracy experiments on each cost item classification experimental model to obtain the impact of the amount of training data on the classification algorithm; According to the classification algorithm accuracy experiment and the impact of the amount of training data on the classification algorithm, the classification algorithm with the best experimental performance is determined as the candidate classification algorithm; Conduct experiments on the impact of the maximum depth of the decision tree on accuracy and select the optimal maximum depth for setting; Perform actual effect tests on the candidate classification algorithms and obtain classification effect evaluations of the candidate classification algorithms; when the classification effect evaluation is expected, set the corresponding candidate classification algorithm as the classification model used by the cost item classifier.

[0060] In a specific embodiment, we combine the classified expense item category data with the invoice amount feature to obtain data for final confirmation of the reimbursement expense item classification.

[0061] The data for this experiment contains two features: one is the major expense item categories from the text classification experiment, and the other is the bill amount, or more precisely, the unit price. For this feature, we preprocessed it based on whether it needs to be capitalized. Unit prices less than 1,000 are marked as "no," and unit prices greater than or equal to 1,000 are marked as "yes."

[0062] The classification algorithms adopted are still the four classification algorithms we have implemented: linear support vector machine, decision tree, multinomial naive Bayes, and logistic regression.

[0063] Experiment 4: Comparison of Classification Algorithms’ Accuracy In order to eliminate the impact of the division of the test data set and the training data set on the algorithm, we cross-validated the classification algorithm in the same way as in Experiment 1. The method used was the K-fold cross-validation method, with the K value of 5. The accuracy of the classification algorithm was calculated, and the average of the 5 results was taken as the accuracy of the classification algorithm. The accuracy experimental results of the four algorithms are as follows: Figure 4 shown.

[0064] according to Figure 5 It can be concluded that among the cost item classification algorithms, the decision tree algorithm has the best classification effect, followed by logistic regression. The accuracy of these two algorithms is above 0.8, while the linear support vector machine and multinomial naive Bayes perform poorly.

[0065] Experiment 5: The impact of the amount of training data on classification accuracy Referring to Experiment 2, three training datasets with the number of 1000, 1500, and 2000 are randomly selected to calculate the accuracy of the classification algorithm. The experimental results are as follows: Figure 5 shown.

[0066] according to Figure 6 It can be concluded that within the range of experimental data, although the accuracy of the classification algorithm will fluctuate slightly with the amount of training data, the ranking of the classification effect of the classification algorithm has not changed. The best effect is still the decision tree algorithm, and the worst is still the multinomial naive Bayes algorithm.

[0067] Combining Experiments 4 and 5, it can be determined that decision tree is the best choice among the four classification algorithms in cost item classification.

[0068] Note: In Experiments 4 and 5, the maximum depth of the decision tree is 4.

[0069] Experiment 6: Experiment on the influence of the maximum depth of decision tree on accuracy Based on the algorithm characteristics of the decision tree, the maximum depth value has a great influence on the classification effect of the decision tree. In order to find the best parameter, we set the maximum depth of the decision tree to integers from 4 to 10 and perform 5-fold cross validation on the accuracy. The experimental results are as follows: Figure 6 .

[0070] according to Figure 7 It can be concluded that when the maximum depth of the decision tree algorithm is less than 8, the greater the maximum depth, the highest the accuracy of the algorithm. However, after reaching 8, the accuracy of the algorithm is fixed and will no longer change with the increase of the maximum depth. Therefore, we set the maximum depth of the decision tree algorithm to 8.

[0071] Experiment 7: The actual effect of the decision tree algorithm In order to study the actual effect of the decision tree algorithm in the classification of expense items, Experiment 7 was conducted, and the detailed evaluation indicators of the algorithm on each reimbursement expense item were obtained, as shown in Table 7.

[0072] Table 7: Detailed evaluation metrics of the decision tree algorithm According to Table 7, except for the two expense items of special materials and equipment purchase, whose evaluation indicators did not reach 1, the evaluation indicators of other expense items all reached 1. This is basically consistent with what we learned from the financial personnel, that is, except for the two expense items of material and equipment purchase, which are related to the invoice amount, the other expense items can basically be determined based on the invoice content.

[0073] The above are only some embodiments of the present application and are not intended to limit the patent scope of the present application. All equivalent structural transformations made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. An intelligent reimbursement classification system based on text classification technology and amount feature dual dimensions, characterized by: include: Invoice information acquisition module, used to obtain the original invoice information of the invoice to be reimbursed; An invoice information preprocessing module is used to perform data preprocessing on the original invoice information to obtain corresponding invoice characteristic information; wherein the invoice characteristic information includes invoice content characteristic information and invoice amount characteristic information; An intelligent reimbursement classifier, which is used to classify invoice feature information and obtain corresponding reimbursement expense items; it includes a text classifier and an expense item classifier; A text classifier is used to perform text classification processing on the feature information of the invoice content, divide the feature information of the invoice content into given expense item categories, and obtain the major category data of the expense items; The expense item classifier is used to perform classification processing based on the expense item category data and invoice amount feature information, determine the reimbursement expense item classification of the invoice to be reimbursed, and obtain the corresponding reimbursement expense item.

2. The intelligent reimbursement classification system based on text classification technology and amount feature dual dimensions according to claim 1 is characterized in that: The invoice information acquisition module takes a photo of a paper invoice or scans and uploads it, directly uploads the electronic file for an electronic invoice, and obtains the original invoice information through the optical character recognition module; wherein, the original invoice information includes the bill content and unit price amount.

3. The intelligent reimbursement classification system based on text classification technology and amount feature dual dimensions according to claim 2 is characterized in that: The invoice information preprocessing module performs word segmentation on the invoice content, deletes non-text information and meaningless stop words, and obtains corresponding invoice content feature information by generating word vectors; By normalizing the unit price amount, the corresponding invoice amount characteristic information is obtained.

4. The intelligent reimbursement classification system based on text classification technology and amount feature dual dimensions according to claim 1 is characterized in that: The intelligent reimbursement classifier is obtained through the following steps: Setting the machine learning algorithm used; wherein the machine learning algorithm is set by a classification algorithm; Select training data and set corresponding classification labels; Extract features from the training data to obtain corresponding features and vectorize them; Use the vectorized features and classification labels to train the machine learning algorithm to obtain the corresponding classification model; Input test data and predict the classification label through the classification model; The classification model whose classification label prediction accuracy meets the requirements is set as the corresponding intelligent reimbursement classifier.

5. The intelligent reimbursement classification system based on text classification technology and amount feature dual dimensions according to claim 4 is characterized in that: The classification effect of the intelligent reimbursement classifier is evaluated by using a confusion matrix and evaluation indicators; wherein the confusion matrix is ​​a two-dimensional matrix composed of test data and classification result data; the evaluation indicators include accuracy evaluation indicator, precision evaluation indicator, recall rate evaluation indicator and F1 value evaluation indicator.

6. The intelligent reimbursement classification system based on text classification technology and amount feature dual dimensions according to claim 5 is characterized in that: Determine the classification model used by the text classifier by following the steps below: Set up the word segmentation algorithm and text segmentation library for the invoice information preprocessing module, and select a text classification experimental model; Conduct classification algorithm accuracy experiments on each text classification experimental model; Set different amounts of training data and conduct classification algorithm accuracy experiments on each text classification experimental model to obtain the impact of the amount of training data on the classification algorithm; According to the classification algorithm accuracy experiment and the impact of the amount of training data on the classification algorithm, the classification algorithm with the best experimental performance is determined as the candidate classification algorithm; Conduct actual effect tests on candidate classification algorithms and obtain classification effect evaluations of candidate classification algorithms; and select test results with unsatisfactory classification effect evaluations; A confusion matrix is ​​generated by comparing the test results with the actual results, and detailed data of all classification errors is obtained based on the confusion matrix. The candidate classification algorithm is optimized based on the detailed data to obtain the classification model used by the text classifier.

7. The intelligent reimbursement classification system based on text classification technology and amount feature dual dimensions according to claim 5 is characterized in that: Determine the classification model used by the expense item classifier by following these steps: Set the expense item categories and invoice amounts as features of the training data, and select the expense item classification experimental model; Conduct classification algorithm accuracy experiments on each cost item classification experimental model; Set different amounts of training data and conduct classification algorithm accuracy experiments on each cost item classification experimental model to obtain the impact of the amount of training data on the classification algorithm; According to the classification algorithm accuracy experiment and the impact of the amount of training data on the classification algorithm, the classification algorithm with the best experimental performance is determined as the candidate classification algorithm; Perform actual effect tests on the candidate classification algorithms and obtain classification effect evaluations of the candidate classification algorithms; when the classification effect evaluation is expected, set the corresponding candidate classification algorithm as the classification model used by the cost item classifier.

8. The intelligent reimbursement classification system based on text classification technology and amount feature dual dimensions according to claim 7 is characterized in that: When the candidate classification algorithm is a decision tree algorithm, an experiment is also conducted to determine the effect of the maximum depth of the decision tree on the accuracy, and the optimal maximum depth is selected for setting.

9. An intelligent reimbursement classification system based on text classification technology and amount feature dual dimensions according to either claim 6 or claim 7, characterized in that: The test data set and training data set of each text classification experimental model or cost item classification experimental model are set through the cross-validation method.

10. An intelligent reimbursement classification system based on text classification technology and amount feature dual dimensions according to either claim 6 or claim 7, characterized in that: The classification algorithms include linear support vector machine, decision tree algorithm, multinomial naive Bayes algorithm and logistic regression algorithm; wherein, the classification model used by the text classifier is set to linear support vector machine, and the classification model used by the cost item classifier is set to decision tree algorithm.

Images