Intelligent finance and tax knowledge data analysis method

Abstract

The invention belongs to the technical field of finance and taxation knowledge data analysis, and particularly relates to an intelligent finance and taxation knowledge data analysis method. A traditional method has defects in the aspects of key field extraction, anomaly detection, income prediction and the like. According to the method, finance and tax data are collected firstly, key fields are extracted by adopting improved natural language processing combined with a DistilBERT distillation technology and a dynamic rarefaction mechanism, and model lightweight and efficiency are optimized; carrying out anomaly detection by using an improved isolated forest algorithm, and accurately identifying anomalies through multi-step processing; and finally, improving a CatBoost algorithm to optimize a loss function so as to realize income prediction, and combining multiple error adaptive weight adjustment and adding a dynamic regularization term. The method effectively improves the intelligent level of finance and tax data processing, and provides powerful support for decision making.

Description

Technical Field

[0001] The present invention belongs to the technical field of financial and taxation knowledge data analysis, and specifically relates to an intelligent financial and taxation knowledge data analysis method. Background Art

[0002] In today's digital age, data in the financial and taxation field has shown explosive growth. Its data is large in scale, complex in structure and from a wide range of sources. Traditional financial and taxation data analysis methods face many challenges. On the one hand, in terms of natural language processing, the efficiency and accuracy of extracting key fields in financial and taxation texts are limited, and the computational complexity is high when processing long texts, which makes it difficult to meet the needs of large-scale data processing. For example, when processing a large amount of text data such as financial statements and tax documents, traditional methods may not be able to quickly and accurately extract key information, such as important fields such as amount, tax rate, and date. On the other hand, in the field of anomaly detection, existing algorithms may not be able to effectively adapt to the characteristics of financial and taxation data, resulting in low accuracy in abnormal transaction detection. Anomalies in financial and taxation data are often hidden and complex. For example, financial fraud, tax fraud and other behaviors may be hidden in seemingly normal transaction data, which is difficult for traditional algorithms to accurately identify. In addition, existing models may not be able to cope well with changes in income distribution and noise interference, and are prone to overfitting, resulting in low reliability of prediction results. For example, in the case of economic fluctuations and changes in market demand, traditional models are difficult to accurately predict financial and taxation revenue, thereby affecting the decision-making of enterprises and governments. Summary of the invention

[0003] In response to the technical problems existing in the above-mentioned background technology, the present invention proposes an intelligent financial and tax knowledge data analysis method that focuses on using innovative technical means to improve the intelligence level of financial and tax data processing, covering data collection, key field extraction, anomaly detection and income forecasting analysis work.

[0004] In order to achieve the above object, the technical solution adopted by the present invention comprises the following steps:

[0005] First, collect financial and tax data, and use improved natural language processing to extract key fields in the text; the improvement of the improved natural language processing is to optimize the lightweight and efficiency improvement of natural language processing by combining DistilBERT distillation technology and dynamic sparsification mechanism, and the steps are: lightweight optimization of the natural language processing model to reduce parameters and computing requirements; improve the efficiency and accuracy of key field extraction in the text; optimize the attention mechanism for processing long texts to reduce computational complexity;

[0006] Then, an improved isolation forest algorithm is used to perform anomaly detection to find abnormal transactions in the data; the improved isolation forest algorithm is implemented by improving weighted feature selection, anomaly score calculation and dynamic threshold adjustment;

[0007] Finally, we use improved CatBoost to provide support for income prediction, improve the loss function, and finally realize income prediction. The specific implementation operations are as follows:

[0008] Step 1: First, combine the mean square logarithmic error and the smoothed absolute error, and adaptively adjust the weights of the two to adapt to changes in income distribution and noise interference. The improved loss function is: Among them, α is the dynamic adjustment weight, is the Huber loss, y i is the true value, is the predicted value;

[0009] Step 2: Add a dynamic regularization term based on the above loss function to control complexity and prevent overfitting: Where λ(t) is the regularization strength that is dynamically adjusted with the number of iterations. is the L2 norm squared of the weights.

[0010] As a preferred implementation, the specific solution to optimize natural language processing by combining DistilBERT distillation technology and dynamic sparsification mechanism is as follows:

[0011] Step 1: Use knowledge distillation technology to compress the complex teacher model knowledge into a lightweight student model, using the distillation loss function L distill Let the student model imitate the output distribution of the teacher model: where p teacher,i Generated by the teacher model to represent the predicted probability of category i, p student,i Generated by the student model, it is used to represent the predicted probability of category i;

[0012] Step 2: Reduce the attention calculation complexity of processing long texts through dynamic sparsification mechanism and introduce local window attention A local , only the dependencies within the window are calculated: Where A is the standard attention matrix, Q and K represent the similarity and information weight between input sequences respectively, and d k is the dimension of the key vector, used to normalize the attention scores, Indicates limiting the interaction range of query Token i and key Token j, indicating that the distance between them must be within the window size w; add global Token attention A global , globally interact with key tokens, and adjust the two attentions through dynamic weighting, A dynamic =λA local +(1-λ)A global, where λ is the weight λ = σ(Wx + b), where σ is the sigmoid function, W and b are training parameters, which are used for dynamic sparse weight calculation, reducing the attention calculation complexity from O(n 2 ) is reduced to O(nw), improving the efficiency of long text processing;

[0013] Step 3: Enhance the adaptability of the model to the financial and taxation fields, improve the extraction accuracy of key fields such as amount, tax rate, and date, optimize the subword embedding, and adjust the pre-trained embedding E pretrained , after adjustment: E domain =E pretrained +γ·ΔE, where γ is the domain weight, ΔE is the domain vocabulary adjustment vector, and the context enhancement feature f is added to the model input contextual :f enhanced =f original +δ·f contextual , where δ is the importance of controlling context features, f original is the original feature;

[0014] Step 4: Finally, further compress the model and optimize the inference efficiency, using low-precision quantization to compress the model weights where ω is the original weight matrix, Where b is the number of quantization bits, and then mixed precision inference is used to improve hardware computing efficiency: Cmixed = C FP32 +ε·C FP16 , where C FP32 , C FP16 are the computational overhead of the high-precision and low-precision parts respectively, and ε is the weight for controlling the FP16 computation part.

[0015] Preferably, the steps of implementing the anomaly detection using the improved isolation forest algorithm to discover abnormal transactions in the data are:

[0016] Step 1: First, clean and standardize the data and extract the time series features;

[0017] Step 2: Calculate the relative importance of each feature: Where d is the number of features, T is the total number, I t (f i ) is the feature f in the tth tree i The importance of, and then according to the weight ω i Adjust feature selection probability: Features with high weights are preferentially selected for splitting;

[0018] Step 3: Train the improved isolation forest, build random trees, and calculate the average path length of data points in all trees: Where h(x) is the path length, the number of steps from the root node of the tree to the isolated leaf node, and t is the number of trees in the forest;

[0019] Step 4: Calculate the anomaly score, linearly normalize the anomaly score, adjust it to the range of [0,1], and then exponentially amplify the normalized score to amplify the difference of high anomaly points: Score amplified (x)=(Score norm (x) β , where β is the amplification factor of 2, which controls the separation degree between high abnormal points and normal points, Score norm (x) is the normalized score;

[0020] Step 5: After amplification, further normalize to the range of [0,1]:

[0021] Step 6: Then dynamically adjust the mean and standard deviation to smooth the abnormal score distribution: where μ S is the mean of the anomaly scores, σ S is the standard deviation of the anomaly score;

[0022] Step 7: Smooth the high outlier points to further improve the discrimination:

[0023] Step 8. Finally, dynamically adjust the threshold according to the distribution of abnormal scores: τ = μ S +2σ S , if Score final (x)>τ is abnormal.

[0024] Preferably, the dynamic adjustment of the weight α in the loss function optimization is achieved based on the variance and skewness of the current income distribution, and the specific formula is: where σ y is the standard deviation of the income distribution.

[0025] Preferably, in the loss function optimization, the regularization strength λ(t)=λ0·exp(-ρ·t) dynamically adjusted with the number of iterations, where λ0 is the initial regularization strength, ρ is the decay rate of the regularization strength, and controls the speed at which the regularization strength decreases.

[0026] Compared with the prior art, the advantages and positive effects of the present invention are that the efficiency and accuracy of key field extraction are greatly improved, and key financial and tax information can be obtained quickly and accurately. In terms of anomaly detection, hidden abnormal transactions are effectively identified by improving the algorithm to improve detection accuracy. When predicting income, the loss function is improved to adapt to various situations and enhance the reliability of prediction. Overall, it reduces computational complexity, reduces resource usage, effectively responds to data growth challenges, improves the level of intelligent processing of financial and tax data, provides more powerful and accurate support for corporate and government decision-making, and promotes the development of data analysis technology in the financial and tax fields. DETAILED DESCRIPTION

[0027] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described below in conjunction with embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0028] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways than those described herein. Therefore, the present invention is not limited to the specific embodiments of the following disclosure.

[0029] Embodiment: In the field of finance and taxation, with the acceleration of the digitalization process, the amount of data has exploded, the data sources have become more extensive, and the structure has become increasingly complex. Traditional financial and taxation data analysis methods expose many problems when faced with massive and complex data. For example, when processing large amounts of text data such as financial statements and tax documents, traditional natural language processing methods are difficult to efficiently and accurately extract key information such as amounts, tax rates, dates, etc. The computational complexity is high when processing long texts, and it is impossible to meet the needs of large-scale data processing. At the same time, existing anomaly detection algorithms are difficult to adapt to the characteristics of financial and taxation data, and have low accuracy in detecting abnormal transactions that are highly concealed, such as financial fraud and tax fraud. In addition, traditional models are prone to overfitting when faced with changes in income distribution and noise interference, and the reliability of prediction results is poor, making it difficult to effectively assist enterprises and governments in making decisions. Therefore, the present invention proposes an intelligent financial and taxation knowledge data analysis method.

[0030] First, in order to improve the efficiency and accuracy of extracting key fields in financial and tax texts and reduce the computational complexity of processing long texts, an improved natural language processing solution combining DistilBERT distillation technology and dynamic sparsification mechanism is adopted. First, the knowledge distillation technology is used to compress the complex teacher model knowledge into a lightweight student model, and the distillation loss function L is used. distill Let the student model imitate the output distribution of the teacher model: where p teacher,i Generated by the teacher model to represent the predicted probability of category i, p student,iGenerated by the student model, it is used to represent the predicted probability of category i; the attention calculation complexity of processing long texts is reduced through the dynamic sparsification mechanism, and the local window attention A is introduced local , only the dependencies within the window are calculated: Where A is the standard attention matrix, Q and K represent the similarity and information weight between input sequences respectively, and d k is the dimension of the key vector, used to normalize the attention scores, Indicates limiting the interaction range of query Token i and key Token j, indicating that the distance between them must be within the window size w; add global Token attention A global , globally interact with key tokens, and adjust the two attentions through dynamic weighting, A dynamic =λA local +(1-λ)A global , where λ is the weight λ = σ(Wx + b), where σ is the sigmoid function, W and b are training parameters, which are used for dynamic sparse weight calculation, reducing the attention calculation complexity from O(n 2 ) to O(nw), improving the efficiency of long text processing; enhancing the adaptability of the model to the financial and tax fields, improving the extraction accuracy of key fields such as amount, tax rate, and date, optimizing the subword embedding, and adjusting the pre-trained embedding E pretrained , after adjustment: E domain =E pretrained +γ·ΔE, where γ is the domain weight, ΔE is the domain vocabulary adjustment vector, and the context enhancement feature f is added to the model input contextual :f enhanced =f original +δ·f contextual , where δ is the importance of controlling context features, f original is the original feature; finally, the model is further compressed and the inference efficiency is optimized, and the model weights are compressed using low-precision quantization where ω is the original weight matrix, Where b is the number of quantization bits, and then mixed precision inference is used to improve hardware computing efficiency: Cmixed = C FP32 +ε·C FP16 , where C FP32 , C FP16 are the computational overhead of the high-precision and low-precision parts, respectively, and ε is the weight for controlling the FP16 computational part. In practical applications, for real-time analysis of large-scale financial and tax data, it can effectively reduce computing time and hardware resource usage.

[0031] In order to improve the accuracy of abnormal transaction detection, the improved isolation forest algorithm is used for anomaly detection. First, the data is cleaned and standardized and the time series features are extracted; the relative importance of each feature is calculated: Where d is the number of features, T is the total number, I t (f i ) is the feature f in the tth tree i The importance of, and then according to the weight ω i Adjust feature selection probability: Features with high weights are preferentially selected for splitting; the improved isolation forest is trained, random trees are built, and the average path length of data points is calculated in all trees: Where h(x) is the path length, the number of steps from the root node of the tree to the isolated leaf node, and t is the number of trees in the forest. The anomaly score is calculated, linearly normalized, adjusted to the range of [0,1], and then the normalized score is exponentially weighted to amplify the difference of high anomaly points: Score amplified (x)=(Score norm (x) β , where β is the amplification factor of 2, which controls the separation degree between high abnormal points and normal points, Score norm (x) is the normalized score; after amplification, it is further normalized to the range of [0,1]: Then the distribution of abnormal scores is smoothly adjusted based on the mean and standard deviation: where μ S is the mean of the anomaly scores, σ S is the standard deviation of the anomaly score; high anomaly points are smoothed to further improve the discrimination: Finally, according to the distribution of abnormal scores, the threshold is dynamically adjusted: τ = μ S +2σ S , if Score final (x)>τ is abnormal. Through multi-step abnormal scoring processing and dynamic threshold adjustment, it can effectively respond to the characteristics of financial and tax data, accurately detect abnormal transactions, and effectively ensure financial and tax security.

[0032] Finally, the improved CatBoost is used to support income prediction and improve the loss function. First, the mean square logarithmic error and smoothed absolute error are combined, and the weights of the two are adaptively adjusted to adapt to changes in income distribution and noise interference. The improved loss function is: Among them, α is the dynamic adjustment weight where σ y is the standard deviation of the income distribution, is the Huber loss, y i is the true value, is the predicted value; a dynamic regularization term is added on the basis of the above loss function to control complexity and prevent overfitting: Where λ(t) is the regularization strength that is dynamically adjusted with the number of iterations, λ(t) = λ0·exp(-ρ·t), where λ0 is the initial regularization strength, ρ is the decay rate of the regularization strength, and controls the speed at which the regularization strength decreases. is the L2 norm square of the weight. The advantages of the improved CatBoost algorithm are significant. Its loss function combines multiple errors and adaptively adjusts weights, which can keenly capture changes in income and resist noise interference. The dynamic regularization term strictly controls the complexity of the model to prevent overfitting, greatly improves the reliability and stability of income forecasting, and helps make scientific and accurate decisions.

[0033] The above description is only a preferred embodiment of the present invention and does not limit the present invention in other forms. Any technician familiar with the profession may use the technical content disclosed above to change or modify it into an equivalent embodiment with equivalent changes and apply it to other fields. However, any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. An intelligent financial and taxation knowledge data analysis method, characterized in that: The following steps are involved: First, collect financial and tax data, and use improved natural language processing to extract key fields in the text; the improvement of the improved natural language processing is to optimize the lightweight and efficiency improvement of natural language processing by combining DistilBERT distillation technology and dynamic sparsification mechanism, and the steps are: lightweight optimization of the natural language processing model to reduce parameters and computing requirements; improve the efficiency and accuracy of key field extraction in the text; optimize the attention mechanism for processing long texts to reduce computational complexity; Then, an improved isolation forest algorithm is used to perform anomaly detection to find abnormal transactions in the data; the improved isolation forest algorithm is implemented by improving weighted feature selection, anomaly score calculation and dynamic threshold adjustment; Finally, we use improved CatBoost to provide support for income prediction, improve the loss function, and finally realize income prediction. The specific implementation operations are as follows: Step 1: First, combine the mean square logarithmic error and the smoothed absolute error, and adaptively adjust the weights of the two to adapt to changes in income distribution and noise interference. The improved loss function is: Among them, α is the dynamic adjustment weight, is the Huber loss, y i is the true value, is the predicted value; Step 2: Add a dynamic regularization term based on the above loss function to control complexity and prevent overfitting: Where λ(t) is the regularization strength that is dynamically adjusted with the number of iterations. is the L2 norm squared of the weights.

2. According to claim 1, the intelligent financial and tax knowledge data analysis method is characterized in that: The specific implementation scheme for optimizing natural language processing by combining DistilBERT distillation technology and dynamic sparsification mechanism is as follows: Step 1: Use knowledge distillation technology to compress the complex teacher model knowledge into a lightweight student model, using the distillation loss function L distill Let the student model imitate the output distribution of the teacher model: where p teacher,i Generated by the teacher model to represent the predicted probability of category i, p student,i Generated by the student model, it is used to represent the predicted probability of category i; Step 2: Reduce the attention calculation complexity of processing long texts through dynamic sparsification mechanism and introduce local window attention A local , only the dependencies within the window are calculated: Where A is the standard attention matrix, Q and K represent the similarity and information weight between input sequences respectively, and d k is the dimension of the key vector, used to normalize the attention scores, Indicates limiting the interaction range of query Token i and key Token j, indicating that the distance between them must be within the window size w; add global Token attention A global , globally interact with key tokens, and adjust the two attentions through dynamic weighting, A dynamic =λA local +(1-λ)A global , where λ is the weight λ = σ(Wx + b), where σ is the sigmoid function, W and b are training parameters, which are used for dynamic sparse weight calculation, reducing the attention calculation complexity from O(n 2 ) is reduced to O(nw), improving the efficiency of long text processing; Step 3: Enhance the adaptability of the model to the financial and taxation fields, improve the extraction accuracy of key fields such as amount, tax rate, and date, optimize the subword embedding, and adjust the pre-trained embedding E pretrained , after adjustment: E domain =E pretrained +γ·ΔE, where γ is the domain weight, ΔE is the domain vocabulary adjustment vector, and the context enhancement feature f is added to the model input contextual :f enhanced =f original +δ·f contextual , where δ is the importance of controlling context features, f original is the original feature; Step 4: Finally, further compress the model and optimize the inference efficiency, using low-precision quantization to compress the model weights where ω is the original weight matrix, Where b is the number of quantization bits, and then mixed precision inference is used to improve hardware computing efficiency: Cmixed = C FP32 +ε·C FP16 , where C FP32 , C FP16 are the computational overhead of the high-precision and low-precision parts respectively, and ε is the weight for controlling the FP16 computation part.

3. According to claim 1, the intelligent financial and tax knowledge data analysis method is characterized in that: The implementation steps of using the improved isolation forest algorithm to perform anomaly detection to discover abnormal transactions in data are as follows: Step 1: First, clean and standardize the data and extract the time series features; Step 2: Calculate the relative importance of each feature: Where d is the number of features, T is the total number, I t (f i ) is the feature f in the tth tree i The importance of, and then according to the weight ω i Adjust feature selection probability: Features with high weights are preferentially selected for splitting; Step 3: Train the improved isolation forest, build random trees, and calculate the average path length of data points in all trees: Where h(x) is the path length, the number of steps from the root node of the tree to the isolated leaf node, and t is the number of trees in the forest; Step 4: Calculate the anomaly score, linearly normalize the anomaly score, adjust it to the range of [0,1], and then exponentially amplify the normalized score to amplify the difference of high anomaly points: Score amplified (x)=(Score norm (x)) β , where β is the amplification factor of 2, which controls the separation degree between high abnormal points and normal points, Score norm (x) is the normalized score; Step 5: After amplification, further normalize to the range of [0,1]: Step 6: Then dynamically adjust the mean and standard deviation to smooth the abnormal score distribution: where μ S is the mean of the anomaly scores, σ S is the standard deviation of the anomaly score; Step 7: Smooth the high outlier points to further improve the discrimination: Step 8. Finally, dynamically adjust the threshold according to the distribution of abnormal scores: τ = μ S +2σ S , if Score final (x)>τ is abnormal.

4. The intelligent financial and tax knowledge data analysis method according to claim 1 is characterized in that: The dynamic adjustment of the weight α in the loss function optimization is achieved based on the variance and skewness of the current income distribution. The specific formula is: where σ y is the standard deviation of the income distribution.

5. The intelligent financial and tax knowledge data analysis method according to claim 1 is characterized in that: In the loss function optimization, the regularization strength λ(t)=λ0·exp(-ρ·t) is dynamically adjusted with the number of iterations, where λ0 is the initial regularization strength and ρ is the decay rate of the regularization strength, which controls the speed at which the regularization strength decreases.

Images