Transform-based financial enterprise customer financial early warning method and system

Abstract

The invention provides a financial enterprise customer finance early warning method and system based on a Transform, and relates to the technical field of enterprise finance analysis, and the method comprises the steps: obtaining multi-year financial data and basic attribute data of an enterprise, and carrying out the data preprocessing, so as to obtain a processed financial data set and a basic attribute data set; according to the processed financial data set and the basic attribute data set, a transpose Transform-based financial prediction model is constructed, and the transpose Transform transposes [L, D] into [D, L] by adjusting the dimension of an input matrix, so that each feature forms an independent time sequence. According to the method, the financial prediction model based on the transpose Transform is constructed, multi-year financial data and basic attribute data of an enterprise are effectively fused, accurate prediction and early warning of the financial condition of the enterprise are achieved, and the risk assessment and credit rating process of financial enterprise customers is optimized.

Description

Technical Field

[0001] The present invention relates to the technical field of enterprise financial analysis, and in particular to a method and system for early financial warning of financial enterprise customers based on a Transformer. Background Art

[0002] Existing technologies are mainly limited to modeling and analyzing single-period or a few-period financial report data, ignoring the important information of long-term financial report data in time series changes. This limitation makes it impossible for the model to fully capture the long-term trends and dynamic changes of the company's financial status, thus affecting the accuracy and reliability of financial forecasts and early warnings. When conducting corporate financial forecasts and early warnings, the basic attribute data of the corporate entity itself is ignored. However, the same financial data may have completely different impacts on companies in different industries and regions. This neglect makes the model unable to fully consider the specificity of the company's environment, thereby reducing the pertinence and effectiveness of forecasts and early warnings.

[0003] When processing time series data, the traditional Transformer model uses time steps as tokens and includes all features in a single time step. This processing method has the following defects:

[0004] Since data points at the same time step have different physical meanings, the acquisition time may not be aligned and the scales are very different, forcibly encoding them into a unified token will eliminate the correlation between multiple variables, making it impossible to learn efficient variable-based representations, and is not suitable for multivariate time series prediction tasks.

[0005] Variables have time lags, and the amount of information contained in a token at a time point is relatively limited. Starting from the token at each moment may not be conducive to modeling the global temporal correlation.

[0006] Traditional Transformers are weak in capturing basic sequence representations and characterizing multivariate correlations, which limits their processing and generalization capabilities for different time series data.

[0007] When modeling the long-term correlation of data along the time direction, as the length of the historical window continues to increase, Transformer will face the challenges of performance degradation and computational explosion, making it difficult to efficiently process long-term time series data. Summary of the invention

[0008] The technical problem to be solved by the present invention is to provide a financial enterprise customer financial early warning method and system based on Transformer, so as to provide financial enterprises with more accurate and efficient financial forecasting and early warning services.

[0009] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0010] In a first aspect, a financial enterprise customer financial early warning method based on a transposed Transformer is provided, the method comprising:

[0011] Obtain the company's multi-year financial data and basic attribute data, and perform data preprocessing to obtain processed financial data sets and basic attribute data sets;

[0012] According to the processed financial data set and basic attribute data set, a financial forecasting model based on the Transpose Transformer is constructed. The Transpose Transformer adjusts the input matrix dimension and transposes [L, D] to [D, L], so that each feature forms an independent time series.

[0013] Input the enterprise financial time series data and the processed basic attribute data set into the financial forecasting model for training to obtain a trained financial forecasting model;

[0014] Obtain the financial and attribute data of the enterprise to be predicted, and input it into the trained financial prediction model to obtain the early warning results and financial indicator prediction results of the enterprise;

[0015] Optimize the risk assessment and credit rating process for financial enterprise customers based on enterprise early warning results and financial indicator forecast results.

[0016] Furthermore, the company's multi-year financial data and basic attribute data are obtained and pre-processed to obtain processed financial data sets and basic attribute data sets, including:

[0017] Extract multi-year financial data and basic attribute data of enterprises from databases and financial statement data sources, including financial indicators, time range, frequency and basic attributes;

[0018] Identify and process missing values ​​in the data source. For financial data, use the average value to fill or delete missing values. For basic attribute data, record missing values ​​as a separate category.

[0019] Align the time of multi-year financial data. If a period of financial data is missing, use the average value of the previous and next values ​​to fill it in to obtain the processed financial data set and basic attribute data set.

[0020] Furthermore, according to the processed financial data set and basic attribute data set, a financial forecasting model based on the Transformer is constructed, wherein the Transformer adjusts the input matrix dimension and transposes [L, D] to [D, L], so that each feature forms an independent time series, including:

[0021] According to the financial forecasting goal, the categorical features in the processed financial data set and basic attribute data set are embedded and converted into vector representation; the numerical features are binned and encoded with one bit effectively to convert them into vector representation; the input data of the transposed Transformer is determined to be a three-dimensional matrix [B, L, D], where the first dimension B represents the batch size of a training, the second dimension L represents the sequence dimension, and the third dimension D represents the feature dimension;

[0022] Adjust the dimension of the input matrix, transpose the position of [L, D], exchange the position of the sequence dimension and the feature dimension, so that the sequence of each single feature at each time point is spliced ​​and merged to form a new input matrix [B, D, L];

[0023] The category feature vector and the numerical feature vector processed by the embedding layer are concatenated according to the transposed dimensions [B, D, L] to obtain the input vector of the multilayer perceptron;

[0024] Use a single-layer fully connected neural network to perform linear transformation on the concatenated input vector to obtain an intermediate result, and use an activation function to perform nonlinear activation on the intermediate result to obtain the output result of the feedforward network layer;

[0025] According to the output results of the feedforward network layer, the linear fully connected neural network is used again to independently process the time series of each feature to extract a high-dimensional feature vector, and the high-dimensional feature vector is linearly mapped to obtain the query vector, key vector and value vector;

[0026] Calculate the dot product of the query vector and the key vector and scale them to get the scaled result;

[0027] Use the flexible maximum function to convert the scaled result into an attention score to obtain a vector represented by multi-head self-attention.

[0028] Perform layer normalization on the vectors represented by the multi-head self-attention to obtain a normalized vector;

[0029] Use a feedforward network layer containing two layers of linear fully connected neural networks to perform nonlinear transformation and combination on the normalized vectors, and concatenate the output results of the feedforward network layer and the time series information to obtain a concatenated feature vector;

[0030] The concatenated feature vector is used as the final feature representation of the financial forecasting model and input into the fully connected layer to build a financial forecasting model based on the transposed Transformer.

[0031] Furthermore, the dot product calculation formula of the query vector and the key vector is:

[0032]

[0033] Where P(Q,K,W) represents the dot product of the query vector Q and the key vector K after linear transformation by the weight matrix W; Q represents the query vector; K represents the key vector; W represents the weight matrix; d represents the dimensions of the query vector, key vector and weight matrix; W ij represents the element in the i-th row and j-th column of the weight matrix; Q j represents the jth component of the query vector; W ik represents the element in the i-th row and k-th column of the weight matrix; K k represents the kth component of the key vector; k represents the index of traversing the key vector; j represents the index of traversing the query vector; i represents the outer sum index.

[0034] Furthermore, the enterprise financial time series data and the processed basic attribute data set are input into the financial forecasting model for training to obtain a trained financial forecasting model, including:

[0035] According to the preset ratio, the enterprise financial time series data and the processed basic attribute data set are divided into a training set and a validation set;

[0036] Initialize the financial forecasting model, including configuring the embedding layer, multivariate attention layer, layer normalization, and feedforward network layer components of the financial forecasting model, and determine the learning rate, batch size, and number of training rounds of the financial forecasting model;

[0037] Input each sample in the training set into the financial forecasting model, perform forward propagation calculations, and output the corresponding forecast results based on the characteristics of each input sample;

[0038] Calculate the loss function value based on the difference between the corresponding prediction result and the true label;

[0039] According to the loss function value, back propagation is performed to calculate the gradient and update the financial forecasting model parameters;

[0040] Repeat the forward propagation, loss calculation, back propagation and parameter update operations until the preset number of training rounds is reached to obtain the trained financial prediction model.

[0041] Furthermore, the calculation formula of the loss function value is:

[0042]

[0043] Where L represents the overall loss function value; n represents the total number of training samples; w h represents the weight of the hth sample; f represents the total number of categories; m represents the index of the category; y hm represents a binary indicator variable; represents the probability that the hth sample predicted by the model belongs to the mth class; γ represents the time series penalty coefficient; T represents Timing penalty term; represents the prediction result of the financial prediction model; y represents the true label; t represents the time information; h represents the index of the sample.

[0044] Furthermore, the financial and attribute data of the enterprise to be predicted are obtained and input into the trained financial prediction model to obtain the early warning results and financial indicator prediction results of the enterprise, including:

[0045] Extract new financial data and basic attribute data from the internal database and financial statement data source of the enterprise to be forecasted, including balance sheet, income statement, and cash flow statement;

[0046] Integrate new financial data and basic attribute data to obtain integrated data;

[0047] Calculate financial ratios, including current ratios, based on consolidated data;

[0048] The financial, attribute data and financial ratios of the enterprise to be predicted are input into the trained model financial prediction model, and the forward propagation calculation is performed to obtain the model output warning results and financial indicator prediction results, including the enterprise's warning level, risk signal and predicted value of financial indicators.

[0049] The second aspect is a financial early warning system for financial enterprise customers based on Transformer, including:

[0050] The acquisition module is used to obtain the financial data and basic attribute data of the enterprise for multiple years, and perform data preprocessing to obtain the processed financial data set and basic attribute data set; based on the processed financial data set and basic attribute data set, a financial forecasting model based on the transposition transformer is constructed, wherein the transposition transformer adjusts the input matrix dimension to transpose [L, D] to [D, L] so that each feature forms an independent time series; the enterprise financial time series data and the processed basic attribute data set are input into the financial forecasting model for training to obtain a trained financial forecasting model;

[0051] The processing module is used to obtain the financial and attribute data of the enterprise to be predicted, and input it into the trained financial prediction model to obtain the enterprise's early warning results and financial indicator prediction results; based on the enterprise's early warning results and financial indicator prediction results, the risk assessment and credit rating process of financial enterprise customers is optimized.

[0052] According to a third aspect, a computing device includes:

[0053] one or more processors;

[0054] The storage device is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method described.

[0055] In a fourth aspect, a computer-readable storage medium stores a program, and when the program is executed by a processor, the method described is implemented.

[0056] The above solution of the present invention includes at least the following beneficial effects:

[0057] The Transposed Transformer algorithm adjusts the input matrix dimension from [L, D] to [D, L], allowing the model to focus on the temporal information of each feature at multiple time steps rather than all features at a single time step. This processing method not only retains more information about the data in terms of time series, but also effectively reduces the differences in indicators caused by inconsistent processing, helps the model process non-stationary sequences, and significantly reduces the noise in the sequence. At the same time, the subsequent multi-head self-attention mechanism can further learn the multivariate correlations between sequence features, and has better temporal representation and multivariate correlation representation capabilities than the traditional Transformer.

[0058] It integrates time-series financial data and non-time-series basic attribute data, enriches the data source, and improves the comprehensiveness and accuracy of the model. At the same time, the Wide&Deep architecture is introduced. The shallow Wide part can directly learn and use the "co-occurrence frequency" representation of basic attribute features, enhancing the financial forecasting model's ability to quickly process and memorize basic data; and the Deep part, which is composed of the deep neural network transposed Transformer, can better represent the time series and feature correlation of financial data, giving the model deep learning powerful generalization and parsing capabilities. This combined architecture enables the model to have the advantages of both general perceptrons and deep neural networks, and achieves accurate parsing of complex financial data.

[0059] Through the trained financial forecasting model, the financial and attribute data of the enterprise to be forecasted can be accurately obtained, and then the early warning results and financial indicator forecast results of the enterprise can be obtained. These results can provide strong decision-making support for financial enterprises and help them optimize the risk assessment and credit rating process of customers. Specifically, the financial forecasting model can identify potential risk customers, issue early warning signals in time, and reduce credit risks; at the same time, for customers with good credit, it can also give accurate credit ratings, providing financial enterprises with more refined customer management strategies. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1It is a flowchart of a financial enterprise customer financial early warning method based on Transformer provided by an embodiment of the present invention.

[0061] Figure 2 It is a schematic diagram of a financial enterprise customer financial early warning system based on a transposition Transformer provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0062] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0063] like Figure 1 As shown, an embodiment of the present invention proposes a financial enterprise customer financial early warning method based on a transposition transformer, and the method comprises the following steps:

[0064] Step 11, obtaining the enterprise's multi-year financial data and basic attribute data, and performing data preprocessing to obtain processed financial data sets and basic attribute data sets;

[0065] Step 12: Based on the processed financial data set and basic attribute data set, a financial forecasting model based on the Transpose Transformer is constructed, wherein the Transpose Transformer adjusts the input matrix dimension and transposes [L, D] to [D, L] so that each feature forms an independent time series.

[0066] Step 13, inputting the enterprise financial time series data and the processed basic attribute data set into the financial forecasting model for training to obtain a trained financial forecasting model;

[0067] Step 14, obtaining the financial and attribute data of the enterprise to be predicted, and inputting the trained financial prediction model to obtain the early warning results and financial indicator prediction results of the enterprise;

[0068] Step 15: Optimize the risk assessment and credit rating process for financial enterprise customers based on the enterprise early warning results and financial indicator forecast results.

[0069] In an embodiment of the present invention, by obtaining the financial data and basic attribute data of the enterprise for many years and performing data preprocessing, such as cleaning, normalization, missing value processing, etc., the integrity and consistency of the data can be ensured, and the effect and accuracy of model training can be improved. At the same time, years of data accumulation help capture the long-term trends and cyclical changes in the financial status of the enterprise, providing a basis for more accurate predictions. As a machine learning algorithm, the Transformer has powerful data processing and feature extraction capabilities. By constructing a financial forecasting model based on the Transformer, it is possible to more deeply explore the potential laws and associations in the financial data of the enterprise, and improve the accuracy and generalization of the prediction.

[0070] Inputting the processed enterprise financial time series data and basic attribute data sample set into the financial forecasting model for training can enable the model to fully learn the changing rules and characteristics of the enterprise's financial status, thereby improving the model's prediction accuracy and stability. The trained model can more accurately predict the future financial status and risk situation of the enterprise. By inputting the financial and attribute data of the enterprise to be predicted into the trained financial forecasting model, the enterprise's early warning results and financial indicator prediction results can be obtained. This helps financial institutions to detect potential risks in a timely manner, take countermeasures in advance, and reduce risk losses. At the same time, the financial indicator prediction results can provide important references for financial institutions' loan decisions, credit ratings, etc.

[0071] Based on the enterprise early warning results and financial indicator forecast results, financial institutions can more accurately assess the credit status and risk level of enterprises, thereby optimizing the risk assessment and credit rating process. This helps financial institutions to formulate loan policies, risk control strategies, etc. more scientifically, and improve business efficiency and profitability. At the same time, the optimized process can reduce human intervention and subjective judgment, and improve the objectivity and fairness of the assessment.

[0072] In a preferred embodiment of the present invention, the above step 11, obtaining the financial data and basic attribute data of the enterprise for multiple years, and performing data preprocessing to obtain the processed financial data set and basic attribute data set, may include:

[0073] Step 112, extracting the enterprise's multi-year financial data and basic attribute data from the database and financial statement data source, including financial indicators, time range, frequency and basic attributes;

[0074] Step 133, identifying and processing missing values ​​of data in the data source, for financial data, using the average value to fill or delete missing values, and for basic attribute data, recording missing values ​​as a separate category;

[0075] Step 114, align the time of multi-year financial data. If a period of financial data is missing, use the average value of the previous and next values ​​to fill in the missing data, so as to obtain the processed financial data set and basic attribute data set.

[0076] In an embodiment of the present invention, the financial data and basic attribute data of an enterprise for multiple years are extracted from a database (such as an SQL database, a NoSQL database, etc.) and a financial statement data source (such as an Excel file, a CSV file, a PDF report, etc.). Financial data includes, but is not limited to, financial indicators such as income, cost, profit, assets, liabilities, cash flow, and the corresponding time range (such as annual, quarterly) and frequency (such as annual, quarterly) of these data. Basic attribute data include the name, industry, region, registered capital, establishment time, legal representative, etc. of the enterprise. For financial data, missing values ​​in the data source are identified and processed. If there are few missing values, the average value of the indicator in other years can be used to fill in; if there are many missing values ​​or it cannot be accurately estimated, consider deleting the indicator or the data of that year. For basic attribute data, such as industry classification, region, etc., if missing, the missing value can be recorded as a separate category (such as "unknown industry", "unspecified region"), or binned according to other relevant information (such as dividing the registered capital into several intervals according to size). Align the time of multi-year financial data to ensure that the data of each year or quarter has a corresponding time point. If a certain period of financial data is missing, the average value of the previous and next values ​​can be used for filling. For example, if data for 2020 are missing, the 2020 data can be estimated using the average of the 2019 and 2021 data.

[0077] Suppose you want to process the financial data and basic attribute data of a retail enterprise from 2018 to 2022. Extract the financial data from 2018 to 2022 from the enterprise's SQL database, including annual revenue, cost, profit, etc. Extract the basic attribute data of the enterprise from the Excel file, such as enterprise name, industry classification (retail industry), region, registered capital (5 million yuan), establishment time (2010), legal representative, etc. In the financial data, it was found that the cost data for 2020 was missing. Since there were few missing values, we decided to use the average of the cost data for 2019 and 2021 to fill it. In the basic attribute data, it was found that the industry classification of some enterprises was missing. The industry classification of these enterprises was recorded as "unknown industry". Make sure that the financial data is aligned by year, that is, each year has corresponding revenue, cost, profit and other data. It was found that the cash flow data for 2019 was missing. We use the average of the cash flow data for 2018 and 2020 for filling.

[0078] By extracting multi-year financial data and basic attribute data from databases and financial statement data sources, the comprehensiveness and continuity of the data are ensured. Missing values ​​in the data source are identified and processed, and reasonable methods (such as filling with average values, deleting missing values, recording as a single category or binning) are used to process them, which improves the completeness and accuracy of the data. The time of multi-year financial data is aligned to ensure that the data of each year or quarter has a corresponding time point, which enhances the consistency and comparability of the data. The average value of the previous and next values ​​is used for filling processing to reduce the analysis bias caused by missing data, making the data of different years or quarters more comparable. The processed financial data set and basic attribute data set provide a reliable data foundation for financial analysis, forecasting, risk assessment, etc. The improvement of data completeness and accuracy helps to reduce errors and deviations in analysis and improve the efficiency and accuracy of analysis. Accurate and comprehensive financial data helps enterprises better understand their own financial status and operating results, and provides strong support for enterprise decision-making and strategic planning. For financial institutions, the processed corporate financial data and basic attribute data provide more accurate and comprehensive information for their risk assessment and credit rating. It helps financial institutions to more accurately assess the credit status and risk level of enterprises, thereby formulating more scientific and reasonable loan policies and risk control strategies.

[0079] In a preferred embodiment of the present invention, in the above step 12, a financial forecasting model based on a transposed Transformer is constructed according to the processed financial data set and the basic attribute data set, wherein the transposed Transformer adjusts the input matrix dimension to transpose [L, D] to [D, L] so that each feature forms an independent time series, including:

[0080] Step 122, according to the financial forecasting goal, embed the categorical features in the processed financial data set and the basic attribute data set and convert them into vector representation; bin the numerical features and perform one-bit effective encoding to convert them into vector representation; determine that the input data of the transposed transformer is a three-dimensional matrix [B, L, D], where the first dimension B represents the batch size of one training, the second dimension L represents the sequence dimension, and the third dimension D represents the feature dimension;

[0081] Step 123, adjust the dimension of the input matrix, transpose the position of [L, D], exchange the position of the sequence dimension and the feature dimension, so that the sequence of each single feature at each time point is spliced ​​and merged to form a new input matrix [B, D, L];

[0082] Step 124, concatenating the category feature vector and the numerical feature vector processed by the embedding layer according to the transposed dimensions [B, D, L] to obtain an input vector of the multilayer perceptron;

[0083] Step 125, using a single-layer fully connected neural network to perform a linear transformation on the concatenated input vector to obtain an intermediate result, and using an activation function to perform a nonlinear activation on the intermediate result to obtain an output result of the feedforward network layer;

[0084] Step 126, based on the output result of the feedforward network layer, the linear fully connected neural network is used again to independently process the time series of each feature to extract a high-dimensional feature vector, and the high-dimensional feature vector is linearly mapped to obtain a query vector, a key vector, and a value vector;

[0085] Step 127, calculating the dot product of the query vector and the key vector, and scaling them to obtain a scaled result;

[0086] Step 128, using a flexible maximum function to convert the scaled result into an attention score to obtain a vector represented by multi-head self-attention;

[0087] Step 129, performing layer normalization processing on the vector represented by the multi-head self-attention to obtain a normalized vector;

[0088] Step 130, using a feedforward network layer including two layers of linear fully connected neural networks, performing nonlinear transformation and combination on the normalized vectors, and concatenating the output result of the feedforward network layer and the time series information to obtain a concatenated feature vector;

[0089] Step 131, the concatenated feature vector is used as the final feature representation of the financial forecasting model and input into the fully connected layer to construct a financial forecasting model based on the transposed Transformer.

[0090] In an embodiment of the present invention, categorical features are embedded and converted to vector representation. Numerical features are binned and encoded with one bit of validity, and also converted to vector representation. The input data of the transposed Transformer is determined to be a three-dimensional matrix [B, L, D], where B represents the batch size, L represents the sequence dimension (number of time periods), and D represents the feature dimension. The dimension of the input matrix is ​​adjusted, and [L, D] is transposed to [D, L], that is, the positions of the sequence dimension and the feature dimension are exchanged. In this way, the sequences of each single feature at each time point are spliced ​​and merged to form a new input matrix [B, D, L].

[0091] Wide part: includes Embedding layer and Multilayer Perceptron layer (MLP). The Embedding layer vectorizes the categorical enterprise attribute features, bins and one-hot processes the numerical enterprise attribute features, and then concatenates them to obtain the input vector of the MLP. MLP is a single-layer fully connected neural network that uses the ReLU activation function to capture complex nonlinear relationships. The calculation formula of the activation function (ReLU) is: ReLU(x) = max(0,x); where ReLU represents a function that performs a nonlinear transformation on the input parameters; x is the input parameter of the ReLU function, representing a real number or a real number vector (in the case of multiple dimensions); max(0,x) is the core part of the ReLU function, which means taking the maximum value between 0 and x; when x is greater than 0, the output of the ReLU function is x itself; when x is less than or equal to 0, the output of the ReLU function is 0. If x is a real number, then max(0,x) is a real number, representing the larger value between x and 0. If x is a vector, then max(0,x) is a vector of the same dimension as x, where each element is the larger of the element of x or 0 at the corresponding position.

[0092] Deep part: It is a transposed Transformer model, whose architecture is similar to the Transformer encoder, including embedding layer, multivariate attention layer, layer normalization (layerNorm) and feed-forward network layer (FFN).

[0093] Embedding layer: Linearly map the time series vector of each transposed variable to obtain a high-dimensional vector.

[0094] Multivariate attention layer: Use a linear fully connected neural network to map the time series vector of each variable into a high-dimensional vector representation. Perform linear mapping on the high-dimensional vector to obtain the query vector (Query), key vector (Key), and value vector (Value) respectively. Calculate the dot product of the query vector and the key vector, and scale it (by dividing by the square root of the dimension of the key vector) to obtain the scaled result. Use the softmax function to convert the scaled result into an attention score, which represents the importance of features between different time steps. The softmax function is: Among them, e b is the bth element in the scaled result vector, and n is the dimension of the vector; the softmax function converts the value of each element into a probability value between 0 and 1, and the sum of the probability values ​​of all elements is 1. The value vector is weighted and summed through the attention score to obtain a vector represented by multi-head self-attention.

[0095] Layer normalization: Standardize the representation sequence of individual variables to reduce differences and noise.

[0096] Feedforward network layer: It is a two-layer linear fully connected neural network with ReLU activation function in the middle to learn different time attributes. The representation vectors of Wide and Deep modules are concatenated. The final prediction result is output through linear projection.

[0097] Suppose there are three consecutive financial report data, including three features: debt-to-asset ratio (a), current ratio (b) and operating income (c). The traditional Transformer model will treat all the features [a1, b1, c1] at each time point as a whole token. In the transposed Transformer, the data is transposed to [[a1, a2, a3], [b1, b2, b3], [c1, c2, c3]], that is, the time series of each individual feature is used as the whole token.

[0098] For the Wide part, categorical features (such as company type) are embedded, numerical features (such as debt-to-asset ratio, current ratio, etc.) are binned and one-hot processed, and then these features are concatenated and input into the MLP.

[0099] For the Deep part, the transposed data is input into the embedding layer to obtain a high-dimensional vector representation of each feature. These vectors are then input into the multivariate attention layer to learn the dependencies between different features. Next, layer normalization will standardize these vectors. Finally, the feedforward network layer will learn the temporal correlation of these vectors and output the final feature representation.

[0100] By transposing the Transformer architecture, the dimension of the input matrix is ​​transposed from [L, D] to [D, L], so that the financial forecasting model can focus on learning the dependencies between different features. This transposition method allows each feature to form an independent time series, which helps the model to more accurately capture and understand the interactions between features, thereby improving the accuracy of financial forecasts. The feedforward network layer (FFN) in the transposed Transformer model is responsible for linearly inferring the future expectations of a single feature. Through the transposition process, the time series data of each feature is processed independently, allowing the FFN to better model the time series correlation of the feature. This helps the model to more accurately grasp the dynamic changes of time series data in financial forecasting. Combining the memory ability of the Wide part and the generalization ability of the Deep part, the model can simultaneously capture the information of low-order feature combination interactions and high-order feature intersections. The Wide part processes low-order feature combinations through linear models to provide stable memory capabilities; while the Deep part processes high-order feature intersections through the transposed Transformer, providing strong generalization capabilities. This fusion makes the model more robust and accurate in financial forecasting.

[0101] The attention mechanism in the Transposed Transformer model can calculate the attention scores between different features, which helps us understand how the model weighs different features when making predictions. By visualizing the attention scores, we can more intuitively understand how the model works, thereby improving the interpretability of the model. Traditional Transformer models tend to process all features at each time point as a whole when processing time series data. However, in the financial series prediction scenario, different features may have different timing patterns and dependencies. The Transposed Transformer enables the model to better adapt to this scenario by transposing the dimensions of the input matrix, thereby improving the accuracy and stability of financial forecasts.

[0102] In a preferred embodiment of the present invention, the dot product calculation formula of the query vector and the key vector is:

[0103]

[0104] Where P(Q,K,W) represents the dot product of the query vector Q and the key vector K after linear transformation by the weight matrix W; Q represents the query vector; K represents the key vector; W represents the weight matrix; d represents the dimensions of the query vector, key vector and weight matrix; W ij represents the element in the i-th row and j-th column of the weight matrix; Q j represents the jth component of the query vector; W ik represents the element in the i-th row and k-th column of the weight matrix; K krepresents the kth component of the key vector; k represents the index of traversing the key vector; j represents the index of traversing the query vector; i represents the outer sum index.

[0105] In an embodiment of the present invention, given a query vector Q, a key vector K, and a weight matrix W, determine the dimension d of the vector and matrix. For each i (from 1 to d), calculate This will give the i-th component of the linearly transformed query vector. Similarly, for each i, calculate This will give the i-th component of the linearly transformed key vector. Multiply the corresponding components of the linearly transformed query vector and key vector and sum them for all i to get the unscaled dot product result. Divide the unscaled dot product result by Get the final dot product P(Q,K,W).

[0106] Assume the following query vector, key vector, and weight matrix:

[0107] Query vector Q = [1, 2, 3]; key vector K = [4, 5, 6]; weight matrix Initialize dimension d=3 (because query vector, key vector and weight matrix are all 3-dimensional).

[0108] For i=1: For i=2 and i=3, perform similar calculations to obtain all components of the linearly transformed query vector and key vector. Multiply the corresponding components of the linearly transformed query vector and key vector and sum them for all i to obtain the unscaled dot product result. Divide the unscaled dot product result by (because d=3), we get the final dot product.

[0109] The introduction of the weight matrix W enables the model to learn and capture the complex relationship between the query vector and the key vector. Through linear transformation, the components of the original vector are recombined and weighted, so that richer and more diverse features can be represented. The query vector and the key vector may come from different feature spaces or have different distributions. Through the linear transformation of the weight matrix, the two vectors can be mapped to a common feature space, making the comparison and matching between them easier and more accurate. The dot product of the original query vector and the key vector may be affected by the length and direction of the vector, resulting in unstable calculation results. Through the linear transformation of the weight matrix, the vector can be normalized and scaled, making the dot product calculation more robust to changes in the length and direction of the vector.

[0110] The attention mechanism is widely used. This formula can be seen as an implementation of the attention mechanism, where the weight matrix W dynamically adjusts the attention to the query vector and the key vector according to the context information, thereby achieving more accurate information extraction and fusion. The weight matrix W is a learnable parameter that can be updated and adjusted through optimization methods such as backpropagation and gradient descent. This enables the model to automatically learn the optimal weight matrix based on the training data, thereby improving the performance and generalization ability of the model.

[0111] In a preferred embodiment of the present invention, the above step 13, inputting the enterprise financial time series data and the processed basic attribute data set into the financial forecasting model for training to obtain the trained financial forecasting model, may include:

[0112] Step 133, dividing the enterprise financial time series data and the processed basic attribute data set into a training set and a validation set according to a preset ratio;

[0113] Step 134, initializing the financial forecasting model, including configuring the embedding layer, multivariate attention layer, layer normalization, and feedforward network layer components of the financial forecasting model, and determining the learning rate, batch size, and number of training rounds of the financial forecasting model;

[0114] Step 135, input each sample in the training set into the financial forecasting model, perform forward propagation calculation, and output the corresponding forecast result according to the input features of each sample;

[0115] Step 136, calculating the loss function value according to the difference between the corresponding prediction result and the true label;

[0116] Step 137, performing back propagation to calculate the gradient according to the loss function value, and updating the financial forecasting model parameters;

[0117] Step 138, repeat the forward propagation, loss calculation, back propagation and parameter update operations until a preset number of training rounds is reached to obtain a trained financial forecasting model.

[0118] In an embodiment of the present invention, the processed enterprise financial time series data and basic attribute data sample set are randomly divided into a training set and a validation set according to a preset ratio (such as 70% training set and 30% validation set). Ensure that the continuity of the time series is maintained when the data is divided to avoid data leakage problems. Initialize the financial forecasting model, including configuring the various components of the model, such as the embedding layer (for converting the classification features into vector representations), the multivariate attention layer (for capturing dynamic dependencies in the time series data), layer normalization (for stabilizing model training) and the feedforward network layer (for generating the final prediction results). Determine the model's learning rate (such as 0.001), batch size (such as 32), number of training rounds (such as 100 rounds) and other hyperparameters. Input each sample in the training set (including time series data and basic attribute data) into the financial forecasting model. The financial forecasting model first converts the classification features into vectors through the embedding layer, then captures the key information in the time series data through the multivariate attention layer, then performs layer normalization, and finally generates the prediction results through the feedforward network layer. The loss function value is calculated based on the difference between the prediction results output by the financial forecasting model and the true label of the sample (such as the company's future financial indicators). Based on the loss function value, back propagation is performed to calculate the gradient, that is, the derivative of the loss function with respect to the model parameters is calculated. The model parameters are updated according to the gradient to minimize the loss function value. The forward propagation, loss calculation, back propagation and parameter update operations are repeated until the preset number of training rounds is reached.

[0119] During the training process, performance indicators (such as accuracy, loss, etc.) on the validation set can be monitored to prevent the model from overfitting. After the training is completed, the performance of the model is evaluated using the validation set to ensure that the model has good generalization ability. The trained financial forecasting model is deployed in practical applications to predict the company's future financial indicators.

[0120] Assume that there is a sample set containing corporate financial time series data and basic attribute data, where the time series data includes the monthly income, cost, profit, etc. of the enterprise, and the basic attribute data includes the industry, scale, region, etc. of the enterprise. The sample set is divided into a training set and a validation set at a ratio of 70% and 30%. Initialize a financial forecasting model containing an embedding layer, a multivariate attention layer, a layer normalization, and a feedforward network layer. Set the learning rate to 0.001, the batch size to 32, and the number of training rounds to 100. For each sample in the training set, its time series data and basic attribute data are input into the financial forecasting model. The financial forecasting model converts the basic attribute data into a vector representation through the embedding layer, then captures the key information in the time series data (such as the growth trend of income, the fluctuation of cost, etc.) through the multivariate attention layer, then performs layer normalization to stabilize the training process, and finally generates the forecast results (such as the revenue forecast for the next year) through the feedforward network layer. Use the mean square error (MSE) as the loss function to calculate the difference between the forecast results output by the financial forecasting model and the true label of the sample. Calculate the gradient according to the loss function value and update the model parameters. Repeat the forward propagation, loss calculation, back propagation and parameter update operations for 100 rounds. Monitor the MSE indicator on the validation set during training to ensure that the model is not overfitting.

[0121] After training is completed, the validation set is used to evaluate the MSE indicator of the financial forecasting model. If the MSE is low and stable, it means that the financial forecasting model has good forecasting performance. The trained financial forecasting model is deployed to the company's financial forecasting system to predict the company's future financial indicators in real time. By inputting the processed corporate financial time series data and basic attribute data into the financial forecasting model, the model can learn the complex patterns and potential relationships in the data, thereby improving the accuracy of forecasting future financial indicators.

[0122] Dividing the data set into a training set and a validation set helps monitor the performance of the financial forecasting model during the training process, and adjusts the parameters of the financial forecasting model through the validation set to ensure that the model not only performs well on the training data, but also maintains stable prediction capabilities on data that has never been seen. The configuration of the financial forecasting model with an embedding layer, a multivariate attention layer, a layer normalization, and a feedforward network layer can efficiently process time series data and basic attribute data, capture key information in the data, and stabilize the training process through layer normalization to improve the convergence speed and stability of the model. By calculating the loss function value, performing backpropagation to calculate the gradient, and updating the parameters of the financial forecasting model, this iterative process can ensure that the model parameters gradually approach the final solution and improve the prediction performance of the model. The automated training process reduces the need for manual intervention and makes model training more efficient. At the same time, accurate financial forecasts help companies better plan cash flow and control costs, thereby saving time and operating costs. The trained financial forecasting model can provide companies with reliable forecasts of future financial indicators. Accurate financial forecasts help companies seize market opportunities, optimize resource allocation, and promote business growth. For example, by predicting future market demand, companies can formulate reasonable production plans and sales strategies to meet market demand and increase market share.

[0123] In a preferred embodiment of the present invention, the calculation formula of the loss function value is:

[0124]

[0125] Where L represents the overall loss function value; n represents the total number of training samples; w h represents the weight of the hth sample; f represents the total number of categories; m represents the index of the category; y hm represents a binary indicator variable; represents the probability that the hth sample predicted by the financial forecasting model belongs to the mth category; γ represents the time series penalty coefficient; T represents Timing penalty term; represents the prediction result of the financial prediction model; y represents the true label; t represents the time information; h represents the index of the sample.

[0126] In the embodiment of the present invention, the total number of training samples n and the weight w of each sample are determined. h , the total number of categories f, and the time series penalty coefficient γ. Initialize the financial forecasting model parameters so that they can be updated during the training process. The probability that the hth sample predicted by the financial forecasting model belongs to the mth category and a binary indicator variable y hm , calculate the loss of sample h on category m. If sample h belongs to category m, then y hm =1, otherwise y hm= 0. Sum the losses of all categories m to get the total loss of sample h. Multiply by the weight w of sample h h , to consider the contribution of different samples to the overall loss. Sum the weighted losses of all training samples h to get the overall loss function value L (without considering the time series penalty term). Calculate the time series penalty term This item is based on the prediction results of the model The real label y and time information t are used to evaluate the accuracy of the prediction results in the time series. The time series penalty term is multiplied by the coefficient γ and added to the overall loss function value L to obtain the final loss function value.

[0127] In a preferred embodiment of the present invention, the above step 14, obtaining the financial and attribute data of the enterprise to be predicted and inputting the trained financial prediction model to obtain the early warning results and financial indicator prediction results of the enterprise, may include:

[0128] Step 144, extracting new financial data and basic attribute data from the internal database and financial statement data source of the enterprise to be predicted, including balance sheet, income statement, and cash flow statement;

[0129] Step 145, integrating the new financial data and the basic attribute data to obtain integrated data;

[0130] Step 146, calculating financial ratios, including current ratios, based on the integrated data;

[0131] Step 147, input the financial, attribute data and financial ratios of the enterprise to be predicted into the trained model financial prediction model, and perform forward propagation calculation to obtain the model output warning results and financial indicator prediction results, including the enterprise's warning level, risk signal and predicted value of financial indicators.

[0132] In an embodiment of the present invention, the latest financial data and basic attribute data are extracted from the internal database and financial statement data source (such as ERP system, financial software, etc.) of the enterprise to be predicted. Financial data include balance sheet, income statement, cash flow statement, etc. These tables contain key information such as assets, liabilities, owner's equity, income, expenses, profits and cash inflows and outflows of the enterprise. Basic attribute data include the industry type, scale, establishment years, number of employees, etc. of the enterprise, which help the model to understand the background of the enterprise more comprehensively. The extracted financial data and basic attribute data are integrated to form a complete data set. The data is cleaned and preprocessed, including removing duplicate values, filling missing values, data standardization, etc., to ensure the accuracy and consistency of the data. Based on the integrated data, a series of financial ratios are calculated, such as current ratio, quick ratio, debt-to-asset ratio, profit margin, etc. These ratios can reflect the situation of the enterprise's debt-paying ability, profitability, operating efficiency, etc., and provide strong support for the prediction of the financial forecasting model. The integrated financial data, basic attribute data and calculated financial ratios are input into the trained financial forecasting model. Perform forward propagation calculations to obtain the output of the model, including the company's warning level, risk signals, and predicted values ​​of financial indicators.

[0133] Assuming that the enterprise to be predicted is a manufacturing company, the balance sheet, income statement, and cash flow statement for 2022 are extracted from the company's ERP system. Basic attribute data such as the number of employees and years of establishment are obtained from the company's human resources department. The extracted financial data and basic attribute data are integrated into an Excel spreadsheet. The data is cleaned, including removing duplicate data and filling in missing financial data, to ensure the accuracy and completeness of the data. Based on the integrated data, the following financial ratios are calculated:

[0134] Current Ratio: In Excel, assuming that "Current Assets" is in cell A2 and "Current Liabilities" is in cell A2, the formula for the current ratio is Among them, B2 represents current assets and B3 represents current liabilities. Enter this formula into the cell that displays the current ratio (for example, C2).

[0135] Quick Ratio: Quick assets are current assets minus inventory and prepaid accounts. In Excel, assuming that "Inventory" is in cell A4 and "Prepaid Accounts" is in cell A5, then the formula for quick assets is M = B2-B4-B5 (if quick assets are not directly listed in the table). Assuming that the calculated quick assets are placed in cell B6, the formula for the quick ratio is Where B6 represents quick assets and B3 represents current liabilities. Enter this formula into the cell that displays the quick ratio (e.g. C3).

[0136] Debt-to-asset ratio: In Excel, assuming that "Total Liabilities" is in cell A7 and "Total Assets" is in cell A8, the formula for the debt-to-asset ratio is: Among them, B7 represents the total liabilities and B8 represents the total assets. Enter this formula into the cell that displays the debt-to-asset ratio (for example, C4).

[0137] The integrated data (including financial data, basic attribute data and calculated financial ratios) are input into the trained financial forecasting model. The model output shows that the company's warning level is "medium risk" and there are potential financial risk signals, such as low current ratio and declining profit margin. At the same time, the model also predicts the company's financial indicators for the next year, such as expected operating income and net profit.

[0138] Through automated data extraction, integration and forecasting processes, enterprises can obtain financial status and risk information faster, thereby improving decision-making efficiency. Prediction models can promptly detect potential risk signals and abnormal situations, helping enterprises enhance risk management capabilities and avoid or reduce financial losses. Based on the forecast results, enterprises can allocate resources more reasonably, improve capital utilization efficiency and operating benefits. Forecast results can provide strong support for enterprises to formulate long-term strategies and short-term plans, helping enterprises better respond to market changes and challenges.

[0139] In a preferred embodiment of the present invention, the above step 15, optimizing the risk assessment and credit rating process of financial enterprise customers according to the enterprise early warning results and financial indicator prediction results, may include:

[0140] In an embodiment of the present invention, based on the early warning results of the enterprise, the financial institution can dynamically adjust the parameters of the risk assessment model to more accurately reflect the current risk status of the enterprise. For example, if the early warning results show that the enterprise has a high liquidity risk, the financial institution can increase the weight of indicators such as the current ratio and the quick ratio in the risk assessment model. Combined with the prediction results of financial indicators, financial institutions can build a more refined credit rating system. By predicting key financial indicators such as the future operating income and net profit of the enterprise, financial institutions can more accurately judge the growth potential and profitability of the enterprise, thereby assigning it a more appropriate credit rating.

[0141] Assume that a technology-based enterprise expects a substantial increase in operating income in the next year, and the forecast results of financial indicators support this expectation. When evaluating the credit rating of the enterprise, financial institutions can give it a higher rating to reflect its good development prospects and debt repayment ability. Using the enterprise early warning results and financial indicator forecast results, financial institutions can establish a real-time monitoring and early warning mechanism. By continuously tracking the financial status and risk changes of the enterprise, financial institutions can promptly identify potential risk points and take corresponding risk management measures.

[0142] like Figure 2 As shown, an embodiment of the present invention further provides a system information processing system 20 based on power data, comprising:

[0143] The acquisition module 21 is used to acquire the financial data and basic attribute data of the enterprise for multiple years, and perform data preprocessing to obtain processed financial data sets and basic attribute data sets; construct a financial forecasting model based on the transposition transformer according to the processed financial data sets and basic attribute data sets, wherein the transposition transformer adjusts the input matrix dimension to transpose [L, D] to [D, L] so that each feature forms an independent time series; input the enterprise financial time series data and the processed basic attribute data set into the financial forecasting model for training to obtain a trained financial forecasting model;

[0144] Processing module 22 is used to obtain the financial and attribute data of the enterprise to be predicted, and input it into the trained financial prediction model to obtain the enterprise's early warning results and financial indicator prediction results; based on the enterprise's early warning results and financial indicator prediction results, optimize the risk assessment and credit rating process of financial enterprise customers.

[0145] It should be noted that the system is a system corresponding to the above method, and all implementation methods in the above method embodiment are applicable to this embodiment and can achieve the same technical effect.

[0146] The embodiment of the present invention further provides a computing device, comprising: a processor, a memory storing a computer program, wherein when the computer program is executed by the processor, the method described above is executed. All implementations in the above method embodiment are applicable to this embodiment and can achieve the same technical effect.

[0147] The embodiment of the present invention also provides a computer-readable storage medium storing instructions, which, when executed on a computer, enable the computer to execute the method described above. All implementations in the above method embodiment are applicable to this embodiment and can achieve the same technical effect.

[0148] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A financial early warning method for financial enterprise customers based on Transformer, characterized in that: The method comprises: Obtain the company's multi-year financial data and basic attribute data, and perform data preprocessing to obtain processed financial data sets and basic attribute data sets; According to the processed financial data set and basic attribute data set, a financial forecasting model based on the Transpose Transformer is constructed. The Transpose Transformer adjusts the input matrix dimension and transposes [L, D] to [D, L], so that each feature forms an independent time series. Input the enterprise financial time series data and the processed basic attribute data set into the financial forecasting model for training to obtain a trained financial forecasting model; Obtain the financial and attribute data of the enterprise to be predicted, and input it into the trained financial prediction model to obtain the early warning results and financial indicator prediction results of the enterprise; Optimize the risk assessment and credit rating process for financial enterprise customers based on enterprise early warning results and financial indicator forecast results.

2. The financial early warning method for financial enterprise customers based on Transformer according to claim 1 is characterized in that: Obtain the company's multi-year financial data and basic attribute data, and perform data preprocessing to obtain processed financial data sets and basic attribute data sets, including: Extract multi-year financial data and basic attribute data of enterprises from databases and financial statement data sources, including financial indicators, time range, frequency and basic attributes; Identify and process missing values ​​in the data source. For financial data, use the average value to fill or delete missing values. For basic attribute data, record missing values ​​as a separate category. Align the time of multi-year financial data. If a period of financial data is missing, use the average value of the previous and next values ​​to fill it in to obtain the processed financial data set and basic attribute data set.

3. The financial early warning method for financial enterprise customers based on Transformer according to claim 2 is characterized in that: According to the processed financial data set and basic attribute data set, a financial forecasting model based on the transposed transformer is constructed. The transposed transformer adjusts the input matrix dimension and transposes [L, D] to [D, L], so that each feature forms an independent time series, including: According to the financial forecasting goal, the categorical features in the processed financial data set and basic attribute data set are embedded and converted into vector representation; the numerical features are binned and encoded with one bit effectively to convert them into vector representation; the input data of the transposed Transformer is determined to be a three-dimensional matrix [B, L, D], where the first dimension B represents the batch size of a training, the second dimension L represents the sequence dimension, and the third dimension D represents the feature dimension; Adjust the dimension of the input matrix, transpose the position of [L, D], exchange the position of the sequence dimension and the feature dimension, so that the sequence of each single feature at each time point is spliced ​​and merged to form a new input matrix [B, D, L]; The category feature vector and the numerical feature vector processed by the embedding layer are concatenated according to the transposed dimensions [B, D, L] to obtain the input vector of the multilayer perceptron; Use a single-layer fully connected neural network to perform linear transformation on the concatenated input vector to obtain an intermediate result, and use an activation function to perform nonlinear activation on the intermediate result to obtain the output result of the feedforward network layer; According to the output results of the feedforward network layer, the linear fully connected neural network is used again to independently process the time series of each feature to extract a high-dimensional feature vector, and the high-dimensional feature vector is linearly mapped to obtain the query vector, key vector and value vector; Calculate the dot product of the query vector and the key vector and scale them to get the scaled result; Use the flexible maximum function to convert the scaled result into an attention score to obtain a vector represented by multi-head self-attention. Perform layer normalization on the vectors represented by the multi-head self-attention to obtain a normalized vector; Use a feedforward network layer containing two layers of linear fully connected neural networks to perform nonlinear transformation and combination on the normalized vectors, and concatenate the output results of the feedforward network layer and the time series information to obtain a concatenated feature vector; The concatenated feature vector is used as the final feature representation of the financial forecasting model and input into the fully connected layer to build a financial forecasting model based on the transposed Transformer.

4. The financial early warning method for financial enterprise customers based on Transformer according to claim 3 is characterized in that: The dot product of the query vector and the key vector is calculated as: Where P(Q,K,W) represents the dot product of the query vector Q and the key vector K after linear transformation by the weight matrix W; Q represents the query vector; K represents the key vector; W represents the weight matrix; d represents the dimensions of the query vector, key vector and weight matrix; W ij represents the element in the i-th row and j-th column of the weight matrix; Q j represents the jth component of the query vector; W ik represents the element in the i-th row and k-th column of the weight matrix; K k represents the kth component of the key vector; k represents the index of traversing the key vector; j represents the index of traversing the query vector; i represents the outer sum index.

5. The financial early warning method for financial enterprise customers based on Transformer according to claim 4 is characterized in that: The enterprise financial time series data and the processed basic attribute data set are input into the financial forecasting model for training to obtain the trained financial forecasting model, including: According to the preset ratio, the enterprise financial time series data and the processed basic attribute data set are divided into a training set and a validation set; Initialize the financial forecasting model, including configuring the embedding layer, multivariate attention layer, layer normalization, and feedforward network layer components of the financial forecasting model, and determine the learning rate, batch size, and number of training rounds of the financial forecasting model; Input each sample in the training set into the financial forecasting model, perform forward propagation calculations, and output the corresponding forecast results based on the characteristics of each input sample; Calculate the loss function value based on the difference between the corresponding prediction result and the true label; According to the loss function value, back propagation is performed to calculate the gradient and update the financial forecasting model parameters; Repeat the forward propagation, loss calculation, back propagation and parameter update operations until the preset number of training rounds is reached to obtain the trained financial prediction model.

6. The financial early warning method for financial enterprise customers based on Transformer according to claim 5 is characterized in that: The calculation formula of the loss function value is: Where L represents the overall loss function value; n represents the total number of training samples; w h represents the weight of the hth sample; f represents the total number of categories; m represents the index of the category; y hm represents a binary indicator variable; represents the probability that the hth sample predicted by the model belongs to the mth class; γ represents the time series penalty coefficient; T represents Timing penalty term; represents the prediction result of the financial prediction model; y represents the true label; t represents the time information; h represents the index of the sample.

7. The financial early warning method for financial enterprise customers based on Transformer according to claim 6 is characterized in that: Obtain the financial and attribute data of the enterprise to be predicted, and input it into the trained financial prediction model to obtain the enterprise's early warning results and financial indicator prediction results, including: Extract new financial data and basic attribute data from the internal database and financial statement data source of the enterprise to be forecasted, including balance sheet, income statement, and cash flow statement; Integrate new financial data and basic attribute data to obtain integrated data; Calculate financial ratios, including current ratios, based on consolidated data; The financial, attribute data and financial ratios of the enterprise to be predicted are input into the trained model financial prediction model, and the forward propagation calculation is performed to obtain the model output warning results and financial indicator prediction results, including the enterprise's warning level, risk signal and predicted value of financial indicators.

8. A financial enterprise customer financial early warning system based on Transformer, the system implements the method according to any one of claims 1 to 7, characterized in that: include: The acquisition module is used to acquire the financial data and basic attribute data of the enterprise for multiple years and perform data preprocessing to obtain the processed financial data set and basic attribute data set; According to the processed financial data set and basic attribute data set, a financial forecasting model based on the transposition transformer is constructed, wherein the transposition transformer adjusts the input matrix dimension and transposes [L, D] to [D, L] so that each feature forms an independent time series; the enterprise financial time series data and the processed basic attribute data set are input into the financial forecasting model for training to obtain the trained financial forecasting model; The processing module is used to obtain the financial and attribute data of the enterprise to be predicted, and input it into the trained financial prediction model to obtain the enterprise's early warning results and financial indicator prediction results; based on the enterprise's early warning results and financial indicator prediction results, the risk assessment and credit rating process of financial enterprise customers is optimized.

9. A computing device, characterized in that include: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images