A financial data analysis and forecasting system and method based on artificial intelligence

Abstract

The present invention discloses a financial data analysis and prediction system based on artificial intelligence, including a financial data collection module, a financial data outlier detection and modification module, a financial data analysis module, a financial data prediction module, and a user interaction module. Also disclosed is a financial data analysis method based on artificial intelligence, which uses a financial data analysis model to collect and clean financial data from a financial data source to obtain a data set; uses an optimized lonely forest algorithm to process data points in the data set, marks and modifies outliers, and integrates to obtain an updated data set; the updated data set is subjected to standardization processing, and then analyzed using an enhanced autoregressive difference moving average model to output a comprehensive feature vector; the financial data prediction model predicts the comprehensive feature vector to obtain a predicted value, and dynamically adjusts the prediction error threshold. The technical solution of the present invention can improve the accuracy of the analysis results, the prediction accuracy, and can ensure long-term use effects.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a financial data analysis and prediction system and method based on artificial intelligence. Background Art

[0002] Currently, in commercial applications, financial data analysis and forecasting systems based on artificial intelligence have demonstrated unique technical advantages. Through automated scanning and identification of financial data from multiple sources, human errors are reduced, the speed and accuracy of data processing are improved, and inflow data is analyzed in real time to quickly respond to market changes, enabling companies to adjust strategies instantly. Using artificial intelligence algorithms, they can learn patterns and trends from historical data, predict future market trends, earnings and risk levels, and provide more accurate forecast results than traditional statistical methods. At the same time, financial data analysis and forecasting systems based on artificial intelligence combine complex data processing capabilities with advanced analytical techniques to achieve data-driven decision-making, which not only improves the operational efficiency of enterprises, but also enhances their ability to adapt to market changes.

[0003] The Chinese patent with application number CN202310828436.9 and publication date of 2023.08.04 discloses a financial data analysis method and system, including: a model library, an acquisition unit, a feature extraction unit, and an execution unit; the acquisition unit is connected to the model library to obtain the enterprise type of the target enterprise and input it into the model library; the model library is connected to the execution unit, and the execution model corresponding to the enterprise type is selected from the model library as the preset execution model; the acquisition unit is also connected to the feature extraction unit to obtain the target enterprise's target year financial data and perform feature extraction to form the current financial feature data, and obtain the target enterprise's target year's financial data for feature extraction to form historical financial feature data; the feature extraction unit is connected to the execution unit, and the historical financial feature data and the current financial feature data are input into the preset execution model to obtain the output data of the preset execution model as the target year behavior data. The technical solution of this patent can simulate the main behaviors performed by the enterprise in the year through the financial data of two consecutive years. On the one hand, it can provide a basis for the enterprise's decision-making in subsequent years, and on the other hand, it can provide a reliable reference for the enterprise's analysis of competitors.

[0004] However, the above technology has at least the following technical problems: the existing technology relies on detailed financial data for two consecutive years, requiring the target company to have complete and accurate financial records. If the financial data records are incomplete or incorrect, the output of the system will be affected, resulting in inaccurate analysis results; relying solely on historical financial data and ignoring external factors will lead to inaccurate predictions in a rapidly changing economic environment; the model needs to be continuously updated and optimized to maintain the effectiveness and accuracy of the model. The existing technology does not mention the update and maintenance mechanism of the model, which may easily lead to a gradual decrease in the effect during long-term use. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a financial data analysis and forecasting system and method based on artificial intelligence to solve the problems existing in the prior art, namely, complete reliance on complete and accurate financial record data, which may lead to inaccurate analysis results when the financial data records are incomplete or erroneous; and only considering historical financial data while ignoring external factors, which may lead to inaccurate forecasts in a rapidly changing economic environment; and the lack of an update and maintenance mechanism for the model, which may lead to a gradual decrease in the effect during long-term use.

[0006] An artificial intelligence-based financial data analysis and forecasting system disclosed in an embodiment of the present invention includes:

[0007] The financial data collection module is used to automatically connect to the financial data source using the financial data analysis model, collect and clean the financial data in the financial data source, and integrate the financial data after standardization and dynamic field mapping as data points into a data set;

[0008] The financial data outlier detection and modification module is used to calculate the isolation index based on the data set through the lonely forest algorithm, automatically set the threshold and mark the outliers, modify the outliers using the linear regression model to obtain the modified values, and integrate the modified values ​​with the data points not marked as outliers to obtain the updated data set;

[0009] The financial data analysis module is used to standardize the updated data set, obtain the enhanced autoregressive difference moving average model by combining seasonal adjustment and exogenous variables, analyze the standardized data set, and output the comprehensive feature vector;

[0010] A financial data prediction module is used to predict the comprehensive feature vector using a financial data prediction model, obtain a predicted value and output it, and dynamically adjust the prediction error threshold of the financial data prediction model through adaptive threshold adjustment;

[0011] The user interaction module is used to receive the predicted values ​​and generate detailed reports and charts to display the indicators and trends of data analysis and prediction.

[0012] Another embodiment of the present application discloses a financial data analysis and prediction method based on artificial intelligence, comprising the following steps:

[0013] S1: Use the financial data parsing model to automatically connect to the financial data source, collect and clean the financial data in the financial data source, and integrate the financial data as data points into a data set after standardization and dynamic field mapping;

[0014] S2: The lonely forest algorithm is optimized by introducing a sample density factor. The optimized lonely forest algorithm calculates the isolation index based on the data points in the data set, automatically sets the threshold and marks the outliers, and uses a linear regression model to modify the outliers to obtain modified values. The modified values ​​and data points that are not marked as outliers are integrated to obtain an updated data set.

[0015] S3: The updated data set is standardized to obtain a standardized data set. The enhanced autoregressive difference moving average model is obtained by combining seasonal adjustment and exogenous variables. The standardized data set is analyzed and a comprehensive feature vector is output;

[0016] S4: The financial data prediction model based on the neural network predicts the comprehensive feature vector to obtain the predicted value, and realizes the dynamic adjustment of the prediction error threshold in the financial data prediction model through adaptive threshold adjustment.

[0017] As a specific implementation, in step S1, the financial data source includes an ERP system, an online transaction platform, a bank interface and a cloud financial service platform.

[0018] As a specific implementation, the “standardization and dynamic field mapping” in step S1 specifically includes:

[0019] Use the pre-trained model of natural language processing to standardize the field names first, and then convert each field name into a high-dimensional semantic vector;

[0020] The weighted cosine similarity between the semantic vectors is calculated to find the field that is most similar to the standard field name and realize dynamic field mapping;

[0021] The calculation formula of weighted cosine similarity is:

[0022]

[0023] Among them, Q(u, v, w) represents weighted cosine similarity, which means the similarity between two vectors u and v adjusted by weight factor w; u and v represent two vectors, which are the representations of field names in a specific vector space; each element u m and v mRepresent the components of vectors u and v in the mth dimension respectively; w is a weight vector; w m represents the mth element in the w vector, representing the weight coefficient of the mth dimension in the similarity calculation; ∑(w m ·u m ·v m ) is the calculation of weighted dot product; and The expressions within the square root sign calculate the magnitudes of the weighted vectors u and v respectively.

[0024] As a specific implementation, in step S2, “optimizing the lonely forest algorithm by introducing a sample density factor” specifically includes:

[0025] S21: Calculate the sample density factor; the sample density factor is normalized and exponentially transformed based on the average distance between the data point and its nearest k neighbors. The specific formula is:

[0026]

[0027] Among them, D(x i ) is a measure used to evaluate the local density of data points in the space where they are located; kNN(x i ) represents the average k-nearest neighbor distance of data point x, that is, the average distance between data point x and its k nearest neighbors; d max represents the maximum k nearest neighbor distance of all data points in the data set L; λ represents a positive adjustment parameter;

[0028] S22: Adjust the logic of building isolated trees; optimize the segmentation logic, and the selection process of segmentation features will be adjusted according to the sample density factor; after determining the segmentation features, select the split value and adjust it according to the sample density factor, and the selected split value should achieve the maximum difference in density of child nodes after segmentation; when building isolated trees, apply the optimized segmentation logic and continuously build trees until all data points are isolated or the maximum depth of the tree is reached;

[0029] S23: Calculate the adjusted path length; when calculating the isolation index of the data point, introduce the sample density factor to adjust the path length, combine the path length with the sample density factor, and make the path length increase proportionally to the inverse of the sample density factor to weaken the influence of the shorter path length in the high-density area.

[0030] As a specific implementation, the specific formula of "obtaining an enhanced autoregressive difference moving average model by combining seasonal adjustment and exogenous variables to analyze the standardized data set" in step S3 is:

[0031]

[0032] in, and are the autoregressive polynomial and the moving average polynomial respectively; B is the backoff operator, which moves each data point in the time series to the beginning of the time series by a specified number of time steps; A and a are the orders of seasonal and non-seasonal differences respectively; S is the seasonal period; (X j ′) t is the standardized data point at time t; ε t is the error term at time t; δ is the coefficient vector of the exogenous variables; Z t is the vector of exogenous variables at time t; T is the transpose operator.

[0033] As a specific implementation, the data point (X) after normalization at time t j ′) t Apply seasonal difference operation (1-B S ) a , the specific formula is:

[0034] (1-B S ) a (X j ′) t =(X j ′) t -(X j ′) t-S

[0035] Among them, (1-B S ) a is the representation of the seasonal difference operation; (X j ′) t is a data point in the time series, t refers to the time index; B is the back-off operator, which means the operation of moving the data points in the time series forward; S is the seasonal cycle, which means the number of time points covered by the complete seasonal cycle; a is the order of difference, which means the number of times the difference operation needs to be repeated, a=1 means one seasonal difference, if a>1, multiple differences are performed.

[0036] As a specific implementation, in step S4, the financial data prediction model is a neural network including an input layer, a long short-term memory layer, a convolutional layer, a fully connected layer, and an output layer; the predicted values ​​include the company's future operating income, profit, and stock price.

[0037] As a specific implementation, in step S4, "predicting the comprehensive feature vector based on the financial data prediction model of the neural network to obtain the predicted value" specifically includes:

[0038] In the long short-term memory layer, an improved structure of a gated recurrent unit (GRU) is introduced, and the comprehensive feature vector input is introduced into the long short-term memory layer of the improved structure of the gated recurrent unit (GRU) to obtain the first output;

[0039] In the convolution layer, the first output of the long short-term memory layer is processed with a one-dimensional convolution kernel to extract the local features of the time series. The convolution operation performs element-wise multiplication and summation on the time series through a sliding window to capture short-term trends and periodic changes to obtain the second output.

[0040] In the fully connected layer, the second output is processed using the ReLU activation function to obtain the third output;

[0041] In the output layer, the third output is processed using the tanh function to obtain the predicted value.

[0042] As a specific implementation, in step S4, "dynamically adjusting the prediction error threshold in the financial data prediction model through adaptive threshold adjustment" is specifically formulated as follows:

[0043] F n+1 =F n +α(|Y real -Y pred |-F n )

[0044] Among them, F n represents the current prediction error threshold at the nth prediction, F n The initial value of is the forecast output of the financial data forecasting model; n+1 The updated prediction error threshold is adjusted according to the latest error data; α is the adjustment rate parameter, ranging from (0, 1), which controls the sensitivity and speed of the prediction error threshold adjustment; |Y real -Y pred | is the absolute error between the actual data point and the predicted data point.

[0045] The technical solution of the embodiment of the present invention has at least the following beneficial effects:

[0046] 1. By automatically scanning and identifying connected financial data sources, such as ERP systems, online trading platforms, bank interfaces, and cloud financial service platforms, it ensures the diversity and richness of financial data and helps improve the quality of decision-making.

[0047] 2. Applying advanced natural language processing pre-trained models, such as BERT, to semantically understand and vectorize field names can effectively solve the problem of inconsistent field names in different data sources. By calculating the similarity between semantic vectors, such as weighted cosine similarity, the model can dynamically identify and map similar fields, which is crucial for integrating data from multiple sources.

[0048] 3. The lonely forest algorithm is used for outlier detection, and by introducing sample density factors for optimization, it can effectively identify and process outliers, reduce the negative impact of outliers on analysis results, and improve the accuracy of financial data analysis.

[0049] 4. The enhanced autoregressive difference-moving average model that combines seasonal adjustment and exogenous variables can better understand the pattern of data fluctuations and achieve more accurate data analysis.

[0050] 5. Use the financial data prediction model based on neural network to predict the comprehensive feature vector. Through deep learning processing of the comprehensive feature vector through activation function and network structure, the complex patterns and associations in the comprehensive feature vector can be discovered, thus improving the accuracy of the prediction.

[0051] 6. The introduced adaptive threshold adjustment module can adjust the threshold according to the real-time performance of the prediction results, providing a more flexible and precise control method to adapt to the changing data characteristics, reduce errors and optimize the prediction performance of the financial data prediction model. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is a module diagram of the financial data analysis and forecasting system based on artificial intelligence according to an embodiment of the present application;

[0053] Figure 2 This is a flowchart of the artificial intelligence-based financial data analysis and forecasting method of an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0056] See attached Figure 1 An artificial intelligence-based financial data analysis and prediction system provided by an embodiment of the present invention includes a financial data collection module, a financial data outlier detection and modification module, a financial data analysis module, a financial data prediction module, and a user interaction module.

[0057] The financial data collection module is used to automatically connect to the financial data source using the financial data analysis model, collect and clean the financial data in the financial data source, and after standardization and dynamic field mapping, integrate the financial data into a data set as data points and output it to the financial data outlier detection and modification module.

[0058] The financial data outlier detection and modification module is used for the data set based on the financial data collection module. It calculates the isolation index through the lonely forest algorithm, automatically sets the threshold and marks the outliers, and uses the linear regression model to modify the outliers to obtain the modified values. The modified values ​​and data points that are not marked as outliers are integrated to obtain the updated data set, which is output to the financial data analysis module.

[0059] The financial data analysis module is used to perform standardization based on the updated data set, obtain the enhanced autoregressive difference moving average model by combining seasonal adjustment and exogenous variables, analyze the standardized data set, and output the comprehensive feature vector to the financial data prediction module.

[0060] The financial data prediction module is used to use the financial data prediction model to predict the comprehensive feature vector, obtain the predicted value, input the predicted company's future operating income, profit, and stock price into the user interaction module, and dynamically adjust the prediction error threshold of the financial data prediction model through adaptive threshold adjustment.

[0061] User interaction module generates detailed reports and charts to display indicators and trends for data analysis and forecasting.

[0062] See attached Figure 2 Another embodiment of the present invention further discloses a financial data analysis and forecasting method based on artificial intelligence, the method comprising the following steps:

[0063] S1: Use the financial data parsing model to automatically connect to the financial data source, collect and clean the financial data in the financial data source, and integrate the financial data into a data set as data points after standardization and dynamic field mapping.

[0064] Considering that the financial data formats output by different financial data sources are not uniform, the financial data parsing model is used to ensure that the financial data formats output by different financial data sources are uniform, which is convenient for subsequent analysis and prediction. For example, the financial data source can be an ERP system, an online trading platform, a bank interface, and a cloud financial service platform; the financial data is one or more of the balance sheet data, income statement data, cash flow statement data, budget and forecast data, which ensures the diversity and richness of financial data and helps improve the quality of decision-making.

[0065] In this step, the process of the financial data parsing model parsing the financial data format is as follows:

[0066] S11: Financial data collection and preprocessing; automatically connect to financial data sources and collect financial data, clean the collected financial data, remove invalid data points, fill in incomplete data points, and correct obvious errors such as format mismatch or data corruption.

[0067] S12: Standardization and dynamic field mapping; use pre-trained models suitable for natural language processing, such as the Bidirectional Encoder Representation Transformer (BERT), which can capture the complex relationships and semantics between words through large-scale corpus learning, first standardize the field names, and then convert each field name into a high-dimensional semantic vector; use similarity metrics in vector space, such as weighted cosine similarity, to perform mathematical calculations and similarity comparisons, calculate the weighted cosine similarity between each semantic vector, find the field that is most similar to the standard field name, and optimize the financial data parsing model, reducing the possibility of human errors and improving the overall accuracy of financial data. It can be continuously optimized and updated as the company's business develops and the financial data source increases.

[0068] Weighted cosine similarity, the specific formula is:

[0069]

[0070] Among them, Q(u, v, w) represents weighted cosine similarity, which means the similarity between two vectors u and v adjusted by weight factor w. It is calculated by vector dot product and modulus length and is used to measure the similarity between texts. The value range of weighted cosine similarity is [0, 1], where 0 means no similarity and 1 means complete similarity. u and v represent two vectors, which are the representations of field names in a specific vector space. By converting text fields into numerical form, the similarity between fields can be calculated by mathematical methods. Each element u m and v m Represent the components of vectors u and v in the mth dimension respectively; w is a weight vector used to adjust the importance of each dimension in the similarity calculation according to domain expertise. In the financial field, specific financial terms are more important than general terms, and the importance can be reflected by increasing the weight of the corresponding dimension of the financial term; w m represents the mth element in the w vector, representing the weight coefficient of the mth dimension in the similarity calculation; ∑(w m ·u m ·v m ) is the calculation of the weighted dot product, that is, the corresponding elements of the two vectors are multiplied and then multiplied by the corresponding weights. The sum of the products constitutes the numerator, which measures the degree of alignment of the two vectors in the weighted multidimensional space; and The expressions inside the square root calculate the modulus of the weighted vectors u and v respectively. The modulus is calculated by taking the square of each element of the vector, multiplying it by the corresponding weight, and then taking the square root. In order to normalize the dot product value, ensure that the obtained similarity is a dimensionless ratio between 0 and 1.

[0071] By calculating weighted cosine similarity, the bidirectional encoder representation converter finds the field that is most similar to the standard field name and realizes dynamic field mapping.

[0072] S13: Financial data integration and output: The standardized and dynamically mapped financial data are integrated as data points into a data set L, where L = [x1, x2, ..., x n ], n represents n data points, x i (i∈1, 2, ..., n) represents the i-th data point.

[0073] S2: The lonely forest algorithm is optimized by introducing a sample density factor. The optimized lonely forest algorithm calculates the isolation index based on the data points in the data set, automatically sets the threshold and marks the outliers, and uses a linear regression model to modify the outliers to obtain modified values. The modified values ​​and data points that are not marked as outliers are integrated to obtain an updated data set.

[0074] In order to increase the accuracy of subsequent analysis and prediction, the existing lonely forest algorithm is used to detect outliers in the data set L to eliminate the interference of outliers on the overall financial data. The lonely forest algorithm is an effective anomaly detection algorithm, which is mainly used to identify abnormal points or outliers in the data set. The core is to calculate the isolation index of each data point. The isolation index reflects the difficulty of isolating the data point.

[0075] The lonely forest algorithm is used to calculate the isolation index of each data point. The specific formula is:

[0076]

[0077] Among them, s(x i , n) represents the data point x i The isolation index of the data point x is i The greater the likelihood of being considered abnormal; x i represents the data point being evaluated; n is the total number of data points; E(h(x i )) means that in the isolation forest, the data point x i The average path length reflects the data point x i The difficulty of being isolated. The shorter the path, the more abnormal the data point. c(n) is a normalization factor used to adjust the path length.

[0078] The calculation formula of the normalization factor is:

[0079]

[0080] Where H(·) represents the Harmon number, which is a normalization adjustment for the average path length;

[0081] In order to reduce the impact of data density on outlier detection, the sample density factor is introduced to optimize the lonely forest algorithm. The sample density factor is used to identify data points in high-density areas. The purpose is to adjust the segmentation probability of data points in isolated trees so that the lonely forest algorithm can handle data points in high-density and low-density areas more fairly, and improve the detection of outliers in high-density areas by affecting the path length of data points in isolated trees.

[0082] The specific process of introducing the sample density factor to optimize the lonely forest algorithm is as follows:

[0083] S21: Calculate the sample density factor; the sample density factor is normalized and exponentially transformed based on the average distance between the data point and its nearest k neighbors. The specific formula is:

[0084]

[0085] Among them, D(x i ) is a measure used to evaluate the local density of data points in the space where they are located. It is calculated based on the average distance between the data point and its nearest k neighbors, and expresses the density or isolation of the data points in an exponential transformation form; kNN(x i ) represents the average k-nearest neighbor distance of data point x, that is, the average distance between data point x and its k nearest neighbors, which is used to evaluate the local density of data point x; d max represents the maximum k nearest neighbor distance of all data points in the dataset L, which is used to normalize kNN(x i ), ensuring that the sample density factor is within a reasonable range; λ represents a positive adjustment parameter, which is used to control the influence of the sample density factor. By adjusting the value of λ, the influence of local density on the isolation index can be enhanced or weakened.

[0086] S22: Adjust the logic of building isolated trees; in the traditional lonely forest algorithm, the split features and split values ​​are randomly selected; in the optimized split logic, the selection process of the split features will be adjusted according to the sample density factor. Specifically, a probability model can be set so that data points in high-density areas have a higher probability of being selected as split points, which helps to isolate points in dense areas more frequently; after determining the split features, the selection of appropriate split values ​​is also adjusted according to the sample density factor. The preferred split value should achieve the maximum difference in sub-node density after the split. Specifically, it can be achieved by maximizing the difference in sample density factors on both sides after the split; when building isolated trees, the optimized split logic is applied, and the tree is continuously built until all data points are isolated or the maximum depth of the tree is reached. The creation of each tree node is based on the optimized split logic, which makes the entire forest more sensitive to density changes in the data set.

[0087] S23: Calculate the adjusted path length; when calculating the isolation index of the data point, introduce the sample density factor to adjust the path length. Specifically, consider combining the path length with the sample density factor, and increase the path length proportionally to the inverse of the sample density factor to weaken the impact of the shorter path length in the high-density area.

[0088] After the above steps, the optimized lonely forest algorithm is obtained. The new isolation index s′(x i , n);

[0089] In order to automatically identify and process outliers, the threshold is automatically set and outliers are marked according to the threshold;

[0090] The threshold setting formula is:

[0091] P=μ+l×γ

[0092] Among them, P is the threshold; μ is the mean of the isolation index, that is, the average of the isolation index of all data points; l is a coefficient used to adjust the strictness of the threshold; γ is the standard deviation of the isolation index, which indicates the degree of dispersion of the isolation index distribution.

[0093] Outlier marking rules:

[0094] If s′(x i , n) ≥ P, then the data points with abnormal isolated index are marked as outliers;

[0095] Replace the data points marked with anomalies in the isolation index with the calculated mean, using the formula:

[0096]

[0097] Among them, x newis the calculated mean, which is the modified value used to replace the outliers in the data set; Used to calculate all x from i=1 to i=m i The sum of the values; m represents the number of data points marked as isolated index anomalies.

[0098] The modified values ​​are obtained through the above steps, and the modified values ​​and the data points not marked as outliers are integrated to obtain the updated data set L′=[X1, X2, ..., X n ], n represents n updated data points, X j (X j ∈1, 2, ..., n) represents the jth updated data point.

[0099] S3: The updated data set is standardized to obtain a standardized data set. The enhanced autoregressive difference moving average model is obtained by combining seasonal adjustment and exogenous variables. The standardized data set is analyzed and the comprehensive feature vector is output.

[0100] The updated data set is standardized to obtain a standardized data set. The specific formula is:

[0101]

[0102] Among them, X j ′ represents the standardized data point; X j Represents the updated data point; Represents the minimum value of the updated data point; Indicates the maximum value of the updated data point.

[0103] In the process of financial data analysis, the standardized data points are analyzed using the existing autoregressive difference moving average model, which is a statistical model widely used in time series data analysis and prediction, and outputs a comprehensive feature vector through the information of historical data.

[0104] In order to further enhance the accuracy of the analysis results, the autoregressive difference moving average model is combined with seasonal adjustment and exogenous variables to obtain the enhanced autoregressive difference moving average model. By introducing seasonal differences, it can deal with data that are obviously affected by seasonal factors, such as retail sales and tourism statistics, and achieve more effective and accurate analysis. By incorporating exogenous variables, such as economic indicators, policy changes, and market conditions, the enhanced autoregressive difference moving average model can further improve the accuracy of the analysis.

[0105] The standardized data points are analyzed using the enhanced autoregressive difference moving average model. The specific formula is:

[0106]

[0107] in, and are the autoregressive polynomial and the moving average polynomial, respectively, which describe the dependency structure of the time series; B is the backoff operator, which is a mathematical tool used to simplify the expression of differential operations. Its function is to move each data point in the time series to the beginning of the time series by a specified number of time steps; A and a are the orders of seasonal and non-seasonal differences, respectively; S is the seasonal period; (X j ′) t is the standardized data point at time t; ε t is the error term at time t; δ is the coefficient vector of the exogenous variable, which describes the contribution of the exogenous variable to the prediction model; Z t is the vector of exogenous variables at time t; T is the transpose operator, which is used to represent the vector δ as a row vector such that δ T Z t is a scalar product, that is, the sum of the products of the coefficient and the corresponding exogenous variable value.

[0108] The standardized data point (X j ′) t Apply seasonal difference operation (1-B S ) a , the specific formula is:

[0109] (1-B S ) a (X j ′) t =(X j ′) t -(X j ′) t-S

[0110] Among them, (1-B S ) a is the representation of the seasonal difference operation; (X j ′) t is a data point in a time series, t refers to the time index, which can be any data recorded in chronological order, such as monthly sales; B is a backward operator, which is used to represent the operation of moving a data point in a time series forward. When B acts on a time series (X j ′) t When B(X j ′) t =(X j ′) t-1, which means moving the data points forward by one time unit; S is the seasonal period, which indicates the number of time points covered by a complete seasonal cycle; a is the order of difference, which indicates the number of times the difference operation needs to be repeated. In the formula, a=1 means performing one seasonal difference. If a>1, multiple differences are performed in order to further eliminate more complex seasonal patterns in the data.

[0111] The data points after seasonal difference operation reflect the pure trend after removing seasonal fluctuations, and the statistical summary (such as mean, variance) of each time point after difference is used as part of the comprehensive feature vector.

[0112] Calculate the exogenous variable vector Z t With the coefficient vector Z t The dot product δ T Z t , which reflects the specific contribution of exogenous variables to time series analysis and takes the contribution of exogenous variables as part of the comprehensive feature vector.

[0113] Using the residual ε t , statistics such as mean, standard deviation, skewness, and kurtosis can be calculated as part of the comprehensive feature vector.

[0114] The standardized data points are accurately analyzed through the enhanced autoregressive difference moving average model to output the comprehensive feature vector The comprehensive feature vector is an array containing characteristic data (such as statistical summaries of the data points after seasonal differencing, contributions of exogenous variables, and statistical properties of the residuals).

[0115] S4: The financial data prediction model based on the neural network predicts the comprehensive feature vector to obtain the predicted value, and realizes the dynamic adjustment of the prediction error threshold in the financial data prediction model through adaptive threshold adjustment.

[0116] The comprehensive feature vector The data is input into the financial data prediction model, which is a neural network including an input layer, a long short-term memory layer, a convolutional layer, a fully connected layer, and an output layer. It can not only effectively process and predict complex financial data, but also improve the accuracy and reliability of the prediction through multi-level nonlinear mapping of the deep network.

[0117] In the long short-term memory layer, an improved structure called gated recurrent unit (GRU) is introduced to simplify the gating mechanism of LSTM and reduce the number of parameters while retaining similar memory capabilities to avoid the gradient vanishing problem common in traditional neural networks, enabling the network to learn dependencies from the distant past to the present.

[0118] The steps of the GRU layer are as follows:

[0119] The first step is to update the door;

[0120] The specific formula for calculating the update gate is:

[0121] Among them, M t Represents the update gate, which controls the previous hidden state N t-1 How much information needs to be retained to the current hidden state N t ; σ represents the Sigmoid activation function, which is used to update the calculation of the gate, limiting the output between 0 and 1, and helps to decide the degree to which information is retained or forgotten; W M represents the weight of the update gate; N t-1 Represents the hidden state at the previous moment: the hidden state passed down from the previous time step contains the information of the previous time step; represents the input feature vector of the current time step; b M Represents the bias term to add additional bias to adjust the gating signal and state update.

[0122] The formula for updating the candidate hidden state is:

[0123]

[0124] Among them, N′ t Represents updating the candidate hidden state, which is calculated based on the current input and the previous hidden state after the reset gate is filtered, and is used to update the current hidden state; tanh activation function; W N represents the weight of the candidate hidden state; r t Represents the reset gate, which controls the update of the candidate hidden state N t How much past information needs to be ignored when calculating the hidden state? This can help the model forget irrelevant information. ⊙ represents element-by-element multiplication, which is used for element-level multiplication operations. For example, when calculating the updated hidden state, the output of the update gate is multiplied by the candidate hidden state element by element. N t-1 Represents the hidden state at the previous moment. The hidden state passed down from the previous time step contains the information of the previous time step; represents the input feature vector of the current time step; b N Represents the bias term to add additional bias to adjust the gating signal and state update.

[0125] Step 2: Reset the door;

[0126] The specific formula for calculating the reset gate is:

[0127]

[0128] Among them, rt Represents the reset gate, which controls the update of the candidate hidden state N′ t How much past information needs to be ignored when the model is in the state of being ignored can help the model forget irrelevant information; σ represents the Sigmoid activation function, which is used to reset the calculation of the gate and limit the output between 0 and 1, which helps to decide the degree of retaining or forgetting information; N t-1 Represents the hidden state at the previous moment. The hidden state passed down from the previous time step contains the information of the previous time step; represents the input feature vector of the current time step; b r represents the bias term to add additional bias to adjust the gating signal and state update;

[0129] The third step is to update the hidden state;

[0130] The specific formula for calculating the hidden state is:

[0131] N t =(1-M t )⊙N t-1 +M t ⊙N t

[0132] N t Represents the hidden state of the current time step: the new hidden state calculated at the current time step will be passed to the next time step; N t-1 Represents the hidden state of the previous moment. The hidden state passed down from the previous time step contains the information of the previous time step; M t Represents the update gate, which controls the previous hidden state N t-1 How much information needs to be retained to the current hidden state N t , helps the model decide how much past information to retain at each time step; N′ t Represents updating the candidate hidden state, which is calculated based on the current input and the previous hidden state filtered by the reset gate, and is used to update the current hidden state.

[0133] The comprehensive feature vector input is introduced into the long short-term memory layer of the GRU improved structure to obtain the output C1 (first output).

[0134] In the convolution layer, a one-dimensional convolution kernel is used to process the output of the long short-term memory layer to extract the local features of the time series. The convolution operation performs element multiplication and summation on the time series through a sliding window to capture short-term trends and periodic changes. The specific formula is:

[0135] C2 = ReLU(Conv1D(C1))

[0136] Among them, C2 is the output (second output) after one-dimensional convolution operation and activation function ReLU processing; ReLU is an activation function used to increase the nonlinear characteristics of the network without affecting the propagation of gradients, which helps to alleviate the gradient vanishing problem and accelerate the convergence of the network; Conv1D(C1) represents a one-dimensional convolution operation on C1. The convolution moves along the input data C1 through a sliding window. Each element in the window is multiplied by the corresponding element of the convolution kernel and then summed to extract features; the convolution kernel size is 5×5, so that the convolution layer can capture a wider range of time-dependent features, while also paying attention to some medium-level local features; the step size is set to 2. The step size determines the speed and coverage of the convolution window in the input data. A step size of 2 means that each window slide will skip a data point, which reduces the temporal resolution of the output data, but improves the computational efficiency and reduces the overlap of the output data.

[0137] In the fully connected layer, using the ReLU activation function can accelerate the convergence speed of the neural network and reduce the gradient disappearance problem. The specific formula is:

[0138] C3=ReLU(W2·C2+b2)

[0139] Among them, C3 is the output of the fully connected layer (the third output), which is obtained by applying a weighted sum operation and a ReLU activation function to C2; the ReLU activation function is a nonlinear activation function used to increase the ability of the neural network to handle nonlinear problems; W2 is the weight matrix of the fully connected layer, which is a parameter used to determine the degree of influence of the previous layer nodes on the output of the current layer. Each element in the weight matrix is ​​a trainable parameter and is optimized by the back propagation algorithm during training; W2·C2 represents the dot product of the weight and the input C2; b2 is the bias vector of the fully connected layer, which is added to the result of W2·C2 and is used to adjust the output, add additional degrees of freedom to the financial data prediction model, and better fit the data. After the dot product of the weight and the input, the bias vector is added to adjust the linear offset of the output result, which helps the financial data prediction model to learn and predict complex functional relationships more effectively.

[0140] In the output layer, the tanh function is used to output the prediction results;

[0141] Y pred =tanh(W3·C3+b3)

[0142] Among them, Y predIt is the predicted output of the financial data prediction model, that is, the predicted value. Due to the diversity of the input financial data, the predicted value can be the company's future operating income, profit, and stock price; tanh is an activation function, which is used to add nonlinear factors to the neural network so that the financial data prediction model can capture complex relationships. The output range of the tanh function is [-1, 1], which can provide stronger negative feedback; W3 is the weight of the output layer; C3 is the output of the fully connected layer; b3 is the bias of the neural network, which is used to add additional degrees of freedom to the financial data prediction model to help the financial data prediction model better fit the data.

[0143] In order to obtain more accurate prediction values ​​and reduce prediction errors caused by improper setting of fixed thresholds, the prediction error threshold of the financial data prediction model is adjusted through adaptive thresholds. The prediction error threshold is used as a standard to judge whether the prediction results of the model are within an acceptable range. The specific formula is:

[0144] F n+1 =F n +α(|Y real -Y pred |-F n )

[0145] Among them, F n Indicates the current prediction error threshold at the time of the nth prediction. It is used to determine whether the prediction error is within an acceptable range. It is a dynamically adjusted benchmark point. It is updated after comparing the actual results of each prediction with the predicted results to reflect the latest data characteristics and prediction accuracy. n The initial value of is the forecast output of the financial data forecasting model; n+1 The updated prediction error threshold is adjusted according to the latest error data, aiming to reflect the changing data characteristics and improve the prediction accuracy; α is the adjustment rate parameter, ranging from (0, 1), which controls the sensitivity and speed of the prediction error threshold adjustment. A higher α value makes the prediction error threshold quickly adapt to the latest error state, while a lower α value makes the adjustment process smoother; |Y real -Y pred | is the absolute error between the actual data point and the predicted data point, reflecting the accuracy of the prediction result and used to adjust the prediction error threshold. real -Y pred |>F n+1 , it is considered a false alarm and the prediction error threshold needs to be further adjusted. With the continuous updating of the prediction error threshold, financial data forecasts can more accurately determine what degree of error is acceptable, thereby reducing misjudgments and improving the accuracy of prediction values.

[0146] In summary, the financial data analysis and prediction system and method based on artificial intelligence provided by the embodiment of the present invention does not completely rely on complete and accurate financial record data, thereby improving the accuracy of the analysis results. At the same time, the analysis not only considers historical financial data, but also introduces external factors to improve the accuracy of the prediction. At the same time, a model update and maintenance mechanism is added to ensure long-term use effect.

[0147] The order of the embodiments of the invention is for description only and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0148] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0149] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A financial data analysis and forecasting method based on artificial intelligence, characterized in that: The following steps are involved: S1: Use the financial data parsing model to automatically connect to the financial data source, collect and clean the financial data in the financial data source, and integrate the financial data into a data set as data points after standardization and dynamic text field mapping. The specific implementation of standardization and dynamic text field mapping is as follows: Use the pre-trained model of natural language processing to standardize the text field names first, and then convert each text field name into a high-dimensional semantic vector; The weighted cosine similarity between the semantic vectors is calculated to find the text field that is most similar to the standard text field name and realize dynamic text field mapping; The calculation formula of the weighted cosine similarity is: Among them, Q(u,v,w) represents weighted cosine similarity, which means the similarity between two vectors u and v adjusted by weight factor w; u and v represent two vectors, which are the representations of text field names in a specific vector space; each element u m and v m Represent the components of vectors u and v in the mth dimension respectively; w is a weight vector; w m represents the mth element in the w vector, representing the weight coefficient of the mth dimension in the similarity calculation; ∑(w m ·u m ·v m ) is the calculation of weighted dot product; and The expressions within the square root sign calculate the modulus of the weighted vectors u and v respectively; S2: The lonely forest algorithm is optimized by introducing the sample density factor. The specific implementation process is as follows: S21: Calculate the sample density factor; the sample density factor is normalized and exponentially transformed based on the average distance between the data point and its k nearest neighbors. The specific formula is: Among them, D(x i ) is a measure used to evaluate the local density of data points in the space where they are located; kNN(x i ) represents the average k-nearest neighbor distance of data point x, that is, the average distance between data point x and its k nearest neighbors; d max represents the maximum k nearest neighbor distance of all data points in the data set L; λ represents a positive adjustment parameter; S22: Adjust the logic of building isolated trees; optimize the segmentation logic, and the selection process of segmentation features will be adjusted according to the sample density factor; after determining the segmentation features, select the split value and adjust it according to the sample density factor, and the selected split value should achieve the maximum difference in density of child nodes after segmentation; when building isolated trees, apply the optimized segmentation logic and continuously build trees until all data points are isolated or the maximum depth of the tree is reached; S23: Calculate the adjusted path length; when calculating the isolation index of the data point, introduce the sample density factor to adjust the path length, combine the path length with the sample density factor, and increase the path length in proportion to the inverse of the sample density factor to weaken the influence of the shorter path length in the high-density area; After the above steps, the optimized lonely forest algorithm is obtained. The optimized lonely forest algorithm is based on the data points in the data set and calculates the isolation index s′(x i ,n), automatically set the threshold and mark the abnormal value according to the threshold; the threshold setting formula is: P=μ+l×γ Where P is the threshold; μ is the mean of the isolation index, that is, the average of the isolation index of all data points; l is a coefficient; γ is the standard deviation of the isolation index; Outlier marking rules: If s′(x i ,n)≥P, then the data point with abnormal isolation index is marked as an outlier; The outliers are modified using a linear regression model to obtain modified values, and the modified values ​​and data points not marked as outliers are integrated to obtain an updated data set; S3: The updated data set is standardized to obtain a standardized data set. The specific formula is: Among them, X j ′ represents the standardized data point; X j represents the updated data point; Represents the minimum value of the updated data point; Indicates the maximum value of the updated data point; By combining seasonal adjustment and exogenous variables, the enhanced autoregressive difference moving average model is obtained to analyze the standardized data set. The specific formula is: in, and are the autoregressive polynomial and the moving average polynomial respectively; B is the backoff operator, which moves each data point in the time series to the beginning of the time series by a specified number of time steps; A and a are the orders of seasonal and non-seasonal differences respectively; S is the seasonal period; (X j ′) t is the standardized data point at time t; ε t is the error term at time t; δ is the coefficient vector of the exogenous variables; Z t is the vector of exogenous variables at time t; T is the transpose operator; The standardized data point (X j ′) t Apply seasonal difference operation (1-B S ) a , the specific formula is: (1-B S ) a (X j ′) t =(X j ′) t -(X j ′) t-s Among them, (1-B S ) a is the representation of the seasonal difference operation; (X j ′) t is a data point in the time series, t refers to the time index; B is the back-off operator, which means the operation of moving the data point in the time series forward; S is the seasonal cycle, which means the number of time points covered by the complete seasonal cycle; a is the order of difference, which means the number of times the difference operation needs to be repeated, a=1 means one seasonal difference, if a>1, multiple differences are performed; The statistical summary of each time point after difference is used as the output comprehensive feature vector; S4: The financial data prediction model based on the neural network predicts the comprehensive feature vector to obtain the predicted value, and dynamically adjusts the prediction error threshold in the financial data prediction model through adaptive threshold adjustment.

2. The financial data analysis and forecasting method based on artificial intelligence according to claim 1, characterized in that: In step S1, the financial data sources include an ERP system, an online trading platform, a bank interface and a cloud financial service platform.

3. The financial data analysis and forecasting method based on artificial intelligence according to claim 1, characterized in that: In step S4, the financial data prediction model is a neural network including an input layer, a long short-term memory layer, a convolutional layer, a fully connected layer, and an output layer.

4. The financial data analysis and forecasting method based on artificial intelligence as claimed in claim 3, characterized in that: In step S4, "predicting the comprehensive feature vector based on the financial data prediction model of the neural network to obtain the predicted value" specifically includes: In the long short-term memory layer, an improved structure of a gated recurrent unit (GRU) is introduced, and the comprehensive feature vector input is introduced into the long short-term memory layer of the improved structure of the gated recurrent unit (GRU) to obtain a first output; In the convolution layer, the first output of the long short-term memory layer is processed with a one-dimensional convolution kernel to extract local features of the time series. The convolution operation performs element multiplication and summation on the time series through a sliding window, thereby capturing short-term trends and periodic changes to obtain a second output. In the fully connected layer, the second output is processed using a ReLU activation function to obtain a third output; In the output layer, the third output is processed using a tanh function to obtain the predicted value.

5. The financial data analysis and forecasting method based on artificial intelligence according to claim 1, characterized in that: In step S4, "dynamic adjustment of the prediction error threshold in the financial data prediction model is achieved through adaptive threshold adjustment", the specific formula is: F n+1 =F n +α(|Y real -Y pred |-F n ) Among them, F n represents the current prediction error threshold at the nth prediction, F n The initial value of is the forecast output of the financial data forecasting model; n+1 The updated prediction error threshold is adjusted according to the latest error data; α is the adjustment rate parameter, ranging from (0,1), which controls the sensitivity and speed of the prediction error threshold adjustment; |Y real -Y pred | is the absolute error between the actual data point and the predicted data point.

6. A financial data analysis and forecasting system based on artificial intelligence, applied to the financial data analysis and forecasting method based on artificial intelligence as claimed in claim 1, characterized in that: Includes the following parts: The financial data collection module is used to automatically connect to the financial data source using the financial data analysis model, collect and clean the financial data in the financial data source, and integrate the financial data after standardization and dynamic field mapping as data points into a data set; A financial data outlier detection and modification module is used to calculate an isolation index based on the data set by a lonely forest algorithm, automatically set a threshold and mark outliers, modify the outliers using a linear regression model to obtain a modified value, and integrate the modified value with data points not marked as outliers to obtain an updated data set; A financial data analysis module is used to standardize the updated data set, obtain an enhanced autoregressive difference moving average model by combining seasonal adjustment and exogenous variables, analyze the standardized data set, and output a comprehensive feature vector; A financial data prediction module, used to predict the comprehensive feature vector using a financial data prediction model, obtain a predicted value and output it, and dynamically adjust the prediction error threshold of the financial data prediction model through adaptive threshold adjustment; The user interaction module is used to receive the predicted values ​​and generate detailed reports and charts to display the indicators and trends of data analysis and prediction.

Images