Financial data analysis and prediction method and system based on big data modeling

Abstract

The invention relates to the field of financial data analysis, in particular to a financial data analysis and prediction method and system based on big data modeling. The method comprises the following steps: segmenting a historical financial data sequence to obtain a plurality of sub-segments, and decomposing the sub-segments into a plurality of historical eigenmode components according to empirical mode decomposition for training a preset prediction model; decomposing the real-time financial data sequence into a plurality of eigenmode components according to empirical mode decomposition, and inputting a target component into a corresponding prediction model to obtain a target prediction value; and integrating the target predicted values, traversing to obtain an integrated value of each eigenmode component, and taking an accumulated result of the integrated values of all the eigenmode components as a financial predicted value of the real-time financial data. Through the technical scheme of the invention, the accuracy of the prediction information of the financial data can be improved.

Description

Technical Field

[0001] The present invention relates to the field of financial data analysis, and more specifically, to a financial data analysis and prediction method and system based on big data modeling. Background Art

[0002] As a direct record of financial market activities, financial data covers key information such as stock prices, trading volumes, interest rates, exchange rates, etc. These data are the basis of financial activities such as financial analysis, investment decisions, and risk management. With the rapid development of financial markets and the increasing complexity of financial products, the analysis and prediction of financial data have become particularly important. Traditional financial data analysis methods mainly rely on statistical and econometric models, which are effective in processing small-scale, structured data. However, in the face of massive, multi-source, and real-time updated financial data, traditional methods have limitations in data processing capabilities, analysis depth, and prediction accuracy. The development of big data technology has brought new opportunities for financial data analysis. Through big data analysis technology, large-scale financial data sets can be processed and analyzed, deep information in the data can be mined, and the accuracy and real-time nature of financial predictions can be improved.

[0003] The existing Chinese patent application document with publication number CN114781745A discloses a method, device, and equipment for predicting the deposit business status of an institution. The method obtains time series data within a preset time period; obtains stationary sequence data and residual sequence data; inputs the stationary sequence data into a first prediction sub-model in a prediction model, and outputs a first prediction result; inputs the residual sequence data into a second prediction sub-model in the prediction model, and outputs a second prediction result; inputs the first prediction result and the second prediction result into a result correction sub-model in the prediction model, and outputs prediction information on the deposit business status of a third institution in a target time period.

[0004] However, the above application documents perform differential processing on time series data, but differential processing cannot guarantee the acquisition of stationary sequence data, and the residual sequence usually refers to the difference between the model prediction value and the actual observation value during the model fitting process. The residual sequence directly obtained by the above application documents may also be inaccurate, resulting in inaccurate prediction information for financial data. Summary of the invention

[0005] In order to solve the problem of inaccurate prediction information, the present invention proposes a financial data analysis and prediction method and system based on big data modeling.

[0006] In a first aspect, the present invention discloses a financial data analysis and prediction method based on big data modeling, comprising: segmenting a historical financial data sequence to obtain a plurality of sub-segments, decomposing the sub-segments into a plurality of historical intrinsic mode components according to empirical mode decomposition, for training a preset prediction model, wherein one historical intrinsic mode component corresponds to one prediction model; decomposing a real-time financial data sequence into a plurality of intrinsic mode components according to empirical mode decomposition, taking any intrinsic mode component as a target component, inputting the target component into a corresponding prediction model to obtain a target prediction value; integrating the target prediction value, traversing to obtain the integrated value of each intrinsic mode component, and accumulating the integrated values ​​of all the intrinsic mode components as the financial prediction value of the real-time financial data.

[0007] Since the model is based on the integration of multiple eigenmode components rather than a single time series, this helps reduce the risk of overfitting historical data and improves the stability of the model in practical applications. In the real-time prediction stage, the eigenmode components obtained by decomposing real-time data are input into the corresponding model to obtain the predicted values ​​of each component, and these predicted values ​​are integrated to form a comprehensive prediction of the entire time series, which combines the prediction results of multiple models and improves the accuracy of the prediction, thus providing a comprehensive and detailed perspective for financial decision-making.

[0008] Preferably, the prediction model is a BP neural network, including an input layer, a hidden layer and an output layer; the input layer is used to receive the input of historical eigenmode components, and after the hidden layer extracts features from the input information, the extracted features are input to the output layer to output the value of the historical eigenmode component at the next moment.

[0009] Preferably, the training process of the prediction model includes: taking the historical eigenmode component of any sub-segment as input information, and taking the value of the historical eigenmode component of any sub-segment at the next moment as a label to obtain a set of training data; inputting the input information in the training data into the prediction model to obtain an output result; calculating the loss value of the prediction model through the output result and the label, back-propagating the error signal according to the loss value, updating the network parameters of the prediction model to reduce the loss value; iteratively updating the network parameters of the prediction model, and when the prediction model reaches the set maximum number of training times or the network loss value is less than the set loss value, stopping the update to obtain a trained prediction model.

[0010] The ability to gradually adjust and refine the model's predictive capabilities ensures that the model has higher accuracy in capturing the dynamic changes of time series. The iterative optimization process ensures that the model has good generalization capabilities when facing new data and can provide more accurate financial forecasts.

[0011] Preferably, the integration of the target prediction value satisfies the relationship: , Represents the real-time financial data sequence The integrated value of the eigenmode components, represents a real-time financial data series, Represents real-time financial data series and sub-segments Between The stationary similarity of the eigenmode components, Represents the real-time financial data sequence Eigenmode component input sub-segment The predicted value output by the corresponding prediction model.

[0012] It can take into account the contribution of different sub-segments to the overall trend, thereby more accurately reflecting the dynamic characteristics of real-time financial data, effectively reducing the possible deviations of a single model, and improving the accuracy and robustness of the forecast.

[0013] Preferably, the stationary similarity satisfies the relationship: , Represents real-time financial data series and sub-segments Between The stationary similarity of the eigenmode components, Represents the real-time financial data sequence The stationary probability of the eigenmode components, Sub-segment No. The stationary probability of the eigenmode components, Represents an exponential function.

[0014] It is able to quantify and emphasize sub-segments that are closer in statistical characteristics to real-time data. In the form of an exponential function, as the difference in stationary probability increases, the similarity decreases rapidly, which helps to give higher weight to historical data that better matches the current market conditions when integrating forecasts.

[0015] Preferably, a unit root test is performed on the eigenmode component to obtain the stationary probability of the eigenmode component.

[0016] Preferably, the stationary similarity also includes: calculating the real-time range of the target component of the real-time financial data sequence; calculating the historical range of the sub-segment target component; and taking the normalized Euclidean distance between the real-time range and the historical range as the stationary similarity.

[0017] It can capture the local characteristics of the time series more accurately and improve the accuracy of the prediction.

[0018] In a second aspect, the present invention discloses a financial data analysis and prediction system based on big data modeling, comprising: a processor; and a memory, wherein the memory stores computer instructions, and when the computer instructions are executed by the processor, the system executes the above-mentioned financial data analysis and prediction method based on big data modeling.

[0019] Beneficial effects of the present invention: 1. Empirical mode decomposition can reveal multiple time scale features in the data. Each feature is modeled and predicted separately, which allows each prediction model to focus on a specific frequency component of the data, thereby improving the accuracy of the prediction information. By training a separate prediction model for each eigenmode component, it can ensure that the model better adapts to the nonlinearity and variability of financial data, thereby improving the model's generalization ability on unknown data.

[0020] 2. When processing large-scale financial data, by decomposing complex time series into multiple simple eigenmode components, the predictions of each component can be processed in parallel, thereby improving computational efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein: Figure 1 It is a flowchart of a financial data analysis and prediction method based on big data modeling according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0023] It should be understood that when the terms "first", "second", etc. are used in the claims, descriptions, and drawings of the present invention, they are only used to distinguish different objects, rather than to describe a specific order. The terms "include" and "comprise" used in the description and claims of the present invention indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their collections.

[0024] The present invention provides a financial data analysis and prediction method based on big data modeling. Figure 1 As shown, a financial data analysis and prediction method based on big data modeling includes steps S1 to S3, which are described in detail below.

[0025] S1, segment the historical financial data sequence to obtain multiple sub-segments, and decompose the sub-segments into multiple historical eigenmode components according to empirical mode decomposition for training the preset prediction model.

[0026] In one embodiment, historical financial data sequences are collected through a web crawler. For example, to analyze and predict stocks, the prices of stocks at historical moments are obtained to obtain the historical financial data sequences.

[0027] Linear interpolation is a common method for dealing with missing values. Linear interpolation is based on known data points and estimates the value of the middle point by constructing a straight line segment. This method is simple and can maintain the continuity of the data to a certain extent.

[0028] According to the period of the historical financial data series, the autocorrelation function and the partial autocorrelation function are used to segment. For example, the historical financial data series is , the period of the historical financial data series is 10, and the first sub-segment is , the second sub-segment is , the third sub-segment is , the fourth sub-segment is , the fifth sub-segment is .

[0029] It should be noted that the volatility of historical financial data series is different in different segments. When using historical financial data series to train prediction models, the volatility of historical financial data series has a decisive influence on the performance of the prediction model. The prediction model trained using data with large volatility is better at predicting the volatility pattern of the data, and the prediction model trained using data with small volatility is better at predicting the stationary pattern of the data. Therefore, it is necessary to divide the data into different segments for training separately.

[0030] According to the empirical mode decomposition, the sub-segment is decomposed into multiple historical eigenmode components. After decomposing a sub-segment, multiple historical eigenmode components and an afterwave are obtained. In the empirical mode decomposition, the afterwave refers to the signal part remaining after the original signal is decomposed into a series of eigenmode components. In this application, the afterwave is regarded as a special historical eigenmode component.

[0031] Prediction models are trained for different intrinsic mode components respectively. For example, the prediction models may be an ARIMA model (AutoRegressive Integrated Moving Average model), an LSTM (Long Short-Term Memory) network, and a BP (back propagation) neural network.

[0032] Preferably, the present application uses a BP neural network. The BP neural network includes an input layer, a hidden layer and an output layer; the input layer is used to receive the input of the historical intrinsic mode component, and the hidden layer extracts the features of the input information and inputs the extracted features into the output layer to output the value of the historical intrinsic mode component at the next moment.

[0033] The training process of the prediction model includes: taking the historical intrinsic mode component of any sub-segment as input information, and taking the value of the historical intrinsic mode component of any sub-segment at the next moment as a label, to obtain a set of training data; inputting the input information in the training data into the prediction model to obtain the output result; calculating the loss value of the prediction model through the output result and the label, back-propagating the error signal according to the loss value, updating the network parameters of the prediction model to reduce the loss value; iteratively updating the network parameters of the prediction model, and when the prediction model reaches the set maximum number of training times or the network loss value is less than the set loss value, stop updating to obtain a trained prediction model. Exemplarily, when the number of training times reaches 1000 times or the loss value is less than 0.0001, stop updating.

[0034] At this point, a trained prediction model corresponding to each historical eigenmode component in a subsegment can be obtained.

[0035] S2, decompose the real-time financial data sequence into multiple eigenmode components according to empirical mode decomposition, and input the target component into the corresponding prediction model to obtain the target prediction value.

[0036] In one embodiment, the real-time financial data sequence is decomposed in the same way as in step S1 to obtain multiple eigenmode components, and any eigenmode component is used as a target component.

[0037] The target component is input into the corresponding prediction model to obtain the target prediction value. For example, the real-time financial data sequence is decomposed into 20 intrinsic mode components. The first intrinsic mode component is input into the prediction model trained by the first historical intrinsic mode component in any sub-segment, and the second intrinsic mode component is input into the prediction model trained by the second historical intrinsic mode component in any sub-segment, and so on. The 20 intrinsic mode components of the real-time financial data sequence can all obtain corresponding prediction values.

[0038] If the number of eigenmode components decomposed from the real-time financial data series is greater than the number of historical eigenmode components decomposed from the historical financial data series, the excess ones are not calculated.

[0039] S3, integrating the target prediction value, traversing to obtain the integrated value of each intrinsic mode component, and accumulating the integrated values ​​of all intrinsic mode components as the financial prediction value of the real-time financial data.

[0040] In one embodiment, a unit root test is performed on the eigenmode component to obtain the stationary probability of the eigenmode component.

[0041] The stationary similarity is calculated based on the stationary probability, and the stationary similarity satisfies the relationship: , Represents real-time financial data series and sub-segments Between The stationary similarity of the eigenmode components, Represents the real-time financial data sequence The stationary probability of the eigenmode components, Sub-segment No. The stationary probability of the eigenmode components, Represents an exponential function.

[0042] The above-mentioned stationary similarity is the similarity between the real-time financial data sequence and any sub-segment in the same eigenmode component. Traversing each sub-segment, the target prediction value is integrated to satisfy the relationship: , Represents the real-time financial data sequence The integrated value of the eigenmode components, represents a real-time financial data series, Represents real-time financial data series and sub-segments Between The stationary similarity of the eigenmode components, Represents the real-time financial data sequence Eigenmode component input sub-segment The predicted value output by the corresponding prediction model.

[0043] Exemplarily, the real-time financial data sequence is decomposed into 20 intrinsic mode components, the historical financial data sequence has 8 sub-segments, each sub-segment corresponds to 20 historical intrinsic mode components, and one sub-segment corresponds to 20 prediction models, the first intrinsic mode component of the real-time financial data sequence is sequentially input into the prediction model corresponding to the first historical intrinsic mode component of the 8 sub-segments to obtain the prediction value, that is, one intrinsic mode component corresponds to 8 prediction values, the prediction value is multiplied by the weight and accumulated to obtain the integrated value of the intrinsic mode component. Similarly, the integrated value of the 20 intrinsic mode components is obtained.

[0044] The result of accumulating the integrated values ​​of all eigenmode components is used as the financial prediction value of real-time financial data to complete the financial data analysis and prediction.

[0045] In another embodiment, calculating the stationary similarity further includes: calculating the real-time range of the target component of the real-time financial data sequence and the historical range of the sub-segment target component; and taking the normalized Euclidean distance between the real-time range and the historical range as the stationary similarity.

[0046] An embodiment of the present invention also discloses a financial data analysis and prediction system based on big data modeling, including a processor and a memory, wherein the memory stores computer program instructions. When the computer program instructions are executed by the processor, a financial data analysis and prediction method based on big data modeling according to the present invention is implemented.

[0047] The above system also includes other components well known to those skilled in the art, such as a communication bus and a communication interface, and their configuration and functions are known in the art, so they will not be described in detail here.

[0048] In the present invention, the aforementioned memory may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus or device. For example, a computer-readable storage medium may be any appropriate magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory RRAM (Resistive Random Access Memory), a dynamic random access memory DRAM (Dynamic Random Access Memory), a static random access memory SRAM (Static Random-Access Memory), an enhanced dynamic random access memory EDRAM (Enhanced Dynamic Random Access Memory), a high-bandwidth memory HBM (High-Bandwidth Memory), a hybrid memory cube HMC (Hybrid Memory Cube), etc., or any other medium that can be used to store the required information and can be accessed by an application, a module or both. Any such computer storage medium may be part of a device or accessible or connectable to a device.

[0049] Although this specification has shown and described a number of embodiments of the present invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art will conceive of many modifications, changes and alternatives without departing from the ideas and spirit of the present invention. It should be understood that in the practice of the present invention, various alternatives to the embodiments of the present invention described herein may be employed.

[0050] The above are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A financial data analysis and prediction method based on big data modeling, characterized in that: include: Segmenting the historical financial data sequence to obtain multiple sub-segments, decomposing the sub-segments into multiple historical eigenmode components according to empirical mode decomposition, and using them to train a preset prediction model, wherein one historical eigenmode component corresponds to one prediction model; Decompose the real-time financial data sequence into multiple eigenmode components according to empirical mode decomposition, take any eigenmode component as the target component, and input the target component into the corresponding prediction model to obtain the target prediction value; The target prediction value is integrated, and the integrated value of each intrinsic mode component is traversed to obtain the integrated value. The result of accumulating the integrated values ​​of all intrinsic mode components is used as the financial prediction value of the real-time financial data.

2. The financial data analysis and prediction method based on big data modeling according to claim 1 is characterized in that: The prediction model is a BP neural network, including an input layer, a hidden layer and an output layer; the input layer is used to receive the input of historical intrinsic mode components, and the hidden layer extracts features from the input information and inputs the extracted features into the output layer to output the value of the historical intrinsic mode components at the next moment.

3. The financial data analysis and prediction method based on big data modeling according to claim 1 is characterized in that: The training process of the prediction model includes: The historical eigenmode components of any sub-segment are used as input information, and the values ​​of the historical eigenmode components of any sub-segment at the next moment are used as labels to obtain a set of training data; Input the input information in the training data into the prediction model to obtain the output result; The loss value of the prediction model is calculated by outputting the results and labels, and the error signal is back-propagated according to the loss value to update the network parameters of the prediction model to reduce the loss value. Iteratively update the network parameters of the prediction model. When the prediction model reaches the set maximum number of training times or the network loss value is less than the set loss value, stop updating and obtain a trained prediction model.

4. The financial data analysis and prediction method based on big data modeling according to claim 1 is characterized in that: The integration of the target prediction value satisfies the relationship: , Represents the real-time financial data sequence The integrated value of the eigenmode components, represents a real-time financial data series, Represents real-time financial data series and sub-segments Between The stationary similarity of the eigenmode components, Represents the real-time financial data sequence Eigenmode component input sub-segment The predicted value output by the corresponding prediction model.

5. The financial data analysis and prediction method based on big data modeling according to claim 4 is characterized in that: The stationary similarity satisfies the relationship: , Represents real-time financial data series and sub-segments Between The stationary similarity of the eigenmode components, Represents the real-time financial data sequence The stationary probability of the eigenmode components, Sub-segment No. The stationary probability of the eigenmode components, Represents an exponential function.

6. The financial data analysis and prediction method based on big data modeling according to claim 5 is characterized in that: The unit root test of the eigenmode component is performed to obtain the stationary probability of the eigenmode component.

7. The financial data analysis and prediction method based on big data modeling according to claim 4 is characterized in that: The stationary similarity also includes: Calculate the real-time range of the target component of the real-time financial data series; Calculate the historical range of the sub-segment target component; The normalized Euclidean distance between the real-time range and the historical range is used as the stationary similarity.

8. A financial data analysis and prediction system based on big data modeling, characterized in that: include: Processor; and A memory storing computer instructions, which, when executed by a processor, enables the system to execute a financial data analysis and prediction method based on big data modeling according to any one of claims 1 to 7.

Images