A method for stock time series processing and prediction

Abstract

The present invention relates to a method for stock time series processing and prediction, and belongs to the technical field of stock data processing and prediction. The method comprises the steps of: S1, acquiring and checking data, collecting daily market data of target stocks and checking the integrity and accuracy thereof; S2, data stabilization, using a stabilization method to perform stabilization processing on stock data; S3, data standardization and division, dividing the data into a training set and a validation set after standardization; S4, data prediction, by constructing an LSTM model, inputting the pre-processed stock data into the model for prediction. The present invention applies data stabilization technology to the task of stock trend prediction, and proposes a stock prediction method based on the model, which not only improves the prediction accuracy, but also improves the robustness and consistency of the model.

Description

Technical Field

[0001] The present invention belongs to the technical field of stock data processing and prediction, and more specifically, relates to a method for stock time series processing and prediction. Background Art

[0002] In the modern stock market, stocks, as an important investment tool, have attracted the participation of a large number of individual and institutional investors. However, since stock prices are affected by many factors, such as macroeconomic conditions, market supply and demand, company performance, and industry policies, they are highly volatile and difficult to predict. In order to obtain higher returns, investors have an urgent need for accurate stock price prediction methods. Against the backdrop of the rapid development of big data, artificial intelligence, and machine learning technologies, the predictability of stock prices has been widely recognized by academia. In this context, researchers are focusing on developing optimal nonlinear models to predict future stock price trends.

[0003] Data in the stock market has significant random volatility and noise, which poses great challenges to prediction. Existing methods perform poorly in dealing with such noise, which may lead to overfitting of the model or over-reliance on short-term data, thereby reducing prediction accuracy.

[0004] Chinese Patent CN115907093A: A stock trend analysis method based on price fluctuation data, which directly decomposes stock data through EMD, is prone to inaccurate decomposition and modal aliasing. When dealing with stock data with complex fluctuation trends and difficult features to analyze, inaccurate decomposition and modal aliasing are prone to problems. EMD is prone to introduce errors when processing the two ends of the time series, resulting in inaccurate decomposition results at both ends of the series.

[0005] Chinese patent CN118245829A: A stock clustering method based on autoencoder and neighbor propagation algorithm. The neighbor propagation algorithm relies on similarity measurement when clustering. For complex nonlinear data, how to reasonably select the similarity measurement method and the number of neighbors may affect the clustering effect. If the similarity measurement is inaccurate or the number of neighbors is not properly selected, the clustering result may be distorted.

[0006] The methods mentioned in the above patents still have many problems when processing long-term stock data series. First, stock data usually contains noise and outliers, which makes it difficult for the model to accurately capture effective trends; second, stock prices have long-term dependencies, but many traditional models have difficulty handling dependencies over long time spans. Stock market data is usually non-stationary, and how to deal with information loss in the stabilization process is also a major problem. Complex models are prone to overfitting historical data, resulting in poor generalization capabilities. During the training process, the model may fall into a local optimal solution, making the final result unsatisfactory. These defects limit their effectiveness in practical applications and urgently need to be improved and optimized.

[0007] In this context, this patent proposes a method that combines stabilization of stock data and LSTM model, which can greatly improve the prediction effect of stock data. Stabilization can make time series data more stable, eliminate unnecessary fluctuations, reduce noise in the model training process, make the training process of LSTM model smoother, and help accelerate the convergence of the model. LSTM performs well in processing time series data with complex dependencies. After stabilization, LSTM can more effectively capture the time dependency in the data and improve the stability and accuracy of the prediction. Summary of the invention

[0007] The present invention aims to solve the following problems: Due to the time series and high-frequency volatility of stock data, the complexity of model processing is greatly increased, which brings great challenges to accurate prediction. Traditional methods are insufficient in processing the complexity and nonlinear characteristics of stock data. Therefore, the present invention proposes a method for stock time series processing and prediction to improve the accuracy and effectiveness of stock data analysis and prediction.

[0008] To solve the above problems, according to one aspect of the present invention, a method for stock time series processing and prediction is provided, which includes the following processing methods: S1, obtaining and checking data; S2, data stabilization; S3, data standardization and division; S4, data prediction.

[0009] To achieve the above object, the present invention adopts the following technical solutions:

[0010] Determine the target stock and time range. First, you need to select the target stock, which is usually determined by the analysis requirements or investment strategy. Use the stock's unique identifier (such as the stock code) to identify the target stock. The time range can be selected based on the needs of historical data, usually including a start date and an end date, to ensure that the data covers a long enough time span for predictive analysis.

[0011] Data inspection: After obtaining stock data, it must be strictly inspected and cleaned to ensure its quality. The first step in data inspection is to check whether there are null values, which may be caused by missing data or crawler acquisition failure. For null values, a common processing method is to fill them with statistical methods such as the average of adjacent data and historical data trends to ensure the continuity and consistency of the data. Outliers refer to data points that deviate significantly from other values ​​in a data set, which may be caused by data input errors, crawler parsing problems, etc. Outliers need to be further analyzed to determine their rationality and source, and erroneous data should be replaced or deleted.

[0012] Data stabilization: The weighted average and VMD algorithms are used successively to stabilize the stock data. Since the weighted average denoising algorithm does not have too many parameter adjustments, it will not introduce too much complexity in many cases and is suitable for avoiding errors caused by over-fitting of data. This is especially important in applications such as signal processing and time series prediction. The VMD algorithm can adaptively decompose signals and decompose complex signals into several sub-modes with narrowband characteristics. This adaptability allows VMD to adapt to various signal types and extract information in different frequency bands without the need to preset specific parameters. The stock data processed by the two algorithms is more stable and more conducive to model prediction. The S2 step includes the following steps: S21 weighted average denoising: stock data sequence x = [x1, x2, ..., x n ], the sliding window size of weighted denoising is v (generally chosen as an odd number for symmetry), and the weight in the window is m = [m1, m2, ...m n ]. The value y of data point i after weighted average denoising i It is expressed as: in, is the offset of the window center relative to the two ends, y i is the value of the i-th data point after denoising. i is the value of the i-th data point in the original data sequence. S22 Construction of variational framework: The goal of VMD is to decompose the input stock data f(t) into several intrinsic mode functions (IMFs), each of which is represented by u k (t), where k represents the mode number. The goal of VMD is to make each mode u k (t) Focused on its center frequency w k to minimize its bandwidth. In the frequency domain, the bandwidth can be characterized by the gradient of the mode. In order to optimize the decomposition of the signal, VMD formulates the decomposition problem as a constrained variational optimization problem: Among them, * is the convolution operation; is a Hilbert transform kernel that generates an analytic signal. The modal u k Move to base frequency w k superior. S23 constraints: To ensure the consistency of the decomposition of the modal function, VMD introduces the following constraints: This constraint states that the sum of all modes should be equal to the original signal f(t). S24 Constrained Optimization: In order to solve the optimization problem with constraints, VMD uses the Lagrange multiplier method. The Lagrange function is constructed as follows: S25 alternating direction multiplier method: VMD optimizes the Lagrangian function by alternating direction multiplier method. It mainly includes the following steps: 1) Update modal u k At fixed w k and λ, for the modal function u k Update. The update formula is: in, Indicates u k (t) Representation in the frequency domain; is the frequency domain representation of the signal f(t). is the representation of Lagrange multipliers in the frequency domain. 2) Update the center frequency W k In updating u k Then, update the center frequency W k , the formula is: By weighted averaging in the frequency domain, the center frequency W of each mode is obtained. k . 3) Update the Lagrange multiplier λ Update the Lagrange multiplier λ to enhance convergence, the formula is: Among them, τ is the step size parameter used to control the convergence speed S26 VMD algorithm iteration: By alternating the update u in each iteration k、 w k and λ, the VMD algorithm continuously optimizes the result of signal decomposition until the convergence condition is met, that is, the iteration is terminated when the following conditions are met: Among them, ∈ is a preset convergence threshold, which is used to control the accuracy of iteration stopping. S27 eigenmode addition: After f(t) is decomposed, several eigenmode functions are added together to obtain the stabilized data y: y=u1(t)+u2(t)+…+u k (t)

[0013] Dataset standardization and partitioning: The data is scaled to fall into a specific range to ensure stability and effectiveness when the data is input into the model. The processed data set is then divided into a training set and a validation set.

[0016] Model training and prediction: Build and train a stock prediction model based on LSTM architecture. The model includes cell state, hidden state, and gating mechanism.

[0018] The present invention effectively solves the shortcomings of traditional methods in dealing with the complexity and nonlinear characteristics of stock data by introducing stabilization processing. Traditional methods usually directly use raw data for modeling, which makes it difficult for the model to capture the effective characteristics of the data, thereby reducing the prediction accuracy. In contrast, stabilization makes the data more stable, thereby improving the model's ability to identify key features and prediction accuracy. Stabilization can also reduce computational complexity, making the model easier to converge, while reducing noise interference caused by short-term fluctuations, making the prediction results more reliable and stable. BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 A flowchart of the method for stabilizing stock data and predicting the stock data according to the present invention; Figure 2 Flowchart for stock data preprocessing; Figure 3 This is a diagram showing the effect of using the weighted average algorithm to process data; Figure 4 This is the effect diagram of VMD algorithm processing data; DETAILED DESCRIPTION In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical or scientific terms used herein shall have the common meanings understood by persons with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the patent application specification and claims of the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as "one" or "a" do not indicate a quantity limitation, but indicate the existence of at least one. like Figure 1 As shown, the present invention provides a method for stock time series processing and prediction, and the specific steps are as follows: S1. Obtain and check data: collect daily market data of target stocks and check its completeness and accuracy; S2. Data stabilization: Use stabilization methods to stabilize stock data. For example, the weighted average and VMD algorithms used in this patent are not limited to these two methods; S3, data standardization and division, the data is divided into training set and validation set after standardization; S4, data prediction, by building an LSTM model, the pre-processed stock data is input into the model for prediction; Specifically, the method for processing stock data and predicting based on stabilization provided in this embodiment includes the following steps: S1. Data acquisition and inspection: acquiring the daily market data of the target stock and inspecting the obtained data. Specifically, Figure 2 As shown in the figure, determine the target stocks to be analyzed and their time range, so that subsequent data collection and processing are more targeted. Then, use the crawler tool to obtain the historical data of the target stock from the specified data source, including information such as opening price, closing price, highest price, lowest price and trading volume. Next, check and clean the acquired data to remove outliers, missing values ​​and duplicate data to ensure the accuracy and consistency of the data, thereby improving data quality. Finally, save the cleaned data to a database or file for subsequent analysis and model training. S2, data stabilization, is to use weighted average and VMD algorithms to stabilize stock data. The S2 step includes the following steps: S21 weighted average denoising: stock data sequence x = [x1, x2, ..., x n ], the sliding window size of weighted denoising is v (generally chosen as an odd number for symmetry), and the weights in the window are m = [m1, m2, ..., m n ]. The value y of data point i after weighted average denoising i It is expressed as: in, is the offset of the window center relative to the two ends, y i is the value of the i-th data point after denoising. i is the value of the i-th data point in the original data sequence. S22 Construction of variational framework: The goal of VMD is to decompose the input stock data f(t) into several intrinsic mode functions (IMFs), each of which is represented by u k (t), where k represents the mode number. The goal of VMD is to make each mode u k (t) Focused on its center frequency w k to minimize its bandwidth. In the frequency domain, the bandwidth can be characterized by the gradient of the mode. In order to optimize the decomposition of the signal, VMD formulates the decomposition problem as a constrained variational optimization problem: Among them, * is the convolution operation; is a Hilbert transform kernel that generates an analytic signal. The modal u k Move to base frequency w k superior. S23 constraints: To ensure the consistency of the decomposition of the modal function, VMD introduces the following constraints: This constraint states that the sum of all modes should be equal to the original signal f(t). S24 Constrained Optimization: In order to solve the optimization problem with constraints, VMD uses the Lagrange multiplier method. The Lagrange function is constructed as follows: S25 alternating direction multiplier method: VMD optimizes the Lagrangian function by alternating direction multiplier method. It mainly includes the following steps: 1) Update modal u k At fixed w k and λ, for the modal function u k Update. The update formula is: in, Indicates u k (t) Representation in the frequency domain; is the frequency domain representation of the signal f(t). is the representation of Lagrange multipliers in the frequency domain. 2) Update the center frequency W k In updating uk Then, update the center frequency W k , the formula is: By weighted averaging in the frequency domain, the center frequency W of each mode is obtained. k . 3) Update the Lagrange multiplier λ Update the Lagrange multiplier λ to enhance convergence, the formula is: Among them, τ is the step size parameter used to control the convergence speed S26 VMD algorithm iteration: By alternating the update u in each iteration k 、w k and λ, the VMD algorithm continuously optimizes the result of signal decomposition until the convergence condition is met, that is, the iteration is terminated when the following conditions are met: Among them, ∈ is a preset convergence threshold, which is used to control the accuracy of iteration stopping. S27 eigenmode addition: After f(t) is decomposed, several eigenmode functions are added together to obtain the stabilized data y: y=u1(t)+u2(t)+…+u k (t) S3, data standardization and division, the data is divided into training set and validation set after standardization; S4. Data prediction: By building an LSTM model, the preprocessed stock data is input into the model for prediction. The above description is merely an exemplary embodiment of the present invention and is not intended to limit the protection scope of the present invention. The protection scope of the present invention is determined by the appended claims.

Claims

1. A method for stock time series processing and prediction, characterized by , including the following steps; S1. Obtain and check data: collect daily market data of target stocks and check its completeness and accuracy; S2. Data stabilization: Use stabilization methods to stabilize stock data. For example, the weighted average and VMD algorithms used in this patent are not limited to these two methods; S3, data standardization and division, the data is divided into training set and validation set after standardization; S4. Data prediction: By building an LSTM model, the preprocessed stock data is input into the model for prediction.

2. A method for stock time series processing and prediction as claimed in claim 1, characterized in that: The S1 step includes the following steps: S11: Determine the target stock and time range. First, you need to select the target stock, which is usually determined by the analysis requirements or investment strategy. Use the stock's unique identifier (such as the stock code) to confirm the target stock. The time range can be selected based on the needs of historical data, usually including a start date and an end date, to ensure that the data covers a long enough time span for predictive analysis. S12: Set up a crawler tool to obtain data. Once the target stock and time range are determined, the data acquisition process is automated by writing and configuring a crawler program. The crawler program obtains the historical trading data of the target stock from the specified URL by sending an HTTP request. The crawler's task is to extract the stock's daily market data from the response. S13: Data inspection. After obtaining the stock data, it must be strictly inspected and cleaned to ensure its quality. The first step of data inspection is to check whether there are null values, which may be caused by missing data or crawler acquisition failure. For null values, the processing method is to fill them with statistical methods such as the average value of adjacent data and historical data trends to ensure the continuity and consistency of the data. Outliers refer to data points that deviate significantly from other values ​​in the data set, which may be caused by data input errors, crawler parsing problems, etc. Outliers need to be further analyzed to determine their rationality and source, and erroneous data should be replaced or deleted. S14: Save data. After cleaning, the stock data needs to be saved in a reliable storage location. You can choose to save the data in a local file, database, or cloud storage. The choice of storage medium depends on the subsequent data usage requirements. This step not only ensures that the data can be easily called in subsequent prediction tasks, but also provides guarantees for the traceability and long-term preservation of the data. When saving data, you need to consider the choice of data format, such as CSV, SQL, Excel, etc., to facilitate compatibility with different platforms and systems. Through the above steps, we can ensure that the acquired stock data has high accuracy and completeness, thereby providing reliable data support for subsequent model training and prediction tasks.

3. A method for stock time series processing and prediction as claimed in claim 1, characterized in that: The data stabilization described in S2 is to use weighted average and VMD algorithms to stabilize the stock data. Since the weighted average denoising algorithm does not have too many parameter adjustments, it will not introduce too much complexity in many cases and is suitable for avoiding errors caused by over-fitting of data. This is especially important in applications such as signal processing and time series prediction. The VMD algorithm can adaptively decompose signals and decompose complex signals into several sub-modes with narrowband characteristics. This adaptability allows VMD to adapt to various signal types and extract information in different frequency bands without the need to preset specific parameters. The stock data processed by the two algorithms is more stable and more conducive to model prediction. The S2 step includes the following steps: S21 weighted average denoising: stock data sequence x = [x1, x2, ..., x n ], the sliding window size of weighted denoising is v (generally chosen as an odd number for symmetry), and the weight in the window is m = [m1, m2, ..., m n ]. The value y of data point i after weighted average denoising i It is expressed as: in, is the offset of the window center relative to the two ends, y i is the value of the i-th data point after denoising. i is the value of the i-th data point in the original data sequence. S22 Construction of variational framework: The goal of VMD is to decompose the input stock data f(t) into several intrinsic mode functions (IMFs), each of which is represented by u k (t), where k represents the mode number. The goal of VMD is to make each mode u k (t) Focused on its center frequency w k to minimize its bandwidth. In the frequency domain, the bandwidth can be characterized by the gradient of the mode. In order to optimize the decomposition of the signal, VMD formulates the decomposition problem as a constrained variational optimization problem: Among them, * is the convolution operation; is a Hilbert transform kernel that generates an analytic signal. The modal u k Move to base frequency w k superior. S23 constraints: To ensure the consistency of the decomposition of the modal function, VMD introduces the following constraints: This constraint states that the sum of all modes should be equal to the original signal f(y). S24 Constrained Optimization: In order to solve the optimization problem with constraints, VMD uses the Lagrange multiplier method. The Lagrange function is constructed as follows: S25 alternating direction multiplier method: VMD optimizes the Lagrangian function by alternating direction multiplier method. It mainly includes the following steps: 1) Update modal u k At fixed w k and λ, for the modal function u k Update. The update formula is: in, Indicates u k (t) Representation in the frequency domain; is the frequency domain representation of the signal f(t). is the representation of Lagrange multipliers in the frequency domain. 2) Update the center frequency w k In updating u k Then, update the center frequency w k , the formula is: By weighted averaging in the frequency domain, the center frequency w of each mode is obtained. k . 3) Update the Lagrange multiplier λ Update the Lagrange multiplier λ to enhance convergence, the formula is: Among them, τ is the step size parameter used to control the convergence speed S26 VMD algorithm iteration: By alternating the update u in each iteration k 、w k and λ, the VMD algorithm continuously optimizes the result of signal decomposition until the convergence condition is met, that is, the iteration is terminated when the following conditions are met: Among them, ∈ is a preset convergence threshold, which is used to control the accuracy of iteration stopping. S27 eigenmode addition: After f(t) is decomposed, several eigenmode functions are added together to obtain the stabilized data y: y=u1(t)+u2(t)+…+u k (t) 4. A method for stock time series processing and prediction as claimed in claim 1, characterized in that: In S3, the standardization method is to scale the data so that it falls into a specific range to ensure the stability and effectiveness of the data when it is input into the model. The processed data set is then divided into a training set and a validation set.

5. A method for stock time series processing and prediction as claimed in claim 1, characterized in that: The basic structure of the LSTM model includes: Cell State: used to store "memory" of long-term dependent information, which is transferred and updated between time steps; Hidden State: used to output the information of the current time step and participate in the calculation of the next time step; Gate mechanism: including input gate, forget gate and output gate, which are used to control the storage, discard and output of information.

6. A method for stock time series processing and prediction according to claim 1, characterized in that: The method also includes a result evaluation step, analyzing the sensitivity of the model to changes in input data, checking the prediction effect of the model at different time periods, ensuring that the model performs stably on new data, and using root mean square error (RMSE) and coefficient of determination (R 2 )) and other indicators to evaluate the prediction results in order to fully verify the performance and practicality of the smoothed stock prediction model.

Images