Data processing method, device and equipment based on LSTM model and readable storage medium
Through the data processing method based on the LSTM model, the characteristic yield in the quantitative investment model is predicted, and the problem of lag in the calculation of traditional Alpha indicators is solved, and more accurate and real-time investment strategy decision support is achieved.
Patent Information
- Application Number
- CN202411527352.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-30
- Publication Date
- 2025-02-14
AI Technical Summary
The traditional Alpha indicator calculation has a lag and it is difficult to assist in the decision-making of quantitative investment models in real time.
The data processing method based on the LSTM model is adopted to calculate the characteristic yield in the target data and use factor data to train the LSTM model with advanced characteristics to predict the characteristic yield in the next period, thereby obtaining the characteristic yield improvement indicator.
Real-time prediction of characteristic yields is achieved, which reduces lag and provides more accurate investment strategy decision support.
Smart Images
Figure CN119444429A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a data processing method, device, equipment and readable storage medium based on an LSTM model. Background Art
[0002] Quantitative investment models use technology and knowledge in mathematics, statistics, computer science and other fields to mine investment opportunities from data in order to obtain excess returns or transform investment concepts into mathematical models. Compared with traditional investment, quantitative investment can quickly obtain and process large amounts of data while being unaffected by emotions and maintaining consistency. Therefore, it is a very effective investment method with broad application prospects.
[0003] Idiosyncratic returns (Alpha indicators) mainly refer to the part of individual stock returns that cannot be explained by market returns or industry returns, that is, the "idiosyncratic" part of individual stocks. This return reflects the unique performance of individual stocks or assets relative to the market or industry. Therefore, in order to build a quantitative investment model, it is necessary to calculate the idiosyncratic returns (Alpha indicators). It is worth noting that the traditional Alpha indicator is relatively simple to calculate, easy to understand and has high accuracy, but there is a certain lag, which is an inevitable disadvantage. . Summary of the invention
[0004] The purpose of the present invention is to provide a data processing method, device, equipment and readable storage medium based on the LSTM model. The present invention processes the target data to obtain a characteristic yield improvement index to assist in making strategic decisions.
[0005] The technical solution of the present invention: a data processing method based on an LSTM model comprises the following steps:
[0006] Step 1: Calculate the idiosyncratic rate of return of the target data in period t based on the pricing model; the formula for calculating the idiosyncratic rate of return of the pricing model is as follows:
[0007]
[0008] in, α is the idiosyncratic rate of return in period t, is the actual rate of return in period t, is the risk-free interest rate in period t, is the beta value at period t, is the expected rate of return in period t;
[0009] Step 2: Obtain factor data related to idiosyncratic returns, and perform data preprocessing on the factor data and idiosyncratic returns;
[0010] Step 3: Take the factor data as input features and the trait rate of return as output targets, and obtain an LSTM model with advanced characteristics after training;
[0011] Step 4: Output the idiosyncratic yield of the t+1th period based on the LSTM model with advanced characteristics, and obtain the idiosyncratic yield improvement index based on the idiosyncratic yield of the tth period and the idiosyncratic yield of the t+1th period.
[0012] In the above-mentioned data processing method based on the LSTM model, in step 1, the factor data includes Alpha factors, volatility index factors, market situation index factors and financial index factors.
[0013] In the aforementioned data processing method based on the LSTM model, in step 2, data preprocessing is to remove missing values and then use the StandardScaler() function for standardization.
[0014] In the aforementioned data processing method based on the LSTM model, in step 2, the LSTM model includes 50 hidden units and a fully connected output layer, and uses the Adam optimizer and the mean square error loss function.
[0015] In the aforementioned data processing method based on the LSTM model, in step 3, the LSTM model with advanced characteristics is set as follows:
[0016]
[0017] in, represents the output characteristic rate of return in period t+1, α t is the characteristic rate of return in period t, F i are other factors, k and b are correlation coefficients.
[0018] In the aforementioned data processing method based on the LSTM model, the characteristic yield improvement index is to output the characteristic yield of the next period minus the characteristic yield of the current period:
[0019]
[0020] in, represents the characteristic yield improvement index of period t, represents the characteristic rate of return in period t+1, α t is the characteristic rate of return in period t.
[0021] A device for a data processing method based on an LSTM model, the device comprising:
[0022] A data acquisition module is used to acquire target data through crawler data;
[0023] A pricing model module, used to calculate the target data to obtain the idiosyncratic rate of return and factor data related to the idiosyncratic rate of return;
[0024] An LSTM module is trained using factor data and the idiosyncratic rate of return output by the pricing model module to output the idiosyncratic rate of return for the t+1th period;
[0025] The indicator acquisition module is used to calculate the characteristic yield improvement index based on the characteristic yield of the t+1th period output by the LSTM module.
[0026] An electronic device, comprising:
[0027] at least one processor; and,
[0028] a memory communicatively connected to the at least one processor; wherein,
[0029] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the steps of the data processing method based on the LSTM model as described in any one of claims 1 to 7.
[0030] A computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the data processing method based on the LSTM model as described in any one of claims 1 to 7 is implemented.
[0031] Compared with the prior art, the data processing method of the present invention aims to obtain the characteristic yield improvement index. The construction method of this index has the following two advantages: First, the characteristic yield prediction used does not need to consider the market factor compared with the general yield prediction, and only focuses on the characteristics of the company itself. Second, the previous indicators only focus on the current Alpha and ignore the relevant predictions. The characteristic yield improvement index obtained by the present invention based on the LSTM model makes up for these shortcomings and can better assist investment. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 It is a schematic diagram of the process of the present invention;
[0033] Figure 2 It is the historical retracement chart of the characteristic yield enhancement indicator verification strategy of the present invention;
[0034] Figure 3 This is a Brinson analysis result diagram of the characteristic yield improvement index verification strategy of the present invention;
[0035] Figure 4 is a schematic diagram of the device of the present invention;
[0036] Figure 5 Schematic diagram of an electronic device of the present invention. DETAILED DESCRIPTION
[0037] The present invention is further described below in conjunction with the accompanying drawings and embodiments, but they are not intended to limit the present invention.
[0038] Embodiment: A data processing method based on LSTM model, such as Figure 1 The method comprises the following steps:
[0039] Step 1: Using monthly stock data, with CSI 500 representing market risk, the Alpha value in the model is calculated based on the capital asset pricing model (CAPM). The Alpha value represents the idiosyncratic return of the stock after eliminating market risk. This embodiment uses the CSI 500 index constituent stocks as the stock pool, and selects monthly data for a total of 5 years from January 1, 2019 to December 31, 2023 as the data sample, and the data source is SuperMind. The formula for calculating the idiosyncratic return of the pricing model is as follows:
[0040]
[0041] in, α is the idiosyncratic rate of return in period t, is the actual rate of return in period t, is the risk-free interest rate in period t, is the beta value at period t, is the expected rate of return in period t.
[0042] Step 2: Use monthly stock data to calculate the factor data that affects the stock's idiosyncratic rate of return. Because the future Alpha value is not only related to the company's historical Alpha value, but also to the company's market and its basic characteristics. Therefore, the present invention will construct a total of 18 factors based on three major indicators: volatility indicators, market indicators, and financial indicators. The specific factor construction is shown in Table 1.
[0043]
[0044]
[0045] Table 1
[0046] Then, in the data preprocessing stage, ST and suspended stocks were removed, missing values were removed, and the StandardScaler() function was used for standardization. The StandardScaler() function calculates the mean and standard deviation of the training set and independently standardizes each factor data. Then, the calculated mean and standard deviation are stored so that the same proportion can be used for standardization on the test set.
[0047] Step 3: Construct a long short-term memory (LSTM) neural network. Take the factor data of the tth period as the input feature and the Alpha value of the t+1th period as the output target. After training, a trait yield prediction model with advanced characteristics is obtained. The prediction model is set as follows:
[0048]
[0049] in, Represents the predicted Alpha for the t+1 period, α t is the historical Alpha value, F i are other factors, and the rest are correlation coefficients.
[0050] The model used in this embodiment is specifically an LSTM neural network. LSTM is a variant of a recurrent neural network (RNN) that is specifically designed to solve the modeling and prediction problems of sequence data. Compared with traditional RNN, LSTM can better handle long-term dependencies and control the flow and retention of information through gating mechanisms (such as input gates, forget gates, and output gates), thereby effectively learning and memorizing long-term patterns and dependencies in sequences. The following are the main formulas and gating mechanisms of LSTM:
[0051] Forget gate: determines how much memory of the previous moment is retained at the current moment;
[0052] f t =σ(W f ·[h t-1 , x t ]+b f );
[0053] Among them, f t is the output of the forget gate, σ is the activation function (usually sigmoid), W f and b f are the weight matrix and bias, h t-1 is the hidden state at the previous moment, x t is the input at the current moment.
[0054] Input gate: determines how much of the current input information is stored in the memory unit;
[0055] i t =σ(W i ·[h t-1 , x t ]+b i );
[0056] Among them, i t is the output of the input gate.
[0057] Candidate memory unit: Generate new candidate memory content;
[0058]
[0059] in, is the output of the candidate memory unit, and tanh is the hyperbolic tangent function.
[0060] Memory unit update: update the state of the memory unit;
[0061]
[0062] Among them, C t-1 is the state of the memory unit at the previous moment, C t is the state of the memory unit at the current moment.
[0063] Output gate: determines the hidden state of the current output;
[0064] o t =σ(W o ·[h t-1 , x t ]+b o );
[0065] Among them, t is the output of the output gate.
[0066] Hidden state update: calculate the hidden state at the current moment;
[0067] h t =o t *tanh(C t );
[0068] Among them, h t is the hidden state at the current moment.
[0069] In summary, LSTM uses these gating mechanisms to finely control the flow and retention of information, thereby effectively capturing long-term dependencies in time series.
[0070] After the model is built, the model index parameters are determined and optimized. The final confirmed model index parameters are shown in Table 3:
[0071] Hidden Units Output Layer Optimizer Loss Function MSE_train MSE_test 50 1 Adam MSE 0.0290 0.0321
[0072] Table 3
[0073] By training the model (the training cycle is 100 epochs, each batch size is 500), the MSE on the training set is reduced from 0.0821 to 0.0290, and the MSE on the test set is 0.0321, indicating that its fitting and prediction effects are good.
[0074] Step 4: Output the idiosyncratic yield of the t+1th period based on the LSTM model with advanced characteristics, and obtain the idiosyncratic yield improvement index based on the idiosyncratic yield of the tth period and the idiosyncratic yield of the t+1th period. The idiosyncratic yield improvement index is the predicted Alpha value minus the current Alpha value:
[0075]
[0076] in, represents the characteristic yield improvement index of period t, Represents the predicted Alpha for the t+1 period, α t is the Alpha value of period t.
[0077] In order to verify the effectiveness of the stock idiosyncratic yield enhancement index with advanced characteristics constructed by the strategy of the present invention, stocks with idiosyncratic yield enhancement index greater than 0 and current Alpha greater than 0 are selected for backtesting comparison on the SuperMind platform. The backtesting period is from January 1, 2023 to December 31, 2023. The initial capital is set to 1,000,000, the transaction fee is 0.02%, and the stock transaction slippage is 0.5%.
[0078] Figure 2 This is the historical retracement chart of the characteristic yield improvement indicator of the present invention. In the historical data, the overall retracement of the indicator is small, no more than 10%, and the overall volatility is not large. This shows that the indicator has a low risk level, the overall loss will not be too large, and it has good stability.
[0079] Figure 3 This is the Brinson analysis result diagram of the characteristic yield improvement index of the present invention. Figure 3 It can be seen that the active return of this indicator is 6.91%, of which the position-related return accounts for a large proportion. Among the position-related returns, the allocation return and management return performed well, which shows that the ability of position management and internal stock management is relatively good. In terms of management return, the return of individual stock selection is the most significant, followed by the return of industry selection, which shows that this indicator has a strong stock selection ability and fully reflects the core of the Alpha indicator.
[0080] The data processing method of the present invention aims to obtain the characteristic yield improvement index. The construction method of the index has the following two advantages: First, the characteristic yield prediction used does not need to consider the market factor compared with the general yield prediction, and only focuses on the characteristics of the company itself. Second, the previous indicators only focus on the current Alpha and ignore the relevant predictions. The characteristic yield improvement index obtained by the present invention based on the LSTM model makes up for these shortcomings and can better assist investment.
[0081] like Figure 4 The figure shows a functional module diagram of the data processing device based on LSTM of the present invention.
[0082] The LSTM-based data processing device 100 of the present invention can be installed in an electronic device. According to the functions to be implemented, the LSTM-based data processing device can include: a data acquisition module 101, a pricing model module 102, an LSTM module 103 and an indicator acquisition module. The module of the present invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.
[0083] In this embodiment, the functions of each module / unit are as follows:
[0084] The data acquisition module 101 is used to acquire target data through crawler data;
[0085] A pricing model module 102, for calculating the target data to obtain the idiosyncratic rate of return and factor data related to the idiosyncratic rate of return;
[0086] The LSTM module 103 is trained using the factor data and the idiosyncratic rate of return output by the pricing model module to output the idiosyncratic rate of return for the t+1th period;
[0087] The indicator acquisition module 104 is used to calculate the characteristic yield improvement indicator according to the characteristic yield of the t+1th period output by the LSTM module.
[0088] like Figure 5 , which is a schematic diagram of the structure of an electronic device for implementing the LSTM-based data processing method of the present invention.
[0089] The electronic device 1 may include a processor 10, a memory 11 and a bus, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as a LSTM-based data processing program 12.
[0090] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. The memory 11 may be an internal storage unit of the electronic device 1 in some embodiments, such as a mobile hard disk of the electronic device 1. The memory 11 may also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 1. Further, the memory 11 may also include both an internal storage unit of the electronic device 1 and an external storage device. The memory 11 may not only be used to store application software and various types of data installed in the electronic device 1, such as the code of the data audit program, etc., but also be used to temporarily store data that has been output or is to be output.
[0091] The processor 10 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips, etc. The processor 10 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes or executes programs or modules (such as data auditing programs, etc.) stored in the memory 11, and calls data stored in the memory 11, so as to execute various functions of the electronic device 1 and process data.
[0092] The bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory 11 and at least one processor 10, etc.
[0093] Figure 5 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 4 The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.
[0094] The LSTM-based data processing program 12 stored in the memory 11 in the electronic device 1 is a combination of multiple instructions. When running in the processor 10, it can achieve:
[0095] Calculate the idiosyncratic rate of return of the target data in period t based on the pricing model;
[0096] Obtain factor data related to idiosyncratic returns, and perform data preprocessing on factor data and idiosyncratic returns;
[0097] Taking factor data as input features and trait yield as output target, an LSTM model with advanced characteristics is obtained after training;
[0098] Based on the LSTM model with advanced characteristics, the characteristic yield of the t+1th period is output, and the characteristic yield improvement index is obtained according to the characteristic yield of the tth period and the characteristic yield of the t+1th period.
[0099] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to Figure 1 The description of the relevant steps in the corresponding embodiments will not be repeated here.
[0100] Furthermore, if the module / unit integrated in the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).
[0101] In an embodiment of the present invention, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the LSTM-based data processing method are implemented.
[0102] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.
[0103] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0104] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.
[0105] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.
Claims
1. A data processing method based on an LSTM model, characterized in that: The steps include: Step 1: Calculate the idiosyncratic rate of return of the target data in period t based on the pricing model; The formula for calculating the idiosyncratic rate of return in the pricing model is as follows: in, α is the idiosyncratic rate of return in period t, is the actual rate of return in period t, is the risk-free interest rate in period t, is the beta value at period t, is the expected rate of return in period t; Step 2: Obtain factor data related to idiosyncratic returns, and perform data preprocessing on the factor data and idiosyncratic returns; Step 3: Take the factor data as input features and the trait rate of return as output targets, and obtain an LSTM model with advanced characteristics after training; Step 4: Output the idiosyncratic yield of the t+1th period based on the LSTM model with advanced characteristics, and obtain the idiosyncratic yield improvement index based on the idiosyncratic yield of the tth period and the idiosyncratic yield of the t+1th period.
2. The data processing method based on the LSTM model according to claim 1, characterized in that: In step 1, the factor data includes Alpha factors, volatility index factors, market conditions index factors and financial index factors.
3. The data processing method based on the LSTM model according to claim 1, characterized in that: In step 2, data preprocessing is to remove missing values and then use the StandardScaler() function to standardize the data.
4. The data processing method based on the LSTM model according to claim 1, characterized in that: In step 2, the LSTM model contains 50 hidden units and a fully connected output layer, using the Adam optimizer and the mean square error loss function.
5. The data processing method based on the LSTM model according to claim 1, characterized in that: In step 3, the LSTM model with look-ahead characteristics is set as follows: in, represents the output characteristic yield of the t+1th period, α t is the characteristic rate of return in period t, F i are other factors, k and b are correlation coefficients.
6. The data processing method based on the LSTM model according to claim 5, characterized in that: The characteristic yield improvement index is to output the characteristic yield of the next period minus the characteristic yield of the current period: in, represents the characteristic yield improvement index of period t, represents the characteristic rate of return in period t+1, α t is the characteristic rate of return in period t.
7. A device for a data processing method based on an LSTM model, characterized in that: The device comprises: A data acquisition module is used to acquire target data through crawler data; A pricing model module, used to calculate the target data to obtain the idiosyncratic rate of return and factor data related to the idiosyncratic rate of return; An LSTM module is trained using factor data and the idiosyncratic rate of return output by the pricing model module to output the idiosyncratic rate of return for the t+1th period; The indicator acquisition module is used to calculate the characteristic yield improvement index based on the characteristic yield of the t+1th period output by the LSTM module.
8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the steps of the data processing method based on the LSTM model as described in any one of claims 1 to 7.
9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the LSTM model-based data processing method as described in any one of claims 1 to 6 is implemented.