Self-adaptive financial data analysis method
By introducing time series data and LSTM models in financial data analysis, combining network crawlers to obtain data, and using factor replacement solutions, the problem of difficult factor effectiveness and cyclicality in the existing technology is solved, and the effectiveness of stock price prediction is improved and adaptability to market changes.
Patent Information
- Application Number
- CN202411426834.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-14
- Publication Date
- 2025-02-18
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In the existing financial data analysis technology, the effectiveness and cyclicality of factor mining are difficult to predict, and there are difficulties in selecting parameters and obtaining data of deep learning methods, resulting in insufficient validity of stock price prediction.
By considering stock time series data in cross-section stock selection, using probability statistics and mean regression theory to perform real-time replacement of stock pools, combining network crawlers to obtain financial data, analyzing stock price trends using LSTM models, and dynamically adjusting stock portfolios through factor replacement schemes.
It improves the prediction and periodic analysis of factor effectiveness, enhances the parameter selection and data acquisition capabilities of deep learning, and improves the effectiveness of stock price prediction and the adaptability of market changes.
Smart Images
Figure CN119477542A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of artificial intelligence technology, and in particular relates to an adaptive financial data analysis method. Background Art
[0002] The current mainstream financial analysis techniques include the following two methods: Factor analysis method: 1. Obtain financial data, including historical stock price information, historical trading volume information, and stock fundamentals information for stocks; 2. Create stock factors based on fundamental information, trading volume information, and stock price information; 3. Sort stocks by stock factors or a combination of stock factors; 4. Select a stock selection strategy, backtest the stock's historical data, and verify the effectiveness of the stock selection strategy.
[0003] Deep learning method: 1. Obtain financial data. For stocks, this includes historical stock price information, historical trading volume information, stock fundamentals information, and various factor information. From the perspective of the traditional deep learning method CNN, it is divided into input layer, convolution layer, downsampling layer, fully connected layer, and output layer. The convolution layer is used to extract image features, the downsampling layer is used to reduce image features and extract representative image features, and the fully connected layer is used to flatten all features and calculate probability values. The same is true for financial data analysis. All input information is processed in multiple layers to obtain the correlation between information and stock prices, thereby predicting stock price fluctuations.
[0004] Current financial data analysis technology is mainly based on stable stock financial report data, and plays an important role in the field of traditional financial data analysis. Traditional factor mining technology has almost found the relationship between traditional financial data and stock prices, but there is a big problem with traditional factors, that is, the effectiveness is uncertain and the periodicity is obvious.
[0005] Financial analysis technology based on network data mining hopes to find a solution that can improve the effectiveness of stock price prediction and explore the potential relationship between stock prices and network data.
[0006] The shortcomings of the two mainstream financial analysis techniques are as follows: Factor analysis method: Currently, factor mining has reached a bottleneck. Some securities firms have mined thousands of factors, but the effectiveness and periodicity of the factors cannot be predicted. At the same time, the profitability of the factors also fluctuates greatly. The current factors are mainly based on active fund management, mainly to obtain investors' satisfaction through investment portfolios and achieve a balance between risk and return. The current problems that need to be solved are: find a method to predict the effectiveness of factors and the effectiveness cycle of factors; find a way to improve the effectiveness of factors.
[0007] Deep learning method: Deep learning is a black box, so the logic inside is difficult to define. Deep learning in the financial field focuses on several aspects: parameter selection, selecting appropriate parameters; learning the number of layers and the role of each layer, so that deep learning can purposefully complete the required tasks; activation function selection, preventing gradient explosion and gradient disappearance, and having good connectivity and dropout capabilities; normalization, so that the effects of each parameter will not have too large or too small effects due to different ranges; regularization, to avoid overfitting. In the process of deep learning, the selection of data and parameters will determine the effect of training, so how to obtain enough data and parameters has become a problem that deep learning methods need to solve. Summary of the invention
[0008] The present invention proposes a solution to the above problem, that is, to consider the impact of stock time series data on stock cross-sectional stock selection on the basis of cross-sectional stock selection, predict stock returns through probability statistics, and replace stocks in the stock pool in real time through the theory of mean reversion.
[0009] To solve the above technical problems, the adaptive financial data analysis method provided by the present invention comprises the following steps: S1. using web crawler technology to obtain financial data from professional market websites and search engines, including international financial data, domestic financial data, industry analysis data, transaction data, market capacity data and market share data; S2. Divide the acquired data into intermittent data and time series data; S3. Perform pairing analysis on time series data and stock factors, determine the correlation and set the correlation threshold; S4. Analyze the correlation between factors and stock prices, extract the top correlated factors through principal component analysis (PCA), and rank the correlations based on absolute values; S5. Determine the paired mean and standard deviation of the two time series by comparing the time series between the factors and the stock prices, and obtain the degree of deviation from the mean and the standard deviation range according to the current pairing situation; S6. The correlation between this factor and the subsequent stock price is predicted by the fluctuations between the previous paired correlations, and the trend of the correlation deviation and the standard deviation distribution in the future period are obtained; S7. Based on the correlation between the current factor and the stock price and the correlation between the predicted factor and the stock price, the change of the correlation between the factor and the stock price in the future period is obtained, and several factors with the smallest change in the correlation between the predicted factor and the stock price in the future period are selected as input factors; S8. Input the priority ranking factors into the long short-term memory network (LSTM) model, analyze the subsequent trend of stock prices, make cross-sectional stock selection, and rank the selected stocks according to expected return rates; S9. Carry out investment risk analysis on the selected stocks. The standard of risk analysis is to make statistics based on the variance of stock investment return rate, determine the balance between the mean stock return rate and the variance of the mean stock return rate, take the mean of the 90% interval of the Gaussian distribution function to calculate the average value, and obtain the most likely mean return rate. Reorder the stock sequence according to this mean return rate, and form an investment portfolio according to the capital allocation.
[0010] S10. Analyze the return of the investment portfolio, analyze the deviation between the actual return and the expected return of the investment portfolio, and analyze the deviation between the actual return and the theoretical return of all stocks in the stock pool. When there is a large deviation between the actual return and the expected return of a stock, if it is a positive deviation, then when this deviation exceeds 50%, it is necessary to consider selecting other stocks from the current stock pool for replacement. This is based on the mean reversion effect between stock price values.
[0011] As a further solution of the present invention, the network data acquisition step in S1 includes: (1) Crawling time series data from professional market websites; (2) Search for keywords through search engines to obtain corresponding web page information, crawl the data of the first 30 web pages through web crawlers, and record the numbers in the text and the keyword expressions and timestamp information corresponding to the numbers to obtain non-continuous data.
[0012] As a further solution of the present invention, in the step of pairing and analyzing the time series data and stock factors in S3, the setting of the correlation threshold is used to determine the factor category. When the correlation is greater than the threshold, they are considered to belong to the same category.
[0013] As a further solution of the present invention, in the correlation analysis step of S4, the top factors extracted by PCA are sorted according to different time periods, and the time periods include 3 months, 1 year, 2 years, 3 years, 5 years and 10 years.
[0014] As a further solution of the present invention, in the stock selection step of S8, the criterion for stock selection is that the average return of the stock price in the subsequent week or month is the highest.
[0015] As a further solution of the present invention, in the investment portfolio step of S9, the standard of the investment portfolio is no less than 3 stocks and no more than 10 stocks.
[0016] Compared with related technologies, the adaptive financial data analysis method provided by the present invention has the following beneficial effects: 1. New data types have been added on top of the original factors and deep learning parameters, enriching the number of factors and deep learning parameters in the financial market; 2. In cross-sectional stock selection, a time series analysis factor is added to allow the influence of time factors to be added to cross-sectional stock selection; 3. A method for analyzing the effectiveness of factors through correlation analysis between factors and stock prices and paired correlation analysis is proposed; 4. A factor replacement scheme is proposed, which allows stocks to replace factors in dynamic time adjustment, thus meeting the changing needs of the financial market; BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.
[0018] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION
[0019] Please refer to Figure 1 ,in, Figure 1 The flowchart of the present invention is shown in FIG. 1 . An adaptive financial data analysis method comprises the following steps: S1. Call Baidu search to search for network data. The searched data include: international financial data, domestic financial data, industry financial analysis data, industry transaction data, company transaction data, industry market capacity data, and company market share data. These data are obtained in two ways. One is to crawl the network based on the information provided by professional market websites to obtain the corresponding data. Generally, this type of data belongs to time series data and is updated daily. The other is to search for keywords through search engines to obtain corresponding web page information. The data of the first 30 web pages are crawled through web crawlers, and the numbers in the text and the keyword expressions and timestamp information corresponding to the numbers are recorded to obtain non-continuous data.
[0020] S2. After acquiring the data, the data is divided into two categories: one is intermittent data, which is data that is not continuous in time, and the other is time series data, which is data updated every day.
[0021] S3. Perform pairing analysis on the acquired time series data in combination with stock factors, analyze the correlation between all time series data and stock factors, and set a correlation threshold. When the pairing correlation is greater than this threshold, it is considered that the two types of factors belong to the same category.
[0022] S4. For all categories of factors, it is necessary to establish correlation analysis with stock prices, and use PCA analysis to get the top few factors of all categories. In this step, the correlation is divided into positive and negative forms. We take the absolute value of the correlation to rank the correlation. This correlation cycle needs to be adjusted according to the actual situation. In this case, six time intervals of 3 months, 1 year, 2 years, 3 years, 5 years, and 10 years are selected as the measurement standard.
[0023] S5. After the correlation analysis is completed, we need to pair and compare the time series between the factors and the stock prices, determine the pairing mean and standard deviation of the two time series, and obtain the degree of deviation from the mean and the standard deviation range based on the current pairing situation.
[0024] S6. After obtaining the degree of deviation and the standard deviation range, the correlation between this factor and the subsequent stock price is predicted through the fluctuations between the previous paired correlations, and the trend of the correlation deviation and the standard deviation distribution in the future period are obtained. This prediction.
[0025] S7. Through the correlation between the current factors and stock prices and the correlation between the predicted factors and stock prices, we can determine the changes in the correlation between factors and stock prices in the future, and select several factors with the smallest changes in the correlation between the predicted factors and stock prices in the future as input factors.
[0026] S8. Add these prioritized factors to the LSTM model, analyze the subsequent trend of the stock price, and make cross-sectional stock selection. The criterion for stock selection is the highest average return of the stock price in the subsequent week or month, and sort the selected stocks according to the expected return rate.
[0027] S9. Analyze the top stocks in the sequence and conduct stock investment risk analysis. The standard of risk analysis is to make statistics based on the variance of stock investment return rate, determine the balance between the mean stock return rate and the variance of the mean stock return rate, take the mean of the 90% interval of the Gaussian distribution function to calculate the average value, and get the most likely mean return rate. Reorder the stock sequence according to this mean return rate and form an investment portfolio according to the capital allocation. The standard of the investment portfolio is no less than 3 stocks and no more than 10 stocks.
[0028] S10. After determining the investment target, you need to analyze the return of the investment portfolio every day during the investment process, analyze the deviation between the actual return of the investment portfolio and the expected return, and also analyze the deviation between the actual return and theoretical return of all stocks in the stock pool. When there is a large deviation between the actual return and the expected return of a stock, if it is a positive deviation, then when this deviation exceeds 50%, you need to consider selecting other stocks from the current stock pool for replacement. This is based on the mean reversion effect between stock price values.
[0029] The present invention provides a new method for financial market analysis. By adding network financial big data to financial market analysis, a new method for financial analysis is provided. Factor correlation, pairing correlation, probability analysis, and deep learning are combined to provide a new method for factor effectiveness analysis. Through a factor replacement scheme, financial analysis can meet the changes in the financial market.
[0030] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.
Claims
1. An adaptive financial data analysis method, characterized in that: The following steps are involved: S1. Use web crawler technology to obtain financial data from professional market websites and search engines, including international financial data, domestic financial data, industry analysis data, transaction data, market capacity data and market share data; S2. Divide the acquired data into intermittent data and time series data; S3. Perform pairing analysis on time series data and stock factors, determine the correlation and set the correlation threshold; S4. Analyze the correlation between factors and stock prices, extract the top correlated factors through principal component analysis (PCA), and rank the correlations based on absolute values; S5. Determine the paired mean and standard deviation of the two time series by comparing the time series between the factors and the stock prices, and obtain the degree of deviation from the mean and the standard deviation range according to the current pairing situation; S6. The correlation between this factor and the subsequent stock price is predicted by the fluctuations between the previous paired correlations, and the trend of the correlation deviation and the standard deviation distribution in the future period are obtained; S7. Based on the correlation between the current factor and the stock price and the correlation between the predicted factor and the stock price, the change of the correlation between the factor and the stock price in the future period is obtained, and several factors with the smallest change in the correlation between the predicted factor and the stock price in the future period are selected as input factors; S8. Input the priority ranking factors into the long short-term memory network (LSTM) model, analyze the subsequent trend of stock prices, make cross-sectional stock selection, and rank the selected stocks according to expected return rates; S9. Carry out investment risk analysis on the selected stocks. The standard of risk analysis is to make statistics based on the variance of stock investment return rate, determine the balance between the mean stock return rate and the variance of the mean stock return rate, take the mean of the 90% interval of the Gaussian distribution function to calculate the average value, and obtain the most likely mean return rate. Reorder the stock sequence according to this mean return rate, and form an investment portfolio according to the capital allocation. 2.S10. Analyze the return of the investment portfolio, analyze the deviation between the actual return and the expected return of the investment portfolio, and analyze the deviation between the actual return and the theoretical return of all stocks in the stock pool. When there is a large deviation between the actual return and the expected return of a stock, if it is a positive deviation, then when this deviation exceeds 50%, it is necessary to consider selecting other stocks from the current stock pool for replacement. This is based on the mean reversion effect between stock price values.
3. The adaptive financial data analysis method according to claim 1, characterized in that: The network data acquisition step in S1 includes: (1) Crawling time series data from professional market websites; (2) Search for keywords through search engines to obtain corresponding web page information, crawl the data of the first 30 web pages through web crawlers, and record the numbers in the text and the keyword expressions and timestamp information corresponding to the numbers to obtain non-continuous data.
4. The adaptive financial data analysis method according to claim 1, characterized in that: In the step of pairing analysis of the time series data and stock factors in S3, the setting of the correlation threshold is used to determine the factor category. When the correlation is greater than the threshold, it is considered to belong to the same category.
5. The adaptive financial data analysis method according to claim 1, characterized in that: In the correlation analysis step of S4, the top factors extracted by PCA are sorted according to different time periods, which include 3 months, 1 year, 2 years, 3 years, 5 years and 10 years.
6. The adaptive financial data analysis method according to claim 1, characterized in that: In the stock selection step of S8, the criterion for stock selection is that the average return of the stock price in the subsequent week or month is the highest.
7. The adaptive financial data analysis method according to claim 1, characterized in that: In the investment portfolio step of S9, the standard of the investment portfolio is no less than 3 stocks and no more than 10 stocks.