IGWO improved LSTM network-based arbitrage price difference prediction method
By introducing IGWO improved heuristic algorithm and random search method into the LSTM network, the problem of difficulty in adjusting hyperparameters when adapting to different data sets is solved, and efficient and programmatic prediction model development is achieved.
Patent Information
- Application Number
- CN202411837749.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-13
- Publication Date
- 2025-03-21
AI Technical Summary
The existing financial time series prediction model based on deep learning has difficulty in adjusting hyperparameters when adapting to different data sets, resulting in bias in search results and high computational burden, and strong subjective factors.
The arbitrage spread prediction method based on IGWO-improved LSTM network is adopted, and high-quality hyperparameter groups are found in a large search space through heuristic algorithms and random search methods, reducing subjective factors for model adjustment and improving the development efficiency of prediction models.
It achieves high-quality search results given under smaller computing requirements, reduces subjective factors in the process of adjusting the prediction model, and promotes the programmatic and efficient prediction model.
Smart Images
Figure CN119671742A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of futures arbitrage spread prediction, and in particular to an arbitrage spread prediction method based on an IGWO-modified LSTM network. Background Art
[0002] With the rapid development of quantitative finance and the intensified competition in the investment market, the role of price data prediction in algorithmic trading has become increasingly prominent. An effective prediction model can help investors discover arbitrage signals in advance and evaluate the investment risks of different trading plans, and ultimately make reasonable decisions to maximize returns. Financial time series prediction models based on recurrent neural networks (such as long short-term memory networks LSTM, etc.) have shown strong competitiveness in capturing sequence characteristics and fitting nonlinear price data (most price data are nonlinear data).
[0003] However, the current prediction models based on deep learning methods at home and abroad are not enough to fully adapt to the rich and diverse data sets in real scenarios. Therefore, before training the model with historical data, it is often necessary to make certain adjustments to the model and the hyperparameters that determine its training process to adapt to different data situations. This work is generally completed through grid search. However, when the selected grid density is too sparse, the final search results may deviate greatly from the actual optimal point in a relatively large search space. The dense grid will bring a greater burden to the computational requirements of the experiment. In this regard, many researchers will subjectively narrow the search range based on previous research results and their own research experience (sometimes by adding more complex model structures). This method with strong subjective factors is not universally applicable and is therefore not suitable for promotion and use in real investment transactions. Summary of the invention
[0004] The purpose of the present invention is to provide an arbitrage spread prediction method based on the LSTM network improved by IGWO, which adopts a random search method with certain rules in a larger search space through a heuristic algorithm to provide higher quality search results with smaller search requirements. This method reduces the subjective factors in the prediction model adjustment process, promotes the programming and efficiency of the prediction model development process, and solves the problems raised in the above background technology.
[0005] To achieve the above object, the present invention provides the following technical solutions: an arbitrage spread prediction method based on an IGWO-modified LSTM network, which obtains the relationship between input data through multi-head self-attention, obtains the optimal parameters of the prediction model through the gradient descent algorithm of the inner network LSTM, searches for the optimal set of hyperparameters through the IGWO group heuristic algorithm of the outer network, and optimizes the optimal LSTM prediction model through the interaction of the hyperparameter group and the fitness function value of the two-layer network;
[0006] The method specifically comprises the following steps:
[0007] Step 1: First, preprocess the historical data, complete the division of training set, validation set and test set, and eliminate the dimensions between data through normalization;
[0008] Step 2: Perform initialization operations, set the population size n and the number of algorithm iterations epoc h_max;
[0009] Step 3: Determine the current search status according to the termination condition: First, determine the number of search rounds, that is, whether the current number of search rounds has reached the maximum number of search rounds. When the number of search rounds t satisfies t>epoc h_max, end the IGWO search and return the representative values of a group of subintervals with the highest current fitness values. They are the final results of hyperparameter optimization, and then jump to step 6;
[0010] Step 4: The inner network calculates the fitness value. First, a neural network model is constructed based on the input hyperparameters, and the data set is cut into feature data D through a sliding window mechanism. feature and label data D targets Finally, the parameters of the neural network are trained by the Adam optimization algorithm based on gradient descent, and the fitness is calculated by the loss function of the training process and formula (23).
[0011]
[0012] The LSTM layer of the neural network is used to extract short-term trends and features in the time series, and the extracted features are converted into interpretable variables through the fully connected layer;
[0013] Step 5: Update the position of the gray wolf according to fitness: Finally, return to step 3 and start the next round of search;
[0014] Step 6: Define and train the neural network model according to the hyperparameters output in step 3. The trained network model is the optimized time series prediction model, and its performance is verified by the divided test set.
[0015] Preferably, the method further includes: setting the multivariate time series historical data to be predicted to be:
[0016]
[0017] Among them, M in It is the number of features at each moment in the time series data set, and there is no missing feature value in the entire data set. In the preprocessing work, the historical data set will be divided into training set, validation set and test set, and the dimensions between different features will be eliminated through normalization.
[0018] Preferably, the LSTM network prediction model consists of a four-layer network, namely, an input layer, an LSTM layer, a fully connected layer, and an output layer; wherein the LSTM layer consists of C1 LSTM feature extraction units, using a traditional gate structure:
[0019]
[0020] The activation function w i ,w o ,w f ,w c is the weight matrix,
[0021] b i ,b o ,b f ,b c is the offset, As the sample feature L in The steps are input into the LSTM layer in this way, but only the last step As the extracted features are input to the fully connected layer, the fully connected layer and the output layer can be regarded as a fully connected network, in which the fully connected layer has C2 neurons and the output layer has M out neurons, which transforms the features extracted from the time series by the LSTM layer into L in +1 Moment M out The predicted value of a feature
[0022] Preferably, the inner optimization network uses the mean square error MSE as the loss function and uses the Adam optimizer to update the model parameters. In addition, before the training starts, the data set is divided into two parts: labels and features through a sliding window mechanism. Assume that the input window length of the time series is L in , then the characteristic data is:
[0023]
[0024] In addition, the output window is 1 time step and the number of predicted features is M out , we can get label data:
[0025]
[0026] During the parameter optimization process of the prediction model, the loss function values of the training set and the validation set will be used to calculate the fitness function value corresponding to the given hyperparameter group. The calculation formula of the fitness function is:
[0027]
[0028] in is the loss function value of the model obtained after i rounds of neural network training on the validation set, It is the loss function value of the model obtained after i rounds of neural network training on the training set, and epochs is the total number of iterations.
[0029] Preferably, the attention mechanism is used to reduce the impact of redundant and noisy data, and a reasonable weight ratio is assigned to the input data samples during training and learning. Let be an input sequence, Let t be the output sequence, t represents the length of the time series, also called the time step, d em is the dimension of each time series, and the number of attention heads is P.
[0030] Preferably, in the outer network, in order to maximize the performance of the prediction model, a group heuristic algorithm, the IGWO search algorithm, is constructed based on the idea of the gray wolf algorithm for hyperparameter tuning. The tunable hyperparameters include C1, C2, the number of iterations, the initial learning rate, and the input window length L. in Hyperparameters.
[0031] Preferably, the IGWO search algorithm includes an initialization phase:
[0032] Assume that the number of hyperparameters to be searched is m, and their search interval is L i (i=1,2,…,m), initialize the search space according to the NSGA-2 algorithm;
[0033] The basic definition and construction of NSGA-2 are as follows: Assume G s is an s-dimensional Euclidean space, where r∈G s , then P n (i)=(r1i1,r2i2,r3i3,...,r n i n ),i=1,2,3,...,n, n represents the number of samples, P n (i) represents the set of good points, and r refers to the good point. Or take r = {e j i}, where k is the smallest prime number satisfying (k-3) / 2≥0;
[0034] Known: The spatial dimension of the population is n, and the population size is m;
[0035] Calculate the r value, r = (r1, r2, r3, ..., r n ),in mi represents the i-th individual;
[0036] Construct a set of good points of number m: P n(i) = {(r1i1,r2i2,...,r n i n )},i=1,2,3,...n;
[0037] P n Mapped to the feasible domain where the population is located: where a j Indicates the lower limit of the current dimension, b j Indicates the upper limit of the current dimension;
[0038] The interval is divided into equally spaced small intervals, so that the population is evenly distributed in the interval, avoiding population accumulation, accelerating population convergence, and reducing the situation of falling into local optimality.
[0039] Preferably, the IGWO search algorithm further comprises: Search phase: the GWO algorithm takes the three leading wolves α, β, δ as the best solution and leads the remaining wolves ω to the promising area, thereby finding the global solution. Wolf hunting mainly includes three steps: encircling, hunting and attacking prey;
[0040] Surrounding: Simulate the situation where a gray wolf surrounds its prey, as shown in equations (4) and (5):
[0041] D=|C×X P -X(t)| (5)
[0042] X(t+1)=X P (t)-A×D (6)
[0043] Where X p is the prey position, X represents the position vector of any gray wolf, t is the current iteration, and C and A are the coefficient vectors calculated by equations (6) and (7):
[0044] A=2×A×r1-a(t) (7)
[0045] C=2×r2 (8)
[0046] Where r1, r2 are random vectors in [0, 1], and a is the convergence factor, as shown in formula (8), which decreases linearly from 2 to 0 as the number of iterations increases;
[0047]
[0048] Hunting: To mathematically model the hunting behavior of wolves, it is assumed that α, β, δ have a better understanding of the location of the prey. Given the locations of the three best solutions α, β, δ, other wolves ω must follow them. The following equations (9) and (10) describe the hunting behavior:
[0049]
[0050]
[0051] Where X α , X β and X δ are the first three best solutions at iteration t. A1, A2 and A3 are calculated by formula (6, D α , D β , D δ Defined by formula (9):
[0052]
[0053] The average value of X1, X2, and X3 cannot highlight the importance of α, β, and δ, so a weighted average strategy and a proportional weight strategy based on fitness value are used; the distance weights between the current gray wolf individual and α, β, and δ are as shown in equations (12)-(14). To prevent the denominator from being 0, ε is added to the denominator, and the value is e -16 , the improved gray wolf position update method is shown in formula (15):
[0054]
[0055] In addition, combined with the random position change of SMA, the wolf pack can jump out of the local optimum, increasing the possibility of searching for the global optimum. The final position update equation is shown in (16):
[0056]
[0057] Where X i (t) and X i (t-1) represents the position of the i-th candidate in the population between the current iteration t and the next iteration t+1; r2 is another random number in the Gaussian distribution; z is a proportional number, randomly selecting some candidates to restart initialization, the default z = 0.03, X A (t) and X B (t) are two candidates randomly selected in the current iteration, v b and v c are two other uniformly distributed random numbers, whose intervals are [-a, a] and [-b, b], where a and b are two variables related to the current number of iterations and the preset maximum iteration time:
[0058]
[0059] p is another proportional number that controls branch selection, which is related to the global optimal fitness value:
[0060]
[0061] Where bF, wF are all fitness values S i The best and worst fitness values among (i=1,2,...,n), Si is the fitness value of all individuals;
[0062] Attack: When the prey stops moving, the hunting process ends and the wolf starts to attack, which is done by the change of the value of a, which decreases linearly from 2 to 0 as the number of iterations increases. Half of the iterations are dedicated to exploration, and through a smooth transition, the other half is allocated to exploitation. In this step, the wolf changes its position to any position between the prey position and the current position.
[0063] According to the change of the value of a, the formula (8) is improved, as shown in formula (16). The graph of the improved convergence factor is a curve based on the change of trigonometric function. It decreases slowly in the early stage of iteration, so that the convergence factor maintains a larger value for a longer time, thereby maintaining a larger value for a longer time, so as to improve the search efficiency. It decreases quickly in the later stage of iteration, so that the smaller value is maintained for a longer time, thereby maintaining a smaller value for a longer time, so as to improve the search accuracy, and balance the global search and local search capabilities of the algorithm.
[0064]
[0065] The algorithm first randomly generates an initial wolf pack in the search space, and the fitness function evaluates the position of the wolves. Then, the above steps are repeated until the stopping criterion is met. The stopping criterion is to reach a predefined number of iterations (Maxiter). In each iteration, the top three wolves with the best skills are considered to be α, β, and δ. Each wolf updates its position according to the above steps of surrounding, hunting, and attacking prey; finally, by repeating these steps, the best position of the prey, i.e., the position of α, is found.
[0066] Preferably, the IGWO search algorithm also includes an output stage: when the search state reaches the termination condition, the IGWO algorithm will return the position with the highest current fitness and output it as a set of globally optimal hyperparameters; the maximum number of iterations set by the IGWO algorithm is reached, that is, t satisfies t>epoch_max. Since an elimination mechanism is introduced into the entire search process, the probability that the optimal value of a single-round search is close to or equal to the global optimal value will continue to increase with the increase in the number of iterations, which ensures the availability of the final output result.
[0067] Compared with the prior art, the present invention has the following beneficial effects:
[0068] The arbitrage spread prediction method based on the IGWO-modified LSTM network proposed in this paper programs the hyperparameter tuning process of building a prediction model, and searches for a set of hyperparameters that best matches historical data under the designed IGWO algorithm. This improves the short-term prediction accuracy of the model and has a certain degree of reference value for the development of prediction models in algorithmic trading. BRIEF DESCRIPTION OF THE DRAWINGS
[0069] Figure 1 It is a system model diagram of the present invention;
[0070] Figure 2 is a detailed flow chart of the present invention;
[0071] Figure 3 It is the pseudo code of the IGWO algorithm of the present invention. DETAILED DESCRIPTION
[0072] In order to make the purpose and technical solution of the present invention clearly and completely described, and the advantages more clearly understood, the embodiments of the present invention are further described in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are part of the embodiments of the present invention, rather than all of the embodiments, and are only used to explain the embodiments of the present invention, and are not used to limit the embodiments of the present invention. All other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0073] See also Figures 1 to 3 The present invention provides a technical solution: an arbitrage spread prediction method based on an IGWO-modified LSTM network, which obtains the relationship between input data through multi-head self-attention, obtains the optimal parameters of the prediction model through the gradient descent algorithm of the inner network LSTM, searches for the optimal set of hyperparameters through the IGWO group heuristic algorithm of the outer network, and optimizes the optimal LSTM prediction model through the interaction of the hyperparameter group and the fitness function value of the two-layer network;
[0074] The method specifically comprises the following steps:
[0075] Step 1: First, preprocess the historical data, complete the division of training set, validation set and test set, and eliminate the dimension between data through normalization; set the multivariate time series historical data to be predicted as:
[0076]
[0077] Among them, M in It is the number of features at each moment in the time series data set, and there is no missing feature value in the entire data set. In the preprocessing work, the historical data set will be divided into training set, validation set and test set, and the dimensions between different features will be eliminated through normalization.
[0078] The LSTM network prediction model consists of four layers, namely the input layer, LSTM layer, fully connected layer and output layer. The LSTM layer consists of C1 LSTM feature extraction units and adopts the traditional gate structure:
[0079]
[0080] The activation function w i ,w o ,w f ,w c is the weight matrix,
[0081] b i ,b o ,b f ,b c is the offset, As the sample feature L in The steps are input into the LSTM layer in this way, but only the last step As the extracted features are input to the fully connected layer, the fully connected layer and the output layer can be regarded as a fully connected network, in which the fully connected layer has C2 neurons and the output layer has M out neurons, which transforms the features extracted from the time series by the LSTM layer into L in +1 Moment M out The predicted value of a feature
[0082] Step 2: Perform initialization operations, set the population size n and the number of algorithm iterations epoc h_max;
[0083] Step 3: Determine the current search status according to the termination condition: First, determine the number of search rounds, that is, whether the current number of search rounds has reached the maximum number of search rounds. When the number of search rounds t satisfies t>epoc h_max, end the IGWO search and return the representative values of a group of subintervals with the highest current fitness values. They are the final results of hyperparameter optimization, and then jump to step 6;
[0084] Step 4: The inner network calculates the fitness value. First, a neural network model is constructed based on the input hyperparameters, and the data set is cut into feature data D through a sliding window mechanism. feature and label data D targets Finally, the parameters of the neural network are trained by the Adam optimization algorithm based on gradient descent, and the fitness is calculated by the loss function of the training process and the formula Error! Reference source not found.
[0085]
[0086] The LSTM layer of the neural network is used to extract short-term trends and features in the time series, and the extracted features are converted into interpretable variables through the fully connected layer;
[0087] The inner optimization network uses the mean square error MSE as the loss function and uses the Adam optimizer to update the model parameters. In addition, before the training starts, the data set is divided into two parts: labels and features through a sliding window mechanism. Assume that the input window length of the time series is L in , then the characteristic data is:
[0088]
[0089] In addition, the output window is 1 time step and the number of predicted features is M out , we can get label data:
[0090]
[0091] During the parameter optimization process of the prediction model, the loss function values of the training set and the validation set will be used to calculate the fitness function value corresponding to the given hyperparameter group. The calculation formula of the fitness function is:
[0092]
[0093] in is the loss function value of the model obtained after i rounds of neural network training on the validation set, It is the loss function value of the model obtained after i rounds of neural network training on the training set, and epochs is the total number of iterations.
[0094] The attention mechanism is used to reduce the impact of redundant and noisy data, and to assign a reasonable weight ratio to the input data samples during training and learning. Let be an input sequence, Let t be the output sequence, t represents the length of the time series, also called the time step, d em is the dimension of each time series, and the number of attention heads is P.
[0095] Step 5: Update the position of the gray wolf according to fitness: Finally, return to step 3 and start the next round of search;
[0096] In the outer network, in order to maximize the performance of the prediction model, a group heuristic algorithm, the IGWO search algorithm, is constructed based on the idea of the gray wolf algorithm for hyperparameter tuning. The tunable hyperparameters include C1, C2, the number of epochs, the initial learning rate, and the input window length L. in Hyperparameters.
[0097] The IGWO search algorithm includes the initialization phase:
[0098] Assume that the number of hyperparameters to be searched is m, and their search interval is L i (i=1,2,…,m), initialize the search space according to the NSGA-2 algorithm;
[0099] The basic definition and construction of NSGA-2 are as follows: Assume G s is an s-dimensional Euclidean space, where r∈G s , then P n (i)=(r1i1,r2i2,r3i3,...,r n i n ),i=1,2,3,...,n, n represents the number of samples, P n (i) represents the set of good points, and r refers to the good point. Or take r = {e j i}, where k is the smallest prime number satisfying (k-3) / 2≥0;
[0100] Known: The spatial dimension of the population is n, and the population size is m;
[0101] Calculate the r value, r = (r1, r2, r3, ..., r n ),in mi represents the i-th individual;
[0102] Construct a set of good points of number m: P n (i) = {(r1i1,r2i2,...,r n i n )},i=1,2,3,...n;
[0103] P n Mapped to the feasible domain where the population is located: where a j Indicates the lower limit of the current dimension, b j Indicates the upper limit of the current dimension;
[0104] The interval is divided into equally spaced small intervals, so that the population is evenly distributed in the interval, avoiding population accumulation, accelerating population convergence, and reducing the situation of falling into local optimality.
[0105] The IGWO search algorithm also includes: Search phase: The GWO algorithm takes the three alpha wolves α, β, and δ as the best solution and leads the rest of the wolves ω to the promising area to find the global solution. The wolf's hunting mainly includes three steps: encircling, hunting, and attacking the prey;
[0106] Surrounding: Simulate the situation where a gray wolf surrounds its prey, as shown in equations (4) and (5):
[0107] D=|C×X P -X(t)| (26)
[0108] X(t+1)=X P (t)-A×D (27)
[0109] Where X p is the prey position, X represents the position vector of any gray wolf, t is the current iteration, and C and A are the coefficient vectors calculated by equations (6) and (7):
[0110] A=2×A×r1-a(t) (28)
[0111] C=2×r2 (29)
[0112] Where r1, r2 are random vectors in [0, 1], and a is the convergence factor, as shown in formula (8), which decreases linearly from 2 to 0 as the number of iterations increases;
[0113]
[0114] Hunting: To mathematically model the hunting behavior of wolves, it is assumed that α, β, δ have a better understanding of the location of the prey. Given the locations of the three best solutions α, β, δ, other wolves ω must follow them. The following equations (9) and (10) describe the hunting behavior:
[0115]
[0116]
[0117] Where X α , X β and X δ are the first three best solutions at iteration t. A1, A2 and A3 are calculated by formula (6). α , D β , D δ Defined by formula (9):
[0118]
[0119] The average value of X1, X2, and X3 cannot highlight the importance of α, β, and δ, so a weighted average strategy and a proportional weight strategy based on fitness value are used; the distance weights between the current gray wolf individual and α, β, and δ are as shown in equations (12)-(14). To prevent the denominator from being 0, ε is added to the denominator, and the value is e -16 , the improved gray wolf position update method is shown in formula (15):
[0120]
[0121] In addition, combined with the random position change of SMA, the wolf pack can jump out of the local optimum, increasing the possibility of searching for the global optimum. The final position update equation is shown in (16):
[0122]
[0123] Where X i (t) and X i (t-1) represents the position of the i-th candidate in the population between the current iteration t and the next iteration t+1; r2 is another random number in the Gaussian distribution; z is a proportional number, randomly selecting some candidates to restart initialization, the default z = 0.03, X A (t) and X B (t) are two candidates randomly selected in the current iteration, v b and v c are two other uniformly distributed random numbers, whose intervals are [-a, a] and [-b, b], where a and b are two variables related to the current number of iterations and the preset maximum iteration time:
[0124]
[0125] p is another proportional number that controls branch selection, which is related to the global optimal fitness value:
[0126]
[0127] Where bF, wF are all fitness values S i The best and worst fitness values among (i=1,2,...,n), S i are all fitness values of all individuals;
[0128] Attack: When the prey stops moving, the hunting process ends and the wolf starts to attack, which is done by the change of the value of a, which decreases linearly from 2 to 0 as the number of iterations increases. Half of the iterations are dedicated to exploration, and through a smooth transition, the other half is allocated to exploitation. In this step, the wolf changes its position to any position between the prey position and the current position.
[0129] According to the change of the value of a, the formula (8) is improved, as shown in formula (16). The graph of the improved convergence factor is a curve based on the change of trigonometric function. It decreases slowly in the early stage of iteration, so that the convergence factor maintains a larger value for a longer time, thereby maintaining a larger value for a longer time, so as to improve the search efficiency. It decreases quickly in the later stage of iteration, so that the smaller value is maintained for a longer time, thereby maintaining a smaller value for a longer time, so as to improve the search accuracy, and balance the global search and local search capabilities of the algorithm.
[0130]
[0131] The algorithm first randomly generates an initial wolf pack in the search space, and the fitness function evaluates the position of the wolves. Then, the above steps are repeated until the stopping criterion is met. The stopping criterion is to reach a predefined number of iterations (Maxiter). In each iteration, the top three wolves with the best skills are considered to be α, β, and δ. Each wolf updates its position according to the above steps of surrounding, hunting, and attacking prey; finally, by repeating these steps, the best position of the prey, i.e., the position of α, is found.
[0132] The IGWO search algorithm also includes an output stage: when the search state reaches the termination condition, the IGWO algorithm will return the position with the highest current fitness and output it as a set of globally optimal hyperparameters; the maximum number of iterations set by the IGWO algorithm is reached, that is, t satisfies t>epoch_max. Since the elimination mechanism is introduced in the entire search process, the probability that the optimal value of a single round of search is close to or equal to the global optimal value will continue to increase with the increase in the number of iterations, which ensures the availability of the final output result.
[0133] Step 6: Define and train the neural network model according to the hyperparameters output in step 3. The trained network model is the optimized time series prediction model, and its performance is verified by the divided test set.
[0134] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. An arbitrage spread prediction method based on IGWO-modified LSTM network, characterized by: The relationship between input data is obtained through multi-head self-attention. The inner network LSTM obtains the optimal parameters of the prediction model through the gradient descent algorithm. The outer network searches for the optimal set of hyperparameters through the IGWO group heuristic algorithm. The two-layer network optimizes the optimal LSTM prediction model through the interaction of the hyperparameter group and the fitness function value. The method specifically comprises the following steps: Step 1: First, preprocess the historical data, complete the division of training set, validation set and test set, and eliminate the dimensions between data through normalization; Step 2: Perform initialization operations, set the population size n and the number of algorithm iterations epoc h_max; Step 3: Determine the current search status according to the termination condition: First, determine the number of search rounds, that is, whether the current number of search rounds has reached the maximum number of search rounds. When the number of search rounds t satisfies t>epoc h_max, end the IGWO search and return the representative values of a group of subintervals with the highest current fitness values. They are the final results of hyperparameter optimization, and then jump to step 6; Step 4: The inner network calculates the fitness value. First, a neural network model is constructed based on the input hyperparameters, and the data set is cut into feature data D through a sliding window mechanism. feature and label data D targets Finally, the parameters of the neural network are trained by the Adam optimization algorithm based on gradient descent, and the fitness is calculated by the loss function of the training process and formula (1). The LSTM layer of the neural network is used to extract short-term trends and features in the time series, and the extracted features are converted into interpretable variables through the fully connected layer; Step 5: Update the position of the gray wolf according to fitness: Finally, return to step 3 and start the next round of search; Step 6: Define and train the neural network model according to the hyperparameters output in step 3. The trained network model is the optimized time series prediction model, and its performance is verified by the divided test set.
2. The arbitrage spread prediction method based on the IGWO-modified LSTM network according to claim 1, characterized in that: Also includes: Set the multivariate time series historical data to be predicted as: Among them, M in It is the number of features at each moment in the time series data set, and there is no missing feature value in the entire data set. In the preprocessing work, the historical data set will be divided into training set, validation set and test set, and the dimensions between different features will be eliminated through normalization.
3. The arbitrage spread prediction method based on the IGWO-modified LSTM network according to claim 2, characterized in that: The LSTM network prediction model consists of four layers, namely the input layer, LSTM layer, fully connected layer and output layer. The LSTM layer consists of C1 LSTM feature extraction units and adopts the traditional gate structure: The activation function w i ,w o ,w f ,w c is the weight matrix, b i ,b o ,b f ,b c is the offset, As the sample feature L in The steps are input into the LSTM layer in this way, but only the last step As the extracted features are input to the fully connected layer, the fully connected layer and the output layer can be regarded as a fully connected network, in which the fully connected layer has C2 neurons and the output layer has M out neurons, which transforms the features extracted from the time series by the LSTM layer into L in +1 Moment M out The predicted value of a feature 4. The arbitrage spread prediction method based on the IGWO-modified LSTM network according to claim 3 is characterized in that: The inner optimization network uses the mean square error MSE as the loss function and uses the Adam optimizer to update the model parameters. In addition, before the training starts, the data set is divided into two parts: labels and features through a sliding window mechanism. Assume that the input window length of the time series is L in , then the characteristic data is: In addition, the output window is 1 time step and the number of predicted features is M out , we can get label data: During the parameter optimization process of the prediction model, the loss function values of the training set and the validation set will be used to calculate the fitness function value corresponding to the given hyperparameter group. The calculation formula of the fitness function is: in is the loss function value of the model obtained after i rounds of neural network training on the validation set, It is the loss function value of the model obtained after i rounds of neural network training on the training set, and epochs is the total number of iterations.
5. The arbitrage spread prediction method based on the IGWO-modified LSTM network according to claim 4 is characterized in that: The attention mechanism is used to reduce the impact of redundant and noisy data, and to assign a reasonable weight ratio to the input data samples during training and learning. Let be an input sequence, Let t be the output sequence, t represents the length of the time series, also called the time step, d em is the dimension of each time series, and the number of attention heads is P.
6. The arbitrage spread prediction method based on the IGWO-modified LSTM network according to claim 1, characterized in that: In the outer network, in order to maximize the performance of the prediction model, a group heuristic algorithm, the IGWO search algorithm, is constructed based on the idea of the gray wolf algorithm for hyperparameter tuning. The tunable hyperparameters include C1, C2, the number of epochs, the initial learning rate, and the input window length L. in Hyperparameters.
7. The arbitrage spread prediction method based on the IGWO-modified LSTM network according to claim 6 is characterized in that: The IGWO search algorithm includes the initialization phase: Assume that the number of hyperparameters to be searched is m, and their search interval is L i (i=1,2,…,m), initialize the search space according to the NSGA-2 algorithm; The basic definition and construction of NSGA-2 are as follows: Assume G s is an s-dimensional Euclidean space, where r∈G s , then P n (i)=(r1i1,r2i2,r3i3,...,r n i n ),i=1,2,3,...,n, n represents the number of samples, P n (i) represents the set of good points, and r refers to the good point. Or take r = {e j i}, where k is the smallest prime number satisfying (k-3) / 2≥0; Known: The spatial dimension of the population is n, and the population size is m; Calculate the r value, r = (r1, r2, r3, ..., r n ),in mi represents the i-th individual; Construct a set of good points of number m: P n (i) = {(r1i1,r2i2,...,r n i n )},i=1,2,3,...n; P n Mapped to the feasible domain where the population is located: where a j Indicates the lower limit of the current dimension, b j Indicates the upper limit of the current dimension; The interval is divided into equally spaced small intervals, so that the population is evenly distributed in the interval, avoiding population accumulation, accelerating population convergence, and reducing the situation of falling into local optimality.
8. The arbitrage spread prediction method based on the IGWO-modified LSTM network according to claim 6, characterized in that: The IGWO search algorithm also includes: Search phase: The GWO algorithm takes the three alpha wolves α, β, and δ as the best solution and leads the rest of the wolves ω to the promising area to find the global solution. The wolf's hunting mainly includes three steps: encircling, hunting, and attacking the prey; Surrounding: Simulate the situation where a gray wolf surrounds its prey, as shown in equations (4) and (5): D=|C×X P -X(t)| (5) X(t+1)=X P (t)-A×D (6) Where X p is the prey position, X represents the position vector of any gray wolf, t is the current iteration, and C and A are the coefficient vectors calculated by equations (6) and (7): A=2×A×r1-a(t) (7) C＝2×r2 (8) Where r1, r2 are random vectors in [0, 1], and a is the convergence factor, as shown in formula (8), which decreases linearly from 2 to 0 as the number of iterations increases; Hunting: To mathematically model the hunting behavior of wolves, it is assumed that α, β, δ have a better understanding of the location of the prey. Given the locations of the three best solutions α, β, δ, other wolves ω must follow them. The following equations (9) and (10) describe the hunting behavior: Where X α , X β and X δ are the first three best solutions at iteration t. A1, A2 and A3 are calculated by formula (6). α , D β , D δ Defined by formula (9): The average value of X1, X2, and X3 cannot highlight the importance of α, β, and δ, so a weighted average strategy and a proportional weight strategy based on fitness value are used; the distance weights between the current gray wolf individual and α, β, and δ are as shown in equations (12)-(14). To prevent the denominator from being 0, ε is added to the denominator, and the value is e -16 , the improved gray wolf position update method is shown in formula (15): In addition, combined with the random position change of SMA, the wolf pack can jump out of the local optimum, increasing the possibility of searching for the global optimum. The final position update equation is shown in (16): Where X i (t) and X i (t-1) represents the position of the i-th candidate in the population between the current iteration t and the next iteration t+1; r2 is another random number in the Gaussian distribution; z is a proportional number, randomly selecting some candidates to restart initialization, the default z = 0.03, X A (t) and X B (t) are two candidates randomly selected in the current iteration, v b and v c are two other uniformly distributed random numbers, whose intervals are [-a, a] and [-b, b], where a and b are two variables related to the current number of iterations and the preset maximum iteration time: p is another proportional number that controls branch selection, which is related to the global optimal fitness value: Where bF, wF are all fitness values S i The best and worst fitness values among (i=1,2,...,n), S i are all fitness values of all individuals; Attack: When the prey stops moving, the hunting process ends and the wolf starts to attack, which is done by the change of the value of a, which decreases linearly from 2 to 0 as the number of iterations increases. Half of the iterations are dedicated to exploration, and through a smooth transition, the other half is allocated to exploitation. In this step, the wolf changes its position to any position between the prey position and the current position. According to the change of the value of a, the formula (8) is improved, as shown in formula (16). The graph of the improved convergence factor is a curve based on the change of trigonometric function. It decreases slowly in the early stage of iteration, so that the convergence factor maintains a larger value for a longer time, thereby maintaining a larger value for a longer time, so as to improve the search efficiency. It decreases quickly in the later stage of iteration, so that the smaller value is maintained for a longer time, thereby maintaining a smaller value for a longer time, so as to improve the search accuracy, and balance the global search and local search capabilities of the algorithm. The algorithm first randomly generates an initial wolf pack in the search space, and the fitness function evaluates the position of the wolves. Then, the above steps are repeated until the stopping criterion is met. The stopping criterion is to reach a predefined number of iterations (Maxiter). In each iteration, the top three wolves with the best skills are considered to be α, β, and δ. Each wolf updates its position according to the above steps of surrounding, hunting, and attacking prey; finally, by repeating these steps, the best position of the prey, i.e., the position of α, is found.
9. The arbitrage spread prediction method based on the IGWO-modified LSTM network according to claim 8, characterized in that: The IGWO search algorithm also includes an output stage: when the search state reaches the termination condition, the IGWO algorithm will return the position with the highest current fitness and output it as a set of globally optimal hyperparameters; the maximum number of iterations set by the IGWO algorithm is reached, that is, t satisfies t>epoch_max. Since the elimination mechanism is introduced in the entire search process, the probability that the optimal value of a single round of search is close to or equal to the global optimal value will continue to increase with the increase in the number of iterations, which ensures the availability of the final output result.