Stock prediction model based on multi-head cross-attention mechanism and improved GRU

Abstract

The invention discloses a stock prediction model based on a multi-head cross-attention mechanism and an improved GRU, and the model comprises an improved GRU model, the improved GRU model comprises an attention mechanism, and the attention mechanism is used for replacing a reset gate; the graph attention network is used for capturing the relation between different stocks to extract the cross section characteristics of the data, the graph attention network comprises an attention mechanism, and different weights are distributed to different nodes according to the relation between the stocks through the attention mechanism; and a multi-head cross attention mechanism, wherein the multi-head cross attention mechanism is used for interaction between the improved GRU model and the output of the graph attention network. According to the method, the potential state interacts with the time characteristics and the cross section characteristics, so that the expression ability of the model and the ability of capturing complex market dynamics are further enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of stock prediction, and in particular to a stock prediction model based on a multi-head cross-attention mechanism and an improved GRU. Background Art

[0002] In recent years, with the advent of the big data era and the rapid development of the global economy, the complexity of financial markets has increased significantly. This trend has posed unprecedented challenges to stock market volatility and unpredictability. Consequently, accurate stock forecasts have become crucial not only for investors and financial institutions, helping them develop more robust investment strategies and risk management measures, but also for policymakers, who rely on these forecasts for macroeconomic regulation and market supervision. Furthermore, for academic researchers, stock forecasting has become a key area for uncovering market dynamics and behavioral patterns, driving research in financial market theory and data-driven methods. This research has not only expanded the theoretical boundaries of financial economics but also provided new research directions and application scenarios for interdisciplinary fields such as machine learning and data science. Consequently, the accuracy and effectiveness of stock forecasts have become a research focus across multiple disciplines, further spurring the extensive exploration of innovative models and methods.

[0003] Time series models, such as GRU and LSTM, have been widely used in stock market forecasting due to their significant advantages in capturing temporal dependencies in sequential data. However, these models have limitations when dealing with long-term dependencies in financial markets. Long-term trends and large fluctuations are often obscured by noise, making it challenging for these models to effectively extract valuable long-term dependency information from noisy data. Furthermore, the highly nonlinear nature of financial markets manifests itself in rapid changes in market behavior, driven by factors such as investor sentiment and unexpected events. These models often lack sufficient sensitivity and flexibility to handle such nonlinearities and unexpected events. Furthermore, they face challenges in flexibly selecting and effectively utilizing key historical information. Given the vast and unordered nature of financial market data, identifying the most relevant predictive features has become a critical issue.

[0004] In recent years, the Transformer model has shown significant potential in capturing long-term dependencies and processing complex nonlinear features thanks to its architecture based on the self-attention mechanism. Unlike traditional RNN models (such as GRU and LSTM), the Transformer is able to simultaneously focus on all time steps in a sequence, making it particularly effective in extracting long-term dependencies. In addition, the Transformer's powerful parallel processing capabilities enable it to efficiently manage large-scale data, which is crucial for processing the vast and diverse stock data in the financial market. However, the application of the Transformer model also faces some challenges. First, the large number of parameters in the Transformer model leads to high computational costs, especially when processing ultra-large-scale financial data, which may limit its application in resource-constrained environments. In addition, although the Transformer is good at capturing complex nonlinear relationships, its performance may be affected when faced with highly noisy financial data.

[0005] The rapid development of artificial intelligence (AI) technologies, particularly graph neural networks (GNNs) and reinforcement learning (RL), has brought unprecedented potential to stock market forecasting. These technologies, through innovative algorithmic design and deep learning models, have enhanced their ability to capture the complex dynamics of financial markets. For example, methods employing GNNs can accurately capture the complex and diverse interdependencies in financial data by modeling relationships between stocks as a graph structure. This approach not only reveals deep connections that are difficult to detect with traditional models but also more effectively reflects the nonlinear nature of the market. Furthermore, RL-based methods gradually learn and optimize trading strategies by simulating continuous interactions with the market environment. These methods are highly adaptable and can dynamically adjust to rapid market changes. However, despite their enormous potential, these emerging methods still face significant challenges in practical application. First, these models often rely on large-scale, high-quality datasets, which are difficult to obtain or construct in real-world scenarios. Their predictive performance can be severely affected when faced with data sparsity and noise—common problems in financial markets. Furthermore, these methods are computationally complex; the training process is not only time-consuming and resource-intensive, but also requires significant computing power, significantly limiting their widespread adoption in real-world financial applications. More critically, a fundamental limitation of current models lies in their inability to effectively capture unobservable underlying market dynamics. Market sentiment, investor expectations, microstructural factors, and participant behavioral patterns play a key role in shaping market dynamics. However, failure to fully account for these underlying factors often leads to a superficial understanding of the market, limiting the model's predictive accuracy and practical application value.

[0006] Technical term: MCI-GRU: Stock prediction model with multi-head cross-attention mechanism and improved GRU. Summary of the Invention

[0007] In response to the shortcomings of the prior art, the present invention aims to provide a stock prediction model based on a multi-head cross-attention mechanism and an improved GRU. By interacting the latent state with temporal and cross-sectional features, the model's expressive power and ability to capture complex market dynamics are further enhanced. To achieve the above-mentioned purpose and other advantages of the present invention, a stock prediction model based on a multi-head cross-attention mechanism and an improved GRU is provided, comprising:

[0008] An improved GRU model, wherein the improved GRU model includes an attention mechanism, wherein the attention mechanism is used to replace a reset gate;

[0009] A graph attention network, wherein the graph attention network is used to capture the relationship between different stocks to extract cross-sectional features of the data, and the graph attention network includes an attention mechanism, through which different weights are assigned to different nodes according to the relationship between the stocks;

[0010] A multi-head cross-attention mechanism is used to interact with the outputs of the improved GRU model and the graph attention network, thereby effectively absorbing relevant information in the outputs of the improved GRU model and the graph attention network and optimizing the potential state representation.

[0011] Preferably, the attention mechanism in the improved GRU model assigns attention weights by calculating the similarity between the query, key, and value, where the hidden state of the previous time step is is considered as a query, the input x at the current time step t As keys and values.

[0012] Preferably, the attention mechanism allocates attention weights by calculating the similarity between the query, key, and value, specifically comprising the following steps:

[0013] The hidden state h t-1 and input x t Linearly transform to query, key and value spaces respectively;

[0014] Attention weights are obtained by calculating the dot product similarity between the query and the key;

[0015] The final reset gate value is obtained by the weighted sum of the values, weighted by the attention weights.

[0016] Preferably, the improved GRU model generates a final hidden state sequence through recursive calculation of multiple time steps, and the last hidden state h t As the representation vector of the entire sequence, the representation vector is used in further feature extraction and model learning stages.

[0017] Preferably, the graph of the graph attention network is constructed in such a way that each node represents a stock, and the edges between nodes represent the relationship between stocks;

[0018] And the weight of the edge, that is, the strength of the relationship, is determined by calculating the historical correlation of stock returns, which is calculated by the Pearson correlation coefficient or other statistical measures.

[0019] Preferably, the attention mechanism of the graph attention network is used to dynamically calculate the influence of each node's neighbors when aggregating information, that is, the attention coefficient, which specifically includes the following:

[0020] The feature vector x of each node it Through the learnable weight matrix W g Perform linear transformation;

[0021] Calculate the attention coefficient between stock i and its neighbor j;

[0022] The attention coefficient is normalized across all neighbors of node i through the softmax function;

[0023] The update of a node is represented by the weighted sum of the transformed features of its neighbors, where the weights consist of the normalized attention coefficients.

[0024] An improved GRU stock prediction model is characterized in that the attention mechanism in the graph attention network is used for.

[0025] Compared with existing technologies, the present invention offers the following advantages: By replacing the reset gate in the traditional GRU model with an attention mechanism, the MCI-GRU model significantly improves the flexibility of selecting and utilizing historical time series information. Secondly, the MCI-GRU employs a graph attention network (GAT) to extract cross-sectional features of stock data. Furthermore, the present invention introduces a multi-head cross-attention mechanism to capture potential unobservable market states. By interacting these potential states with temporal and cross-sectional features, the model's expressive power and ability to capture complex market dynamics are further enhanced. DETAILED DESCRIPTION

[0026] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0027] The present invention considers a set of stocks, denoted as S = {s1, s2, ..., sN}, where each s i represents an individual stock, and N is the total number of stocks in the dataset. i , the data related to the t-th trading day is represented as a vector in and Respectively represent the opening price, closing price, highest price, lowest price, trading volume and turnover of the tth day. x To represent the number of features used for each stock on each day. i The data sequence on day t is represented as a set x i ={x i1 ,x i2 ,…,x it The complete data set of all stocks is represented by X = {x1, x2, ..., x N}.

[0028] A stock prediction model based on a multi-head cross-attention mechanism and an improved GRU, comprising: an improved GRU model, wherein the improved GRU model includes an attention mechanism, wherein the attention mechanism is used to replace a reset gate; in time series prediction tasks, the GRU model is widely used because it can effectively capture the temporal dependencies in sequence data. However, traditional GRU models have certain limitations in capturing complex temporal relationships, especially when dealing with long-term dependencies, and they may find it difficult to extract deeper features from the sequence. In order to solve this problem, the present invention adopts an enhanced GRU model, which replaces the reset gate with an attention mechanism, thereby improving the model's representation and learning capabilities for temporal data. In the classic GRU model, the hidden state h t Updates are performed through two gating mechanisms: update gate z t and reset gate r t The formula is as follows: t =σ(W z x t +U z h t-1 +b z ), r t =σ(W r x t +U r h t-1 +b r )in, represents the stock input at the current time step, represents the hidden state of the previous time step, d h is the dimension of the hidden state. and is the weight matrix, and is the bias term, σ is the activation function, usually the Sigmoid function. Update gate and reset gate Used to adjust the hidden state h of the previous time step t-1 Candidate hidden state for the current time step The specific formula for calculating the impact is as follows: in, is the weight matrix, is the bias term, ⊙ represents element-wise multiplication. Finally, the hidden state h at the current time step is t By update gate z t Controls as follows: In these equations, the reset gate r t Determines the hidden state h of the previous time step t-1 The degree of reset in the current time step. However, this mechanism has certain limitations in capturing long-term dependencies. To overcome the above limitations, the model proposed in this paper introduces an attention mechanism to replace the traditional reset gate r in GRU t The attention mechanism can dynamically assign weights to different time steps in the sequence, thereby more accurately capturing the key information in the time series data. Specifically, the traditional reset gate r t is replaced by the attention-based weight coefficient α t , and its calculation formula is as follows: t =Attention(h t-1 ,x t ) where α t ∈R is the attention weight vector. The core idea of the attention mechanism is to assign attention weights by calculating the similarity between query, key, and value. In this model, the hidden state of the previous time step is is considered as a query, and the input x at the current time step is t The attention calculation process includes the following steps:

[0029] Linear transformation of query, key and value: First, the hidden state h t-1 and input x t Linearly transform to query, key and value spaces respectively: q t =W q h t-1 ,k t =W k x t ,v t =W v x t.in, and is a learnable linear transformation matrix. and d q Represents the dimension of the query, used to calculate the attention weight; and d k Represents the dimension of the key; and d v Indicates the dimension of the value, usually the same as d q and d k Same, that is, d q =d k =d v .

[0030] Attention weight calculation: The attention weight is obtained by calculating the dot product similarity between the query and the key: Among them, d k Used to scale the dot product result to prevent numerical instabilities.

[0031] Weighted Sum: The final reset gate value is obtained by the weighted sum of the values, weighted by the attention weight: r' t =α t v t Among them, the new Dynamically select the current input x t and the previous hidden state h t-1 The most important part of the model, thereby enhancing the model's ability to capture long-term dependencies.

[0032] Furthermore, by introducing a new reset gate r' t After that, the update formula of the GRU hidden state is adjusted as follows: Among them, z t is the update gate. This updated hidden state calculation combines the dynamic information selection capability provided by the attention mechanism, enabling the model to better extract long-term dependency information and key features from time series data. Through recursive calculations of multiple time steps, the improved GRU model generates the final hidden state sequence H = h1, h2, ..., h t ], where each h t Both contain information from past time steps, and due to the introduction of the attention mechanism, the model's attention to important time steps is enhanced. For subsequent processing, the last hidden state h t As the representation vector of the entire sequence, it is recorded as This output A1 will serve as input for further feature extraction and model learning stages.

[0033] Furthermore, the graph attention network is used to capture the relationship between different stocks to extract the cross-sectional features of the data, and the graph attention network includes an attention mechanism, through which different weights are assigned to different nodes according to the relationship between stocks; GAT expands the traditional graph convolutional network by introducing the attention mechanism, so that the model can assign different weights to different nodes according to the relationship between nodes (stocks). In this model, the input of the GAT layer is a matrix that represents the features of all stocks at a specific time step. The dimension of the input matrix is (N, d x ), where N is the number of stocks, representing nodes in the graph, and d x is the dimension of the feature vector for each stock. This input representation is derived from the raw stock data. Therefore, the GAT layer is particularly concerned with capturing the cross-sectional dependencies between stocks. In the GAT layer, the graph is constructed in such a way that each node represents a stock and the edges between nodes represent the relationships between these stocks. The weight of the edge, that is, the strength of the relationship, is determined by calculating the historical correlation of stock returns. Typically, these correlations are calculated using the Pearson correlation coefficient or other statistical measures. In order to determine the relationship between two stocks s i and s j To determine the strength of the relationship between them, the present invention first calculates the historical correlation of their stock returns in the past year. i (t') and r j (t') respectively represent stock s i and s j Return at time t, where t' = 1, 2, ..., T represents the number of trading days in the past year. For example, there are 252 trading days per year. Returns are usually calculated using the logarithmic return formula: Among them, r i (t') represents stock s i The closing price at time t'. Next, the present invention calculates the Pearson correlation coefficient ρ (s i ,s j ) to measure their linear correlation. The calculation formula of Pearson correlation coefficient is:

[0034]

[0035] in, and Represents stocks i and s j The average return in the past year. According to the correlation coefficient ρ(s i ,s j ), the present invention is a stock s i and s jThe relationship between them is assigned an edge weight. i,j Usually set to ρ(s i ,s j ). To optimize the learning process, not all relations are included in the graph, but rather a threshold-based filtering method called judge is applied. value This parameter enables the model to retain only the edges that represent important relationships, thereby effectively reducing noise and focusing on the most relevant connections.

[0036] Furthermore, the core of the GAT layer lies in its attention mechanism, which dynamically calculates the influence (attention coefficient) of each node's neighbors when aggregating information. For each node i in the graph, the GAT layer performs the following operations:

[0037] Linear transformation: feature vector x of each node it Through the learnable weight matrix W g Perform linear transformation: h' i =W g x it Among them, x it is the stock s at time step t i Features, is the transformed eigenvector, d g is the dimension of the hidden layer.

[0038] Attention coefficient calculation: The attention coefficient between stock i and its neighbor j is calculated by the following formula: in, is a learnable attention vector, || represents the connection operation, and LeakyReLU is an activation function that introduces nonlinearity, which aims to introduce nonlinear features into attention calculation.

[0039] Normalization: The attention coefficients are then normalized across all neighbors of node i using the softmax function: Among them, σ ij is the normalized attention score between stock i and stock j, N i represents the neighbor set of node i.

[0040] Feature Aggregation: Finally, the update of a node represents the transformed features of its neighbors by weighted summation, where the weights consist of the normalized attention coefficients: Where σ is a nonlinear activation function, usually ReLU. The final output of the GAT layer is a matrix Each row corresponds to an updated feature vector for a stock, which now includes information from its neighbors. This output captures the cross-sectional dependencies between stocks and serves as input to the next stage of the model, where the features are further processed by the market latent state learning layer.

[0041] Furthermore, a multi-head cross-attention mechanism is used to interact with the output of the improved GRU model and the graph attention network, thereby effectively absorbing relevant information in the output of the improved GRU model and the graph attention network and optimizing the latent state representation. The market latent state learning layer is a key component of the model, which aims to capture and represent the latent states of the market, which cannot be directly observed in the original data. This layer is specifically designed to model potential market conditions that affect stock behavior, so that the model can better understand and predict stock fluctuations by utilizing these hidden states. The process of initializing the market potential state starts with initializing a set of learnable market potential state vectors. These vectors represent different potential market conditions or factors that may affect stock prices. The initialization process is as follows:

[0042] The number of potential states (d r ): Model initialization d r latent state vectors. r is a hyperparameter that can be adjusted based on the complexity of the market being modeled. r This enables the model to capture a wider variety of potential factors.

[0043] Dimension (d i ): The dimension of each potential state vector is d i , should match the dimensions of the stock features the model is currently learning. This ensures that these latent states are compatible with the outputs of the previous layer.

[0044] The initialized potential state vectors are denoted as R1 and R2, which correspond to the two types of information processed by the model: time series information, which comes from the improved GRU output A1, and cross-sectional information, which comes from the GAT output A2. The dimensions of R1 and R2 are both (d r ,d i ).

[0045] Furthermore, the core of the field latent state learning process is the multi-head cross attention mechanism. This mechanism enables the latent state vector to interact with the outputs of the improved GRU and GAT layers, effectively absorbing relevant information in these outputs and optimizing the latent state representation. The multi-head cross attention between R1 and A1 is specifically:

[0046] In the multi-head cross-attention mechanism, the interaction between A1 and R1 can be modeled through the cross-attention process, where A1 serves as the query and R1 as the key and value. The specific operation is as follows:

[0047] Linear transformation: For each attention head i, the attention score between A1 (query) and R1 (key and value) is calculated as follows:

[0048]

[0049] in, And set d h =d i For the head, is the learnable weight matrix of the query, is the learnable weight matrix of the keys, is a learnable weight matrix of values. in is the dimension of each attention head, and h' is the number of attention heads.

[0050] Scaled dot product attention: For each head i, compute the attention score between query A1 and key R1:

[0051]

[0052] in, Given the attention weight, is a scaling factor to avoid excessively large dot product values. The softmax function ensures that the sum of the attention weights is 1.

[0053] Concatenation of attention heads: After calculating the attention of each head, concatenate the outputs of all heads:

[0054]

[0055] Among them, W O Is a learnable matrix used to project the concatenated result into the output space. The final output Captures the complex relationship between two sets of features.

[0056] The multi-head cross attention between R2 and A2 is as follows: The secondary multi-head cross attention operation is similar, but this time it is performed between the latent state vector R2 and the GAT output A2. By a similar method, we can calculate After applying the multi-head crisscross attention mechanism, the outputs B1 and B2 are treated as enhanced latent state representations. These vectors now capture both the time series and cross-sectional characteristics of the market, making them powerful representations for subsequent tasks such as stock volatility prediction or market state recognition.

[0057] Furthermore, the model of the present application also includes a loss calculation layer, which is responsible for integrating the outputs of the aforementioned layers and generating prediction results. This layer also defines how the model is trained, by calculating the difference between the predicted value and the actual value, and minimizing this difference during the training process. The input of the loss calculation layer comes from three key components of the model: time series representation A1, cross-sectional representation A2, and potential market state representations B1 and B2. The time series representation A1 comes from the improved GRU layer, which captures the time series dependencies in the stock market data; the cross-sectional representation A2 comes from the GAT layer, which models the relationship between stocks; the potential market state representations B1 and B2 come from the market potential state learning layer, which represents the hidden market factors learned from the time series and cross-sectional data. In order to prepare the final prediction, the model splices these outputs into a comprehensive feature vector that integrates all relevant information. The splicing can be expressed as:

[0058] Z=Concat(A1,A2,B1,B2)

[0059] in, is the comprehensive feature vector used for the final prediction, and d z =2d h +2d g , depends on the dimensions of A1, A2, B1 and B2.

[0060] Furthermore, the final prediction is made using the GAT layer as follows:

[0061] Once the feature vector Z is obtained, it is passed to additional GAT layers for the final prediction. The purpose of these layers is to optimize the concatenated features by considering the relationships between stocks (nodes), as GAT layers are well suited for modeling graph-structured data. The GAT layers in this section are similar to the previous GAT layers, but now they operate on a more comprehensive feature set that combines time series and cross-sectional information, as well as a representation of the underlying market state. The structure of the GAT layer is as follows:

[0062] Graph Construction: The graph structure is the same as the earlier GAT layer. Nodes represent individual stocks, and edges represent relationships between stocks based on historical correlations over the past year. These correlations are judged by value The threshold is an adjustable parameter that determines which relationships are included in the graph.

[0063] Attention Mechanism: The attention mechanism calculates the importance of each stock’s neighbors using the following formula:

[0064]

[0065] Feature aggregation:

[0066] The final output of each stock is calculated by weighted summation of its neighbor features, where the attention weight σ' ij Determines the contribution of each neighbor:

[0067]

[0068] in The information from neighboring stocks is effectively integrated to form a refined feature representation that blends time series, cross-sectional, and underlying market characteristics.

[0069] Dimensionality reduction: The output Z' of the first GAT layer i Input to the second GAT layer for dimensionality reduction. The calculation method is the same as above, and the final output is Z' i ' As the prediction result for each stock.

[0070] Loss Function: After obtaining the final prediction from the GAT layer, the next step is to calculate the loss, which measures the difference between the stock returns predicted by the model and the actual values. The choice of loss function is crucial because it guides the model's training process and affects its performance. In this model, we use the mean squared error (MSE) for the stock prediction task. It is defined as:

[0071]

[0072] where Z' i ' is the predicted value of stock i, y i is the actual value. During the training process, the model parameters are optimized to minimize the selected loss function, thereby improving the prediction performance. The present invention uses Adam as a gradient-based optimization algorithm to update the model parameters.

[0073] The number of devices and processing scales described herein are intended to simplify the description of the present invention, and the application, modification, and variation of the present invention will be apparent to those skilled in the art. Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiment. They can be applied to various fields suitable for the present invention. For those skilled in the art, additional modifications can be easily implemented. Therefore, the present invention is not limited to the specific details and shown here without departing from the general concept defined by the claims and their equivalents.

Claims

1. A stock prediction model based on multi-head cross-attention mechanism and improved GRU, characterized by: include: An improved GRU model, wherein the improved GRU model includes an attention mechanism, wherein the attention mechanism is used to replace a reset gate; A graph attention network, wherein the graph attention network is used to capture the relationship between different stocks to extract cross-sectional features of the data, and the graph attention network includes an attention mechanism, through which different weights are assigned to different nodes according to the relationship between the stocks; A multi-head cross-attention mechanism is used to interact with the outputs of the improved GRU model and the graph attention network, thereby effectively absorbing relevant information in the outputs of the improved GRU model and the graph attention network and optimizing the potential state representation.

2. A stock prediction model based on a multi-head cross-attention mechanism and an improved GRU as claimed in claim 1, characterized in that: The attention mechanism in the improved GRU model assigns attention weights by calculating the similarity between query, key, and value, where the hidden state of the previous time step is is considered as a query, the input x at the current time step t As keys and values.

3. A stock prediction model based on a multi-head cross-attention mechanism and an improved GRU as claimed in claim 2, characterized in that: The attention mechanism assigns attention weights by calculating the similarity between queries, keys, and values. Specifically, it includes the following steps: The hidden state h t-1 and input x t Linearly transform to query, key and value spaces respectively; Attention weights are obtained by calculating the dot product similarity between the query and the key; The final reset gate value is obtained by the weighted sum of the values, weighted by the attention weights.

4. A stock prediction model based on a multi-head cross-attention mechanism and an improved GRU as claimed in claim 3, characterized in that: The improved GRU model generates the final hidden state sequence through recursive calculation of multiple time steps, and the last hidden state h t As the representation vector of the entire sequence, the representation vector is used in further feature extraction and model learning stages.

5. The stock prediction model based on the multi-head cross-attention mechanism and the improved GRU according to claim 1, characterized in that: The graph of the graph attention network is constructed in such a way that each node represents a stock, and the edges between nodes represent the relationship between stocks; And the weight of the edge, that is, the strength of the relationship, is determined by calculating the historical correlation of stock returns, which is calculated by the Pearson correlation coefficient or other statistical measures.

6. A stock prediction model based on a multi-head cross-attention mechanism and an improved GRU as claimed in claim 5, characterized in that: The attention mechanism of the graph attention network is used to dynamically calculate the influence of each node's neighbors when aggregating information, that is, the attention coefficient, which specifically includes the following: The feature vector x of each node it Through the learnable weight matrix W g Perform linear transformation; Calculate the attention coefficient between stock i and its neighbor j; The attention coefficient is normalized across all neighbors of node i through the softmax function; The update of a node is represented by the weighted sum of the transformed features of its neighbors, where the weights consist of the normalized attention coefficients. An improved GRU stock prediction model is characterized in that the attention mechanism in the graph attention network is used for.

Images