A spatiotemporal multi-granularity representation learning method for large-scale multivariate time series

Abstract

This invention discloses a spatiotemporal multi-granularity representation learning method for large-scale multivariate time series, specifically relating to the field of time series analysis technology. The method includes input information, elimination of redundant information between different granularities, adaptive spatiotemporal feature mining, and dynamic spatiotemporal multi-granularity information fusion. This invention fully mines and extracts important feature information in the time and spatial dimensions of large-scale multivariate time series, providing high-quality input representations for different downstream tasks. It designs a multi-granularity residual learning component to model multi-granularity information in the time dimension, eliminating redundant information between different granularities while mining the characteristics of multi-granularity data. Through an adaptive spatiotemporal graph attention method, it dynamically establishes spatial relationships between nodes and further achieves effective large-scale multivariate time series representation learning, providing high-quality representation information for downstream tasks.

Description

Technical Field

[0001] This invention relates to the field of time series analysis technology, and more specifically, to a spatiotemporal multi-granularity representation learning method for large-scale multivariate time series. Background Technology

[0002] Multivariate time series is a widely used type of data, prevalent in complex spatiotemporal systems. By recording the state changes of multiple observation points and the inherent relationships between them, multivariate time series can represent the complex states and changes of spatiotemporal systems in both time and space dimensions. Due to the complex patterns in both time and space, effective feature mining and modeling of multivariate time series is a very challenging task. In the time dimension, changes in time series often contain complex patterns such as nonlinearity and non-stationarity; in the spatial dimension, there are significant pattern dependencies between time series. Currently, with the continuous improvement of the number and efficiency of sensors, and the continuous enhancement of big data storage and computing power, the scale of observation and analysis of spatiotemporal systems is also constantly expanding, making the mining and modeling of large-scale multivariate time series a major challenge. In recent years, spatiotemporal graph neural networks and Transformers have achieved extensive research in the field of multivariate time series modeling. Spatiotemporal graph neural networks use graph structure as prior knowledge and graph convolution as a tool to fully explore the spatial relationships between different nodes. Transformers use self-attention as the main tool to model long-term dependencies between large-scale long-sequence contexts.

[0003] Large-scale multivariate time series data have two key characteristics: (1) multi-granularity: time series data can usually be obtained at different granularities according to different sampling intervals. Coarse-grained data has a large sampling interval and mainly reflects the overall trend and seasonality; fine-grained data has a small sampling interval and mainly reflects local details. Making full use of multi-granular data can bring more different information to the model, thereby improving the prediction effect; (2) spatial dependence: due to the significant increase in the number of spatial nodes, the difficulty of modeling spatial dependence has increased significantly. On the one hand, predefined graphs based on prior knowledge are difficult to fully model the dependence between different nodes. On the other hand, as the number of spatial nodes increases, the patterns of data from different nodes are complex and constantly changing. Effectively capturing the correlation between nodes can further improve the performance of the model. However, when conducting research on the characteristics of large-scale multivariate time series, it was found that the existing technology is not sufficient for mining multi-granular information in the time dimension and the dependence in the spatial dimension. First, the existing methods mainly use integration and splicing to utilize multi-granular information, ignoring the redundancy of information. Second, the existing predefined graphs based on prior knowledge cannot effectively mine the complex spatial relationships of nodes at a large scale, resulting in insufficient feature mining by the existing methods. In addition, existing technologies only design modeling frameworks for specific downstream tasks, making it difficult to extend the representational information mined by the model to other tasks. To solve the above problems, a technical solution is provided. Summary of the Invention

[0004] To overcome the aforementioned deficiencies of the prior art, this invention provides a spatiotemporal multi-granularity representation learning method for large-scale multivariate time series, thereby solving the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A spatiotemporal multi-granularity representation learning method for large-scale multivariate time series includes the following steps:

[0007] Step 1, Input Information: Extract features from the target image, obtain data of different granularities through time series sampling at different intervals, and input the data of different granularities from coarse to fine into the multi-granularity residual learning module;

[0008] Step 2, eliminate redundant information between different granularities: construct a multi-granularity residual learning module based on the basic characteristics of time series data. The functions of the multi-granularity residual learning module are to determine the location of redundant information between multiple granularities, match the feature dimensions of different granularities, and eliminate redundant information.

[0009] Step 3, Adaptive Spatiotemporal Feature Mining: Generate the connection relationships between spatial nodes, and realize the interaction of node information by constructing an adaptive spatiotemporal graph attention module;

[0010] Step 4, Dynamic Spatiotemporal Multi-granularity Information Fusion: The feature vector mined by attention-based spatiotemporal multi-granularity information fusion is used to obtain the final representation information for downstream modeling.

[0011] As a further aspect of the present invention, the modeling steps of the multi-granularity residual learning module are as follows:

[0012] Step 1, Data Preparation: The input to the multi-granularity residual learning module is multivariate time series with different granularities. Specifically, the multivariate time series with different granularities are:

[0013] ;

[0014] In the formula: The number of multi-granularity residual learning modules, For multivariate time series sets of different granularities, This is the coarsest granular data. For the finest granular data, The number of multivariate time series. for It is a dimension of tensor, The length of the window for looking back at history;

[0015] Step 2, Determining the location of redundant information among multiple granularities: A segmented sampling method is used to map the locations of coarse-grained and fine-grained redundant information, with the coarsest granular data... The dimension transformation is as follows:

[0016] ;

[0017] In the formula: Let be the dimension. The length of the coarsest-grained historical lookback window. and They are equal in size;

[0018] Based on the coarsest dimension, all fine-grained data are transformed into a three-dimensional tensor through piecewise sampling. The transformation of the fine-grained data is shown below:

[0019] ;

[0020] For the finest-grained data after transformation, the segmented sampling results in the first... subsequence The data format is as follows:

[0021] ;

[0022] In the formula: For the first subsequence set;

[0023] Step 3, Matching Feature Dimensions of Different Granularities: The finest-grained input data is transformed using segmented sampling techniques. Matrix correspondence is achieved through a multilayer perceptron, reducing the dimension of the finest-grained data from... Transform into And to achieve information embedding, the formula is as follows:

[0024] ;

[0025] ;

[0026] In the formula: It is a fine-grained feature. This is the weight matrix. For bias vectors, For activation function, The size after fine-grained feature embedding;

[0027] Step 4: Elimination of Redundant Information: Redundant information between fine-grained and coarse-grained data is eliminated through residual join. The formula for residual join is:

[0028] ;

[0029] In the formula: This serves as the input to the spatiotemporal adaptive graph attention module. The input features are after dimensionality mapping processing. This refers to the coarse-grained information that needs to be eliminated in the current fine-grained setting.

[0030] As a further aspect of the present invention, the dimension C of the finest-grained data is equal to the dimension C of the coarsest-grained data.

[0031] As a further aspect of the present invention, the dimensionality of different finest-grained data is kept consistent before the multi-granularity residual learning module is modeled.

[0032] As a further aspect of this invention, the steps for generating connection relationships between spatial nodes and realizing node information interaction by constructing an adaptive spatiotemporal graph attention module are as follows:

[0033] Step Z1, Modeling the spatial relationships between nodes: Initialize the diagonal matrix and randomly initialize the node embedding matrix. The values ​​of the node embedding matrix are iteratively obtained through neural network training. The formula for obtaining the spatial relationships between nodes is:

[0034] ;

[0035] ;

[0036] ;

[0037] In the formula: It is a two-dimensional matrix. It is a diagonal matrix. Embed the matrix for the node. This is the transpose of the node embedding matrix;

[0038] Step Z2, spatial interaction is achieved through an attention mechanism: For a missing node, the spatial association matrix with its connected nodes is calculated using the following formula:

[0039] ;

[0040] In the formula: It is a spatial incidence matrix. and They are nodes and nodes The characteristics are: concatenate refers to two tensors being joined together, and FC refers to a fully connected layer. These are the weighting coefficients;

[0041] Step Z3, Normalized Spatial Similarity: The formula for normalized spatial similarity is:

[0042] ;

[0043] In the formula: The normalized spatial correlation matrix is ​​represented by SoftMax and LeakyReLU, which are activation functions, respectively.

[0044] Step Z4, Reasoning of Node Representation Information: The spatial association matrix and input features are weighted and summed to achieve the reasoning of node representation information. The formula for weighted summation of the spatial association matrix and input features is as follows:

[0045] ;

[0046] In the formula: This represents the number of nodes connected to the missing node. The new feature information obtained after the missing node passes through the adaptive spatiotemporal graph attention module;

[0047] Step Z5, repeat: obtain the first step by repeating the above steps. Feature vectors mined at each granularity .

[0048] As a further aspect of the present invention, in step Z1, when the two-dimensional matrix... Okay, number When the value of a column is greater than 0, there is a connection between the two nodes; when the value of a column in a two-dimensional matrix is ​​greater than 0, there is a connection between the two nodes. Okay, number When the value of a column is less than or equal to 0, there is no connection between the two nodes.

[0049] As a further aspect of this invention, the final representation information for downstream modeling is obtained from the feature vector mined by spatiotemporal multi-granularity information fusion based on attention. The steps for obtaining the final representation information for downstream modeling are as follows:

[0050] Step Q1: Based on the spatiotemporal adaptive graph attention module, obtain representation information at different granularities:

[0051] ;

[0052] In the formula: This refers to sets of representational information at different granularities.

[0053] Step Q2: Fuse multi-granularity information from different time steps using a dynamic fusion strategy. The attention distribution for the features at the first time step is as follows:

[0054] ;

[0055] ;

[0056] In the formula: This is the feature vector of the first time step. This provides the characterization information for the m-th granularity at the first time step. The attention distribution of features at the first time step. Let m be the attention distribution of the m-th granularity among the features of the first time step;

[0057] Step Q3: Based on the attention distribution, obtain the representation information for the first time step through weighted summation:

[0058] ;

[0059] In the formula: This represents the information for the first time step. The feature vector at the first time step, The attention distribution of features at the first time step;

[0060] Step Q4: Repeat the above steps to obtain representation information at each time step, and use a stitching method and a multilayer perceptron to obtain representation information for downstream tasks. .

[0061] The technical effects and advantages of the spatiotemporal multi-granularity representation learning method for large-scale multivariate time series are as follows:

[0062] 1. This invention designs a multi-granularity residual learning component to model multi-granularity information in the time dimension, which can eliminate redundant information between different granularities while mining the characteristics of multi-granularity data;

[0063] 2. This invention addresses the problem of complex spatial relationships in large-scale multivariate time series by designing an adaptive spatiotemporal graph attention method, which enables the dynamic establishment of spatial relationships between nodes;

[0064] 3. Based on the designed dynamic spatiotemporal multi-granularity information fusion module and the self-supervised learning strategy, this invention further realizes effective large-scale multivariate time series representation learning, providing high-quality representation information for downstream tasks. Attached Figure Description

[0065] Figure 1 This is a flowchart illustrating a spatiotemporal multi-granularity representation learning method for large-scale multivariate time series according to the present invention. Detailed Implementation

[0066] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] A spatiotemporal multi-granularity representation learning method for large-scale multivariate time series includes the following steps:

[0068] Step 1, Input Information: Extract features from the target image, obtain data of different granularities through time series sampling at different intervals, and input the data of different granularities from coarse to fine into the multi-granularity residual learning module;

[0069] Step 2, eliminate redundant information between different granularities: construct a multi-granularity residual learning module based on the basic characteristics of time series data. The functions of the multi-granularity residual learning module are to determine the location of redundant information between multiple granularities, match the feature dimensions of different granularities, and eliminate redundant information.

[0070] Step 3, Adaptive Spatiotemporal Feature Mining: Generate the connection relationships between spatial nodes, and realize the interaction of node information by constructing an adaptive spatiotemporal graph attention module;

[0071] Step 4, Dynamic Spatiotemporal Multi-granularity Information Fusion: The feature vector mined by attention-based spatiotemporal multi-granularity information fusion is used to obtain the final representation information for downstream modeling.

[0072] The modeling steps of the multi-granularity residual learning module in this embodiment of the invention are as follows:

[0073] Step 1, Data Preparation: The input to the multi-granularity residual learning module is multivariate time series with different granularities. Specifically, the multivariate time series with different granularities are:

[0074] ;

[0075] In the formula: The number of multi-granularity residual learning modules, For multivariate time series sets of different granularities, This is the coarsest granular data. For the finest granular data, The number of multivariate time series. for It is a dimension of tensor, The length of the window for looking back at history;

[0076] Step 2, Determining the location of redundant information among multiple granularities: A segmented sampling method is used to map the locations of coarse-grained and fine-grained redundant information, with the coarsest granular data... The dimension transformation is as follows:

[0077] ;

[0078] In the formula: Let be the dimension. The length of the coarsest-grained historical lookback window. and They are equal in size;

[0079] Based on the coarsest dimension, all fine-grained data are transformed into a three-dimensional tensor through piecewise sampling. The transformation of the fine-grained data is shown below:

[0080] ;

[0081] For the finest-grained data after transformation, the segmented sampling results in the first... subsequence The data format is as follows:

[0082] ;

[0083] In the formula: For the first subsequence set;

[0084] Step 3, Matching Feature Dimensions of Different Granularities: The finest-grained input data is transformed using segmented sampling techniques. Matrix correspondence is achieved through a multilayer perceptron, reducing the dimension of the finest-grained data from... Transform into And to achieve information embedding, the formula is as follows:

[0085] ;

[0086] ;

[0087] In the formula: It is a fine-grained feature. This is the weight matrix. For bias vectors, For activation function, The size after fine-grained feature embedding;

[0088] Step 4: Elimination of Redundant Information: Redundant information between fine-grained and coarse-grained data is eliminated through residual join. The formula for residual join is:

[0089] ;

[0090] In the formula: This serves as the input to the spatiotemporal adaptive graph attention module. The input features are after dimensionality mapping processing. This refers to the coarse-grained information that needs to be eliminated in the current fine-grained setting.

[0091] By designing multi-granularity residual learning components to model multi-granularity information in the time dimension, it is possible to mine the characteristics of multi-granularity data while eliminating redundant information between different granularities.

[0092] In this embodiment of the invention, the dimension C of the finest-grained data is equal to the dimension C of the coarsest-grained data.

[0093] In this embodiment of the invention, the dimension of different finest-grained data is kept consistent before the multi-granularity residual learning module is modeled.

[0094] In this embodiment of the invention, the steps for generating the connection relationships between spatial nodes and realizing the interaction of node information by constructing an adaptive spatiotemporal graph attention module are as follows:

[0095] Step Z1, Modeling the spatial relationships between nodes: Initialize the diagonal matrix and randomly initialize the node embedding matrix. The values ​​of the node embedding matrix are iteratively obtained through neural network training. The formula for obtaining the spatial relationships between nodes is:

[0096] ;

[0097] ;

[0098] ;

[0099] In the formula: It is a two-dimensional matrix. It is a diagonal matrix. Embed the matrix for the node. This is the transpose of the node embedding matrix;

[0100] Step Z2, spatial interaction is achieved through an attention mechanism: For a missing node, the spatial association matrix with its connected nodes is calculated using the following formula:

[0101] ;

[0102] In the formula: It is a spatial incidence matrix. and They are nodes and nodes The characteristics are: concatenate refers to two tensors being joined together, and FC refers to a fully connected layer. These are the weighting coefficients;

[0103] Step Z3, Normalized Spatial Similarity: The formula for normalized spatial similarity is:

[0104] ;

[0105] In the formula: The normalized spatial correlation matrix is ​​represented by SoftMax and LeakyReLU, which are activation functions, respectively.

[0106] Step Z4, Reasoning of Node Representation Information: The spatial association matrix and input features are weighted and summed to achieve the reasoning of node representation information. The formula for weighted summation of the spatial association matrix and input features is as follows:

[0107] ;

[0108] In the formula: This represents the number of nodes connected to the missing node. The new feature information obtained after the missing node passes through the adaptive spatiotemporal graph attention module;

[0109] Step Z5, repeat: obtain the first step by repeating the above steps. Feature vectors mined at each granularity .

[0110] To address the complex spatial relationships in large-scale multivariate time series, an adaptive spatiotemporal graph attention method is designed to dynamically establish spatial relationships between nodes.

[0111] In step Z1 of this embodiment of the invention, when the two-dimensional matrix is... Okay, number When the value of a column is greater than 0, there is a connection between the two nodes; when the value of a column in a two-dimensional matrix is ​​greater than 0, there is a connection between the two nodes. Okay, number When the value of a column is less than or equal to 0, there is no connection between the two nodes.

[0112] This invention provides a method for obtaining final representation information for downstream modeling by using feature vectors mined from spatiotemporal multi-granularity information fusion based on attention. The steps for obtaining the final representation information for downstream modeling are as follows:

[0113] Step Q1: Based on the spatiotemporal adaptive graph attention module, obtain representation information at different granularities:

[0114] ;

[0115] In the formula: This refers to sets of representational information at different granularities.

[0116] Step Q2: Fuse multi-granularity information from different time steps using a dynamic fusion strategy. The attention distribution for the features at the first time step is as follows:

[0117] ;

[0118] ;

[0119] In the formula: This is the feature vector of the first time step. This provides the characterization information for the m-th granularity at the first time step. The attention distribution of features at the first time step. Let m be the attention distribution of the m-th granularity among the features of the first time step;

[0120] Step Q3: Based on the attention distribution, obtain the representation information for the first time step through weighted summation:

[0121] ;

[0122] In the formula: This represents the information for the first time step. The feature vector at the first time step, The attention distribution of features at the first time step;

[0123] Step Q4: Repeat the above steps to obtain representation information at each time step, and use a stitching method and a multilayer perceptron to obtain representation information for downstream tasks. .

[0124] Based on the designed dynamic spatiotemporal multi-granularity information fusion module and the self-supervised learning strategy, effective large-scale multivariate time series representation learning is further realized, providing high-quality representation information for downstream tasks.

[0125] This invention fully mines and extracts important feature information in the time and spatial dimensions of large-scale multivariate time series, providing high-quality input representations for different downstream tasks. It designs a multi-granularity residual learning component to model multi-granular information in the time dimension, eliminating redundant information between different granularities while mining multi-granular data characteristics. Through an adaptive spatiotemporal graph attention method, it dynamically establishes spatial relationships between nodes and further achieves effective large-scale multivariate time series representation learning, providing high-quality representation information for downstream tasks. This addresses the problem of insufficient data analysis and mining in existing technologies and improves the scalability of the model's application value.

[0126] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0127] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A spatiotemporal multi-granularity representation learning method for large-scale multivariate time series, characterized in that, Includes the following steps: Step 1, Input Information: Extract features from the target image, obtain data of different granularities through time series sampling at different intervals, and input the data of different granularities from coarse to fine into the multi-granularity residual learning module; Step 2, eliminate redundant information between different granularities: construct a multi-granularity residual learning module based on the basic characteristics of time series data. The functions of the multi-granularity residual learning module are to determine the location of redundant information between multiple granularities, match the feature dimensions of different granularities, and eliminate redundant information. Step 3, Adaptive Spatiotemporal Feature Mining: Generate the connection relationships between spatial nodes, and realize the interaction of node information by constructing an adaptive spatiotemporal graph attention module; Step 4, Dynamic Spatiotemporal Multi-granularity Information Fusion: The feature vector mined by attention-based spatiotemporal multi-granularity information fusion is used to obtain the final representation information for downstream modeling; The modeling steps for the multi-granularity residual learning module are as follows: Step 1, Data Preparation: The input to the multi-granularity residual learning module is multivariate time series with different granularities. Specifically, the multivariate time series with different granularities are: ； In the formula: The number of multi-granularity residual learning modules, For multivariate time series sets of different granularities, This is the coarsest granular data. For the finest granular data, The number of multivariate time series. for It is a dimension of tensor, The length of the window for looking back at history; Step 2, Determining the location of redundant information among multiple granularities: A segmented sampling method is used to map the locations of coarse-grained and fine-grained redundant information, with the coarsest granular data... The dimension transformation is as follows: ； In the formula: For dimension, The length of the coarsest-grained historical lookback window. and They are equal in size; Based on the coarsest dimension, all fine-grained data are transformed into a three-dimensional tensor through piecewise sampling. The transformation of the fine-grained data is shown below: ； For the finest-grained data after transformation, the segmented sampling results in the first... subsequence The data format is as follows: ； In the formula: For the first subsequence set; Step 3, Matching Feature Dimensions of Different Granularities: The finest-grained input data is transformed using segmented sampling techniques. Matrix correspondence is achieved through a multilayer perceptron, reducing the dimension of the finest-grained data from... Transform into And to achieve information embedding, the formula is as follows: ； ； In the formula: It is a fine-grained feature. This is the weight matrix. For bias vectors, For activation function, The size after fine-grained feature embedding; Step 4: Elimination of Redundant Information: Redundant information between fine-grained and coarse-grained data is eliminated through residual join. The formula for residual join is: ； In the formula: This serves as the input to the spatiotemporal adaptive graph attention module. The input features are after dimensionality mapping processing. This refers to the coarse-grained information that needs to be eliminated in the current fine-grained setting.

2. The spatiotemporal multi-granularity representation learning method for large-scale multivariate time series according to claim 1, characterized in that, Before modeling, the multi-granularity residual learning module ensures that the dimensionality of different finest-grained data is consistent.

3. The spatiotemporal multi-granularity representation learning method for large-scale multivariate time series according to claim 1, characterized in that, The steps to generate connections between spatial nodes and achieve node information interaction by constructing an adaptive spatiotemporal graph attention module are as follows: Step Z1, Modeling the spatial relationships between nodes: Initialize the diagonal matrix and randomly initialize the node embedding matrix. The values ​​of the node embedding matrix are iteratively obtained through neural network training. The formula for obtaining the spatial relationships between nodes is: ； ； ； In the formula: It is a two-dimensional matrix. It is a diagonal matrix. Embed the matrix for the node. This is the transpose of the node embedding matrix; Step Z2, spatial interaction is achieved through an attention mechanism: For a missing node, the spatial association matrix with its connected nodes is calculated using the following formula: ； In the formula: It is a spatial incidence matrix. and They are nodes and nodes The characteristics are: concatenate refers to two tensors being joined together, and FC refers to a fully connected layer. These are the weighting coefficients; Step Z3, Normalized Spatial Similarity: The formula for normalized spatial similarity is: ； In the formula: The normalized spatial correlation matrix is ​​represented by SoftMax and LeakyReLU, which are activation functions, respectively. Step Z4, Reasoning of Node Representation Information: The spatial association matrix and input features are weighted and summed to achieve the reasoning of node representation information. The formula for weighted summation of the spatial association matrix and input features is as follows: ； In the formula: This represents the number of nodes connected to the missing node. The new feature information obtained after the missing node passes through the adaptive spatiotemporal graph attention module; Step Z5, repeat: obtain the first step by repeating the above steps. Feature vectors mined at each granularity .

4. The spatiotemporal multi-granularity representation learning method for large-scale multivariate time series according to claim 3, characterized in that, In step Z1, when the two-dimensional matrix is... Okay, number When the value of a column is greater than 0, there is a connection between the two nodes; when the value of a column in a two-dimensional matrix is ​​greater than 0, there is a connection between the two nodes. Okay, number When the value of a column is less than or equal to 0, there is no connection between the two nodes.

5. The spatiotemporal multi-granularity representation learning method for large-scale multivariate time series according to claim 1, characterized in that, The final representation information for downstream modeling is obtained by fusing and mining feature vectors based on attention-based spatiotemporal multi-granularity information. The steps for obtaining the final representation information for downstream modeling are as follows: Step Q1: Based on the spatiotemporal adaptive graph attention module, obtain representation information at different granularities: ； In the formula: This refers to sets of representational information at different granularities. Step Q2: Fuse multi-granularity information from different time steps using a dynamic fusion strategy. The attention distribution for the features at the first time step is as follows: ； ； In the formula: This is the feature vector of the first time step. This provides the characterization information for the m-th granularity at the first time step. The attention distribution of features at the first time step. Let m be the attention distribution of the m-th granularity among the features of the first time step; Step Q3: Based on the attention distribution, obtain the representation information for the first time step through weighted summation: ； In the formula: This represents the information for the first time step. The feature vector at the first time step, The attention distribution of features at the first time step; Step Q4: Repeat the above steps to obtain representation information at each time step, and use a stitching method and a multilayer perceptron to obtain representation information for downstream tasks. .

Images