Spatial-temporal data prediction method fusing attention mechanism

Abstract

The invention discloses a spatio-temporal data prediction method fusing an attention mechanism, and belongs to the field of data prediction. The objective of the invention is to solve the problem that the data prediction precision of the Internet of Things is limited due to non-deep spatial-temporal feature extraction of a traditional prediction model. According to the technical scheme, firstly, self-adaptive denoising and reconstruction are carried out on original data through ensemble empirical mode decomposition, and non-stationarity and noise interference are inhibited; then, a time channel attention and causal time attention mechanism is embedded in the time feature extraction stage, and refined modeling of the multi-level time dependency relationship is achieved; in the spatial feature extraction stage, spatial channel attention and dynamic spatial attention mechanisms are combined, and static topology and dynamic feature similarity are fused to adaptively model spatial dependence; and finally, carrying out training and prediction through an EEMD-STAtt-STGCN integration model. According to the method, the precision and robustness of spatio-temporal data prediction are remarkably improved, and the generalization ability of the model to non-stationary nonlinear data is enhanced.

Description

Technical Field

[0001] This invention belongs to the field of data prediction and relates to a spatiotemporal data prediction method that incorporates an attention mechanism. Background Technology

[0002] Sensing data in smart cities exhibits distinct spatiotemporal coupling characteristics, simultaneously containing complex temporal dependencies and dynamic spatial correlations, posing a significant challenge to accurate prediction. In the temporal dimension, data sequences often display significant nonlinearity and nonstationarity, such as the morning and evening rush hour phenomena and holiday effects in traffic flow, and are frequently mixed with substantial noise interference. In the spatial dimension, the interactions between different urban units, such as traffic intersections or monitoring stations, are not fixed but dynamically evolve over time; for example, the direction of traffic flow during the morning rush hour may be completely opposite to that during the evening rush hour. This dynamic spatial dependency makes it difficult for traditional models based on fixed topological relationships to effectively capture their true patterns.

[0003] Despite the significant progress made by deep learning technology in the field of temporal prediction, existing models still have obvious limitations when directly applied to the complex scenarios mentioned above. Traditional temporal models are good at capturing temporal dependencies but often ignore spatial relationships; while the standard Spatio-Temporal Graph Convolutional Network (STGCN), although considering both spatio-temporal dimensions, suffers from problems such as insufficient feature extraction and inadequate dynamic dependency modeling ability when dealing with strong noise and complex dynamic patterns, due to its inherent static graph structure assumption and lack of fine-grained filtering mechanisms for multi-dimensional features and time steps. This limits further improvement in prediction accuracy.

[0004] Therefore, under the current technological background, there is an urgent need for a prediction method that can deeply integrate signal preprocessing technology and has the ability to extract refined spatiotemporal features, in order to adaptively handle the non-stationary characteristics of smart city data, while accurately capturing its complex temporal patterns and dynamic spatial interaction modes, thereby ultimately achieving more reliable and robust prediction results. This invention is proposed precisely to address this technological challenge. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention designs a spatiotemporal data prediction method that integrates an attention mechanism, aiming to solve the technical problem of low prediction accuracy caused by insufficient extraction of temporal and spatial features in traditional prediction models. For example... Figure 1 As shown, the method includes the following steps: S1. Obtain the IoT dataset and preprocess it using the improved EEMD method to obtain high-quality training and testing datasets. S2. In the temporal feature extraction stage, a temporal convolution mechanism with enhanced temporal attention is designed. Important feature dimensions are filtered through channel attention, and key time steps are dynamically focused on in combination with causal temporal attention to achieve refined modeling of multi-level temporal dependencies. S3. In the spatial feature extraction stage, a graph convolution mechanism with enhanced spatial attention is constructed. Through dual screening of channel attention and spatial attention, and combined with static topology and dynamic feature similarity, adaptive spatial dependency modeling is achieved. S4. An integrated model based on EEMD-STAtt-STGCN is constructed by embedding EEMD preprocessing, temporal attention-enhanced convolution, and spatial attention-enhanced convolution modules into the traditional STGCN structure, forming a prediction system of "denoising preprocessing - temporal modeling - spatial dependency optimization". This significantly improves the accuracy and robustness of spatiotemporal data prediction while maintaining the overall network structure.

[0006] In S1, the method includes an adaptive signal reconstruction step based on EEMD, wherein the input time-series signal is decomposed into several IMF components using EEMD; for each IMF component, significance indicators such as sample entropy, correlation coefficient with the original signal, and energy proportion are calculated to characterize the component's complexity, consistency with the main trend, and energy contribution to the overall signal, respectively; subsequently, the indicators are normalized, and the comprehensive score of each IMF component is calculated according to the comprehensive evaluation formula: (9) in, , , They represent the first Normalized sample entropy, correlation coefficient, and energy percentage of each IMF component. , , The weighting coefficients are set empirically; key IMF components are automatically selected based on the ranking results of the comprehensive scores and the adaptive threshold, and the key components are superimposed and reconstructed to obtain a denoised and feature-enhanced target signal for use by subsequent algorithm modules.

[0007] In S2, the method is characterized in that the temporal attention-enhanced temporal convolution mechanism includes two core components: temporal channel attention and causal temporal attention, and its processing flow is as follows: First, for the input time series feature map Intermediate features are obtained through one-dimensional causal convolution. Causal convolution can ensure that the model only uses information from the current and historical time steps, preventing the leakage of future information; Subsequently, a temporal channel attention mechanism is applied to recalibrate the importance of feature P. Specifically, this involves compressing each feature channel into a representative scalar over the entire temporal dimension using global average pooling, learning the importance weights of each channel through two fully connected layers, and finally multiplying the learned importance weights with the original features to recalibrate the feature channels, thereby enhancing important features and suppressing minor features. (10) (11) in This is a global average pooling operation. , For the weights of the fully connected layer, To reduce the ratio, It is the ReLU activation function. This is the Sigmoid activation function.

[0008] Next, a causal temporal attention mechanism is introduced to ensure that the model only focuses on information from the current and historical time steps. First, the calibrated features are queried, key-value projected, and then... Then, a causal mask is introduced when calculating the attention score: (12) Then, the temporal attention weights are obtained through Softmax normalization, and the value features are weighted and aggregated based on these attention weights. Finally, the temporal attention output is combined with the other branch feature Q through a Sigmoid gating mechanism, and the module output is added to the input feature through residual connections to ensure effective gradient propagation and preserve the original information.

[0009] This time-attention enhancement mechanism can dynamically focus on important feature dimensions and key time steps while ensuring causality, effectively capturing recent fluctuations, cyclical patterns and long-term trends in time-series data.

[0010] In S3, the method is characterized in that the spatial attention-enhanced graph convolution mechanism includes two core components: spatial channel attention and dynamic spatial attention, and its processing flow is as follows: First, regarding the output features of the graph convolutional layer Applying spatial channel attention mechanism to achieve node-level importance filtering: Global max pooling is used to extract the maximum value of all feature channels for each node. Then, node importance weights are learned based on two fully connected layers. Finally, the sigmoid function is used to generate node importance scores. These importance weights are then applied to the original features to enhance the features of important nodes. (13) (14) in , For spatial attention weights.

[0011] Subsequently, a dynamic spatial attention mechanism is constructed to generate dynamic adjacency relationships based on node feature similarity: Query and key projection of importance-weighted node features ( , Then, the query-key dot product is calculated to obtain the original similarity matrix, and a dynamic adjacency matrix is ​​generated by Softmax normalization. Finally, the dynamic adjacency matrix and the static topology matrix are adaptively fused. (15) in The prior adjacency matrix is ​​based on distance. These are learnable fusion parameters.

[0012] Then, spatial feature aggregation is performed based on the fused adjacency matrix: Value projection of importance-weighted node features ( Then, using the fused adjacency matrix as a guide, graph convolution operations are performed, followed by nonlinear activation and residual connections to stabilize the training process. Finally, layer normalization is applied to standardize the feature distribution. (16) This spatial attention enhancement mechanism can simultaneously utilize fixed topology and dynamic feature similarity to adaptively adjust the spatial dependencies between nodes, effectively capturing complex spatial interaction patterns that change over time in the spatial domain.

[0013] In S4, the network model is EEMD-STAtt-STGCN, which embeds EEMD preprocessing, temporal attention-enhanced convolution, and spatial attention-enhanced convolution modules on the basis of traditional STGCN.

[0014] First, the input IoT spatiotemporal data undergoes EEMD decomposition and adaptive reconstruction to obtain multi-scale, denoised, and stable signals. Then, the temporal feature extraction part combines channel attention and causal temporal attention to achieve feature dimension filtering and key time step focusing. The spatial feature extraction part integrates spatial channel attention and dynamic spatial attention, combining static topology and feature similarity to adaptively model spatial dependencies. The temporal and spatial attention modules are deeply integrated within the spatiotemporal convolutional block, forming a sequential processing flow of "temporal attention convolution—spatial attention convolution—temporal attention convolution," thereby achieving multi-level spatiotemporal feature collaborative optimization.

[0015] Through this structure, the model maintains the efficiency of STGCN while integrating EEMD denoising, temporal and spatial attention mechanisms, realizing a unified framework from data preprocessing to spatiotemporal feature modeling, which significantly improves the prediction accuracy and generalization ability for non-stationary and nonlinear data.

[0016] The beneficial effects of this invention are as follows: First, in terms of data preprocessing, the original spatiotemporal data is adaptively denoised and reconstructed using the ensemble empirical mode decomposition method, which effectively suppresses non-stationarity and noise interference, generating high-quality, smooth input signals. This lays a solid foundation for subsequent feature extraction, thereby enhancing the model's ability to process complex data.

[0017] Secondly, in terms of time feature modeling, thanks to the deep integration of time channel attention and causal time attention mechanisms, the model can dynamically filter important feature dimensions and focus on key time steps, achieving refined capture of multi-level time dependencies, and significantly improving the recognition accuracy of time series dynamics such as recent fluctuations, periodic patterns and long-term trends.

[0018] At the spatial feature extraction level, by combining spatial channel attention and dynamic spatial attention mechanisms, the model adaptively integrates static topological structure and dynamic feature similarity to accurately capture the interaction patterns between nodes that change with time and scene. This solves the limitations of traditional methods caused by fixed graph structures and enhances the flexibility of spatial dependency modeling.

[0019] Finally, the EEMD-STAtt-STGCN integrated model organically unifies preprocessing, temporal modeling, and spatial optimization to form an end-to-end prediction system. While maintaining computational efficiency, it significantly improves the robustness and generalization ability of the prediction results, making it suitable for diverse IoT scenarios in smart cities such as traffic flow and environmental monitoring, and ensuring the model has reliable practicality in real-world environments.

[0020] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0021] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a structural diagram of EEMD-STatt-STGCN in this invention; Detailed Implementation To provide a complete and clear description of the technical solution, application purpose, and advantages of this invention, a detailed description is provided below in conjunction with the accompanying drawings and specific examples. It should be understood that the examples provided in the drawings are not within the scope of protection of this invention, but are merely selected examples. Those skilled in the art can fully understand the intent of this invention based on the content of this specification, both partially and holistically. This invention can be widely implemented for different application purposes, and similar improvements can be made based on the viewpoints in this specification. All other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this invention.

[0022] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0023] Please see Figure 1 , Figure 1 This is a flowchart illustrating a spatiotemporal data prediction method incorporating an attention mechanism, provided as an embodiment of this application. The method aims to process common spatiotemporal data in smart cities (such as traffic flow, environmental monitoring indicators, and public facility load), achieving high-precision prediction through integrated preprocessing and attention mechanisms. The method specifically includes the following steps: S1: Adaptive signal reconstruction based on improved EEMD; S1.1 EEMD Decomposition: In this embodiment, the raw time-series signal of smart city IoT data is used. The input is the original non-stationary time-series signal. EEMD is used to perform multiple perturbation decompositions on the signal. Specifically, zero-mean Gaussian white noise is added to the signal during each decomposition process. Generate perturbation samples Then, empirical mode decomposition is performed on each perturbation sample to obtain several IMF components. This process is repeated N times, and the average of the IMF components of the same order under all samples is calculated to obtain the final EEMD decomposition result: (17) in The number of IMFs obtained from the decomposition.

[0024] S1.2 Significance Indicators and Comprehensive Score Calculation: After obtaining the IMF components of each order, a significance index is calculated for each component to quantify its importance in signal reconstruction. The selected significance indices include sample entropy, correlation coefficient, and energy proportion. Sample entropy measures the complexity and randomness of the signal component; noise components typically have higher sample entropy. The correlation coefficient measures the linear correlation between the IMF component and the original signal, reflecting the degree to which the component retains the main trend of the signal. The energy proportion measures the proportion of the IMF component's energy in the overall signal energy, thereby assessing the component's energy contribution.

[0025] To facilitate comparison and weighted fusion of indicators with different dimensions, the three significance indicators mentioned above were normalized to obtain normalized sample entropy. Normalized correlation coefficient and normalized energy percentage The overall score for each IMF component is then calculated using the following comprehensive evaluation formula: (18) in, , , The weighting coefficients are set empirically. This comprehensive score takes into account the complexity, correlation, and energy characteristics of IMF components, ensuring the rationality and stability of the screening results.

[0026] S1.3 IMF Screening and Signal Reconstruction: Based on the calculated composite score, each IMF component is sorted, and an adaptive threshold method is used to automatically determine the inflection point position by analyzing the distribution curve of the composite score, which serves as the boundary for component screening. The selected key IMF components are then superimposed and reconstructed to obtain the denoised signal. (19) in The number of selected IMFs. Through the above adaptive EEMD signal reconstruction process, the effective components containing the main information can be automatically retained and high-noise components can be removed without manual intervention, thereby generating a denoised and feature-enhanced target signal. This provides high-quality input for subsequent spatiotemporal feature extraction, significantly improving the accuracy and robustness of the prediction model.

[0027] S2: Temporal feature extraction with enhanced temporal attention; S2.1 Causal Convolution and Channel Attention: The reconstructed signal is used as the input feature. First, intermediate features are obtained through one-dimensional causal convolution. Causal convolution ensures that the model uses only information from the current and historical time steps, preventing the leakage of future information. Subsequently, a temporal channel attention mechanism is applied to recalibrate the importance of feature P. This involves compressing each feature channel into a representative scalar across the entire time dimension using global average pooling, and learning the importance weights of each channel through two fully connected layers. (20) (twenty one) in , For the weights of the fully connected layer, To reduce the ratio, It is the ReLU activation function. The Sigmoid activation function is used. Finally, the learned importance weights are multiplied by the original features to recalibrate the feature channels, enhancing important features and suppressing minor features. (twenty two) S2.2 Causal Temporal Attention: Features after Calibration Building upon this foundation, a causal temporal attention mechanism is introduced. First, the calibrated features are subjected to query, key-value projection: (twenty three) in It is a learnable projection matrix.

[0028] A causal mask is introduced when calculating the attention score to ensure that the model only focuses on information from the current and historical time steps, preventing the leakage of future information. (twenty four) Finally, the temporal attention weights are obtained through Softmax normalization, and the value features are then weighted and aggregated based on these attention weights. (25) S2.3 Gated Fusion and Residual Output: Combining Temporal Attention Output with Another Branch Feature By combining the Sigmoid gating mechanism with the residual connection, the module output is added to the input features to ensure effective gradient propagation and preserve the original information. (26) This time-attention enhancement mechanism can dynamically focus on important feature dimensions and key time steps while ensuring causality, effectively capturing recent fluctuations, periodic patterns and long-term trends in smart city time-series data.

[0029] S3: Spatial feature extraction with enhanced spatial attention; S3.1 Spatial Channel Attention: For the output features of graph convolutional layers A spatial channel attention mechanism is applied to achieve node-level importance filtering. Global max pooling is used to extract the maximum value from all feature channels of each node, forming a node description vector. (27) Subsequently, the importance weights of nodes are learned based on two fully connected layers: (28) in , This represents the spatial attention weights. A node importance score is generated using the Sigmoid function, and these importance weights are applied to the original features to enhance the features of important nodes. (29) S3.2 Dynamic Spatial Attention and Fusion: Constructing a dynamic spatial attention mechanism. This involves querying and projecting key features onto importance-weighted node features. (30) in This is the spatial attention projection matrix.

[0030] The query-key dot product is calculated to obtain the original similarity matrix, and then a dynamic adjacency matrix is ​​generated by Softmax normalization. (31) This matrix reflects the dynamic association strength between nodes based on real-time features. Next, the dynamic adjacency matrix is ​​combined with a static topology matrix constructed based on prior knowledge (geographical distance, road connections, etc.). Adaptive fusion is performed to form the final fused adjacency matrix: (32) in These are learnable fusion parameters.

[0031] S3.3 Spatial Feature Aggregation and Output: Based on the fused adjacency matrix, value projection is performed on the importance-weighted node features, and then graph convolution operation is executed to achieve spatial feature aggregation. (33) (34) Then, the ReLU activation function is applied to the aggregation result to introduce nonlinearity: (35) Finally, the training process is stabilized through residual connections and layer normalization: (36) This mechanism can simultaneously utilize fixed topology and dynamic feature similarity to adaptively adjust the spatial dependencies between nodes, effectively capturing the complex spatial interaction patterns that change over time in smart city networks.

[0032] S4: Prediction based on the integrated model of EEMD-STAtt-STGCN; Please see Figure 2 This is a structural diagram of EEMD-STatt-STGCN in this invention. This step integrates all the aforementioned modules into an end-to-end prediction model, EEMD-STAtt-STGCN. This model optimizes the traditional STGCN framework: Input end: The integrated S1 EEMD adaptive signal reconstruction module automatically denoises and smooths the original smart city spatiotemporal data.

[0033] The core of feature extraction is to embed the temporal attention enhancement module of S2 and the spatial attention enhancement module of S3 into the spatiotemporal convolutional block of the traditional STGCN. The model adopts an alternating stacked structure of "temporal convolution (with attention) - spatial convolution (with attention) - temporal convolution (with attention)" to achieve deep fusion and joint optimization of temporal and spatial features.

[0034] Output: The final extracted refined spatiotemporal feature map is input into a fully connected prediction layer, which outputs the predicted smart city data for a specific future time window.

[0035] This integrated model is trained end-to-end through backpropagation, enabling the EEMD preprocessing, temporal attention, and spatial attention modules to work together to optimize the prediction target, ultimately significantly improving the prediction accuracy and generalization ability for non-stationary and nonlinear spatiotemporal data sequences in smart cities.

Claims

1. A spatiotemporal data prediction method incorporating an attention mechanism, characterized in that, The method includes: S1. Data preprocessing stage: Obtain the original spatiotemporal data sequence of the Internet of Things, and use the ensemble empirical mode decomposition method to adaptively denoise and reconstruct the data to suppress non-stationarity and noise interference, and generate high-quality training and test datasets. S2, Temporal Feature Extraction Stage: The preprocessed data is input into the temporal attention enhancement module, which extracts temporal features through one-dimensional causal convolution and embeds temporal channel attention mechanism and causal temporal attention mechanism to dynamically filter important feature dimensions and key time steps, thereby achieving refined modeling of multi-level temporal dependencies. S3, Spatial Feature Extraction Stage: The temporal feature output is input into the spatial attention enhancement module, spatial features are extracted through graph convolution operation, and spatial channel attention mechanism and dynamic spatial attention mechanism are embedded. Combined with static topology and dynamic feature similarity, adaptive spatial dependency modeling is achieved. S4. Prediction Output Stage: Based on the EEMD-STAtt-STGCN integrated model, the temporal attention module and spatial attention module are deeply integrated into the spatiotemporal graph convolutional network structure to form an end-to-end prediction system. This system is used to train and predict IoT spatiotemporal data to improve prediction accuracy and robustness.

2. The spatiotemporal data prediction method incorporating an attention mechanism according to claim 1, characterized in that: In step S1, the ensemble empirical mode decomposition process includes decomposing the input time-series signal into multiple intrinsic mode function (IMF) components, calculating the sample entropy, correlation coefficient with the original signal, and energy proportion for each component as significance indicators, normalizing these indicators, and calculating the comprehensive score of each IMF component according to the comprehensive evaluation formula. (1) in, , , They represent the first Normalized sample entropy, correlation coefficient, and energy percentage of each IMF component. , , The weighting coefficients are set empirically. Based on the ranking results of the comprehensive score and the adaptive threshold, key IMF components are automatically selected and superimposed and reconstructed to obtain a denoised and feature-enhanced target signal for use by subsequent algorithm modules.

3. The spatiotemporal data prediction method incorporating an attention mechanism according to claim 1, characterized in that: In S2, for the input time series feature map Intermediate features are obtained through one-dimensional causal convolution. Causal convolution ensures that the model uses only information from the current and historical time steps, preventing the leakage of future information; Applying temporal channel attention mechanism to features Importance recalibration is performed by compressing each feature channel into a representative scalar over the entire time dimension using global average pooling, learning the importance weights of each channel through two fully connected layers, and finally multiplying the learned importance weights with the original features to recalibrate the feature channels, thereby enhancing important features and suppressing minor features. (2) (3) in This is a global average pooling operation. , For the weights of the fully connected layer, To reduce the ratio, It is the ReLU activation function. This is the Sigmoid activation function.

4. The spatiotemporal data prediction method incorporating an attention mechanism according to claim 1, characterized in that: In step S2, a causal temporal attention mechanism is introduced to ensure that the model only focuses on current and historical time step information; firstly, the calibrated features are queried. Q ,key K ,value V Projection, followed by the introduction of a causal mask when calculating the attention score: (4) Then, the temporal attention weights are obtained through Softmax normalization, and the value features are weighted and aggregated based on the attention weights. Finally, the temporal attention output is combined with another branch feature Q through the Sigmoid gating mechanism, and the module output is added to the input feature through residual connection to ensure effective gradient propagation and preserve the original information.

5. The spatiotemporal data prediction method incorporating an attention mechanism according to claim 1, characterized in that: In S3, the output features of the graph convolutional layer Applying spatial channel attention mechanism to achieve node-level importance filtering: Global max pooling is used to extract the maximum value of all feature channels for each node. Then, node importance weights are learned based on two fully connected layers. Finally, the sigmoid function is used to generate node importance scores. These importance weights are then applied to the original features to enhance the features of important nodes. (5) (6) in , For spatial attention weights.

6. The spatiotemporal data prediction method incorporating an attention mechanism according to claim 1, characterized in that: In step S3, a dynamic spatial attention mechanism is constructed to generate dynamic adjacency relationships based on node feature similarity. Querying and key projection of node features weighted by importance, i.e. and Then, the query-key dot product is calculated to obtain the original similarity matrix, and a dynamic adjacency matrix is ​​generated by Softmax normalization. Finally, the dynamic adjacency matrix and the static topology matrix are adaptively fused. (7) in The prior adjacency matrix is ​​based on distance. These are learnable fusion parameters.

7. The spatiotemporal data prediction method incorporating an attention mechanism according to claim 1, characterized in that: In step S3, spatial feature aggregation is performed based on the fused adjacency matrix: Value projection of importance-weighted node features Then, using the fused adjacency matrix as a guide, graph convolution operations are performed, followed by nonlinear activation and residual connections to stabilize the training process. Finally, layer normalization is applied to standardize the feature distribution. (8)。 8. The spatiotemporal data prediction method incorporating an attention mechanism according to claim 1, characterized in that: In S4, the EEMD-STAtt-STGCN integrated model embeds EEMD preprocessing, temporal attention module and spatial attention module within the spatiotemporal graph convolutional network framework, forming a sequential processing flow of "temporal attention convolution-spatial attention convolution-temporal attention convolution", realizing multi-level spatiotemporal feature collaborative optimization, and finally outputting the prediction result through a fully connected layer.

Images