Power communication optical cable DAS signal event recognition method based on spatiotemporal feature extraction

Abstract

The method for identifying DAS signal events of power communication optical cable based on spatiotemporal feature extraction relates to the technical field of distributed power optical fiber sensor signal identification, and solves the problems of large amount of calculation, unstable identification performance and identification confusion of events with similar spatiotemporal features in the existing network during DAS signal identification. The present invention constructs a DAS signal optical fiber vibration event data set; constructs a 1DCNNs‑gMLP model, and uses the DAS signal optical fiber vibration event data set to train the 1DCNNs‑gMLP model; obtains the trained 1DCNNs‑gMLP model; and performs DAS signal identification. The present invention fully extracts the features of spatiotemporal signals, establishes the dependency and globality between the features, thereby improving the accuracy of DAS signal identification and reducing the problem of event identification confusion caused by similar spatiotemporal features.

Description

Technical Field

[0001] The present invention relates to the technical field of distributed power optical fiber sensor signal recognition, and in particular to a method for identifying power communication optical cable DAS signal events based on spatiotemporal feature extraction. Background Art

[0002] Power communication optical cable is the core foundation of the power communication network. It carries important services such as line protection, dispatching data network, information wide area network, dispatching exchange, and power consumption information collection. The stable operation of optical cable has become an important part of power grid security. If the optical cable fails, it will not only affect the transmission of power communication information, but also bring inestimable economic losses to the power system.

[0003] Existing online monitoring equipment for power communication optical cables mainly includes video monitoring equipment, tension sensors, micro-meteorological sensors and temperature sensors. These electronic sensors have shortcomings such as poor power supply reliability, insufficient monitoring range and susceptibility to weather, making it difficult to meet the current high reliability and long-distance monitoring requirements. It is very convenient to use distributed optical fiber sensing technology to detect optical cables. Power communication optical cables themselves have optical fibers, which can be directly used as sensors to connect to distributed sensing systems for real-time monitoring. The principle is that the distributed optical fiber sensing system uses the process of light transmission in the optical fiber, which will be affected by the optical fiber state such as the refractive index, causing the propagation state of the light to change. By monitoring the changes in signals such as light intensity, the vibration state of the position, intensity, etc. of the sensing signal on the optical fiber can be reflected, thereby demodulating the physical quantity to be measured.

[0004] The signal collected by the distributed optical fiber vibration sensing (DAS) system based on the phase-sensitive optical time domain reflectometry principle (Ф-OTDR) is the DAS signal. It is very meaningful and valuable to study the DAS signal event recognition method, which can be applied to the actual monitoring of power communication optical cable projects. However, in practical applications, the DAS signal event recognition capability has always been a bottleneck restricting its widespread application. In a complex and changeable environment, there will be a variety of interfering noise signals, which can easily cause the vibration signal to be submerged in the noise signal, and harmless human or machine interference may cause a high nuisance alarm rate. Therefore, studying the DAS signal event recognition algorithm and improving the recognition ability of DAS signal events are of great significance for monitoring power communication optical cables.

[0005] There have been many studies on the current DAS signal event recognition methods. In the early stage, the method of "fixed artificial feature extraction combined with a specific recognition model" was mainly used. This method cannot adapt to the data changes of a large number of distributed nodes and has poor adaptability in the actual application environment. In recent years, drawing on the application of deep learning in the fields of image / speech, people have begun to use deep learning networks to extract and abstract DAS signal event features for detection and recognition. It can automatically extract features, and its performance is more stable than machine learning methods in complex environments, and its recognition accuracy is significantly improved compared to machine learning.

[0006] Currently, convolutional neural network (CNN) is the most commonly used method for DAS signal event recognition. Because DAS signals have rich temporal information, most DAS signal event recognition method studies mainly extract the temporal features of the signal at each sampling node for recognition, while ignoring the spatial relationship of the signal at each spatial node. 1DCNN mainly focuses on temporal features and ignores spatial features. 2DCNN uses image processing to extract local spatiotemporal features. In 2020, IEEE Access "A new DAS signal recognition method based on 1DCNNs-BiLSTM network spatiotemporal information extraction" proposed multiple groups of parallel one-dimensional convolution combined with bidirectional long short-term memory network (1DCNNs-BiLSTM), using 1DCNNs to extract the temporal features of the vibration signal of each spatial node and using BiLSTM to mine its spatial relationship. Since the memory unit of BiLSTM has a forgetting mechanism, it is easy to forget the earlier input information and the amount of calculation is large, resulting in unstable recognition performance and confusion in the recognition of events with similar spatiotemporal features.

[0007] Inspired by the above 1DCNNs-BiLSTM model, the 1DCNNs-gMLP model is proposed. Multiple sets of parallel 1DCNNs are used to extract the temporal features of the vibration signal of each spatial node, and gMLP is used to mine the spatial relationship of the signal on the node. The spatial gating unit (SGU) of gMLP can perform spatial interaction on different feature vectors on the node, and choose to amplify or suppress some spatial features according to the importance of the features. Therefore, 1DCNNs-gMLP can better focus on the differences between different events and improve the problem of event confusion caused by the similarity of their spatiotemporal features. Summary of the invention

[0008] In order to solve the problems of large amount of calculation, unstable recognition performance and recognition confusion of typical power optical cable vibration events such as digging, knocking, shaking and other events with similar spatiotemporal characteristics in the existing network during the DAS signal event recognition of power communication optical cables, the present invention provides a power communication optical cable DAS signal event recognition method based on spatiotemporal feature extraction.

[0009] A method for identifying DAS signal events in power communication optical cables based on spatiotemporal feature extraction is implemented as follows:

[0010] Construct a DAS signal optical fiber vibration event dataset;

[0011] Constructing a 1DCNNs-gMLP model, and using the DAS signal optical fiber vibration event data set to train the 1DCNNs-gMLP model; obtaining a trained 1DCNNs-gMLP model;

[0012] The trained 1DCNNs-gMLP model is used to identify DAS signals. The specific process is as follows:

[0013] 1DCNNs are used to extract the temporal features of DAS signals;

[0014] The spatial features of DAS signals were extracted using gMLP;

[0015] The temporal and spatial features of the DAS signal are classified through a fully connected layer to achieve DAS signal recognition.

[0016] Beneficial effects of the invention: The invention provides a method for fully mining the spatiotemporal information of DAS signal events of power communication optical cables, and by fully extracting spatiotemporal features, establishing dependencies and globality between features, the accuracy of DAS signal recognition is improved, and the problem of event recognition confusion caused by similar spatiotemporal features is reduced. It has the following advantages:

[0017] 1. In the DAS signal recognition method described in the present invention, an end-to-end 1DCNNs-gMLP method is proposed, which can fully exploit the temporal and spatial characteristics of DAS signals. This method uses 1DCNNs to extract the temporal characteristics of the vibration signals of each spatial node, and uses the spatial gating unit (SGU) in gMLP to mine the spatial feature relationship between spatial sampling nodes. Because its SGU can selectively amplify or suppress certain spatial features according to the importance of the features, it is more conducive to distinguishing vibration events with similar temporal and spatial features.

[0018] 2. The model structure and optimal parameters of 1DCNNs-gMLP for DAS signal vibration event detection and recognition were determined through parameter selection experiments; the rationality of combining 1DCNNs and gMLP was verified by comparing 1DCNNs-gMLP with 1DCNNs, gMLP, and 2DCNN-gMLP through ablation experiments; the ability of 1DCNNs-gMLP to extract spatiotemporal features was compared with other DAS signal recognition models (1DCNNs model that only extracts time features, 2DCNN model that processes DAS signal features in the form of images, and 1DCNNs-BiLSTM model that has already mined the time and space features of DAS signals) to verify that the 1DCNNs-gMLP model has better spatiotemporal feature extraction capabilities than the compared models, thereby better distinguishing events; by comparing the convergence speed, recognition accuracy, and ability to distinguish various types of events with other DAS signal recognition models, it was verified that 1DCNNs-gMLP has a faster convergence speed, higher recognition accuracy, and reduces the problem of event confusion caused by similar spatiotemporal features compared to other compared models. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flow chart of events for identifying power communication optical cable DAS signal events based on spatiotemporal feature extraction according to the present invention;

[0020] Figure 2 This is the structure diagram of the 1DCNNs-gMLP model built based on this experimental data;

[0021] Figure 3 The ablation experiment of the 1DCNNS-gMLP model compares the recognition accuracy of 1DCNNs, gMLP, and 2DCNNs-gMLP;

[0022] Figure 4 This is the effect diagram of 1DCNNs;

[0023] Figure 5 This is the effect diagram of 2DCNN;

[0024] Figure 6 This is the effect diagram of 1DCNNs-BiLSTM;

[0025] Figure 7 This is the effect diagram of 1DCNNs-gMLP in the method of the present invention;

[0026] Figure 8 This is a graph comparing the convergence speed and recognition accuracy of 1DCNNs-gMLP with other current DAS signal recognition models;

[0027] Fig. 9The figures are to compare the confusion degree effect diagrams of DAS signal event recognition of each recognition model; among them, (a) is the effect diagram of 1DCNNs, (b) is the effect diagram of 2DCNN, (c) is the effect diagram of 1DCNNs-BiLSTM, and (d) is the effect diagram of 1DCNNs-gMLP in the method of the present invention. DETAILED DESCRIPTION

[0028] Combination Figures 1 to 9 This embodiment describes a method for identifying power communication optical cable DAS signal events based on spatiotemporal feature extraction. This method fully analyzes the temporal and spatial features of DAS signals and mines the relationship between deeper features to improve the event type identification accuracy of DAS signals. Figure 1 As shown, the method of this embodiment includes the steps of data collection, extracting time features from the processed data using 1DCNNs, mining the spatial relationship between features using gMLP, and identifying event types using a fully connected layer. The specific implementation process is as follows:

[0029] Step 1: Collect the spatiotemporal matrix signal of the optical fiber vibration event, and cut the matrix signal to retain the data of the effective spatial sampling points; then mark the data of the sampling points with event type labels to construct the optical fiber vibration event data set; specifically:

[0030] The Ф-OTDR distributed sensing optical fiber system is used to collect signal data of interference events. The optical fiber vibration data of six types of events, including background noise, digging, knocking, watering, shaking, and walking, are collected. The collected signals are labeled according to their interference types, namely 0, 1, 2, 3, 4, and 5.

[0031] In this embodiment, the detection optical fiber in the Ф-OTDR distributed sensing optical fiber system uses a 5 / 10km bare optical fiber. The detection optical cable is placed in a soundproof box to reduce external interference. The detection optical fiber is connected to a 100-meter trailing optical fiber, and the first 50 meters of the trailing optical fiber is used to collect data. When the system collects data samples, the parameters are set to an output wavelength of 1550.12nm, an output power of 15dBm, an optical fiber length of 5km or 10km, a repetition frequency of 12.5kHz or 8kHz, a pulse width of 400ns, and a sampling rate of 10MSamp / s. The spatiotemporal signal samples of each event consist of 10,000 points in the time domain, and the acquisition time is 1.25s. The 12 adjacent points near the interference position are removed at the spatial node. The single sample is in the form of a two-dimensional array with a size of 10,000×12, including 3,094 background noises, 2,512 digging, 2,530 knocking, 2,298 watering, 2,728 shaking, and 2,450 walking samples. There are 15,162 samples in total for the six types of events. And it is used as a fiber vibration event dataset (the above fiber vibration event dataset uses the dataset in the existing published literature);

[0032] The optical fiber vibration event data set is randomly extracted as the training data and test data of the experiment in a ratio of 8:2, and there is no overlap between the two. The collected signal is in the form of a two-dimensional space-time matrix;

[0033] X∈R 12×10000

[0034] Where 12 is the number of spatial sampling points taken for data acquisition, and 10000 is the time series length of the signal at each spatial sampling point.

[0035] Step 2: Construct a 1DCNNs-gMLP model; combine the two network structures of 1DCNNs and gMLP to let them play their respective advantages to fully extract spatiotemporal information. Let the denoised DAS signal dataset be trained and tested in the constructed 1DCNNs-gMLP. The 1DCNNs-gMLP method can be divided into three parts. First, 1DCNNs is used to extract the temporal characteristics of the signal at each node. Secondly, gMLP is used to mine the spatial feature relationship between the signals at each spatial sampling node. Finally, the fully connected layer is used for event classification. Observe the model convergence and recognition accuracy, determine the best model parameters of 1DCNNs-gMLP under its dataset, and then compare the model recognition performance with the currently popular DAS signal recognition method.

[0036] Step 2-1: Figure 2As shown in the figure, in this embodiment, 1DCNNs are used to extract the time features of the DAS signal. A group of 1DCNNs is used to extract the time features of the vibration signal at a single spatial sampling node, and multiple groups of parallel 1DCNNs are used to extract the signal time features at each sampling node, which are recorded as 1DCNNs.

[0037] 1DCNNs are used to extract the temporal features of the signals at each spatial node, specifically:

[0038] Assume that the input time series signal is x, where x is a vector with a length of 10,000 at a spatial sampling point. Use a convolution kernel of length K to perform a convolution operation on x to obtain a new feature vector y. The convolution operation can be expressed as:

[0039] y[i]=f(x[i:i+K-1]×w+b)

[0040] Among them, x[i:i+K-1] represents a subsequence of the input signal, w is the weight of the convolution kernel, b is the scalar bias, and f is the activation function ReLU.

[0041] After the convolution operation extracts features, the maximum pooling operation is performed. Max pooling is a commonly used pooling operation used to reduce the dimension of the feature vector. In this problem, the maximum pooling is selected to extract the most significant features of each feature vector.

[0042] z[i]=max(y[i])

[0043] Among them, z[i] is the feature value after pooling at the i-th location.

[0044] By performing multiple sets of one-dimensional convolutions in parallel, the temporal features of each spatial node can be extracted independently. Suppose that the output feature vector X after a convolution layer of 1DCNNs is:

[0045] X∈R n×r

[0046] In the formula, n is the number of spatial nodes and the length of the feature sequence is r;

[0047] Step 2-2: In order to mine the spatial relationship between signals on spatial nodes, the gMLP network is used. The deeper feature vector X processed by 1D-CNNs in step 2-1 is used as input. The gMLP uses the spatial gating unit (SGU) to realize the spatial interaction between the feature vectors of different sampling nodes, thereby mining the overall spatial connection. The specific process is as follows:

[0048] In this implementation, the feature vector X is normalized before entering the SGU, and the normalized feature vector is mapped to a higher-dimensional feature space using a linear transformation, and its nonlinear ability is enhanced by an activation function as shown in the formula:

[0049] Z=δ(XU)

[0050] Where δ represents the activation function GeLU, and U is defined as the linear projection along the channel dimension.

[0051] The SGU first divides the input feature vector into Two feature sequences Z1 and Z2 of equal channel dimensions. To multiply feature sequences Z1 and Z2 element by element to make them interact with each other, the spatial dimension contraction operation must be performed in SGU, and Z2 is linearly projected to introduce spatial information. The simplest linear projection is as follows:

[0052] f w,b =QZ2+b1

[0053] Where Q∈R n×n is a matrix whose size is the same as the number of spatial nodes, which is n. b1 is the deviation of the specified sequence. Z2 is f after linear projection. w,b , f w,b When multiplying Z1 element by element, S(·) represents the output of the linear gate, where ⊙ represents element by element multiplication, and its formula is:

[0054] S(Z)=Z1⊙f w,b (Z2)

[0055]

[0056] The feature sequence after spatial gating When using linear transformation to perform channel projection operation, the features are mapped back to the original dimension, and V defines a linear projection with a length equal to the channel dimension, as shown in the formula:

[0057]

[0058] The last projection vector Y is fused with the input feature vector X residual connection to calculate the new vector Y with spatial interaction n ∈R n×r , as the input of the next layer of gMLP, the number of gMLP layers is set according to the experimental data, as shown in the formula:

[0059] Y n =Y+X

[0060] Finally, the output Y of the last layer of gMLP nThe feature vector is averaged and converted into a one-dimensional array. The formula is as follows:

[0061]

[0062] Finally, its spatiotemporal feature vector L is placed in the fully connected layer for classification:

[0063] q=w1L+b2

[0064] P=βq

[0065] In the formula, w1 is the weight matrix, b2 is the vector bias, and β is the activation function Softmax. Finally, a 1×T probability vector is obtained to represent the inferred probability value P of the corresponding sample under T category = [p1,p2,...,p T ], in this embodiment, T=6.

[0066] The method described in this embodiment obtains a feature vector X of a sample through signal acquisition and cropping and splicing, and extracts the signal time features at each sampling node through multiple sets of parallel 1DCNNs, which are recorded as 1DCNNs. The time features of the DAS signal mined by 1DCNN are used as the input of the gMLP model, and the spatial gating unit (SGU) in the gMLP is used to perform spatial interactions between feature vectors at different sampling nodes, and the relationship between the vibration signals at each sampling node in terms of spatial features is mined. By combining two 1DCNNs with different purposes with the gMLP network, each part can give full play to its advantages to fully extract the time and space information of the DAS signal, thereby improving the accuracy of DAS signal event recognition.

[0067] Combination Figure 3-Figure 9 This embodiment is described, and the rationality of the design of the 1DCNNs-gMLP described in this embodiment is verified; the performance of 1DCNNs-gMLP is compared with that of 1DCNN, 2DCNN, and 1DCNNs-BiLSTM commonly used in current DAS signal recognition, and it is verified that 1DCNNs-gMLP has better recognition performance.

[0068] like Figure 3As shown in the figure, in order to verify the rationality of the proposed 1DCNNs-gMLP, ablation experiments were conducted with the 1DCNNs model that only extracts the temporal features of the signal on each spatial node, the gMLP model that focuses on spatial features, and the 2DCNN-gMLP model that uses 2DCNN as an image processing to extract local features and then uses gMLP to mine spatial connections. The highest recognition accuracy of 1DCNNs-gMLP, 1DCNNs, gMLP, and 2DCNN-gMLP are 99.71%, 95%, 93.71%, and 90.91%, respectively, and the convergence speed of 1DCNNs-gMLP is significantly faster than that of the comparison model, thus verifying the rationality of combining 1DCNNs with gMLP.

[0069] like Figure 4-Figure 7 As shown in the figure, in order to more intuitively observe the discrimination of the four popular models of DAS signals for events, the features learned from the sample data by the comparison model and its optimal model are visualized respectively. The high-dimensional features of the above four trained networks are simplified into three-dimensional space through the linear discriminant analysis (LDA) algorithm. Among the four compared models, 2DCNN has the worst feature extraction ability. 1DCNNs, 1DCNNs-BiLSTM and 1DCNNs-gMLP can distinguish basic events.

[0070] The feature extraction capabilities of 1DCNNs-BiLSTM and 1DCNNs-gMLP are stronger than those of 1DCNNs and 2DCNN. However, for the three types of events, background noise, knocking, and shaking, 1DCNNs-gMLP is better than 1DCNNs-BiLSTM in extracting spatiotemporal features.

[0071] like Figure 8 As shown in the figure, it can be seen that among the four deep learning methods, 1DCNNs-gMLP has the fastest convergence speed. After 3 cycles (epoch) training, the test accuracy of this method can reach 98.5%, and the recognition accuracy is the highest among the compared methods, with a maximum accuracy of 99.71%. 2DCNN has the slowest convergence speed, and it takes 50 epochs to reach convergence. The highest test accuracy of 2DCNN is 92.6%. The convergence speed and recognition accuracy of 1DCNNs and 1DCNNs-BiLSTM are at the middle level of the four methods, and their highest recognition accuracies are 95% and 98.31% respectively. Table 1 shows the performance index values ​​of the best models of each model. The evaluation indicators we choose are the normalized average recall rate (NAR), F1 score (F1-score) and average recognition accuracy (Accuracy). It can be intuitively seen that the 1DCNNs-gMLP method has the best performance in DAS signal event recognition.

[0072]

[0073] pass Fig. 9 The confusion matrix shows more clearly the ability of the four methods to distinguish events with similar spatiotemporal features in the vibration signals of power optical cables collected by distributed optical fiber sensing technology. In the compared deep learning models, 1DCNNs and 2DCNN still have a large number of events confused with each other. 1DCNNs-BiLSTM has a small number of events confused. 1DCNNs-gMLP is the best among the four methods in distinguishing events in the four models.

[0074] Compared with 1DCNNs-BiLSTM, 1DCNNs-gMLP reduces the problem of confusion in the identification of some events. For label 1, the background noise event is reduced by 0.8%, for label 2, the mining event is reduced by 1.99%, and for label 4, the watering event is reduced by 6.16%. For label 3, knocking, label 5, shaking, and label 6, the correct events identified by the two methods differ by no more than 1 sample.

[0075] Through the above experimental verification, 1DCNNs-gMLP makes full use of the spatiotemporal information of DAS signals, can better and faster identify various events of DAS signals of power communication optical cables, and can reduce the problem of confusion in the identification of DAS signals of power communication optical cables.

[0076] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0077] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for identifying DAS signal events in power communication optical cables based on spatiotemporal feature extraction, characterized by: The implementation process of this method is: Construct a DAS signal optical fiber vibration event dataset; Constructing a 1DCNNs-gMLP model, wherein the 1DCNNs-gMLP model consists of two network structures, 1DCNNs and gMLP; The 1DCNNs extracts the temporal features of the signal at each spatial node to obtain the feature vector X at each spatial node; Taking the feature vector X as the input of gMLP, and realizing the spatial interaction between feature vectors of different sampling nodes through the SGU of the gMLP; The SGU divides the input feature vector X into The two-part feature sequences Z1 and Z2 in the channel dimension; r is the length of the feature vector; The feature sequences Z1 and Z2 are multiplied element by element for feature interaction, and the feature sequence output by SGU is linearly transformed for channel projection. Finally, the spatiotemporal feature vector output by gMLP is averaged and placed in the fully connected layer for classification. Using the DAS signal optical fiber vibration event data set to train the 1DCNNs-gMLP model; obtaining the trained 1DCNNs-gMLP model; The trained 1DCNNs-gMLP model is used to identify DAS signals. The specific process is as follows: 1DCNNs are used to extract the temporal features of DAS signals; The spatial features of DAS signals were extracted using gMLP; The temporal and spatial features of the DAS signal are classified through a fully connected layer to achieve DAS signal recognition.

2. The method for identifying power communication optical cable DAS signal events based on spatiotemporal feature extraction according to claim 1 is characterized in that: The specific process of extracting the temporal features of DAS signals using 1DCNNs is as follows: A set of 1DCNNs is used to extract the temporal features of the vibration signal at a spatial sampling node, and multiple sets of parallel 1DCNNs are used to extract the temporal features of the signal at each sampling node.

3. The method for identifying power communication optical cable DAS signal events based on spatiotemporal feature extraction according to claim 2 is characterized in that: Set the input time series signal to x, use a convolution kernel with a length of K to perform a convolution operation on x, and obtain the feature vector y after the convolution operation; After the convolution operation is performed to extract features, the maximum pooling operation is performed to output the pooled feature values; After performing multiple sets of convolution operations in parallel, the feature vector X on each spatial node is extracted.

4. The method for identifying power communication optical cable DAS signal events based on spatiotemporal feature extraction according to claim 1 is characterized in that: The process of multiplying feature sequences Z1 and Z2 element by element for feature interaction is: The feature sequence Z2 is linearly projected as follows: f w,b =QZ2+b1 Where Q∈R n×n is a matrix, b1 is the deviation of the specified sequence, f w,b is the feature sequence after linear projection of Z2, f w,b Then multiply it element by element with Z1, which can be expressed as follows: S(Z)=Z1⊙f w,b (Z2) Where S(·) is the output of the linear gate, and ⊙ represents element-by-element multiplication; The feature sequence output by the linear gate Use linear transformation to perform channel projection operation and map the features back to the original dimension, which can be expressed as follows: Where the length of V is equal to the linear projection of the channel dimension; The last projection vector Y is fused with the input feature vector X by residual connection, and finally the spatiotemporal feature vector Y output by the last layer of gMLP is n The features are averaged and placed in a fully connected layer for classification.

5. The method for identifying power communication optical cable DAS signal events based on spatiotemporal feature extraction according to claim 4 is characterized in that: The spatiotemporal feature vector Y output by the last layer of gMLP n The features are averaged, that is, they are converted into a one-dimensional array form, and finally the spatiotemporal feature vector L in the one-dimensional array form is placed in the fully connected layer for classification.

Images