A cross-domain behavior recognition method based on spatiotemporal frequency domain features and adversarial learning

Abstract

The present invention discloses a cross-domain behavior recognition method based on spatiotemporal frequency domain features and adversarial learning. First, the source domain and target domain sensor data are obtained, and a sliding window is used to segment the sensor data. The segmented sensor data is passed through a spatiotemporal frequency domain feature extraction network to obtain a source domain feature vector and a target domain feature vector. In the process of cross-domain behavior recognition, the spatiotemporal features in the sensor data are expanded and extracted using a spatiotemporal feature extraction network to obtain more sufficient spatiotemporal feature information. The frequency domain feature extraction network is used to fully mine the frequency domain features in the sensor data, and the frequency domain features between the target domain and the source domain are calculated to obtain a similarity matrix. Combined with adversarial learning, the interference of non-target behaviors on behavior recognition is suppressed, the feature distribution between the source domain and the target domain is shortened, the generalization ability of the model is enhanced, and the model is effectively recognized for cross-domain behaviors.

Description

Technical Field

[0001] The present application belongs to the technical field of sensor data behavior recognition, and specifically relates to a cross-domain behavior recognition method based on spatiotemporal frequency domain features and adversarial learning. Background Art

[0002] Behavior recognition based on sensor data is a highly anticipated and promising research field, which aims to perceive the motion characteristics of the human body through sensor data signals, and analyze and identify the current behavior state of the human body. In actual application scenarios, the distribution of sensor data from different users and different environments varies greatly, resulting in differences in the distribution between the training set and the test set during model training, making it impossible to effectively train the behavior recognition model. Therefore, cross-domain human action recognition is needed to improve the accuracy of the model.

[0003] In recent years, with the rapid development of the Internet of Things industry, the popularization of technologies such as wireless sensor networks and wearable devices, and the widespread application of various sensors, a rich source of data has been provided for cross-domain behavior recognition research based on sensor data. At the same time, the widespread application of technologies such as machine learning and deep learning has made significant progress in cross-domain behavior recognition based on sensor data. Compared with behavior recognition using videos and images, behavior recognition based on sensor data is not subject to geographical restrictions. Current smart terminal devices have integrated a variety of sensor devices, including accelerometers, magnetometers, gyroscopes, etc., which provides feasibility for the use of smart terminal devices for cross-domain behavior recognition.

[0004] However, despite the widespread application of various sensors and deep learning technologies to provide data and theoretical support for cross-domain behavior recognition, this field still faces some challenges that need to be addressed. First, the existing cross-domain behavior recognition maps all source domain samples and target domain samples into the same subspace, where there may be huge distribution differences between some source domain samples and target domain samples, making it difficult to give accurate labels at the class level, so a more fine-grained method is needed to reduce the distribution difference. Secondly, in the current transfer learning-based models, there are situations where the data distribution difference between the source domain and the target domain is too large or the quality of the source domain data is poor, resulting in the knowledge learned in the source domain having a negative effect on the learning in the target domain, i.e., negative transfer. Therefore, how to mine the information in the source domain that is useful to the target domain, eliminate interference information and noise information, and design an effective transfer learning algorithm on this basis to improve the target domain action recognition performance is an issue that needs to be explored urgently. Summary of the invention

[0005] The purpose of this application is to provide a cross-domain behavior recognition method based on spatiotemporal frequency domain features and adversarial learning. The spatiotemporal and frequency domain features of sensor data are fully mined through a spatiotemporal feature extraction network and a frequency domain feature extraction network. The source domain and the target domain are aligned through adversarial learning combined with a frequency domain similarity matrix, so that the feature distribution of the source domain and the target domain is as consistent as possible, thereby improving the robustness and accuracy of cross-domain behavior recognition.

[0006] To achieve the above purpose, the technical solution of this application is as follows:

[0007] A cross-domain behavior recognition method based on spatiotemporal frequency domain features and adversarial learning, comprising:

[0008] Obtain source and target domain sensor data and use sliding windows for segmentation;

[0009] The segmented sensor data is passed through the spatiotemporal and frequency domain feature extraction network to obtain the source domain feature vector and the target domain feature vector respectively;

[0010] The spatiotemporal frequency domain feature extraction network comprises a spatiotemporal feature extraction network and a frequency domain feature extraction network. The spatiotemporal feature extraction network extracts source domain spatiotemporal features and target domain spatiotemporal features respectively, and the frequency domain feature extraction network extracts source domain frequency domain features and target domain frequency domain features respectively. The source domain spatiotemporal features and source domain frequency domain features are spliced ​​and fused to obtain a source domain feature vector, and the target domain spatiotemporal features and target domain frequency domain features are spliced ​​and fused to obtain a target domain feature vector.

[0011] Calculate the similarity matrix between the source domain frequency domain features and the target domain frequency domain features;

[0012] The source domain feature vector and the target domain feature vector are input into the classifier for classification respectively, and adversarial learning is performed through the discriminator combined with the similarity matrix;

[0013] The spatiotemporal frequency domain feature extraction network and classifier are used for the target domain sensor data, and the predicted category corresponding to the target domain sensor data is output through the classifier.

[0014] Furthermore, the spatiotemporal feature extraction network includes a Transformer encoder, a temporal feature extraction module and a spatial feature extraction module; the spatiotemporal feature extraction network performs the following operations:

[0015] The input data passes through the Transformer encoder to obtain high-dimensional features;

[0016] Then the high-dimensional features are extracted by the time feature extraction module to obtain the time features, and by the space feature extraction module to obtain the space features;

[0017] The temporal and spatial features are concatenated to obtain the spatiotemporal features.

[0018] Furthermore, the time feature extraction module performs the following operations:

[0019] Perform temporal normalization on high-dimensional features to obtain high-frequency features;

[0020] The extended features are obtained by concatenating the temporal high-frequency features and the high-dimensional features;

[0021] The expanded features are input into the sliding dilation temporal attention module to obtain the temporal features.

[0022] Furthermore, the sliding dilation time attention module performs the following operations:

[0023] The input features are selected using a sliding window to select feature components, which are input into the corresponding sliding dilation time attention layer to perform the sliding dilation time attention operation;

[0024] The outputs of each sliding dilation time attention layer are fused to obtain the temporal features;

[0025] Among them, each sliding dilation time attention layer controls the sparsity of the input feature components through the dilation rate, and then applies the self-attention mechanism to obtain the intermediate features, and splices the intermediate features under different dilation rates to obtain the output of the sliding dilation time attention layer.

[0026] Furthermore, the spatial feature extraction module performs the following operations:

[0027] Perform spatial normalization on high-dimensional features to obtain spatial local features;

[0028] The high-dimensional features and the local spatial features are concatenated to obtain the expanded features;

[0029] The expanded features are passed through the self-attention mechanism to obtain spatial features.

[0030] Furthermore, the frequency domain feature extraction network includes a preset number of frequency domain feature extraction layers, and each frequency domain feature extraction layer performs the following operations:

[0031] The input features are converted from the time domain to the frequency domain using discrete Fourier transform to obtain the frequency spectrum components;

[0032] The frequency spectrum components are concatenated with the input features, and then a multi-head attention mechanism is performed to obtain the associated components;

[0033] The input features, frequency spectrum components and correlation components are concatenated and passed through a feedforward network to obtain the frequency domain feature components output by the frequency domain feature extraction layer.

[0034] Furthermore, the frequency domain feature extraction network concatenates the frequency domain feature components output by all frequency domain feature extraction layers to obtain the frequency domain features finally output by the frequency domain feature extraction network.

[0035] This application proposes a cross-domain behavior recognition method based on spatiotemporal frequency domain features and adversarial learning. In the process of cross-domain behavior recognition, a spatiotemporal feature extraction network is used to expand and extract the spatiotemporal features in the sensor data to obtain more sufficient spatiotemporal feature information. The frequency domain feature extraction network is used to fully mine the frequency domain features in the sensor data, and the frequency domain features between the target domain and the source domain are calculated to obtain a similarity matrix. Combined with adversarial learning, the interference of non-target behaviors on behavior recognition is suppressed, the feature distribution between the source domain and the target domain is shortened, the generalization ability of the model is enhanced, and the model effectively recognizes cross-domain behaviors. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a flow chart of the cross-domain behavior recognition method based on spatiotemporal frequency domain features and adversarial learning in this application.

[0037] Figure 2 This is a schematic diagram of the adversarial learning network structure for this application.

[0038] Figure 3 Schematic diagram of the spatiotemporal feature extraction network structure of an embodiment of the present application.

[0039] Figure 4 This is a structural diagram of the sliding dilation time attention module of an embodiment of the present application.

[0040] Figure 5 Schematic diagram of the structure of the frequency domain feature extraction network in the embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0042] An embodiment of the present application, such as Figure 1 and Figure 2 As shown, a cross-domain behavior recognition method based on spatiotemporal frequency domain features and adversarial learning is provided, including:

[0043] Step S1: Acquire source domain and target domain sensor data and use a sliding window to segment them.

[0044] The raw sensor data is recorded as , the original sensor data in the source domain is denoted as , the original sensor data in the target domain is recorded as This embodiment adopts the sliding window segmentation technology, uses a sliding window with a fixed length of 200 and an overlap rate of 50%, places the sliding window at the beginning of the time series, and then moves the sliding window backward to form another subsequence with a length of 200, and so on, to segment a long time series sample into multiple short time series samples.

[0045] It should be noted that the data samples in the source domain have real labels and are used for training the network model.

[0046] Step S2: The segmented sensor data is extracted through a spatiotemporal frequency domain feature extraction network to obtain a source domain feature vector and a target domain feature vector.

[0047] Figure 2 The structure of the entire adversarial learning network is shown, wherein the spatiotemporal frequency domain feature extraction network includes a spatiotemporal feature extraction network and a frequency domain feature extraction network. The spatiotemporal feature extraction network extracts the source domain spatiotemporal features and the target domain spatiotemporal features respectively, and the frequency domain feature extraction network extracts the source domain frequency domain features and the target domain frequency domain features respectively; the source domain spatiotemporal features and the source domain frequency domain features are spliced ​​and fused to obtain a source domain feature vector, and the target domain spatiotemporal features and the target domain frequency domain features are spliced ​​and fused to obtain a target domain feature vector.

[0048] like Figure 3 As shown, the spatiotemporal feature extraction network includes a Transformer encoder, a temporal feature extraction module, and a spatial feature extraction module.

[0049] For the spatiotemporal feature extraction network, in this embodiment, the segmented sensor data is first converted into high-dimensional features with rich contextual information through the Transformer encoder. Then the high-dimensional features Extract its time features through the time feature extraction module , extract its spatial features through the spatial feature extraction module , and finally the time feature and spatial characteristics Output spatiotemporal features after splicing .

[0050] For the time feature extraction module, in this embodiment, the high-dimensional feature First, the time features are expanded by time normalization to obtain high-frequency features. , for the time window, It is a high-dimensional feature No. feature components, express The high-frequency feature components can be obtained by The calculation formula is as follows:

[0051] (1)

[0052] (2)

[0053] in, is a tiny constant term used to maintain numerical stability. and Respectively represent The mean and standard deviation of . and is a pair of learnable variables. Represents a concatenation operation. By extracting high-frequency feature components to enrich the feature space, the model is allowed to capture features outside of itself and better identify long-term dependencies in time series.

[0054] The obtained high-frequency features and high-dimensional features Splicing to get the expanded features , its formula can be expressed as:

[0055] (3)

[0056] Input to the sliding dilation temporal attention (SWDTA) module, such as Figure 4 As shown, sliding dilation time attention is used to extract temporal features, that is, temporal feature extraction and expansion are achieved through local attention mechanism and sparse selection strategy.

[0057] In a specific embodiment, the sliding dilation time attention module performs the following operations:

[0058] The input features are selected using a sliding window to select feature components, which are input into the corresponding sliding dilation time attention layer to perform the sliding dilation time attention operation;

[0059] The outputs of each sliding dilation time attention layer are fused to obtain the temporal features;

[0060] Among them, each sliding dilation time attention layer controls the sparsity of the input feature components through the dilation rate, and then applies the self-attention mechanism to obtain the intermediate features, and splices the intermediate features under different dilation rates to obtain the output of the sliding dilation time attention layer.

[0061] In this embodiment, a sliding window is used to expand the features Make a selection, select The characteristic components are Represents. They are input into the corresponding sliding dilation time attention layer respectively to perform the sliding dilation time attention operation.

[0062] In each corresponding sliding dilation time attention layer, the input feature components are linearly changed to obtain the query vector Q, key vector K and value vector V of the self-attention mechanism, and then some features are selected through the sparsity strategy of skip sampling. In this embodiment, the expansion rate Control the sparsity and determine the interval of feature selection within the window. Then apply the self-attention mechanism, which can be expressed as follows:

[0063] (4)

[0064] (5)

[0065] (6)

[0066] ℎ t i = SWDTA Q t i , K t i , V t i , r = Linear ( Concat [ z 0 i ,…, z r i ]) (7)

[0067] in, express No. feature components, is the input of sliding dilation time attention, represents the learnable weight matrix, represents the sliding dilation time attention, The time characteristic feature components. represents the expansion rate, When it is 0, take the adjacent time window. for Jump time window for sampling. Take 0 to 3, Respectively In the expansion rate The feature matrix generated by Represents the mapping function related to the expansion rate. express The dimension of ensures the stability of gradient calculation. Through the self-attention module, the input features With expansion rate Combined with the corresponding expansion rate, The intermediate features below . represents the sliding dilation time attention, The time characteristic feature components, and the intermediate features under different expansion rates You can get it by splicing .

[0068] The output of each layer of sliding dilation time attention is fused to generate the expanded features The global representation of , the formula can be expressed as:

[0069] H t ' = Norm ( Linear ( Concat [ ℎ t 1 ,…, ℎ t n ])) (8)

[0070] in, The time characteristic feature components, Representation layer normalization, represents the linear connection of parameters, That is, the time feature finally output by the time feature extraction module.

[0071] Similarly, for the spatial feature extraction module, high-dimensional features First, the spatial features are expanded by spatial normalization to obtain the spatial local features. For the time window, It is a high-dimensional feature No. feature components, express The local feature components can be obtained The calculation formula is as follows:

[0072] (9)

[0073] (10)

[0074] in, is a tiny constant term used to maintain numerical stability. and Respectively represent The mean and standard deviation of . and is a pair of learnable variables. By extracting local feature components, the global attention mechanism can be better used for association.

[0075] The obtained spatial local features and high-dimensional features Splicing to get the expanded features , its formula can be expressed as:

[0076] (11)

[0077] Input into the spatial attention network, and extract spatial features through the self-attention mechanism to obtain spatial features , the formula can be expressed as:

[0078] (12)

[0079] (13)

[0080] H s ' = Norm ( Linear ( Concat [ ℎ s 1 ,…, ℎ s n ])) (14)

[0081] in, express No. feature components, is the input of the spatial attention network, represents the learnable weight matrix, The spatial feature feature components, Representation layer normalization, Represents a linear connection of parameters.

[0082] Then the extracted time features and spatial characteristics Perform splicing and fusion to obtain the final spatiotemporal features , the formula is as follows:

[0083] (15)

[0084] Source domain sensor data Input the above spatiotemporal feature extraction network to obtain the source domain spatiotemporal features , target domain sensor data Input the above spatiotemporal feature extraction network to obtain the target domain spatiotemporal features .

[0085] For frequency domain feature extraction network, such as Figure 5 As shown, it includes multiple frequency domain feature extraction layers, and each frequency domain feature extraction layer performs the following operations:

[0086] The input features are converted from the time domain to the frequency domain using discrete Fourier transform to obtain the frequency spectrum components;

[0087] The frequency spectrum components are concatenated with the input features, and then a multi-head attention mechanism is performed to obtain the associated components;

[0088] The input features, frequency spectrum components and correlation components are concatenated and passed through a feedforward network to obtain the frequency domain feature components output by the frequency domain feature extraction layer.

[0089] Specifically, for the Layer, use represents the frequency spectrum component, represents the associated component, Represents the frequency domain characteristic component, which can be expressed as the following formula:

[0090] (16)

[0091] (17)

[0092] (18)

[0093] (19)

[0094] in, It is The frequency domain feature component of the layer output, when (i.e., first layer), using sensor features Alternative Perform calculations. is the discrete Fourier transform, are the input of multi-head attention, represents the learnable weight matrix, Indicates multiple attentions, It represents a feed-forward neural network.

[0095] This embodiment further improves the robustness and generalization ability of feature expression.

[0096] The frequency domain feature components output by all frequency domain feature extraction layers are concatenated to obtain the frequency domain features of the final output of the frequency attention module. , which can be expressed as the following formula:

[0097] E out = Norm ( Linear ( Concat [ E 1 ,…, E N ])) (20)

[0098] in, Indicates In this embodiment, a four-layer frequency domain feature extraction network is used to jointly extract frequency domain features to fully capture the multi-level representation of frequency domain information. Source domain sensor data Input the above frequency domain feature extraction network to obtain the source domain frequency domain features , target domain sensor data The above frequency domain feature extraction network obtains the target domain frequency domain features .

[0099] Source domain spatiotemporal characteristics and source domain frequency domain characteristics The source domain feature vector can be obtained by splicing and fusion , which can be expressed as the following formula:

[0100] (twenty one)

[0101] Similarly, the target domain spatiotemporal features and target domain frequency domain features The target domain feature vector can be obtained by splicing and fusion , which can be expressed as the following formula:

[0102] (twenty two).

[0103] Step S3: Calculate the similarity matrix between the source domain frequency domain features and the target domain frequency domain features.

[0104] In this embodiment, according to the source domain frequency domain characteristics and target domain frequency domain features , and calculate its similarity matrix. The similarity matrix is ​​obtained by calculating the cosine similarity of the source domain frequency domain features and the target domain frequency domain features. , which can be expressed as the following formula:

[0105] (twenty three)

[0106] in, Represents the total number of dimensions of frequency domain features, Indicates that the source domain frequency domain features are in dimension The characteristic components on Indicates that the target domain frequency domain features are in the dimension The characteristic components on .

[0107] Step 4: Input the source domain feature vector and the target domain feature vector into the classifier for classification, and perform adversarial learning through the discriminator combined with the similarity matrix.

[0108] In this step, the source domain feature vector and the target domain feature vector They are input into the classifier for classification respectively, and adversarial learning is performed through the discriminator combined with the similarity matrix.

[0109] Specifically, in this embodiment, the target domain features are classified by combining the adversarial network with the similarity matrix. The source domain data provides the real label, and supervises the learning through the classifier to help the feature extraction network learn discriminative features. The output probability distribution of the classifier is used as conditional information and combined with the features of the feature extraction network. This joint representation captures the relationship between the feature and the category information and can better reflect the difference in conditional distribution. A discriminator is used to perform binary classification on the joint representation to determine whether its source is the source domain or the target domain, thereby bringing the feature distribution of the target domain and the source domain closer.

[0110] In this embodiment, the source domain feature vector and the target domain feature vector are respectively input into the classifier for classification. The classifier is composed of a series of linear layers. Except for the last layer, each linear layer applies Dropout and ReLU activation functions to enhance the nonlinear expression ability of the network and prevent overfitting. The loss function during the classifier training is defined as follows:

[0111] (twenty four)

[0112] in, is the cross entropy loss function, is the total number of categories, Indicates that the activity belongs to a category The true probability of Indicates that the behavior of the classifier output belongs to the category The predicted probability of .

[0113] In this embodiment, adversarial learning is performed by combining a discriminator with a similarity matrix. The discriminator consists of a layer of fully connected neural network and a Sigmoid activation function. and the target domain feature vector After a layer of fully connected neural network, the probability value of the sample coming from the source domain is output through the Sigmoid activation function. In order to enable the discriminator to better distinguish the source of the sample, this embodiment uses a weighted adversarial loss function for optimization. The weighted adversarial loss function can be defined as follows:

[0114] (25)

[0115] in, Represents the source domain feature vector The total number of dimensions, Represents the target domain feature vector The total number of dimensions, represents the weighted adversarial loss, represents the discriminator, Represents source domain features No. Quantity, Represents the target domain characteristics No. A quantity.

[0116] The total loss function of this embodiment is It can be expressed as the following formula:

[0117] (26)

[0118] in, Represents the weight coefficient, which is used to balance the importance of classification task and identification task.

[0119] It should be noted that the learning process of adversarial learning is a relatively mature technology in this field. The weight parameters in the network are updated through adversarial learning, which will not be described in detail here.

[0120] Step 5: Apply the spatiotemporal frequency domain feature extraction network and classifier to the target domain sensor data, and output the predicted category corresponding to the target domain sensor data through the classifier.

[0121] After adversarial learning, the spatiotemporal frequency domain feature extraction network and classifier are used for the target domain sensor data, and the classifier outputs the predicted category corresponding to the target domain sensor data.

[0122] In this application, the technical solution of this application is also verified through experiments. The experiment is based on the DSADS dataset and the PAMAP2 dataset, and is compared with the current mainstream cross-domain behavior recognition algorithm. The accuracy and F1 score are used to measure the recognition performance. The experimental results are shown in Table 1:

[0123] Table 1

[0124]

[0125] As can be seen from the data in Table 1, the performance of the proposed method on both DSADS and PAMAP2 datasets is significantly better than other methods, reflecting its excellent feature extraction and cross-domain adaptation capabilities. On the DSADS dataset, the accuracy and F1 score of the proposed method reached 84.39% and 82.64%, respectively, which were 5.64% and 6.41% higher than the second-best method (CALDA); on the PAMAP2 dataset, its accuracy and F1 score reached 86.15% and 85.72%, respectively, which were 7.81% and 9.24% higher than CALDA. Compared with traditional methods (such as SVM, RF) and some transfer learning methods (such as TCA, SA), the proposed method showed significant advantages in cross-domain feature alignment on both datasets, especially in high-complexity tasks, showing strong robustness and generalization capabilities.

[0126] In summary, the method of the present application has certain advantages over other algorithms. It can more comprehensively understand the behavior reflected by the sensor data and efficiently extract behavioral features from the sensor data, thereby significantly improving the accuracy and robustness of cross-domain behavior recognition.

[0127] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A cross-domain behavior recognition method based on spatiotemporal frequency domain features and adversarial learning, characterized in that: The cross-domain behavior recognition method based on spatiotemporal frequency domain features and adversarial learning includes: Obtain source and target domain sensor data and use sliding windows for segmentation; The segmented sensor data is passed through the spatiotemporal and frequency domain feature extraction network to obtain the source domain feature vector and the target domain feature vector respectively; The spatiotemporal frequency domain feature extraction network comprises a spatiotemporal feature extraction network and a frequency domain feature extraction network. The spatiotemporal feature extraction network extracts source domain spatiotemporal features and target domain spatiotemporal features respectively, and the frequency domain feature extraction network extracts source domain frequency domain features and target domain frequency domain features respectively. The source domain spatiotemporal features and source domain frequency domain features are spliced ​​and fused to obtain a source domain feature vector, and the target domain spatiotemporal features and target domain frequency domain features are spliced ​​and fused to obtain a target domain feature vector. Calculate the similarity matrix between the source domain frequency domain features and the target domain frequency domain features; The source domain feature vector and the target domain feature vector are input into the classifier for classification respectively, and adversarial learning is performed through the discriminator combined with the similarity matrix; The spatiotemporal frequency domain feature extraction network and classifier are used for the target domain sensor data, and the predicted category corresponding to the target domain sensor data is output through the classifier.

2. The cross-domain behavior recognition method based on spatiotemporal frequency domain features and adversarial learning according to claim 1 is characterized in that: The spatiotemporal feature extraction network includes a Transformer encoder, a temporal feature extraction module and a spatial feature extraction module; the spatiotemporal feature extraction network performs the following operations: The input data passes through the Transformer encoder to obtain high-dimensional features; Then the high-dimensional features are extracted by the time feature extraction module to obtain the time features, and by the space feature extraction module to obtain the space features; The temporal and spatial features are concatenated to obtain the spatiotemporal features.

3. The cross-domain behavior recognition method based on spatiotemporal frequency domain features and adversarial learning according to claim 2 is characterized in that: The time feature extraction module performs the following operations: Perform temporal normalization on high-dimensional features to obtain high-frequency features; The extended features are obtained by concatenating the temporal high-frequency features and the high-dimensional features; Input the expanded features into the sliding dilation time attention module to obtain the temporal features; The sliding dilation time attention module performs the following operations: The input features are selected using a sliding window to select feature components, which are input into the corresponding sliding dilation time attention layer to perform the sliding dilation time attention operation; The outputs of each sliding dilation time attention layer are fused to obtain the temporal features; Among them, each sliding dilation time attention layer controls the sparsity of the input feature components through the dilation rate, and then applies the self-attention mechanism to obtain the intermediate features, and splices the intermediate features under different dilation rates to obtain the output of the sliding dilation time attention layer.

4. The cross-domain behavior recognition method based on spatiotemporal frequency domain features and adversarial learning according to claim 2 is characterized in that: The spatial feature extraction module performs the following operations: Perform spatial normalization on high-dimensional features to obtain spatial local features; The high-dimensional features and the local spatial features are concatenated to obtain the expanded features; The expanded features are passed through the self-attention mechanism to obtain spatial features.

5. The cross-domain behavior recognition method based on spatiotemporal frequency domain features and adversarial learning according to claim 1 is characterized in that: The frequency domain feature extraction network includes a preset number of frequency domain feature extraction layers, each of which performs the following operations: The input features are converted from the time domain to the frequency domain using discrete Fourier transform to obtain the frequency spectrum components; The frequency spectrum components are concatenated with the input features, and then a multi-head attention mechanism is performed to obtain the associated components; The input features, frequency spectrum components and correlation components are concatenated and passed through a feedforward network to obtain the frequency domain feature components output by the frequency domain feature extraction layer.

6. The cross-domain behavior recognition method based on spatiotemporal frequency domain features and adversarial learning according to claim 5 is characterized in that: The frequency domain feature extraction network concatenates the frequency domain feature components output by all frequency domain feature extraction layers to obtain the frequency domain features finally output by the frequency domain feature extraction network.

Images