Human activity identification method and system based on lightweight hybrid neural network and self-attention migration and medium thereof

Abstract

The invention relates to the technical field of human activity recognition, and particularly discloses a human activity recognition method and system based on a lightweight hybrid neural network and self-attention migration and a medium thereof, and the method comprises the steps: 1, employing an HHAR data set, and carrying out the preprocessing; step 2, constructing a teacher model based on a CNN-LSTM-Transform hybrid architecture, and constructing a teacher model based on the CNN-LSTM-Transform hybrid architecture; step 3, constructing a student model of a hybrid architecture based on CNN-LSTM-Transform; and step 4, utilizing a self-attention migration mechanism to guide the student model to learn the attention distribution mode of the teacher model, and realizing efficient migration of knowledge. According to the method, the calculation complexity and the storage requirement can be remarkably reduced while the expression ability of the model is maintained.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning and artificial intelligence, in particular to a human activity recognition method and system based on a lightweight hybrid neural network and self-attention transfer and a medium thereof. BACKGROUND

[0002] Human Activity Recognition (HAR) refers to a technology that automatically identifies the type, state and pattern of human activities through various sensors to obtain human activity data, and through signal processing and pattern recognition techniques. With the rapid development of smart wearable devices, smartphones and Internet of Things technology, HAR technology has been widely applied in health monitoring, sports training, smart home, elderly care and rehabilitation medicine and many other fields.

[0003] Sensor-based HAR systems are usually based on data from inertial measurement units, including accelerometers, gyroscopes and magnetometers, etc. These sensors can capture the motion state and posture changes of the human body in three-dimensional space. With the progress of sensor miniaturization and low-power technology, these sensors have been widely integrated into smartphones, smartwatches and professional sports tracking devices, providing a rich source of data for HAR systems.

[0004] Human activity recognition technology has undergone an evolution from traditional methods to deep learning methods:

[0005] Traditional machine learning methods: Early HAR systems mainly used a combination of hand-crafted feature extraction and traditional machine learning algorithms. Researchers usually extract statistical features from the time domain (mean, variance, peak value, etc.) and frequency domain (spectral energy distribution, frequency domain entropy, etc.), and then use classifiers such as Support Vector Machine (SVM), Random Forest (RF), K-Nearest Neighbors (KNN) for activity recognition. This method requires domain experts to design features, and its performance largely depends on the quality of feature design.

[0006] Deep learning methods: With the rapid development of deep learning, the HAR field gradually adopted end-to-end deep neural network architectures to automatically learn hierarchical feature representations from raw sensor data. The main deep learning models include:

[0007] Convolutional Neural Network (CNN): uses convolution operations to automatically extract local features in time and spatial dimensions, effectively capturing local patterns and hierarchical structures of activity signals.

[0008] Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM): Specialized for processing sequential data, capable of capturing long and short-term temporal dependencies in activity data, effectively modeling the temporal dynamics of activities.

[0009] Transformer architecture: Based on self-attention mechanism, capable of parallel processing of sequential data, capturing global dependencies, and demonstrating superior performance in the HAR field in recent years.

[0010] Hybrid architecture: Combining the advantages of CNN for extracting spatial features and RNN / LSTM / Transformer for handling temporal relationships, such as CNN-LSTM and CNN-Transformer hybrid models, which perform well in complex activity recognition tasks.

[0011] However, high-performance models in the prior art usually have complex network structures and large numbers of parameters, such as typical HAR Transformer models that may contain 5-10M parameters, making it difficult to efficiently run on resource-constrained edge devices. SUMMARY

[0012] Therefore, the present application provides a human activity recognition method based on a lightweight hybrid neural network and self-attention transfer to overcome the technical problem that existing technology is difficult to efficiently run on resource-constrained edge devices.

[0013] To achieve the above purpose, the basic scheme of the present application provides a human activity recognition method based on a lightweight hybrid neural network and self-attention transfer, comprising:

[0014] Step 1: Use the HHAR dataset and perform preprocessing;

[0015] Step 2: Construct a teacher model based on a CNN-LSTM-Transformer hybrid architecture;

[0016] Step 3: Construct a student model based on a CNN-LSTM-Transformer hybrid architecture;

[0017] Step 4: Use the self-attention transfer mechanism to guide the student model to learn the attention distribution pattern of the teacher model, achieving efficient knowledge transfer.

[0018] In one possible design, the teacher model based on the CNN-LSTM-Transformer hybrid architecture in step 2 includes,

[0019] Step 21, convolution feature extraction is performed, and a three-layer cascaded residual convolution structure is adopted. The calculation of each layer of residual convolution block can be represented as:

[0020] H l =F l (H l-1 )+G l (H l-1 )

[0021] Wherein, F l represents the main path mapping, G l represents the shortcut connection, and when the input and output channel numbers do not match, G l is realized by 1x1 convolution to adjust the dimension; otherwise, it is an identity mapping. The channel configuration of the three residual convolution blocks is as follows:

[0022]

[0023] Step 22, the time sequence dependence relationship is processed by using a multi-layer LSTM network, and the core calculation process of the LSTM unit is as follows:

[0024] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0025] i t =σ(W i ·[h t-1 ,x t ]+b i )

[0026]

[0027] o t =σ(W o ·[h t-1 ,x t ]+b o )

[0028] h t =o t ⊙tanh(C t )

[0029] Wherein, f t represents the forgetting gate output, which is mapped to the interval [0, 1] through the sigmoid activation function, and controls the retention degree of the cell state C t-1 at the last time; i t is the input gate output, which is also mapped through the sigmoid function, and determines the update proportion of the current input information. C represents the candidate cell state, mapped to [-1, 1] interval through tanh function, indicating new information to be added; C t is the current cell state, obtained by adding the historical information controlled by the forget gate and the new information controlled by the input gate; o t is the output gate, generated by the sigmoid function, which determines the amount of information transmitted from the cell state to the hidden state; h t is the current hidden state, i.e., the LSTM unit output, obtained by multiplying the output gate and the cell state activated by tanh, where W f , W i , W C , and W o are the weight matrices of each gate, b f , b i , b C , and b o are the corresponding bias vectors, and [h t-1 , x t ] represents the connection of the previous hidden state h t-1 and the current input x t in the feature dimension, and the symbol ⊙ represents the Hadamard product, i.e., element-level multiplication operation;

[0030] The teacher model adopts a 3-layer LSTM, each layer having 256 hidden units, and after the LSTM output, layer normalization is applied:

[0031] H norm = LayerNorm(H lstm )

[0032] For the LayerNorm layer normalization operation: H n orm represents the output feature after layer normalization; H l stm represents the original output feature of the LSTM network;

[0033] where the layer normalization operation is defined as:

[0034]

[0035] where γ and β are learnable scaling and offset parameters to restore the expression ability of the feature; μ represents the mean of the feature at each sample; σ 2 represents the variance of the feature at each sample; ε is a very small constant for numerical stability to avoid division by zero error;

[0036] Step 23: Directly associate all time steps in the sequence through the self-attention mechanism of the Transformer layer, and the calculation process of multi-head self-attention is as follows:

[0037] MultiHead(Q, K, V) = Concat(head1,..., head h )W O

[0038] where the computation of each attention head is:

[0039]

[0040] For the multi-head self-attention mechanism of the Transformer layer, where: Q, K, V represent the query matrix Query, the key matrix Key, and the value matrix Value, respectively, which are the representations of the input sequence after linear transformation; head i represents the output of the i-th attention head; is the weight matrix corresponding to the i-th attention head, used to project the input to different feature subspaces; W O is the linear projection matrix of the multi-head attention output; h represents the number of attention heads; Concat represents the concatenation operation in the feature dimension,

[0041] The attention function is defined as:

[0042]

[0043] The softmax function normalizes the attention scores into a probability distribution, ensuring that the sum of all weights is 1; QK T represents the similarity matrix between the query and the key, measuring the strength of the association between elements in the sequence; is a scaling factor, d k represents the dimension of the key vector, and the scaling operation can make the gradient more stable; V is the value matrix, containing the information of each element in the sequence.

[0044] In one possible design, in step 3, the backbone network of the student model includes a depth separable convolution layer, a single-layer LSTM network, and a lightweight Transformer layer, and the steps for constructing the student model include,

[0045] Step 31, the depth separable convolution decomposes the standard convolution into two independent steps of depth convolution and pointwise convolution, that is:

[0046] Y = Y point (Y depth (X))

[0047] Y represents the final output feature map of the depth separable convolution; Y point represents the output of the pointwise convolution; Y depthY represents the output of the depth convolution; X represents the input feature map, with dimensions [batch size, input channel number, sequence length], and the depth separable convolution significantly reduces the computational complexity and parameter quantity by decomposing the standard convolution into two independent operations,

[0048] The depth convolution independently applies the convolution kernel to each channel of the input to capture spatial features, and the calculation is as follows:

[0049] Y depth [:,i,:] = X[:,i,:]*K i

[0050] X[:,i,:] represents the i-th channel of the input feature map; K i represents the convolution kernel acting on the i-th input channel; the symbol * represents the convolution operation,

[0051] The point-wise convolution uses a 1x1 convolution kernel to fuse channel information:

[0052]

[0053] Y point [:,j,:] represents the j-th channel of the point-wise convolution output; C in represents the number of channels of the input feature map; W i,j represents the weight parameter connecting the i-th input channel and the j-th output channel; the summation symbol ∑ represents the weighted summation operation on all input channels, and the point-wise convolution uses a 1x1 convolution kernel to realize cross-channel information fusion, linearly combines the channel features obtained by the depth convolution, and generates the final output feature;

[0054] Step 32, after the single-layer LSTM network outputs, layer normalization is also applied to ensure the stability of the feature distribution;

[0056] Step 33, the lightweight Transformer layer realizes efficient attention calculation and reduces the number of attention heads from 8 to 4 through parameter sharing and dimension reduction, and simplifies the query, key, and value matrices to:

[0057] Q = K = V = XW QKV

[0058] Where: Q, K, and V represent the query matrix Query, the key matrix Key, and the value matrix Value, respectively.

[0059] In one possible design, in step 3, the depth separable convolution layer is configured with two depth separable convolution blocks, and the channel configuration is:

[0060]

[0061] In one possible design, in step 4, first, the multi-head attention weights are extracted from the teacher model and the student model. In the Transformer architecture, each attention head generates an attention weight matrix, which represents the correlation strength between elements in the sequence. For the teacher model, the attention weight matrix can be represented as:

[0062]

[0063] where B represents the batch size, H t represents the number of attention heads of the teacher model, S t represents the sequence length after three pooling operations of the teacher model,

[0064] Correspondingly, the attention weight matrix of the student model can be represented as:

[0065]

[0066] where H s represents the number of attention heads of the student model, S s represents the sequence length after two pooling operations of the student model;

[0067] Secondly, the attention weight matrix alignment is performed. The alignment operation can be implemented by an adaptive average pooling function f align

[0068]

[0069] where f align is specifically:

[0070]

[0071] where b represents the batch index; h represents the attention head index; i, j represent the target index, ranging from {0, 1,..., S t-1}; p, q represent the source index, ranging from {0, 1,..., S s-1}; R i represents the set of source indexes mapped to the target index i; |R i | represents the size of the set R i , for the adaptive average pooling, the construction method of R i is as follows:

[0072]

[0073] where for each position (p, q) in the attention weight matrix of the student model, the position (i, j) in the attention weight matrix of the teacher model that it maps to is calculated, and then the average value of all source positions that map to the same position is taken; ​

[0074] Again, after the alignment of the attention weight matrices, the teacher model and the student model need to be aggregated respectively for the attention head dimension, and the information of multi-head attention is integrated into a single attention distribution:

[0075]

[0076] where |·| represents the absolute value operation, and subscript i represents the i-th attention head,

[0077] To ensure that the comparison is the relative pattern of the attention distribution rather than the absolute intensity, L2 norm normalization is applied to the aggregated attention weight matrix:

[0078]

[0079] where b represents the b-th sample in the batch, and ||·||2 represents the L2 Euclidean norm,

[0080] The attention transfer loss is obtained by calculating the L2 distance square between the normalized teacher and student attention vectors:

[0081]

[0082] where β is a hyperparameter that controls the weight of the attention transfer loss, and 2 in the denominator is a normalization factor to keep the loss value within a proper range. The total training loss is composed of the standard cross-entropy loss and the weighted attention transfer loss:

[0083]

[0084] where L CE represents the cross-entropy loss function, y is the true label, is the prediction output of the student model.

[0085] In one possible design, in step 2, when building the teacher model, after obtaining the deep feature representation of the sequence, the model needs to convert the variable-length sequence feature into a fixed-length vector representation for final classification. A learning-based attention pooling mechanism is used, which is calculated as follows:

[0086]

[0087] where c represents the global feature vector after attention weighting; T represents the length of the input sequence, i.e., the total number of time steps; α t represents the attention weight at the t-th time step; h t represents the hidden state feature vector at the t-th time step;

[0088] where the attention weight α t is calculated and normalized by the feedforward network:

[0089]

[0090] e t denotes the raw attention score at the t-th time step; W a denotes the weight matrix of attention computation, mapping the hidden state to the attention score; b a denotes the bias parameter of attention computation; exp function denotes the exponential operation, converting the raw score to a positive value; the denominator is the normalization term for all time step attention scores, ensuring the sum of all attention weights is 1, forming a valid probability distribution,

[0091] The classification module consists of multiple layers of fully connected networks, finally outputting the class probability distribution:

[0092] y = softmax(W3(ReLU(W2(ReLU(W1c + b1)) + b2)) + b3)

[0093] where y denotes the final class probability distribution vector; c is the global feature vector from attention pooling, serving as the input to the classifier; W1, W2, W3 represent the weight matrices of the three layers of fully connected networks, respectively, realizing nonlinear feature transformation; b1, b2, b3 are the corresponding bias vectors; ReLU represents the rectified linear unit activation function; the softmax function converts the output of the last layer to a probability distribution, with each element representing the predicted probability of the corresponding activity class, and the sum of all probabilities is 1.

[0094] In one possible design, the HHAR dataset is collected by multiple volunteers wearing smartphones while performing a series of standardized activities in a controlled environment, recording six common human activities, including walking, running, standing, sitting, climbing stairs, and descending stairs.

[0095] In one possible design, in step 1, preprocessing includes the following processing steps: data loading, data cleaning, time series segmentation, activity labeling, and dataset construction,

[0096] Data loading first verifies the integrity and format consistency of the data file, ensuring that the column names match the expected structure, and then converts the key sensor data columns to appropriate numerical types, laying the foundation for subsequent processing;

[0097] Data cleaning is responsible for handling outliers and missing values in the original data, removing records containing missing values in sensor readings (x, y, z) or activity labels, ensuring data integrity;

[0098] Time series segmentation, the continuous data stream is segmented into fixed length time series fragments by using sliding window technology, the window size is 200, the compensation is 100, that is, the overlap rate is 50%;

[0099] Dataset construction, first, the activity label in text form is converted into numerical encoding using LabelEncoder, and the encoding mapping relationship is saved, ensuring that the prediction result can be correctly decoded in the inference stage, the dimension structure of the data segment is rearranged from [sample number, time series length, feature channel number] to [sample number, feature channel number, time series length], a hierarchical sampling strategy is used to divide the dataset into a training set, a validation set and a test set, and the proportion of activity classes in each subset is kept consistent, first, the data is divided into an initial training set and a test set in a ratio of 8:2; then the initial training set is divided into a final training set and a validation set in a ratio of 8:2.

[0100] The application also provides a system comprising a memory, a control processor and a computer program stored on the memory and executable on the control processor, the control processor executing the program to implement the human activity recognition method based on lightweight hybrid neural network and self-attention transfer as described above.

[0101] The application also provides a computer readable storage medium storing computer executable instructions for implementing the human activity recognition method based on lightweight hybrid neural network and self-attention transfer as described above.

[0102] Compared with the prior art, the application has the following advantages:

[0103] 1. Lightweight model architecture innovation

[0104] When deploying the HAR system on resource-constrained devices, the computational complexity and storage requirements of the model are key limiting factors. Traditional model compression methods usually use simple channel reduction or layer reduction strategies, which are difficult to achieve significant complexity reduction while maintaining recognition performance.

[0105] The application proposes a series of lightweight architecture innovations, including: (1) using depth separable convolution instead of standard convolution, reducing the computational complexity by about 9 times; (2) designing a lightweight Transformer layer, reducing the number of attention heads and optimizing the feedforward network structure; (3) reducing the number of LSTM layers and reducing the dimension of the hidden units; (4) optimizing the time series pooling strategy, while preserving key time series features, reducing the parameter amount. These innovations work together to significantly reduce the computational complexity and storage requirements while maintaining the model's expressive ability.

[0106] 2. Self-attention transfer strategy

[0107] The application proposes an innovative self-attention transfer strategy, which has the following key innovations compared with traditional methods.

[0108] Firstly, the application realizes the paradigm shift from feature mapping to self-attention transfer. Traditional knowledge distillation methods such as FitNets mainly focus on matching the intermediate layer feature mapping. Although this method has achieved some success in static image processing field, it cannot effectively capture the time dependence in sequence data. The application innovatively shifts the focus of knowledge distillation to the self-attention mechanism in the Transformer architecture, directly transferring the understanding of the importance of different time steps by the model, rather than simply the feature representation. This shift enables the student model to inherit the core ability of the teacher model to recognize complex temporal patterns, rather than just surface features, thereby significantly reducing the model complexity while maintaining high recognition accuracy.

[0109] The classic attention transfer technology proposed by Zagoruyko and Komodakis mainly focuses on the spatial attention of CNN, defining attention as the square sum of activation values. This method faces major challenges when dealing with the multi-head self-attention of the Transformer architecture, and cannot effectively handle the rich complementary information in multi-head attention. The self-attention transfer strategy proposed by the application is specifically designed for the multi-head self-attention mechanism, effectively utilizing the complementary information between attention heads, and enabling efficient knowledge transfer even when the number of attention heads in the teacher model and the student model is different, which is particularly important in complex human activity recognition tasks. BRIEF DESCRIPTION OF DRAWINGS

[0110] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0111] Figure 1 The teacher model structure schematic diagram in the embodiments of the present application is shown;

[0112] Figure 2 The student model structure schematic diagram in the embodiments of the present application is shown;

[0113] Figure 3 The self-attention transfer strategy schematic diagram in the embodiments of the present application is shown;

[0114] Figure 4 The teacher model confusion matrix diagram in the embodiments of the present application is shown;

[0115] Figure 5 The student model confusion matrix diagram without using self-attention transfer is shown;

[0116] Figure 6 A student model confusion matrix plot using self-attention transfer is shown. DETAILED DESCRIPTION

[0117] To further illustrate the embodiments, the present application provides drawings, which are part of the disclosure of the present application, mainly used to illustrate the embodiments, and can be used to explain the operating principle of the embodiments in conjunction with the related description of the specification. Those skilled in the art should understand other possible implementations and advantages of the present application by referring to these contents. The components in the drawings are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0118] As shown in Figures 1-4 , Figure 6 A human activity recognition method based on lightweight hybrid neural network and self-attention transfer, comprising the following steps:

[0119] 1. Data set selection and preprocessing

[0120] The data set used by the present application is HHAR (Heterogeneous Human Activity Recognition) data set, which is a public data set widely used for research and evaluation of human activity recognition algorithm.

[0121] The HHAR data set is collected by a number of volunteers wearing various brands and models of smartphones (such as Samsung, LG, HTC, etc.) when performing a series of standardized activities in a controlled environment, recording six common human activities, including walking, running, standing, sitting, climbing stairs and descending stairs. The original data is stored in CSV format, containing fields such as timestamp, sensor reading, user identification, device information and activity label. Sensor data is usually collected at a frequency of about 100Hz, providing sufficient time resolution to capture the details of human motion.

[0122] The data preprocessing framework includes five sequentially executed processing steps: data loading, data cleaning, time series segmentation, activity labeling and dataset construction.

[0123] Data loading first verifies the integrity and format consistency of the data file, ensuring that the column names match the expected structure. Then the key sensor data columns are converted to appropriate numerical types, laying the foundation for subsequent processing. During the loading process, the system generates a data statistics report, including the total number of samples, the sample distribution of each activity category, and the possible existence of missing values, which helps subsequent error debugging.

[0124] Data cleaning is responsible for handling outliers and missing values in raw data, removing records containing missing values in sensor readings (x, y, z) or activity labels, and ensuring data integrity.

[0125] Time series segmentation uses a sliding window technique to segment continuous data streams into fixed-length time series segments. The window size is 200, and the compensation is 100, which is a 50% overlap rate.

[0126] Since the sliding window may span the transition phase of multiple activities, the present application uses a majority voting method to determine the activity label of the window, that is, the activity type with the highest frequency in the window is assigned to the window.

[0127] Finally, the data set construction part first converts the text form activity label (such as "walking", "sitting") into numerical encoding for model processing. This conversion is realized using LabelEncoder, and the encoding mapping relationship is saved to ensure that the prediction results can be correctly decoded in the inference stage. To adapt to the input requirements of the convolutional neural network, the dimension structure of the data segment is rearranged from [sample number, time series length, feature channel number] to [sample number, feature channel number, time series length]. To scientifically evaluate the performance of the model, the present application uses stratified sampling strategy to divide the data set into training set, validation set and test set, keeping the proportion of activity categories consistent in each sub-set. First, the data is divided into initial training set and test set in the ratio of 8:2; then the initial training set is divided into final training set and validation set in the ratio of 8:2.

[0128] 2. Teacher model design

[0129] The present application designs a teacher model based on a hybrid architecture of CNN-LSTM-Transformer, as shown in Figure 1 The model fully integrates the advantages of convolutional neural networks in local feature extraction, long short-term memory networks in sequence modeling, and Transformers in capturing long-range dependencies, forming a high-performance human activity recognition model. The teacher model uses a multi-level feature extraction and fusion strategy, which can effectively process the spatio-temporal patterns in sensor data and achieve accurate activity classification.

[0130] The feature extraction module uses a three-layer cascaded residual convolution structure. The calculation of each layer of residual convolution block can be represented as:

[0131] H l =F l (H l-1 )+G l (H l-1 )

[0132] where F l represents the main path mapping, and Gl Indicates a shortcut connection. When the number of input and output channels does not match, G l Dimension adjustment is achieved by 1x1 convolution; otherwise, it is an identity mapping. The channel configuration of the three residual convolution blocks is:

[0133]

[0134] After the convolution feature extraction, the model further processes the time-dependent relationship by using a multi-layer LSTM network. The core calculation process of the LSTM unit is as follows:

[0135] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0136] i t =σ(W i ·[h t-1 ,x t ]+b i )

[0137]

[0138] o t =σ(W o ·[h t-1 ,x t ]+b o )

[0139] h t =o t ⊙tanh(C t )

[0140] Where f t represents the forgetting gate output, which is mapped to the interval [0, 1] through the sigmoid activation function, and controls the preservation degree of the cell state C t-1 at the last time; i t is the input gate output, which is also mapped through the sigmoid function, and determines the update proportion of the current input information; represents the candidate cell state, which is mapped to the interval [-1, 1] through the tanh function, and represents the new information to be added; C t is the current cell state, which is obtained by adding the historical information controlled by the forgetting gate and the new information controlled by the input gate; o t is the output gate, which is generated by the sigmoid function, and determines the amount of information transmitted from the cell state to the hidden state; h t is the current hidden state, i.e., the LSTM unit output, which is obtained by multiplying the output gate and the cell state activated by tanh, where W f, W i , W C , W o are the weight matrix of each gate, b f , b i , b C , b o are the corresponding bias vectors, [h t-1 , x t ] represents the concatenation of the previous hidden state h t-1 and the current input x t in the feature dimension, and the symbol ⊙ represents the Hadamard product, i.e., element-level multiplication operation;

[0141] The teacher model adopts a 3-layer LSTM, each layer having 256 hidden units. After the LSTM output, layer normalization is applied:

[0142] H norm = LayerNorm(H lstm )

[0143] For the LayerNorm layer normalization operation: H n orm represents the output feature after layer normalization; H l stm represents the original output feature of the LSTM network.

[0144] where the layer normalization operation is defined as:

[0145]

[0146] where γ and β are learnable scaling and offset parameters to restore the expressiveness of the feature; μ represents the mean of the feature at each sample; σ 2 represents the variance of the feature at each sample; ε is a very small constant (usually set to 10 -5 ) for numerical stability to avoid division by zero errors;

[0147] The Transformer layer establishes direct connections between all time steps in the sequence through a self-attention mechanism. The calculation process of multi-head self-attention is as follows:

[0148] MultiHead(Q,K,V)=Concat(head1,…,head h )W O

[0149] where the calculation of each attention head is:

[0150]

[0151] For the multi-head self-attention mechanism of the Transformer layer, where: Q, K, V represent the query matrix Query, the key matrix Key, and the value matrix Value, respectively, which are the representations of the input sequence after linear transformation; head i represents the output of the i-th attention head; is the weight matrix corresponding to the i-th attention head, used to project the input into different feature subspaces; W O is the linear projection matrix of the multi-head attention output; h represents the number of attention heads; Concat represents the concatenation operation in the feature dimension,

[0152] The attention function is defined as:

[0153]

[0154] The softmax function normalizes the attention scores into a probability distribution, ensuring that the sum of all weights is 1; QK T represents the similarity matrix of the query and the key, measuring the correlation strength between elements in the sequence; is a scaling factor, d k represents the dimension of the key vector, and the scaling operation can make the gradient more stable; V is the value matrix, containing the information of each element in the sequence.

[0155] The teacher model implements an 8-head self-attention mechanism, with a feature dimension of 32 for each attention head and a total feature dimension of 256.

[0156] After obtaining the deep feature representation of the sequence, the model needs to convert the variable-length sequence features into fixed-length vector representations for final classification. The invention uses a learning-based attention pooling mechanism. The calculation is as follows:

[0157]

[0158] where c represents the global feature vector after attention weighting; T represents the length of the input sequence, i.e., the total number of time steps; α t represents the attention weight of the t-th time step; h t represents the hidden state feature vector of the t-th time step;

[0159] where the attention weight α t is calculated and normalized through a feedforward network:

[0160]

[0161] e t represents the original attention score of the t-th time step; W a represents the weight matrix of attention calculation, mapping the hidden state to the attention score; ba bias parameter representing attention computation; exp function represents exponential operation, converting original score to positive value; denominator is a normalization term for all time step attention scores, ensuring the sum of all attention weights is 1, forming a valid probability distribution,

[0162] The classification module is composed of a multi-layer fully connected network, and finally outputs a class probability distribution:

[0163] y = softmax(W3(ReLU(W2(ReLU(W1c + b1) + b2) + b3)

[0164] where y represents the final class probability distribution vector; c is the global feature vector from attention pooling, as the input of the classifier; W1, W2, W3 represent the weight matrices of the three-layer fully connected network, respectively, realizing nonlinear feature transformation; b1, b2, b3 are the corresponding bias vectors; ReLU represents the rectified linear unit activation function; the softmax function converts the output of the last layer into a probability distribution, and each element represents the prediction probability of the corresponding activity class, and the sum of all probabilities is 1.

[0165] 3. Student model design

[0166] The student model of the application adopts a lightweight neural network architecture, which significantly reduces the parameter quantity and computational complexity while maintaining a similar overall structure as the teacher model. The student model is also based on a hybrid architecture of CNN-LSTM-Transformer, such as Figure 2 .

[0167] The backbone network of the student model is composed of a deep separable convolution layer, a single-layer LSTM, and a lightweight Transformer layer. The deep separable convolution decomposes the standard convolution into two independent steps of depth convolution and pointwise convolution, i.e.

[0168] Y = Y point (Y depth (X))

[0169] Y represents the final output feature map of the deep separable convolution; Y point represents the output of pointwise convolution (PointwiseConvolution); Y depth represents the output of depthwise convolution (Depthwise Convolution); X represents the input feature map, with dimensions [batch size, input channel number, sequence length], the deep separable convolution decomposes the standard convolution into two independent operations, significantly reducing the computational complexity and parameter quantity,

[0170] Deep convolution independently applies a convolution kernel to each channel of the input, capturing spatial features, calculated as follows:

[0171] Y depth [:, i, : ] = X[:, i, : ] * K i

[0172] X[:, i, : ] represents the i-th channel of the input feature map; K i represents the convolution kernel acting on the i-th input channel; the symbol * represents the convolution operation,

[0173] Point-wise convolution fuses channel information using a 1x1 convolution kernel:

[0174]

[0175] Y point [:, j, : ] represents the j-th channel of the point-wise convolution output; represents the number of channels of the input feature map; W i,j represents the weight parameter connecting the i-th input channel and the j-th output channel; the summation symbol ∑ represents the weighted summation operation on all input channels. Point-wise convolution uses a 1x1 convolution kernel to realize cross-channel information fusion, linearly combines the channel features obtained by deep convolution, and generates the final output feature;

[0176] This decomposition significantly reduces the parameter quantity and computational complexity, theoretically reducing the computational complexity of the standard convolution by about 9 times, while maintaining similar receptive fields and feature extraction capabilities. The student model is configured with two deep separable convolution blocks, with channel configurations:

[0177]

[0178] Compared with the 64-128-256 channel configuration of the teacher model, the feature dimension is greatly reduced.

[0179] The time series modeling module uses a single-layer LSTM network instead of the three-layer structure in the teacher model, reducing the number of hidden units from 256 to 64 while maintaining basic time series modeling capabilities and significantly reducing the parameter quantity. After LSTM output, layer normalization is also applied to ensure the stability of the feature distribution.

[0180] The last layer uses a lightweight Transformer layer, and the lightweight Transformer of the student model realizes efficient attention calculation through parameter sharing and dimension reduction. The query, key, and value matrices are shared, simplified as:

[0181]

[0182] Wherein: Q, K, V represent query matrix Query, key matrix Key and value matrix Value respectively.

[0183] And reduce the number of attention heads from 8 to 4; Simplify the feedforward network structure, adopt more compact dimension configuration. These optimizations make the parameter quantity of the Transformer layer greatly reduced, while retaining the core ability to capture global dependencies.

[0184] 4. Self-attention transfer strategy

[0185] The self-attention transfer mechanism is the core innovation of the present application. This mechanism guides the student model to learn the attention distribution pattern of the teacher model, achieving efficient knowledge transfer.

[0186] The self-attention transfer mechanism first needs to extract multi-head attention weights from the teacher model and the student model. In the Transformer architecture, each attention head generates an attention weight matrix, which represents the correlation strength between elements in the sequence. For the teacher model, the attention weight matrix can be represented as:

[0187]

[0188] Where B represents the batch size, H t represents the number of attention heads of the teacher model (8 in the present application), S t represents the sequence length after three times of pooling of the teacher model (25 in the present application).

[0189] Correspondingly, the attention weight matrix of the student model can be represented as:

[0190]

[0191] Where H s represents the number of attention heads of the student model (4 in the present application), S s represents the sequence length after two times of pooling of the student model (50 in the present application).

[0192] Due to the differences in the number of attention heads and sequence length between the teacher and student models, attention weight matrix alignment is needed. The alignment operation can be realized by an adaptive average pooling function f align :

[0193]

[0194] Where f align Specifically:

[0195]

[0196] where b denotes batch index; h denotes attention head index; i, j denote target index, range {0, 1,..., S t-1} and p, q denote source index, range {0, 1,..., S s-1}; R i denotes the set of source indices that map to target index i; |R i | denotes the size of set R i .

[0197] For adaptive average pooling, R i is constructed as follows:

[0198]

[0199] where for each position (p, q) in the attention weight matrix of the student model, it is mapped to position (i, j) in the attention weight matrix of the teacher model, and then the average value of all source positions that map to the same position is taken.

[0200] After the alignment of the attention weight matrix, the teacher model and the student model need to be aggregated respectively for the attention head dimension, and the information of multi-head attention is integrated into a single attention distribution:

[0201]

[0202] where |·| denotes the absolute value operation, and subscript i denotes the i-th attention head.

[0203] To ensure that the comparison is the relative pattern of the attention distribution rather than the absolute intensity, L2 norm normalization is applied to the aggregated attention weight matrix:

[0204]

[0205] where b denotes the b-th sample in the batch, and ||·||2 denotes the L2 Euclidean norm.

[0206] The attention transfer loss is obtained by calculating the L2 distance square between the normalized teacher and student attention vectors:

[0207]

[0208] where β is a hyperparameter that controls the weight of the attention transfer loss, and the typical value is set to 0.5. The 2 in the denominator is a normalization factor to keep the loss value in a proper range.

[0209] The total training loss is composed of the standard cross-entropy loss and the weighted attention transfer loss:

[0210]

[0211] where L CE denotes the cross-entropy loss function, y is the true label, is the prediction output of the student model.

[0212] Verification experiment:

[0213] The server used in the experiment of the application adopts an Intel Xeon Platinum 8474C 16-core processor, an NVIDIA RTX 4090D graphics processor, and is equipped with a 24GB graphics memory. Ubuntu 20.04 operating system is adopted, equipped with Python3.8 interpreter and PyTorch 1.11.0 deep learning framework, and GPU acceleration is realized through CUDA 11.3.

[0214] In the training process, the implementation of the self-attention transfer mechanism follows the following process: first, the input data is propagated through the forward propagation of the teacher model to extract the multi-head attention weight matrix, while the teacher model parameters are frozen and the gradient is not calculated; then the same input data is passed through the student model to extract the attention weight matrix and classification output of the student model; then the attention weight matrix alignment is performed, the attention transfer loss and the classification loss are calculated; finally, the weighted total loss is calculated, and only the student model parameters are updated, and after the training is completed, only the student model is retained for actual deployment, and the teacher model is only used as a knowledge source to participate in the training phase.

[0215] The human activity recognition method based on the lightweight hybrid neural network and the self-attention transfer proposed in the application is verified and evaluated through a series of experiments.

[0216] The experimental results show that the application can significantly reduce the model complexity while maintaining the recognition accuracy close to that of the teacher model, providing an effective solution for efficient human activity recognition in resource-constrained environments.

[0217] The confusion matrix of the following figure shows the accurate performance of the three models on the human activity recognition test set:

[0218] Figure 4 For the confusion matrix of the teacher model, it can be seen that the teacher model performs well and achieves a very high recognition accuracy on all activity categories, with an average recognition accuracy of more than 99%. The teacher model has very little confusion among categories, indicating its strong feature extraction and discrimination ability.

[0219] Figure 5 For the confusion matrix of the student model without using the self-attention transfer mechanism, it can be seen that the classification accuracy on each category has a very obvious decrease, which is due to the serious decline in model generalization ability caused by the simplification of model structure and parameters.

[0220] Figure 6 For the performance of the student model using self-attention transfer on the test set, it can be seen that compared with the student model without using self-attention transfer, the self-attention transfer can significantly improve the generalization ability of the student model. The confusion matrix data fully proves that by guiding the student model to learn the attention distribution pattern of the teacher model, the subtle temporal feature differences captured in the complex model can be effectively transmitted, thereby maintaining a high activity recognition accuracy while significantly reducing the model complexity.

[0221] Table 1 presents a comprehensive performance comparison of the teacher model and the student models trained in two modes.

[0222]

[0223]

[0224] Table 1

[0225] As can be seen from Table 1, although the teacher model has excellent recognition performance with an accuracy of 99.41%, its large parameter quantity (2M), storage requirement (15.03MB), and computational complexity (130M FLOPs) make it difficult to deploy on resource-constrained devices.

[0226] In contrast, the student models have achieved extremely significant lightweight, with a parameter quantity (0.06M) of only 3% of the teacher model, a storage space (only 0.75MB) reduced by about 95%, and a computational quantity (5M FLOPs) reduced by about 96%.

[0227] However, under the same lightweight architecture, the student model using the self-attention transfer mechanism significantly outperforms the traditionally trained student model in various performance indicators. The accuracy of the student model using self-attention transfer is 95.22%, which is only about 4 percentage points lower than that of the teacher model; while the traditionally trained student model, although having the same parameter quantity and computational complexity, has an accuracy of 91.13%, which is significantly lower than the model using self-attention transfer, and the performance gap with the teacher model is widened to more than 8 percentage points.

[0228] This comparison result powerfully proves the key role of the self-attention transfer mechanism in the knowledge compression process, which can effectively guide the lightweight model to learn the key temporal discrimination patterns captured by the complex model, while significantly reducing the computational burden, still maintaining the recognition ability close to the high-performance model, providing an ideal solution for human activity recognition in resource-constrained environments.

[0229] The application further provides a system comprising a memory, a control processor, and a computer program stored on the memory and executable on the control processor, wherein the control processor executes the program to implement the human activity recognition method based on lightweight hybrid neural network and self-attention transfer as described above.

[0230] The application further provides a computer readable storage medium storing computer executable instructions for implementing the human activity recognition method based on lightweight hybrid neural network and self-attention transfer as described above.

[0231] For the technical solutions of the application, there are various alternative implementation manners that can also achieve similar human activity recognition effects, but each has advantages and disadvantages.

[0232] In terms of lightweight model architecture, a MobileNetV2-based feature extraction backbone network can be used to replace the depth separable convolution layer, which has stronger feature extraction capability due to its inverted residual structure and linear bottleneck design; or a Gated Recurrent Unit (GRU) can be used to replace the LSTM, which has fewer parameters and more stable training, but has slightly insufficient long-term memory capability; or a Squeeze-and-Excitation attention module can be used to replace the Transformer layer to realize a lighter channel attention mechanism, but the global dependency modeling capability will be weakened.

[0233] In terms of knowledge transfer mechanism, a multi-level distillation method based on feature response can be considered, which transfers hierarchical feature knowledge by applying matching loss on feature maps at different depths of the network; or a feature correlation distillation method can be used, which focuses on the mutual relationship between features rather than absolute values.

[0234] In terms of sequence alignment technology, a Dynamic Time Warping (DTW) algorithm can be used to replace the adaptive pooling function to realize more flexible time sequence alignment, but the computational complexity will be significantly increased; or a learning sequence alignment network can be introduced to realize adaptive mapping through end-to-end training, but this will introduce additional parameters to increase the model complexity. In addition, advanced technologies such as contrastive representation learning or adversarial distillation can also be considered, but these methods are unstable in training and have high sensitivity to hyperparameters. In summary, although there are various alternative solutions, the time sequence attention transfer mechanism proposed by the application has unique advantages in balancing model complexity, computational efficiency, and recognition accuracy, and is particularly suitable for human activity recognition tasks in resource-constrained scenarios.

[0235] Although the methods are illustrated and described above as a series of acts, it will be appreciated that the methods are not limited by the order of acts, as some acts may, in accordance with one or more embodiments, occur simultaneously or in different orders than shown and described herein. It will also be appreciated that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality, without reference to a particular sequence of acts or specific hardware or software components. Such functionality can be implemented in a variety of ways, and in one embodiment, can be implemented using firmware or software embodied in computer-readable media. The term "computer-readable media" includes both machine-storage media and communications media. Machine-storage media includes any medium that provides (i.e., stores and / or transmits) data in a form accessible by a machine, including non- transitory machine-storage media, such as various types of flash memory, RAM, ROM, programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, magnetic tapes, magnetic disks, optical disks, hard drives, floppy disks, compact disks, DVDs, and other types of storage media. Communications media includes any medium that facilitates the communication of data or instructions, such as various types of electrical signals, electromagnetic signals, optical signals, radio frequency signals, and other types of signals.Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media can be any available media that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. Disk and disc, as used herein, include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray® disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0236] Although the above methods have been illustrated and described as a series of acts, it will be appreciated that not all of the acts need to be performed in the order illustrated or in any particular order, and that one or more acts can be performed concurrently or in any order with respect to one another. In addition, one or more acts can be performed at the same time as one or more other acts.

[0237] The above description is embodied in the best mode, and is not intended to limit the present application. Since any modification, equivalent, or improvement made within the idea and technical scope of the present application shall fall within the scope of the present application.

Claims

1. A human activity recognition method based on lightweight hybrid neural network and self-attention transfer, characterized in that, Comprise: Step 1, using HHAR dataset and preprocessing; Step 2, constructing a teacher model based on a hybrid architecture of CNN-LSTM-Transformer; Step 3, constructing a student model based on a hybrid architecture of CNN-LSTM-Transformer; Step 4, using a self-attention transfer mechanism to guide the student model to learn the attention distribution pattern of the teacher model, achieving efficient knowledge transfer.

2. The human activity recognition method based on lightweight hybrid neural network and self-attention transfer according to claim 1, characterized in that, The construction of the teacher model based on the hybrid architecture of CNN-LSTM-Transformer in step 2 includes, Step 21, convolutional feature extraction, using a three-layer cascaded residual convolution structure, the calculation of each residual convolution block can be represented as: H l = F l (H l-1 ) + G l (H l-1 ) where F l represents the main path mapping, G l represents the shortcut connection, when the number of input and output channels does not match, G l dimension adjustment is realized by 1x1 convolution; otherwise, it is an identity mapping, and the channel configuration of the three residual convolution blocks is: Step 22, using a multi-layer LSTM network to process the time-dependent relationship, the core calculation process of the LSTM unit is as follows: f t = σ(W f · [h t-1 , x t ]+ b f ) i t = σ(W i · [h t-1 , x t ]+ b i ) o t = σ(W o · [h t-1 , x t ]+b o ) h t = o t ⊙ tanh(C t ) where f t represents the forget gate output, mapped to the interval [0, 1] by a sigmoid activation function, controlling the retention degree of the cell state C t-1 at the previous time step; i t is the input gate output, also mapped by a sigmoid function, determining the update proportion of the current input information; C t represents the candidate cell state, mapped to the interval [-1, 1] by a tanh function, indicating the new information to be added; C t is the current cell state, obtained by adding the historical information controlled by the forget gate and the new information controlled by the input gate; o t is the output gate, generated by a sigmoid function, determining the amount of information transmitted from the cell state to the hidden state; h f is the current hidden state, i.e., the LSTM unit output, obtained by multiplying the output gate and the cell state activated by a tanh function, where W i , W C , W o , and W f are the weight matrices of the respective gates, b i , b C , b o are the corresponding bias vectors, and [h t-1 , x t ] represents the connection of the previous hidden state h t-1 and the current input x t in the feature dimension, and the symbol ⊙ represents the Hadamard product, i.e., element-level multiplication operation; The teacher model uses 3 layers of LSTM, each with 256 hidden units, and applies layer normalization after the LSTM output: H norm = LayerNorm(H lstm ) For LayerNorm layer normalization operation: H n orm represents the output feature after layer normalization; H l stm represents the original output feature of the LSTM network; Where the layer normalization operation is defined as: where γ and β are learnable scaling and shift parameters to recover the expressivity of the features; μ denotes the mean of the features at each sample; σ 2 denotes the variance of the features at each sample; ε is a constant for numerical stability to avoid division by zero errors; Step 23, through the self-attention mechanism of the Transformer layer, the direct relationship between all time steps in the sequence is established, and the calculation process of multi-head self-attention is as follows: MultiHead(Q, K, V) = Concat(head1,..., head h ) W O Where the calculation of each attention head is: For the multi-head self-attention mechanism of the Transformer layer, where: Q, K, V represent the query matrix Query, the key matrix Key, and the value matrix Value, respectively, which are representations of the input sequence after linear transformation; head i represents the output of the i-th attention head; is the weight matrix corresponding to the i-th attention head, used to project the input to different feature subspaces; W O is the linear projection matrix of the multi-head attention output; h represents the number of attention heads; Concat represents the concatenation operation in the feature dimension, The attention function is defined as: The softmax function normalizes the attention scores into a probability distribution, ensuring that the sum of all weights is 1; QK T represents the similarity matrix of queries and keys, measuring the strength of association between elements in the sequence; is a scaling factor, d k represents the dimension of the key vector, and the scaling operation can make the gradient more stable; V is the value matrix, containing the information of each element in the sequence.

3. The human activity recognition method based on lightweight hybrid neural network and self-attention transfer according to claim 1 or 2, characterized in that, In step 3, the backbone network of the student model includes a depth separable convolution layer, a single-layer LSTM network, and a lightweight Transformer layer, and the steps to build the student model include, Step 31, depth separable convolution decomposes standard convolution into two independent steps: depth convolution and pointwise convolution, that is: Y = Y point (Y depth (X)) Y represents the final output feature map of the depth separable convolution; Y point represents the output of the point-wise convolution; Y depth represents the output of the depth convolution; X represents the input feature map, the dimension of which is [batch size, input channel number, sequence length], the depth separable convolution significantly reduces the computational complexity and parameter quantity by decomposing the standard convolution into two independent operations, Depth convolution applies convolution kernels to each channel of the input independently to capture spatial features, and the calculation is as follows: Y depth [:,i,:] = X[:,i,:]*K i X[:, i, :] denotes the i-th channel of the input feature map; K i denotes the convolution kernel acting on the i-th input channel; the symbol * denotes the convolution operation, Pointwise convolution uses a 1x1 convolution kernel to fuse channel information: Y point [:,j,:] represents the jth channel of the pointwise convolution output; C in represents the number of channels of the input feature map; W i,j represents the weight parameter connecting the ith input channel and the jth output channel; the summation symbol ∑ represents the weighted summation operation on all input channels, the pointwise convolution uses a 1×1 convolution kernel to realize cross-channel information fusion, linearly combines the channel features obtained by deep convolution, and generates the final output feature; Step 32, after the output of the single-layer LSTM network, layer normalization is also applied to ensure the stability of the feature distribution; Step 33, the lightweight Transformer layer achieves efficient attention calculation and reduces the number of attention heads from 8 to 4 through parameter sharing and dimension reduction, and the query, key, and value matrices are shared and simplified as: Q = K = V = XW QKV Where: Q, K, V represent the query matrix Query, the key matrix Key, and the value matrix Value.

4. The human activity recognition method based on lightweight hybrid neural network and self-attention transfer according to claim 3, characterized in that, In step 3, the depth separable convolution layer is configured with two depth separable convolution blocks, and the channel configuration is:

5. The human activity recognition method based on lightweight hybrid neural network and self-attention transfer according to claim 4, characterized in that, In step 4, first, extract the multi-head attention weight from the teacher model and the student model, in the Transformer architecture, each attention head will generate an attention weight matrix, representing the association strength between elements in the sequence, for the teacher model, the attention weight matrix can be represented as: where B denotes batch size, H t denotes the number of attention heads of the teacher model, S t denotes the sequence length after three times of pooling of the teacher model, Correspondingly, the attention weight matrix of the student model can be represented as: where H s denotes the number of attention heads of the student model, S s denotes the sequence length after two pooling of the student model; Secondly, attention weight matrix alignment is performed, and the alignment operation can be implemented by an adaptive average pooling function f align Implementation: wherein f align Specifically: where b denotes batch index; h denotes attention head index; i, j denote target index, ranging from {0, 1,..., S t-1}; p, q denote source index, ranging from {0, 1,..., S s-1}; R i denotes the set of source indices that are mapped to target index i; |R i | denotes the size of set R i ; for adaptive average pooling, R i is constructed as follows: Where, for each position (p, q) in the attention weight matrix of the student model, calculate the position (i, j) it maps to in the teacher model's attention weight matrix, and then take the average of all source positions that map to the same position; Again, after aligning the attention weight matrix, the teacher model and the student model need to aggregate the attention head dimension respectively, and integrate the multi-head attention information into a single attention distribution: where |·| denotes the absolute value operation, and subscript i denotes the i-th attention head, To ensure that the comparison is the relative pattern of attention distribution rather than the absolute intensity, L2 norm normalization is applied to the aggregated attention weight matrix: where b denotes the b-th sample in the batch, and ||·||2 denotes the L2 Euclidean norm, The attention transfer loss is obtained by calculating the square of the L2 distance between the normalized teacher and student attention vectors: where β is a hyperparameter that controls the weight of the attention transfer loss, and 2 in the denominator is a normalization factor to keep the loss value within a proper range. The total training loss is composed of the standard cross-entropy loss and the weighted attention transfer loss: where L CE represents a cross-entropy loss function, y is a true label, is the prediction output of the student model.

6. The human activity recognition method based on lightweight hybrid neural network and self-attention transfer according to claim 5, characterized in that, In step 2, when constructing the teacher model, after obtaining the deep feature representation of the sequence, the model needs to convert the variable-length sequence features into fixed-length vector representation for final classification. The learning-based attention pooling mechanism is used to calculate as follows: where c denotes the global feature vector after attention weighting; T denotes the length of the input sequence, i.e., the total number of time steps; a t denotes the attention weight of the t-th time step; h t denotes the hidden state feature vector of the t-th time step; where the attention weight a t computed and normalized by the feedforward network: e t denotes the raw attention scores for the t-th time step; W a denotes the weight matrix for the attention computation, mapping the hidden states to the attention scores; b a denotes the bias parameter for the attention computation; the exp function denotes the exponential operation, converting the raw scores to positive values; the denominator is a normalization term over all time step attention scores, ensuring that the sum of all attention weights is 1, forming a valid probability distribution, The classification module is composed of multiple fully connected networks, and the final output is the class probability distribution: y = softmax(W3(ReLU(W2(ReLU(W1c + b1) + b2) + b3)) where y represents the final class probability distribution vector; c is the global feature vector from attention pooling, which is used as the input of the classifier; W1, W2, and W3 represent the weight matrices of the three fully connected networks, respectively, which realize nonlinear feature transformation; b1, b2, and b3 are the corresponding bias vectors; ReLU represents the rectified linear unit activation function; the softmax function converts the output of the last layer into a probability distribution, with each element representing the predicted probability of the corresponding activity class, and the sum of all probabilities is 1.

7. The human activity recognition method based on lightweight hybrid neural network and self-attention transfer according to any one of claims 1-3 or 5, characterized in that, The HHAR dataset is collected by multiple volunteers wearing smartphones while performing a series of standardized activities in a controlled environment, recording six common human activities, including walking, running, standing, sitting, climbing stairs, and descending stairs.

8. The human activity recognition method based on lightweight hybrid neural network and self-attention transfer according to claim 7, characterized in that, In step 1, preprocessing includes the following processing steps: data loading, data cleaning, time series segmentation, activity labeling, and dataset construction, Data loading first verifies the integrity and format consistency of the data file, ensures that the column names match the expected structure, and then converts the key sensor data columns to appropriate numerical types, laying the foundation for subsequent processing; Data cleaning is responsible for handling outliers and missing values in the original data, removing records containing missing values in sensor readings (x, y, z) or activity labels, ensuring data integrity; Time series segmentation uses a sliding window technique to divide continuous data streams into fixed-length time series segments, with a window size of 200 and a compensation of 100, i.e., a 50% overlap rate; Dataset construction, first use LabelEncoder to convert the activity label in text form into numerical encoding, and save the encoding mapping relationship, ensure that the prediction result can be correctly decoded in the inference stage, rearrange the dimension structure of the data segment from [sample number, time series length, feature channel number] to [sample number, feature channel number, time series length], adopt stratified sampling strategy to divide the dataset into training set, validation set and test set, keep the proportion of activity categories in each sub-set consistent, first divide the data into initial training set and test set according to the proportion of 8:2; Then divide the initial training set into final training set and validation set according to the proportion of 8:

2.

9. A system, characterized by A computer program product comprising a memory, a control processor and a computer program stored on the memory and executable on the control processor, the control processor executing the program to implement the human activity recognition method based on lightweight hybrid neural network and self-attention transfer according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer executable instructions for implementing the human activity recognition method based on lightweight hybrid neural network and self-attention transfer according to any one of claims 1-7.

Images