Mobile device human body action recognition method based on pulse neural network
Through the CSHAR model based on pulsed neural network, using built-in sensors for mobile devices to collect and process time series data, the problem of high computing resources and energy consumption in human behavior recognition is solved, and a more efficient recognition effect is achieved.
Patent Information
- Application Number
- CN202510483726.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-17
- Publication Date
- 2025-08-15
AI Technical Summary
When the prior art recognizes human behavior on sensor-based mobile devices, there are problems with high computing resources and energy consumption, and traditional ANN models are difficult to effectively process time series data, resulting in low recognition accuracy.
The CSHAR model based on pulsed neural network is adopted, including time slice units, position coding units, coordination attention mechanism units and ANN-SNN mapping units. Time series data is collected and normalized through built-in sensors, features are extracted and mapped into the SNN network, and identification is combined with a one-dimensional K nearest neighbor classifier.
It significantly reduces power consumption and improves the accuracy of human body movement recognition. It is suitable for mobile devices with high power consumption requirements and small sensors, improving recognition performance.
Smart Images
Figure CN120493041A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of behavior recognition technology, and more particularly to a method for human motion recognition in a mobile device based on a pulse neural network. Background Art
[0002] Extensive research has been conducted on human behavior recognition, with application cases in healthcare, security monitoring, smart homes, human-computer interaction, and other fields. Data acquisition methods can be categorized as sensor-based and video-based. Compared to sensor data, video data is large and susceptible to complex environmental influences, such as lighting, obstructions, and camera angles. Video equipment requires separate configuration and is expensive, making video-based human behavior recognition often more challenging. Sensor data offers inherent advantages in stability, lightweightness, personal privacy, and device compatibility. Because smartphone-based human behavior analysis does not require complex experimental environments or expensive equipment, smartphone-based human behavior recognition and analysis has attracted significant research interest.
[0003] Traditionally, researchers have used traditional artificial neural networks and simple classifiers to accomplish HAR tasks. With the development of deep learning, researchers have begun using various improved artificial neural networks to optimize accuracy, such as using convolutional neural networks with varying kernel dimensions to extract features, employing simple temporal attention within CNNs, or simply stacking CNNs, LSTMs, and other models and training a meta-learner on the stacked models. Because gradient descent optimization can automatically determine the most appropriate parameters, ANNs have demonstrated excellent performance on a variety of datasets.
[0004] However, because the 32-bit floating-point (FP32) operations commonly used by ANNs are more complex than integer operations, and many weight parameters and activation values in their networks are non-zero, i.e., they have low sparsity, ANNs typically require significant computational resources and energy. This is a significant issue for battery-powered wearable devices; for example, a 32-bit network requires 58 times more operations than a 1-bit network. Furthermore, standard ANNs are not adept at processing time series data because they lack mechanisms to capture temporal dependencies and cannot remember long-term dependencies. While this issue can be addressed using recurrent neural networks (RNNs) or long short-term memory networks (LSTMs), these models are typically more complex and computationally expensive.
[0005] Limited by the high efficiency and low energy requirements of wearable mobile devices, reducing the memory and computational costs of HAR models while maintaining accuracy has always been a key issue. Methods such as using conditional feature extractors and smaller, low-dimensional filters have reduced computational costs to some extent, but still lack user-side optimization.
[0006] Therefore, it is necessary to improve one or more problems existing in the above-mentioned related technical solutions.
[0007] It should be noted that this section is intended to provide background or context for the technical solutions of the present disclosure stated in the claims. The description herein is not admitted to be prior art by virtue of being included in this section. Summary of the Invention
[0008] The purpose of the embodiments of the present disclosure is to provide a method for human motion recognition in a mobile device based on a pulse neural network, thereby overcoming one or more problems caused by the limitations and defects of related technologies to at least a certain extent.
[0009] According to a first aspect of an embodiment of the present disclosure, a method for human motion recognition in a mobile device based on a spiking neural network is provided, the method comprising: Use the built-in sensors of mobile devices to collect user time series data and normalize the time series data; Construct a CSHAR model; the CSHAR model includes a feature extraction module and a classification recognition module. The feature extraction module includes a time slicing unit, a position encoding unit, a coordinated attention mechanism unit, and an ANN-SNN mapping unit. Use the feature extraction module to extract features from the normalized time series data to obtain time features; The one-dimensional K-nearest neighbor classifier of the classification and recognition module is used to classify the temporal features to obtain the action recognition results.
[0010] Furthermore, the step of collecting the user's time series data through the built-in sensor of the mobile device and normalizing the time series data includes: Utilize the built-in sensors of mobile devices to collect time series data of users; Normalize the time series data to a preset range of [0, 1].
[0011] Furthermore, the step of extracting features from the normalized time series data using a feature extraction module to obtain time features includes: The time slicing unit divides the normalized time series data into multiple time windows to capture temporal patterns. The time slicing unit divides the data into fixed time windows, and the window length is dynamically adjusted according to the sensor sampling rate. The position encoding unit assigns position information to each time window to indicate the data order; The coordinated attention mechanism unit generates location-aware attention weights by decomposing global pooling into feature encodings in the time dimension and sensor dimension; The ANN network in the ANN-SNN mapping unit extracts the features of the normalized time series data and maps the features to the SNN network in the ANN-SNN mapping unit. The ReLU-LIF conversion method is used to map the output of the ReLU activation function to the input current of the LIF neuron, and the gradient optimization is achieved through the proxy gradient method to obtain the time features.
[0012] Furthermore, the coordinated attention mechanism unit generates location-aware attention weights by decomposing the global pooling into feature encodings in the time dimension and sensor dimension, including: Pooling is performed along the time dimension to capture important information within the time window; Pooling along the sensor dimension to determine the sensor from which the data originated; The two-dimensional pooling results are connected and the intermediate features are generated through shared convolution. After segmentation, they are processed separately and the final attention weights are generated through broadcast addition.
[0013] Furthermore, the method further comprises: Dynamically adjust the activation threshold of LIF neurons. The threshold changes according to the exponential decay model based on historical spike activity. Specifically:
[0014] in, is the discharge threshold, is the time step, is the baseline threshold, is the increase in threshold after each activation, The time when the last pulse point was issued. is the time constant of threshold decay.
[0015] Furthermore, the membrane potential update formula of ReLU-LIF mapping is:
[0016] in, is the time step The membrane potential, is the time step The membrane potential, is the weight coefficient, is the input signal, is the output value of the ReLU function, and gradient optimization is achieved by replacing the triangular gradient function in the proxy gradient method:
[0017] in, for Presynaptic neuron membrane potential The partial derivative of is the membrane potential of the presynaptic neuron at time t, is the threshold voltage.
[0018] Furthermore, the output modulation formula of the coordinated attention mechanism is:
[0019] in, is the input feature, is the sigmoid function, is the output feature after the input information is modulated by the sensor dimension, is the output feature after the input information is modulated by the time dimension, ⊙ is the element-wise product, and ⊕ is the broadcast addition along the channel dimension.
[0020] The technical solutions provided by the embodiments of the present disclosure may have the following beneficial effects: In the embodiments of the present disclosure, the aforementioned spiking neural network-based method for human action recognition on mobile devices involves, firstly, the built-in sensors on the mobile device collect time series data from the user, which is then normalized to a fixed range suitable for feature extraction. In the feature extraction module, a time slicing unit divides the time series sensor data into smaller segments or time windows to capture temporal patterns, while a position encoding unit indicates the order or position of the data in each time window, ensuring that the data is fully prepared for feature extraction. A coordinated attention mechanism unit generates position-aware attention weights by decomposing global pooling into feature encodings in the time and sensor dimensions. The artificial neural network (ANN) extracts relevant features from the input data, while the ANN-SNN mapping unit maps these features to the SNN network. Through the spiking behavior of its neurons, the SNN network can effectively capture the temporal dynamics of the input data to extract temporal features. Finally, a one-dimensional K-nearest neighbor classifier is used for classification, thereby achieving recognition of user activities. Furthermore, this method significantly reduces power consumption, making it more suitable for human action recognition tasks based on mobile devices or small sensors with high power requirements, while also improving recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] The accompanying drawings are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and those skilled in the art can derive other drawings based on these drawings without inventive effort.
[0022] Figure 1 A method for human motion recognition of a mobile device based on a spiking neural network in an exemplary embodiment of the present disclosure is shown; Figure 2 A specific flow chart showing a method for human motion recognition on a mobile device based on a spiking neural network in an exemplary embodiment of the present disclosure is shown; Figure 3 FIG. 1 shows an overall framework diagram of the CSHAR model in an exemplary embodiment of the present disclosure; Figure 4 A schematic diagram showing a flow chart of an SNN network in an exemplary embodiment of the present disclosure; Figure 5 A schematic diagram showing a process of coordinating an attention mechanism unit in an exemplary embodiment of the present disclosure is shown; Figure 6 A diagram showing energy consumption comparisons of CNNbaseline, CNN with FS, RepHAR, SFCN, and GS-HAR on the UCI-HAR dataset in exemplary embodiments of the present disclosure is shown; Figure 7 A comparison of energy consumption on the HHAR dataset among CNNbaseline, CNN with FS, RepHAR, SFCN, and GS-HAR in an exemplary embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0023] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
[0024] In addition, the accompanying drawings are merely schematic illustrations of embodiments of the present disclosure and are not necessarily drawn to scale. Like reference numerals in the figures represent like or similar parts, and thus repeated descriptions thereof will be omitted. Some of the blocks shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically separate entities.
[0025] This example embodiment provides a method for human motion recognition on a mobile device based on a pulse neural network. Figure 1 As shown in , the method for human motion recognition of a mobile device based on a pulse neural network may include: steps S101 to S104.
[0026] Step S101: using the built-in sensor of the mobile device to collect the user's time series data, and normalizing the time series data; Step S102: constructing a CSHAR model; wherein the CSHAR model includes a feature extraction module and a classification recognition module, and the feature extraction module includes a time slicing unit, a position encoding unit, a coordinated attention mechanism unit, and an ANN-SNN mapping unit; Step S103: using a feature extraction module to extract features from the normalized time series data to obtain time features; Step S104: using the one-dimensional K-nearest neighbor classifier of the classification and recognition module to classify the time features to obtain an action recognition result.
[0027] In the aforementioned spiking neural network-based method for human action recognition on mobile devices, the built-in sensors on the mobile device collect time series data from the user. This data is then normalized to a fixed range suitable for feature extraction. In the feature extraction module, the time slicing unit divides the time series sensor data into smaller segments or time windows to capture temporal patterns, while the position encoding unit indicates the order or position of the data in each time window, ensuring that the data is fully prepared for feature extraction. The coordinated attention mechanism unit decomposes the global pooling into feature encodings in the time and sensor dimensions, generating position-aware attention weights. The ANN network extracts relevant features from the input data, while the ANN-SNN mapping unit maps these features to the SNN network. The spiking behavior of its neurons effectively captures the temporal dynamics of the input data to extract temporal features. Finally, a one-dimensional K-nearest neighbor classifier is used for classification, thereby achieving user activity recognition. Furthermore, this method significantly reduces power consumption, making it more suitable for power-sensitive human action recognition tasks on mobile devices or small sensors, while also improving recognition accuracy.
[0028] Below, we will refer to Figures 1 to 7 Each step of the above-mentioned method for human motion recognition on a mobile device based on a spiking neural network in this example embodiment is described in more detail.
[0029] In step S101 , the user's time series data is collected using the built-in sensor of the mobile device, and the time series data is normalized.
[0030] Specifically, such as Figure 2 The figure shows a detailed flow chart of the method for human motion recognition on mobile devices based on spiking neural networks. It mainly consists of three parts: (1) data collection and preprocessing, (2) feature extraction, and (3) classification and recognition. First, the built-in sensors on the mobile device collect the user's time series data, which is then normalized to a fixed range suitable for feature extraction.
[0031] In steps S102 to S104, a CSHAR model is constructed; wherein, the CSHAR model includes a feature extraction module and a classification and recognition module, and the feature extraction module includes a time slicing unit, a position encoding unit, a coordinated attention mechanism unit and an ANN-SNN mapping unit; the feature extraction module is used to extract features from the normalized time series data to obtain time features; the one-dimensional K nearest neighbor classifier of the classification and recognition module is used to classify the time features to obtain action recognition results.
[0032] Specifically, the time slicing unit divides the normalized time series data into multiple time windows to capture temporal patterns; wherein, the time slicing unit divides the data by fixed time windows, and the window length is dynamically adjusted according to the sensor sampling rate.
[0033] The position encoding unit assigns position information to each time window to indicate the data sequence.
[0034] The coordinated attention mechanism unit generates location-aware attention weights by decomposing global pooling into feature encodings in the time dimension and sensor dimension. Specifically, it includes: pooling along the time dimension to capture important information within the time window; pooling along the sensor dimension to determine the sensor from which the data originated; connecting the two-dimensional pooling results and generating intermediate features through shared convolution, which are then split and processed separately and the final attention weights are generated through broadcast addition.
[0035] The ANN network in the ANN-SNN mapping unit extracts the features of the normalized time series data and maps the features to the SNN network. The ReLU-LIF conversion method is used to map the output of the ReLU activation function to the input current of the LIF neuron, and the gradient optimization is achieved through the proxy gradient method to obtain the time features.
[0036] More specifically, Figure 3 As shown in the figure, the overall framework diagram of the CSHAR model.
[0037] 1. ANN-SNN conversion using ReLU-LIF mapping: like Figure 4The figure shows a flow chart of the SNN network. The ReLU activation function is a widely used activation function in artificial neural networks, and the LIF (Leaky Integrate-and-Fire) pulse neuron model is a simplified model of a biological neuron, which integrates the input signal and triggers a pulse when the membrane potential exceeds a certain threshold. The ReLU-LIF mapping technology aims to map the output of the ReLU activation function to the input current of the LIF neuron model, and approximate the linear part of the ReLU function to the membrane potential in the LIF neuron model, and use the LIF neuron to approximate the linear process of ReLU. For each discrete time step, the output value of the ReLU neuron is converted to the input current of the corresponding IF neuron by multiplying the output value by the weight coefficient and adding it to the membrane potential of the IF neuron. Mathematically, this can be expressed as: (1) in is the membrane potential at time step t, is the weight coefficient, is the input signal, is the output value of the ReLU function. Since the process of LIF neurons simulating temporal features and generating binary pulses is mathematically represented by a nonlinear pulse function, it is difficult to calculate gradients using conventional backpropagation algorithms. To address this problem, this application uses a proxy gradient method, using a proxy gradient function to approximate the behavior of the original discharge function near the threshold. This allows the calculation of non-zero gradients at these points, making gradient-based optimization possible. Specifically, a triangular surrogate gradient is used: (2) in, Presynaptic neuron membrane potential The partial derivative of , used to approximate the gradient of the non-differentiable pulse emission function, is the membrane potential of the presynaptic neuron at time t, is the threshold voltage, and when the membrane potential of a neuron reaches this value, an impulse (action potential) is triggered.
[0038] By constructing a function that gradually flattens near the firing threshold, the triangular surrogate gradient effectively provides an approximate gradient for the neuron's firing behavior near the threshold. This allows the algorithm to adjust the neuron's weights using gradient descent even when the membrane potential is at or near the firing threshold. This method is both simple and effective in implementation, allowing SNNs to be optimized using stochastic gradient descent.
[0039] At the same time, this application proposes an adaptive threshold model that allows LIF neurons to adaptively adjust their thresholds based on historical activity levels. After the neuron fires an action potential, the threshold is raised and gradually decays to the original threshold, thereby reducing the possibility of overfitting. The method can be calculated as follows: (3) in is the basic threshold, is the increase in threshold after each activation, The time when the last pulse point was issued. The time constant of the threshold decay is used to control the speed of the threshold recovery.
[0040] 2. Designed a lightweight "coordinated attention mechanism" to improve the network's recognition performance for human movements like Figure 5 The figure shows a flow chart of the coordinated attention mechanism unit. The global pooling of the traditional SE attention is decomposed into two one-dimensional feature encoding operations, which enables the attention module to accurately locate the information in the time series in two dimensions and capture long-term interactions. This method is applied to the HAR task. Specifically, given an input x, two pooling kernels with a spatial range of (1, W) or (H, 1) are used to perform pooling operations along the time dimension and sensor dimension respectively. The time dimension pooling captures important information or average trends within a specific time window, and the sensor dimension pooling determines which sensor the data at a given time point comes from. By combining the two, the complex relationship between the data can be captured. Therefore, for the output of feature extraction, the output of sensor s at time t can be expressed as: (4) Similarly, the output of the feature at sensor s at each time can be expressed as: (5) The above two transformations aggregate features along two dimensions respectively to generate position awareness, which is different from the squeezing operation in traditional SE attention methods. These two transformations also allow the attention block to capture long-range dependencies along one temporal direction and retain precise position information along another spatial direction, which helps the network locate objects of interest more accurately.
[0041] Equations (4) and (5) describe the realization of the global receptive field and the encoding of position information. In order to make the design as simple and low-cost as possible so that it can be applied to wearable mobile devices, it can accurately capture position information, highlight the area of interest, and capture the relationship between data. The two aggregation methods are then connected and fed into a shared 1×1 convolution F: (6) in is a nonlinear activation function, and f is an intermediate feature used to encode position information in two dimensions. The feature encoding process is similar to the module in SE attention, which contains the control mechanism of ratio and size. Then f is split into two independent tensors and , allowing the model to process and analyze information in different dimensions separately. Then use two 1×1 convolutions and The two tensors are transformed separately, and after the transformation, the features processed by their respective dimensions are ensured to be consistent with the original input, so as to facilitate subsequent feature fusion or further processing.
[0042] (7) (8) Finally, the output and will be expanded and used as attention weights respectively. Finally, the output Y of the coordinated attention block can be written as: (9) Among them, ⊙ represents element-wise product (Hadamard product), and ⊕ represents broadcast addition along the channel dimension. Under this mechanism, through and The calculated attention weights are processed and used to modulate each feature in the input data x. This coordinate-weighted approach allows the model to focus more closely on the key data of the features while maintaining low computational complexity, thereby improving overall performance and efficiency.
[0043] In a specific embodiment, the present application designs a general "coordinated attention" mechanism. Compared with the existing attention mechanism, it improves the performance of the network in processing time series information with a more lightweight network structure, and is more suitable for the sparse characteristics of HAR task sensor data; by focusing attention on the time dimension and spatial dimension of the sensor respectively, and combining channel compression and expansion, the feature expression capability is enhanced.
[0044] A new SNN algorithm, the CSHAR model, is designed to adaptively adjust the trigger threshold. Its recognition accuracy is superior to existing fixed-threshold methods. The proposed algorithm also features low power consumption and is suitable for HAR tasks based on mobile sensors.
[0045] Finally, the coordinated attention mechanism is embedded into the CSHAR model, which significantly reduces the power consumption of HAR tasks in mobile devices while maintaining high accuracy and improves the accuracy of action recognition.
[0046] In a specific embodiment, six ANN benchmark models, one benchmark SNN model and the CSHAR model of the present application were used for comparative experiments. Each model was run 5 times and the average value was taken. The experimental results are shown in Table 1. From the experimental results, it can be seen that compared with the traditional SNN, the CSHAR model has an average improvement of 2.3% in recognition accuracy in three data sets, and these improvements are very significant. Compared with other ANN networks, except that the performance on the UCI-HAR data set is slightly worse than that of CNN with FS, the performance of the CSHAR model is also better than that of other ANN networks on other data sets, and the amount of computation is greatly reduced. At the same time, the performance of GS-HAR under different seeds is more stable than that of other models.
[0047] Table 1 Performance of ANN, improved ANN model, FSCN and GS-HAR network models on three datasets
[0048] This application mainly evaluates the performance of the proposed CSHAR model, the baseline SNN model, the ReLU-based ANN baseline, and the improved CNN model on the UCI-HAR and HHAR datasets based on two indicators: activation sparsity and energy consumption. Figure 6 and Figure 7 The results are visualized in the figure (blue chart). ANNs typically have a sparsity of around 45%, while LIF neurons show a higher sparsity of around 75%. Therefore, SNNs can save more operations in inference and avoid using more weights for calculation. The energy consumption was estimated using the SATA energy simulator. The total energy considered consists of two parts: computational energy and data movement energy. Due to the binary activation and higher sparsity of SNNs, they have an advantage in computational energy consumption. Figure 5 On the right, CSHAR reduces power consumption by 74.7% on UCIHAR and 73% on HHAR compared to traditional artificial neural networks. Compared with FSCN, although the model of this application adopts a coordinated attention mechanism and an adaptive threshold model, due to the lightweight mechanism, the CSHAR model achieves a significant improvement in task accuracy without significantly increasing power consumption compared to traditional SNNs. As for the improved model of ANN, although the recognition effect has been improved, it has paid a huge energy price due to the complexity of its network. At the same time, in the field of HAR, the data movement energy of SNN is also lower than that of ANN. The input data in HAR is enhanced multiple times to generate features in the time dimension, but SNN does not need to increase the dimension of the intermediate features to adapt to the time dimension, thereby reducing the cost of data movement. In short, SNN not only brings better task performance, but also brings great efficiency in energy consumption. Among them, Figure 6 The energy consumption comparison chart of CNNbaseline, CNN with FS, RepHAR, SFCN and GS-HAR on the UCI-HAR dataset; the energy consumption of CNN is normalized to 1; Figure 6 (A) is activation sparsity; Figure 6 (B) Calculate energy; Figure 6 (C) Data movement energy. Figure 7 The energy consumption comparison chart of CNNbaseline, CNN with FS, RepHAR, SFCN and GS-HAR on the HHAR dataset; the energy consumption of CNN is normalized to 1; Figure 7 (A) is activation sparsity; Figure 7 (B) is the calculated energy; Figure 7 (C) is the data movement energy.
[0049] In the aforementioned spiking neural network-based method for human action recognition on mobile devices, the built-in sensors on the mobile device collect time series data from the user. This data is then normalized to a fixed range suitable for feature extraction. In the feature extraction module, the time slicing unit divides the time series sensor data into smaller segments or time windows to capture temporal patterns, while the position encoding unit indicates the order or position of the data in each time window, ensuring that the data is fully prepared for feature extraction. The coordinated attention mechanism unit decomposes the global pooling into feature encodings in the time and sensor dimensions, generating position-aware attention weights. The ANN network extracts relevant features from the input data, while the ANN-SNN mapping unit maps these features to the SNN network. The spiking behavior of its neurons effectively captures the temporal dynamics of the input data to extract temporal features. Finally, a one-dimensional K-nearest neighbor classifier is used for classification, thereby achieving user activity recognition. Furthermore, this method significantly reduces power consumption, making it more suitable for power-sensitive human action recognition tasks on mobile devices or small sensors, while also improving recognition accuracy.
[0050] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification.
[0051] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the appended claims.
Claims
1. A method for human motion recognition in mobile devices based on pulse neural networks, characterized in that: The method includes: Use the built-in sensors of mobile devices to collect user time series data and normalize the time series data; Construct a CSHAR model; the CSHAR model includes a feature extraction module and a classification recognition module. The feature extraction module includes a time slicing unit, a position encoding unit, a coordinated attention mechanism unit, and an ANN-SNN mapping unit. Use the feature extraction module to extract features from the normalized time series data to obtain time features; The one-dimensional K-nearest neighbor classifier of the classification and recognition module is used to classify the temporal features to obtain the action recognition results.
2. The method for human motion recognition of mobile devices based on pulse neural network according to claim 1, characterized in that: The steps of collecting user time series data through the built-in sensor of the mobile device and normalizing the time series data include: Utilize the built-in sensors of mobile devices to collect time series data of users; Normalize the time series data to a preset range of [0, 1].
3. The method for human motion recognition of mobile devices based on pulse neural network according to claim 1, characterized in that: The step of extracting features from the normalized time series data using a feature extraction module to obtain time features includes: The time slicing unit divides the normalized time series data into multiple time windows to capture temporal patterns. The time slicing unit divides the data into fixed time windows, and the window length is dynamically adjusted according to the sensor sampling rate. The position encoding unit assigns position information to each time window to indicate the data order; The coordinated attention mechanism unit generates location-aware attention weights by decomposing global pooling into feature encodings in the time dimension and sensor dimension; The ANN network in the ANN-SNN mapping unit extracts the features of the normalized time series data and maps the features to the SNN network in the ANN-SNN mapping unit. The ReLU-LIF conversion method is used to map the output of the ReLU activation function to the input current of the LIF neuron, and the gradient optimization is achieved through the proxy gradient method to obtain the time features.
4. The method for human motion recognition of mobile device based on pulse neural network according to claim 3, characterized in that: The coordinated attention mechanism unit generates location-aware attention weights by decomposing the global pooling into feature encodings in the time dimension and sensor dimension, including the following steps: Pooling is performed along the time dimension to capture important information within the time window; Pooling along the sensor dimension to determine the sensor from which the data originated; The two-dimensional pooling results are connected and the intermediate features are generated through shared convolution. After segmentation, they are processed separately and the final attention weights are generated through broadcast addition.
5. The method for human motion recognition of mobile device based on pulse neural network according to claim 4, characterized in that: The method further includes: Dynamically adjust the activation threshold of LIF neurons. The threshold changes according to the exponential decay model based on historical spike activity. Specifically: in, is the discharge threshold, is the time step, is the baseline threshold, is the increase in threshold after each activation, The time when the last pulse point was issued. is the time constant of threshold decay.
6. The method for human motion recognition of mobile devices based on pulse neural networks according to claim 5, characterized in that: The membrane potential update formula of ReLU-LIF mapping is: in, is the time step The membrane potential, is the time step The membrane potential, is the weight coefficient, is the input signal, is the output value of the ReLU function, and gradient optimization is achieved by replacing the triangular gradient function in the proxy gradient method: in, for Presynaptic neuron membrane potential The partial derivative of is the membrane potential of the presynaptic neuron at time t, is the threshold voltage.
7. The method for human motion recognition of mobile device based on pulse neural network according to claim 6, characterized in that: The output modulation formula of the coordinated attention mechanism is: in, is the input feature, is the sigmoid function, is the output feature after the input information is modulated by the sensor dimension, is the output feature after the input information is modulated by the time dimension, ⊙ is the element-wise product, and ⊕ is the broadcast addition along the channel dimension.