Lightweight human behavior recognition methods, systems, and devices based on wearable sensors

Abstract

This invention provides a lightweight human behavior recognition method, system, and device based on wearable sensors. The method includes: collecting data from multiple sensors and preprocessing it; calculating the Euler angles of the multiple sensor data as fused data; initializing the fused data as input data; constructing a human behavior recognition network based on machine learning; processing the input data; and outputting the recognition result. The human behavior recognition network includes a sequentially connected improved bidirectional gated recurrent network, a combined attention network, and a dendritic network; and recognizing human behavior based on the recognition result. This solution can reduce the complexity of the network model and significantly reduce computational overhead. Through the improvement, combination, and application of the three-part network structure, it has significant advantages over current human behavior recognition algorithms in terms of model complexity, computational overhead, and recognition accuracy.

Description

Technical Field

[0001] This invention relates to the fields of multi-sensor data processing, machine learning, and human behavior data processing and recognition, specifically to a lightweight human behavior recognition method, system, and device based on wearable sensors. Background Technology

[0002] Human behavior recognition has demonstrated positive impacts in various fields, such as healthcare, sports training, and film and television production. Mainstream human behavior recognition methods primarily utilize optical sensors, electromagnetic sensors, and wearable sensors. Optical sensor-based methods capture image sequences of user activities using cameras to perform classification, but suffer from issues such as privacy concerns, environmental limitations, and inconvenience in carrying the device. Electromagnetic sensor-based methods acquire limb data from emitted and reflected electromagnetic waves to perform classification, but suffer from low accuracy, environmental limitations, and severe noise interference. Wearable sensor-based methods acquire human activity data using wearable sensors to achieve behavior recognition, offering multiple advantages such as low power consumption, small size, high accuracy, and no spatial limitations or occlusion issues.

[0003] While wearable sensor-based human behavior recognition offers numerous advantages, the sensor data cannot be directly applied to behavior recognition. In practical use, wearable sensors require data processing based on factors such as the orientation during installation and the relative displacement with the limbs during movement, in order to acquire accurate human motion data.

[0004] In recent years, benefiting from the rapid development of machine learning, numerous methods for human behavior recognition based on wearable sensors have been proposed. Existing technologies mainly include the following approaches: 1. Using a random forest binary classifier to first classify human behavior into static and dynamic activities, then employing a support vector machine to identify static activities, and a deep learning model based on convolutional neural networks to identify movement activities. 2. Using inertial sensor data from smartphones and smartwatches as input, then using a combined model of convolutional networks and bidirectional gated recurrent units to identify individual behavior. 3. Using accelerometer data, processing total acceleration, linear acceleration, and angular velocity separately using convolutional neural networks, and finally integrating the features through dense layers. These methods exhibit good performance in human motion recognition; however, they possess high complexity in feature processing and model structure. Summary of the Invention

[0005] In view of the problems existing in the prior art, this invention proposes a new lightweight human behavior recognition method based on wearable sensors. The model combines an improved BGRU, a combined attention network (CoDA), and a dendritic network (DD). It maintains a lightweight structure while achieving a high recognition rate, reducing training and inference time. Compared with traditional CNN networks and other structures, it has greater advantages.

[0006] Specifically, the present invention provides the following technical solutions: On one hand, the present invention provides a lightweight human behavior recognition method based on wearable sensors, the method comprising: S1. Collect data from multiple sensors, preprocess the data, calculate the Euler angles of the multiple sensor data as fused data, initialize the fused data as input data; the multiple sensors include a gyroscope, an accelerometer, and a magnetometer; S2. Construct a human behavior recognition network, process the input data, and output the recognition result; the human behavior recognition network includes a sequentially connected improved bidirectional gated recurrent network, a combined attention network, and a dendritic network; S3. Based on the recognition results, identify human behavior.

[0007] Preferably, in step S1, the preprocessing includes: The collected sensor data is filtered and fused to obtain fused data; Define the initial attitude and collect initial attitude data for a fixed duration. Calculate the average value of the initial attitude data as the static deviation. In the initial posture, actual data of nodes wearing multiple sensors are collected, and the error is calculated based on the difference in the actual data of the nodes to obtain the wearing error; When initializing the fused data, static bias is removed and error compensation is performed based on the static error to form the input data.

[0008] Preferably, the improved bidirectional gated loop network consists of a forward gated loop unit and a reverse gated loop unit. The structure of the gated loop unit has been improved, and the update gate and reset gate are calculated using only the hidden state of the previous moment. The gated cyclic unit model is as follows: ; ; in, This indicates the hidden state of the previous time step. Indicates an update to the door. This represents the weight matrix used to update the hidden state at the current time step in front of the gate. The weight matrix represents the hidden state at the current time step before resetting the gate. This indicates that the door is being reset. This represents the Sigmoid activation function.

[0009] Preferably, the combined attention network model is: ;

[0010] ;

[0011] ; in, Indicates pairwise affinity between elements. Parametric functions representing linear / nonlinear projections, Represents the elements of sequence matrix A. Represents the elements of sequence matrix B. Represents the scaling constant. This represents a measure of dissimilarity between elements. Represents the scaling constant. Parametric functions representing linear / nonlinear projections, express norm, This represents the combined attention matrix.

[0012] Preferably, a dendritic network is used as the decision layer of the human behavior recognition network, and the dendritic network model is as follows: ; in, This represents the input to the dendritic network. This represents the output of the dendritic network. Represents positive integers. Indicates the total number of dendritic units. This indicates a specific module. This represents the weight matrix.

[0013] Preferably, the preprocessing further includes performing fusion filtering on the filtered data from multiple sensors, wherein the fusion filtering includes prediction and update equations: The prediction equation is: ; ; The update equation is: ; ; ; in, Indicates the predicted state value. and Let represent the optimal estimates at times k and k-1, respectively. This represents the state prediction covariance. and This represents the state estimation covariance at times k and k-1. Let A represent the Kalman state estimation covariance gain, B represent the control variable matrix, and A represent the state transition matrix. Let Q represent the state control vector, R represent the system noise covariance matrix, I represent the measurement noise covariance matrix, and I represent the identity matrix. This represents the actual measured value. This represents the measurement matrix.

[0014] Preferably, the combined attention network uses two sets of outputs from a gated recurrent network as its input sequence; Calculate the pairwise affinity matrix of the first set of input sequences and the dissimilarity metric matrix of the second set of input sequences; The pairwise affinity matrix is ​​activated by the tanh function; the dissimilarity metric matrix is ​​activated by the sigmoid function. The activation results are then subjected to Hadamard matrix operations to obtain the combined attention matrix.

[0015] Preferably, the pairwise affinity matrix is ​​calculated as follows: ; in, Let represent the value of the element in the i-th row and j-th column of the pairwise affinity matrix. Parametric functions representing linear / nonlinear projections, This represents the elements of the first input sequence. This represents the elements of the second set of input sequences. This represents the scaling constant.

[0016] Preferably, the dissimilarity metric matrix is ​​calculated as follows: ; in, This represents the elements of the first input sequence. This represents the elements of the second set of input sequences. This represents the value of the element in the i-th row and j-th column of the dissimilarity metric matrix. Represents the scaling constant. Parametric functions representing linear / nonlinear projections, express The norm of .

[0017] Preferably, the plurality of sensors include a gyroscope, an accelerometer, and a magnetometer.

[0018] On the other hand, the present invention also provides a lightweight human behavior recognition system based on wearable sensors, the system comprising: Multiple sensors are used to collect human behavior data; The data preprocessing module is used to preprocess human behavior data collected by multiple sensors, calculate the Euler angles of the data from multiple sensors as fused data, and initialize the fused data as input data. The data recognition module includes a constructed human behavior recognition network, which processes the input data and outputs the recognition result; the human behavior recognition network includes a sequentially connected improved bidirectional gated recurrent network, a combined attention network, and a dendritic network. The human behavior recognition module is used to recognize human behavior based on the recognition results.

[0019] In another aspect, the present invention also provides a lightweight human behavior recognition device based on wearable sensors, the device including multiple sensors, a processor and a memory; The multiple sensors are worn as wearable devices on the human body; The processor calls the computer instructions stored in the memory to execute the lightweight human behavior recognition method based on wearable sensors as described above.

[0020] Compared with existing technologies, the technical solution of this invention can reduce network model complexity and significantly reduce computational overhead. This solution reduces the number of parameters for update and reset gates by improving BGRU, and assigns different weights to learned features using a combined attention network. The combined attention mechanism has superior performance compared to traditional attention mechanisms. Dendritic networks replace dense layer decision-making, reducing computational complexity and improving the logical relationship between input and output. Through the improvement, combination, and application of these three parts, this solution has significant advantages in model complexity, computational overhead, and recognition accuracy compared to current human recognition algorithms. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the human behavior recognition system framework according to an embodiment of the present invention; Figure 2 This is a flowchart of human behavior recognition according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the improved BGRU network framework according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the CoDA network framework according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the DD network framework according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the system structure according to an embodiment of the present invention. Detailed Implementation

[0023] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0024] Those skilled in the art should understand that the following specific embodiments or implementation methods are a series of optimized configurations listed to further explain the specific content of the invention. These configuration methods can be combined or used in conjunction with each other, unless the invention explicitly states that some or a specific embodiment or implementation method cannot be associated with or used in conjunction with other embodiments or implementation methods. Furthermore, the following specific embodiments or implementation methods are merely optimized configurations and are not intended to limit the scope of protection of the invention.

[0025] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0026] The framework of this article is as follows Figure 1 As shown, raw data is first extracted using wearable devices and their various sensors. This raw data is then preprocessed and trained using a lightweight human behavior recognition machine learning model (LWearSor) based on wearable sensors. Classification is then performed to complete the recognition of human behavior. The following section combines... Figure 1 , Figure 2 The specific identification process of this scheme will be explained.

[0027] 1. Data Preprocessing First, the collected raw data needs to be preprocessed. In this embodiment, the preferred processing method is as follows: (1) Filtering of the original signal Due to noise interference from the environment, sensors, or other sources, the raw data contains noise interference, requiring filtering of the acquired data. Preferably, in this embodiment, Butterworth filtering is used to remove noise interference from the environment, sensors, or other sources, making the signal clearer. Furthermore, this filtering can reduce high-frequency components in the signal, making the signal smoother and thus obtaining more stable measurement results. In this embodiment, three types of sensors are used as examples: gyroscope, accelerometer, and magnetometer.

[0028] (2) Data fusion operation Because gyroscopes have cumulative integration errors during use, accelerometers cannot measure vertical rotation, and magnetometers can only measure the north and south magnetic field strength, using data from only one or two sensors as input introduces errors. However, using all the data would result in an excessively large input dataset. To reduce the amount of data used, it is necessary to fuse the data from the gyroscope, accelerometer, and magnetometer. In this solution, if... Figure 1 As shown, the data after filtering in step (1) above is fused using a fusion filter to combine the gyroscope, accelerometer and magnetometer. This not only combines the advantages of the three sensors, but also reduces the data volume to the size of a single sensor.

[0029] The input consists of gyroscope, accelerometer, and magnetometer data after filtering in step (1) above. The Euler angle obtained by integrating the gyroscope data is the optimal estimated value, and the Euler angle obtained from the accelerometer and magnetometer data is the actual measured value. Fusion filtering and iterative solution are performed. The final Euler angle solution combines the advantages of the gyroscope, accelerometer, and magnetometer, which is the optimal attitude solution. The formula is as follows: predict: ; ; renew: ; ; ; In the above formula, Indicates the predicted state value. and Let represent the optimal estimates at times k and k-1, respectively. This represents the state prediction covariance. and This represents the state estimation covariance at times k and k-1. Let A represent the Kalman state estimation covariance gain, B represent the control variable matrix, and A represent the state transition matrix. Let Q represent the state control vector, R represent the system noise covariance matrix, I represent the measurement noise covariance matrix, and I represent the identity matrix. This represents the actual measured value. This represents the measurement matrix.

[0030] (3) Data initialization Preferably, in this embodiment, the data after data fusion in step (2) above needs to be initialized, including eliminating static errors and eliminating wearing errors. The initialized data can then be used as input data for a lightweight human behavior recognition network.

[0031] Eliminating static error: The initial state of the human body when standing is defined as the standard posture for initialization. In this process, data (i.e., initial posture data) after data fusion in step (2) above is collected for a fixed duration (e.g., 1 minute, 2 minutes, etc.) and the average value is calculated as the static error. Subsequent data collected after data fusion in step (2) above is subtracted from the static error to eliminate the static error.

[0032] Eliminating Wearing Errors: When collecting data after data fusion in step (2) above, due to differences in each person's limbs and the different orientations when wearing the device, it is necessary to eliminate wearing errors during installation. For example, the elimination of wearing errors can be achieved in the following way: First, it is stipulated that the human body stands vertically downward and facing south, with the y-axis of the sensor pointing in the positive direction of gravity and the x-axis pointing south. Therefore, the gravitational acceleration value should be a=[0,a,0] and the magnetometer value should be g=[0,g,0]. Then, the actual gravitational acceleration and magnetic field strength of the wearing node are measured. Through the theory of rotation matrix, the deviations between the measured values ​​of gravitational acceleration and magnetic field strength and the true values ​​are calculated respectively. The final wearing error is obtained by fusing these two deviations. The subsequent data collected after data fusion in step (2) above needs to be subtracted from this wearing error to eliminate the error caused by wearing.

[0033] 2. Lightweight human behavior recognition network, namely LWearSor network Next, the identification network structure proposed in this scheme will be described.

[0034] (1) Improved bidirectional gated loop network (i.e., improved BGRU) In this embodiment, the bidirectional gated recurrent network is an extension of the GRU (Gated Recurrent Unit). It consists of two GRUs: one is a forward GRU model that learns from the forward input; the other is a backward GRU model that learns from the backward input. It can simultaneously consider the past and future information of the input sequence, better capturing patterns and relationships in the sequence data. The formula is as follows: ; ; ; In the above formula, Indicates a positive GRU, Indicates reverse GRU, Represents the input vector. This indicates the output result, and GRU() represents a gated loop unit.

[0035] The formula for the GRU model is as follows: ; ; In the above formula, Indicates the updated hidden state. Indicates the candidate hidden state. Indicates an update to the door. Represents the weight matrix. Represents the weight matrix. This indicates that the door is being reset. Let g represent the bias vector with dimension 1*h, and g() represent the tanh activation function.

[0036] The expressions for the two gate units are: ; ; In the above formula, Indicates an update to the door. This represents the Sigmoid activation function. , , , This represents the input weight matrix. Represents the input vector. This indicates the hidden state at the previous moment. This indicates that the door is being reset. , This represents the bias vector.

[0037] Assumption It is the (external) m-dimensional input vector at time t. If it is an n-dimensional hidden state, then according to the rules of matrix operations, we know... It is an n×m matrix. It is an n×n matrix, and Since it is an n×1 matrix, the number of parameters for the two gates is... .

[0038] In the BGRU gating mechanism, GRU regulates the information flow of the hidden state through update and reset gates to solve the problems of long-term dependencies and gradient vanishing. The weights of these gates are updated via backpropagation stochastic gradient descent. Each parameter update involves information about the entire network state, leading to redundancy. Therefore, in this embodiment, more preferably, we only use the hidden state from the previous time step to calculate the update and reset gates. That is, the two gated recurrent units of the improved gated recurrent network preferably adopt the following formula: ; ; The parameters of the two gates are The number of parameters has been reduced. .

[0039] Improved BGRU network framework such as Figure 3 As shown, when calculating the update gate and reset gate of GRU, only the hidden state of the previous time step is used, eliminating the input and bias, which greatly reduces the total number of parameters.

[0040] (2) Coordinated Attention Network (CoDA) CoDA, or Combinatorial Attention Network, learns whether to add, subtract, or remove a vector during learning. Compared to traditional attention mechanisms, it learns a pool of token combinations instead of learning relative importance (i.e., traditional weighted sums) before deciding whether to add, subtract, or remove input tokens, resulting in better performance. CoDA uses a tanh transformation on the pairwise affinity matrix E to control vector addition and subtraction, and a sigmoid transformation on the pairwise dissimilarity metric matrix N to introduce a gating mechanism, erasing unnecessary pairwise scores to zero when necessary. In this embodiment, the Combinatorial Attention Network we use is as follows: Figure 4 As shown, the input consists of two sets of feature sequences output by the improved BGRU network. First, for the two different input sequences 1 and 2, their respective pairwise affinity matrices E and dissimilarity metric matrices N are calculated. For example, the pairwise affinity matrix E is calculated based on input sequence 1, and the dissimilarity metric matrix N is calculated based on input sequence 2. Then, the pairwise affinity matrix E is activated using tanh, and the dissimilarity metric matrix N is activated using... After activation, the results from both parts are subjected to Hadamard matrix operations to obtain the features with combined attention. The calculation formula is as follows: ; ; ;

[0041] in, Let represent the value of the element in the i-th row and j-th column of the pairwise affinity matrix E. Parametric functions representing linear / nonlinear projections, This represents the elements of the input sequence matrix (denoted as matrix A). This represents the elements of the input sequence matrix 2 (denoted as matrix B). Represents the scaling constant. Let represent the value of the element in the i-th row and j-th column of the dissimilarity metric matrix N. Represents the scaling constant. Parametric functions representing linear / nonlinear projections, express norm, This represents the combined attention matrix.

[0042] (3) Dendritic network (DD) For example, this invention uses a DD network as the final decision network. This network can represent whether the logical relationship between input and output is highly correlated, and has higher expressive power than residual connected dense layers, which can only represent additive relationships. The combined attention matrix is ​​used as the input data for the dendritic network. In this embodiment, the dendritic network is designed with multiple dendritic network units (i.e., dendritic units).

[0043] We define the expression for a single DD network model as follows: ; Where X is the input of DD, A 0 Let X, A l-1 A represents the output of the previous dendritic network. l W represents the current output of the dendritic network. l,l-1 It is A l-1 The weight matrix, This is the Hadamard product, a binary operation that takes two matrices of the same dimension and produces another matrix of the same dimension as operands, where each element i, j is the product of the elements i, j of the original two matrices. In this embodiment, the dendritic network we use is as follows: Figure 5 As shown, the feature matrix (i.e., the combined attention matrix) of the CoDA output is used as input, and then processed by the DD module. The result of each processing step is used as the input for the next step, and the classification result is output after the final processing. The complete DD architecture model is as follows: ; Where P+ represents a positive integer, and L represents the total number of dendritic units. This indicates a specific module. This represents the weight matrix.

[0044] Thus, through the dendritic network's identification and classification, and based on the data acquired by the sensors, the identification results of human behavior are obtained.

[0045] In yet another specific embodiment, this solution can also be implemented systematically. (See reference...) Figure 6 As shown, the system includes: Multiple sensors are used to collect human behavior data; The data preprocessing module is used to preprocess human behavior data collected by multiple sensors, calculate the Euler angles of the data from multiple sensors as fused data, and initialize the fused data as input data. The data recognition module includes a constructed human behavior recognition network, which processes the input data and outputs the recognition result; the human behavior recognition network includes a sequentially connected improved bidirectional gated recurrent network, a combined attention network, and a dendritic network. The human behavior recognition module is used to recognize human behavior based on the recognition results.

[0046] Preferably, the plurality of sensors include a gyroscope, an accelerometer, and a magnetometer.

[0047] Preferably, the preprocessing in the above data preprocessing module includes: The collected sensor data is filtered, and the fused data is calculated. Define the initial attitude and collect initial attitude data for a fixed period of time, and calculate the average value of the initial attitude data as the static error; In the initial posture, actual data of nodes wearing multiple sensors are collected, and the error is calculated based on the difference in the actual data of the nodes to obtain the wearing error; When initializing the fused data, static bias is removed and error compensation is performed based on wearing error to form input data.

[0048] Preferably, the improved bidirectional gated loop network consists of a forward gated loop unit and a reverse gated loop unit. The structure of the gated loop unit has been improved, and the update gate and reset gate are calculated only using the hidden state of the previous moment. The gated cyclic unit model is as follows: ; ; in, This indicates the hidden state of the previous time step. Indicates an update to the door. This represents the weight matrix used to update the hidden state at the current time step in front of the gate. The weight matrix represents the hidden state at the current time step before resetting the gate. This indicates that the door is being reset. This represents the Sigmoid activation function.

[0049] Preferably, the combined attention network model is: ; ; ;

[0050] in, Indicates pairwise affinity between elements. Parametric functions representing linear / nonlinear projections, Represents the elements of sequence matrix A. Represents the elements of sequence matrix B. Represents the scaling constant. This represents a measure of dissimilarity between elements. Represents the scaling constant. Parametric functions representing linear / nonlinear projections, express norm, This represents the combined attention matrix.

[0051] Preferably, a dendritic network is used as the decision layer of the human behavior recognition network, and the dendritic network model is as follows: ; in, This represents the input to the dendritic network. This represents the output of the dendritic network. Represents positive integers. Indicates the total number of dendritic units. This indicates a specific module. This represents the weight matrix.

[0052] Preferably, the method for calculating the fused data is as follows: The Euler angles obtained by integrating the data collected by the gyroscope are used as the optimal estimated values, while the Euler angles calculated from the data collected by the accelerometer and magnetometer are used as the actual measured values. The fusion filtering iterative solution is performed to obtain fused data. The fusion filtering equation includes prediction and update equations: The prediction equation is: ; ; The update equation is: ; ; ; in, Indicates the predicted state value. and Let represent the optimal estimates at times k and k-1, respectively. This represents the state prediction covariance. and This represents the state estimation covariance at times k and k-1. Let A represent the Kalman state estimation covariance gain, B represent the control variable matrix, and A represent the state transition matrix. Let Q represent the state control vector, R represent the system noise covariance matrix, I represent the measurement noise covariance matrix, and I represent the identity matrix. This represents the actual measured value. This represents the measurement matrix.

[0053] Preferably, the combined attention network uses two sets of outputs from an improved bidirectional gated recurrent network as input sequences; Calculate the pairwise affinity matrix of the first set of input sequences and the dissimilarity metric matrix of the second set of input sequences; The pairwise affinity matrix is ​​activated by the tanh function; the dissimilarity metric matrix is ​​activated by the sigmoid function. The activation results are then subjected to Hadamard matrix operations to obtain the combined attention matrix.

[0054] The pairwise affinity matrix is ​​calculated as follows: ; in, Let represent the value of the element in the i-th row and j-th column of the pairwise affinity matrix. Parametric functions representing linear / nonlinear projections, This represents the elements of the first input sequence. This represents the elements of the second set of input sequences. This represents the scaling constant.

[0055] The dissimilarity metric matrix is ​​calculated as follows: ; in, This represents the elements of the first input sequence. This represents the elements of the second set of input sequences. This represents the value of the element in the i-th row and j-th column of the dissimilarity metric matrix. Represents the scaling constant. Parametric functions representing linear / nonlinear projections, express The norm of .

[0056] Therefore, the combined attention matrix M is calculated as follows: ;

[0057] Where E represents the pairwise affinity matrix and N represents the dissimilarity metric matrix.

[0058] Preferably, the system also includes an output device that can display the recognition results of human behavior for subsequent use or for user reference.

[0059] In another embodiment, this solution can also be implemented by a device, which may include a corresponding module or circuit that performs each or several steps of the various embodiments and implementations described above. Therefore, each or several steps of the various implementations described above can be performed by a corresponding module, and the device may include one or more of these modules. A module may be one or more hardware modules specifically configured to perform a corresponding step, or implemented by a processor configured to perform a corresponding step, or stored in a computer-readable medium for implementation by a processor, or implemented through some combination thereof.

[0060] This device can be implemented using a bus architecture. A bus architecture can include any number of interconnect buses and bridges, depending on the specific application of the hardware and overall design constraints. The bus connects various circuits, including one or more processors, memory, and / or hardware modules. The bus can also connect various other circuits such as peripherals, voltage regulators, power management circuitry, external antennas, etc.

[0061] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this solution includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which the embodiments of this solution pertain. The processor performs the various methods and processes described above. For example, the method embodiments of this solution can be implemented as software programs tangibly contained in a machine-readable medium, such as memory. In some embodiments, part or all of the software program can be loaded and / or installed via memory and / or a communication interface. When the software program is loaded into memory and executed by the processor, one or more steps of the methods described above can be performed. Alternatively, in other embodiments, the processor can be configured to perform one of the methods described above by any other suitable means (e.g., by means of firmware).

[0062] Those skilled in the art will understand that all or part of the processes in the above embodiments or implementations can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods.

[0063] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A lightweight human behavior recognition method based on wearable sensors, characterized in that, The method includes: S1. Collect data from multiple sensors, preprocess the data, calculate the Euler angles of the multiple sensor data as fused data, initialize the fused data as input data; the multiple sensors include a gyroscope, an accelerometer, and a magnetometer; S2. Construct a human behavior recognition network, process the input data, and output the recognition result; the human behavior recognition network includes a sequentially connected improved bidirectional gated recurrent network, a combined attention network, and a dendritic network; S3. Based on the recognition results, identify human behavior; In step S1, the preprocessing includes: The collected sensor data is filtered and fused to obtain fused data; Define the initial attitude and collect initial attitude data for a fixed period of time, and calculate the average value of the initial attitude data as the static error; In the initial posture, actual data of nodes wearing multiple sensors are collected, and the error is calculated based on the difference in the actual data of the nodes to obtain the wearing error; When initializing the fused data, static errors are removed and error compensation is performed based on wearing errors to form input data; The combined attention network model is as follows: in, Indicates pairwise affinity between elements. Parametric functions representing linear / nonlinear projections, Represents the elements of sequence matrix A. Represents the elements of sequence matrix B. Represents the scaling constant. This represents a measure of dissimilarity between elements. Represents the scaling constant. Parametric functions representing linear / nonlinear projections, express norm, Represents the combined attention matrix; Using a dendritic network as the decision layer of the human behavior recognition network, the dendritic network model is as follows: Where X represents the input of the dendritic network, Y represents the output of the dendritic network, P+ represents a positive integer, and L represents the total number of dendritic units. This indicates a specific module. Represents the weight matrix; The method for calculating fused data is as follows: The Euler angles obtained by integrating the data collected by the gyroscope are used as the optimal estimated values, while the Euler angles calculated from the data collected by the accelerometer and magnetometer are used as the actual measured values. The fusion filtering process is iteratively solved to obtain fused data. The fusion filtering includes prediction and update equations. The prediction equation is: The update equation is: in, Indicates the predicted state value. and Let represent the optimal estimates at times k and k-1, respectively. This represents the state prediction covariance. and This represents the state estimation covariance at times k and k-1. Let A represent the Kalman state estimation covariance gain, B represent the control variable matrix, and A represent the state transition matrix. Let Q represent the state control vector, R represent the system noise covariance matrix, I represent the measurement noise covariance matrix, and I represent the identity matrix. This represents the actual measured value. This represents the measurement matrix.

2. The method according to claim 1, characterized in that, The improved bidirectional gated loop network consists of a forward gated loop unit and a reverse gated loop unit, and only uses the hidden state of the previous moment to calculate the update gate and reset gate; The improved gated loop unit model is as follows: in, This indicates the hidden state of the previous time step. Indicates an update to the door. This represents the weight matrix used to update the hidden state at the current time step in front of the gate. The weight matrix represents the hidden state at the current time step before resetting the gate. This indicates that the door is being reset. This represents the Sigmoid activation function.

3. The method according to claim 1, characterized in that, The combined attention network uses two sets of outputs from an improved bidirectional gated recurrent network as input sequences. Calculate the pairwise affinity matrix of the first set of input sequences and the dissimilarity metric matrix of the second set of input sequences; The pairwise affinity matrix is ​​activated by the tanh function; the dissimilarity metric matrix is ​​activated by the sigmoid function. The activation results are then subjected to Hadamard matrix operations to obtain the combined attention matrix.

4. A lightweight human behavior recognition system based on wearable sensors, characterized in that, The system is used to perform the method as described in any one of claims 1-3, the system comprising: Multiple sensors are used to collect human behavior data; The data preprocessing module is used to preprocess human behavior data collected by multiple sensors, calculate the Euler angles of the data from multiple sensors as fused data, and initialize the fused data as input data. The data recognition module includes a constructed human behavior recognition network, which processes the input data and outputs the recognition result; the human behavior recognition network includes a sequentially connected improved bidirectional gated recurrent network, a combined attention network, and a dendritic network. The human behavior recognition module is used to recognize human behavior based on the recognition results.

5. The system according to claim 4, characterized in that, The multiple sensors include gyroscopes, accelerometers, and magnetometers.

6. A lightweight human behavior recognition device based on wearable sensors, characterized in that, The device includes multiple sensors, a processor, and a memory; The multiple sensors are worn as wearable devices on the human body; The processor calls the computer instructions stored in the memory to execute the lightweight human behavior recognition method based on wearable sensors as described in any one of claims 1-3.

Images