A Behavior Recognition Method Based on Distributed Sensor Fusion Network

Abstract

This invention relates to the fields of computer vision and pattern recognition, and particularly to a behavior recognition method based on a distributed sensor fusion network. The method includes sampling acceleration sequences to ensure consistent sequence lengths; feeding acceleration sequences from different sensors into parallel feature learning modules to extract sensor embeddings; using a Transformer Encoder as a multi-sensor feature fusion module to obtain fused features; sampling angular velocity sequences to ensure the angular velocity sequence length matches the frame number; synthesizing angular velocity tensors from different sensors; and feeding the angular velocity tensors into a CNN model to extract features. This invention addresses the challenges posed by the high complexity of real-world motion scenes, which hinders robustness in behavior recognition tasks and results in low recognition accuracy.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and pattern recognition technology, and in particular to a behavior recognition method based on a distributed sensor fusion network. Background Technology

[0002] Human behavior recognition has been extensively studied in the fields of computer vision and pattern recognition and has become one of the most popular research topics.

[0003] Human behavior recognition methods can be categorized into three types based on different acquisition devices: those based on RGB sensors, those based on depth sensors, and those based on inertial sensors. However, RGB sensors are sensitive to changes in ambient light, which significantly affects the accuracy of the information recorded by the acquisition device. Processing depth image data requires substantial computational resources. While extracting 3D skeleton data from depth images for behavior recognition significantly reduces the computational load, the accuracy of joint coordinates is affected by pose estimation algorithms and viewpoints, leading to missing or incorrect calibration. Using only a single inertial sensor fixed to a local limb of a moving object cannot provide a comprehensive description of the overall motion.

[0004] Real-world motion scenarios are often highly complex, posing a challenge to the robustness of action recognition tasks. To overcome this problem, multimodal data fusion may be useful. Compared with using a single type of motion data, multimodal data fusion for action recognition can provide higher recognition accuracy and stronger robustness. Summary of the Invention

[0005] To address the shortcomings of existing algorithms, this invention proposes two branches: one based on acceleration and the other on angular velocity. In the acceleration-based branch, the acceleration sequence is first sampled to ensure consistent sequence length. Acceleration sequences from different sensors are fed into parallel feature learning modules to extract sensor embeddings. A Transformer Encoder is then used as a multi-sensor feature fusion module to obtain fused features. Features from the acceleration branch are input into a classification network to obtain classification scores. In the angular velocity-based branch, the angular velocity sequence is first sampled to ensure its length matches the frame count. Angular velocity sequences from different sensors are synthesized into an angular velocity tensor. This tensor is then fed into a CNN model to extract features. Features from the angular velocity branch are input into a classification network to obtain classification scores. Finally, the classification scores from both the acceleration and angular velocity branches are fused at the decision level for prediction.

[0006] The technical solution adopted in this invention is: a behavior recognition method based on a distributed sensor fusion network, comprising the following steps:

[0007] Step 1: Sample the acceleration sequence to ensure consistent sequence length; feed the acceleration sequences from different sensors into parallel feature learning modules to extract sensor embeddings; use TransformerEncoder as a multi-sensor feature fusion module to obtain fused features; input the features from the acceleration branch into a classification network to obtain classification scores.

[0008] Furthermore, the formula for the feature learning strategy for extracting Sensor Embedding is as follows:

[0009] SE(i)=AFLB(a(i))

[0010] Where SE is an abbreviation for Sensor Embedding, i∈[1,N] is the sensor number, and N represents the number of sensors; AFLB is an abbreviation for acceleration feature learning block, and a(i) represents the triaxial acceleration data of the i-th sensor.

[0011] Furthermore, the feature learning module uses a 2D convolutional layer for data filtering in the AFLB block; then uses a 2D convolutional layer to enhance the feature representation by injecting acceleration information from three directions; and uses a 2D convolutional layer in the SAFLB block for data filtering and feature representation enhancement.

[0012] Furthermore, using Transformer Encoder as a multi-sensor feature fusion module to obtain fused features involves projecting Sensor Embeddings into a high-dimensional space and then inputting them into three multi-head self-attention layers. In the multi-head self-attention layers, q, k, and v are each divided into eight parts. Finally, the output vector is input into a fully connected network.

[0013] Step 2: Sample the angular velocity sequence to ensure the sequence length matches the frame number; angular velocity sequences from different sensors are synthesized into an angular velocity tensor; the angular velocity tensor is fed into a CNN model to extract features; features from the angular velocity branch are input into a classification network to obtain classification scores;

[0014] Furthermore, the CNN model includes: a local motion learning block, a global motion learning block, and a feature learning block. The local motion learning block consists of two 2D convolutional layers and one ReLU layer, with the tensor dimension axis adjusted in the transposed layer. The global motion learning block consists of two 2D convolutional layers, one Dropout layer, and two max pooling layers. The feature learning block consists of two 2D convolutional layers, two ReLU layers, two MaxPool2d layers, and two dropout layers.

[0015] Furthermore, specifically including:

[0016] In the local motion learning block, two convolutional layers with filter sizes of 1×1 and 1×3 are used to learn the features of a single sensor joint in time sequence and the features of adjacent sensor joints in tensor arrangement, respectively. Then, a transpose operation is used to transform the angular velocity signal axis and the sensor joint axis. In the global motion learning block, two convolutional layers with filter size of 3×3 are used to learn the global features of all joints in the channel dimension and output feature maps. Then, the feature maps are input into two feature map learning blocks with filter size of 3×3 to learn features. Finally, the feature maps are unfolded into a vector and input into the classification network to obtain scores for different categories.

[0017] Step 3: The classification scores from the acceleration branch and the angular velocity branch are fused at the decision level for prediction.

[0018] The beneficial effects of this invention are:

[0019] 1. A new network model for behavior recognition based on distributed sensor data fusion, called DSFNet (Distributed Sensor Fusion Network), was constructed for multi-sensor data.

[0020] 2. Based on the characteristic that acceleration sequences can reflect motion trends, a learning one-to-many dependency branch is proposed.

[0021] 3. To address the characteristic that angular velocity sequences can reflect the direction and velocity of local attitude, a branch for learning local-global features is proposed.

[0022] 4. The method of the present invention was evaluated on the self-built multi-sensor dataset CZU-MHAD. The experimental results show that the method of the present invention outperforms the existing methods. Attached Figure Description

[0023] Figure 1 This is a flowchart of the behavior recognition method based on a distributed sensor fusion network of the present invention;

[0024] Figure 2 This is a branch structure diagram based on acceleration of the present invention;

[0025] Figure 3 These are diagrams illustrating the three feature learning strategies of this invention;

[0026] Figure 4 This is a diagram showing the joint positions corresponding to the sensor wearing positions of the present invention;

[0027] Figure 5 This invention relates to a CNN layer based on angular velocity branching. Detailed Implementation

[0028] The present invention will be further described below with reference to the accompanying drawings and embodiments. The drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0029] The experimental platform used in this invention is a desktop computer, with a B460M MORTAR motherboard, an Intel i7 10700 CPU with a clock speed of 2.9GHz, 16GB of memory, and Windows 10 Professional operating system. The GPU is a Tesla V100 with 32GB of video memory, and the operating system is Ubuntu. The deep learning framework used in this paper is PyTorch, and the software tools used in the experiments are PyCharm and Anaconda3.

[0030] like Figure 1 As shown, a behavior recognition method based on a distributed sensor fusion network includes the following steps:

[0031] Step 1: Sample the acceleration sequence to ensure consistent sequence length; feed the acceleration sequences from different sensors into parallel feature learning modules to extract sensor embeddings; use TransformerEncoder as a multi-sensor feature fusion module to obtain fused features; input the features from the acceleration branch into a classification network to obtain classification scores.

[0032] Unlike other datasets that provide data from a single inertial sensor, CZU-MHAD provides multi-sensor data; to ensure consistency in sequence length, 100 samples were sampled at equal intervals for each acceleration sequence. Figure 2 The left side shows a human skeleton model with inertial sensors attached to the CZU-MHAD dataset, where the red nodes indicate the positions where the inertial sensors are worn.

[0033] Action recognition based on triaxial acceleration, such as Figure 2 As shown, the acceleration-based behavior recognition branch is divided into two stages. The first stage is feature learning based on single sensor data. Acceleration data is input into the feature learning module to generate a sensor embedding representing a single sensor. In the second stage, the Transformer Encoder layer is used to model the spatial position dependencies of multiple sensors to achieve multi-sensor feature fusion representation.

[0034] Three feature learning strategies are designed to extract sensor embeddings, such as... Figure 3 As shown; the input to the feature learning module is sampled acceleration data; the first learning strategy ( Figure 3(a) directly learns the acceleration features in three directions from the raw acceleration data and then generates the Sensor Embedding; the formula is as follows:

[0035] SE(i)=AFLB(a(i))

[0036] Where SE is an abbreviation for Sensor Embedding, i∈[1,N] is the sensor number, and N represents the number of sensors; AFLB is an abbreviation for acceleration feature learning block, and a(i) represents the triaxial acceleration data of the i-th sensor.

[0037] In the second learning strategy ( Figure 3 In (b), a synthetic acceleration feature learning block is added to learn the original acceleration features and the resultant velocity features of the triaxial accelerations respectively; then, the triaxial acceleration features and the resultant velocity features are fused to generate Sensor Embedding, which is formulated as follows:

[0038]

[0039] SE(i)=AFLB(a(i))+SAFLB(A(i))

[0040] Among them, A t This represents the resultant velocity of the three-axis accelerations at time t. SAFLB is an abbreviation for the synergistic acceleration feature learning block.

[0041] In the third learning strategy ( Figure 3 In (c), the idea of ​​an attention mechanism is introduced; in the second learning strategy, acceleration and synthetic acceleration play equally important roles in the generation of Sensor Embedding; however, this may not be the case in the real world. Assigning different attention weights to the two during fusion can reduce the interference problem caused by the second strategy to some extent; the formula is:

[0042] SE(i)=Attn(AFLB(a(i)),SAFLB(A(i)))

[0043] Attn represents the attention weight fusion. After experimental verification, the first learning strategy was ultimately selected to extract Sensor Embedding.

[0044] The training parameters of the feature learning module are shown in Table 1. In the AFLB block, a 2D convolutional layer (kernel size 1x1, stride 1) is first used for data filtering, and then a 2D convolutional layer (kernel size 1x3, stride 1, number of channels 128) is used to enhance the feature representation by injecting acceleration information from three directions. In the SAFLB block, a 2D convolutional layer (kernel size 1x1, stride 1, number of channels 128) is used for data filtering and to enhance the feature representation.

[0045] Table 1 Training parameters of the feature learning module

[0046]

[0047] First, sensor embeddings are extracted from acceleration data, and then they are input into the Transformer Encoder module to learn the spatial location dependencies of multiple sensors and achieve multi-sensor feature fusion. In this model, the sensor embeddings are first projected into a high-dimensional space of 256, and then input into three multi-head self-attention layers. In the multi-head self-attention layers, q, k, and v are each divided into 8 parts. Finally, the output vector is input into a fully connected network for classification.

[0048] Table 2 Parameters of Acceleration Branch Classification Network

[0049]

[0050] Step 2: Sample the angular velocity sequence to ensure the sequence length matches the frame number; angular velocity sequences from different sensors are synthesized into an angular velocity tensor; the angular velocity tensor is fed into a CNN model to extract features; features from the angular velocity branch are input into a classification network to obtain classification scores;

[0051] Angular velocity is the ratio of the angle Δθ rotated by a particle around an axis in time Δt to the time Δt. Angular velocity describes how fast an object moves around the center of a circle. Unlike acceleration, angular velocity is more inclined to express local motion characteristics. Acceleration can reflect the direction of motion and spatial displacement, and can reflect the change of an object's current position in space relative to its original position. However, angular velocity can only express the direction and speed of local attitude changes. Although both acceleration and angular velocity are measured by wearable sensors attached to a part of the human body, angular velocity is weaker in expressing global information than acceleration. Therefore, this invention proposes a local-global two-stage feature learning method.

[0052] Angular velocity data is a time series; therefore, to obtain a tensor with the same scale as the skeleton data from the angular velocity data, the angular velocities are first sampled to obtain an angular velocity sequence with the same length as the skeleton sequence; then, a tensor with a scale of T×N×C is extracted from the angular velocity sequence, where T is the sequence length, N is the number of skeleton joints, and C is the dimension of the angular velocity signal; since three-axis angular velocities are used, the dimension of the angular velocity tensor is 3; as shown... Figure 4 In the extracted tensors, the positions of the wearable sensors correspond one-to-one with the joints; in reality, the number of wearable sensors is less than the number of joints; therefore, in the angular velocity tensor, the values ​​for those without sensors are all set to 0.

[0053] The CNN layer based on angular velocity branching is as follows: Figure 5 As shown; the local motion learning blocks consist of two 2D convolutional layers and one ReLU layer, with parameters of (1,1,64) and (3,1,32) for the convolutional layers, respectively; in the transposed layer, the tensor dimension axis changes from (0,1,2) to (0,2,1); the global motion learning blocks consist of two 2D convolutional layers, one Dropout layer, and two max-pooling layers, with parameters of (3,3,32) and (3,3,64) for the convolutional layers, a dropout rate of 0.5, and a pooling layer window size of 2; the feature map learning blocks consist of two 2D convolutional layers, two ReLU layers, two MaxPool2d layers, and two dropout layers, with parameters of (3,3,128) and (3,3,256) for the two convolutional layers, a pooling layer window size of 2, and a dropout rate of 0.5.

[0054] In the local motion learning block, two convolutional layers with filter sizes of 1×1 and 1×3 are used to learn the features of a single sensor joint in time sequence and the features of adjacent sensor joints in tensor arrangement, respectively. Then, a transpose operation is used to transform the angular velocity signal axis and the sensor joint axis. In the global motion learning block, two convolutional layers with filter size of 3×3 are used to learn the global features of all joints in the channel dimension and output feature maps. Then, the feature maps are input into two feature map learning blocks with filter size of 3×3 to learn features. Finally, the feature maps are unfolded into a vector and input into the classification network to obtain scores for different categories.

[0055] Table 3 Angular velocity branch classification network parameters

[0056]

[0057]

[0058] Step 3: The classification scores from the acceleration branch and the angular velocity branch are fused at the decision level for prediction;

[0059] Decision-level fusion occurs during the prediction process, where decisions from different models are combined to make a final decision. In machine learning, decision-level fusion methods primarily include probability-based methods, opinion pool-based methods, and voting-based methods. In deep learning, decision-level fusion makes a final decision by combining scores from different branches, such as maximum value fusion and average value fusion.

[0060] Experimental data

[0061] This experiment was conducted on the self-built dataset CZU-MHAD, which contains 880 motion samples. Each sample has a corresponding depth map sequence, skeleton sequence, three-axis acceleration sequence, and three-axis angular velocity sequence. The depth map sequence was acquired by a depth sensor, with each depth map having a resolution of 424×512. The three-axis acceleration and three-axis angular velocity sequences were acquired by a wearable inertial sensor. The skeleton data was extracted from the depth sequences, with each skeleton having 25 joints, each described by four real numbers: screen coordinates, depth value, and confidence score. The 880 motion samples in the dataset can be divided into 22 different motion categories, including right high wave (A01), left high wave (A02), right horizontal wave (A03), left horizontal wave (A04), hammer (A05), grab (A06), and draw X (A07). (A07) Draw X with left hand (A08) Draw circle with right hand (A09) Draw circle with left hand (A10) Right foot kick forward (A11) Left foot kick forward (A12) Right foot kick side (A13) Left foot kichside (A14) Clap (A15) Bend down (A16) Wave up and down (A17) Sur place (A18) Left body turning movement (A19) Right body turning movement (A20) Left lateral movement (A21) Right lateral movement (A22)

[0062] Experimental setup

[0063] To compare with existing methods, the samples in the dataset were divided into two experimental settings, referring to the literature.

[0064] In Experimental Setup 1, there are four experimental strategies for randomized validation. In the first experimental strategy, 3 / 8 of the samples are used as the training set, and the rest are used as the test set. In the second experimental strategy, 4 / 8 of the samples are used as the training set, and the rest are used as the test set. In the third experimental strategy, 5 / 8 of the samples are used as the training set, and the rest are used as the test set. In the fourth experimental strategy, 6 / 8 of the samples are used as the training set, and the rest are used as the test set. T1 to T4 are used to represent the above four experimental strategies.

[0065] In Experimental Setup 2, there are three experimental strategies for cross-validation. In the first experimental strategy, the actions of subjects 1 and 2 are used as the training set, and the rest are used as the test set. In the second experimental strategy, the actions of subjects 1, 2, and 5 are used as the training set, and the rest are used as the test set. In the third experimental strategy, the actions of subjects 1, 2, 3, and 5 are used as the training set, and the rest are used as the test set. T5 to T7 are used to represent the above three experimental strategies.

[0066] ablation experiment

[0067] Experimental results for different learning strategies:

[0068] The performance of the Sensor Embedding extraction module was evaluated in Experiment 1 and Experiment 2. Three feature learning strategies were attempted when extracting Sensor Embeddings in the acceleration-based human behavior recognition model. Therefore, the effectiveness of the three feature learning strategies was verified during the model performance evaluation process, and the evaluation results are shown in Tables 2 and 3. In the first learning strategy, we only used triaxial acceleration; in the second learning strategy, we fused the resultant velocity of acceleration based on the first learning strategy; in the third learning strategy, we added attention weights to the feature fusion process of the second learning strategy.

[0069] The evaluation results for Experiment 1 are shown in Table 2. We found that the first learning strategy outperformed the other two learning strategies in T1, T2, and T4. In the second learning strategy, the resultant velocity of acceleration was fused; however, the model's performance decreased significantly in T1, T2, and T4. In the third learning strategy, attention weights were added during fusion; compared with the second learning strategy, the model's performance was significantly improved in T1, T3, and T4.

[0070] Table 4 shows the results of the acceleration-based human behavior recognition model in Experimental Setup 1.

[0071]

[0072] The evaluation results for Experiment 2 are shown in Table 3. The performance of the first learning strategy remains superior; in the second learning strategy, the model performance still shows varying degrees of decline; in the third learning strategy, the model achieves the best performance on T6. With the increase of training samples, the performance on T7 is on par with the second learning strategy.

[0073] Table 5 shows the results of the acceleration-based human behavior recognition model in Experimental Setup 2.

[0074]

[0075] The experimental analysis above reveals that the resultant velocity of acceleration is a scalar value, lacking directional information, thus introducing noise into the original acceleration after fusion. A comparison of the experimental results of the second and third learning strategies shows that adding attention weights during fusion effectively balances the noise. Therefore, in experimental strategies T1, T3, T4, and T6, the third learning strategy demonstrates superior performance. Especially in experimental strategies T3 and T6, it even surpasses the first learning strategy.

[0076] Multimodal fusion recognition results

[0077] The performance of the fused modal branches was evaluated in Experimental Setup 1 and Experimental Setup 2. The evaluation results of the fused model are shown in Tables 4 and 5. The evaluation results in Experimental Setup 1 are shown in Table 4. We found that the angular velocity-based branch showed better performance than the acceleration-based branch in T1, T2, T3, and T4. Before fusion, the best results for each branch in Experimental Setup 1 were 95.27%, 94.55%, 97.88%, and 99.09%, respectively. The fused model improved by 3.46%, 2.50%, 1.21%, and 0.91% compared to the unfused model.

[0078] Table 6 Results of multimodal fusion in Experimental Setup 1

[0079]

[0080] The evaluation results on Experimental Setup 2 are shown in Table 5. It was found that the acceleration-based branch outperformed the angular velocity-based branch at times T5 and T7. Before fusion, the best results for each branch on Experimental Setup 2 were 90.91%, 91.19%, and 95.45%, respectively. The fused model showed improvements of 0.19%, 3.41%, and 1.71% compared to the unfused model, respectively.

[0081] Table 7 Results of multimodal fusion in Experimental Setup 2

[0082]

[0083] The experimental analysis above reveals that the angular velocity-based branch outperforms the acceleration-based branch in tests with randomly partitioned datasets, making it more suitable for recognizing identical actions performed by the same subject. Conversely, the acceleration-based branch outperforms the angular velocity-based branch in subject-based cross-validation tests, making it more suitable for recognizing identical actions performed by different subjects. Finally, decision-level fusion of the two modalities significantly improves the model's performance.

[0084] Results compared with existing methods

[0085] The model of this invention was evaluated using existing action recognition methods across all experimental settings, where its accuracy significantly outperformed other methods. The results of existing methods on Experimental Setting 1 are shown in Table 6; the DMM-LBP method achieved the best result on T1, with an accuracy of 97.09%. The DMM-HOG method achieved the best results on T2 and T3, reaching 97.27% and 97.58%, respectively. On T4, the DMM-GLAC method achieved the highest accuracy, reaching 98.67%. The model of this invention represents an improvement of 1.64%, 1.14%, 1.51%, and 1.33% compared to the current best results, respectively.

[0086] Table 8 compares experimental setup 1 with other methods.

[0087]

[0088] The results of existing methods on experimental setup 2 are shown in Table 7. The DMM-HOG method achieved the best results on T5, with an accuracy of 88.26%. The DMM-GLAC method achieved the best results on T6 and T7, respectively, with accuracy of 91.25% and 92.50%. The model of this invention improves upon the current best results by 2.84%, 3.35%, and 4.66%, respectively.

[0089] Table 9 compares experimental setup 2 with other methods.

[0090]

[0091] This invention proposes a novel distributed sensor fusion network that designs two parallel branches to process acceleration and angular velocity data respectively, based on their different characteristics. In the acceleration-based branch, the network benefits from modeling single-sensor to multi-sensor dependencies at the spatial scale. In the angular velocity-based branch, an end-to-end network is designed to learn local and global relationships at the temporal scale. Experimental results on the CZU-MHAD dataset demonstrate that the different learning branches designed in this invention exhibit competitive performance. Furthermore, the fusion of the two modalities significantly improves model performance compared to using data from only one sensor modality.

[0092] Based on the above-described preferred embodiments of the present invention, and through the foregoing description, those skilled in the art can make various changes and modifications without departing from the inventive concept. The technical scope of this invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. A behavior recognition method based on a distributed sensor fusion network, characterized in that, Includes the following steps: Step 1: Sample the acceleration sequences; feed multiple acceleration sequences into parallel feature learning modules to extract sensor embeddings. The AFLB block of the feature learning module uses 1×1 2D convolutional layers, ReLU function, 1x3 2D convolutional layers, Flatten operation, RelU function, Dropout and Linear layer; The SAFLB block of the feature learning module uses 1×1 2D convolutional layers, Flatten operation, RelU function, Dropout and Linear layer; Attention weights are introduced to fuse SAFLB and AFLB blocks to generate Sensor Embeddings, as shown in the formula: , For attention weight fusion, express The resultant velocity of the three-axis accelerations at a given moment. Indicates the first Triaxial acceleration data from one sensor; ; The sensor embedding is input into the Transformer Encoder module, which projects the sensor embedding into a high-dimensional space; it is then input into three multi-head self-attention layers; in each multi-head self-attention layer, q, k, and v are each divided into eight parts; the output vector is then input into a fully connected network. Step 2: Sampling of angular velocity sequences; multiple angular velocity sequences are synthesized into an angular velocity tensor; the angular velocity tensor is fed into a CNN model to extract features; the features of the angular velocity branch are input into the second classification network; The CNN model includes: a local motion learning block, a global motion learning block, and a feature learning block. The local motion learning block consists of two 2D convolutional layers and one ReLU layer, with parameters (1,1,64) and (3,1,32) for the convolutional layers, respectively. In the transpose layer, the tensor dimensions are adjusted from (0,1,2) to (0,2,1). The global motion learning block consists of two 2D convolutional layers, one Dropout layer, and two max-pooling layers, with parameters (3,3,32) and (3,3,64) for the convolutional layers, respectively. The feature learning block consists of two 2D convolutional layers, two ReLU layers, two MaxPool2d layers, and two dropout layers, with parameters (3,3,128) and (3,3,256) for the two convolutional layers, respectively. Specifically, the CNN model includes: in the local motion learning block, two filters with sizes of... and The convolutional layers learn the features of a single sensor joint in time sequence and the features of adjacent sensor joints in tensor arrangement, respectively; then, a transpose operation is used to transform the angular velocity signal axis and the sensor joint axis; in the global motion learning block, two filters of size are used. The convolutional layer learns global features of all joints in the channel dimension and outputs a feature map; the feature map is input to two filters of size . Features are learned in the feature map learning blocks; finally, the feature map is unfolded into a vector and input into the classification network to obtain scores for different categories; Step 3: The classification scores from the acceleration branch and the angular velocity branch are fed into the decision-level fusion for prediction.

2. The behavior recognition method based on a distributed sensor fusion network according to claim 1, characterized in that, The formula for the feature learning strategy for extracting Sensor Embedding is: in, For the sensor number, Indicates the number of sensors; For acceleration feature learning blocks, Indicates the first Triaxial acceleration data from one sensor.

Images