A Residual Modular Network Design Method for Recognizing Human Activities

Abstract

The technical solution adopted in this invention is a residual module network design method for recognizing human activities, comprising: denoising sensor data in a dataset using a low-pass filtering method; normalizing the denoised sensor data; segmenting the normalized sensor data using a sliding window; constructing a residual module network model with a local loss function; training the residual module network model to obtain a trained residual module network model; and extracting features from the trained residual module network model to obtain the recognition results of human activities. This method greatly reduces the memory requirements and memory usage when training the residual network.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and in particular to a residual module network design method for recognizing human activities. Background Technology

[0002] Human activity recognition using sensor data has become a research hotspot in recent years. Analyzing various human activities through embedded sensors can be viewed as a time-series pattern recognition problem. Sliding window techniques are used to segment the data and extract features to achieve recognition and classification results. Currently, sensor-based human activity recognition is widely used in smart homes, intelligent monitoring systems, healthcare, behavior detection, augmented manufacturing, and pedestrian navigation.

[0003] Human activity recognition typically employs traditional machine learning methods, such as multilayer perceptrons, decision trees, and support vector machines, and has made significant progress in these areas. However, traditional machine learning methods often rely on heuristic, manual feature extraction, which is not only time-consuming and labor-intensive but also limits its widespread application in most everyday activity recognition tasks. With the development of deep learning, sensor-based human activity recognition has transitioned from feature design to network design. Deep learning can significantly reduce the workload of feature design and has achieved unparalleled performance results in computer vision and natural language processing. In recent years, due to advancements in computing power, deep learning has achieved remarkable results in HAR time series, surpassing state-of-the-art shallow algorithms that require tedious manual feature extraction. Significantly increasing network depth can bring substantial performance improvements, but it inevitably requires a large amount of memory to store network activations. As networks become deeper, the ever-increasing memory burden limits the widespread application of deep learning in human activity recognition. Therefore, reducing memory consumption and effectively improving recognition performance has become an important challenge in human activity recognition. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a residual module network design method for recognizing human activities that can reduce memory usage and effectively improve recognition performance.

[0005] The technical solution adopted in this invention is a residual module network design method for identifying human activities, which includes the following steps:

[0006] S1. Use low-pass filtering to denoise the sensor data in the dataset to obtain the denoised sensor data.

[0007] S2. Normalize the noise-reduced sensor data to obtain normalized sensor data.

[0008] S3. Use a sliding window to segment the normalized sensor data to obtain the segmented sensor data;

[0009] S4. Construct a residual module network model with a local loss function; the residual module network model sequentially includes a first residual module, a local loss function, and a second residual module, wherein both the first and second residual modules sequentially include a first convolutional layer, a first batch normalization layer, and a first R... e The system consists of a ReLU activation function, a second convolutional layer, a second batch normalization layer, and a second ReLU activation function; the local loss function is obtained by weighting the similarity matching loss function and the prediction loss function.

[0010] S5. Divide the segmented sensor data obtained in step S3 into two parts. One part of the sensor data is input into the residual module network model in step S4 for training, to obtain the trained residual module network model. The specific process includes the following steps:

[0011] S5.1, Sensor data y h Enter the first residual module to process the sensor data y. h Perform mapping processing to obtain h(y) h Meanwhile, sensor data y h After being processed sequentially through the first convolutional layer, the first batch normalization layer, the first ReLU activation function, the second convolutional layer, the second batch normalization layer, and the second ReLU activation function of the first residual module, the output result a2(a1(y) is obtained. h )), h(y h ) and a2(a1(y h After weighting, the output of the first residual module is obtained: y h+1 =h(y h )+a2(a1(y h ));

[0012] S5.2, convert the output result y of the first residual module h+1 The input is fed into a local loss function for similarity matching loss processing and prediction loss processing, resulting in the output of the local loss function l(y). h+1 ): l(y h+1 )=αl s +(1-α)l p Among them, l s This indicates that similarity matching loss processing is performed, l s =||S(c(y) h+1))-S(Y)||2, where Y represents a set of labels encoded in a single pass, Y=(y1,y2,...yn), c(·) is the convolutional layer in the similarity matching loss; S(·) is the cosine similarity matrix operation; l p This indicates that loss prediction processing is being performed. in Y represents the weight matrix of the linear classifier, Y represents the label matrix of the one-hot encoded target; α represents a positive real number, α <= 1; CrossEntropy represents a cross-entropy function;

[0013] S5.3, Output the local loss function result l(y) h+1 The input is fed into the second residual module, and the output result l(y) is processed. h+1 After mapping, we get h(l(y)). h+1 Meanwhile, the output result l(y) h+1 The data sequentially passes through the first convolutional layer, the first batch normalization layer, and the first R layer of the second residual module. e LU activation function, second convolutional layer, second batch normalization layer, and second R e After processing with the LU activation function, the output result is a2(a1(l(y)). h+1 ))), h(l(y h+1 )) and a2(a1(l(y h+1 After weighted processing, the output of the second residual module is obtained: Y = h(l(y h+1 ))+a2(a1(l(y h+1 Activity identification and classification are performed using the output of the second residual module.

[0014] S5.4. Following steps S5.1 to S5.3, input sensor data multiple times and iterate multiple times to complete the training of the residual module network model.

[0015] S6. Input another part of the sensor data into the trained residual module network model obtained in step S5 for feature extraction to obtain the recognition result of human activities.

[0016] The beneficial effects of this invention are as follows: By employing the aforementioned residual module network design method for recognizing human activities, this method uses residual modules to omit the interconnections between convolutional layers, reducing network depth and avoiding the gradient vanishing problem caused by training deeper convolutions. Furthermore, because it uses a local loss function, it does not rely on globally generated losses, and gradients do not propagate backwards throughout the network. That is, hidden weights can be updated during forward propagation, unlike global backpropagation. After the weights of each residual block are updated, gradients and activations no longer need to be held in memory, significantly reducing memory requirements during residual network training. Although the entire network is trained simultaneously, the locally generated loss can be used to train the remaining blocks one at a time, further reducing memory usage.

[0017] Preferably, in step S2, the formula for normalization is: Where i = 1, 2, ..., n, This represents the normalized sensor data, where m is the number of channels. and These represent the maximum and minimum values ​​of the i-th sensor channel, respectively.

[0018] Preferably, in step S3, the sliding window has a length of L and an overlap coverage rate of 50%. Attached Figure Description

[0019] Figure 1 This is a flowchart of a residual module network design method for identifying human activities according to the present invention.

[0020] Figure 2 A bar chart showing the accuracy of activity categories using the method of this invention. Detailed Implementation

[0021] The invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can implement it based on the description. The scope of protection of the invention is not limited to these specific embodiments.

[0022] This invention relates to a residual module network design method for recognizing human activities, such as... Figure 1 As shown, the method includes the following steps:

[0023] S1. The mean filtering method with a window length of 5 is used to denoise the sensor data in the dataset to obtain the denoised sensor data.

[0024] S2. Normalize the noise-reduced sensor data to obtain normalized sensor data; this ensures that all data values ​​are within a similar range, which helps solve the model learning problem.

[0025] S3. Use a sliding window with a length of 200 and an overlap coverage of 50% to segment the normalized sensor data to obtain the segmented sensor data.

[0026] S4. Construct a residual module network model with a local loss function; the residual module network model sequentially includes a first residual module, a local loss function, and a second residual module, wherein both the first and second residual modules sequentially include a first convolutional layer, a first batch normalization layer, a first ReLU activation function, a second convolutional layer, a second batch normalization layer, and a second ReLU activation function. e LU activation function; the local loss function is obtained by weighting the similarity matching loss function and the prediction loss function;

[0027] S5. Divide the segmented sensor data obtained in step S3 into two parts. One part of the sensor data is input into the residual module network model in step S4 for training, to obtain the trained residual module network model. The specific process includes the following steps:

[0028] S5.1, Sensor data y h Enter the first residual module to process the sensor data y. h Perform mapping processing to obtain h(y) h Meanwhile, sensor data y h After being processed sequentially through the first convolutional layer, the first batch normalization layer, the first ReLU activation function, the second convolutional layer, the second batch normalization layer, and the second ReLU activation function of the first residual module, the output result a2(a1(y) is obtained. h )), h(y h ) and a2(a1(y h After weighting, the output of the first residual module is obtained: y h+1 =h(y h )+a2(a1(y h ));

[0029] S5.2, convert the output result y of the first residual module h+1 The input is fed into a local loss function for similarity matching loss processing and prediction loss processing, resulting in the output of the local loss function l(y). h+1 ): l(y h+1 )=αl s +(1-α)l p Among them, l s This indicates that similarity matching loss processing is performed, l s =||s(c(y) h+1))-S(Y)||2, where Y represents a set of labels encoded in a single pass, Y=(y1,y2,...yn), c(·) is the convolutional layer in the similarity matching loss; S(·) is the cosine similarity matrix operation; l p This indicates that loss prediction processing is being performed. in Y represents the weight matrix of the linear classifier, Y represents the label matrix of the one-hot encoded target, α represents a positive real number, α <= 1, and CrossEntropy represents a cross-entropy function.

[0030] S5.3, Output the local loss function result l(y) h+1 The input is fed into the second residual module, and the output result l(y) is processed. h+1 After mapping, we get h(l(y)). h+1 Meanwhile, the output result l(y) h+1 After passing through the first convolutional layer, first batch normalization layer, first ReLU activation function, second convolutional layer, second batch normalization layer, and second ReLU activation function of the second residual module, the output result a2(a1(l(y)) is obtained. h+1 ))), h(l(y h+1 )) and a2(a1(l(y h+1 After weighted processing, the output of the second residual module is obtained: Y = h(l(y h+1 ))+a2(a1(l(y h+1 Activity identification and classification are performed using the output of the second residual module.

[0031] S5.4. Following steps S5.1 to S5.3, input sensor data multiple times and iterate multiple times to complete the training of the residual module network model.

[0032] S6. Input another part of the sensor data into the trained residual module network model obtained in step S5 for feature extraction to obtain the recognition result of human activities.

[0033] The residual module network design method described above for recognizing human activities eliminates the need for interconnections between convolutional layers, reducing network depth and avoiding the vanishing gradient problem caused by training deeper convolutions. Furthermore, because it uses a local loss function, it does not rely on globally generated losses, and gradients are not backpropagated throughout the network. That is, hidden weights can be updated during forward propagation, unlike global backpropagation. After the weights of each residual block are updated, gradients and activations no longer need to be held in memory, significantly reducing memory requirements during residual network training. Although the entire network is trained simultaneously, the locally generated loss can be used to train the remaining blocks one at a time, further reducing memory usage.

[0034] In step S2, the formula for normalization is: Where i = 1, 2, ..., n, This represents the normalized sensor data, where m is the number of channels. and These represent the maximum and minimum values ​​of the i-th sensor channel, respectively.

[0035] Table 2 shows the confusion matrix obtained to verify the residual module network design method for recognizing human activities according to the present invention:

[0036]

[0037] In Table 2, the horizontal axis represents the activity category predicted by the machine, and the vertical axis represents the actual action category. The data in the table represents the sample value of each predicted and actual activity category. For example, when the predicted activity category is "standing" and the actual activity category is "walking," the sample value for this situation is 0, meaning this prediction will not occur. When both the predicted and actual activity categories are "sitting," the sample value for this situation is 439, meaning there are 439 samples where the prediction and the actual activity correspond. In Table 2, although there are some misclassifications for "walking in place" and "running in place" because these two actions are somewhat similar, very good accuracy is still achieved for other action recognition (e.g., ...). Figure 2 As shown in the figure, excellent recognition performance was achieved. It can be seen that the method described in this paper has high recognition performance.

Claims

1. A residual module network design method for recognizing human activities, characterized in that: The method includes the following steps: S1. Use low-pass filtering to denoise the sensor data in the dataset to obtain the denoised sensor data. S2. Normalize the noise-reduced sensor data to obtain normalized sensor data. S3. Use a sliding window to segment the normalized sensor data to obtain the segmented sensor data; S4. Construct a residual module network model with a local loss function; the residual module network model sequentially includes a first residual module, a local loss function, and a second residual module. Both the first and second residual modules sequentially include a first convolutional layer, a first batch normalization layer, a first ReLU activation function, a second convolutional layer, a second batch normalization layer, and a second ReLU activation function; the local loss function is obtained by weighting a similarity matching loss function and a prediction loss function. S5. Divide the segmented sensor data obtained in step S3 into two parts. One part of the sensor data is input into the residual module network model in step S4 for training, to obtain the trained residual module network model. The specific process includes the following steps: S5.1, Sensor data y h Enter the first residual module to process the sensor data y. h Perform mapping processing to obtain h(y) h Meanwhile, sensor data y h After being processed sequentially through the first convolutional layer, the first batch normalization layer, the first ReLU activation function, the second convolutional layer, the second batch normalization layer, and the second ReLU activation function of the first residual module, the output result a2(a1(y) is obtained. h )), h(y h ) and a2(a1(y h After weighting, the output of the first residual module is obtained: y h+1 =h(y h )+a2(a1(y h )); S5.2, convert the output result y of the first residual module h+1 The input is fed into a local loss function for similarity matching loss processing and prediction loss processing, resulting in the output of the local loss function l(y). h+1 ): l(y h+1 )=αl s +(1-α)l p Among them, l s This indicates that similarity matching loss processing is performed, l s =||S(c(y) h+1 ))-S(Y)||2, where Y represents a set of labels encoded in a single pass, Y=(y1,y2,...yn), c(·) is the convolutional layer in the similarity matching loss; S(·) is the cosine similarity matrix operation; l p This indicates that loss prediction processing is being performed. in Y represents the weight matrix of the linear classifier, Y represents the label matrix of the one-hot encoded target; α represents a positive real number, α <= 1; CrossEntropy represents a cross-entropy function; S5.3, Output the local loss function result l(y) h+1 The input is fed into the second residual module, and the output result l(y) is processed. h+1 After mapping, we get h(l(y)). h+1 Meanwhile, the output result l(y) h+1 After passing through the first convolutional layer, first batch normalization layer, first ReLU activation function, second convolutional layer, second batch normalization layer, and second ReLU activation function of the second residual module, the output result a2(a1(l(y)) is obtained. h+1 ))), h(l(y h+1 )) and a2(a1(l(y h+1 After weighted processing, the output of the second residual module is obtained: Y = h(l(y h+1 ))+a2(a1(l(y h+1 Activity identification and classification are performed using the output of the second residual module. S5.

4. Following steps S5.1 to S5.3, input sensor data multiple times and iterate multiple times to complete the training of the residual module network model. S6. Input another part of the sensor data into the trained residual module network model obtained in step S5 for feature extraction to obtain the recognition result of human activities.

2. The residual module network design method for identifying human activities according to claim 1, characterized in that: In step S2, the formula for normalization is: Where i = 1, 2, ..., n, This represents the normalized sensor data, where m is the number of channels. and These represent the maximum and minimum values ​​of the i-th sensor channel, respectively.

3. The residual module network design method for identifying human activities according to claim 2, characterized in that: In step S3, the sliding window has a length of L and an overlap coverage rate of 50%.

Images