Human activity recognition method based on multi-scale group information distillation and refinement network

Abstract

The application discloses a human activity recognition method based on multi-scale group information distillation and refinement network, relates to the technical field of artificial intelligence and edge computing, and comprises the following steps: constructing an input tensor based on measurement data of an inertial measurement unit; inputting a trained activity type recognition model; and calculating, which comprises the following steps: processing the input tensor through a multi-scale feature extraction module to obtain multi-scale fusion features; processing the input features through a group information distillation module in a group information distillation stage to obtain residual enhancement features; processing the residual enhancement features through a feature aggregation module to obtain aggregated features; processing the aggregated features through a feature integration module to obtain a global feature vector; and processing the global feature vector through a classification module to obtain an activity type classification result. The method provided by the application can significantly improve the recognition accuracy under the premise that the parameter quantity and the calculation cost are controlled, can be directly deployed on a smart phone, and can provide an efficient and reliable solution for real-time medical monitoring and motion analysis.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and edge computing technology, and in particular to a method for human activity recognition based on multi-scale group information distillation and refinement networks. Background Technology

[0002] With the widespread adoption of smartphones and wearable devices, devices equipped with inertial measurement units (IMUs) have made continuous human activity monitoring increasingly convenient, providing a technological foundation for numerous applications such as medical monitoring, motion analysis, elderly care, and human-computer interaction.

[0003] Currently, some related technologies can automatically extract discriminative features from raw data measured by inertial measurement units using deep learning methods, thereby eliminating the need for manual feature engineering and achieving accurate human activity recognition. However, most neural network models typically require significant computational resources, making it difficult to apply these models to edge devices with limited hardware, such as mobile phones. Furthermore, this approach faces three ongoing challenges: On the one hand, traditional Convolutional Neural Networks (CNNs) are limited by fixed receptive fields and lack sufficient temporal modeling capabilities, making it difficult to effectively represent multi-scale motion patterns in human activities, such as the differences between rapid gestures and slow posture transitions. On the other hand, Transformer models can solve the problems of CNNs through global attention mechanisms, but the computational complexity of Transformer models increases quadratically with sequence length, resulting in excessive computational overhead in real-time applications and making them difficult to deploy on edge devices. Furthermore, related technologies can also set up lightweight architectures designed for edge devices, but such architectures rely on post-training compression techniques such as pruning, quantization, and knowledge distillation. Compared to full-scale models, the application of these compression techniques leads to a decrease in model recognition accuracy, making it difficult to distinguish subtle differences in human activity types. Moreover, most existing lightweight architectures lack explicit progressive feature refinement and cross-scale information fusion mechanisms, which are crucial for maintaining recognition performance under parameter-constrained conditions.

[0004] Therefore, there is an urgent need for an accurate and efficient method for recognizing human activity. Summary of the Invention

[0005] This application provides a human activity recognition method based on multi-scale group information distillation and refinement networks to overcome the shortcomings of the aforementioned related technologies. The technical solution is as follows: In a first aspect, this application provides a method for human activity recognition based on a multi-scale group information distillation and refinement network, applied to a user terminal, the method comprising: Acquire measurement data collected by the inertial measurement unit on the user terminal and construct the input tensor; The input tensor is input into the trained activity type recognition model; the activity type recognition model includes a multi-scale feature extraction module, a group information distillation module, a feature aggregation module, a feature integration module, and a classification module. The computation steps of the activity type recognition model include: The input tensor is processed by multiple convolutional paths of different scales in the multi-scale feature extraction module to obtain multi-scale fused features; The multi-scale fused features are input into the group information distillation module, and the input features are processed through multiple cascaded group information distillation stages to obtain the residual enhancement features of each group information distillation stage. The feature aggregation module performs a weighted summation of the residual enhancement features from each group's information distillation stage, and outputs the aggregated features. The global feature vector is obtained by the feature integration module based on the aggregated feature processing. The activity type classification result is obtained by processing the global feature vector through the classification module.

[0006] In one alternative embodiment of the first aspect, the step of acquiring measurement data collected by the inertial measurement unit mounted on the user terminal and constructing the input tensor includes: The system collects measurement data from the inertial measurement unit, including data from the triaxial accelerometer and the triaxial gyroscope. The input tensor is constructed based on the sampled data obtained by sampling the data from the three-axis accelerometer and the three-axis gyroscope through a preset sliding window.

[0007] In one alternative embodiment of the first aspect, the step of processing the input tensor through multiple convolutional paths of different scales in the multi-scale feature extraction module to obtain multi-scale fused features includes: The input tensor is convolved in parallel by multiple convolutional paths of different scales in the multi-scale feature extraction module. Each convolutional path has a convolutional kernel of a different size. Global hierarchical features are extracted through the largest convolutional kernel, while local hierarchical features are extracted through the other convolutional kernels. Based on each local and global hierarchical feature, features are concatenated along the channel dimension and then fused through convolution to obtain multi-scale fused features.

[0008] In one alternative embodiment of the first aspect, the step of processing the input features through multiple cascaded group information distillation stages to obtain residual enhancement features for each group information distillation stage includes: In each group information distillation stage, the input features are segmented according to the channel dimension, the first segmentation feature is input into the information path, and the second segmentation feature is input into the distillation path; In the information path, multiple convolutional kernels of different sizes are used to perform parallel convolution based on the first segmentation feature. The convolutional features output by each convolutional kernel are concatenated in the channel dimension to obtain the information path concatenation feature. The information path splicing features are processed by an enhanced attention mechanism to obtain attention features; In the distillation path, the second segmentation feature is preserved through identity mapping; The second segmentation feature and the attention feature are concatenated and then fused and convolved to obtain the residual enhancement feature of the current group information distillation stage. The input features for the first group of information distillation stage are the multi-scale fusion features, and the input features for the other groups of information distillation stages are the residual enhancement features output from the adjacent previous group of information distillation stage.

[0009] In one alternative to the first aspect, the process of processing the information path splicing features through an enhanced attention mechanism to obtain attention features includes: The enhanced attention mechanism includes channel attention branch and spatial attention branch; In the channel attention branch, the channel attention weights are obtained by sequentially processing global average pooling, 1×1 convolution, SiLU activation function, 1×1 convolution, and Sigmoid activation function. In the spatial attention branch, spatial attention weights are obtained by sequentially processing the data through 1×1 convolution and the Sigmoid activation function. The information path splicing features are weighted based on the channel attention weight and the spatial attention weight to obtain attention features.

[0010] In one alternative embodiment of the first aspect, the residual enhancement features of each group of information distillation stage are weighted and summed by the feature aggregation module, and the output aggregated features include: The residual enhancement features of each group of information distillation stage are averaged by the feature aggregation module, and then the channel dimension is unified by 1×1 convolution to obtain the aligned features. Each aligned feature is projected onto a preset embedding dimension using a 1×1 convolution to obtain the projected features. Based on the weight parameters learned during training and the Softmax function, normalization is performed to obtain the aggregate weight of each projected feature. The projected features are weighted and summed based on the aggregation weights to output aggregated features.

[0011] In one alternative embodiment of the first aspect, the training process of the activity type recognition model includes the following steps: During each type of activity carried by the user, the measurement data of the inertial measurement unit on the user terminal is sampled, and a sample input tensor is constructed based on the sample data of each sampling point to determine the type of activity performed by the user at the corresponding sampling point. A sample set is constructed by using the sample input tensor as the sample input and the corresponding activity type as the sample label; The activity type identification model is trained based on the sample set, and a loss function is constructed based on the difference between the predicted activity type obtained by the activity type identification model based on the sample input and the sample label. Backpropagation is performed based on the loss function to update the model parameters of the activity type identification model; If the activity type recognition model is determined to be converged according to the loss function, the trained activity type recognition model is output.

[0012] Secondly, this application also provides a human activity recognition device based on a multi-scale group information distillation and refinement network, comprising: The data acquisition unit is used to acquire measurement data collected by the inertial measurement unit mounted on the user terminal and construct the input tensor. The data processing unit is used to input the input tensor into the trained activity type recognition model. The activity type recognition model includes a multi-scale feature extraction module, a group information distillation module, a feature aggregation module, a feature integration module, and a classification module. The computation steps of the activity type recognition model include: The input tensor is processed by multiple convolutional paths of different scales in the multi-scale feature extraction module to obtain multi-scale fused features; The multi-scale fused features are input into the group information distillation module, and the input features are processed through multiple cascaded group information distillation stages to obtain the residual enhancement features of each group information distillation stage. The feature aggregation module performs a weighted summation of the residual enhancement features from each group's information distillation stage, and outputs the aggregated features. The global feature vector is obtained by the feature integration module based on the aggregated feature processing. The activity type classification result is obtained by processing the global feature vector through the classification module.

[0013] Thirdly, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method provided by the first aspect of this application or any implementation thereof.

[0014] Fourthly, this application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method provided by the first aspect of this application or any implementation thereof.

[0015] The beneficial effects of the technical solution provided in this application include at least the following: This application utilizes a multi-scale feature extraction module to process inertial measurement data in parallel through multiple convolutional paths of different scales, simultaneously capturing multi-scale motion patterns such as rapid gestures and slow posture transitions, overcoming the temporal modeling limitations caused by a fixed receptive field. Multi-scale features are input into a cascaded group information distillation module, progressively refining features and reducing redundancy. A lightweight channel-space attention mechanism is combined to achieve selective feature emphasis, avoiding accuracy loss caused by compression techniques such as pruning and quantization. A feature aggregation module performs learnable weighted summation on the outputs of each distillation stage, achieving adaptive cross-scale information fusion and enhancing feature robustness. Finally, the activity recognition result is output through a feature integration and classification module. The method provided in this application significantly improves recognition accuracy under controlled parameter count and computational overhead, and can be directly deployed on resource-constrained platforms such as smartphones, providing an efficient and reliable solution for real-time medical monitoring and motion analysis. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating a human activity recognition method based on a multi-scale group information distillation and refinement network provided in an embodiment of this application. Figure 2 This is a schematic diagram of the structure of a human activity recognition device based on a multi-scale group information distillation and refinement network provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or modules is not limited to the steps or modules listed, but may optionally include steps or modules not listed, or may optionally include other steps or modules inherent to such process, method, product, or apparatus.

[0020] It should be noted that the terms "first" and "second" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects. It is understood that "first" and "second" can be interchanged in a specific order or sequence where permitted. It should be understood that the objects distinguished by "first" and "second" can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in an order other than those described or illustrated herein.

[0021] The present application will now be described in detail with reference to specific embodiments.

[0022] Next, combine Figure 1 This paper introduces a human activity recognition method based on a multi-scale group information distillation and refinement network, provided by embodiments of this application. For details, please refer to... Figure 1 , Figure 1 This diagram illustrates a flowchart of a human activity recognition method based on a multi-scale group information distillation and refinement network, as provided in an embodiment of this application. Figure 1 As shown, the method includes the following steps: S101: Obtain measurement data collected by the inertial measurement unit mounted on the user terminal and construct the input tensor.

[0023] Specifically, the user terminal may include various portable electronic devices equipped with inertial measurement units, such as mobile terminals, watches, and bracelets, but this application embodiment does not limit this.

[0024] Specifically, the measurement data collected by the inertial measurement unit on the user terminal is the six-axis inertial measurement unit (IMU) data, which includes three-axis accelerometer signals (ax, ay, az) and three-axis gyroscope signals (gx, gy, gz).

[0025] Specifically, the acquired continuous sensor signals can be processed by sliding window segmentation to form an input tensor X∈R^(B×6×128), where B is the batch size, 6 is the number of sensor channels (triaxial acceleration + triaxial angular velocity), and 128 is the number of sampling points in the time window.

[0026] S102, input the input tensor into the trained activity type recognition model; wherein, the activity type recognition model includes a multi-scale feature extraction module, a group information distillation module, a feature aggregation module, a feature integration module, and a classification module, and the calculation steps of the activity type recognition model include: S1021, the input tensor is processed by multiple convolutional paths of different scales in the multi-scale feature extraction module to obtain multi-scale fused features, specifically including: S10211 performs parallel convolution on the input tensor through multiple convolutional paths of different scales in the multi-scale feature extraction module.

[0027] S10212, each convolutional path has a convolutional kernel of different size. Global hierarchical features are extracted through the largest convolutional kernel, and local hierarchical features are extracted through the other convolutional kernels. For example, three parallel one-dimensional convolutional paths can be set up with kernel sizes of 3, 5 and 7 respectively. With such a setup, multiple convolutional paths of different scales can extract multi-granular temporal features.

[0028] It should be noted that, through this design, smaller convolutional kernels (e.g., kernels of size 3) can capture micro-movements, such as rapid changes like joint rotation and muscle deformation, when extracting local hierarchical features based on the input tensor; moderately sized convolutional kernels (e.g., kernels of size 5) can capture meso-scale movements, such as medium-time changes like gesture transitions and posture adjustments, when extracting local hierarchical features based on the input tensor; and larger convolutional kernels (e.g., kernels of size 7) can capture meso-scale movements and model macro-activity, such as long-time scale changes like walking cycles and arm swings, when extracting global hierarchical features based on the input tensor.

[0029] With this setup, the embodiments of this application ensure the independence and parallelism of feature extraction at each scale, enabling the acquisition of rich multi-scale feature representations in a single forward propagation.

[0030] S10213 performs feature concatenation on the channel dimension based on each local and global hierarchical feature, and then performs fusion convolution to obtain multi-scale fused features.

[0031] For example, the convolution kernel that can be merged can be set to 1×1.

[0032] S1022, the multi-scale fusion features are input into the group information distillation module, and the input features are processed through multiple cascaded group information distillation stages to obtain the residual enhancement features of each group information distillation stage.

[0033] The input features for the first group of information distillation stage are the multi-scale fusion features, and the input features for the other groups of information distillation stages are the residual enhancement features output from the adjacent previous group of information distillation stage.

[0034] Specifically, each group information distillation stage includes the following steps: S201, the input features are segmented according to the channel dimension, the first segmentation feature is input into the information path, and the second segmentation feature is input into the distillation path.

[0035] For example, 30% of the channels can be segmented as the first segmentation feature, which is input as the Info Path; 70% of the channels can be segmented as the second segmentation feature, which is input as the Distillation Path.

[0036] S202, in the information path, multiple convolution kernels of different sizes are used to perform parallel convolution based on the first segmentation feature, and the convolution features output by each convolution kernel are concatenated in the channel dimension to obtain the information path concatenation feature.

[0037] For example, parallel convolution can be performed using three convolution kernels: 3×3, 5×5, and 7×7.

[0038] S203, the information path splicing features are processed through an enhanced attention mechanism to obtain attention features.

[0039] Specifically, the enhanced SE Attention mechanism includes channel attention branch and spatial attention branch; In the channel attention branch, the channel attention weights are obtained by sequentially processing global average pooling, 1×1 convolution, SiLU activation function, 1×1 convolution, and Sigmoid activation function. In the spatial attention branch, spatial attention weights are obtained by sequentially processing the data through 1×1 convolution and the Sigmoid activation function. The information path splicing features are weighted based on the channel attention weight and the spatial attention weight to obtain attention features.

[0040] S204, In the distillation path, the second segmentation feature is preserved by identity mapping; S205, the second segmentation feature and the attention feature are concatenated and fused convolutional to obtain the residual enhancement feature of the current group information distillation stage.

[0041] Specifically, the features of the distillation path and the information path can be concatenated and then processed by 1×1 fusion convolution, and residual connections can be achieved through learnable residual weights γ to obtain residual enhanced features.

[0042] S1023, the feature aggregation module performs weighted summation on the residual enhancement features of each group of information distillation stage, and outputs the aggregated features.

[0043] Specifically, the residual enhancement features of each group of information distillation stage can be averaged by the feature aggregation module, and then the channel dimension can be unified by 1×1 convolution to obtain the aligned features. Each aligned feature is projected onto a preset embedding dimension using a 1×1 convolution to obtain the projected features. Based on the weight parameters learned during training and the Softmax function, normalization is performed to obtain the aggregate weight of each projected feature. The projected features are weighted and summed based on the aggregation weights to output the aggregated features, denoted as H_agg.

[0044] S1024, the global feature vector is obtained by the feature integration module based on the aggregated feature processing.

[0045] Specifically, the aggregated feature H_agg enters the global feature integration module, and is compressed into a global feature vector, denoted as f_g, by sequentially passing through a one-dimensional convolution (Conv1D) with a kernel size of 3, batch normalization (BatchNorm), SiLU activation function, and global average pooling (GAP).

[0046] S1025, the activity type classification result is obtained by processing the global feature vector through the classification module.

[0047] Specifically, the final classification can be performed using a lightweight dual Dropout classifier. The processing sequence can be: first Dropout layer → FC Layer 1 (first fully connected layer) → SiLU activation function → second Dropout → FC Layer 2 (second fully connected layer) → activity type classification result.

[0048] In the classification stage, unlike the single fully connected layer commonly used in traditional deep learning frameworks, this application embodiment uses a lightweight two-layer classifier to replace the heavy fully connected layer and is equipped with double Dropout regularization, with the dropout rate of the first Dropout layer being greater than that of the second Dropout layer.

[0049] For example, the feature vector after global average pooling is first passed through a first Dropout layer with a dropout rate of 0.2 to improve generalization performance, then fed into a first fully connected layer and activated by the SiLU activation function, followed by a second Dropout layer with a dropout rate of 0.1, and finally the activity type classification result is output by the second fully connected layer. This dual Dropout design provides stronger regularization while maintaining the model's expressive power.

[0050] It should be noted that, in the embodiments of this application, the first Dropout layer (dropout rate 0.2) effectively prevents over-reliance on specific feature dimensions in the early stages of feature extraction, forcing the network to learn more robust and distributed feature representations; the second Dropout layer (dropout rate 0.1) performs mild regularization before classification decision, avoiding excessive discarding of refined feature information, and achieving a balance between preventing overfitting and maintaining classification accuracy; compared with a single Dropout layer, the dual-layer decreasing design adopted in the embodiments of this application provides finer regularization control.

[0051] In some embodiments, the activity type recognition model can be trained based on the following steps, the training process including the following steps: S301, during each type of activity carried by the user terminal, the measurement data of the inertial measurement unit on the user terminal is sampled, a sample input tensor is constructed based on the sample data of each sampling point, and the type of activity performed by the user at the corresponding sampling point is determined.

[0052] S302, using the sample input tensor as the sample input and the corresponding activity type as the sample label, a sample set is constructed.

[0053] In some embodiments, the data source for the sample set can be the UCI-HAR dataset, which can be collected by 30 volunteers aged 19-48 years using a smartphone of brand A placed on the waist, with a sampling frequency of 50Hz, and includes 6 daily activities (walking, going upstairs, going downstairs, sitting, standing, and lying down).

[0054] In some embodiments, the data source for the sample set can be the MotionSense dataset: collected from 24 participants using a smartphone of brand B, at a sampling frequency of 50Hz, including 6 activities (walking, jogging, going upstairs, going downstairs, sitting, and standing), totaling 15,630 samples.

[0055] Meanwhile, a weighted random sampling strategy is adopted to deal with the potential class imbalance problem, and the processed data is divided into training set, validation set and test set.

[0056] S303, the activity type identification model is trained based on the sample set, and a loss function is constructed based on the difference between the predicted activity type obtained by the activity type identification model based on the sample input and the sample label.

[0057] S304, Backpropagation is performed based on the loss function to update the model parameters of the activity type identification model.

[0058] S305, if the activity type recognition model is determined to be converged according to the loss function, the trained activity type recognition model is output.

[0059] In some specific implementations, experiments can be conducted on a high-performance computing server equipped with an Intel Xeon Silver 4410Y CPU, 128GB of RAM, and an NVIDIA RTX 4090 GPU. The implementation uses the PyTorch deep learning framework, with a batch size of 64, training for 100 epochs, employing the Adam optimizer, and an initial learning rate of 0.001.

[0060] The experimental results are shown in Table 1. The activity type recognition model provided in this application achieved the highest accuracy (ACC), F1 score, and recall on all three datasets. On the UCI-HAR dataset, the F1 score reached 96.87%; on the MotionSense dataset, the F1 score reached 98.82%; and on the most challenging SHL dataset, the F1 score reached 93.76%, surpassing the second-place DBC-CNN method by 3.14 percentage points. The average F1 score across the three datasets was 96.48%.

[0061] Regarding model complexity, the results are shown in Table 2. The activity type recognition model provided in this application has only 79K parameters (Params(K)), 0.158M computational cost (FLOPs), and a model size of 0.30MB of storage space, making it one of the models with the lowest computational cost among all compared methods. Compared to the MHCNLS method with a similar number of parameters (approximately 78K parameters), the average F1 score is improved by 7.30 percentage points. Compared to the DBC-CNN method with a larger number of parameters (1033K parameters, which is 1205% of that of this invention), the method provided in this application still leads in recognition performance, while its computational cost is only 1 / 1600 of that.

[0062] Table 1. Performance comparison of the method provided in this application with other methods on three benchmark datasets.

[0063] Table 2. Comprehensive Comparison of Human Activity Recognition Methods

[0064] In some embodiments, to verify the deployment feasibility of the method provided in this application on actual edge devices, inference latency was evaluated on a mobile device of brand C (hardware parameters: 16GB RAM, quad-core Cortex-A76 @ 2.4GHz), and the results are shown in Table 3. All models were converted to ONNX format and executed with four threads via ONNX Runtime. After warming up each model for 20 iterations, the average latency of 200 inferences was recorded.

[0065] Experimental results show that the activity type recognition model provided in this application achieves the optimal accuracy-efficiency tradeoff with an average inference latency of 0.381ms (approximately 2626 FPS): it achieves the highest accuracy (91.8%) among all methods, while the model size is only 342KB. Although MHCNLS has a slightly faster inference speed (0.332ms), its accuracy is only 69.6%, which is 22.2 percentage points lower and cannot meet the needs of practical applications. The heavier baseline methods DBC (3.154ms) and X-LiteHAR (7.525ms) are 8.3 times and 19.8 times slower, respectively, and fail to provide any accuracy improvement.

[0066] Table 3 Device Deployment Performance of Mobile Devices in Brand C

[0067] The following are apparatus embodiments of this application, which can be used to execute the method embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the method embodiments of this application.

[0068] Please see below. Figure 2 The image below is a schematic diagram of a human activity recognition device based on a multi-scale group information distillation and refinement network, provided as an exemplary embodiment of this application. The device includes: The data acquisition unit is used to acquire measurement data collected by the inertial measurement unit mounted on the user terminal and construct the input tensor. The data processing unit is used to input the input tensor into the trained activity type recognition model. The activity type recognition model includes a multi-scale feature extraction module, a group information distillation module, a feature aggregation module, a feature integration module, and a classification module. The computation steps of the activity type recognition model include: The input tensor is processed by multiple convolutional paths of different scales in the multi-scale feature extraction module to obtain multi-scale fused features; The multi-scale fused features are input into the group information distillation module, and the input features are processed through multiple cascaded group information distillation stages to obtain the residual enhancement features of each group information distillation stage. The feature aggregation module performs a weighted summation of the residual enhancement features from each group's information distillation stage, and outputs the aggregated features. The global feature vector is obtained by the feature integration module based on the aggregated feature processing. The activity type classification result is obtained by processing the global feature vector through the classification module.

[0069] It should be noted that the apparatus provided in the above embodiments, when executing a human activity recognition method based on multi-scale group information distillation and refinement networks, is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and their implementation process is detailed in the method embodiments, which will not be repeated here.

[0070] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any of the methods described above.

[0071] Please see Figure 3 This is a structural block diagram of an electronic device provided in an embodiment of this application.

[0072] like Figure 3 As shown, the electronic device includes a processor and a memory.

[0073] In this embodiment, the processor is the control center of the computer system, and can be a processor of a physical machine or a processor of a virtual machine. The processor may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor can be implemented using at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array).

[0074] A processor can also include a main processor and a coprocessor. The main processor is used to process data in the wake-up state and is also called the CPU (Central Processing Unit). The coprocessor is a low-power processor used to process data in the standby state.

[0075] The memory may include one or more computer-readable storage media, which may be non-transitory. The memory may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments of this application, the non-transitory computer-readable storage media in the memory are used to store at least one instruction, which is executed by a processor to implement the methods in the embodiments of this application.

[0076] In some embodiments, the electronic device further includes a peripheral device interface and at least one peripheral device. The processor, memory, and peripheral device interface are connected via a bus or signal line. Each peripheral device is connected to the peripheral device interface via a bus, signal line, or circuit board. Specifically, the peripheral device includes: a display screen, a camera, and audio circuitry. The peripheral device interface can be used to connect at least one I / O (Input / Output) related peripheral device to the processor and memory.

[0077] In some embodiments of this application, the processor, memory, and peripheral device interfaces are integrated on the same chip or circuit board; in other embodiments of this application, any one or two of the processor, memory, and peripheral device interfaces can be implemented on separate chips or circuit boards. This application does not specifically limit the implementation in this regard.

[0078] The electronic device structural block diagrams shown in the embodiments of this application do not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0079] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the methods in any of the foregoing embodiments. The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0080] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of software products. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0081] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for human activity recognition based on multi-scale group information distillation and refinement networks, characterized in that, When applied to a user terminal, the method includes: Acquire measurement data collected by the inertial measurement unit on the user terminal and construct the input tensor; The input tensor is input into the trained activity type recognition model; the activity type recognition model includes a multi-scale feature extraction module, a group information distillation module, a feature aggregation module, a feature integration module, and a classification module. The computation steps of the activity type recognition model include: The input tensor is processed by multiple convolutional paths of different scales in the multi-scale feature extraction module to obtain multi-scale fused features; The multi-scale fused features are input into the group information distillation module, and the input features are processed through multiple cascaded group information distillation stages to obtain the residual enhancement features of each group information distillation stage. The feature aggregation module performs a weighted summation of the residual enhancement features from each group's information distillation stage, and outputs the aggregated features. The global feature vector is obtained by the feature integration module based on the aggregated feature processing. The activity type classification result is obtained by processing the global feature vector through the classification module.

2. The method for human activity recognition based on multi-scale group information distillation and refinement network according to claim 1, characterized in that, The step of acquiring measurement data collected by the inertial measurement unit mounted on the user terminal and constructing an input tensor includes: The system collects measurement data from the inertial measurement unit, including data from the triaxial accelerometer and the triaxial gyroscope. The input tensor is constructed based on the sampled data obtained by sampling the data from the three-axis accelerometer and the three-axis gyroscope through a preset sliding window.

3. The human activity recognition method based on a multi-scale group information distillation and refinement network according to claim 1, characterized in that, The process of processing the input tensor through multiple convolutional paths of different scales in the multi-scale feature extraction module to obtain multi-scale fused features includes: The input tensor is convolved in parallel by multiple convolutional paths of different scales in the multi-scale feature extraction module. Each convolutional path has a convolutional kernel of a different size. Global hierarchical features are extracted through the largest convolutional kernel, while local hierarchical features are extracted through the other convolutional kernels. Based on each local and global hierarchical feature, features are concatenated along the channel dimension and then fused through convolution to obtain multi-scale fused features.

4. The method for human activity recognition based on a multi-scale group information distillation and refinement network according to claim 1, characterized in that, The input features are processed through multiple cascaded group information distillation stages to obtain residual enhancement features for each group information distillation stage, including: In each group information distillation stage, the input features are segmented according to the channel dimension, the first segmentation feature is input into the information path, and the second segmentation feature is input into the distillation path; In the information path, multiple convolutional kernels of different sizes are used to perform parallel convolution based on the first segmentation feature. The convolutional features output by each convolutional kernel are concatenated in the channel dimension to obtain the information path concatenation feature. The information path splicing features are processed by an enhanced attention mechanism to obtain attention features; In the distillation path, the second segmentation feature is preserved through identity mapping; The second segmentation feature and the attention feature are concatenated and then fused and convolved to obtain the residual enhancement feature of the current group information distillation stage. The input features for the first group of information distillation stage are the multi-scale fusion features, and the input features for the other groups of information distillation stages are the residual enhancement features output from the adjacent previous group of information distillation stage.

5. The human activity recognition method based on a multi-scale group information distillation and refinement network according to claim 4, characterized in that, The process of processing the information path concatenation features through an enhanced attention mechanism to obtain attention features includes: The enhanced attention mechanism includes channel attention branch and spatial attention branch; In the channel attention branch, the channel attention weights are obtained by sequentially processing global average pooling, 1×1 convolution, SiLU activation function, 1×1 convolution, and Sigmoid activation function. In the spatial attention branch, spatial attention weights are obtained by sequentially processing the data through 1×1 convolution and the Sigmoid activation function. The information path splicing features are weighted based on the channel attention weight and the spatial attention weight to obtain attention features.

6. The human activity recognition method based on a multi-scale group information distillation and refinement network according to claim 4, characterized in that, The residual enhancement features of each group of information distillation stage are weighted and summed through the feature aggregation module, and the output aggregated features include: The residual enhancement features of each group of information distillation stage are averaged by the feature aggregation module, and then the channel dimension is unified by 1×1 convolution to obtain the aligned features. Each aligned feature is projected onto a preset embedding dimension using a 1×1 convolution to obtain the projected features. Based on the weight parameters learned during training and the Softmax function, normalization is performed to obtain the aggregate weight of each projected feature. The projected features are weighted and summed based on the aggregation weights to output aggregated features.

7. A method for human activity recognition based on a multi-scale group information distillation and refinement network according to any one of claims 1-6, characterized in that, The training process of the activity type recognition model includes the following steps: During each type of activity carried by the user, the measurement data of the inertial measurement unit on the user terminal is sampled, and a sample input tensor is constructed based on the sample data of each sampling point to determine the type of activity performed by the user at the corresponding sampling point. A sample set is constructed by using the sample input tensor as the sample input and the corresponding activity type as the sample label; The activity type identification model is trained based on the sample set, and a loss function is constructed based on the difference between the predicted activity type obtained by the activity type identification model based on the sample input and the sample label. Backpropagation is performed based on the loss function to update the model parameters of the activity type identification model; If the activity type recognition model is determined to be converged according to the loss function, the trained activity type recognition model is output.

8. A human activity recognition device based on a multi-scale group information distillation and refinement network, characterized in that, include: The data acquisition unit is used to acquire measurement data collected by the inertial measurement unit mounted on the user terminal and construct the input tensor. The data processing unit is used to input the input tensor into the trained activity type recognition model. The activity type recognition model includes a multi-scale feature extraction module, a group information distillation module, a feature aggregation module, a feature integration module, and a classification module. The computation steps of the activity type recognition model include: The input tensor is processed by multiple convolutional paths of different scales in the multi-scale feature extraction module to obtain multi-scale fused features; The multi-scale fused features are input into the group information distillation module, and the input features are processed through multiple cascaded group information distillation stages to obtain the residual enhancement features of each group information distillation stage. The feature aggregation module performs a weighted summation of the residual enhancement features from each group's information distillation stage, and outputs the aggregated features. The global feature vector is obtained by the feature integration module based on the aggregated feature processing. The activity type classification result is obtained by processing the global feature vector through the classification module.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images