A human behavior detection method based on LHNFNet network model

Abstract

This invention proposes a method for detecting human behavior based on the LHNFNet network model. The optimized LHNFNet network model is used to detect human behavior images. In this optimized model, a C2f-Star module with star operations is designed in the backbone layer. This module combines input features from different subspaces through element-wise multiplication, creating numerous interaction terms that significantly increase the feature space dimensionality. This allows computation within a compact, low-dimensional feature space while benefiting from implicit high-dimensional features, thus enhancing feature extraction capabilities. A lightweight detection head with a parameter-sharing convolutional layer structure is also designed, using continuous channels from the front or back ends for computation, reducing the number of parameters and memory accesses generated during computation. Furthermore, a channel pruning algorithm is applied to constrain the hierarchical structure of the LHNFNet network model, identifying and automatically pruning unimportant channels, compressing the model size, and improving detection performance, enabling the model to meet deployment requirements on resource-constrained edge devices.

Description

Technical Field

[0001] This invention relates to the field of image intelligent recognition and human behavior target detection technology, specifically to a human behavior detection method based on the LHNFNet network model. Background Technology

[0002] With the increasing demand for intelligent monitoring and security, personnel behavior recognition technology has been widely applied in public safety, intelligent transportation, and security monitoring. Accurate identification of personnel behavior not only helps in real-time monitoring of potential security risks but also improves the speed of emergency response, thereby effectively safeguarding public safety. Therefore, how to efficiently and accurately identify personnel behavior has become one of the current research hotspots.

[0003] Traditional methods for human behavior recognition often rely on rule-based analysis and manual feature extraction from video surveillance. These methods are easily limited in complex scenarios and have poor adaptability to changes in human movement and environmental interference, resulting in low accuracy. Furthermore, their real-time performance is poor, failing to meet the real-time detection requirements of modern intelligent monitoring systems. Therefore, there is an urgent need for an effective technological means to improve the accuracy and real-time performance of human behavior recognition to address ever-changing practical needs.

[0004] In recent years, the rapid development of deep learning technology has provided new solutions for personnel behavior recognition. Through a multi-layered neural network framework, deep learning can automatically extract complex features and patterns, significantly outperforming traditional algorithms in image and video understanding. Especially in areas such as object detection and image segmentation, deep learning has been widely applied and has achieved remarkable results. This makes deep learning-based personnel behavior recognition methods an effective means to improve the performance of surveillance systems.

[0005] Currently, deep learning-based methods for human behavior recognition are mainly divided into two categories: one is a two-stage detection algorithm, such as the R-CNN series, and the other is a one-stage detection algorithm represented by the YOLO network model. The YOLO (You Only Look Once) network model treats object detection as a regression problem, enabling the prediction of multiple objects simultaneously in a single network, which greatly improves detection speed and accuracy, making it particularly suitable for real-time behavior analysis applications.

[0006] However, several challenges remain in the practical process of human behavior recognition. Existing models identify human behavior through feature extraction, target localization, and target prediction. By replacing convolutions and changing the structure, the computational load of the models has been reduced, achieving basic target detection at the edge. However, existing models have weak high-dimensional nonlinear feature extraction capabilities and high computational load, making them difficult to deploy on edge devices with limited performance resources. Furthermore, the accuracy of commonly used lightweight models that can be deployed on edge devices still has room for improvement, and they still suffer from high false negative and false positive rates.

[0007] Therefore, in situations of imbalanced samples and diverse behavioral behaviors, developing an efficient and accurate human behavior recognition model has become a key focus of further research. Enhancing the adaptability and accuracy of human behavior recognition in diverse and complex scenarios has significant theoretical and practical value, and will have a profound impact on promoting the development of intelligent monitoring and security technologies. Summary of the Invention

[0008] To address the shortcomings of the existing technologies, this invention provides a person behavior detection method based on the LHNFNet network model. This method achieves lightweight computation for person behavior detection and recognition while reducing the false positive and false negative rates, thereby improving the accuracy of person behavior detection. It also solves the problems of insufficient accuracy, high false negative and false negative rates in lightweight deployments on edge devices.

[0009] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0010] A method for detecting human behavior based on the LHNFNet network model includes the following steps:

[0011] Obtain a dataset of human behavior image samples containing both abnormal and normal behavior image samples;

[0012] Based on the YOLO network model architecture, the backbone network layer is optimized by using the C2f-Star module that integrates star operation operations, and a lightweight detection head is used to construct the LHNFNet network model.

[0013] The LHNFNet network model was trained using the aforementioned human behavior image sample dataset, and the hierarchical structure of the trained LHNFNet network model was constrained by the channel pruning algorithm to obtain the trained LHNFNet network model.

[0014] The trained LHNFNet network model is used to detect the human behavior images to be detected, and the detection results of the human behavior images to be detected are obtained.

[0015] Specifically, the personnel behavior detection model includes a backbone network layer, a neck network layer, and a detection head;

[0016] The backbone network layer comprises a convolutional layer, three convolutional fusion units, one convolutional fusion star operation unit, and one spatial pyramid pooling module connected in sequence; wherein, each convolutional fusion unit comprises a cascaded convolutional layer and a C2f module; the convolutional fusion star operation unit comprises a cascaded convolutional layer and a C2f-Star module;

[0017] The neck network layer includes two feature fusion upsampling units and two convolutional connection feature fusion units connected in sequence. The feature fusion upsampling unit includes an upsampling module, a connection layer, and a C2f module connected in sequence; the convolutional connection feature fusion unit includes a convolutional layer, a connection layer, and a C2f module connected in sequence.

[0018] The detection head obtains the detection results of the human behavior image based on the output of the neck network layer, and uses it as the output of the LHNFNet network model;

[0019] In this model, the input of the LHNFNet network model serves as the input of the backbone network layer. Within the backbone network layer, the outputs of the second and third convolutional fusion units also serve as the inputs to the connection layers of the second and first feature fusion upsampling units in the neck network layer, respectively. In addition to serving as the input of the spatial pyramid pooling module in the backbone network layer to the neck network layer, the output of the spatial pyramid pooling module also serves as the input to the connection layer of the second convolutional connection feature fusion unit in the neck network layer. Furthermore, within the neck network layer, the output of the first feature reconstruction upsampling unit also serves as the input to the connection layer of the first convolutional connection feature fusion unit. The outputs of the second feature fusion upsampling unit and the two convolutional connection feature fusion units also serve as the inputs to the detection head.

[0020] Specifically, the C2f-Star module includes a cascaded first convolutional layer, a split module, several star operation bottleneck modules, a connection layer, and a second convolutional layer. The two split outputs of the split module serve as the inputs of the first fast convolutional module and the connection layer, respectively. The outputs of each star operation bottleneck module also serve as the inputs of the connection layer. The output of the second convolutional layer serves as the output of the C2f-Star module.

[0021] Specifically, the processing procedure of the star operation bottleneck module includes:

[0022] First, the input feature map is subjected to a first depthwise convolution. Then, two extension branches are used to perform channel expansion processing on the result of the first depthwise convolution. One extension branch is expanded by 4 times through a fully connected layer before ReLU activation function operation. The other extension branch is first batch normalized and then expanded by 4 times through a fully connected layer. The channel expansion results of the two extension branches are then subjected to star operation operation. The star operation operation result is then compressed back to 1 channel through a fully connected layer. Batch normalization and a second depthwise convolution are then performed in sequence. Finally, the output of the second depthwise convolution is superimposed on the input feature map and used as the output of the bottleneck module of star operation operation.

[0023] Specifically, the star operation is represented as follows:

[0024]

[0025] in, This diagram shows the channel expansion results for the two extended branches. This represents the channel expansion result diagram. The image of any i-th channel, This represents the channel expansion result diagram. The image of any j-th channel, i,j = 1, 2, ..., 4d; d represents the number of channels in the input feature map of the bottleneck module of the star operation, and 4d is the number of channels after quadrupling the channels using the extended branches; W1 and W2 represent the channel expansion results of the star operation for the two extended branches, respectively. The weight matrix; This represents the weight value of the i-th channel in the weight matrix W1. This represents the weight value of the j-th channel in the weight matrix W2; * is the star operator; T is the transpose symbol; X * This indicates the output of the star operation.

[0026] Specifically, the processing procedure of the detection head includes:

[0027] First, the input feature map of the detection head is processed by partial convolution PConv and point-directed convolution PWConv operations in sequence. Then, the processing results are fed into the convolutional layers of the localization regression detection channel and the classification detection channel respectively, and the localization regression detection result and the classification detection result are output respectively.

[0028] Specifically, the LHNFNet network model is trained in the following manner:

[0029] S101: Obtain the personnel behavior image sample dataset, label the abnormal behavior image samples and normal behavior image samples in the personnel behavior image sample dataset, and divide them into training set, validation set and test set according to a preset ratio;

[0030] S102: Input the training set into the LHNFNet network model for training. Optimize the parameters of the LHNFNet network model with the goal of minimizing the loss function until the LHNFNet network model converges, thus obtaining the pre-trained LHNFNet network model.

[0031] S103: The channel pruning algorithm is used to perform channel pruning on the pre-trained LHNFNet network model to obtain the trained and pruned LHNFNet network model.

[0032] S104: Test the trained and pruned LHNFNet network model using a test set to confirm the recognition performance of the LHNFNet network model; if the recognition performance meets the requirements, end the training of the LHNFNet network model to obtain the trained LHNFNet network model; otherwise, input the training set into the trained and pruned LHNFNet network model for retraining until the LHNFNet network model converges to obtain the trained LHNFNet network model.

[0033] Specifically, the process of performing channel pruning on the LHNFNet network model using the channel pruning algorithm includes:

[0034] First, the number of channels in each batch normalized layer of the trained LHNFNet network model is counted. The output of each channel in each batch normalized layer is multiplied by a sparsity coefficient γ and then L1 regularized to obtain the performance contribution value for that channel. Channels in the batch normalized layer whose performance contribution value is less than a preset contribution threshold are designated as pruned channels and are masked, retaining the remaining channels of the batch normalized layer. This completes the channel pruning adjustment for each batch normalized layer. Then, corresponding channel pruning adjustments are performed on the convolutional and linear layers in the upstream and downstream operations of each normalized layer to ensure that the number of channels in the upstream and downstream operations is consistent. This completes the channel pruning processing of the LHNFNet network model, and the parameters of the LHNFNet network model after channel pruning are saved.

[0035] Specifically, in the loss function, the localization regression loss is the total loss function formed by the sum of the CIoU loss function and the DFL loss function, and the classification loss is the variable focal length (VFL) loss function.

[0036] Compared with the prior art, the present invention has the following technical effects:

[0037] 1. The personnel behavior detection method based on the LHNFNet network model proposed in this invention can reduce the false negative rate and false positive rate while achieving lightweight computation for personnel behavior detection and recognition, and improve the accuracy of personnel behavior detection. It effectively solves the problems of insufficient accuracy, high false negative rate and false positive rate of personnel behavior detection when implementing lightweight deployment on edge devices.

[0038] 2. In the LHNFNet network model used in this invention, a C2f-Star module that integrates star operation is designed in the backbone network layer. By combining input features from different subspaces through element-wise multiplication, a large number of interaction terms are created, which significantly improves the feature space dimension. This enables computation in a compact low-dimensional feature space and benefits from implicit high-dimensional features, thereby enhancing feature extraction capabilities.

[0039] 3. In the LHNFNet network model used in this invention, a lightweight detection head with a parameter-sharing convolutional layer structure is also designed. It takes continuous channels from the front or back end for calculation, which reduces the number of parameters and memory accesses generated by the detection head during calculation.

[0040] 4. This invention also applies a channel pruning algorithm to constrain the hierarchical structure of the LHNFNet network model, enforce channel-level sparsity, identify and automatically prune unimportant channels, compress the model volume and improve detection performance, so that the model meets the deployment requirements on edge devices with limited performance resources. Attached Figure Description

[0041] To make the objectives, technical solutions, and advantages of the invention clearer, the invention will now be described in further detail with reference to the accompanying drawings, wherein:

[0042] Figure 1 This is a flowchart of the personnel behavior detection method based on the LHNFNet network model of the present invention;

[0043] Figure 2 This is a structural diagram of the LHNFNet network model constructed in the method of this invention;

[0044] Figure 3 This is a structural diagram of the C2f-star module in the LHNFNet network model;

[0045] Figure 4 This is a diagram of the parameter-sharing convolutional layer structure of the detection head in the LHNFNet network model.

[0046] Figure 5 A schematic diagram showing the comparison of the model before and after channel pruning;

[0047] Figure 6 This is a sample diagram of human behavior in the embodiment;

[0048] Figure 7 This is a diagram showing the actual detection results of the model in the embodiment. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but only to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0050] The present invention will now be described in further detail with reference to the accompanying drawings.

[0051] This invention discloses a method for detecting human behavior based on the LHNFNet network model, such as... Figure 1 As shown, it includes the following steps:

[0052] Obtain a dataset of human behavior image samples containing both abnormal and normal behavior image samples;

[0053] Based on the YOLO network model architecture, the backbone network layer is optimized by using the C2f-Star module that integrates star operation operations, and a lightweight detection head is used to construct the LHNFNet network model.

[0054] The LHNFNet network model was trained using the aforementioned human behavior image sample dataset, and the hierarchical structure of the trained LHNFNet network model was constrained by the channel pruning algorithm to obtain the trained LHNFNet network model.

[0055] The trained LHNFNet network model is used to detect the human behavior images to be detected, and the detection results of the human behavior images to be detected are obtained.

[0056] This invention presents a method for detecting human behavior based on the LHNFNet network model. It utilizes an optimized LHNFNet network model to detect human behavior images. After extracting features from the human behavior images through the backbone network layer that integrates star operations, the neck network layer is used to retain detailed information from the extracted features. Various detailed information is then fused, and finally, the detection results of the human behavior images are obtained through a lightweight detection head.

[0057] The method for detecting human behavior according to the present invention and the LHNFNet network model used therein will be described in more detail below.

[0058] 1. LHNFNet network model

[0059] The LHNFNet network model used in the method of this invention is obtained by optimizing and improving the YOLO network model architecture, such as... Figure 2 As shown, its model structure includes a backbone network layer, a neck network layer, and a detection head.

[0060] The backbone network layer comprises sequentially connected convolutional layers, three convolutional fusion units, one convolutional fusion star operation unit, and one spatial pyramid pooling module. Each convolutional fusion unit includes cascaded convolutional layers and a C2f module; the convolutional fusion star operation unit includes cascaded convolutional layers and a C2f-Star module. Introducing the C2f-Star module for fusion star operations into the backbone network layer helps improve the feature extraction capability of the backbone network.

[0061] The neck network layer comprises two feature fusion upsampling units and two convolutional connected feature fusion units connected in sequence. Each feature fusion upsampling unit includes an upsampling module, a connection layer, and a C2f module connected in sequence; the convolutional connected feature fusion unit includes a convolutional layer, a connection layer, and a C2f module connected in sequence. The neck network layer helps retain detailed information in the extracted features and performs feature fusion on various details.

[0062] The detection head obtains the detection results of the person's behavior image based on the output of the neck network layer, and uses this as the output of the LHNFNet network model. In this invention, a lightweight detection head is constructed by replacing convolutions and changing the convolutional layer structure, thereby reducing the computational load of the detection head and helping to achieve a lightweight model.

[0063] Among them, such as Figure 2 As shown, the input of the LHNFNet network model serves as the input to the backbone network layer. Within the backbone network layer, the outputs of the second and third convolutional fusion units also serve as the inputs to the connection layers of the second and first feature fusion upsampling units in the neck network layer, respectively. In addition to serving as the input to the neck network layer, the output of the spatial pyramid pooling module in the backbone network layer also serves as the input to the connection layer of the second convolutional connection feature fusion unit in the neck network layer. Furthermore, the output of the first feature reconstruction upsampling unit in the neck network layer also serves as the input to the connection layer of the first convolutional connection feature fusion unit. The outputs of the second feature fusion upsampling unit and the two convolutional connection feature fusion units also serve as the inputs to the detection head.

[0064] In the LHNFNet network model constructed in this invention, a C2f-Star module is created by fusing star operations to enhance the feature extraction capability of the backbone network layer. A lightweight detection head is constructed by replacing convolutions and changing the convolutional layer structure, reducing the computational load of the detection head. At the same time, a channel pruning algorithm is applied to constrain the hierarchical structure of the LHNFNet network model, forcing channel-level sparsity, identifying and automatically pruning unimportant channels, further making the LHNFNet network model architecture lightweight. Thus, while achieving computational lightweighting for human behavior detection and recognition, the accuracy of human behavior detection is improved, enabling the model to better meet the deployment requirements of edge devices with limited performance resources.

[0065] Next, we will introduce in detail the main optimizations and improvements in the LHNFNet network model.

[0066] 1.1 Design of C2f-Star Feature Extraction Module

[0067] To enhance the feature extraction capability of the backbone network layers, this invention incorporates the concept of star operations to design the C2f-Star module. For example... Figure 3 As shown, the C2f-Star module includes a cascaded first convolutional layer, a split module, several star operation bottleneck modules, a connection layer, and a second convolutional layer. The two split outputs of the split module serve as the inputs to the first fast convolutional module and the connection layer, respectively. The outputs of each star operation bottleneck module also serve as the inputs to the connection layer. The output of the second convolutional layer serves as the output of the C2f-Star module.

[0068] The backbone network layer designed in this way, based on the original backbone network structure of the YOLO network model, changes the conventional bottleneck module Bottleneck to a star operation bottleneck module Bottleneck_Star that integrates star operation operations. For example... Figure 3As shown, the processing procedure of the star operation bottleneck module is as follows: First, the input feature map undergoes a first depthwise convolution. Then, two extension branches are used to perform channel expansion processing on the result of the first depthwise convolution. One extension branch performs ReLU activation function operation after a 4x channel fully connected expansion, while the other extension branch first performs batch normalization and then a 4x channel fully connected expansion. After the channel expansion results of the two extension branches are subjected to star operation, the star operation result is fully connected and compressed back to 1x channel. Then, batch normalization and a second depthwise convolution are performed sequentially. Finally, the output of the second depthwise convolution is superimposed on the input feature map and used as the output of the star operation bottleneck module. The star operation combines input features from different subspaces through element-wise multiplication, creating a large number of interaction terms. By expanding the channels and then compressing and backtracking, the feature space dimension is significantly increased, effectively amplifying the feature dimension and outlining a more accurate decision boundary, thereby enhancing the feature extraction capability.

[0069] The star operation in the bottleneck module of star operation can be represented as:

[0070]

[0071] in, This diagram shows the channel expansion results for the two extended branches. This represents the channel expansion result diagram. The image of any i-th channel, This represents the channel expansion result diagram. The image of any j-th channel, i,j = 1, 2, ..., 4d; d represents the number of channels in the input feature map of the bottleneck module of the star operation, and 4d is the number of channels after quadrupling the channels using the extended branches; W1 and W2 represent the channel expansion results of the star operation for the two extended branches, respectively. The weight matrix; This represents the weight value of the i-th channel in the weight matrix W1. This represents the weight value of the j-th channel in the weight matrix W2; * is the star operator; T is the transpose symbol; X * This indicates the output of the star operation.

[0072] Each term in the star operation exhibits a non-linear correlation, indicating that they are independent implicit dimensions. Therefore, by using the computationally efficient star operation to perform channel expansion and compression backtracking computation in the original d-channel dimension space of the input feature map, the implicit high-dimensional features can be represented in the space, significantly amplifying the feature dimension without generating additional computational overhead. This enhances the feature extraction capability without increasing the computational load.

[0073] 1.2 Parameter Sharing Convolutional Layer Structure Design for the Detection Head

[0074] To achieve model lightweighting, this invention redesigns the detection head of the LHNFNet network model, proposing a Parameter Sharing Detection Head (PSDH) with an optimized convolutional layer structure. For example... Figure 2 As shown, the processing procedure of the detection head in the LHNFNet network model is as follows: First, the input feature map of the detection head is processed by partial convolution PConv and point-directed convolution PWConv operations in sequence. Then, the processing results are respectively connected to the convolutional layers of the localization regression detection channel and the classification detection channel for processing, and the localization regression detection result and the classification detection result are output respectively.

[0075] Figure 4 A schematic diagram illustrating the structural changes of the detection head before and after the improvement is shown. Figure 4 (1) The original detection head structure of the YOLO network model is shown. The original detection head uses two separate channels, one for localization / regression detection and the other for classification detection, to complete the output. This invention changes the original detection head structure by integrating the front-end convolutions of the two output layers of the detection head together, as shown below. Figure 4 (2) shows; secondly, the ordinary convolutional layer at the front end is replaced with partial convolution PConv and point-to-point convolution PWConv, which have lower computational and memory access requirements, for computation, as shown in Figure 2. Figure 4 (3) As shown; Finally, the results processed by partial convolution PConv and point-directed convolution PWConv are then layered and connected to the back-end convolution of the localization regression detection channel and the classification detection channel to complete the output.

[0076] like Figure 4 As shown in (4), unlike traditional convolutions that process all input features, PConv utilizes the similarity of feature maps to perform convolutions only on a subset of input feature channels, selecting continuous channels from the beginning or end for computation, while leaving the rest unprocessed. Without sacrificing generality, it assumes that the input and output features have the same number of channels, and selects continuous c... p Each channel is computed. To fully utilize the information from all channels, a point-wise convolution PWConv is appended after PConv. The combined PWConv has an effective receptive field on the input similar to a T-Conv. This combination can better focus on the weights in the central region, and its decomposed structure can better utilize the redundancy of the filter, requiring fewer parameters and memory accesses than T-Conv.

[0077] Compared to the original detection head structure of the YOLO network model, the redesigned parameter-shared convolutional layer structure of the detection head integrates two output convolutional layers into two cascaded convolutional layers in a single layer at the front end, and uses PConv and PWConv, which have lower computational cost, to effectively reduce the computational cost of the detection head without affecting the detection performance, enabling the model to be effectively applied in resource-constrained edge devices.

[0078] 1.3 Model Training and Channel Pruning Algorithm Design

[0079] While optimizing the model structure, this invention also applies a channel pruning algorithm during model training to constrain the hierarchical structure of the LHNFNet network model, enforce channel-level sparsity, identify and automatically prune unimportant channels, and further make the LHNFNet network model architecture lightweight.

[0080] The training process for the LHNFNet network model is as follows:

[0081] S101: Obtain a dataset of human behavior image samples. Label both abnormal and normal behavior image samples in the dataset and divide them into training, validation, and test sets according to a preset ratio. To avoid data overlap, the dataset is usually divided into training, validation, and test sets in a 7:1:2 ratio.

[0082] S102: Input the training set into the LHNFNet network model for training. Optimize the parameters of the LHNFNet network model with the goal of minimizing the loss function until the LHNFNet network model converges, resulting in a pre-trained LHNFNet network model.

[0083] S103: The channel pruning algorithm is used to perform channel pruning on the pre-trained LHNFNet network model to obtain the trained and pruned LHNFNet network model.

[0084] S104: Test the trained and pruned LHNFNet network model using a test set to confirm the recognition performance of the LHNFNet network model; if the recognition performance meets the requirements, end the training of the LHNFNet network model to obtain the trained LHNFNet network model; otherwise, input the training set into the trained and pruned LHNFNet network model for retraining until the LHNFNet network model converges to obtain the trained LHNFNet network model.

[0085] The channel pruning algorithm is introduced during the model training process. The channel pruning algorithm takes the preliminary pre-trained model as input and performs sparsity regularization on each batch normalized layer (BN layer) in the model. During the training process, it automatically identifies and prunes unimportant channels in the network, thereby producing a lightweight model with considerable accuracy and a more compact design. The process of channel pruning for the LHNFNet network model using the channel pruning algorithm is as follows: First, count the number of channels in each batch normalized layer of the trained LHNFNet network model. Then, multiply the output of each channel in each batch normalized layer by the sparsity coefficient γ and perform L1 regularization to obtain the performance contribution value of the corresponding channel. Channels in the batch normalized layer whose performance contribution value is less than a preset contribution threshold are designated as pruning channels and are masked, while the remaining channels of the batch normalized layer are retained. This completes the channel pruning adjustment for each batch normalized layer. Next, perform corresponding channel pruning adjustments for the convolutional and linear layers in the upstream and downstream operations of each normalized layer to ensure that the number of channels in the upstream and downstream operations is consistent. This completes the channel pruning process for the LHNFNet network model, and the parameters of the LHNFNet network model after channel pruning are saved.

[0086] In the channel pruning algorithm, the performance contribution value of each channel in each batch normalization layer is calculated. By adding a sparsity coefficient γ parameter to the channel output of the BN layer and then performing L1 regularization, the magnitude of the channel's contribution to model performance is evaluated, achieving sparse training of the model channels. Then, channels with small performance contribution values ​​(i.e., those with relatively small contributions to model performance) are selected and pruned, retaining only the channels with larger performance contributions to ensure the pruned model can run normally and maintain its recognition performance. Finally, the number of channels operated on downstream is adjusted to complete the channel pruning process. Figure 6 As shown, channels with smaller performance contributions will be identified and pruned, resulting in a more compact model on the right. The model's performance may temporarily decrease during channel pruning, but this can be compensated for by retraining and fine-tuning. This retraining process typically involves adjusting parameters such as the learning rate and optimizer to ensure the pruned model maintains good recognition capabilities. The pruned and fine-tuned model possesses a more compact network structure and stronger generalization ability, reducing the risk of overfitting and, in some cases, achieving higher detection accuracy than the initial model.

[0087] In the loss functions used during model training, the localization regression loss is the sum of the CIoU loss function and the DFL loss function, while the classification loss is the variable focal length (VFL) loss function. This is similar to the training loss function in the YOLO network, which helps to balance the convergence and regression speed and training accuracy of the model.

[0088] 2. Example

[0089] In this embodiment, the method of the present invention, as well as other methods, are tested and verified to better demonstrate the technical advantages of the method of the present invention. These are described in detail below.

[0090] 2.1 Experimental Parameters

[0091] The deep learning experimental environment configured in this embodiment is as follows: CPU: Intel(R) Xeon(R) Bronze3204 CPU (1.9GHz); GPU: NVIDIA GeForce RTX 2060, 6144MiB; compiler: Python-3.8; deep learning framework: PyTorch-1.13.1; acceleration module: CUDA-11.7. The hyperparameters configured during the experiment are as follows: Epochs: 100; Batch-size: 32; Optimizer: SGD; Ir: 0.01; Momentum: 0.937.

[0092] 2.2 Comparative Experiment

[0093] 2.2.1 Comparison of C2f-Star Modules

[0094] To verify the effectiveness of the C2f-Star module, the proposed C2f-Star module was compared with modules such as C2f-Faster (incorporating FasterNet ideas), C2f-DLKA (incorporating a deformable large kernel attention mechanism), C2f-RVB (incorporating RepViT Block), C2fCIB from YOLOv10, and C2f-iRMB (incorporating a reverse residual moving block). The experimental results are shown in Table 1.

[0095] Table 1 Comparison of experimental results for different modules

[0096]

[0097] As shown in Table 1, under the same conditions, the proposed C2f-Star module can effectively improve accuracy while reducing model computation. The recall rate of the fused C2f-Star model reached 0.893, and the mAP50 value reached 0.949, leading the C3, C2f, C2FCIB, C2f-DLKA, C2f-iRMB, C2f-RVB, C2f-DCNv2, and C2f-Faster modules by 0.2, 2.1, 3.5, 0.5, 0.8, 0.3, 0.2, and 1.4 percentage points, respectively. The mAP50-95 value reached 0.701, which is the highest among all compared modules, leading C3 and C2f. The C2f-Star module, compared to other modules such as C2FCIB, C2f-DLKA, C2f-iRMB, C2f-RVB, C2f-DCNv2, and C2f-Faster, shows a performance reduction of 3.7, 3.5, 3.0, 1.4, 2.0, 1.0, 1.4, and 2.4 percentage points respectively. It has 2.77M parameters, a 7.9% reduction compared to the baseline model, and a computational cost of 7.5G, a 6.3% reduction compared to the baseline model, making it the lowest among all modules. Its FPS reaches 53.2 frames per second, the fastest inference speed among all compared models. In summary, the C2f-Star proposed in this invention helps the model reduce computational load and improve detection accuracy. The model incorporating the C2f-Star module has a more compact network structure and lower computational cost. It can combine input features from different subspaces through element-wise multiplication and create numerous interaction terms during detection to increase the dimensionality of the feature space. It obtains implicit high-dimensional features after computation in a low-dimensional space, effectively capturing complex feature dimensions and possessing a clearer decision boundary.

[0098] 2.2.2 Comparison of detection heads

[0099] To verify the effectiveness of the detection head proposed in this invention, the PSDH detection head with shared convolutional layer structure proposed in this invention was compared with other detection heads such as Dyhead (designed with an attention mechanism), Detect-Aux (designed with reference to YOLOV7), Detect-SEAM (designed with a SEAM attention mechanism), and Detect-AFPN (designed with a feature pyramid layer). The experimental results are shown in Table 2.

[0100] Table 2. Comparison of experimental results for different detection heads

[0101]

[0102] As shown in Table 2, the detection head module proposed in this invention is effective under the same conditions, maintaining high accuracy while significantly reducing the number of parameters. The detection head using the parameter-sharing convolutional layer structure of this invention achieves a recall rate of 0.873 and an mAP50 value of 0.948, respectively leading the Detect-v8, Dyhead, Detect-Aux, Detect-SEAM, Detect-AFPN, and Detect-v10 comparison modules by 2.0, 2.1, 0.7, 0.5, 5.7, and 8.0 percentage points; the mAP50-95 value reaches 0.682, leading the Detect-v8, Dyhead, Detect-AFPN, and Detect-v10 comparison modules by 1.6, 1.5, 6.9, and 9.6 percentage points; the number of parameters is 2.42M, a 19.6% reduction compared to the baseline model; the computational cost is 5.5G, a 32.1% reduction compared to the baseline model, the lowest among all modules; and the FPS reaches 52.3 frames / second, leading most of the comparison detection heads. In summary, the detection head proposed in this invention can maintain accuracy while reducing computational load. After fusing the PSDH detection head, the model can share parameters, and select some channels for calculation using the integrated PConv and PWConv, effectively reducing the number of model parameters and computational load, and better addressing resource-constrained edge device application scenarios.

[0103] 2.2.3 Comparison of Model Pruning Parameter Settings

[0104] To further reduce computational load and improve real-time inference speed, a channel pruning algorithm is used to constrain the structure at different levels of the model, identifying and automatically pruning unimportant channels. The pruning process consists of two main steps: first, importing a pre-trained model for sparse training; second, setting a pruning rate to prune the model; and finally, performing network fine-tuning to restore model accuracy after pruning.

[0105] (1) Sparsification training

[0106] During sparse training, a suitable sparsity coefficient γ needs to be set to filter channels. If the sparsity coefficient is too small, the sparsification process will be too slow, and the model will not be able to effectively identify unimportant channels; if the sparsity coefficient is too large, the sparsification process will be too fast, resulting in too much loss of model accuracy. To determine the optimal sparsity coefficient γ, training was conducted with coefficients set to 0.005, 0.01, 0.02, 0.03, 0.04, and 0.05, respectively. The experimental results are shown in Table 3. Under the same pruning rate, the model performs best when the sparsity coefficient γ is set to 0.04. At this value, the model's mAP50 value reaches 0.945, the mAP50-95 value reaches 0.697, and the FPS reaches 53.8 frames / second. The model can identify unimportant channels to the greatest extent while maintaining accuracy. Therefore, the sparsity coefficient γ is set to 0.04.

[0107] Table 3 Experimental results of model performance under different sparsity coefficients

[0108]

[0109] (2) Model pruning and network fine-tuning

[0110] After sparse training, the model was pruned, and the optimal pruning rate was determined based on the specific results. After pruning, the network was fine-tuned to obtain the final model. Table 4 shows the training results after fine-tuning the model under different pruning rates. The model achieved the highest accuracy when the pruning rate was 2.4. At this rate, the model's mAP50 value reached 0.952, the mAP50-95 value reached 0.703, and the FPS reached 55.2 frames per second.

[0111] Table 4. Experimental results of model performance under different pruning rates

[0112]

[0113] 2.2.4 Comparison of different models

[0114] To further verify the effectiveness of the LHNFNet network model proposed in this invention, experiments were conducted on the same dataset under the same environment, comparing it with mainstream object detection algorithms such as Faster-RCNN, TOOD, Cascad-RCNN, YOLOX, YOLOv9, and YOLOv10. The comparison results are shown in Table 5.

[0115] Table 5 Comparison of experimental results of the model

[0116]

[0117] The LHNFNet network model proposed in this invention outperforms comparative models in terms of detection accuracy, number of parameters, and inference speed, demonstrating superior accuracy and efficiency in resource-constrained detection scenarios. The LHNFNet network model achieves an mAP50 value of 0.952, a 2.4 percentage point improvement over the baseline model, and outperforms Faster-RCNN, Cascad-RCNN, TOOD, YOLOX, ATSS, RetinaNet, RTMDet, CenterNet, RTDETR, GFL-R50, SSD, EfficientNet, YOLOv5, YOLOv7-tiny, YOLOv9-tiny, and YOLOv10 by 5 points respectively. The percentages of 8, 5.6, 6.7, 4.5, 8.5, 5.3, 7.1, 7.5, 8.5, 11.1, 9.2, 9.2, 1.1, 5.5, 6.1, and 5.4 percentage points indicate that the LHNFNet network model significantly improves the feature space dimensionality by combining input features from different subspaces through element-wise multiplication, creating a large number of interaction terms. This effectively captures complex image features and delineates more precise decision boundaries. The mAP50-95 value reached 0.703, an improvement of 3.7 percentage points compared to the baseline model, and comparable to Faster-RCNN and Cas... Compared to cad-RCNN, TOOD, YOLOX, ATSS, RetinaNet, RTMDet, CenterNet, RTDETR, GFL-R50, SSD, EfficientNet, YOLOv5, YOLOv7-tiny, YOLOv9-tiny, and YOLOv10, the performance improvements are 9.1, 9.2, 5.9, 2.4, 5.6, 3.2, 2.9, 7.6, 3.9, 7.1, 2.5, 5.2, 2.3, 6.4, 9.4, and 6 respectively. The LHNFNet network model achieves a 0.2 percentage point improvement; with a computational cost of 0.96M and a parameter count of 2.0G, it is the lightest among all comparison models. Its FPS reaches 55.2 frames per second, outperforming most of the comparison models. This demonstrates that the LHNFNet network model utilizes the high similarity of feature maps across different channels to extract features from some inputs while keeping the remaining channels unchanged. This reduces feature map redundancy and memory access, efficiently captures spatial features, and uses channel pruning algorithms to constrain the structure at different levels of the model, identifying and automatically pruning unimportant channels, compressing the model size, and improving detection performance. In conclusion, the LHNFNet network model boasts a lighter structure and higher accuracy.

[0118] 2.3 Ablation Experiment

[0119] To verify the performance advantages of the design module of this invention, ablation experiments were conducted on three parts: C2f-Star, PSDH parameter-sharing detection head, and channel pruning. The experimental results are shown in Table 6.

[0120] Table 6 Comparison Results of Ablation Experiments

[0121]

[0122] By integrating C2f-Star, the model recall reached 0.893, an improvement of 6.8 percentage points compared to the baseline model. The mAP50 and mAP50-95 values ​​reached 0.949 and 0.701, respectively, improvements of 2.1 and 3.5 percentage points compared to the baseline model, indicating that the model's feature extraction capability was enhanced, enabling it to delineate more accurate boundaries for image recognition. By integrating a parameter-sharing detection head, the model recall reached 0.873, and the mAP50 and mAP50-95 values ​​reached 0.948 and 0.682, respectively. The number of parameters was 2.42M, a reduction of 19.6% compared to the baseline model, and the computational cost was 5.5G, a reduction of 32.1% compared to the baseline model. This indicates that the proposed detection head only extracts features from a portion of the channels for training, maintaining high accuracy while reducing computational cost.

[0123] By integrating C2f-Star and a parameter-sharing detector, the model achieved mAP50 and mAP50-95 values ​​of 0.953 and 0.699, respectively, with 2.18M parameters and 4.8G computation. The model achieved a slight increase in accuracy while further reducing the number of parameters. Channel pruning further improved the model, resulting in mAP50 and mAP50-95 values ​​of 0.952 and 0.703, respectively, representing improvements of 2.4 and 3.7 percentage points compared to the baseline model. The number of parameters was reduced to 0.96M, a 68.1% reduction compared to the baseline model, and the computation was reduced to 2.0G, a 75.3% reduction compared to the baseline model. The FPS reached 55.2 frames per second, the fastest among all experiments, indicating that the pruning process resulted in a more compact network structure and stronger generalization ability, leading to enhanced performance and faster inference speed.

[0124] 2.4 Visual Verification

[0125] Figure 6 These are four different sample images of human behavior used in this visualization recognition verification. Figure 7 The results of different models for detecting four sample images of human behavior are shown. Figure 6 and Figure 7 As can be seen, when detecting images of human behavior, the LHNFNet network model proposed in this invention outperforms other comparative models in both detection accuracy and performance. Specifically, in actual detection, this model can achieve accurate bounding box localization, not only correctly classifying the category but also significantly improving confidence. This performance indicates that the model has superior detection performance in practical applications and can more reliably identify human behavioral characteristics.

[0126] During testing Figure 6 In images (1), (2), and (3), the LHNFNet network model proposed in this invention exhibits higher detection confidence, meaning it has a stronger ability to accurately identify human behavior. In comparison, other comparative models perform slightly worse in these aspects. Furthermore, in detection... Figure 6 In the case of image (4), some other models misidentified the background as human behavior, while the LHNFNet network model proposed in this invention can effectively distinguish between the background and the target object, avoiding false detection problems, and its confidence level in this scenario is as high as 0.92. This result further proves the superiority of the LHNFNet network model proposed in this invention in terms of performance and accuracy, and it is indeed superior to other comparative models. Through this accurate discrimination ability and high confidence, this invention can provide a more reliable and effective solution for human behavior detection.

[0127] 3. Overview

[0128] This invention proposes a method for detecting human behavior based on the LHNFNet network model. The optimized LHNFNet network model is used to detect human behavior images. In this optimized model, a C2f-Star module with star operations is designed in the backbone layer. This module combines input features from different subspaces through element-wise multiplication, creating numerous interaction terms that significantly increase the feature space dimensionality. This allows for computation within a compact, low-dimensional feature space while benefiting from implicit high-dimensional features, thus enhancing feature extraction capabilities. A lightweight detection head with a parameter-sharing convolutional layer structure is also designed, using continuous channels from the front or back ends for computation, reducing the number of parameters and memory accesses generated during computation. Furthermore, a channel pruning algorithm is applied to constrain the hierarchical structure of the LHNFNet network model, enforcing channel-level sparsity, identifying and automatically pruning unimportant channels, compressing the model size, and improving detection performance. This enables the model to meet deployment requirements on resource-constrained edge devices.

[0129] Experimental results show that the proposed LHNFNet network model achieves a recall rate of 0.886, which is 6.1 percentage points higher than the baseline model; the mAP50 and mAP50-95 values ​​reach 0.952 and 0.703, respectively, which are 2.4 and 3.7 percentage points higher than the baseline model. The number of parameters is 0.96M, which is 68.1% lower than the baseline model, and the computational cost is 2.0G, which is 75.3% lower than the baseline model. Compared with models such as Faster-RCNN, TOOD, Cascad-RCNN, YOLOX, YOLOv9, and YOLOv10, the proposed LHNFNet network model demonstrates superior effectiveness in terms of detection accuracy, number of parameters, and inference speed. Therefore, the LHNFNet network model and method proposed in this invention are feasible.

[0130] In summary, the personnel behavior detection method based on the LHNFNet network model proposed in this invention can reduce the false negative rate and false positive rate while achieving lightweight computation for personnel behavior detection and recognition, and improve the accuracy of personnel behavior detection. It effectively solves the problems of insufficient accuracy, high false negative rate and false positive rate of personnel behavior detection when implementing lightweight deployment on edge devices.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described with reference to preferred embodiments, those skilled in the art should understand that various changes in form and detail can be made without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A method for detecting human behavior based on the LHNFNet network model, characterized in that, Includes the following steps: Obtain a dataset of human behavior image samples containing both abnormal and normal behavior image samples; Based on the YOLO network model architecture, the backbone network layer is optimized by using the C2f-Star module that integrates star operation operations, and a lightweight detection head is used to construct the LHNFNet network model. The LHNFNet network model was trained using the aforementioned human behavior image sample dataset, and the hierarchical structure of the trained LHNFNet network model was constrained by the channel pruning algorithm to obtain the trained LHNFNet network model. The trained LHNFNet network model is used to detect the human behavior images to be detected, and the detection results of the human behavior images to be detected are obtained.

2. The method for detecting human behavior based on the LHNFNet network model according to claim 1, characterized in that, The LHNFNet network model includes a backbone network layer, a neck network layer, and a detection head. The backbone network layer comprises a convolutional layer, three convolutional fusion units, one convolutional fusion star operation unit, and one spatial pyramid pooling module connected in sequence; wherein, each convolutional fusion unit comprises a cascaded convolutional layer and a C2f module; the convolutional fusion star operation unit comprises a cascaded convolutional layer and a C2f-Star module; The neck network layer includes two feature fusion upsampling units and two convolutional connection feature fusion units connected in sequence. The feature fusion upsampling unit includes an upsampling module, a connection layer, and a C2f module connected in sequence; the convolutional connection feature fusion unit includes a convolutional layer, a connection layer, and a C2f module connected in sequence. The detection head obtains the detection results of the human behavior image based on the output of the neck network layer, and uses it as the output of the LHNFNet network model; In this model, the input of the LHNFNet network model serves as the input of the backbone network layer. Within the backbone network layer, the outputs of the second and third convolutional fusion units also serve as the inputs to the connection layers of the second and first feature fusion upsampling units in the neck network layer, respectively. In addition to serving as the input of the spatial pyramid pooling module in the backbone network layer to the neck network layer, the output of the spatial pyramid pooling module also serves as the input to the connection layer of the second convolutional connection feature fusion unit in the neck network layer. Furthermore, within the neck network layer, the output of the first feature reconstruction upsampling unit also serves as the input to the connection layer of the first convolutional connection feature fusion unit. The outputs of the second feature fusion upsampling unit and the two convolutional connection feature fusion units also serve as the inputs to the detection head.

3. The method for detecting human behavior based on the LHNFNet network model according to claim 2, characterized in that, The C2f-Star module includes a cascaded first convolutional layer, a split module, several star operation bottleneck modules, a connection layer, and a second convolutional layer. The two split outputs of the split module serve as the inputs to the first fast convolutional module and the connection layer, respectively. The outputs of each star operation bottleneck module also serve as the inputs to the connection layer. The output of the second convolutional layer serves as the output of the C2f-Star module.

4. The method for detecting human behavior based on the LHNFNet network model according to claim 3, characterized in that, The processing procedure of the star operation bottleneck module includes: First, the input feature map is subjected to a first depthwise convolution. Then, two extension branches are used to perform channel expansion processing on the result of the first depthwise convolution. One extension branch is expanded by 4 times through a fully connected layer before ReLU activation function operation. The other extension branch is first batch normalized and then expanded by 4 times through a fully connected layer. The channel expansion results of the two extension branches are then subjected to star operation operation. The star operation operation result is then compressed back to 1 channel through a fully connected layer. Batch normalization and a second depthwise convolution are then performed in sequence. Finally, the output of the second depthwise convolution is superimposed on the input feature map and used as the output of the bottleneck module of star operation operation.

5. The method for detecting human behavior based on the LHNFNet network model according to claim 3, characterized in that, The star operation is represented as follows: in, This diagram shows the channel expansion results for the two extended branches. This represents the channel expansion result diagram. The image of any i-th channel, This represents the channel expansion result diagram. The image of any j-th channel, i,j = 1, 2, ..., 4d; d represents the number of channels in the input feature map of the bottleneck module of the star operation, and 4d is the number of channels after quadrupling the channels using the extended branches; W1 and W2 represent the channel expansion results of the star operation for the two extended branches, respectively. The weight matrix; This represents the weight value of the i-th channel in the weight matrix W1. This represents the weight value of the j-th channel in the weight matrix W2; * is the star operator; T is the transpose symbol; X * This indicates the output of the star operation.

6. The method for detecting human behavior based on the LHNFNet network model according to claim 2, characterized in that, The processing procedure of the detection head includes: First, the input feature map of the detection head is processed by partial convolution PConv and point-directed convolution PWConv operations in sequence. Then, the processing results are fed into the convolutional layers of the localization regression detection channel and the classification detection channel respectively, and the localization regression detection result and the classification detection result are output respectively.

7. The method for detecting human behavior based on the LHNFNet network model according to claim 2, characterized in that, The LHNFNet network model is trained in the following manner: S101: Obtain the personnel behavior image sample dataset, label the abnormal behavior image samples and normal behavior image samples in the personnel behavior image sample dataset, and divide them into training set, validation set and test set according to a preset ratio; S102: Input the training set into the LHNFNet network model for training. Optimize the parameters of the LHNFNet network model with the goal of minimizing the loss function until the LHNFNet network model converges, thus obtaining the pre-trained LHNFNet network model. S103: The channel pruning algorithm is used to perform channel pruning on the pre-trained LHNFNet network model to obtain the trained and pruned LHNFNet network model. S104: Test the trained and pruned LHNFNet network model using a test set to confirm the recognition performance of the LHNFNet network model; if the recognition performance meets the requirements, end the training of the LHNFNet network model to obtain the trained LHNFNet network model. Otherwise, the training set is input into the pruned LHNFNet network model for retraining until the LHNFNet network model converges, resulting in the trained LHNFNet network model.

8. The method for detecting human behavior based on the LHNFNet network model according to claim 7, characterized in that, The process of performing channel pruning on the LHNFNet network model using the channel pruning algorithm includes: First, the number of channels in each batch normalized layer of the trained LHNFNet network model is counted. The output of each channel in each batch normalized layer is multiplied by a sparsity coefficient γ and then L1 regularized to obtain the performance contribution value for that channel. Channels in the batch normalized layer whose performance contribution value is less than a preset contribution threshold are designated as pruned channels and are masked, retaining the remaining channels of the batch normalized layer. This completes the channel pruning adjustment for each batch normalized layer. Then, corresponding channel pruning adjustments are performed on the convolutional and linear layers in the upstream and downstream operations of each normalized layer to ensure that the number of channels in the upstream and downstream operations is consistent. This completes the channel pruning processing of the LHNFNet network model, and the parameters of the LHNFNet network model after channel pruning are saved.

9. The method for detecting human behavior based on the LHNFNet network model according to claim 7, characterized in that, In the loss function, the localization regression loss is the total loss function formed by the sum of the CIoU loss function and the DFL loss function, and the classification loss is the variable focal length (VFL) loss function.

Images