Personnel violation behavior detection method based on lightweight model
By introducing lightweight modules and attention mechanisms into the YOLOv8 network and replacing the detection head, the problem of YOLOv8 training time in high-resolution image processing is solved, and fast and efficient small object detection is achieved.
Patent Information
- Application Number
- CN202411670451.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-21
- Publication Date
- 2025-04-04
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
When YOLOv8 processes high-resolution images, due to the large amount of parameters and calculation complexity, the training time is too long, making it difficult to accurately capture and identify small targets such as smoke and mobile phones in the chemical plant area.
By improving the YOLOv8 network, a custom lightweight module C2f_GSBlock, PKConv module and GSAM attention mechanism were introduced, and the original detection head was replaced with the FASFHead detection head, reducing the computational complexity and enhancing the feature expression ability.
It achieves faster and more efficient target detection, can accurately capture and identify small targets in the picture, and is suitable for detection of violations by chemical plant personnel, and has significantly decreased model size, parameter quantity and GFLOPs.
Smart Images
Figure CN119763180A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of target detection, and in particular to a method for detecting personnel violation behaviors based on a lightweight model. Background Art
[0002] With the rapid development of industrial automation, the application of machine vision technology in the field of chemical production safety is becoming more and more extensive. Especially in chemical plant areas, machine vision systems are of great significance for real-time monitoring and identification of personnel violations. However, in practical applications, the detection of smoking and phone calls still faces many difficulties: ① The shape of smoke is small and the proportion of smoke in the image is often very small, which makes it particularly difficult to capture smoke. ② In the surveillance video of the chemical plant area, when people make phone calls, mobile phones often appear as small target objects. Since the proportion of mobile phones in the image is small, it is difficult for machine vision systems to accurately capture and identify these small targets, and YOLOv8 needs to maintain high efficiency when processing high-resolution images, resulting in large model parameters, reduced processing speed and long training time. Summary of the invention
[0003] In view of the technical problem that when YOLOv8 uses weights to train the target detection model, the training time is too long due to the large number of parameters and GFLOPs, the present technical solution provides a personnel violation detection method based on a lightweight model, and uses the YOLOv8 network model as a benchmark to improve YOLOv8s-GPGF. The improved detection model backbone network is more lightweight, has low computational complexity, and has strong feature expression capabilities; it can quickly and effectively capture and accurately identify small targets in images, and can be applied to personnel violation detection in chemical plants; it effectively solves the above problems.
[0004] The present invention is achieved through the following technical solutions:
[0005] A method for detecting personnel violation behavior based on a lightweight model, specifically comprising the following steps:
[0006] Step 1: Use a self-built dataset of chemical plant personnel violations and divide it into a training set, a validation set, and a test set;
[0007] Step 2: Improve YOLOv8 and build a new YOLOv8s-GPGF model; the specific operation method is:
[0008] Step 2.1: Based on the backbone structure of YOLOv8, use the customized lightweight module C2f_GSBlock to replace the C2f module in the network. C2f_GSBlock uses multi-scale convolution, residual connection and feature concatenation to extract local and global feature information at the same time;
[0009] Step 2.2: Introduce the PKConv module into the network to replace the Conv module in the network; PKConv captures and fuses spatial information in different directions and reweights the original features;
[0010] Step 2.3: Add a custom GSAM attention mechanism to the backbone network, and use the combination of spatial channel attention and multi-layer perceptron to combine the original features with the features extracted by the attention mechanism to enhance the feature expression capability;
[0011] Step 2.4: In the detection head part, use the customized FASFHead detection head to replace the original detection head. The FASFHead detection head combines the attention mechanism of scale, space and task dimensions, enabling the model to optimize features from multiple angles;
[0012] Step 3: Use the training set in step 1 to train the improved target detection model, and evaluate and judge the improved model based on the training results;
[0013] Step 4: Use the trained improved model to test the test set in step 1 and output the test results.
[0014] Furthermore, the specific operation method of step 2.1 is:
[0015] Step 2.1.1: Input feature map X1,X2,…,X n It is split into multiple branches, each branch processes a part of the input data;
[0016] Step 2.1.2: In each branch, first use 1×1 convolution to reduce the dimension or adjust the number of channels;
[0017] Step 2.1.3: Extract spatial features through k×k convolution;
[0018] Step 2.1.4: Use 1×1 convolution* to restore the number of channels;
[0019] Step 2.1.5: For each branch, after all operations, add the result to the corresponding branch feature map, connect the residuals, and retain the original information of the input features;
[0020] Step 2.1.6: Combine the output results of all branches through the splicing operation. The feature map generated after splicing will contain the feature information from each branch;
[0021] Step 2.1.7: Repeat steps 2.1.1 to 2.1.7 multiple times to further extract rich features.
[0022] Furthermore, let X∈R H×W×Cis the input feature, where R is the resolution or size of the input feature, H is the height of the input feature, W is the width of the input feature, and C is the number of channels of the input feature; after 1×1 convolution transformation, the channel dimension of X increases to n×C; X is divided into n different groups, denoted as {X i}, where i∈1,2,3,…,n; in order to reduce the computational cost, n=3; except X1, each group must go through a reverse bottleneck layer, denoted as IB k×k (·), where k represents the kernel size, to obtain Y i ; Y i The mathematical expression is:
[0023]
[0024] According to the formula, it acts as a cross-level connection and retains information from the previous layer; all splits are connected and 1×1 convolution is applied to interact between all splits, each of which encodes features at a different scale; 1×1 convolution is also used to adjust the number of channels when the network goes deeper.
[0025] Furthermore, the specific operation mode of the PKConv module described in step 2.2 is:
[0026] Step 2.2.1: The input tensor dimension is C×H×W. The input directly passes through a residual connection, and the output still maintains the dimension of C×H×W.
[0027] Step 2.2.2: Through X and Y average pooling, the average pooling in the X direction performs average pooling on the input feature map in the horizontal width W, and the output dimension is C×1×W; the average pooling in the Y direction performs average pooling on the input feature map in the vertical height H, and the output dimension is C×H×1;
[0028] Step 2.2.3: Concatenate the average pooling results in the X and Y directions in the channel dimension to obtain a tensor with a dimension of C×1×(W+H); the concatenated tensor passes through a convolution layer to obtain an output with a dimension of C×1×(W+H); the result after convolution is normalized and processed by a nonlinear activation function, and the output dimension remains unchanged, still C×1×(W+H); the processed tensor is split into two tensors, corresponding to the X and Y directions, with dimensions of C×1×W and C×1×H respectively; apply separate convolution layers to these two tensors respectively, and obtain two output tensors with dimensions of C×1×W and C×1×H respectively;
[0029] Step 2.2.4: The output of each convolutional layer passes through the Sigmoid activation function, and the output tensor dimensions remain unchanged, which are C×1×W and C×1×H respectively; the tensor after Sigmoid activation is multiplied by the initial residual connection result, and the output dimension after reweighting is C×H×W;
[0030] Step 2.2.5: The final output tensor has the same dimensions as the input, C×H×W.
[0031] Furthermore, the specific process of step 2.3 is as follows:
[0032] Step 2.3.1: Input features. The input of the module is a multi-dimensional feature map. Perform permutation operations on the input features to adjust the data format or channel arrangement to adapt to subsequent operations.
[0033] Step 2.3.2: The permuted features enter a multi-layer perceptron (MLP), which consists of a linear layer, a ReLU activation function, and another linear layer; further extract and adjust the features on the channel;
[0034] Step 2.3.3: After processing by MLP, perform an inverse permutation operation to restore the original feature arrangement;
[0035] Step 2.3.4: Use the Sigmoid activation function to compress the feature value between 0 and 1 to generate channel attention weights; channel attention weights represent the importance of each channel;
[0036] Step 2.3.5: By multiplying the channel attention weights with the input features point by point, the response to important channels is enhanced and unimportant channels are suppressed. After obtaining the channel-enhanced features, a 7×7 convolution layer is passed through, which is used to capture the spatial relationship and position information of the feature map. The size of the 7×7 convolution kernel helps to obtain a larger range of spatial information.
[0037] Step 2.3.6: Perform batch normalization on the convolutional features to ensure the stability of network training, and then use the ReLU activation function to increase nonlinear expression capabilities; then, another layer of 7×7 convolution is performed, which further enhances the extraction of spatial features, especially in conjunction with the first layer of convolution to capture more complex spatial dependencies; batch normalization is performed again to prepare for subsequent activation;
[0038] Step 2.3.7: Compress the spatial weight of the feature to between 0 and 1 through the Sigmoid activation function to generate the spatial attention weight; the spatial attention weight is used to emphasize the importance of a specific spatial position; the spatial weight is multiplied point by point with the channel-enhanced feature to enhance the feature expression of the specific spatial position;
[0039] Step 2.3.8: Add the features enhanced by channel and spatial attention to the original input features to obtain the final output features; this process is similar to residual connection, which ensures the combination of original features and features extracted by attention mechanism and enhances the feature expression ability.
[0040] Furthermore, the FASFHead detection head described in step 2.4 includes three main parts, including attention modules for processing scale attention, spatial attention and task attention respectively; each attention module extracts and enhances features of different dimensions on the input respectively, combines the attention mechanism parameters of the generative model, and finally outputs the model.
[0041] Furthermore, the specific operation steps of the FASFHead detection head include:
[0042] Step 2.4.1: Processing scale attention:
[0043] Step 2.4.1.1: The input feature map is first subjected to an average pooling operation to extract global features;
[0044] Step 2.4.1.2: The pooled features are linearly combined between channels through a 1x1 convolutional layer to reduce the feature dimension and extract higher-level features;
[0045] Step 2.4.1.3: The convolutional features are activated by ReLU function to add nonlinear features;
[0046] Step 2.4.1.4: The features are processed by the HardSigmoid function to generate scale attention weights. The generated scale attention weights are multiplied element-by-element with the input feature map to adjust the scale importance of the input features.
[0047] Step 2.4.2: Processing spatial attention:
[0048] Step 2.4.2.1: The input features first pass through the Index module to perform position encoding or indexing operations so that the convolutional layer can handle spatial correlations;
[0049] Step 2.4.2.2: After the 3x3 convolutional layer, extract local spatial features;
[0050] Step 2.4.2.3: The convolutional features are divided into two branches: one is to generate a spatial attention map through the Sigmoid activation function, and the other is to adjust the position or shape of the feature map through the Offset module;
[0051] Step 2.4.2.4: Multiply the generated spatial attention weights by the original input feature map to further adjust the spatial weights of the features;
[0052] Step 2.4.3: Handling Task Attention:
[0053] Step 2.4.3.1: The input features are first average pooled to extract global information;
[0054] Step 2.4.3.2: The pooled features are passed through a fully connected layer to extract features related to a specific task;
[0055] Step 2.4.3.3: Then add nonlinear features through ReLU, and then pass through another fully connected layer to output the task-related feature map;
[0056] Step 2.4.3.4: Normalize the feature map and adjust the value range;
[0057] Step 2.4.3.5: Multiply the normalized feature map by a predefined vector [1, 0, 0, 0] element-wise to generate the task attention weight.
[0058] Furthermore, the self-built data set described in step 1 contains 5312 images, which are divided into a training set, a validation set, and a test set in a ratio of 8:1:1; the images contain two violation categories of chemical plant personnel: smoking and making phone calls.
[0059] Beneficial Effects
[0060] The present invention proposes a method for detecting personnel violations based on a lightweight model, which has the following beneficial effects compared with the prior art:
[0061] (1) The present invention proposes a more lightweight model for detecting violations of chemical plant personnel. The C2f module in the network is replaced by a customized lightweight module C2f_GSBlock. C2f_GSBlock uses multi-scale convolution, residual connection and feature splicing to extract local and global feature information at the same time. Through efficient channel compression and feature aggregation, the computational complexity is reduced. At the same time, a customized new lightweight convolution PKConv is introduced into the network to replace the Conv module in the network. PKConv captures and fuses spatial information in different directions and reweights the original features to make the backbone network more lightweight. A customized GSAM attention mechanism is added to the backbone network. The combination of spatial channel attention and multi-layer perceptron is used to ensure the combination of original features and features extracted by the attention mechanism, thereby enhancing the feature expression capability. The detection head of YOLOv8 is also improved. The customized FASFHead detection head is used to replace the original detection head. The FASFHead detection head combines the attention mechanisms of scale, space and task dimensions, so that the model can optimize features from multiple angles, making the output features more accurate and targeted. The improved detection model can quickly and effectively capture and identify small targets in images, and can be applied to the detection of personnel violations in chemical plants. BRIEF DESCRIPTION OF THE DRAWINGS
[0062] Figure 1 It is a schematic diagram of the overall process of the present invention.
[0063] Figure 2 This is the network structure diagram of YOLOv8s-GPGF in the present invention.
[0064] Figure 3 This is the network structure diagram of the C2f_GSBlock module in the present invention.
[0065] Figure 4 This is the network structure diagram of the PKConv module in the present invention.
[0066] Figure 5 It is a structural diagram of GSAM in the present invention.
[0067] Figure 6 The structural diagram of the detection head FASFHead in the present invention
[0068] Figure 7 This is a diagram of the training results of YOLOv8s-GPGF in the present invention. DETAILED DESCRIPTION
[0069] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. The described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Under the premise of not departing from the design concept of the present invention, various modifications and improvements made by ordinary persons in the art to the technical solutions of the present invention should all fall within the protection scope of the present invention.
[0070] Embodiment 1:
[0071] like Figure 1 As shown, a method for detecting personnel violation behavior based on a lightweight model specifically includes the following steps:
[0072] Step 1: Use a self-built dataset of chemical plant personnel violation behaviors and divide it into training set, validation set and test set. The self-built dataset contains 5312 images, which are divided into training set, validation set and test set in a ratio of 8:1:1. The images contain two violation categories: smoking and making phone calls by chemical plant personnel.
[0073] Step 2: Improve YOLOv8 and build a new YOLOv8s-GPGF model; the modified YOLOv8 model backbone network is as follows Figure 2 As shown in the figure, a new network is composed of a layer of Conv, C2f_GSBlock and PKConv modules, and a custom GSAM attention mechanism is added to the YOLOv8 backbone network. Finally, the detection head is replaced with a custom FASFHead detection head. The specific operation method of improving YOLOv8 is:
[0074] Step 2.1: Based on the backbone structure of YOLOv8, use the custom lightweight module C2f_GSBlock to replace the C2f module in the network. The C2f_GSBlock module structure is as follows: Figure 3 As shown in the figure, it uses multi-scale convolution, residual connection and feature splicing to extract local and global feature information at the same time; the specific operation method is:
[0075] Step 2.1.1: Input feature map X1,X2,…,X n It is split into multiple branches, each branch processes a part of the input data.
[0076] Step 2.1.2: In each branch, first use 1×1 convolution to reduce the dimension or adjust the number of channels.
[0077] Step 2.1.3: Extract spatial features through k×k convolution.
[0078] Step 2.1.4: Use 1×1 convolution* to restore the number of channels.
[0079] Step 2.1.5: For each branch, after all operations, add the result to the corresponding branch feature map, connect the residuals, and retain the original information of the input features.
[0080] Step 2.1.6: Combine the output results of all branches through the splicing operation. The feature map generated after splicing will contain the feature information from each branch.
[0081] Step 2.1.7: Repeat steps 2.1.1 to 2.1.7 multiple times to further extract rich features.
[0082] Let X∈R H×W×C is the input feature, where R is the resolution or size of the input feature, H is the height of the input feature, W is the width of the input feature, and C is the number of channels of the input feature; after 1×1 convolution transformation, the channel dimension of X increases to n×C; X is divided into n different groups, denoted as {X i}, where i∈1,2,3,…,n; in order to reduce the computational cost, n=3; except X1, each group must go through a reverse bottleneck layer, denoted as IB k×k (·), where k represents the kernel size, to obtain Y i ; Y i The mathematical expression is:
[0083]
[0084] According to the formula, it acts as a cross-level connection and retains information from the previous layer; all splits are connected and 1×1 convolution is applied to interact between all splits, each of which encodes features at a different scale; 1×1 convolution is also used to adjust the number of channels when the network goes deeper.
[0085] Step 2.2: Introduce the PKConv module into the network to replace the Conv module in the network; the PKConv module structure is as follows Figure 4 As shown in the figure, PKConv captures and fuses spatial information in different directions and reweights the original features. The specific operation of the PKConv module is as follows:
[0086] Step 2.2.1: The input tensor dimension is C×H×W. The input directly passes through a residual connection, and the output still maintains the dimension of C×H×W.
[0087] Step 2.2.2: Through X and Y average pooling, the average pooling in the X direction performs average pooling on the input feature map in the horizontal width W, and the output dimension is C×1×W; the average pooling in the Y direction performs average pooling on the input feature map in the vertical height H, and the output dimension is C×H×1.
[0088] Step 2.2.3: Concatenate the average pooling results in the X and Y directions in the channel dimension to obtain a tensor with a dimension of C×1×(W+H); pass the concatenated tensor through a convolution layer to obtain an output with a dimension of C×1×(W+H); the result after convolution is normalized and processed by a nonlinear activation function, and the output dimension remains unchanged, still C×1×(W+H); the processed tensor is split into two tensors, corresponding to the X and Y directions, with dimensions of C×1×W and C×1×H respectively; apply separate convolution layers to these two tensors respectively, and obtain two output tensors with dimensions of C×1×W and C×1×H respectively.
[0089] Step 2.2.4: The output of each convolutional layer passes through the Sigmoid activation function, and the output tensor dimensions remain unchanged, namely C×1×W and C×1×H respectively; the tensor after Sigmoid activation is multiplied by the initial residual connection result, and the output dimension after reweighting is C×H×W.
[0090] Step 2.2.5: The final output tensor has the same dimensions as the input, C×H×W.
[0091] Step 2.3: Add a custom GSAM attention mechanism to the backbone network. The GSAM structure is as follows: Figure 5 As shown in the figure, it uses the combination of spatial channel attention and multi-layer perceptron to combine the original features with the features extracted by the attention mechanism to enhance the feature expression ability; the specific process is:
[0092] Step 2.3.1: Input features. The input of the module is a multi-dimensional feature map. The input features are permuted to adjust the data format or channel arrangement to adapt to subsequent operations.
[0093] Step 2.3.2: The permuted features enter a multi-layer perceptron (MLP), which consists of a linear layer, a ReLU activation function, and another linear layer; further extract and adjust the features on the channel.
[0094] Step 2.3.3: After processing through the MLP, perform an inverse permutation operation to restore the original feature arrangement.
[0095] Step 2.3.4: Use the Sigmoid activation function to compress the feature value between 0 and 1 and generate channel attention weights; channel attention weights indicate the importance of each channel.
[0096] Step 2.3.5: Multiply the channel attention weights by the input features point by point to improve the response to important channels and suppress unimportant channels. After obtaining the channel-enhanced features, pass through a 7×7 convolution layer, which is used to capture the spatial relationship and position information of the feature map. The size of the 7×7 convolution kernel helps to obtain a larger range of spatial information.
[0097] Step 2.3.6: Perform batch normalization on the convolutional features to ensure the stability of network training, and then use the ReLU activation function to increase the nonlinear expression ability; then, another layer of 7×7 convolution is performed, which further enhances the extraction of spatial features, especially in conjunction with the first layer of convolution to capture more complex spatial dependencies; batch normalization is performed again to prepare for subsequent activation.
[0098] Step 2.3.7: Through the Sigmoid activation function, the spatial weight of the feature is compressed to between 0 and 1 to generate the spatial attention weight; the spatial attention weight is used to emphasize the importance of a specific spatial position; the spatial weight is multiplied point by point with the channel-enhanced feature to enhance the feature expression of the specific spatial position.
[0099] Step 2.3.8: Add the features enhanced by channel and spatial attention to the original input features to obtain the final output features; this process is similar to residual connection, which ensures the combination of original features and features extracted by attention mechanism and enhances the feature expression ability.
[0100] Step 2.4: In the detection head part, use the customized FASFHead detection head to replace the original detection head. The structure of the FASFHead detection head is as follows: Figure 6 As shown in the figure, the FASFHead detection head combines the attention mechanisms in three dimensions: scale, space, and task, enabling the model to optimize features from multiple angles.
[0101] The FASFHead detection head consists of three main parts, including attention modules for processing scale attention, spatial attention, and task attention respectively; each attention module extracts and enhances features of different dimensions of the input, combines the attention mechanism parameters of the generative model, and finally outputs the model. The specific operation steps of the FASFHead detection head include:
[0102] Step 2.4.1: Processing scale attention:
[0103] Step 2.4.1.1: The input feature map is first subjected to an average pooling operation to extract global features.
[0104] Step 2.4.1.2: The pooled features are linearly combined between channels through a 1x1 convolutional layer to reduce the feature dimension and extract higher-level features.
[0105] Step 2.4.1.3: The convolutional features are activated by the ReLU function to add nonlinear features.
[0106] Step 2.4.1.4: The features are processed through the HardSigmoid function to generate scale attention weights. The generated scale attention weights are multiplied element-wise with the input feature map to adjust the scale importance of the input features.
[0107] Step 2.4.2: Processing spatial attention:
[0108] Step 2.4.2.1: The input features first pass through the Index module to perform positional encoding or indexing operations so that the convolutional layer can handle spatial correlations.
[0109] Step 2.4.2.2: After the 3x3 convolutional layer, local spatial features are extracted.
[0110] Step 2.4.2.3: The convolutional features are divided into two branches: one generates a spatial attention map through the Sigmoid activation function, and the other adjusts the position or shape of the feature map through the Offset module.
[0111] Step 2.4.2.4: Multiply the generated spatial attention weights and the original input feature map to further adjust the spatial weights of the features.
[0112] Step 2.4.3: Handling Task Attention:
[0113] Step 2.4.3.1: The input features are first average pooled to extract global information.
[0114] Step 2.4.3.2: The pooled features are passed through a fully connected layer to extract features related to the specific task.
[0115] Step 2.4.3.3: Then add nonlinear features through ReLU, and then pass through another fully connected layer to output the task-related feature map.
[0116] Step 2.4.3.4: Normalize the feature map and adjust the value range.
[0117] Step 2.4.3.5: Multiply the normalized feature map by a predefined vector [1, 0, 0, 0] element-wise to generate the task attention weight.
[0118] Step 3: Use the training set in step 1 to train the improved target detection model, and evaluate and judge the improved model based on the training results.
[0119] Step 4: Use the trained improved model to test the test set in step 1 and output the test results.
[0120] In this embodiment, the inventors conducted a comparative experiment on this solution and the original YOLOv8 network. The specific comparative experiment is as follows:
[0121] The original YOLOv8 network and the improved YOLOv8 network are trained as follows:
[0122] Original YOLOv8 network: Use YOLOv8s.pt weights and YOLOv8 network to iteratively train the dataset. Running environment: CUDA11.8; Pytorch11.8; PPUNVIDIAGeForceRTX2060super8G. Model size: 22.6MB, number of parameters: 11133324, GFLOPs: 28.4.
[0123] The model was trained with 300 iterations and weight s files, which resulted in a large number of parameters, complex model training, and a long training time.
[0124] The experiment shows that after 216 information processing layers, the number of parameters is 11126358 parameters, the GFLOPs reaches 28.4, and the model size is 22.6MB.
[0125] Improved YOLOv8 network: Use YOLOv8s.pt weights and YOLOv8 improved network to iteratively train the data set, running environment: CUDA11.8; Pytorch11.8; PPUNVIDIAGeForceRTX2060super8G. Model size: 8.9MB, number of parameters: 4105682, GFLOPs: 14.5. The comparison of the experimental results of this embodiment and YOLOv8 is shown in Table 1:
[0126] The GFLOPs size dropped by 48.9% from 28.4 to 14.5; the number of model parameters dropped by 63.1% from 11,126,358 to 4,105,682, which is a significant effect.
[0127] Table 1 Comparative experimental results
[0128] YOLO Parameter quantity Model size GFLOPs YOLOv8 11126358 22.6 28.4 Our 4105682 8.9 14.5
[0129] According to the data in Table 1, the YOLOv8 network in the present invention has a significant decrease in model size, parameter amount, and GFLOPs compared to the original YOLOv8 network, wherein the model size decreases by: (22.6-8.9) / 22.6=60.6%;
[0130] Parameter quantity decreased: (11126358-4105682) / 11126358=63.1%;
[0131] GFLOPs decrease: (28.4-14.5) / 28.4=48.9%.
[0132] Experimental results show that, on the self-built dataset of chemical plant personnel violations, the improved YOLOv8 model is 60.6% smaller than the original YOLOv8 network model (22.6M); the number of model parameters is reduced by 63.1%, and GFLOPs is reduced by 48.9%, showing a significant lightweight effect.
[0133] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with the technology to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent transformation or modification made according to the spirit of the present invention should be included in the present invention.
Claims
1. A method for detecting personnel violations based on a lightweight model, characterized in that: The specific steps include: Step 1: Use a self-built dataset of chemical plant personnel violations and divide it into a training set, a validation set, and a test set; Step 2: Improve YOLOv8 and build a new YOLOv8s-GPGF model; the specific operation method is: Step 2.1: Based on the backbone structure of YOLOv8, use the customized lightweight module C2f_GSBlock to replace the C2f module in the network. C2f_GSBlock uses multi-scale convolution, residual connection and feature concatenation to extract local and global feature information at the same time; Step 2.2: Introduce the PKConv module into the network to replace the Conv module in the network; PKConv captures and fuses spatial information in different directions and reweights the original features; Step 2.3: Add a custom GSAM attention mechanism to the backbone network, and use the combination of spatial channel attention and multi-layer perceptron to combine the original features with the features extracted by the attention mechanism to enhance the feature expression capability; Step 2.4: In the detection head part, use the customized FASFHead detection head to replace the original detection head. The FASFHead detection head combines the attention mechanism of scale, space and task dimensions, enabling the model to optimize features from multiple angles; Step 3: Use the training set in step 1 to train the improved target detection model, and evaluate and judge the improved model based on the training results; Step 4: Use the trained improved model to test the test set in step 1 and output the test results.
2. According to claim 1, a method for detecting personnel violations based on a lightweight model is characterized by: The specific operation method of step 2.1 is: Step 2.1.1: Input feature map X1,X2,…,X n It is split into multiple branches, each branch processes a part of the input data; Step 2.1.2: In each branch, first use 1×1 convolution to reduce the dimension or adjust the number of channels; Step 2.1.3: Extract spatial features through k×k convolution; Step 2.1.4: Use 1×1 convolution* to restore the number of channels; Step 2.1.5: For each branch, after all operations, add the result to the corresponding branch feature map, connect the residuals, and retain the original information of the input features; Step 2.1.6: Combine the output results of all branches through the splicing operation. The feature map generated after splicing will contain the feature information from each branch; Step 2.1.7: Repeat steps 2.1.1 to 2.1.7 multiple times to further extract rich features.
3. According to claim 2, a method for detecting personnel violations based on a lightweight model is characterized by: Let X∈R ×W×H is the input feature, where R is the resolution or size of the input feature, H is the height of the input feature, W is the width of the input feature, and C is the number of channels of the input feature; after 1×1 convolution transformation, the channel dimension of X increases to n×C; X is divided into n different groups, denoted as {X i }, where i∈1,2,3,…,n; except X1, each group must go through a reverse bottleneck layer, denoted as IB k×k (·), where k represents the kernel size, to obtain Y i ; Y i The mathematical expression is: According to the formula, it acts as a cross-level connection and retains information from the previous layer; all splits are connected and 1×1 convolution is applied to interact between all splits, each of which encodes features at a different scale; 1×1 convolution is also used to adjust the number of channels when the network goes deeper.
4. According to claim 1, a method for detecting personnel violations based on a lightweight model is characterized by: The specific operation mode of the PKConv module described in step 2.2 is: Step 2.2.1: The input tensor dimension is C×H×W. The input directly passes through a residual connection, and the output still maintains the dimension of C×H×W. Step 2.2.2: Through X and Y average pooling, the average pooling in the X direction performs average pooling on the input feature map in the horizontal width W, and the output dimension is C×1×W; the average pooling in the Y direction performs average pooling on the input feature map in the vertical height H, and the output dimension is C×H×1; Step 2.2.3: Concatenate the average pooling results in the X and Y directions in the channel dimension to obtain a tensor with a dimension of C×1×(W+H); the concatenated tensor passes through a convolution layer to obtain an output with a dimension of C×1×(W+H); the result after convolution is normalized and processed by a nonlinear activation function, and the output dimension remains unchanged, still C×1×(W+H); the processed tensor is split into two tensors, corresponding to the X and Y directions, with dimensions of C×1×W and C×1×H respectively; Apply separate convolutional layers to these two tensors, obtaining two output tensors of dimensions C×1×W and C×1×H respectively. Step 2.2.4: The output of each convolutional layer passes through the Sigmoid activation function, and the output tensor dimensions remain unchanged, which are C×1×W and C×1×H respectively; the tensor after Sigmoid activation is multiplied by the initial residual connection result, and the output dimension after reweighting is C×H×W; Step 2.2.5: The final output tensor has the same dimensions as the input, C×H×W.
5. According to claim 1, a method for detecting personnel violations based on a lightweight model is characterized by: The specific process of step 2.3 is as follows: Step 2.3.1: Input features. The input of the module is a multi-dimensional feature map. Perform permutation operations on the input features to adjust the data format or channel arrangement to adapt to subsequent operations. Step 2.3.2: The permuted features enter a multi-layer perceptron (MLP), which consists of a linear layer, a ReLU activation function, and another linear layer; further extract and adjust the features on the channel; Step 2.3.3: After processing by MLP, perform an inverse permutation operation to restore the original feature arrangement; Step 2.3.4: Use the Sigmoid activation function to compress the feature value between 0 and 1 to generate channel attention weights; channel attention weights represent the importance of each channel; Step 2.3.5: By multiplying the channel attention weights with the input features point by point, the response to important channels is enhanced and unimportant channels are suppressed. After obtaining the channel-enhanced features, a 7×7 convolution layer is passed through, which is used to capture the spatial relationship and position information of the feature map. The size of the 7×7 convolution kernel helps to obtain a larger range of spatial information. Step 2.3.6: Perform batch normalization on the convolutional features to ensure the stability of network training, and then use the ReLU activation function to increase nonlinear expression capabilities; then, another layer of 7×7 convolution is performed, which further enhances the extraction of spatial features, especially in conjunction with the first layer of convolution to capture more complex spatial dependencies; batch normalization is performed again to prepare for subsequent activation; Step 2.3.7: Compress the spatial weight of the feature to between 0 and 1 through the Sigmoid activation function to generate the spatial attention weight; the spatial attention weight is used to emphasize the importance of a specific spatial position; the spatial weight is multiplied point by point with the channel-enhanced feature to enhance the feature expression of the specific spatial position; Step 2.3.8: Add the channel- and spatial-attention-enhanced features to the original input features to obtain the final output features. This process is similar to residual connection, which ensures the combination of original features and features extracted by attention mechanism and enhances the feature expression ability.
6. The method for detecting personnel violations based on a lightweight model according to claim 1, characterized in that: The FASFHead detection head described in step 2.4 includes three main parts, including attention modules for processing scale attention, spatial attention and task attention respectively; each attention module extracts and enhances features of different dimensions on the input respectively, combines the attention mechanism parameters of the generative model, and finally outputs the model.
7. A method for detecting personnel violations based on a lightweight model according to claim 1 or 6, characterized in that: The specific operation steps of the FASFHead detection head include: Step 2.4.1: Processing scale attention: Step 2.4.1.1: The input feature map is first subjected to an average pooling operation to extract global features; Step 2.4.1.2: The pooled features are linearly combined between channels through a 1x1 convolutional layer to reduce the feature dimension and extract higher-level features; Step 2.4.1.3: The convolutional features are activated by ReLU function to add nonlinear features; Step 2.4.1.4: The features are processed by the HardSigmoid function to generate scale attention weights. The generated scale attention weights are multiplied element-by-element with the input feature map to adjust the scale importance of the input features. Step 2.4.2: Processing spatial attention: Step 2.4.2.1: The input features first pass through the Index module to perform position encoding or indexing operations so that the convolutional layer can handle spatial correlations; Step 2.4.2.2: After the 3x3 convolutional layer, extract local spatial features; Step 2.4.2.3: The convolutional features are divided into two branches: one is to generate a spatial attention map through the Sigmoid activation function, and the other is to adjust the position or shape of the feature map through the Offset module; Step 2.4.2.4: Multiply the generated spatial attention weights by the original input feature map to further adjust the spatial weights of the features; Step 2.4.3: Handling Task Attention: Step 2.4.3.1: The input features are first average pooled to extract global information; Step 2.4.3.2: The pooled features are passed through a fully connected layer to extract features related to a specific task; Step 2.4.3.3: Then add nonlinear features through ReLU, and then pass through another fully connected layer to output the task-related feature map; Step 2.4.3.4: Normalize the feature map and adjust the value range; Step 2.4.3.5: Multiply the normalized feature map by a predefined vector [1, 0, 0, 0] element-wise to generate the task attention weight.
8. The method for detecting personnel violation behavior based on a lightweight model according to claim 1, characterized in that: The self-built dataset described in step 1 contains 5312 images, which are divided into training set, validation set and test set in a ratio of 8:1:1; the images contain two violation categories: smoking and making phone calls by chemical plant personnel.
Citation Information
Cited By
SAR image-based wounded person detection method and electronic equipment
CN120260081A