A method for identifying violations of aerial power work

Abstract

The application relates to the technical field of target detection, in particular to a high-altitude electric power operation violation identification method, which discloses the following steps: collecting high-definition images and low-quality images of an electric power operation site by using a drone, and making an electric power site high-definition data set; constructing an image enhancement network based on a local information enhancement module and a global information enhancement module; building a violation identification model based on an improved YOLOv8; selecting a suitable optimizer and a loss function; training and optimizing the constructed model. The optimal model after training and optimization is converted into an RKNN model in the front end, and is deployed to the front end of the drone. Real-time detection is realized by relying on the camera in the front end, and whether the operation personnel violate the rules is judged by the judgment unit according to the detection result. The application provides a high-altitude electric power operation violation identification method, which considers the construction of an operation data set, the enhancement of low-quality images, and simultaneously proposes a violation identification network model based on an improved YOLOv8, so that the accuracy of detection is maximally improved.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, specifically to a method for identifying violations in high-altitude power operations. Background Technology

[0002] High-altitude work sites typically involve complex environments, such as overlapping construction sites or three-span structures. The confined and variable working spaces impose high technical demands on drone operations. Drone operation faces significant challenges in these environments, especially in adverse weather conditions such as rain, low light, and strong sunlight, where pilots are prone to focusing difficulties and flight instability. Furthermore, violations during high-altitude operations often pose substantial safety risks. However, due to the dynamic nature of the target object and the limited computing resources of edge devices, real-time identification of violations using existing technologies is difficult. This results in untimely detection of safety risks, necessitating manual judgment to identify potential hazards, which carries the risks of judgment lag and insufficient accuracy.

[0003] Therefore, improving the accuracy and real-time performance of identifying violations during high-altitude operations, as well as lowering the operational threshold, has become an urgent problem to be solved. Summary of the Invention

[0004] Purpose of the invention: In order to overcome the shortcomings of the prior art, the present invention provides a method for identifying violations in high-altitude power operations, which solves the problem of low accuracy in identifying violations in high-altitude operations.

[0005] Technical solution: This invention provides a method for identifying violations in high-altitude electrical work, the method comprising:

[0006] High-definition and low-quality images of high-altitude power operation sites are collected by drones. A high-definition dataset of power sites is created based on the high-definition and low-quality images. The high-definition dataset of power sites is obtained by annotating the high-definition images with targets to obtain the first tag file corresponding to the images.

[0007] The low-quality image is processed using a first processing model and a second processing model. The first processing method corresponding to the first processing model includes: introducing grouping branches into the first three convolutional layers of the dynamic contrast enhancement network structure; after processing the original low-quality image using the first processing method, a corresponding first feature map is finally obtained. The second processing method corresponding to the second processing model includes: firstly performing a convolution operation on the original low-quality image; then extracting image features through a multi-head attention module; and finally generating corresponding adjustment matrices and transformation values ​​through multilayer perception and linear transformation, respectively. The adjustment matrix is ​​used to correct the color of the image features, while the transformation value is used to adjust the illumination of the image features, thereby obtaining a corresponding second feature map. The first feature map and the second feature map are fused to obtain the final optimized image.

[0008] The optimized image is labeled with targets to obtain the corresponding second label file. The optimized dataset is obtained by combining the second label file with the corresponding image. The first processing model and the second processing model are trained using the optimized dataset until the final training result is achieved.

[0009] The high-definition power field dataset and the optimized dataset are combined to form the final dataset, which is then divided into a training set, a validation set, and a test set.

[0010] A traffic violation recognition model based on an improved YOLOv8 is constructed. The improved YOLOv8 traffic violation recognition model includes: replacing the backbone network in the YOLOv8 network structure with a Transformer model, replacing the C2f module in the YOLOv8 network structure with a C2f-CBDC module, and finally, the model outputs the result and label data corresponding to the traffic violation recognition.

[0011] The improved YOLOv8 violation recognition model is trained using the training set until convergence.

[0012] The trained model is evaluated using the test set, and its performance is judged based on the overall accuracy. When the accuracy stabilizes and improves, the final model is output; if the expected accuracy is not achieved, the model is retrained.

[0013] The optimal model after training and optimization is converted into an RKNN model for the front end and deployed to the front end of the drone;

[0014] Real-time detection is achieved by relying on front-end cameras. The detection results are then sent to the analysis unit to determine whether the relevant personnel have violated regulations, and the results are saved.

[0015] Furthermore, including:

[0016] The dynamic contrast enhancement network structure includes 7 convolutional layers. In the first 3 convolutional layers, channel separation is performed to add group branches. In each group branch, 1×1 convolution is used to increase the dimensionality. The number of feature channels in the first 3 convolutional layers is halved and then concatenated with the corresponding group branch to restore the original number of channels. The first 6 convolutional layers use the ReLU activation function, and the 7th convolutional layer uses the Tanh activation function.

[0017] Furthermore, including:

[0018] The dynamic contrast enhancement network structure further includes an illumination adjustment layer, which is used to adjust the illumination range of the image features output from the convolutional layer, and then output the first feature map. The illumination adjustment layer is based on an estimation formula representing the relationship between different features, and the estimation formula is expressed as:

[0019]

[0020] Where I represents the current image feature; a is a set threshold parameter, which is the same as the dimension of the input image feature. Based on this formula, all pixels in the image are normalized, and the graphic formed by the mapping relationship is used as the corresponding color channel to achieve the adjustment of the illumination range.

[0021] Furthermore, including:

[0022] The second processing model includes: a convolution module, an encoding module, a first-layer normalization module, a multi-head attention module, a second-layer normalization module, a multi-layer perception module, and a linear transformation module. The original low-quality image is input into the convolution module, which includes a 3×3 convolutional layer to obtain a first feature map with 64 channels. After dimensionality transformation by the encoding module, the first feature map is sequentially input into the first-layer normalization module and the multi-head attention module. The module extracts the corresponding second feature. The second feature passes through the second normalization module and then through the multi-layer perception module to generate a 3×3 adjustment matrix. This matrix is ​​then linearly transformed with the second feature output by the multi-head attention module to obtain a 1×1 transformation value.

[0023] Furthermore, including:

[0024] The process of fusing the first feature map and the second feature map to obtain the final optimized image specifically includes:

[0025] The second feature map is input into a convolutional layer with a size of 1×1 and 1 channel to obtain the third feature. The third feature is multiplied by the first feature to obtain the fourth feature. The fourth feature is then exponentially calculated with the third feature to output the final optimized image.

[0026] Furthermore, including:

[0027] The step of replacing the backbone network in the YOLOv8 network structure with a Transformer model includes: using the EfficientFormerV2 model in the Transformer model as the backbone network of the model.

[0028] Furthermore, including:

[0029] The replacement of the C2f module in the YOLOv8 network structure with the C2f-CBDC module includes:

[0030] In the EfficientFormerV2 model, Block1, Block2, and Block3 are all connected to a convolutional layer in the Neck part of the YOLOv8 network model, and the Fast Spatial Pyramid Pooling (SPPF) layer is also connected to a convolutional layer in the Neck part of the YOLOv8 network model. The C2f module in the Neck part of the YOLOv8 network model is replaced with a C2f-CBDC module. The C2f-CBDC module is a C2f module with a CBDC attention module added. The CBDC attention module integrates the CBAM attention module and deformable convolution. The CBAM attention module combines channel attention and spatial attention to refine the feature map in two different dimensions.

[0031] Furthermore, including:

[0032] The CBDC attention module integrates the CBAM attention module and deformable convolution, specifically including:

[0033] The Channel Attention Module (CAM) performs max pooling and average pooling operations on the input feature map to compress the information of each channel and generate two weight vectors. These two weight vectors are mapped through a shared network to obtain the weight matrix of each channel, and then multiplied element-wise with the original input feature map to enhance the feature response of important channels and suppress redundant or irrelevant channel information.

[0034] The Spatial Attention Module (SAM) focuses on spatial dimension features based on the output of the Channel Attention Module. In this module, max pooling and average pooling operations are applied to all channels at each location to generate two two-dimensional feature maps. Then, through deformable convolution operations, the channel information is fused, and the spatial attention matrix is ​​multiplied element-wise with the output feature map of the Channel Attention Module.

[0035] Furthermore, including:

[0036] The step of converting the trained and optimized optimal model into a front-end RKNN model and deploying it to the front end of the drone includes:

[0037] The trained image augmentation and object detection models are converted into a model format suitable for the RKNN framework using Python scripts. This conversion process includes the reconstruction of model structure and parameters to ensure that the model can run efficiently in the RKNN engine.

[0038] After the conversion is completed, a series of optimization operations are performed on the resulting RKNN model, including adaptive hybrid quantization. This process aims to improve the model's execution efficiency on embedded platforms by reducing the model's computational and storage requirements, while maintaining its accuracy level as much as possible.

[0039] Through multiple experiments, the impact of different quantization methods on model performance was verified, and the detection accuracy of each category was analyzed in detail to ensure that the model can maintain a high recognition accuracy in different scenarios.

[0040] The optimized RKNN model is deployed to the front end of the drone. During deployment, the model is loaded onto the hardware platform on the drone and the inference process is accelerated by the integrated neural network processing unit (NPU).

[0041] Furthermore, including:

[0042] The method of relying on a front-end camera to achieve real-time detection, sending the detection results to the analysis unit to determine whether the relevant workers have violated regulations, and saving the results, specifically includes:

[0043] The inspection drone began reading the real-time video stream and performing continuous frame-by-frame detection;

[0044] The drone detected the construction tower using the violation detection algorithm in the RKNN model and guided the drone to fly toward the tower.

[0045] After flying above the construction tower, the drone began to fly around the tower using preset latitude and longitude coordinates;

[0046] When workers are detected on the construction tower, the analysis unit judges whether the workers are working in accordance with regulations and whether the prescribed safety measures are in place within the worker's pixel frame based on the detection results.

[0047] Once a violation is identified, the detection and analysis results will be saved and sent back to the backend system, which will then conduct further analysis and processing of the data.

[0048] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0049] This invention proposes an image processing method based on two processing modes. This method can be trained using only low-quality images in an environment without paired data, greatly reducing the training difficulty. Furthermore, the method combines a first processing model and a second processing model to fully extract details and overall features from the image. In the first processing model, grouped branches and dilated convolutional layers are used to improve the network's sensitivity to details, making the enhancement process more refined and effectively preserving detailed information in the image, avoiding the color distortion and detail loss problems common in traditional methods. In the second processing model, the multi-head attention mechanism of the Transformer network is combined to capture the overall contextual information in the image, improving the overall contrast and brightness distribution of the image, thus effectively solving the common problems of over-enhancing bright areas and under-enhancing dark areas, further improving the visual effect of the image. In summary, the low-light image enhancement method proposed in this invention, with its innovative design of training without paired data and combining detailed and overall information, not only significantly reduces the training difficulty and data requirements but also achieves more refined and flexible image enhancement effects, possessing broad application prospects and excellent practical value.

[0050] In power operation environments, this invention improves upon the basic framework of YOLOv8: First, EfficientFormerV2 is selected as the feature extraction network to enhance the algorithm's ability to capture overall information. Second, the original C2f module is replaced with a C2f-CBDC module. This module integrates the CBAM attention module and deformable convolution, effectively reducing the model size while maintaining performance, dynamically adjusting the receptive field, focusing on detailed target features, significantly reducing background interference, and improving feature extraction capabilities. Finally, a feature fusion pyramid structure is adopted to better integrate and utilize features at different scales. In summary, after adopting this technical solution, the performance of target detection is comprehensively improved, especially in power operation environments, enabling more accurate target identification and localization, reducing the impact of background noise, and improving the model's adaptability to multi-scale targets. Attached Figure Description

[0051] Figure 1 This is a flowchart of the method for identifying violations in power operations according to an embodiment of the present invention;

[0052] Figure 2 This is a diagram showing the overall structure of the image enhancement network of the first and second processing models described in the embodiments of the present invention.

[0053] Figure 3 This is a network structure diagram of the first processing model described in this embodiment of the invention;

[0054] Figure 4This is the overall structure diagram of the improved YOLOv8 network model described in the embodiments of the present invention;

[0055] Figure 5 yes Figure 3 C2f-CBDC module structure diagram in network structural unit;

[0056] Figure 6 yes Figure 4 Internal module structure diagram of the CBDC attention module. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] This invention provides a method for identifying violations during high-altitude power operations. This method considers the construction of an operation dataset and the enhancement of low-quality images. It also proposes a violation identification network model based on an improved YOLOv8 to maximize detection accuracy. The method includes:

[0059] High-definition and low-quality images of high-altitude power operation sites are collected by drones. A high-definition dataset of power sites is created based on the high-definition and low-quality images. The high-definition dataset of power sites is obtained by annotating the high-definition images with targets to obtain the first tag file corresponding to the images.

[0060] The low-quality image is processed using a first processing model and a second processing model. The first processing method corresponding to the first processing model includes: introducing grouping branches into the first three convolutional layers of the dynamic contrast enhancement network structure; after processing the original low-quality image using the first processing method, a corresponding first feature map is finally obtained. The second processing method corresponding to the second processing model includes: firstly performing a convolution operation on the original low-quality image; then extracting image features through a multi-head attention module; and finally generating corresponding adjustment matrices and transformation values ​​through multilayer perception and linear transformation, respectively. The adjustment matrix is ​​used to correct the color of the image features, while the transformation value is used to adjust the illumination of the image features, thereby obtaining a corresponding second feature map. The first feature map and the second feature map are fused to obtain the final optimized image.

[0061] The optimized image is labeled with targets to obtain the corresponding second label file. The optimized dataset is obtained by combining the second label file with the corresponding image. The first processing model and the second processing model are trained using the optimized dataset until the final training result is achieved.

[0062] The high-definition power field dataset and the optimized dataset are combined to form the final dataset, which is then divided into a training set, a validation set, and a test set.

[0063] A traffic violation recognition model based on an improved YOLOv8 is constructed. The improved YOLOv8 traffic violation recognition model includes: replacing the backbone network in the YOLOv8 network structure with a Transformer model, replacing the C2f module in the YOLOv8 network structure with a C2f-CBDC module, and finally, the model outputs the result and label data corresponding to the traffic violation recognition.

[0064] The improved YOLOv8 violation recognition model is trained using the training set until convergence.

[0065] The trained model is evaluated using the test set, and its performance is judged based on the overall accuracy. When the accuracy stabilizes and improves, the final model is output; if the expected accuracy is not achieved, the model is retrained.

[0066] The optimal model after training and optimization is converted into an RKNN model for the front end and deployed to the front end of the drone;

[0067] Real-time detection is achieved by relying on front-end cameras. The detection results are then sent to the analysis unit to determine whether the relevant personnel have violated regulations, and the results are saved.

[0068] The dynamic contrast enhancement network structure includes 7 convolutional layers. In the first 3 convolutional layers, channel separation is performed to add group branches. In each group branch, 1×1 convolution is used to increase the dimensionality. The number of feature channels in the first 3 convolutional layers is halved and then concatenated with the corresponding group branch to restore the original number of channels. The first 6 convolutional layers use the ReLU activation function, and the 7th convolutional layer uses the Tanh activation function.

[0069] The dynamic contrast enhancement network structure further includes an illumination adjustment layer, which is used to adjust the illumination range of the image features output from the convolutional layer, and then output the first feature map. The illumination adjustment layer is based on an estimation formula representing the relationship between different features, and the estimation formula is expressed as:

[0070]

[0071] Where I represents the current image feature; a is a set threshold parameter, which is the same as the dimension of the input image feature. Based on this formula, all pixels in the image are normalized, and the graphic formed by the mapping relationship is used as the corresponding color channel to achieve the adjustment of the illumination range.

[0072] The second processing model includes: a convolution module, an encoding module, a first-layer normalization module, a multi-head attention module, a second-layer normalization module, a multi-layer perception module, and a linear transformation module. The original low-quality image is input into the convolution module, which includes a 3×3 convolutional layer to obtain a first feature map with 64 channels. After dimensionality transformation by the encoding module, the first feature map is sequentially input into the first-layer normalization module and the multi-head attention module. The module extracts the corresponding second feature. The second feature passes through the second normalization module and then through the multi-layer perception module to generate a 3×3 adjustment matrix. This matrix is ​​then linearly transformed with the second feature output by the multi-head attention module to obtain a 1×1 transformation value.

[0073] The process of fusing the first feature map and the second feature map to obtain the final optimized image specifically includes:

[0074] The second feature map is input into a convolutional layer with a size of 1×1 and 1 channel to obtain the third feature. The third feature is multiplied by the first feature to obtain the fourth feature. The fourth feature is then exponentially calculated with the third feature to output the final optimized image.

[0075] Furthermore, including:

[0076] The step of replacing the backbone network in the YOLOv8 network structure with a Transformer model includes: using the EfficientFormerV2 model in the Transformer model as the backbone network of the model.

[0077] The replacement of the C2f module in the YOLOv8 network structure with the C2f-CBDC module includes:

[0078] In the EfficientFormerV2 model, Block1, Block2, and Block3 are all connected to a convolutional layer in the Neck part of the YOLOv8 network model, and the Fast Spatial Pyramid Pooling (SPPF) layer is also connected to a convolutional layer in the Neck part of the YOLOv8 network model. The C2f module in the Neck part of the YOLOv8 network model is replaced with a C2f-CBDC module. The C2f-CBDC module is a C2f module with a CBDC attention module added. The CBDC attention module integrates the CBAM attention module and deformable convolution. The CBAM attention module combines channel attention and spatial attention to refine the feature map in two different dimensions.

[0079] The CBDC attention module integrates the CBAM attention module and deformable convolution, specifically including:

[0080] The Channel Attention Module (CAM) performs max pooling and average pooling operations on the input feature map to compress the information of each channel and generate two weight vectors. These two weight vectors are mapped through a shared network to obtain the weight matrix of each channel, and then multiplied element-wise with the original input feature map to enhance the feature response of important channels and suppress redundant or irrelevant channel information.

[0081] The Spatial Attention Module (SAM) focuses on spatial dimension features based on the output of the Channel Attention Module. In this module, max pooling and average pooling operations are applied to all channels at each location to generate two two-dimensional feature maps. Then, through deformable convolution operations, the channel information is fused, and the spatial attention matrix is ​​multiplied element-wise with the output feature map of the Channel Attention Module.

[0082] The step of converting the trained and optimized optimal model into a front-end RKNN model and deploying it to the front end of the drone includes:

[0083] The trained image augmentation and object detection models are converted into a model format suitable for the RKNN framework using Python scripts. This conversion process includes the reconstruction of model structure and parameters to ensure that the model can run efficiently in the RKNN engine.

[0084] After the conversion is completed, a series of optimization operations are performed on the resulting RKNN model, including adaptive hybrid quantization. This process aims to improve the model's execution efficiency on embedded platforms by reducing the model's computational and storage requirements, while maintaining its accuracy level as much as possible.

[0085] Through multiple experiments, the impact of different quantization methods on model performance was verified, and the detection accuracy of each category was analyzed in detail to ensure that the model can maintain a high recognition accuracy in different scenarios.

[0086] The optimized RKNN model is deployed to the front end of the drone. During deployment, the model is loaded onto the hardware platform on the drone and the inference process is accelerated by the integrated neural network processing unit (NPU).

[0087] The method of relying on a front-end camera to achieve real-time detection, sending the detection results to the analysis unit to determine whether the relevant workers have violated regulations, and saving the results, specifically includes:

[0088] The inspection drone began reading the real-time video stream and performing continuous frame-by-frame detection;

[0089] The drone detected the construction tower using the violation detection algorithm in the RKNN model and guided the drone to fly toward the tower.

[0090] After flying above the construction tower, the drone began to fly around the tower using preset latitude and longitude coordinates;

[0091] When workers are detected on the construction tower, the analysis unit judges whether the workers are working in accordance with regulations and whether the prescribed safety measures are in place within the worker's pixel frame based on the detection results.

[0092] Once a violation is identified, the detection and analysis results will be saved and sent back to the backend system, which will then conduct further analysis and processing of the data.

[0093] To further and more clearly describe the solution of this application, the following embodiments are provided:

[0094] like Figure 1 As shown, this invention proposes a method for identifying violations in power operations based on unmanned aerial vehicles (UAVs) and an improved YOLOv8 target detection algorithm. The specific implementation steps are as follows:

[0095] Step 1: Collect high-definition images of power operation sites using drones and create a high-definition dataset of power operation sites;

[0096] The process of creating a high-definition dataset of the power field in step 1 is as follows:

[0097] Step 1.1: Collect high-definition and low-quality images of the power operation site using drones;

[0098] Step 1.2: Use the LabelImg annotation tool to annotate the high-definition image data to obtain the label file corresponding to the image; finally, use a Python script to crop and classify the annotated data to create a violation dataset, and divide the dataset into training set, validation set and test set in a random form according to a fixed ratio.

[0099] Step 2: Construct an image processing network based on the first and second processing modules;

[0100] The overall architecture of the network is as follows: Figure 2 As shown, it mainly consists of two parts:

[0101] In the first processing module, this invention makes several key improvements to the original DCE-Net (Dynamic Contrast Enhancement Network) to further enhance the network's performance, computational efficiency, and adaptability to illumination adjustment. The main improvements are detailed below:

[0102] First, a grouping branch is introduced into the DCE-Net network structure. By separating the feature maps of the first three layers into channels, more features can be captured from the same batch of feature maps. This design facilitates more refined processing and attention to the input features. Each grouping branch contains three different types of convolutional layers: a 1×1 convolutional layer, a 3×3 dilated convolutional layer, and a 1×1 convolutional layer. The 1×1 convolutional layer acts as a dimensionality adjuster in the network, effectively maintaining the feature extraction efficiency of dilated convolution without increasing computational cost. The 3×3 dilated convolutional layer expands the receptive field, enabling the network to capture a wider range of contextual information and enhancing its ability to focus on detailed features.

[0103] Secondly, to expand the range of illumination adjustment and further reduce the number of model parameters and computation time, the dynamic contrast enhancement network structure also includes an illumination adjustment layer. This layer adjusts the illumination range of the image features output from the convolutional layer, thereby outputting the first feature map. The illumination adjustment layer is based on an estimation formula representing the relationship between different features. This estimation formula is expressed as:

[0104]

[0105] Where I represents the current image feature; a is a set threshold parameter, which has the same dimension as the input image feature. Based on this formula, all pixels in the image are normalized, and the graphic formed by the mapping relationship is used as the corresponding color channel to achieve the adjustment of the illumination range. This alternative method not only improves the computational efficiency of the model, but also allows for better control over the adjustment range of illumination mapping, thereby improving the quality and adaptability of detail processing.

[0106] In the second processing module, this invention employs a Transformer network for image enhancement. A multi-head attention module captures the relationship between individual pixels and surrounding pixels, obtaining rich overall feature information. This addresses the issues of over-enhancement in bright areas and insufficient enhancement in dark areas, improving the overall contrast of the image. Specifically, first, a 3×3 convolution is performed on the input image to obtain a feature map with 64 channels. Next, after dimensionality transformation by the encoding module, the feature map is input to the multi-head attention module, which extracts overall features. Subsequently, a 3×3 adjustment matrix and a 1×1 transform value are generated through a multilayer perceptron (MLP) and linear transformation. The adjustment matrix is ​​used for color correction of the RGB three color channels, while the transform value is used for non-linear adjustment of image illumination. Finally, the final image optimization is completed.

[0107] Specifically, such as Figure 3 As shown, the first processing network of this invention consists of 7 convolutional layers and introduces skip connections to preserve features extracted in stages. Based on this, the DCE-Net structure is improved by implementing channel separation operations in the first 3 convolutional layers, increasing the number of group branches. In each group branch, 1×1 convolutions are used for dimensionality increase, and dilated convolutions with different dilation rates are used to extract features, thereby ensuring the early nature of feature extraction and the continuity of feature information.

[0108] To avoid increasing network parameters and computational overhead, the number of feature channels in the first three layers is halved and then concatenated with the grouped branches to restore the original number of channels. In the first processing model network, the first six layers use the ReLU activation function, responsible for feature extraction and training acceleration. The last layer uses the Tanh activation function to ensure the reasonableness of the output results and meet the requirements of the image enhancement task.

[0109] The second processing module is as follows: Figure 2As shown: First, a 3×3 convolution is performed on the input image to obtain a feature map with 64 channels. Next, after dimensionality transformation by the encoding module, the feature map is input to a multi-head attention module, which extracts the overall features. Subsequently, a 3×3 color adjustment matrix and a 1×1 transform value are generated through a multilayer perceptron (MLP) and linear transformation. The adjustment matrix is ​​used for color correction of the RGB three color channels, while the transform value is used for non-linear adjustment of the overall image illumination.

[0110] Step 3: Build a traffic violation recognition model based on the improved YOLOv8;

[0111] Considering the high similarity between categories during UAV flight detection, this invention improves upon the YOLOv8 framework to enable the model to fully learn global and local information from images. First, EfficientFormerV2 is selected as the backbone network to enhance the algorithm's ability to capture global information. Second, the original C2f module is replaced with a C2f-CBDC module. This module integrates CBAM and deformable convolution, effectively reducing the model size while maintaining performance, dynamically adjusting the receptive field, focusing on local target features, significantly reducing background interference, and improving feature extraction capabilities. Finally, a feature fusion pyramid structure is adopted to better integrate and utilize features at different scales, thereby further improving the model's recognition accuracy.

[0112] Step 3, which improves the YOLOv8 network model, is as follows: Figure 3 As shown, the specific process is as follows:

[0113] Step 3.1: Replace the original backbone network with a Transformer model:

[0114] Because the Transformer model can capture the overall dependencies between pixels through its self-attention mechanism, it can efficiently acquire overall information. Therefore, this invention introduces the Transformer into the backbone network.

[0115] Because the Transformer model can capture the overall dependencies between pixels through its self-attention mechanism, it can efficiently acquire overall information.

[0116] Therefore, this invention introduces a Transformer into the backbone network. Specifically, as... Figure 4As shown, this invention selects EfficientFormerV2 as the backbone network of the model. Its structure includes Stem, Subsample, Feedforward Network (FFN), Attention Feedforward Network (AttenFFN), and Attention DownSample. The Stem, Subsample, and Attention DownSample modules are jointly responsible for downsampling the feature maps, reducing redundant information. The FFN consists of pointwise convolutions and 3×3 depthwise separable convolutions. The pointwise convolutions are mainly used for feature channel compression and expansion, while the depthwise separable convolutions focus on extracting detailed features. Finally, the model combines Multi-Head Self-Attention (MHSA) with the FFN to construct the AttenFFN module, further enhancing its ability to capture overall information. This design allows the model to simultaneously consider details and overall dependencies when processing images, thereby improving overall detection and recognition performance.

[0117] In EfficientFormerV2, a multi-head self-attention downsampling module combining detailed information is designed. This module achieves downsampling of detailed information through strided convolution, while downsampling of overall information is completed by MHSA. Within the MHSA framework, the query matrix is ​​used to determine the resolution of the output feature map. To this end, pooling layers and 1×1 convolutional layers are used to downsample the query matrix. Subsequently, the fusion and interaction between different information are achieved by combining the 1×1 convolutional layers through element-wise addition operations. The detailed calculation process is shown in Equation (2).

[0118] L GQ =BN(Conv 1×1 (Pool 3×3，stride＝2 (Q)+Conv 3×3，stride＝2 (Q)));

[0119] In the formula: L GQ The query matrix is ​​processed by two different downsampling strategies; BN is the batch normalization operation; Conv 1×1 For pointwise convolution operations; Pool 3×3，stride＝2 This represents an average pooling operation with a 3×3 pooling kernel and a stride of 2; Conv 3×3，stride＝2 This represents a convolution operation with a pooling kernel of 3×3 and a stride of 2.

[0120] Specifically, this invention selects EfficientFormerV2 as the backbone network of the model, and its structure is as follows: Figure 2The backbone is shown in the diagram. In Blocks 1-4, the FFN (Feedforward Neural Network) network is repeated 2, 2, 4, and 2 times respectively. In Blocks 3 and 4, the AttenFFN module is constructed by combining the Multi-Head Self-Attention (MHSA) mechanism with the FFN, and it is added after the FFN network to further improve the model's ability to capture overall information about the target.

[0121] Step 3.2: The original C2f module was replaced with a C2f-CBDC module.

[0122] like Figure 5 As shown, the resolution of the input image gradually decreases with the increase of the number of network layers, which may weaken the detection performance of the model. Furthermore, since C2f uses a fixed receptive field, the feature extraction of this fixed structure is not ideal for targets with large scale variations.

[0123] To address the aforementioned issues, this invention incorporates a CBDC attention module into the existing C2f module. The CBDC module combines the CBAM attention module and a deformable convolution module, dynamically increasing the model's receptive field while enhancing its focus on important feature channels. This allows the convolutional kernel to better adapt to the actual shape of the object. This improvement enables the module to focus more intently on the target region and effectively reduces background interference.

[0124] Specifically, in CBDC, the input feature map is first subjected to max pooling and average pooling operations by the channel attention module to compress the information of each channel, generating two weight vectors. These two weight vectors are mapped through a shared network to obtain the weight matrix for each channel, and then multiplied element-wise with the original input feature map, thereby enhancing the feature response of important channels and suppressing redundant or irrelevant channel information. Next, the spatial attention module focuses on spatial dimension features based on the output of the channel attention module. In this module, max pooling and average pooling operations are applied to all channels at each location, generating two two-dimensional feature maps. Then, through deformable convolution operations, the channel information is fused, and the spatial attention matrix is ​​multiplied element-wise with the output feature map of the channel attention module, further enhancing the model's ability to perceive key location information, thereby improving the model's localization accuracy of the target region.

[0125] In summary, by adding deformable convolution and CBAM dual attention mechanism to the feature enhancement module, the model's ability to track targets can be effectively improved, and the performance of target detection and recognition in complex environments can be enhanced.

[0126] Specifically, such as Figure 3As shown, a CBDC attention module is added to the original C2f module. This module combines the CBAM attention module and deformable convolution, which can effectively reduce the model size while ensuring model performance, dynamically adjust the receptive field, focus on the detailed features of the target, significantly reduce background interference, and improve feature extraction capabilities.

[0127] Specifically, in the CBDC module, firstly as follows: Figure 6 As shown in the left figure, the input feature map is subjected to max pooling and average pooling operations through the Channel Attention Module (CAM) to compress the information of each channel, generating two weight vectors. These two weight vectors are mapped through a shared network to obtain the weight matrix for each channel, and then multiplied element-wise with the original input feature map, thereby enhancing the feature responses of important channels and suppressing redundant or irrelevant channel information. Next, as... Figure 6 As shown in the right figure, the Spatial Attention Module (SAM) focuses on spatial features based on the output of the Channel Attention Module. In this module, max pooling and average pooling operations are applied to all channels at each location, generating two two-dimensional feature maps. Then, through deformable convolution, the channel information is fused, and the spatial attention matrix is ​​multiplied element-wise with the output feature map of the channel attention module, further enhancing the model's ability to perceive key location information, thereby improving the model's localization accuracy of the target region.

[0128] Step 3.3: Construct the feature fusion pyramid.

[0129] This invention proposes a feature fusion pyramid structure.

[0130] In YOLOv8, an FPN+PAN structure is used for feature fusion. Although the FPN+PAN structure achieves complementarity between semantic and localization information, there is still room for improvement. On the one hand, feature fusion increases computational cost, and feature maps lose some original feature information after multiple upsampling and downsampling. On the other hand, during feature fusion, invalid feature information is easily extracted, and some detailed information is ignored, leading to a decrease in target detection accuracy.

[0131] Specifically, such as Figure 3 As shown in the Neck structure, firstly, to fully utilize the low-level feature maps to enrich the detailed information of the features, a 1×1 convolution operation is introduced to adjust the number of channels in the feature maps, making the network more efficient in operations such as feature fusion and upsampling. Then, while fusing feature maps at different scales, an element-wise subtraction operation is performed to eliminate redundant background features and retain target information. For example, when the size of the feature image at layer m is described as h... m ×w m ×c mLet f be the feature information of the i-th layer. i ={x i x i+1 x i+2 ,…,x n The feature information of the (i+1)th layer should be f. i+1 ={x i+1 x i+2 x i+3 ,…,x n}

[0132] Typically, from f i to f i+1 During the forward propagation process, x i While some background information may be ignored, it could still contain detailed target information. During drone flight, certain frames may contain limited target details, making them easily overlooked by the model during forward propagation. Therefore, element-wise subtraction of features at different scales can eliminate redundant background features and preserve target information. In summary, because the module does not add extra computation, it is more lightweight than FPN+PAN, has fewer parameters, and is better suited to front-end requirements.

[0133] Step 4: Select a suitable optimizer and loss function based on Step 3;

[0134] The specific process is as follows: This invention compares the Adam optimizer and the SGD optimizer, and stochastic gradient descent (SGD)... c Gradient Descent (SGD), as a fundamental optimization algorithm, is widely used in training deep learning models. SGD updates model parameters based on the gradient of the current sample, gradually reducing the loss function. However, SGD has a relatively slow convergence speed, and the update process is susceptible to noise interference, potentially leading to convergence instability because the gradient direction is not always aligned with the optimal solution. In contrast, the Adam (Adaptive Moment Estimation) optimizer is an adaptive learning rate optimization algorithm that combines momentum and adaptive learning rate methods. It assigns different weights to different gradients, making the learning rate more stable during training and helping the model converge to the optimal solution faster and more smoothly. Therefore, this invention chooses the Adam optimizer for model training to achieve more efficient and stable convergence performance.

[0135] This invention uses MPDIoU (Maximum Possible Distance Intersection over Union) as the bounding box regression loss function for training the model. Compared to the classic IoU, MPDIoU, by introducing a metric of the maximum possible distance between targets, can more accurately assess the relative position and geometric relationship between targets, avoiding some of the limitations of IoU. Especially when dealing with small targets, closely packed targets, or complex scenes, MPDIoU provides a more refined and robust evaluation metric, helping to improve the accuracy and stability of target detection tasks.

[0136] The formula for MPDIoU is shown below:

[0137]

[0138] In the formula: superscripts a and b represent the annotation box and the prediction box, respectively; subscripts 1 and 2 represent the upper right corner and lower left corner of the corresponding box, respectively; in formula (3) This represents the normalization operation on the diagonal to facilitate a comparison of the diagonal distance with IoU on the same scale; c1 is used to adjust the weight of the diagonal distance to better measure the impact of position offset on IoU.

[0139] Step 5: Train and optimize the model built in Steps 2 and 3;

[0140] The specific process is as follows:

[0141] Step 5.1: Set training parameters, including maximum number of iterations, learning rate, optimizer momentum, and weight initialization;

[0142] Step 5.2: Use the low-quality images from the high-resolution images acquired in Step 1.1 as input to the image enhancement module and train it to generate higher-order curves. Then, use these higher-order curves to adjust the dynamic range of the input image at the pixel level, thereby obtaining the enhanced image.

[0143] Step 5.3: Train the improved YOLOv8 network using a high-resolution dataset. After each training round, adjust the weights according to the difficulty of the category until training is complete and the optimal detection model is obtained.

[0144] Step 5.4: Evaluate the trained model using the test set and judge its performance based on the overall accuracy. When the accuracy stabilizes and improves, output the final model; if the expected accuracy is not achieved, retrain.

[0145] Step 6: Convert the optimal model trained and optimized in Step 5 into an RKNN model for the front end and deploy it to the drone front end.

[0146] The specific process is as follows:

[0147] Step 6.1: Use a Python script to convert the trained image augmentation and object detection models (usually in .pt format) into a model format suitable for the RKNN framework. This conversion process includes refactoring the model structure and parameters to ensure that the model can run efficiently in the RKNN engine.

[0148] Step 6.2: After the conversion, a series of optimization operations are performed on the resulting RKNN model, including adaptive hybrid quantization. This process aims to improve the model's execution efficiency on embedded platforms by reducing its computational and storage requirements, while maintaining its accuracy as much as possible.

[0149] Step 6.3: Through multiple experiments, verify the impact of different quantization methods on model performance, and conduct a detailed analysis of the detection accuracy of each category to ensure that the model can maintain a high recognition accuracy in different scenarios.

[0150] Step 6.4: Deploy the optimized RKNN model to the drone's front end. During deployment, the model is loaded onto the drone's hardware platform, and the inference process is accelerated by the integrated NPU (Neural Processing Unit).

[0151] Step 7: Real-time detection is achieved using the front-end camera. The detection results are then sent to the analysis unit to determine whether the workers have violated regulations. The results are saved and sent back to the backend.

[0152] The specific process is as follows:

[0153] The camera mounted on the front of the drone is responsible for acquiring real-time video streams of the surrounding environment, which are then input into an image enhancement model for processing. The enhanced images are then passed to a target detection model for continuous real-time detection. During this process, the model rapidly processes each frame of the image and transmits it to the judgment unit. Based on the model's output, the judgment unit determines whether the personnel working on the tower are correctly equipped with safety measures (such as safety helmets) within their pixel bounding boxes, thereby assessing whether the personnel have engaged in any violations and outputting the judgment results in real time.

[0154] Step 7.1: The inspectors launch the inspection drone, which begins to read the real-time video stream and performs continuous frame-by-frame detection.

[0155] Step 7.2: The drone detects the construction pole using a violation detection algorithm and is guided to fly towards the pole;

[0156] Step 7.3: After flying above the construction tower, the drone begins to fly around the tower using the preset latitude and longitude.

[0157] Step 7.4: After a worker is detected on the construction tower, the analysis unit judges whether the worker is working in accordance with regulations and whether the prescribed safety measures are in place within the worker's pixel frame based on the detection results;

[0158] Step 7.5: Once a violation is identified, the detection and analysis results are saved and transmitted back to the backend system. The backend system further analyzes and processes this data, such as generating statistical reports, recording the time and location of the violation, and providing support for subsequent decision-making. This entire process achieves a closed loop from data collection and model inference to result feedback, enabling drones to perform violation detection tasks efficiently and accurately.

[0159] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0160] Obviously, those skilled in the art can make various modifications and variations to the embodiments of the present invention without departing from the spirit and scope of the embodiments of the present invention. Thus, if these modifications and variations to the embodiments of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention also intends to include these modifications and variations.

Claims

1. A method for identifying violations in high-altitude electrical work, characterized in that, The method comprises: Collecting high-definition images and low-quality images of high-altitude power operation sites by a drone, and making a power site high-definition dataset according to the high-definition images and the low-quality images, wherein the power site high-definition dataset is obtained by target labeling of the high-definition images to obtain a first label file corresponding to the images; Performing image processing on the low-quality images by using a first processing model and a second processing model, wherein the first processing method corresponding to the first processing model comprises introducing a grouping branch in the first three convolution layers in a dynamic contrast enhancement network structure, and finally obtaining a corresponding first feature map by processing the original low-quality image by using the first processing method; the second processing method corresponding to the second processing model comprises: firstly performing convolution operation on the original low-quality image, then extracting image features by a multi-head attention module, and finally generating a corresponding adjustment matrix and a transformation value by a multi-layer perception and a linear transformation, respectively, wherein the adjustment matrix is used for color correction of the image features, and the transformation value is used for adjustment of the image feature illumination, thereby obtaining a corresponding second feature map; the first feature map and the second feature map are fused to obtain a final optimized image; The second processing model comprises a convolution module, an encoding module, a first layer normalization module, a multi-head attention module, a second layer normalization module, a multi-layer perception module and a linear transformation module; the original low-quality image is input into the convolution module, the convolution module comprises a 3*3 convolution layer, and a first feature with a channel number of 64 is obtained; after dimension conversion by the encoding module, the first feature is sequentially input into the first layer normalization module and the multi-head attention module, and a corresponding second feature is extracted by the module; the second feature is input into the multi-layer perception module through the second normalization module, a 3*3 adjustment matrix is generated, and a 1*1 transformation value is obtained by linear transformation of the second feature output by the multi-head attention module; Target labeling is performed on the optimized image to obtain a corresponding second label file, an optimized dataset is obtained according to the second label file and the corresponding image, the first processing model and the second processing model are trained by using the optimized dataset until the final training result is achieved; The power site high-definition dataset and the optimized dataset are combined to form a final dataset, and the final dataset is divided into a training set, a validation set and a test set in form; An improved YOLOv8-based violation identification model is constructed, wherein the improved YOLOv8-based violation identification model comprises: replacing a backbone network in a YOLOv8 network structure by using a Transformer model, replacing a C2f module in the YOLOv8 network structure by using a C2f-CBDC module, and finally outputting a result corresponding to violation identification and label data by the model; The backbone network in the YOLOv8 network structure is replaced by a Transformer model, including: an EfficientFormerV2 model in the Transformer model is used as the backbone network of the model; Block1, Block2 and Block3 in the EfficientFormerV2 model are connected with a convolution layer of the Neck part of the YOLOv8 network model, and a fast spatial pyramid pooling layer SPPF is connected with a convolution layer of the Neck part of the YOLOv8 network model, a C2f module of the Neck part of the YOLOv8 network model is replaced by a C2f-CBDC module, the CBDC attention module is added to the C2f module, the CBAM attention module and the deformable convolution are fused into the CBDC attention module, and the CBAM attention module combines channel attention and spatial attention to refine the feature map in two different dimensions; The improved YOLOv8 violation identification model is trained by using the training set until convergence is achieved; The trained model is evaluated by using the test set, and the effect is determined according to the overall precision; when the precision is stable and improved, the final model is output; if the expected precision is not reached, retraining is performed; The optimal model after training and optimization is converted into an RKNN model of the front end and deployed to the front end of the unmanned aerial vehicle; Real-time detection is realized by relying on the camera of the front end, the detection result is handed over to the research and judgment unit to judge whether the relevant personnel are in violation, and the result is saved.

2. The high-altitude power work violation recognition method of claim 1, wherein The dynamic contrast enhancement network structure includes 7 convolution layers, and channel separation operations are implemented in the first 3 convolution layers, a grouping branch is added, and in each grouping branch, 1*1 convolution is used for dimension increasing, and the feature channel number of the first 3 convolution layers is halved, and after splicing with the corresponding grouping branch, it is restored to the original channel number; and the first 6 convolution layers use the ReLU activation function, and the 7th convolution layer uses the Tanh activation function.

3. The high-altitude power work violation identification method of claim 2, wherein The dynamic contrast enhancement network structure further comprises an illumination adjustment layer, which is configured to adjust the image features output from the convolution layer in an illumination range, and output the first feature map. The illumination adjustment layer is based on an estimation formula representing the relationship between different features, and the estimation formula is expressed as: wherein, I represents the current image feature; is a set threshold parameter, which has the same dimension size as the input image feature. On the basis of the estimation formula, all pixels in the image are normalized, and the graph formed by the mapping relationship is taken as a corresponding color channel to realize the range adjustment of the illumination.

4. The high-altitude power work violation identification method of claim 3, wherein The first feature map and the second feature map are fused to obtain a final optimized image, specifically including: The second feature map is input into a convolution layer with a size of 1*1 and a channel number of 1 to obtain a third feature, the third feature is multiplied by the first feature map to obtain a fourth feature, and the fourth feature is exponentially calculated with the third feature to output the final optimized image.

5. The aerial power work violation identification method of claim 4, wherein, The CBDC attention module fuses the CBAM attention module and the deformable convolution, specifically including: Through the channel attention module CAM, maximum pooling and average pooling operations are performed on the input feature map to compress the information of each channel, generate two weight vectors, map the two weight vectors through a shared network to obtain a weight matrix of each channel, and multiply the weight matrix with the original input feature map element by element, thereby enhancing the feature response of important channels and suppressing redundant or irrelevant channel information. The spatial attention module (SAM) focuses on the features in the spatial dimension based on the output of the channel attention module. In this module, max-pooling and average-pooling operations are performed on all channels of each position to generate two two-dimensional feature maps. Then, the channel information is fused through a deformable convolution operation. The spatial attention matrix is multiplied element-wise with the output feature map of the channel attention module.

6. The aerial power work violation identification method of claim 1, wherein The optimal model obtained by completing the training optimization is converted into an RKNN model for the front end and deployed to the front end of the unmanned aerial vehicle, including: The image enhancement model and the target detection model obtained by training are converted into a model format suitable for the RKNN framework using a Python script. The conversion process includes reconstruction of the model structure and parameters to ensure that the model can run efficiently in the RKNN engine. After the conversion is completed, a series of optimization operations are performed on the RKNN model obtained by the conversion, including adaptive hybrid quantization. This process aims to improve the execution efficiency of the model on the embedded platform by reducing the computational load and storage occupancy of the model, while trying to maintain its accuracy level. Through multiple experiments, the influence of different quantization methods on the performance of the model is verified, and the detection accuracy of each category is analyzed in detail to ensure that the model can still maintain a high recognition accuracy in different scenarios. The optimized RKNN model is deployed to the front end of the unmanned aerial vehicle, and during the deployment process, the model is loaded onto the hardware platform on the unmanned aerial vehicle and the integrated neural network processing unit (NPU) is used to accelerate the inference process.

7. The aerial power work violation identification method of claim 1, wherein The camera on the front end is used to realize real-time detection, and the detection results are submitted to the research and judgment unit to determine whether the relevant workers have violated the regulations. The results are saved, including: The inspection unmanned aerial vehicle starts reading real-time video streams and performs uninterrupted frame extraction and detection. The unmanned aerial vehicle detects the construction tower using the violation recognition algorithm in the RKNN model and guides the unmanned aerial vehicle to fly towards the tower. After flying over the construction tower, the unmanned aerial vehicle starts flying around the tower based on the pre-set latitude and longitude. When detecting workers on the construction tower, the research and judgment unit determines whether the workers are working according to the regulations and whether the workers are equipped with the required safety measures in the pixel box. Once a violation is identified, the detection results and the judgment results are saved and data is returned to the background system. The background system further analyzes and processes these data.

Images