Night electric worker protection equipment detection method based on deep cascade network architecture

Abstract

The invention discloses a night electric worker protection equipment detection method based on a deep cascade network architecture. The method comprises the following steps: constructing an improved YOLO v8 model; carrying out the preprocessing of a low-illumination and high-noise image through the input part of the improved YOLO v8 model, and generating an optimized image tensor; performing feature extraction on the optimized image tensor through a backbone network of an improved YOLO v8 model to obtain feature maps of different scales; capturing a content relationship between image features of the large-scale feature map through a Transform module in the backbone network; transmitting the feature map to a detection head of an improved YOLO v8 model for detection; and when the improved YOLO v8 model is trained, a sliding loss function is introduced, and a detection result with the annotation information of the improved YOLO v8 model is output. According to the invention, the performance and accuracy of target detection under the low-light condition are significantly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and image processing, and particularly relates to a method for detecting night power worker protection equipment based on a deep cascade network architecture. Background Art

[0002] As the guardians of the power system, power workers work in limited lighting conditions and changing working environments, which pose high risks. Therefore, identifying and monitoring power workers and their personal protective equipment in complex environments is a challenging task. Incomplete use of safety equipment cannot guarantee the safety of workers. However, due to insufficient safety awareness, power workers often fail to correctly use safety equipment. For example, they only wear safety helmets but do not wear insulating shoes and insulating gloves. The night scene poses challenges in identifying the correct use of safety equipment. Detecting the safety equipment of power workers in complex environments is an effective tool to ensure their operation safety.

[0003] Existing general detection networks for power worker protection equipment detection can be roughly divided into two categories: those based on traditional machine learning and those based on deep learning. For methods based on traditional machine learning, shape features (such as geometric shape descriptors, moment features) and texture features (such as gray-level co-occurrence matrices, local binary patterns) are manually extracted, and then decision trees, naive Bayes classifiers, etc. are used for classification. Then, the feature extraction of this type of method depends on artificial design, requires prior knowledge and experience, lacks generality, often needs to design different feature extraction methods for different equipment, and it is difficult to comprehensively and accurately describe the equipment features in the face of complex and changing actual scenarios; it has poor robustness to complex backgrounds and changes in equipment appearance, posture, and lighting, is easily interfered by background noise and irrelevant objects, and the detection accuracy drops significantly when conditions change; the computational efficiency is low, and when dealing with large-scale data sets, the feature calculation and classifier training are time-consuming and require a large amount of memory.

[0004] To overcome the limitations of traditional machine learning in complex environments, researchers have proposed deep learning methods. These methods are more diverse. In the R-CNN series, R-CNN first generates candidate regions through selective search, then extracts features using a CNN and fine-tunes them with an SVM classifier and regressor; Fast R-CNN shares feature maps and optimizes through RoIPooling; Faster R-CNN introduces an RPN to generate candidate regions and uses a multi-task loss function. In single-stage object detection algorithms, YOLO divides the image into grids for prediction and continuously optimizes the architecture; SSD detects on multi-scale feature maps and designs a specific loss function; RetinaNet uses focal loss to address sample imbalance and combines with FPN to enhance detection capabilities. For methods based on Transformer, there are pure Transformer architectures that directly process image patches, and there are also combinations of Transformer and CNN, such as integrating Transformer modules into the CNN backbone to improve the detection performance of protective equipment. However, deep learning methods also have some problems. First, they rely heavily on large amounts of data. When the data is insufficient, the model is prone to overfitting, with poor generalization ability, and the detection accuracy will be greatly reduced in other complex scenarios. Second, changes in lighting conditions have a significant impact on the model performance. Low light can cause image feature distortion, making it difficult for the model to extract key features. Pre-trained models are usually based on normal lighting images and cannot be effectively transferred in low-light scenarios, resulting in a significant drop in detection accuracy and frequent false detections and missed detections.

[0005] On the other hand, with the continuous development of deep learning and computer vision technologies, image preprocessing (such as 3D filtering technology, context awareness technology, etc.) has become an important research direction in night complex environment detection technologies. Image preprocessing simplifies data through grayscale conversion, removes noise interference through denoising, enhances contrast to improve image clarity, and normalizes the size to unify the image format, laying a foundation for subsequent detection.

[0006] Therefore, although existing methods have made certain progress in image processing, they still face the problem that denoising is prone to blurring the key details of protective equipment, affecting feature extraction and detection accuracy. Summary of the Invention

[0007] Most current object detection algorithms are designed under normal lighting conditions, resulting in problems such as low detection accuracy, false detections, and leaks in the model under low-light conditions. To overcome the above technical deficiencies, the present invention provides a method for detecting the protective equipment of night-time power workers based on a deep cascade network architecture.

[0008] Term Explanation:

[0009] 1. CNN: Convolutional Neural Network, convolutional neural network.

[0010] 2. R-CNN: Regions with CNN features, region detection based on convolutional neural network.

[0011] 3. SVM: Support Vector Machine, support vector machine.

[0012] 4. Faster R-CNN: Faster Regions with CNN features, fast region detection based on convolutional neural network.

[0013] 5. RoIPooling: Region of Interest Pooling, region of interest pooling.

[0014] 6. YOLO: You Only Look Once, real-time object detection algorithm.

[0015] 7. SSD: Single Shot Multibox Detector, single-stage multibox detector.

[0016] 8. FPN: Feature Pyramid Network, feature pyramid network.

[0017] 9. RetinaNet: A single-stage model for object detection and recognition tasks.

[0018] 10. CenterNet: A single-stage object detection algorithm.

[0019] 11. RTDETR: Real-Time Detection Transformer, an object detection algorithm based on Transformer.

[0020] 12. Transformer: A neural network architecture designed for natural language processing tasks.

[0021] 13. IoU: Intersection over Union, intersection over union.

[0022] 14. MAP: Mean Average Precision, mean average precision.

[0023] The technical solution adopted by the present invention to overcome its technical problems is:

[0024] A method for detecting night power worker protection equipment based on a deep cascaded network architecture, comprising the following steps:

[0025] S1. Construct an improved YOLO v8 model:

[0026] The YOLO v8 model at least includes an input part, a backbone network, a neck network, and a detection head. An improved YOLO v8 model is obtained by at least embedding a Transformer module in the backbone network, adjusting the feature pyramid network structure in the backbone network, and modifying the detection head. The input part is used for image preprocessing and transmitting the preprocessed image to the backbone network. The backbone network is used for feature extraction and unidirectionally inputting the extracted features to the neck network. The neck network is used for multi-scale fusion of the features input by the backbone network and fully connecting the multi-scale fused features to the detection head. The detection head is used for target localization and class prediction to generate prediction results.

[0027] S2. Preprocess the low-light and high-noise images through the input part of the improved YOLO v8 model, extract the illumination features and noise features in the images, and generate an optimized image tensor.

[0028] S3. Extract features from the optimized image tensor in step S2 through the backbone network of the improved YOLO v8 model to obtain feature maps of different scales of height and width, including small-scale feature maps, medium-scale feature maps, and large-scale feature maps.

[0029] S4. Capture the content relationship between the image features of the large-scale feature map through the Transformer module in the backbone network of the improved YOLO v8 model.

[0030] S5. Transmit the feature map processed in step S4 to the detection head of the improved YOLO v8 model for detection.

[0031] S6. When training the improved YOLO v8 model, introduce a sliding loss function to adjust the dynamic loss weights of samples with different levels of difficulty, and finally output the detection results with the annotation information of the improved YOLO v8 model.

[0032] Further, in step S1, an improved YOLO v8 model is obtained by at least embedding a Transformer module in the backbone network, adjusting the feature pyramid network structure in the backbone network, and modifying the detection head, which specifically includes:

[0033] (1) In the backbone network, embed a Transformer module for processing large-scale feature maps. The Transformer module uses the multi-head attention mechanism and the multi-layer perceptron to strengthen the capture of global image information.

[0034] (2) By adjusting the connection method and weight distribution of the Feature Pyramid Network structure, the Feature Pyramid Network performs information interaction and complementation on feature maps of different scales;

[0035] (3) Modify the detection head, including at least optimizing the regression algorithm and anchor mechanism, and adding trajectory smoothing and motion compensation techniques, so as to combine the enhanced features of the Transformer module to achieve accurate object localization and classification.

[0036] Furthermore, adjusting the connection method and weight distribution of the Feature Pyramid Network structure specifically includes the following:

[0037] The adjustment of the connection method includes expanding cross-layer connections and introducing a dynamic connection mechanism. Among them, expanding cross-layer connections is configured to establish direct connections between small-scale feature maps and large-scale feature maps, between large-scale feature maps and small-scale feature maps, and between non-adjacent layers across multiple levels to enhance the model's perception ability of objects of different sizes. The dynamic connection mechanism is configured to use the lightweight meta-network in the Feature Pyramid Network to output a connection matrix according to the input image content, adaptively adjust the feature map connections, and strengthen the connections of corresponding scale feature maps according to the characteristics of the image object size.

[0038] The adjustment of weight distribution includes adaptive weight learning, task-oriented weight distribution, and weight distribution based on feature differences. Among them, adaptive weight learning is configured to automatically learn fusion weights through the adaptive weight module according to the feature distribution and information richness of the feature map. Task-oriented weight distribution is configured to optimize the weights during model training with the help of a sliding loss function according to the different attentions of different detection tasks to the target scale, so that the model adapts to the preset tasks. Weight distribution based on feature differences is configured to increase the fusion weights of feature maps exceeding the preset feature difference value and reduce the weights of feature maps not exceeding the preset feature difference value by calculating the statistics of the feature maps.

[0039] Furthermore, in step S2, the input part of the YOLO v8 model includes an illumination estimation module and a self-correction module. The illumination estimation module estimates the illumination of low-light and high-noise images, and then the self-correction module correlates the images processed by the illumination estimation module to obtain an optimized image tensor.

[0040] Furthermore, step S2 specifically includes the following steps:

[0041] S21. Use the illumination estimation module to adopt a progressive learning method, learn the residual representation between the low-light observation image and the ideal illumination image through the parameter θ, the structure H, and the weight sharing mechanism, and deduce the module to be obtained after optimizing the illumination estimation module from the training loss and data, where the mapping H with the parameter θ is introducedθ To learn about illumination, the illumination estimation formula is as follows:

[0042] {F(x) = x t+1 = x t + u t |u t = H θ (x t ), x 0 = y} (1)

[0043] In formula (1), x represents the illumination component, x 0 and y represent the initial low-illumination image, u t and x t respectively represent the residual term and the illumination component at the t-th stage, x t+1 represents the illumination component of the next stage of the t-th stage, that is, the illumination component of the (t + 1)-th stage, where t = 0, 1, …, T - 1;

[0044] S22. The image processed by the illumination estimation module in step S21 forms a new image tensor x t , which is used as the input of the self-correction module. The self-correction module uses the output result of the previous stage as the input x t-1 at the (t - 1)-th stage and associates it with the new image tensor x t . The result after association is used as the input to affect the illumination estimation output x t+1 at the next stage. Finally, at the (T - 1)-th stage, the original image tensor is fused with the output feature map to generate the final result G(x). The self-correction association formula is:

[0045]

[0046] In formula (2), the value of t is t ≥ 1, K θ is a parameter θ that can be learned by the introduced parameterized operator; the image is composed of multiple pixel points, and each pixel point has a corresponding value. The operation represents the division calculation of the corresponding pixel values in y and x t . The basic unit conversion at the t-th stage is expressed as:

[0047] F(x t ) = F(G(x t )) (3).

[0048] Furthermore, step S3 specifically includes the following steps:

[0049] The improved YOLO v8 model extracts features from the image tensor optimized in step S2 through a Feature Pyramid Network (FPN) to obtain feature maps of different scales. Feature maps of different scales correspond to objects of different sizes. In the backbone network of the improved YOLO v8 model, the image tensor is divided into three-scale feature maps P3, P4, and P5. Among them, P3, P4, and P5 represent small-scale feature maps, medium-scale feature maps, and large-scale feature maps respectively. P3, P4, and P5 are all four-dimensional tensors, and the four dimensions are batch size, channels, height, and width.

[0050] Furthermore, step S4 specifically includes the following steps:

[0051] S41. Preprocessing of input features: First, the feature map P5 processed by the backbone network of the improved YOLO v8 model is unfolded and reshaped into a two-dimensional sequence form, and spatial position information is injected through positional encoding. Then, the features are decomposed into three parts Q, K, and V through a linear transformation and divided into multiple heads, where Q, K, and V represent query vectors, key vectors, and value vectors respectively.

[0052] S42. Multi-head attention mechanism processing: Calculate the attention weights of each head obtained in step S41 and merge the attention weights of all heads, specifically as follows:

[0053]

[0054] In formula (4), Attention(Q, K, V) is used to calculate the output of the attention mechanism. By operating on the input query vector Q, key vector K, and value vector V, the importance of different input features is determined, and a weighted output representation is generated. K T represents the transpose matrix of the key vector K, and D K represents the dimension of the key vector.

[0055] The calculation formula for the attention weight of each head is:

[0056] Head i = Attention(QW i q , KW i k , VW i v ) (5)

[0057] In formula (5), QW i q , QW i q , QW i q represent the weight matrices of the i-th head respectively.

[0058] Then, the attention results of all heads are combined and fused through a linear transformation to form the output of the multi-head attention mechanism. The splicing formula is as follows:

[0059] MultiHead(Q, K, V) = CONCAT(Head1, …, Head n )W0 (6)

[0060] In formula (6), W0 represents the linear transformation matrix of the output, and CONCAT represents the combination of the attention of all heads;

[0061] S43. Non-linear processing of the multi-layer perceptron: Non-linear transformation and feature extraction are performed on the information processed by the multi-head attention mechanism in step S42. The multi-layer perceptron in the Transformer module includes two fully connected layers, a non-linear activation function inserted between the two fully connected layers, and an output layer; the multi-layer perceptron first performs a weighted combination and offset adjustment of the features of the input data through a linear transformation W1q + b1, then applies the non-linear activation function f to non-linearly map the result of the linear transformation so that the model can learn complex patterns and features in the data, then performs the second linear transformation W2, and finally adds the bias b2 and obtains the final output through the activation function g of the output layer. The formula is as follows:

[0062] MLP(q) = g(W2f(W1q + b1) + b2) (7)

[0063] In formula (7), MLP represents the multi-layer perceptron, W1 and W2 are weight matrices, which learn the feature relationship of the data by weighting each dimension of the input vector; b1 and b2 are bias vectors, which add fixed values to the output and adjust the output range.

[0064] Furthermore, step S5 specifically includes the following steps:

[0065] S51. Fuse the feature map P5 enhanced by the Transformer module with the feature maps P3 and P4 not enhanced by the Transformer module at three scales to complete the multi-scale fusion of features;

[0066] S52. The feature map processed in step S51 is passed to the detection head of the improved YOLO v8 model. The detection head is used for object localization and class prediction. Specifically, the detection head predicts the bounding box of the object through a regression algorithm and uses the anchor mechanism to predict the offset of the bounding box relative to the anchor point to achieve precise localization of the object. At the same time, the detection head performs class prediction on each bounding box and then outputs the probability that the object belongs to each class, and normalizes the class probabilities through the Softmax function. The formula of the Softmax function is as follows:

[0067]

[0068] In formula (8), m represents the total number of categories of protective equipment, j represents the protective equipment of the jth category, and s j is the original score of the jth category of protective equipment, and e is the base of the natural logarithm function. Formula (8) means dividing s j by the total score of all categories of protective equipment to obtain the probability ratio of the protective equipment of the jth category.

[0069] Further, after step S52, there is also step S53, and the step S53 includes the following:

[0070] In the detection stage, a trajectory smoothing method and a motion compensation method are adopted. Among them, the trajectory smoothing method smooths the motion trajectory of the object through time series analysis, and the time series analysis adopts at least any one of weighted average or Kalman filtering; the motion compensation method calculates the difference between the trajectory smoothed by the trajectory smoothing method and the original trajectory, and performs geometric transformation on the image frame, and the geometric transformation includes at least translation and rotation.

[0071] Further, step S6 specifically includes the following steps:

[0072] S61. Sample classification based on intersection over union: The sliding loss function determines the difficulty of the sample by calculating the intersection over union between the predicted bounding box and the ground truth bounding box. Let the intersection over union be IoU: When the value of IoU is lower than the preset threshold, the sample is difficult to classify; otherwise, the sample is easy to classify.

[0073] S62. Dynamic adjustment of loss weights:

[0074] The sliding loss function dynamically adjusts the loss weights of samples with different difficulties, so that the model pays attention to difficult-to-classify samples during the training process, specifically as follows:

[0075]

[0076] In formula (9), z represents the value of IoU between the predicted bounding box and the ground truth bounding box, and p represents the average value of IoU, which is used as the preset threshold.

[0077] The beneficial effects of the present invention are as follows:

[0078] Firstly, through a carefully designed cascaded illumination learning process and a clever weight sharing mechanism, the present invention deeply optimizes low-light and high-noise images. Then, a Transformer module is embedded in the backbone network of the YOLO v8 model to process large-scale feature maps, enhancing the model's multi-scale feature fusion ability and fully understanding image feature information through the self-attention mechanism. Furthermore, by adjusting the connection method and weight distribution of the feature pyramid network structure, the feature pyramid network conducts information interaction and complementarity for feature maps of different scales. The detection head is also modified, including at least optimizing the regression algorithm and the anchor mechanism and adding trajectory smoothing and motion compensation techniques, thereby combining the enhanced features of the Transformer module to achieve accurate object localization and classification. Finally, by introducing a sliding loss function to enhance the importance of boundary samples, the imbalance problem between samples of different difficulties is solved. The present invention significantly improves the performance of object detection under low-light conditions, ensures the correct detection and recognition of target objects in images, and thus can effectively identify the protective equipment of electric power workers in complex environments, improving the overall detection effect and playing a role in warning the safety of electric power workers. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 It is a flowchart of the method for detecting the protective equipment of electric power workers at night based on the deep cascaded network architecture described in the embodiment of the present invention.

[0080] Figure 2 It is a schematic diagram of the image preprocessing structure described in the embodiment of the present invention.

[0081] Figure 3 It is a schematic diagram of the large-scale feature extraction structure described in the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0082] To facilitate better understanding of the present invention by those skilled in the art, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The following is merely exemplary and does not limit the protection scope of the present invention.

[0083] The present invention discloses a method for detecting the protective equipment of electric power workers at night based on a deep cascaded network architecture, including the following steps:

[0084] S1. Construct an improved YOLO v8 model;

[0085] S2. Preprocess low-light and high-noise images through the input part of the improved YOLO v8 model, extract the illumination features and noise features in the images, and generate an optimized image tensor;

[0086] S3. Feature extract the optimized image tensor in step S2 through the backbone network of the improved YOLO v8 model to obtain feature maps with different scales of height and width, including small-scale feature maps, medium-scale feature maps, and large-scale feature maps;

[0087] S4. Capture the content relationship between the image features of the large-scale feature map through the Transformer module in the backbone network of the improved YOLO v8 model;

[0088] S5. Transmit the feature map processed in step S4 to the detection head of the improved YOLO v8 model for detection;

[0089] S6. When training the improved YOLO v8 model, introduce a sliding loss function to adjust the dynamic loss weights of samples with different levels of difficulty, and finally output the detection results with the annotation information of the improved YOLO v8 model.

[0090] To better understand the above technical solution, the exemplary embodiments of the present invention will be described in detail below with reference to the accompanying drawings. These are only the exemplary embodiments of the present invention. However, it should be understood that the present invention can be implemented in various forms and is not limited to the embodiments described here. These embodiments are to make those skilled in the art understand the present invention more clearly and thoroughly.

[0091] A method for detecting night electrician protection equipment based on a deep cascade network architecture described in this embodiment, as Figure 1 shown, includes the following steps:

[0092] S1. Construct an improved YOLO v8 model.

[0093] The YOLO v8 model at least includes an input part, a backbone network, a neck network, and a detection head.

[0094] This embodiment improves the YOLO v8 model. The improvements at least include: embedding a Transformer module in the backbone network, adjusting the Feature Pyramid Network (FPN) structure in the backbone network to meet the processing requirements of different-scale feature maps, and modifying the detection head to better combine the features enhanced by the Transformer module for detection. The improved YOLO v8 model is obtained through the improvement and optimization of the above several parts.

[0095] In the improved YOLO v8 model, the input part is used for image preprocessing and transferring the preprocessed image to the backbone network. The backbone network is used for feature extraction and unidirectionally inputting the extracted features into the neck network. The neck network is used for multi-scale fusion of the features input by the backbone network and fully connecting the multi-scale fused features to the detection head. The detection head is used for object localization and class prediction to generate prediction results.

[0096] In step S1 of this embodiment, at least by embedding a Transformer module in the backbone network, adjusting the feature pyramid network structure in the backbone network, and modifying the detection head, an improved YOLO v8 model is obtained, which specifically includes:

[0097] (1) In the backbone network, a Transformer module is embedded to process large-scale feature maps. The Transformer module uses the multi-head attention mechanism and the multi-layer perceptron to enhance the capture of global image information.

[0098] (2) By adjusting the connection method and weight assignment of the feature pyramid network structure, the feature pyramid network performs information interaction and complementarity on feature maps of different scales.

[0099] Specifically, in this embodiment, the adjustment of the connection method includes expanding cross-layer connections and introducing a dynamic connection mechanism.

[0100] First, expand cross-layer connections, breaking the limitations of traditional adjacent-layer connections. It is configured to establish direct connections between small-scale feature maps and large-scale feature maps, between large-scale feature maps and small-scale feature maps, and between non-adjacent layers across multiple levels, so as to promote the circulation of detailed information and semantic information and enhance the model's perception ability of targets of different sizes. Among them, the direct connection between small-scale feature maps and large-scale feature maps is specifically that in the feature pyramid network, the small-scale feature map (such as the high-level feature map) is directly connected to the large-scale feature map (such as the low-level feature map). For example, the small-scale feature map is enlarged to the same size as the large-scale feature map through an upsampling operation, and then feature fusion is performed; the direct connection between large-scale feature maps and small-scale feature maps is specifically that the large-scale feature map can also be reduced to the same size as the small-scale feature map through a downsampling operation, and then feature fusion is performed; the direct connection between non-adjacent layers across multiple levels is specifically that in addition to the connections between adjacent layers, direct connections can also be established between non-adjacent layers. For example, in a deep network, the feature map of a shallower layer is directly connected to the feature map of a deeper layer, so that the model can simultaneously obtain the detailed information of the low layer and the semantic information of the high layer.

[0101] Second, the dynamic connection mechanism is configured to output a connection matrix based on the input image content by means of a lightweight meta-network in the feature pyramid network, adaptively adjust the feature map connections, and strengthen the connections of the corresponding scale feature maps according to the characteristics of the image target size.

[0102] The adjustment of weight distribution includes adaptive weight learning, task-oriented weight distribution, and weight distribution based on feature differences.

[0103] First, adaptive weight learning is configured to automatically learn the fusion weights through an adaptive weight module according to the feature distribution and information richness of the feature maps, enabling the model to focus on valuable features.

[0104] Second, task-oriented weight distribution is configured to optimize the weights during model training by means of a sliding loss function according to the different attentions of different detection tasks to the target scales, so that the model can be adapted to the preset tasks. For example, pedestrian detection focuses on medium scales, and traffic sign detection focuses on small scales, which can make the model adapt to specific tasks.

[0105] Third, weight distribution based on feature differences is configured to increase the fusion weights for feature maps that exceed the preset feature difference value (i.e., large feature differences and strong complementarity) and reduce the weights for feature maps that do not exceed the preset feature difference value (i.e., small feature differences) by calculating statistics such as the similarity and variance of the feature maps, effectively integrating information, avoiding information redundancy, and finally achieving efficient information interaction and complementarity of feature maps of different scales.

[0106] (3) Modify the detection head, including at least optimizing the regression algorithm and the anchor mechanism and adding trajectory smoothing and motion compensation techniques to combine the enhanced features of the Transformer module to achieve accurate target positioning and classification.

[0107] S2. Preprocess the low-light and high-noise images through the input part of the improved YOLO v8 model, extract the illumination features and noise features in the images, and generate an optimized image tensor.

[0108] In this embodiment, the input part of the YOLO v8 model includes an illumination estimation module and a self-correction module. The illumination estimation module estimates the illumination of the low-light and high-noise images, and then the self-correction module correlates the images processed by the illumination estimation module to obtain an optimized image tensor. As Figure 2 shown is the schematic diagram of the image preprocessing structure.

[0109] Specifically, step S2 specifically includes the following steps:

[0110] S21. Illumination estimation is the basis for low-quality image optimization. The illumination estimation module uses a progressive learning method. Through parameters θ, structure H, and a weight sharing mechanism, that is, the same architecture H and weights θ are used in each stage, to learn the residual representation between the low-light observation image and the ideal illumination image, and to derive the module to be obtained after optimization of the illumination estimation module from the training loss and data. Among them, the mapping H with parameter θ is introduced θ to learn the illumination. The illumination estimation formula is:

[0111] {f(x) = x t+1 = x t + u t |u t = H θ (x t ), x 0 = y} (1)

[0112] In formula (1), x represents the illumination component, x 0 and y represent the initial low-light image, u t and x t respectively represent the residual term and the illumination component at the t-th stage, x t+1 represents the illumination component of the next stage at the t-th stage, that is, the illumination component at the (t + 1)-th stage, where t = 0, 1,..., T - 1.

[0113] S22. The image processed by the illumination estimation module in step S21 forms a new image tensor x t , which is used as the input of the self-calibration module. The self-calibration module uses the output result of the previous stage as the input x t-1 at the (t - 1)-th stage and associates it with the new image tensor x t . The result after association is used as the input to affect the illumination estimation output x t+1 at the next stage. Finally, at the (T - 1)-th stage, the original image tensor is fused with the output feature map to generate the final result G(x). The self-calibration association formula is:

[0114]

[0115] In formula (2), the value of t is t ≥ 1, K θ is the parameter θ that can be learned by the introduced parameterized operator; the image is composed of multiple pixel points, and each pixel point has a corresponding value, The operation represents the division calculation of the corresponding pixel values in y and x t ; The basic unit conversion at the t-th stage is expressed as:

[0116] F(x t ) = F(G(x t )) (3).

[0117] S3. The backbone network of the improved YOLO v8 model extracts features from the image tensor optimized in step S2, obtaining feature maps with different scales of height and width, including small-scale feature maps, medium-scale feature maps, and large-scale feature maps.

[0118] Further, step S3 specifically includes the following steps:

[0119] The improved YOLO v8 model extracts features from the image tensor optimized in step S2 through a feature pyramid network, obtaining feature maps of different scales to facilitate the processing of targets of different scales. Feature maps of different scales correspond to targets of different sizes, thereby improving the model's detection ability for targets of different sizes. In the backbone network of the improved YOLO v8 model, the image tensor is divided into three-scale feature maps P3, P4, and P5. Among them, P3, P4, and P5 represent small-scale feature maps, medium-scale feature maps, and large-scale feature maps respectively. P3, P4, and P5 are all four-dimensional tensors, and the four dimensions are batch size, channels, height, and width. Specifically, P3 is used to detect small targets, has a high spatial resolution, but fewer channels; P4 is used to detect medium targets, balancing spatial resolution and the number of channels; P5 is used to detect large targets, has a low spatial resolution, but more channels, and is suitable for capturing global information.

[0120] S4. The Transformer module in the backbone network of the improved YOLO v8 model captures the content relationship between the image features of the large-scale feature map.

[0121] Based on the investigation and research of the electric power construction site, it is found that among the same targets in different scenarios and different targets in the same scenario, large target objects are in the majority. To make it more conducive for the model to capture the feature information of the detection objects, in this embodiment, the Transformer module is applied to the P5 output channel in the backbone network of the YOLO v8 model to adaptively learn the internal connection of the input data and enhance the information capture ability. As Figure 3 shown in the schematic diagram of the large-scale feature extraction structure, step S4 is specifically as follows:

[0122] S41. Preprocessing of input features: First, the feature map P5 processed by the backbone network of the improved YOLO v8 model is unfolded and reshaped, converted into a two-dimensional sequence form, and spatial position information is injected through positional encoding. Then, the features are decomposed into three parts, Q, K, and V, by linear transformation and divided into multiple heads to adapt to the multi-head attention mechanism. This process not only preserves the spatial information of the feature map but also provides multi-perspective feature representations for the Transformer module, thereby enhancing the model's global perception ability and detection accuracy for targets under low-light conditions. Here, Q, K, and V represent the query vector, key vector, and value vector respectively.

[0123] S42. Processing by multi-head attention mechanism: The core of the multi-head attention mechanism is to divide the input features into multiple "heads" (Heads), then calculate the attention weights of each head, and merge the attention weights of all heads, as follows:

[0124]

[0125] In formula (4), Attention(Q, K, V) is used to calculate the output of the attention mechanism. By operating on the input query vector Q, key vector K, and value vector V, the importance of different input features is determined, and a weighted output representation is generated. K T represents the transpose matrix of the key vector K, and D K represents the dimension of the key vector;

[0126] The calculation formula for the attention weight of each head is:

[0127] Head i = Attention(QW i q , KW i k , VW i v ) (5)

[0128] In formula (5), QW i q , QW i q , QW i q represent the weight matrices of the i-th head respectively;

[0129] Then, the attention results of all heads are merged and fused through linear transformation to form the output of the multi-head attention mechanism. The splicing formula is:

[0130] MultiHead(Q, K, V) = CONCAT(Head1,..., Head n )W0 (6)

[0131] In formula (6), W0 represents the output linear transformation matrix, and CONCAT represents the attention for combining all heads.

[0132] S43. Multilayer perceptron non-linear processing: Further non-linear transformation and feature extraction are performed on the information processed by the multi-head attention mechanism in step S42. While further enhancing the global perception ability of features, the non-linear transformation of the multi-layer perceptron can improve the stability of the model, making it more robust when processing images under low-light conditions. The multi-layer perceptron in the Transformer module includes two fully connected layers, a non-linear activation function inserted between the two fully connected layers, and an output layer; the multi-layer perceptron first performs weighted combination and offset adjustment of features on the input data through a linear transformation W1q + b1, then applies the non-linear activation function f to perform non-linear mapping on the result of the linear transformation so that the model can learn complex patterns and features in the data, then performs the second linear transformation W2, and finally adds the bias b2 and obtains the final output through the activation function g of the output layer. The formula is as follows:

[0133] MLP(q) = g(W2f(W1q + b1) + b2) (7)

[0134] In formula (7), MLP represents the multi-layer perceptron, W1 and W2 are weight matrices, which learn the feature relationship of the data by weighting each dimension of the input vector; b1 and b2 are bias vectors, which add fixed values to the output and adjust the output range.

[0135] S5. Transfer the feature map processed in step S4 to the detection head of the improved YOLO v8 model for detection.

[0136] In this embodiment, step S5 specifically includes the following steps:

[0137] S51. Perform multi-scale feature map fusion on the feature map P5 enhanced by the Transformer module, the feature maps P3 and P4 not enhanced by the Transformer module, so as to complete multi-scale fusion of features. This fusion not only retains the local details of the target, but also enhances the detection ability of the target under low-light conditions by capturing global dependency relationships; at the same time, the position encoding introduced by the Transformer module retains the spatial position information of the feature map, providing an important spatial reference for subsequent object detection.

[0138] S52. The feature map processed in step S51 is passed to the detection head of the improved YOLO v8 model. The detection head is used for object localization and class prediction. Specifically, the detection head predicts the bounding box of the object through a regression algorithm and uses the anchor mechanism to predict the offset of the bounding box relative to the anchor point to achieve precise localization of the object. At the same time, the detection head performs class prediction on each bounding box, then outputs the probability that the object belongs to each class, and normalizes the class probabilities through the Softmax function. The formula of the Softmax function is as follows:

[0139]

[0140] In formula (8), m represents the total number of categories of protective equipment, j represents the protective equipment of the j-th category, and s j is the original score of the j-th category of protective equipment, e is the base of the natural logarithm function. Formula (8) means dividing s j by the total score of all categories of protective equipment to obtain the probability ratio of the protective equipment of the j-th category.

[0141] Preferably in this embodiment, after step S52, step S53 is further included. The step S53 includes the following:

[0142] To further optimize the detection results, a trajectory smoothing method and a motion compensation method are introduced in the detection stage. Among them, the trajectory smoothing method smooths the motion trajectory of the object through time series analysis, reduces the jitter of the trajectory, and improves the continuity and stability of object detection. The time series analysis at least adopts any one of weighted average or Kalman filtering; the motion compensation method calculates the difference between the trajectory smoothed by the trajectory smoothing method and the original trajectory, and performs geometric transformation on the image frame to eliminate the jitter during the motion process and restore the stable effect of the video. These technologies significantly improve the stability and accuracy of detection under low light conditions. The geometric transformation at least includes translation and rotation.

[0143] S6. When training the improved YOLO v8 model, a sliding loss function is introduced to adjust the dynamic loss weights of samples with different levels of difficulty to solve the problem of sample imbalance, and finally output the detection results with the annotation information of the improved YOLO v8 model.

[0144] Sample imbalance means that in the model training data, the number of samples that are easy to classify is much larger than the number of samples that are difficult to classify. This may cause the model to be biased towards the majority class, thereby reducing the recognition ability for the minority class (i.e., samples that are difficult to classify). The core idea of the sliding loss function is to dynamically adjust the loss weights according to the difficulty of the samples, thereby enhancing the model's learning ability for samples that are difficult to classify.

[0145] In this embodiment, step S6 specifically includes the following steps:

[0146] S61. Sample classification based on intersection over union (IoU): The sliding loss function determines the difficulty level of samples by calculating the IoU between the predicted bounding box and the ground truth bounding box. Let the IoU be denoted as IoU. When the value of IoU is lower than the preset threshold, the sample is difficult to classify; otherwise, the sample is easy to classify.

[0147] S62. Dynamic adjustment of loss weights:

[0148] The sliding loss function dynamically adjusts the loss weights of samples with different difficulty levels, enabling the model to focus on difficult-to-classify samples during training. This not only improves the model's ability to recognize minority classes but also enhances the overall robustness of the model, as follows:

[0149]

[0150] In formula (9), z represents the value of IoU between the predicted bounding box and the ground truth bounding box, and p represents the average value of IoU, serving as the preset threshold. When z is small (i.e., the sample is difficult to classify), the loss weight is large, thereby enhancing the model's learning ability for these samples; when z is large (i.e., the sample is easy to classify), the loss weight is small, preventing the model from being overly biased towards these samples. This optimization not only enhances the model's learning ability for minority classes but also improves the overall robustness and generalization ability of the model, significantly enhancing the performance of object detection under low-light conditions and ensuring the correct detection and recognition of target objects in images.

[0151] As shown in Table 1, the method of the present invention is compared with several current advanced detection methods on five scenario datasets. Specifically, the dataset selected in this embodiment consists of images of electric power workers in five different scenarios. In this embodiment, seven currently advanced detection algorithms are selected, namely CenterNet, RTDETR, SSD, Faster R-CNN, YOLO v8, YOLO v9, and YOLO v10. The evaluation metrics used in this embodiment include MAP, recall rate, precision, and F1 score. The value ranges of these metrics are all [0, 1]. The larger the values of these metrics, the better the performance of the model usually indicates. As can be seen from Table 1, the method of the present invention performs best in all five evaluation metrics.

[0152] Table 1 Quantitative comparison between the method of the present invention and seven advanced detection methods on five scenario datasets

[0153]

[0154] In summary, the present invention can effectively improve the adaptability of the model to complex scenarios, significantly improve the detection performance and accuracy of the model under low-light conditions, optimize the learning ability for difficult samples, enhance the overall detection effect, and play a role in warning the safety of electric power workers.

[0155] The above only describes the basic principles and preferred embodiments of the present invention. Those skilled in the art can make many changes and improvements based on the above description, and these changes and improvements should fall within the protection scope of the present invention.

Claims

1. A method for detecting protective equipment for power workers at night based on a deep cascade network architecture, characterized in that: The steps include: S1. Build the improved YOLO v8 model: The YOLO v8 model at least includes an input part, a backbone network, a neck network and a detection head, and at least a Transformer module is embedded in the backbone network, the feature pyramid network structure in the backbone network is adjusted, and the detection head is modified to obtain an improved YOLO v8 model; the input part is used for image preprocessing and the preprocessed image is passed to the backbone network, the backbone network is used for feature extraction and the extracted features are unidirectionally input to the neck network, the neck network is used for multi-scale fusion of the features input by the backbone network and the multi-scale fused features are fully connected to the detection head, and the detection head is used for target positioning and category prediction to generate prediction results; S2, preprocess the low-light and high-noise image through the input part of the improved YOLO v8 model, extract the lighting features and noise features in the image, and generate an optimized image tensor; S3, extracting features from the image tensor optimized in step S2 through the backbone network of the improved YOLO v8 model to obtain feature maps of different height and width scales, including small-scale feature maps, medium-scale feature maps, and large-scale feature maps; S4, capturing the content relationship between image features of large-scale feature maps through the Transformer module in the backbone network of the improved YOLO v8 model; S5, passing the feature map processed in step S4 to the detection head of the improved YOLO v8 model for detection; S6. When training the improved YOLO v8 model, a sliding loss function is introduced to adjust the dynamic loss weights of samples of different difficulty levels, and finally the detection results with the annotation information of the improved YOLO v8 model are output.

2. The method for detecting protective equipment for electric power workers at night based on a deep cascade network architecture according to claim 1 is characterized in that: In step S1, at least the Transformer module is embedded in the backbone network, the feature pyramid network structure in the backbone network is adjusted, and the detection head is modified to obtain an improved YOLO v8 model, specifically including: (1) In the backbone network, a Transformer module is embedded to process large-scale feature maps. The Transformer module uses a multi-head attention mechanism and a multi-layer perceptron to enhance the capture of global image information. (2) By adjusting the connection mode and weight distribution of the feature pyramid network structure, the feature pyramid network can interact and complement the feature maps of different scales; (3) Modify the detection head, including at least optimizing the regression algorithm and anchor mechanism and adding trajectory smoothing and motion compensation techniques to combine the enhanced features of the Transformer module to achieve accurate target positioning and classification.

3. The method for detecting protective equipment for electric power workers at night based on a deep cascade network architecture according to claim 2 is characterized in that: The connection mode and weight distribution of the feature pyramid network structure are adjusted as follows: The adjustment of the connection mode includes expanding cross-layer connections and introducing a dynamic connection mechanism, wherein the expanded cross-layer connection is configured to establish direct connections between small-scale feature maps and large-scale feature maps, between large-scale feature maps and small-scale feature maps, and between non-adjacent layers across multiple levels to enhance the model's perception of objects of different sizes; the dynamic connection mechanism is configured to output a connection matrix based on the input image content with the help of a lightweight meta-network in the feature pyramid network, adaptively adjust the feature map connection, and strengthen the corresponding scale feature map connection based on the image target size characteristics; The adjustment of weight distribution includes adaptive weight learning, task-oriented weight distribution and weight distribution based on feature difference, wherein adaptive weight learning is configured to automatically learn fusion weights through an adaptive weight module according to the feature distribution and information richness of the feature map; task-oriented weight distribution is configured to optimize the weights with the help of a sliding loss function during model training according to the different attention paid to the target scale by different detection tasks so that the model can adapt to the preset tasks; weight distribution based on feature difference is configured to increase the fusion weight for feature maps that exceed the preset feature difference value by calculating the statistics of the feature map, and reduce the weight for feature maps that do not exceed the preset feature difference value.

4. The method for detecting protective equipment for electric power workers at night based on a deep cascade network architecture according to claim 1 is characterized in that: In step S2, the input part of the YOLO v8 model includes an illumination estimation module and a self-correction module. The illumination estimation module is used to perform illumination estimation on the low-light and high-noise image, and then the self-correction module is used to associate the image processed by the illumination estimation module to obtain an optimized image tensor.

5. The method for detecting protective equipment for electric power workers at night based on a deep cascade network architecture according to claim 4 is characterized in that: Step S2 specifically includes the following steps: S21. Use the illumination estimation module to adopt a progressive learning method to learn the residual representation between the low-light observation image and the ideal illumination image through parameters θ, structure H and weight sharing mechanism, and derive the module to be obtained after the illumination estimation module is optimized from the training loss and data, where the mapping H with parameter θ is introduced. θ To learn about lighting, the lighting estimation formula is: {F(x)=x t+1 =x t +u t |u t =H θ (x t ),x 0 =y} (1) In formula (1), x represents the illumination component, x 0 and y represent the initial low-light image, u t and x t Represent the residual term and illumination component of the tth stage, respectively, x t+1 represents the illumination component of the next stage of the t-th stage, that is, the illumination component of the t+1-th stage, where t=0, 1, ..., T-1; S22: The image processed by the illumination estimation module in step S21 forms a new image tensor x t , as the input of the self-correction module, the self-correction module uses the output of the previous stage as the input x of the t-1th stage t-1 and with the new image tensor x t The result after association is used as input to influence the lighting estimation output x of the next stage t+1 Finally, the T-1 stage fuses the original image tensor with the output feature map to generate the final result G(x). The self-correction correlation formula is: In formula (2), the value of t is t≥1, K θ is the parameter θ that can be learned by the introduced parameterized operator; the image is composed of multiple pixels, each pixel has a corresponding value, The operation represents the operation on y and x t The value of the corresponding pixel point is divided by the value of the corresponding pixel point in the t-th stage; the basic unit conversion is expressed as: F(x t )=f(G(x t )) (3)。 6. The method for detecting protective equipment for nighttime electric power workers based on a deep cascade network architecture according to claim 1 is characterized in that: Step S3 specifically includes the following steps: The improved YOLO v8 model extracts features from the image tensor optimized in step S2 through a feature pyramid network to obtain feature maps of different scales. Feature maps of different scales correspond to targets of different sizes. In the backbone network of the improved YOLO v8 model, the image tensor is divided into feature maps of three scales, P3, P4, and P5, where P3, P4, and P5 represent small-scale feature maps, medium-scale feature maps, and large-scale feature maps, respectively. P3, P4, and P5 are all four-dimensional tensors, and the four dimensions are batch size, channel, height, and width, respectively.

7. The method for detecting protective equipment for electric power workers at night based on a deep cascade network architecture according to claim 6 is characterized in that: Step S4 specifically includes the following steps: S41. Preprocessing of input features: First, the feature map P5 processed by the backbone network of the improved YOLO v8 model is expanded and reshaped, converted into a two-dimensional sequence form, and the spatial position information is injected through position encoding; then, the feature is decomposed into three parts Q, K, and V by linear transformation and divided into multiple heads, where Q, K, and V represent query vectors, key vectors, and value vectors respectively; S42, multi-head attention mechanism processing: calculate the attention weight of each head obtained in step S41, and merge the attention weights of all heads, as follows: In formula (4), Attention(Q, K, V) is used to calculate the output of the attention mechanism. It operates on the input query vector Q, key vector K, and value vector V to determine the importance of different input features and generate a weighted output representation, K T represents the transposed matrix of the key vector K, D K represents the dimension of the key vector; The attention weight calculation formula for each head is: Head i =Attention(QW i q ,KW i k ,VW i v ) (5) In formula (5), QW i q , QW i q , QW i q Respectively represent the weight matrix of the i-th head; Then, the attention results of all heads are merged and fused through linear transformation to form the output of the multi-head attention mechanism. The splicing formula is: MultiHead(Q,K,V)=CONCAT(Head1,…,Head n )W0 (6) In formula (6), W0 represents the output linear transformation matrix, and CONCAT represents the attention used to merge all heads; S43, multi-layer perceptron nonlinear processing: nonlinear transformation and feature extraction are performed on the information processed by the multi-head attention mechanism in step S42. The multi-layer perceptron in the Transformer module includes two fully connected layers, a nonlinear activation function inserted between the two fully connected layers, and an output layer; the multi-layer perceptron first performs a weighted combination and offset adjustment of the input data through a linear transformation W1q+b1, and then applies a nonlinear activation function f to perform nonlinear mapping on the result after the linear transformation so that the model can learn the complex patterns and features in the data, and then performs a second linear transformation W2, and finally adds a bias b2 and obtains the final output through the activation function g of the output layer. The formula used is as follows: MLP(q)=g(W2f(W1q+b1)+b2) (7) In formula (7), MLP represents a multi-layer perceptron, W1 and W2 are weight matrices that learn the data feature relationship by weighting each dimension of the input vector; b1 and b2 are bias vectors that add fixed values ​​to the output and adjust the output range.

8. The method for detecting protective equipment for electric power workers at night based on a deep cascade network architecture according to claim 1 is characterized in that: Step S5 specifically includes the following steps: S51, fusing the feature map P5 enhanced by the Transformer module with the feature maps P3 and P4 not enhanced by the Transformer module to form a three-scale feature map, thereby completing the multi-scale fusion of features; S52, the feature map processed by step S51 is passed to the detection head of the improved YOLO v8 model, which is used for target positioning and category prediction. Specifically, the detection head predicts the bounding box of the target through a regression algorithm and uses an anchor mechanism to predict the offset of the bounding box relative to the anchor point to achieve accurate positioning of the target. At the same time, the detection head predicts the category of each bounding box and outputs the probability that the target belongs to each category, and normalizes the category probability through the Softmax function, where the formula of the Softmax function is: In formula (8), m represents the total number of protective equipment categories, j represents the protective equipment of the jth category, and s j is the original score of the jth protective equipment category, e is the base of the natural logarithm function, and formula (8) represents s j Divide by the total score of all protective equipment categories to obtain the probability proportion of protective equipment in the jth category.

9. The method for detecting protective equipment for electric power workers at night based on a deep cascade network architecture according to claim 8 is characterized in that: Step S52 is followed by step S53, wherein step S53 includes the following: The detection stage adopts a trajectory smoothing method and a motion compensation method. The trajectory smoothing method smoothes the motion trajectory of the target through time series analysis, and the time series analysis adopts at least one of weighted average or Kalman filtering; the motion compensation method calculates the difference between the trajectory smoothed by the trajectory smoothing method and the original trajectory, and performs a geometric transformation on the image frame. The geometric transformation at least includes translation and rotation.

10. The method for detecting protective equipment for electric power workers at night based on a deep cascade network architecture according to claim 1 is characterized in that: Step S6 specifically includes the following steps: S61. Sample classification based on intersection over union: The sliding loss function determines the difficulty of a sample by calculating the intersection over union between the predicted bounding box and the true bounding box. Let the intersection over union be IoU: when the value of IoU is lower than the preset threshold, the sample is difficult to classify; otherwise, the sample is easy to classify. S62, Dynamic loss weight adjustment: The sliding loss function dynamically adjusts the loss weights of samples of different difficulty levels, so that the model pays attention to samples that are difficult to classify during training, as follows: In formula (9), z represents the IoU value between the predicted bounding box and the true bounding box, and p represents the average value of IoU, which is used as the preset threshold.

Images