Method and device for establishing defect detection model of lightweight power equipment, defect detection method and chip

Abstract

The invention relates to the technical field of intelligent detection and chip design, in particular to a method and a device for establishing a defect detection model of lightweight power equipment, a defect detection method and a chip. The building method comprises the following steps of: introducing a scene depth estimation modulation weight based on a mahalanobis distance to improve a color attenuation priori theory, and further realizing defogging processing of an image; inputting the fogless image into a lightweight detection network model for model training to obtain a power equipment defect detection model, and introducing a regional routing attention mechanism to enhance the multi-level feature representation capability to obtain a plurality of enhanced fusion feature maps with different resolutions; the detection head outputs a prediction information tensor to generate a prediction frame, and model parameters are adjusted by comparing the prediction frame with a real frame and calculating loss. According to the method, the detection precision and the detection speed of the defects of the power equipment in the haze environment are remarkably improved while lightweight deployment of the edge side and the end side is realized, and real-time intelligent inspection of the power equipment is realized.

Description

Technical Field

[0001] This disclosure relates to the fields of intelligent detection and chip design technology, specifically to a method and apparatus for establishing a defect detection model for lightweight power equipment, a defect detection method, and a chip. Background Technology

[0002] With the continuous expansion of the power grid, the safety inspection of high-voltage transmission lines has become an indispensable and crucial part of operation and maintenance. Various electrical devices are installed on transmission lines, and the operating status of these devices directly affects the safety and stability of the entire transmission system. Therefore, the inspection of these electrical devices is a vital component of transmission line inspection.

[0003] Due to the limitations of manual inspections, drone inspection technology has emerged. However, its widespread application has also brought new challenges: in mountainous, humid, and foggy environments, images of power equipment captured by drones often suffer from blurriness and difficulty in effectively identifying surface defects. Although deep learning-based detection algorithms exist for power equipment image processing, these methods are typically computationally complex and rely on high-performance servers, making real-time processing difficult on edge devices with limited computing power. Furthermore, because power equipment generally exhibits low color saturation, traditional image dehazing algorithms easily misclassify it as distant areas, resulting in poor dehazing effects or even introducing distortion, further impacting the accuracy of defect identification. Therefore, developing a power equipment defect detection method suitable for lightweight edge deployment in low-visibility environments such as fog and haze, enabling real-time and accurate defect identification, has become a key challenge in promoting the intelligent operation and maintenance of transmission lines. Summary of the Invention

[0004] To address the problems in the related technologies, this disclosure provides a method and apparatus for establishing a defect detection model for lightweight power equipment, as well as a defect detection method and chip.

[0005] In a first aspect, embodiments of this disclosure provide a method for establishing a defect detection model for lightweight power equipment, the method being implemented by a computer, the method comprising: Obtain a training set, which includes multiple fog-free images of power equipment; The training set is input into a lightweight detection network model for training to obtain a power equipment defect detection model. The lightweight detection network model includes a backbone network, a neck network, and multiple detection heads. The backbone network includes multiple depthwise separable convolutional modules, and the neck network includes multiple region routing attention mechanism modules. The model training includes: For any fog-free image of power equipment in the training set, features are extracted from the fog-free image of power equipment using the multiple depthwise separable convolutional modules to obtain multiple power equipment feature maps with different resolutions. Enhanced feature fusion is performed on the multiple power equipment feature maps using the multiple region routing attention mechanism modules to obtain multiple enhanced fused feature maps with different resolutions. Multiple detection heads detect the multiple enhanced fused feature maps respectively, outputting multiple prediction information tensors corresponding to each enhanced fused feature map. Multiple prediction boxes are generated based on the multiple prediction information tensors, thereby obtaining multiple prediction boxes for the fog-free image of power equipment. The multiple prediction boxes of the fog-free image of power equipment are compared with the corresponding ground truth boxes, and the model parameters of the lightweight detection network model are adjusted.

[0006] According to embodiments of this disclosure, the fog-free image of the power equipment is an image obtained by defogging a foggy image of the power equipment collected in a hazy environment; wherein, the defogging process includes: For the foggy image of the power equipment, calculate the scene depth estimation modulation weight for each pixel; The foggy image of the power equipment is converted from the RGB color space to the HSV color space to obtain the brightness and saturation of each pixel. Based on the scene depth estimation modulation weight, the brightness coefficient and saturation coefficient of the corresponding pixels are set; Based on the brightness and saturation of each pixel in the foggy image of the power equipment, as well as the brightness coefficient and saturation coefficient of the corresponding pixel, the scene depth estimation formula for the foggy image of the power equipment is determined. The fog-free image of the power equipment is obtained based on the scene depth estimation formula and the atmospheric scattering model.

[0007] According to embodiments of this disclosure, calculating the scene depth estimation modulation weights for each pixel includes: For the foggy image of the power equipment, the foggy image of the power equipment is converted from the RGB color space to the YCbCr color space to obtain the blue chromaticity component and the red chromaticity component of each pixel in the foggy image of the power equipment. The blue chromaticity component and the red chromaticity component of the pixel form the chromaticity vector of the corresponding pixel. Calculate the Mahalanobis distance between the chroma vector of each pixel in the foggy image of the power equipment and the typical chroma vector of the power equipment, and map the Mahalanobis distance to the scene depth estimation modulation weight of the corresponding pixel.

[0008] According to embodiments of this disclosure, the step of extracting features from the fog-free image of the power equipment using the plurality of depthwise separable convolutional modules to obtain multiple power equipment feature maps with different resolutions includes: The fog-free image of the power equipment is subjected to feature extraction by a series of sequentially set depthwise separable convolutional modules. The first depthwise separable convolutional module uses the fog-free image of the power equipment as the image to be processed to extract features and obtain a power equipment feature map of the corresponding resolution. One or more depthwise separable convolutional modules other than the first depthwise separable convolutional module use the power equipment feature map of the corresponding resolution output by the previous depthwise separable convolutional module as the image to be processed to extract features and obtain a power equipment feature map of the corresponding resolution.

[0009] According to embodiments of this disclosure: the image to be processed includes multiple input channels and multiple spatial locations, the input channels covering the multiple spatial locations; The method further includes: when the depth-separable convolution module extracts features from the image to be processed, performing two-dimensional convolution on each input channel of the image to be processed to generate an intermediate feature map group corresponding to the input channel; and for multiple spatial locations covered by the input channel, performing point-by-point convolution on the intermediate feature map groups corresponding to the multiple input channels respectively to obtain a power equipment feature map of corresponding resolution.

[0010] According to embodiments of this disclosure, the step of performing enhanced feature fusion on the feature maps of the multiple power devices through the multiple regional routing attention mechanism modules to obtain multiple enhanced fused feature maps with different resolutions includes: The M power equipment feature maps are arranged in ascending order of resolution; The first and second power equipment feature maps are fused to obtain the first intermediate fused feature map. Starting with the first intermediate fused feature map and the third power equipment feature map, the following steps are performed sequentially: The nth intermediate fused feature map is fused with the mth power equipment feature map to obtain the (n+1)th intermediate fused feature map; the (n+1)th intermediate fused feature map is then fused with the (m+1)th power equipment feature map, and so on, until the (M-2)th intermediate fused feature map is fused with the Mth power equipment feature map, thus obtaining the first enhanced fused feature map. During the final feature fusion execution, the corresponding regional routing attention mechanism module is used. , ; Starting with the first enhanced fused feature map and the (M-2)th intermediate fused feature map, the following steps are performed sequentially: The p-th enhanced fused feature map and the q-th intermediate fused feature map are fused using the corresponding region routing attention mechanism module to obtain the (p+1)-th enhanced fused feature map; the (p+1)-th enhanced fused feature map is then fused with the (q-1)-th intermediate fused feature map using the corresponding region routing attention mechanism module, until the feature fusion of the (M-2)-th and first intermediate fused feature maps is completed, thus obtaining the (M-1)-th enhanced fused feature map. , ; The M-1th enhanced fusion feature map and the first power equipment feature map are fused using the corresponding regional routing attention mechanism module to obtain the Mth enhanced fusion feature map.

[0011] According to embodiments of this disclosure, when using the region routing attention mechanism module to perform feature fusion on two feature maps to be processed, the method further includes: The two feature maps to be processed are stitched together to obtain a stitched feature map; The stitched feature map is evenly divided into multiple non-overlapping regions; Average pooling is performed on each non-overlapping region to generate a region-level query vector, a region-level key vector, and a region-level value vector for each non-overlapping region in the stitched feature map. Based on the region-level query vector and region-level key vector of the multiple non-overlapping regions, the correlation between any two non-overlapping regions is calculated to form a region correlation adjacency matrix. For the non-overlapping regions, based on the region relevance adjacency matrix, the top-k routes are used to select the k most relevant regions. The region-level attention score between the non-overlapping regions and each of the k most relevant regions is calculated, and the calculated region-level attention scores are used to weight and fuse the region-level value vectors of the k most relevant regions to obtain the corresponding enhanced fusion feature map.

[0012] According to embodiments of this disclosure, the plurality of detection heads respectively detect the plurality of enhanced fusion feature maps and output a plurality of prediction information tensors corresponding to each of the plurality of enhanced fusion feature maps, including: for any enhanced fusion feature map, the detection head outputs a plurality of prediction information tensors corresponding to each spatial position in the enhanced fusion feature map and a plurality of corresponding prior boxes, wherein the prediction information tensors correspond to the prior boxes; The step of generating multiple prediction boxes for the corresponding enhanced fusion feature map based on the multiple prediction information tensors includes: generating multiple prediction boxes corresponding to each spatial location based on the multiple prediction information tensors corresponding to each spatial location in the enhanced fusion feature map.

[0013] According to embodiments of this disclosure, the prediction information tensor includes: class confidence, normalized offset, and logarithmic scaling factor.

[0014] Secondly, embodiments of this disclosure provide a method for detecting defects in lightweight power equipment, the method comprising: A fog-free image of the power equipment to be processed is acquired, and the fog-free image of the power equipment to be processed is input into the power equipment defect detection model to obtain the power equipment defect detection result. The power equipment defect detection result includes the category, location and confidence level of the power equipment. The power equipment defect detection model is established through the following steps: Obtain a training set, which includes multiple fog-free images of power equipment; The training set is input into a lightweight detection network model for training to obtain a power equipment defect detection model. The lightweight detection network model includes a backbone network, a neck network, and multiple detection heads. The backbone network includes multiple depthwise separable convolutional modules, and the neck network includes multiple region routing attention mechanism modules. The model training includes: For any fog-free image of power equipment in the training set, features are extracted from the fog-free image of power equipment using the multiple depthwise separable convolutional modules to obtain multiple power equipment feature maps with different resolutions. Enhanced feature fusion is performed on the multiple power equipment feature maps using the multiple region routing attention mechanism modules to obtain multiple enhanced fused feature maps with different resolutions. Multiple detection heads detect the multiple enhanced fused feature maps respectively, outputting multiple prediction information tensors corresponding to each enhanced fused feature map. Multiple prediction boxes are generated based on the multiple prediction information tensors, thereby obtaining multiple prediction boxes for the fog-free image of power equipment. The multiple prediction boxes of the fog-free image of power equipment are compared with the corresponding ground truth boxes, and the model parameters of the lightweight detection network model are adjusted.

[0015] According to embodiments of this disclosure, acquiring a fog-free image of the power equipment to be processed includes: Acquire foggy images of the power equipment to be processed; For the foggy image of the power equipment to be processed, calculate the scene depth estimation modulation weight for each pixel; The foggy image of the power equipment to be processed is converted from the RGB color space to the HSV color space to obtain the brightness and saturation of each pixel. Based on the scene depth estimation modulation weight, the brightness coefficient and saturation coefficient of the corresponding pixels are set; Based on the brightness and saturation of each pixel in the foggy image of the power equipment to be processed, as well as the brightness coefficient and saturation coefficient of the corresponding pixel, the scene depth estimation formula of the foggy image of the power equipment to be processed is determined. The fog-free image of the power equipment to be processed is obtained based on the scene depth estimation formula and the atmospheric scattering model.

[0016] According to embodiments of this disclosure, calculating the scene depth estimation modulation weights for each pixel includes: For the foggy image of the power equipment to be processed, the foggy image of the power equipment to be processed is converted from the RGB color space to the YCbCr color space to obtain the blue chromaticity component and the red chromaticity component of each pixel in the foggy image of the power equipment to be processed. The blue chromaticity component and the red chromaticity component of the pixel form the chromaticity vector of the corresponding pixel. The Mahalanobis distance between the chroma vector of each pixel in the foggy image of the power equipment to be processed and the typical chroma vector of the power equipment is calculated, and the Mahalanobis distance is mapped to the scene depth estimation modulation weight of the corresponding pixel.

[0017] Thirdly, embodiments of this disclosure provide an apparatus for establishing a lightweight defect detection model for power equipment, comprising: The acquisition module is configured to acquire a training set, which includes multiple fog-free images of power equipment. The model training module is configured to input the training set into a lightweight detection network model for model training to obtain a power equipment defect detection model. The lightweight detection network model includes a backbone network, a neck network, and multiple detection heads. The backbone network includes multiple depthwise separable convolutional modules, and the neck network includes multiple region routing attention mechanism modules. The model training includes: For any fog-free image of power equipment in the training set, features are extracted from the fog-free image of power equipment using the multiple depthwise separable convolutional modules to obtain multiple power equipment feature maps with different resolutions. Enhanced feature fusion is performed on the multiple power equipment feature maps using the multiple region routing attention mechanism modules to obtain multiple enhanced fused feature maps with different resolutions. Multiple detection heads detect the multiple enhanced fused feature maps respectively, outputting multiple prediction information tensors corresponding to each enhanced fused feature map. Multiple prediction boxes are generated based on the multiple prediction information tensors, thereby obtaining multiple prediction boxes for the fog-free image of power equipment. The multiple prediction boxes of the fog-free image of power equipment are compared with the corresponding ground truth boxes, and the model parameters of the lightweight detection network model are adjusted.

[0018] According to an embodiment of this disclosure, the fog-free image of the power equipment is an image obtained by defogging a foggy image of the power equipment collected in a hazy environment; the device further includes: a first defogging processing module; The first dehazing module is configured to calculate the scene depth estimation modulation weight for each pixel in the foggy image of the power equipment. The foggy image of the power equipment is converted from the RGB color space to the HSV color space to obtain the brightness and saturation of each pixel. Based on the scene depth estimation modulation weight, the brightness coefficient and saturation coefficient of the corresponding pixels are set; Based on the brightness and saturation of each pixel in the foggy image of the power equipment, as well as the brightness coefficient and saturation coefficient of the corresponding pixel, the scene depth estimation formula for the foggy image of the power equipment is determined. The fog-free image of the power equipment is obtained based on the scene depth estimation formula and the atmospheric scattering model.

[0019] Fourthly, embodiments of this disclosure provide a lightweight defect detection chip for power equipment, comprising: The detection module is configured to acquire a fog-free image of the power equipment to be processed, input the fog-free image of the power equipment to be processed into the power equipment defect detection model, and obtain the power equipment defect detection result, which includes the category, location and confidence level of the power equipment. The power equipment defect detection model is established through the following steps: Obtain a training set, which includes multiple fog-free images of power equipment; The training set is input into a lightweight detection network model for training to obtain a power equipment defect detection model. The lightweight detection network model includes a backbone network, a neck network, and multiple detection heads. The backbone network includes multiple depthwise separable convolutional modules, and the neck network includes multiple region routing attention mechanism modules. The model training includes: For any fog-free image of power equipment in the training set, features are extracted from the fog-free image of power equipment using the multiple depthwise separable convolutional modules to obtain multiple power equipment feature maps with different resolutions. Enhanced feature fusion is performed on the multiple power equipment feature maps using the multiple region routing attention mechanism modules to obtain multiple enhanced fused feature maps with different resolutions. Multiple detection heads detect the multiple enhanced fused feature maps respectively, outputting multiple prediction information tensors corresponding to each enhanced fused feature map. Multiple prediction boxes are generated based on the multiple prediction information tensors, thereby obtaining multiple prediction boxes for the fog-free image of power equipment. The multiple prediction boxes of the fog-free image of power equipment are compared with the corresponding ground truth boxes, and the model parameters of the lightweight detection network model are adjusted.

[0020] According to embodiments of this disclosure, it further includes: Second defogging module; The second defogging module is configured to acquire a foggy image of the power equipment to be processed; For the foggy image of the power equipment to be processed, calculate the scene depth estimation modulation weight for each pixel; The foggy image of the power equipment to be processed is converted from the RGB color space to the HSV color space to obtain the brightness and saturation of each pixel. Based on the scene depth estimation modulation weight, the brightness coefficient and saturation coefficient of the corresponding pixels are set; Based on the brightness and saturation of each pixel in the foggy image of the power equipment to be processed, as well as the brightness coefficient and saturation coefficient of the corresponding pixel, the scene depth estimation formula of the foggy image of the power equipment to be processed is determined. The fog-free image of the power equipment to be processed is obtained based on the scene depth estimation formula and the atmospheric scattering model.

[0021] Fifthly, embodiments of this disclosure provide a computer-readable storage medium having computer instructions stored thereon that, when executed by a processor, implement the method as described in any of the first aspects.

[0022] In a sixth aspect, embodiments of this disclosure provide a computer program product including computer instructions that, when executed by a processor, implement the method as described in any of the first aspects.

[0023] This disclosure trains a lightweight detection network model by inputting a training set containing multiple fog-free images of power equipment into the model. The lightweight detection network model includes a backbone network, a neck network, and multiple detection heads. The backbone network utilizes multiple depthwise separable convolutional modules to extract multiple feature maps of power equipment at different resolutions. The neck network utilizes multiple region routing attention mechanism modules to obtain multiple enhanced fusion feature maps at different resolutions. Multiple detection heads then output multiple prediction information tensors to generate corresponding prediction boxes. By comparing the prediction boxes of the fog-free power equipment images with the corresponding ground truth boxes, the parameters of the lightweight detection network model are adjusted, thereby achieving the training of the lightweight detection network model.

[0024] This disclosure significantly improves the detection accuracy and speed of power equipment defects in hazy environments while achieving lightweight design. It can be deployed on edge devices such as drones, providing a complete visual perception solution for intelligent inspection of power systems. It can detect potential hazards of power equipment on transmission lines in real time under complex weather conditions, significantly improving the reliability, safety and operation and maintenance efficiency of the power grid, and laying a scalable technical foundation for building smart grids and the energy internet.

[0025] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0026] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments, taken in conjunction with the accompanying drawings. In the drawings: Figure 1 A flowchart illustrating a method for establishing a defect detection model for lightweight power equipment according to an embodiment of the present disclosure is shown. Figure 2 This diagram illustrates the structure of a backbone network according to an embodiment of the present disclosure. Figure 3 This diagram illustrates an enhanced feature fusion of feature maps of multiple power devices using multiple regional routing attention mechanism modules, according to an embodiment of the present disclosure. Figure 4 This illustration shows another schematic diagram of enhancing feature fusion of multiple power device feature maps using multiple area routing attention mechanism modules according to an embodiment of the present disclosure; Figure 5A flowchart illustrating a defect detection method for lightweight power equipment according to an embodiment of the present disclosure is shown. Figure 6 A structural diagram of an apparatus for establishing a defect detection model for lightweight power equipment according to an embodiment of the present disclosure is shown. Figure 7 A structural diagram of a lightweight defect detection chip for power equipment according to an embodiment of the present disclosure is shown. Figure 8 A schematic diagram of the structure of a computer system suitable for implementing a method for establishing a defect detection model according to an embodiment of the present disclosure is shown. Detailed Implementation

[0027] In the following, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings to enable those skilled in the art to readily implement them. Furthermore, for clarity, portions unrelated to the description of exemplary embodiments have been omitted from the drawings.

[0028] In this disclosure, it should be understood that terms such as “comprising” or “having” are intended to indicate the presence of features, figures, steps, behaviors, components, parts or combinations thereof disclosed in this specification, and are not intended to exclude the possibility of the presence or addition of one or more other features, figures, steps, behaviors, components, parts or combinations thereof.

[0029] It should also be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other. This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0030] As mentioned earlier, the use of drone inspection technology to inspect power equipment has gradually replaced manual inspection and become an important means of power transmission line safety inspection. However, this has also brought some problems. In mountainous areas, humid and foggy environments, the images of power equipment collected by drones are often blurry and surface defects are difficult to identify effectively. The inventors noticed that existing deep learning-based detection algorithms can be used for power equipment image processing, but these methods are usually computationally complex and rely on high-performance servers, making it difficult to achieve real-time processing on edge devices with limited computing power. This contradiction leads to insufficient real-time diagnosis and early warning capabilities for potential hazards in power transmission lines, and cannot meet the inspection needs under complex weather conditions.

[0031] This disclosure provides a method for establishing a lightweight defect detection model for power equipment, the method being implemented by a computer, and the method comprising: Obtain a training set, which includes multiple fog-free images of power equipment; The training set is input into a lightweight detection network model for training to obtain a power equipment defect detection model. The lightweight detection network model includes a backbone network, a neck network, and multiple detection heads. The backbone network includes multiple depthwise separable convolutional modules, and the neck network includes multiple region routing attention mechanism modules. The model training includes: For any fog-free image of power equipment in the training set, features are extracted from the fog-free image of power equipment using the multiple depthwise separable convolutional modules to obtain multiple power equipment feature maps with different resolutions. Enhanced feature fusion is performed on the multiple power equipment feature maps using the multiple region routing attention mechanism modules to obtain multiple enhanced fused feature maps with different resolutions. Multiple detection heads detect the multiple enhanced fused feature maps respectively, outputting multiple prediction information tensors corresponding to each enhanced fused feature map. Multiple prediction boxes are generated based on the multiple prediction information tensors, thereby obtaining multiple prediction boxes for the fog-free image of power equipment. The multiple prediction boxes of the fog-free image of power equipment are compared with the corresponding ground truth boxes, and the model parameters of the lightweight detection network model are adjusted.

[0032] This disclosure significantly improves the detection accuracy and speed of power equipment defects in hazy environments while achieving lightweight design. It can be deployed on edge devices such as drones, providing a complete visual perception solution for intelligent inspection of power systems from the edge side, thereby achieving the effect of real-time detection of potential power equipment hazards on transmission lines under complex weather conditions.

[0033] Figure 1 A flowchart illustrating a method for establishing a defect detection model for lightweight power equipment according to an embodiment of the present disclosure is shown, the method being implemented by a computer.

[0034] In this disclosure, the power equipment may be insulators, transformers, instrument transformers, relay protection devices, etc.

[0035] like Figure 1 As shown, the method includes steps S101 to S102.

[0036] In step S101, a training set is obtained, which includes multiple fog-free images of power equipment.

[0037] The fog-free image of the power equipment is obtained by defogging the foggy image of the power equipment collected in a foggy environment.

[0038] Further, the dehazing process includes: calculating the scene depth estimation modulation weight for each pixel in the foggy image of the power equipment; converting the foggy image of the power equipment from the RGB color space to the HSV color space to obtain the brightness and saturation of each pixel; setting the brightness coefficient and saturation coefficient of the corresponding pixel according to the scene depth estimation modulation weight; determining the scene depth estimation formula for the foggy image of the power equipment based on the brightness and saturation of each pixel in the foggy image of the power equipment and the brightness coefficient and saturation coefficient of the corresponding pixel; and obtaining the fog-free image of the power equipment based on the scene depth estimation formula and the atmospheric scattering model.

[0039] Furthermore, the scene depth estimation modulation weights for each pixel are calculated, including: for the foggy image of the power equipment, converting the foggy image of the power equipment from the RGB color space to the YCbCr color space to obtain the blue chromaticity component and the red chromaticity component of each pixel in the foggy image of the power equipment, and the blue chromaticity component and the red chromaticity component of the pixel forming the chromaticity vector of the corresponding pixel; calculating the Mahalanobis distance between the chromaticity vector of each pixel in the foggy image of the power equipment and the typical chromaticity vector of the power equipment, and mapping the Mahalanobis distance to the scene depth estimation modulation weights of the corresponding pixel.

[0040] According to embodiments of this disclosure, a foggy image of the power equipment is converted from the RGB color space to the YCbCr color space using a photometric function formula. The photometric function formula can be defined based on different standards, such as ITU-R BT.601, ITU-R BT.709, ITU-R BT.2020, etc., but is not limited thereto.

[0041] In one specific implementation, the photometric function formula defined based on the ITU-R BT.601 standard is as follows: ; ; in, For the pixels of the foggy image of the power equipment The blue chromaticity component at a given location refers to the degree to which the pixel color deviates from blue. For the pixels of the foggy image of the power equipment The red chromaticity component at the location refers to the degree to which the pixel color deviates from red; R represents the number of pixels in the foggy image of the power equipment. The intensity of the red light component at the location, where G is the pixel value of the foggy image of the power equipment. The intensity of the green light component at the location, where B represents the number of pixels in the foggy image of the power equipment. The intensity of the blue light component at that location.

[0042] According to embodiments of this disclosure, the blue and red chromaticity components of the pixel form the chromaticity vector of the pixel, including: [the following is a process described in the original text, but the translation is incomplete and requires further context.] chromaticity value Consider it as a two-dimensional vector: ;in, This is for the transpose operation.

[0043] The inventors noted that existing color attenuation prior (CAP) models typically use a scene depth estimation formula: To estimate the scene depth of each pixel in the foggy image, a scene depth map of the foggy image is obtained. For pixels Brightness at that location, For pixels Saturation at that point For pixels Scene depth at that location , , For the fixed parameters of the linear model, specifically, =0.121779, =0.959710, =−0.780245, these parameters are fixed values ​​obtained through supervised learning on a large dataset of foggy / foggy images.

[0044] However, in practical applications, existing schemes that use fixed parameters to estimate scene depth for dehazing are increasingly unable to meet the dehazing requirements of hazy images. There is an urgent need for a scheme to enhance the restoration quality of hazy images. This disclosure effectively enhances features by assigning a coefficient to the brightness and saturation of the pixels, thereby significantly improving the restoration quality of hazy images.

[0045] Specifically, the Mahalanobis distance is calculated between the chroma vector of each pixel in the foggy image of the power equipment and the typical chroma vector of the power equipment. This Mahalanobis distance measures the similarity between the chroma of a pixel and the typical chroma of the power equipment. The Mahalanobis distance is calculated using the following formula. : ; in, The larger the value, the greater the difference between the color of that pixel and the typical color of the power equipment; the smaller the value, the more similar the pixel is to the power equipment. The mean vector representing the typical chromaticity vector of electrical equipment can be obtained by collecting a large number of sample images and taking the mean. Color values ​​of power equipment The inverse of the covariance matrix represents , The degree of dispersion around its mean and the correlation between the two components.

[0046] According to embodiments of this disclosure, the calculated Mahalanobis distance is mapped to scene depth estimation modulation weights for the corresponding pixels, and the brightness and saturation coefficients of the corresponding pixels are set according to the scene depth estimation modulation weights. Specifically, for each pixel... With electrical equipment at this pixel Mahalanobis distance between typical chromaticity vectors Mapped to pixels Scene depth estimation modulation weights , ;in, It is a natural exponential function, which guarantees that the output value is in the range (0, 1]. The bandwidth parameter is used to control the rate of weight decay; then the modulation weights are estimated based on the scene depth. Calculate pixels The lightness coefficient and saturation coefficient, where the lightness coefficient is... The saturation coefficient is .

[0047] For the saturation coefficient, the following was introduced: It is because It is a negative number with a large absolute value (because the denser the fog and the farther the distance, the lower the saturation), if we want to achieve this without prior information (i.e. In the context of =0), If it can still play a strong role similar to the existing model, then the weight of the saturation term needs to be rescaled to compensate for the possible impact caused by the introduction of the modulation mechanism and to ensure the reasonableness of the numerical range of the depth estimation in the background region.

[0048] in, The closer a value is to 1, the better the pixel's chromaticity matches the typical chromaticity of the power equipment; the closer a value is to 0, the greater the difference between the pixel's chromaticity and the typical chromaticity of the power equipment.

[0049] Therefore, based on the brightness and saturation of each pixel in the foggy image of the power equipment, as well as the brightness coefficient and saturation coefficient of the corresponding pixel, the scene depth estimation formula for the foggy image of the power equipment is determined, specifically including: ,in, This is the new scene depth estimate.

[0050] According to embodiments of this disclosure, based on the scene depth estimation formula and the atmospheric scattering model... A fog-free image of the power equipment was obtained.

[0051] in, For pixels The power equipment there has a foggy image. For pixels The image of the power equipment is free of fog. A is the global atmospheric light value (usually selected from the brightest (i.e., deepest) 0.1% pixel in the scene depth map of the power equipment, and then the pixel with the highest brightness value is found in the corresponding position in the original foggy image as A). This is the atmospheric scattering coefficient (the larger the value, the higher the degree of smog); For pixels Projection rate at that location , The attenuation factor represents the remaining proportion of light that passes through the haze from the scene point and reaches the camera (drone).

[0052] Therefore, a fog-free image of power equipment can be obtained using the formula: get.

[0053] In step S102, the training set is input into the lightweight detection network model for model training to obtain a power equipment defect detection model. The lightweight detection network model includes a backbone network, a neck network, and multiple detection heads. The backbone network includes multiple depthwise separable convolutional modules, and the neck network includes multiple region routing attention mechanism modules.

[0054] Specifically, for any fog-free image of power equipment in the training set, features are extracted from the fog-free image of power equipment using the multiple depthwise separable convolutional modules to obtain multiple power equipment feature maps with different resolutions; the multiple region routing attention mechanism modules are used to perform enhanced feature fusion on the multiple power equipment feature maps to obtain multiple enhanced fused feature maps with different resolutions; the multiple detection heads detect the multiple enhanced fused feature maps respectively, and output multiple prediction information tensors corresponding to each of the multiple enhanced fused feature maps; based on the multiple prediction information tensors, multiple prediction boxes of the corresponding enhanced fused feature maps are generated, thereby obtaining multiple prediction boxes of the fog-free image of power equipment; the multiple prediction boxes of the fog-free image of power equipment are compared with the corresponding ground truth boxes, and the model parameters of the lightweight detection network model are adjusted.

[0055] According to embodiments of this disclosure, the step of extracting features from the fog-free image of the power equipment using the plurality of depthwise separable convolutional modules to obtain multiple power equipment feature maps with different resolutions includes: The fog-free image of the power equipment is subjected to feature extraction by a series of sequentially set depthwise separable convolutional modules. The first depthwise separable convolutional module uses the fog-free image of the power equipment as the image to be processed to extract features and obtain a power equipment feature map of the corresponding resolution. One or more depthwise separable convolutional modules other than the first depthwise separable convolutional module use the power equipment feature map of the corresponding resolution output by the previous depthwise separable convolutional module as the image to be processed to extract features and obtain a power equipment feature map of the corresponding resolution.

[0056] According to embodiments of this disclosure, the image to be processed includes multiple input channels and multiple spatial locations, the input channels covering the multiple spatial locations.

[0057] Specifically, when the depthwise separable convolution module extracts features from the image to be processed, it performs two-dimensional convolution on each input channel of the image to be processed to generate an intermediate feature map group corresponding to the input channel; for multiple spatial locations covered by the input channel, it performs point-by-point convolution on the intermediate feature map groups corresponding to the multiple input channels respectively to obtain a power equipment feature map of the corresponding resolution. In this disclosure, the image to be processed for the depthwise separable convolution module is either a fog-free image of the power equipment input or a power equipment feature map of the corresponding resolution input from the previous depthwise separable convolution module.

[0058] Specifically, such as Figure 2 As shown, Figure 2 A schematic diagram of a backbone network according to an embodiment of the present disclosure is shown; Figure 2 The example uses three depthwise separable convolutional modules, which will be understood by those skilled in the art. Figure 2 The number of depthwise separable convolutional modules shown is not intended to limit the scope of protection of this disclosure. The depthwise separable convolutional modules can also be 5, 7, etc.

[0059] exist Figure 2In the backbone network, three depthwise separable convolutional modules are sequentially arranged: depthwise separable convolutional module 1, depthwise separable convolutional module 2, and depthwise separable convolutional module 3. For the input fog-free image of the power equipment, depthwise separable convolutional module 1 processes it to obtain a power equipment feature map of the first resolution. This first-resolution feature map is output from the backbone network to the neck network and simultaneously input to depthwise separable convolutional module 2 for feature extraction, resulting in a power equipment feature map of the second resolution. Similarly, this second-resolution feature map is output from the backbone network to the neck network and simultaneously input to depthwise separable convolutional module 3 for feature extraction, resulting in a power equipment feature map of the third resolution. That is, through the three depthwise separable convolutional modules arranged in the above sequence, three power equipment feature maps of different resolutions are obtained, with the first resolution being greater than the second resolution, and the second resolution being greater than the third resolution.

[0060] It is known that in deep convolutional networks, the number of output channels increases as the resolution decreases; for example, in... Figure 2 In the example shown, the first resolution power equipment feature map output by the first depthwise separable convolution module has a high resolution, but its output channel count is low, for example, it only extracts some detailed information; while the second resolution power equipment feature map output by the second depthwise separable convolution module has a lower resolution than the first resolution, but its output channel count is increased, and it can extract more complex patterns; the third resolution power equipment feature map output by the third depthwise separable convolution module has a lower resolution than the second resolution, but its output channel count is the highest, thus it can extract the overall feature information of the object.

[0061] This disclosure utilizes a depthwise separable convolution module for feature extraction. By extracting features channel by channel and performing pointwise convolution, the computational load for feature extraction of images is greatly reduced.

[0062] In this disclosure, after obtaining feature maps of power equipment at multiple resolutions, these feature maps need to be further processed by the neck network.

[0063] According to embodiments of this disclosure, the step of performing enhanced feature fusion on the feature maps of the multiple power devices through the multiple regional routing attention mechanism modules to obtain multiple enhanced fused feature maps with different resolutions includes: The M power equipment feature maps are arranged in ascending order of resolution; The first and second power equipment feature maps are fused to obtain the first intermediate fused feature map. Starting with the first intermediate fused feature map and the third power equipment feature map, the following steps are performed sequentially: The nth intermediate fused feature map is fused with the mth power equipment feature map to obtain the (n+1)th intermediate fused feature map; the (n+1)th intermediate fused feature map is then fused with the (m+1)th power equipment feature map, and so on, until the (M-2)th intermediate fused feature map is fused with the Mth power equipment feature map, thus obtaining the first enhanced fused feature map. During the final feature fusion execution, the corresponding regional routing attention mechanism module is used. , ; Starting with the first enhanced fused feature map and the (M-2)th intermediate fused feature map, the following steps are performed sequentially: The p-th enhanced fused feature map and the q-th intermediate fused feature map are fused using the corresponding region routing attention mechanism module to obtain the (p+1)-th enhanced fused feature map; the (p+1)-th enhanced fused feature map is then fused with the (q-1)-th intermediate fused feature map using the corresponding region routing attention mechanism module, until the feature fusion of the (M-2)-th and first intermediate fused feature maps is completed, thus obtaining the (M-1)-th enhanced fused feature map. , ; The M-1th enhanced fusion feature map and the first power equipment feature map are fused using the corresponding regional routing attention mechanism module to obtain the Mth enhanced fusion feature map.

[0064] That is, when this disclosure uses multiple regional routing attention mechanism modules to perform enhanced feature fusion on multiple power equipment feature maps, it does not simply use one regional routing attention mechanism module to perform enhanced feature fusion on any two power equipment feature maps. Instead, it first determines a group of power equipment feature maps arranged in order of increasing resolution, and then fuses the power equipment feature maps in the group in order to obtain multiple intermediate fused feature maps. Finally, it uses multiple regional routing attention mechanism modules to obtain multiple final enhanced fused feature maps based on different combinations of intermediate fused feature maps, power equipment feature maps, and enhanced fused feature maps.

[0065] To more clearly illustrate the use of multiple regional routing attention mechanism modules to enhance feature fusion of feature maps of multiple power devices, two specific implementation methods are provided below. Those skilled in the art should understand that these two specific implementation methods are not intended to limit the scope of protection of this disclosure.

[0066] Figure 3 This diagram illustrates an enhanced feature fusion of feature maps of multiple power devices using multiple area routing attention mechanism modules, according to an embodiment of the present disclosure.

[0067] In this embodiment, it is assumed that there are 3 power equipment feature maps, and they are ordered from low to high resolution as follows: Power Equipment Features Figure 1 Characteristics of power equipment Figure 2 Characteristics of power equipment Figure 3 In the neck network, feature fusion is performed on the feature maps of the three power devices through three regional routing attention mechanism modules, that is, at least three regional routing attention mechanism modules are provided in the neck network.

[0068] like Figure 3 As shown, the characteristics of power equipment Figure 1 Characteristics of power equipment Figure 2 Perform feature fusion (usually ordinary feature fusion, for example, combining features of power equipment). Figure 1 After upsampling and power equipment characteristics Figure 2 Alternatively, feature fusion can be achieved using a region routing attention mechanism module (this disclosure does not impose any restrictions on this method) to obtain intermediate fused features. Figure 1 Then, the intermediate fusion features are fused. Figure 1 and characteristics of power equipment Figure 3 Feature fusion is performed through the region routing attention mechanism module 1 to obtain enhanced fused features. Figure 1 Further enhance fusion features Figure 1 and intermediate fusion features Figure 1 Feature fusion is performed through the region routing attention mechanism module 2 to obtain enhanced fused features. Figure 2 Finally, the fusion features will be enhanced. Figure 2 and characteristics of power equipment Figure 1 Feature fusion is performed through the region routing attention mechanism module 3 to obtain enhanced fused features. Figure 3 .

[0069] Figure 4 This diagram illustrates another embodiment of the present disclosure that utilizes multiple regional routing attention mechanism modules to perform enhanced feature fusion on feature maps of multiple power devices.

[0070] In this embodiment, it is assumed that there are 5 power equipment feature maps, and they are ordered from low to high resolution as follows: Power Equipment Features Figure 1 Characteristics of power equipment Figure 2 Characteristics of power equipment Figure 3 Characteristics of power equipment Figure 4 Characteristics of power equipment Figure 5 In the neck network, feature fusion is performed on the feature maps of the five power devices through five regional routing attention mechanism modules, that is, at least five regional routing attention mechanism modules are provided in the neck network.

[0071] like Figure 4 As shown, the characteristics of power equipment Figure 1 Characteristics of power equipment Figure 2 Perform feature fusion (usually ordinary feature fusion, for example, combining features of power equipment). Figure 1 After upsampling and power equipment characteristics Figure 2 Alternatively, feature fusion can be achieved using a region routing attention mechanism module (this disclosure does not impose any restrictions on this method) to obtain intermediate fused features. Figure 1 Then, the intermediate fusion features are fused. Figure 1 and characteristics of power equipment Figure 3 Perform feature fusion to obtain intermediate fused features. Figure 2 Then, the intermediate fusion features are... Figure 2 and characteristics of power equipment Figure 4 Perform feature fusion to obtain intermediate fused features. Figure 3 Then, the intermediate fusion features are... Figure 3 and characteristics of power equipment Figure 5 Feature fusion is performed through the region routing attention mechanism module 1 to obtain enhanced fused features. Figure 1 This will enhance the integration features. Figure 1 and intermediate fusion features Figure 3 Feature fusion is performed through the region routing attention mechanism module 2 to obtain enhanced fused features. Figure 2 This will enhance the integration features. Figure 2 and intermediate fusion features Figure 2 Feature fusion is performed through the region routing attention mechanism module 3 to obtain enhanced fused features. Figure 3 This will enhance the integration features. Figure 3 and intermediate fusion features Figure 1 Feature fusion is performed through the region routing attention mechanism module 4 to obtain enhanced fused features. Figure 4 Finally, the fusion features will be enhanced. Figure 4 and characteristics of power equipment Figure 1 Feature fusion is performed through the region routing attention mechanism module 5 to obtain enhanced fused features. Figure 5 .

[0072] According to embodiments of this disclosure, when fusing features from two feature maps to be processed using the region routing attention mechanism module, the method further includes: concatenating the two feature maps to be processed to obtain a concatenated feature map; uniformly dividing the concatenated feature map into multiple non-overlapping regions; performing average pooling on each non-overlapping region to generate a region-level query vector, a region-level key vector, and a region-level value vector for each non-overlapping region in the concatenated feature map; calculating the correlation between any two non-overlapping regions based on the region-level query vectors and region-level key vectors of the multiple non-overlapping regions to form a region-related adjacency matrix; for the non-overlapping regions, using Top-k routing to select the corresponding k most relevant regions based on the region-related adjacency matrix; calculating the region-level attention score between the non-overlapping region and each of the corresponding k most relevant regions, and using the calculated region-level attention score to perform weighted fusion on the region-level value vectors of the k most relevant regions to obtain the corresponding enhanced fused feature map.

[0073] Specifically, the query vector, key vector, and value vector of each pixel in the stitched feature map are generated by linearly projecting the stitched feature map to obtain the query vector Q, key vector K, and value vector V of each pixel in the stitched feature map.

[0074] According to embodiments of this disclosure, a region-level query vector is calculated for each of the plurality of feature map regions. Region-level key vectors Regional value vector This can be achieved through the following formula: , , , This is a regional-level average pooling.

[0075] According to embodiments of this disclosure, based on the region-level query vectors and region-level key vectors of the plurality of non-overlapping regions, the correlation between any two non-overlapping regions is calculated to form a region correlation adjacency matrix. The calculation is performed using the following formula: ;in, for The transpose of .

[0076] According to embodiments of this disclosure, for a non-overlapping region, based on the region-related adjacency matrix... The top-k routes are used to filter out the k most relevant regions, using the following formula: ,in, This represents an indexing operation that retains the top k maximum values ​​row by row, where I is the routing index matrix, which represents the K other non-overlapping regions that are most relevant to the non-overlapping regions.

[0077] According to embodiments of this disclosure, an attention mechanism is established among the k feature map regions to obtain the corresponding enhanced fusion feature map, which is achieved through the following formula: ,in, This represents the region-level query vector for all tokens within the i-th non-overlapping region. This represents the region-level key vectors of the k most relevant regions selected from the routing index matrix I. This is a scaling factor used to prevent gradient explosion. Let be the region-level value vectors of the k most relevant regions.

[0078] According to embodiments of this disclosure, the plurality of detection heads respectively detect the plurality of enhanced fusion feature maps and output a plurality of prediction information tensors corresponding to each of the plurality of enhanced fusion feature maps, including: for any enhanced fusion feature map, the detection head outputs a plurality of prediction information tensors corresponding to each spatial position in the enhanced fusion feature map and a plurality of corresponding prior boxes, wherein the prediction information tensors correspond to the prior boxes.

[0079] For each enhanced fusion feature map, the corresponding detection head outputs multiple prediction information tensors based on each spatial location on the enhanced fusion feature map and the corresponding multiple prior boxes. Each prediction information tensor includes: class confidence. The normalized offset value of the spatial position relative to the center of the prior box. The logarithmic scaling factor of the spatial position relative to the width and height of the prior bounding box. .

[0080] Specifically, the number of detection heads is consistent with the number of augmented fusion feature maps; that is, one detection head needs to be allocated to each augmented fusion feature map for detection. Multiple prior boxes are obtained by clustering the width and height of all ground truth boxes in the training set (e.g., using the k-means clustering algorithm). Each spatial location in the augmented fusion feature map shares this group of prior boxes. For example, assuming an augmented fusion feature map has a resolution of 80x80, it has 6400 spatial locations, and there are 3 prior boxes. These 3 prior boxes need to appear once at each of these 6400 spatial locations. Since the corresponding detection head outputs a prediction information tensor for each spatial location and each prior box in the multiple prior boxes, the corresponding detection head can obtain multiple prediction information tensors for each spatial location.

[0081] According to embodiments of this disclosure, generating multiple prediction boxes for corresponding enhanced fusion feature maps based on the multiple prediction information tensors includes: generating multiple prediction boxes corresponding to each spatial location based on the multiple prediction information tensors corresponding to each spatial location in the enhanced fusion feature map, wherein the prediction boxes... The formula is expressed as follows: Where, , and , where a represents the index number of the prior box. Represents a two-dimensional real number space. Let be the baseline width of the a-th prior box. The reference height of the a-th prior box.

[0082] According to embodiments of this disclosure, multiple predicted bounding boxes of the fog-free image of the power equipment are compared with their corresponding ground truth bounding boxes, and the model parameters of the lightweight detection network model are adjusted. This can be achieved through the following steps: Calculate the prediction box B and the true value box B gt intersection ratio Use the following formula: ;calculate Loss value: ;in, For those with penalties Loss value, It is an adjustable exponent; then... The total loss is obtained by adding the loss value to the classification loss and the confidence loss. , ;in, For class loss weights, For category cross-entropy loss, Weight the target confidence level loss. Foreground / background binary classification, cross-entropy loss is used. for Boundary regression loss.

[0083] Subsequently, using the loss function as the optimization objective, the spatial difference between the predicted bounding box and the ground truth bounding box is quantitatively evaluated, and the accurate localization error is calculated. This error gradient is backpropagated to the lightweight detection network model through a backpropagation mechanism to adjust the model parameters, thereby gradually improving the accuracy of bounding box regression and localization.

[0084] This disclosure significantly improves the detection accuracy and speed of power equipment defects in hazy environments while achieving lightweight design. It can be deployed on edge devices such as drones, providing a complete visual perception solution for intelligent inspection of power systems. It can detect potential hazards of power equipment on transmission lines in real time under complex weather conditions, significantly improving the reliability, safety and operation and maintenance efficiency of the power grid, and laying a scalable technical foundation for building smart grids and the energy internet.

[0085] Figure 5 A flowchart illustrating a defect detection method for lightweight electrical equipment according to an embodiment of the present disclosure is shown.

[0086] like Figure 5 As shown, the method includes step S501, acquiring a fog-free image of the power equipment to be processed, inputting the fog-free image of the power equipment to be processed into the power equipment defect detection model, and obtaining the power equipment defect detection result, which includes the category, location and confidence level of the power equipment.

[0087] The power equipment defect detection model is established through the following steps: Obtain a training set, which includes multiple fog-free images of power equipment; The training set is input into a lightweight detection network model for training to obtain a power equipment defect detection model. The lightweight detection network model includes a backbone network, a neck network, and multiple detection heads. The backbone network includes multiple depthwise separable convolutional modules, and the neck network includes multiple region routing attention mechanism modules. The model training includes: For any fog-free image of power equipment in the training set, features are extracted from the fog-free image of power equipment using the multiple depthwise separable convolutional modules to obtain multiple power equipment feature maps with different resolutions. Enhanced feature fusion is performed on the multiple power equipment feature maps using the multiple region routing attention mechanism modules to obtain multiple enhanced fused feature maps with different resolutions. Multiple detection heads detect the multiple enhanced fused feature maps respectively, outputting multiple prediction information tensors corresponding to each enhanced fused feature map. Multiple prediction boxes are generated based on the multiple prediction information tensors, thereby obtaining multiple prediction boxes for the fog-free image of power equipment. The multiple prediction boxes of the fog-free image of power equipment are compared with the corresponding ground truth boxes, and the model parameters of the lightweight detection network model are adjusted.

[0088] According to embodiments of this disclosure, acquiring a fog-free image of the power equipment to be processed includes: A foggy image of the power equipment to be processed is acquired; for the foggy image of the power equipment to be processed, the scene depth estimation modulation weight of each pixel is calculated; the foggy image of the power equipment to be processed is converted from the RGB color space to the HSV color space to obtain the brightness and saturation of each pixel; the brightness coefficient and saturation coefficient of the corresponding pixel are set according to the scene depth estimation modulation weight; the scene depth estimation formula of the foggy image of the power equipment to be processed is determined according to the brightness and saturation of each pixel in the foggy image of the power equipment to be processed and the corresponding brightness coefficient and saturation coefficient; the fog-free image of the power equipment to be processed is obtained according to the scene depth estimation formula and the atmospheric scattering model.

[0089] According to embodiments of this disclosure, the calculation of scene depth estimation modulation weights for each pixel includes: for the foggy image of the power equipment to be processed, converting the foggy image of the power equipment to be processed from the RGB color space to the YCbCr color space to obtain the blue chromaticity component and the red chromaticity component of each pixel in the foggy image of the power equipment to be processed, wherein the blue chromaticity component and the red chromaticity component of the pixel form the chromaticity vector of the corresponding pixel; calculating the Mahalanobis distance between the chromaticity vector of each pixel in the foggy image of the power equipment to be processed and the typical chromaticity vector of the power equipment, and mapping the Mahalanobis distance to the scene depth estimation modulation weights of the corresponding pixel.

[0090] This disclosure can be deployed on edge devices such as drones, thereby providing a complete visual perception solution for power system monitoring at the edge, improving the intelligent inspection solution of the power system, greatly reducing the dependence on high-performance processing equipment, and enabling the real-time detection of potential hazards in power equipment on transmission lines under complex weather conditions.

[0091] Figure 6 A structural diagram of a lightweight electrical equipment defect detection model building apparatus 600 is shown according to an embodiment of the present disclosure.

[0092] like Figure 6 As shown, the device 600 includes: an acquisition module 610, a model training module 620, and a first dehazing processing module 630.

[0093] The acquisition module 610 is configured to acquire a training set, which includes multiple fog-free images of power equipment.

[0094] The fog-free image of the power equipment is obtained by defogging the foggy image of the power equipment collected in a foggy environment.

[0095] The model training module 620 is configured to input the training set into a lightweight detection network model for model training to obtain a power equipment defect detection model. The lightweight detection network model includes a backbone network, a neck network, and multiple detection heads. The backbone network includes multiple depthwise separable convolutional modules, and the neck network includes multiple region routing attention mechanism modules.

[0096] The model training includes: for any fog-free image of power equipment in the training set, extracting features from the fog-free image of power equipment using the multiple depthwise separable convolutional modules to obtain multiple power equipment feature maps with different resolutions; performing enhanced feature fusion on the multiple power equipment feature maps using the multiple region routing attention mechanism modules to obtain multiple enhanced fused feature maps with different resolutions; detecting the multiple enhanced fused feature maps by the multiple detection heads respectively, and outputting multiple prediction information tensors corresponding to each of the multiple enhanced fused feature maps; generating multiple prediction boxes for the corresponding enhanced fused feature maps based on the multiple prediction information tensors, thereby obtaining multiple prediction boxes for the fog-free image of power equipment; comparing the multiple prediction boxes of the fog-free image of power equipment with the corresponding ground truth boxes, and adjusting the model parameters of the lightweight detection network model.

[0097] The first dehazing processing module 630 is configured to: calculate the scene depth estimation modulation weight for each pixel in the foggy image of the power equipment; convert the foggy image of the power equipment from the RGB color space to the HSV color space to obtain the brightness and saturation of each pixel; set the brightness coefficient and saturation coefficient of the corresponding pixel according to the scene depth estimation modulation weight; determine the scene depth estimation formula for the foggy image of the power equipment based on the brightness and saturation of each pixel in the foggy image of the power equipment and the brightness coefficient and saturation coefficient of the corresponding pixel; and obtain the fog-free image of the power equipment based on the scene depth estimation formula and the atmospheric scattering model.

[0098] Figure 7 A structural diagram of a lightweight power device defect detection chip according to an embodiment of the present disclosure is shown.

[0099] like Figure 7 As shown, the defect detection chip 700 includes: a detection module 710 and a second defogging processing module 720.

[0100] The detection module 710 is configured to acquire a fog-free image of the power equipment to be processed, input the fog-free image of the power equipment to be processed into the power equipment defect detection model, and obtain the power equipment defect detection result, which includes the category, location and confidence level of the power equipment.

[0101] The power equipment defect detection model is established through the following steps: Obtain a training set, which includes multiple fog-free images of power equipment; The training set is input into a lightweight detection network model for training to obtain a power equipment defect detection model. The lightweight detection network model includes a backbone network, a neck network, and multiple detection heads. The backbone network includes multiple depthwise separable convolutional modules, and the neck network includes multiple region routing attention mechanism modules. The model training includes: For any fog-free image of power equipment in the training set, features are extracted from the fog-free image of power equipment using the multiple depthwise separable convolutional modules to obtain multiple power equipment feature maps with different resolutions. Enhanced feature fusion is performed on the multiple power equipment feature maps using the multiple region routing attention mechanism modules to obtain multiple enhanced fused feature maps with different resolutions. Multiple detection heads detect the multiple enhanced fused feature maps respectively, outputting multiple prediction information tensors corresponding to each enhanced fused feature map. Multiple prediction boxes are generated based on the multiple prediction information tensors, thereby obtaining multiple prediction boxes for the fog-free image of power equipment. The multiple prediction boxes of the fog-free image of power equipment are compared with the corresponding ground truth boxes, and the model parameters of the lightweight detection network model are adjusted.

[0102] The second dehazing module 720 is configured to: acquire a foggy image of the power equipment to be processed; calculate the scene depth estimation modulation weight for each pixel in the foggy image of the power equipment to be processed; convert the foggy image of the power equipment to be processed from the RGB color space to the HSV color space to obtain the brightness and saturation of each pixel; set the brightness coefficient and saturation coefficient of the corresponding pixel according to the scene depth estimation modulation weight; determine the scene depth estimation formula for the foggy image of the power equipment to be processed based on the brightness and saturation of each pixel in the foggy image of the power equipment to be processed and the brightness coefficient and saturation coefficient of the corresponding pixel; and obtain the fog-free image of the power equipment to be processed based on the scene depth estimation formula and the atmospheric scattering model.

[0103] This disclosure enables complex detection schemes with extremely low power consumption, reducing the load on edge devices such as drones and thus significantly extending their battery life. It allows drones to complete equipment defect identification (such as insulator breakage or bolt detachment) the moment they are captured, without waiting for data to be transmitted back to the control center.

[0104] Figure 8A schematic diagram of the structure of a computer system suitable for implementing a method for establishing a defect detection model according to an embodiment of the present disclosure is shown.

[0105] like Figure 8 As shown, the computer system includes a processing unit that can execute various methods described above based on a program stored in a read-only memory (ROM) or a program loaded from a storage portion into a random access memory (RAM). The RAM also stores various programs and data required for the operation of the computer system. The processing unit, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0106] The following components are connected to the I / O interface: input sections including keyboards, mice, etc.; output sections including cathode ray tubes (CRTs), liquid crystal displays (LCDs), and speakers; storage sections including hard disks; and communication sections including network interface cards such as LAN cards and modems. The communication section performs communication processes via a network such as the Internet. Drives are also connected to the I / O interface as needed. Removable media, such as disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on the drive as needed so that computer programs read from them can be installed into the storage section as required. The processing unit can be implemented as a CPU, GPU, TPU, FPGA, NPU, etc.

[0107] In particular, according to embodiments of this disclosure, the methods described above can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program containing program code for performing the methods described above. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium.

[0108] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0109] The units or modules described in the embodiments of this disclosure can be implemented in software or programmable hardware. The described units or modules can also be located in a processor, and the names of these units or modules do not necessarily constitute a limitation on the unit or module itself.

[0110] In another aspect, this disclosure also provides a computer-readable storage medium, which may be a computer-readable storage medium included in the electronic device or computer system described above; or it may be a standalone computer-readable storage medium not assembled into a device. The computer-readable storage medium stores one or more programs, which are used by one or more processors to perform the methods described in this disclosure.

[0111] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

Claims

1. A method for establishing a defect detection model for lightweight power equipment, characterized in that, The method is implemented by a computer, and the method includes: Obtain a training set, which includes multiple fog-free images of power equipment; The training set is input into a lightweight detection network model for training to obtain a power equipment defect detection model. The lightweight detection network model includes a backbone network, a neck network, and multiple detection heads. The backbone network includes multiple depthwise separable convolutional modules, and the neck network includes multiple region routing attention mechanism modules. The model training includes: For any fog-free image of power equipment in the training set, features are extracted from the fog-free image of power equipment using the multiple depthwise separable convolutional modules to obtain multiple power equipment feature maps with different resolutions. Enhanced feature fusion is performed on the multiple power equipment feature maps using the multiple region routing attention mechanism modules to obtain multiple enhanced fused feature maps with different resolutions. Multiple detection heads detect the multiple enhanced fused feature maps respectively, outputting multiple prediction information tensors corresponding to each enhanced fused feature map. Multiple prediction boxes are generated based on the multiple prediction information tensors, thereby obtaining multiple prediction boxes for the fog-free image of power equipment. The multiple prediction boxes of the fog-free image of power equipment are compared with the corresponding ground truth boxes, and the model parameters of the lightweight detection network model are adjusted.

2. The method for establishing according to claim 1, characterized in that, The fog-free image of the power equipment is obtained by defogging a foggy image of the power equipment captured in a hazy environment; wherein, the defogging process includes: For the foggy image of the power equipment, calculate the scene depth estimation modulation weight for each pixel; The foggy image of the power equipment is converted from the RGB color space to the HSV color space to obtain the brightness and saturation of each pixel. Based on the scene depth estimation modulation weight, the brightness coefficient and saturation coefficient of the corresponding pixels are set; Based on the brightness and saturation of each pixel in the foggy image of the power equipment, as well as the brightness coefficient and saturation coefficient of the corresponding pixel, the scene depth estimation formula for the foggy image of the power equipment is determined. The fog-free image of the power equipment is obtained based on the scene depth estimation formula and the atmospheric scattering model.

3. The method for establishing according to claim 2, characterized in that, The calculation of the scene depth estimation modulation weights for each pixel includes: For the foggy image of the power equipment, the foggy image of the power equipment is converted from the RGB color space to the YCbCr color space to obtain the blue chromaticity component and the red chromaticity component of each pixel in the foggy image of the power equipment. The blue chromaticity component and the red chromaticity component of the pixel form the chromaticity vector of the corresponding pixel. Calculate the Mahalanobis distance between the chroma vector of each pixel in the foggy image of the power equipment and the typical chroma vector of the power equipment, and map the Mahalanobis distance to the scene depth estimation modulation weight of the corresponding pixel.

4. The method for establishing according to claim 1, characterized in that, The process of extracting features from the fog-free image of the power equipment using the multiple depthwise separable convolutional modules yields multiple power equipment feature maps with different resolutions, including: The fog-free image of the power equipment is subjected to feature extraction by a series of sequentially set depthwise separable convolutional modules. The first depthwise separable convolutional module uses the fog-free image of the power equipment as the image to be processed to extract features and obtain a power equipment feature map of the corresponding resolution. One or more depthwise separable convolutional modules other than the first depthwise separable convolutional module use the power equipment feature map of the corresponding resolution output by the previous depthwise separable convolutional module as the image to be processed to extract features and obtain a power equipment feature map of the corresponding resolution.

5. The method according to claim 4, characterized in that: The image to be processed includes multiple input channels and multiple spatial locations, and the input channels cover the multiple spatial locations; The method further includes: when the depth-separable convolution module extracts features from the image to be processed, performing two-dimensional convolution on each input channel of the image to be processed to generate an intermediate feature map group corresponding to the input channel; and for multiple spatial locations covered by the input channel, performing point-by-point convolution on the intermediate feature map groups corresponding to the multiple input channels respectively to obtain a power equipment feature map of corresponding resolution.

6. The method for establishing according to claim 1, characterized in that, The enhancement feature fusion of the multiple power equipment feature maps through the multiple regional routing attention mechanism modules yields multiple enhanced fused feature maps with different resolutions, including: The M power equipment feature maps are arranged in ascending order of resolution; The first and second power equipment feature maps are fused to obtain the first intermediate fused feature map. Starting with the first intermediate fused feature map and the third power equipment feature map, the following steps are performed sequentially: The nth intermediate fused feature map is fused with the mth power equipment feature map to obtain the (n+1)th intermediate fused feature map; the (n+1)th intermediate fused feature map is then fused with the (m+1)th power equipment feature map, and so on, until the (M-2)th intermediate fused feature map is fused with the Mth power equipment feature map, thus obtaining the first enhanced fused feature map. During the final feature fusion execution, the corresponding regional routing attention mechanism module is used. , ; Starting with the first enhanced fused feature map and the (M-2)th intermediate fused feature map, the following steps are performed sequentially: The p-th enhanced fused feature map and the q-th intermediate fused feature map are fused using the corresponding region routing attention mechanism module to obtain the (p+1)-th enhanced fused feature map; the (p+1)-th enhanced fused feature map is then fused with the (q-1)-th intermediate fused feature map using the corresponding region routing attention mechanism module, until the feature fusion of the (M-2)-th and first intermediate fused feature maps is completed, thus obtaining the (M-1)-th enhanced fused feature map. , ; The M-1th enhanced fusion feature map and the first power equipment feature map are fused using the corresponding regional routing attention mechanism module to obtain the Mth enhanced fusion feature map.

7. The method for establishing according to claim 6, characterized in that, When using the region routing attention mechanism module to perform feature fusion on two feature maps to be processed, the method further includes: The two feature maps to be processed are stitched together to obtain a stitched feature map; The stitched feature map is evenly divided into multiple non-overlapping regions; Average pooling is performed on each non-overlapping region to generate a region-level query vector, a region-level key vector, and a region-level value vector for each non-overlapping region in the stitched feature map. Based on the region-level query vector and region-level key vector of the multiple non-overlapping regions, the correlation between any two non-overlapping regions is calculated to form a region correlation adjacency matrix. For the non-overlapping regions, based on the region relevance adjacency matrix, the top-k routes are used to select the k most relevant regions. The region-level attention score between the non-overlapping regions and each of the k most relevant regions is calculated, and the calculated region-level attention scores are used to weight and fuse the region-level value vectors of the k most relevant regions to obtain the corresponding enhanced fusion feature map.

8. The method for establishing according to claim 1, characterized in that: The plurality of detection heads respectively detect the plurality of enhanced fusion feature maps and output a plurality of prediction information tensors corresponding to each of the plurality of enhanced fusion feature maps, including: for any enhanced fusion feature map, the detection head outputs a plurality of prediction information tensors corresponding to each spatial position in the enhanced fusion feature map and a plurality of prior boxes, wherein the prediction information tensors correspond to the prior boxes; The step of generating multiple prediction boxes for the corresponding enhanced fusion feature map based on the multiple prediction information tensors includes: generating multiple prediction boxes corresponding to each spatial location based on the multiple prediction information tensors corresponding to each spatial location in the enhanced fusion feature map.

9. The method for establishing according to claim 8, characterized in that, The prediction information tensor includes: class confidence, normalized offset, and logarithmic scaling factor.

10. A method for defect detection in lightweight power equipment, characterized in that, The method includes: A fog-free image of the power equipment to be processed is acquired, and the fog-free image of the power equipment to be processed is input into the power equipment defect detection model to obtain the power equipment defect detection result. The power equipment defect detection result includes the category, location and confidence level of the power equipment. The power equipment defect detection model is established through the following steps: Obtain a training set, which includes multiple fog-free images of power equipment; The training set is input into a lightweight detection network model for training to obtain a power equipment defect detection model. The lightweight detection network model includes a backbone network, a neck network, and multiple detection heads. The backbone network includes multiple depthwise separable convolutional modules, and the neck network includes multiple region routing attention mechanism modules. The model training includes: For any fog-free image of power equipment in the training set, features are extracted from the fog-free image of power equipment using the multiple depthwise separable convolutional modules to obtain multiple power equipment feature maps with different resolutions. Enhanced feature fusion is performed on the multiple power equipment feature maps using the multiple region routing attention mechanism modules to obtain multiple enhanced fused feature maps with different resolutions. Multiple detection heads detect the multiple enhanced fused feature maps respectively, outputting multiple prediction information tensors corresponding to each enhanced fused feature map. Multiple prediction boxes are generated based on the multiple prediction information tensors, thereby obtaining multiple prediction boxes for the fog-free image of power equipment. The multiple prediction boxes of the fog-free image of power equipment are compared with the corresponding ground truth boxes, and the model parameters of the lightweight detection network model are adjusted.

11. The defect detection method according to claim 10, characterized in that, The process of acquiring a fog-free image of the power equipment to be processed includes: Acquire foggy images of the power equipment to be processed; For the foggy image of the power equipment to be processed, calculate the scene depth estimation modulation weight for each pixel; The foggy image of the power equipment to be processed is converted from the RGB color space to the HSV color space to obtain the brightness and saturation of each pixel. Based on the scene depth estimation modulation weight, the brightness coefficient and saturation coefficient of the corresponding pixels are set; Based on the brightness and saturation of each pixel in the foggy image of the power equipment to be processed, as well as the brightness coefficient and saturation coefficient of the corresponding pixel, the scene depth estimation formula of the foggy image of the power equipment to be processed is determined. The fog-free image of the power equipment to be processed is obtained based on the scene depth estimation formula and the atmospheric scattering model.

12. The defect detection method according to claim 11, characterized in that, The calculation of the scene depth estimation modulation weights for each pixel includes: For the foggy image of the power equipment to be processed, the foggy image of the power equipment to be processed is converted from the RGB color space to the YCbCr color space to obtain the blue chromaticity component and the red chromaticity component of each pixel in the foggy image of the power equipment to be processed. The blue chromaticity component and the red chromaticity component of the pixel form the chromaticity vector of the corresponding pixel. The Mahalanobis distance between the chroma vector of each pixel in the foggy image of the power equipment to be processed and the typical chroma vector of the power equipment is calculated, and the Mahalanobis distance is mapped to the scene depth estimation modulation weight of the corresponding pixel.

13. A device for establishing a defect detection model for lightweight power equipment, characterized in that, include: The acquisition module is configured to acquire a training set, which includes multiple fog-free images of power equipment. The model training module is configured to input the training set into a lightweight detection network model for model training to obtain a power equipment defect detection model. The lightweight detection network model includes a backbone network, a neck network, and multiple detection heads. The backbone network includes multiple depthwise separable convolutional modules, and the neck network includes multiple region routing attention mechanism modules. The model training includes: For any fog-free image of power equipment in the training set, features are extracted from the fog-free image of power equipment using the multiple depthwise separable convolutional modules to obtain multiple power equipment feature maps with different resolutions. Enhanced feature fusion is performed on the multiple power equipment feature maps using the multiple region routing attention mechanism modules to obtain multiple enhanced fused feature maps with different resolutions. Multiple detection heads detect the multiple enhanced fused feature maps respectively, outputting multiple prediction information tensors corresponding to each enhanced fused feature map. Multiple prediction boxes are generated based on the multiple prediction information tensors, thereby obtaining multiple prediction boxes for the fog-free image of power equipment. The multiple prediction boxes of the fog-free image of power equipment are compared with the corresponding ground truth boxes, and the model parameters of the lightweight detection network model are adjusted.

14. The apparatus for establishing according to claim 13, characterized in that, The fog-free image of the power equipment is obtained by defogging the foggy image of the power equipment collected in a foggy environment; the device further includes: a first defogging processing module; The first dehazing module is configured to calculate the scene depth estimation modulation weight for each pixel in the foggy image of the power equipment. The foggy image of the power equipment is converted from the RGB color space to the HSV color space to obtain the brightness and saturation of each pixel. Based on the scene depth estimation modulation weight, the brightness coefficient and saturation coefficient of the corresponding pixels are set; Based on the brightness and saturation of each pixel in the foggy image of the power equipment, as well as the brightness coefficient and saturation coefficient of the corresponding pixel, the scene depth estimation formula for the foggy image of the power equipment is determined. The fog-free image of the power equipment is obtained based on the scene depth estimation formula and the atmospheric scattering model.

15. A lightweight defect detection chip for power equipment, characterized in that, include: The detection module is configured to acquire a fog-free image of the power equipment to be processed, input the fog-free image of the power equipment to be processed into the power equipment defect detection model, and obtain the power equipment defect detection result, which includes the category, location and confidence level of the power equipment. The power equipment defect detection model is established through the following steps: Obtain a training set, which includes multiple fog-free images of power equipment; The training set is input into a lightweight detection network model for training to obtain a power equipment defect detection model. The lightweight detection network model includes a backbone network, a neck network, and multiple detection heads. The backbone network includes multiple depthwise separable convolutional modules, and the neck network includes multiple region routing attention mechanism modules. The model training includes: For any fog-free image of power equipment in the training set, features are extracted from the fog-free image of power equipment using the multiple depthwise separable convolutional modules to obtain multiple power equipment feature maps with different resolutions. Enhanced feature fusion is performed on the multiple power equipment feature maps using the multiple region routing attention mechanism modules to obtain multiple enhanced fused feature maps with different resolutions. Multiple detection heads detect the multiple enhanced fused feature maps respectively, outputting multiple prediction information tensors corresponding to each enhanced fused feature map. Multiple prediction boxes are generated based on the multiple prediction information tensors, thereby obtaining multiple prediction boxes for the fog-free image of power equipment. The multiple prediction boxes of the fog-free image of power equipment are compared with the corresponding ground truth boxes, and the model parameters of the lightweight detection network model are adjusted.

16. The detection chip according to claim 15, characterized in that, Also includes: Second defogging module; The second defogging module is configured to acquire a foggy image of the power equipment to be processed; For the foggy image of the power equipment to be processed, calculate the scene depth estimation modulation weight for each pixel; The foggy image of the power equipment to be processed is converted from the RGB color space to the HSV color space to obtain the brightness and saturation of each pixel. Based on the scene depth estimation modulation weight, the brightness coefficient and saturation coefficient of the corresponding pixels are set; Based on the brightness and saturation of each pixel in the foggy image of the power equipment to be processed, as well as the brightness coefficient and saturation coefficient of the corresponding pixel, the scene depth estimation formula of the foggy image of the power equipment to be processed is determined. The fog-free image of the power equipment to be processed is obtained based on the scene depth estimation formula and the atmospheric scattering model.

17. A computer-readable storage medium storing computer instructions thereon, characterized in that, When the computer instructions are executed by the processor, they implement the method of any one of claims 1 to 9.

18. A computer program product, characterized in that, It includes computer instructions that, when executed by a processor, implement the method of any one of claims 1 to 9.

Images