Insulator defect detection method and device, storage medium and computer device

Abstract

The present disclosure relates to the technical field of smart grid inspection, and provides an insulator defect detection method and device, a storage medium and computer equipment, which comprises: collecting polarization, infrared and visible light images of a target insulator through an image extraction device carried by a UAV; extracting a polarization gradient feature map of the polarization image, a temperature attention feature map of the infrared image and a contour guide feature map of the visible light image respectively; performing pixel-level spatial alignment on the temperature attention feature map and the contour guide feature map according to the polarization gradient feature map to obtain aligned infrared and visible light feature maps; determining a spatially reinforced fusion feature map based on the polarization gradient feature map and the aligned infrared and visible light feature maps; and determining a defect detection result based on the polarization gradient feature map and the spatially reinforced fusion feature map. The present disclosure improves the coverage, spatial consistency, robustness and accuracy of insulator defect detection through a polarization-guided multi-modal image feature fusion method.

Description

Technical Field

[0001] This disclosure relates to the field of smart grid inspection technology, and more specifically, to an insulator defect detection method, device, storage medium, and computer equipment. Background Technology

[0002] Insulators are core components for electrical isolation in high-voltage transmission lines, and their operational status directly affects the safety and stability of the power grid. With the advancement of smart grid construction, drone inspections, due to their advantages of mobility, efficiency, and lack of personnel safety risks, have become a primary means of routine maintenance for 110kV to 1000kV transmission lines. Drones equipped with imaging devices cruise along the lines, collecting image data of insulators. This data can be used to identify critical defects such as micro-cracks, contamination buildup, spontaneous breakage, and flashover precursors, enabling remote monitoring and assessment of the insulator's operational status.

[0003] While there are methods in related technologies that utilize drones equipped with imaging devices to inspect insulators and identify defects based on image processing or deep learning, these methods generally suffer from limited sensing capabilities and difficulty in simultaneously addressing the needs for detecting different types of defects. Summary of the Invention

[0004] This disclosure provides at least one insulator defect detection method, apparatus, storage medium, and computer equipment. By using a polarization-leading multimodal image feature fusion method, the coverage, spatial consistency, robustness, and accuracy of insulator defect detection are improved.

[0005] This disclosure provides an insulator defect detection method, including:

[0006] An image set of the target insulator is acquired using an image extraction device mounted on a drone; wherein the image set includes polarization images, infrared images, and visible light images;

[0007] Polarization features are extracted from the polarization image to obtain a polarization gradient feature map; temperature features are extracted from the infrared image to obtain a temperature attention feature map; and contour features are extracted from the visible light image to obtain a contour guidance feature map.

[0008] Based on the polarization gradient feature map, the temperature attention feature map and the contour guidance feature map are spatially aligned at the pixel level to obtain the aligned infrared feature map and the aligned visible light feature map.

[0009] Based on the polarization gradient feature map, the aligned infrared feature map, and the aligned visible light feature map, a spatial enhancement fusion feature map is determined;

[0010] Based on the polarization gradient feature map and the spatial reinforcement fusion feature map, the defect detection result of the target insulator is determined.

[0011] This disclosure provides an insulator defect detection device, including:

[0012] The image acquisition module is used to acquire a set of images of the target insulator using an image extraction device mounted on a UAV; wherein the set of images includes polarization images, infrared images, and visible light images;

[0013] The feature extraction module is used to extract polarization features from the polarization image to obtain a polarization gradient feature map; and to extract temperature features from the infrared image to obtain a temperature attention feature map; and to extract contour features from the visible light image to obtain a contour guidance feature map.

[0014] The feature alignment module is used to perform pixel-level spatial alignment of the temperature attention feature map and the contour guidance feature map based on the polarization gradient feature map, so as to obtain an aligned infrared feature map and an aligned visible light feature map.

[0015] The feature enhancement module is used to determine a spatial enhancement fusion feature map based on the polarization gradient feature map, the aligned infrared feature map, and the aligned visible light feature map.

[0016] The defect detection module is used to determine the defect detection result of the target insulator based on the polarization gradient feature map and the spatial reinforcement fusion feature map.

[0017] This disclosure provides a storage medium storing a computer program that, when executed by a processor, implements the insulator defect detection method as described in any of the possible embodiments above.

[0018] This disclosure provides a computer device including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor. When the processor executes the computer program, it implements the insulator defect detection method described in any of the possible embodiments above.

[0019] The insulator defect detection method, apparatus, storage medium, and computer equipment provided in this disclosure construct a "polarization-thermal-visible light" three-modal sensing system and design a polarization-guided multimodal image feature fusion method. This achieves complementary fusion of multimodal information, correction of spatial consistency, robust fusion guided by physical characteristics, and accurate detection of defect areas, effectively improving the coverage, spatial consistency, robustness, and accuracy of insulator defect detection.

[0020] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings referenced in the embodiments will be briefly described below. These drawings are incorporated in and constitute a part of this specification. They illustrate embodiments conforming to this disclosure and, together with the specification, serve to explain the technical solutions of this disclosure. It should be understood that the following drawings only show some embodiments of this disclosure and should not be considered as limiting the scope. Those skilled in the art can obtain other related drawings based on these drawings without creative effort.

[0022] Figure 1 A flowchart of an insulator defect detection method provided by an embodiment of this disclosure is shown;

[0023] Figure 2 A flowchart of a method for extracting polarization features from a polarization image provided in an embodiment of this disclosure is shown;

[0024] Figure 3 A flowchart of a method for extracting temperature features from infrared images provided in an embodiment of this disclosure is shown;

[0025] Figure 4 A flowchart of a feature map pixel-level spatial alignment method provided by an embodiment of this disclosure is shown;

[0026] Figure 5 A flowchart of a method for determining spatial enhancement fusion feature maps provided in an embodiment of this disclosure is shown;

[0027] Figure 6 A flowchart is shown below illustrating a method for determining defect detection results using a segmented detection head, as provided in an embodiment of this disclosure.

[0028] Figure 7 A schematic diagram of the structure of an insulator defect detection device provided in an embodiment of this disclosure is shown;

[0029] Figure 8 A schematic diagram of the structure of a computer device provided in an embodiment of this disclosure is shown. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0031] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0032] In this document, the term "and / or" merely describes a relationship, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0033] As new power systems upgrade towards "intelligent, unmanned, and full-coverage" operations, high-voltage transmission line inspections are undergoing a comprehensive transformation from "manual tower climbing + ground observation" to "autonomous drone inspections." Drones, with their advantages of maneuverability, high inspection efficiency, and no personnel safety risks, have become core equipment for the routine operation and maintenance of 110kV-1000kV transmission lines. Drones equipped with imaging devices cruise along the lines, simultaneously collecting visual and thermal data of insulators for long-distance, precise identification of critical defects such as micro-cracks, early contamination, spontaneous breakage, and flashover precursors. As the core insulating component separating conductors and towers, the operating status of insulators directly determines the safety and stability of the power grid. Early detection and precise location of defects are crucial to avoiding major accidents such as line tripping and large-scale power outages. Therefore, the defect identification accuracy and robustness of drone inspections directly determine the reliability and practical value of smart grid operation and maintenance.

[0034] Research has revealed that UAV insulator inspection faces extremely complex field environments and technical challenges, with multiple core contradictions between its sensing data and defect detection tasks. First, strong environmental interference is prevalent: insulators are mostly made of white / brown insulating materials, and their surfaces are prone to specular reflection, leading to overexposure of images in strong light, completely masking minute defects such as microcracks. Simultaneously, adverse weather conditions such as rain, fog, and dusk cause image blurring and reduced contrast, further exacerbating the difficulty of defect identification. Second, defect types are complex and heterogeneous: insulator defects encompass two main categories: "thermal" (such as flashover precursors and temperature anomalies caused by internal aging) and "non-thermal" (such as microcracks, early contamination, and spontaneous breakage). The physical characteristics of different types of defects differ significantly, making it difficult to capture them comprehensively using a single sensing dimension. Third, detecting minute defects is extremely difficult: Potential defects such as microcracks with a width <0.2mm and early contamination accumulation have weak morphological features, low contrast with the background, and scarce labeled data, making effective identification difficult using traditional methods. Finally, multimodal data suffers from heterogeneity and misalignment: the various imaging devices carried by the drone (visible light, infrared) experience spatial misalignment and temporal delays in data acquisition due to installation position deviations, flight vibrations, and asynchronous triggering, increasing the difficulty of multi-source information fusion. These factors collectively result in insufficient robustness of drone-based insulator defect detection in real-world scenarios, making it difficult to meet the high reliability requirements of refined power grid operation and maintenance.

[0035] Based on the above research, this disclosure provides an insulator defect detection method, apparatus, storage medium, and computer equipment. First, a UAV is used to acquire polarization, infrared, and visible light images of the target insulator, achieving complementary acquisition of multimodal information. Further, polarization feature extraction is performed on the polarization image to obtain a polarization gradient feature map, which enhances the response to microstructural defects on the insulator surface. Temperature feature extraction is performed on the infrared image to obtain a temperature attention feature map, which helps highlight temperature anomaly areas. Contour feature extraction is performed on the visible light image to obtain a contour-guided feature map, which preserves the insulator's morphology and contour information. Based on this, pixel-level spatial alignment is performed on the temperature attention feature map and the contour-guided feature map, guided by the polarization gradient feature map. This eliminates spatial misalignment between multimodal images caused by differences in camera installation position or flight jitter, improving the accuracy of multimodal information fusion. Then, a spatial enhancement fusion feature map is further determined based on the aligned multimodal features, enabling the fusion process to conform to the physical distribution characteristics of the defect area, achieving spatial focusing of defect features and suppression of background interference. Finally, the defect detection result is determined based on the polarization gradient feature map and the spatial enhancement fusion feature map, which helps improve the accuracy and reliability of insulator defect detection.

[0036] In this embodiment of the disclosure, by constructing a "polarization-thermal-visible light" three-modal sensing system and designing a polarization-guided multimodal image feature fusion method, complementary fusion of multimodal information, correction of spatial consistency, robust fusion guided by physical characteristics, and accurate detection of defect areas are achieved, effectively improving the coverage, spatial consistency, robustness, and accuracy of insulator defect detection.

[0037] To facilitate understanding of this embodiment, the executing entity of the insulator defect detection method provided in this disclosure will first be described in detail. The executing entity of the insulator defect detection method provided in this disclosure is a computer device. This computer device can be a terminal device or a server. The terminal device can also be a mobile device, user terminal, terminal, handheld device, computing device, vehicle-mounted device, wearable device, etc. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud storage, big data, and artificial intelligence platforms. Optionally, this method can also be applied to an implementation environment composed of computer devices and servers.

[0038] The insulator defect detection method provided in this application embodiment will be described in detail below with reference to the accompanying drawings. See also Figure 1 The diagram shown is a flowchart of an insulator defect detection method provided in an embodiment of this disclosure. The insulator defect detection method includes the following steps S101 to S105:

[0039] S101 uses an image extraction device mounted on a drone to collect a set of images of the target insulator.

[0040] It is understood that a drone refers to an unmanned aerial vehicle used in power transmission line inspections. Equipped with image extraction equipment, it autonomously flies along a pre-set route or is remotely controlled by ground personnel to conduct close-range observations of key components along the line. In this disclosure, the drone is equipped with image extraction equipment for acquiring images of target insulators during flight. This image extraction equipment can include various imaging sensors such as polarization cameras, infrared thermal imagers, and visible light cameras to acquire information about different physical dimensions of the target insulators.

[0041] Specifically, target insulators refer to the insulator strings that need to be inspected in high-voltage transmission lines. They are the core components of the transmission line insulation system, used to achieve electrical isolation between conductors and towers. A set of images of the target insulators can be acquired using polarization cameras, infrared thermal imagers, and visible light cameras on a drone. This image set includes polarization images, infrared images, and visible light images.

[0042] Here, a polarization image refers to an image acquired using polarization imaging technology. It records information about the changes in the polarization state of light after reflection from the insulator surface. A polarization image can include polarization angle images in four directions: 0°, 45°, 90°, and 135°. Each polarization angle image represents the light intensity distribution along that specific vibration direction. Polarization images are primarily used to analyze the microstructural characteristics of the insulator surface. For example, minute defects such as microcracks and early contamination can cause changes in surface roughness, leading to alterations in the polarization state of reflected light, thus forming identifiable features in the polarization image. For instance, specular reflection from a complete insulator surface has strong polarization characteristics (linear polarization value close to 1), while defects such as microcracks and contamination, due to increased surface roughness or material changes, can cause polarization state disorder (significantly reduced linear polarization value and irregular polarization angle distribution). Infrared images are thermal radiation images acquired using infrared thermal imagers. They record temperature distribution information on the surface and shallow areas of insulators and are primarily used to detect thermal defects in insulators, such as flashover precursors and internal aging, which can cause localized temperature anomalies, appearing as differences in temperature values ​​compared to normal areas in infrared images. Visible light images are color or grayscale images acquired using visible light cameras. They can represent the appearance, texture details, and color distribution of insulators and are mainly used to provide the macroscopic geometric outline of the insulator and contextual information.

[0043] S102, polarization features are extracted from the polarization image to obtain a polarization gradient feature map; temperature features are extracted from the infrared image to obtain a temperature attention feature map; and contour features are extracted from the visible light image to obtain a contour guidance feature map.

[0044] Furthermore, after obtaining the image set of the target insulator, in order to perceive the defect information of the insulator from multiple dimensions, feature extraction processing can be performed on the polarization image, infrared image and visible light image respectively to achieve preliminary extraction of features in different physical dimensions and determine the feature representation under each mode.

[0045] Specifically, the polarization gradient feature map refers to a feature map extracted based on polarization images that reflects the microstructural changes and edge information of the insulator surface. It is used to characterize the difference in polarization characteristics between defect areas and normal areas and can serve as a physical prior signal for subsequent fusion processes. The temperature attention feature map is a feature map used to characterize the degree of abnormality in temperature distribution on the insulator surface. Temperature abnormal areas are assigned higher response values ​​to highlight the location and extent of thermal defects. The contour-guided feature map represents a visible light feature map after suppressing illumination interference and can be used to provide the macroscopic morphological contour and texture details of the insulator.

[0046] In some possible embodiments, since the polarization image contains polarization angle images in four directions, it can record the polarization state of light from different angles. To fully utilize this information and extract polarization features that characterize microscopic defects, when extracting polarization features from the polarization image, reference is made to... Figure 2 As shown, the steps S201~S203 may be included:

[0047] S201, Calculate the linear polarization feature map based on the polarization angle images of the target insulator in the four directions of 0 degrees, 45 degrees, 90 degrees and 135 degrees.

[0048] Here, the linear polarization degree feature map refers to a feature map calculated by analyzing the intensity differences of light in polarization angle images in different directions. It is a physical quantity describing the proportion of polarization components in reflected light. The value of each pixel in the linear polarization degree feature map represents the proportion of polarization components of the reflected light at that location relative to the total light intensity, which can be calculated by applying Stokes parameters to the polarization angle images in four directions. The calculation formula can be expressed as:

[0049] ;

[0050] In the formula, i represents the pixel index, which is used to identify the position of each pixel in the linear polarization feature map; It is represented as the linear polarization degree value at pixel position i, reflecting the proportion of polarized components in the total light intensity of the reflected light at that position; It is represented as the gray value corresponding to the 0-degree polarization angle image at pixel position i, representing the light intensity component vibrating in that direction; It is represented as the gray value of the image at a 45-degree polarization angle at pixel position i, representing the light intensity component vibrating in that direction; It is represented as the gray value of the image at a 90-degree polarization angle at pixel position i, representing the light intensity component vibrating in that direction; It is represented as the gray value of the image at a polarization angle of 135 degrees at pixel position i, representing the light intensity component vibrating in that direction; It represents the intensity amplitude of linearly polarized light at pixel position i, which is calculated by taking the square root of the sum of the squares of the differences in orthogonal polarization angle images, and is used to quantify the strength of the polarization component. It is expressed as the total light intensity of the four polarization angle images at pixel position i, that is, the total intensity of the incident light.

[0051] S202, perform a convolution operation on the linear polarization feature map to extract the region in the linear polarization feature map whose gray-level changes meet the preset conditions, and obtain the initial gradient feature map.

[0052] Furthermore, to highlight the areas of dramatic gray-level changes in the linear polarization feature map—i.e., potential defect edge regions—a convolution operation can be performed on the linear polarization feature map, and an edge extraction algorithm such as Gaussian difference can be used to extract regions in the linear polarization feature map whose gray-level changes meet preset conditions, thereby determining the edge position and contour information of the defect. Compared with the linear polarization feature map described above, the initial gradient feature map focuses more on regions where gray-level changes occur, suppressing background regions with gradual gray-level changes.

[0053] Here, the preset conditions refer to the criteria used to filter the degree of grayscale change in the image. This is to identify pixel regions that may correspond to defect edges from the linear polarization feature map, thus determining the boundary location information of the defects. For example, it can be set as a local extremum region detected by the difference of Gaussian operator. That is, the linear polarization feature map is convolved with Gaussian kernels of different scales and then subtracted. Pixels with difference response values ​​exceeding a set threshold (such as 0.2, 0.35) are identified as edge points that meet the preset conditions. Alternatively, it can be set as a local maximum point based on the gradient magnitude. That is, the gradient magnitude of each pixel in the linear polarization feature map is calculated, and pixels with gradient magnitudes that are maximum in their neighborhood and higher than the background noise level are identified as regions with drastic grayscale changes.

[0054] For example, in the process of extracting the initial gradient feature map, the Gaussian difference method can be used to extract the high-frequency features of the image. This method uses two Gaussian kernels with different standard deviations to smooth the original image, resulting in two images with different degrees of blur. The two blurred images are then subtracted pixel by pixel. Through this difference operation, low-frequency background information that is consistent in the two images can be canceled out, while high-frequency edge and detail information that differs is preserved and enhanced. Finally, a gradient image that can focus on the key structural features of the image can be output.

[0055] The process of extracting the initial gradient feature map can be achieved through gradient calculation, and the gradient calculation formula can be expressed as:

[0056] ;

[0057] In the formula, It is represented as the gradient component of the linear polarization feature map in the horizontal direction, reflecting the rate of change of gray values ​​in the image along the horizontal direction; This is represented as a two-dimensional convolution operation, used to filter the input feature map; Represented as a linear polarization degree characteristic map; Represented as a horizontal gradient operator convolution kernel, it is used to extract horizontal edge features of an image; This is represented as the boundary padding parameter in the convolution operation, which is to pad the input feature map with a ring of zero values ​​to keep the feature map size unchanged before and after convolution; It is represented as the gradient component of the linear polarization feature map in the vertical direction, reflecting the rate of change of gray values ​​in the image along the vertical direction; Represented as a gradient operator convolution kernel in the vertical direction, it is used to extract vertical edge features of an image.

[0058] The expression for calculating the gradient magnitude and denoising the horizontal and vertical gradient components can be expressed as:

[0059] ;

[0060] In the formula, The initial gradient feature map is a gradient magnitude map obtained by integrating gradient information in the horizontal and vertical directions. This is represented as gradient magnitude calculation, used to synthesize the gradient components in the horizontal and vertical directions into the overall gradient intensity of each pixel; This is represented as a Gaussian blur operation, used for smoothing and denoising gradient magnitude maps; This represents the kernel size used for Gaussian blurring, i.e., a 3x3 filter window. It is represented as the standard deviation parameter of the Gaussian kernel and is used to control the degree of blurring.

[0061] S203, perform multi-scale enhancement and normalization processing on the initial gradient feature map to generate the polarization gradient feature map.

[0062] Understandably, to improve the adaptability of gradient features to defects of different scales and to unify the numerical range of features, multi-scale enhancement and normalization processing can be applied to the initial gradient feature map. Multi-scale enhancement refers to using convolutional kernels or filters of different scales to further process the initial gradient feature map to capture edge information of varying thickness. Normalization processing refers to mapping the numerical range of the feature map to a unified interval, such as between 0 and 1 or between -1 and 1, to eliminate the inconsistency in numerical distribution caused by differences in lighting or acquisition conditions between different images, generating a polarization gradient feature map with a unified numerical range and more prominent defect edges.

[0063] Here, the expression for multi-scale enhancement of the initial gradient feature map can be expressed as:

[0064] ;

[0065] In the formula, This is represented as a gradient feature map after multi-scale enhancement processing, obtained by subtracting the original gradient map from its large-scale blurred version, used to highlight defect edge information at different scales; kernel=5×5 indicates the size of the convolution kernel used for Gaussian blur, i.e., a 5x5 filtering window is used. =2 represents the standard deviation parameter of the Gaussian kernel, used to control the degree of blurring; a larger value here indicates this. The value corresponds to a stronger smoothing effect.

[0066] The expression for normalizing the gradient feature map after multi-scale enhancement can be expressed as:

[0067] ;

[0068] In the formula, The final output polarization gradient feature map is represented by the normalized gradient feature; max represents the value of taking... The maximum value of all pixels in the feature map is used to normalize the numerical range of the feature map to between 0 and 1.

[0069] In this embodiment, the polarization characteristics of the insulator surface are quantified into physical features that can distinguish between normal and defective regions by calculating the degree of linear polarization. Gradient extraction and Gaussian difference processing are combined to accurately capture the defect edge region and suppress background interference. Then, multi-scale enhancement and normalization processing are performed to improve the adaptability and numerical stability of the features to defects of different scales, providing high-quality physical prior signals for subsequent multi-modal fusion.

[0070] In some possible embodiments, there is a definite mapping relationship between the original grayscale values ​​of the infrared image and the actual temperature of the target surface. To extract feature information characterizing the abnormal temperature state of the insulator from the infrared image, temperature feature extraction can be performed on the infrared image to locate and highlight thermal defects. (Refer to...) Figure 3 As shown, the process of extracting temperature features from an infrared image may include the following steps S301~S305:

[0071] S301, perform calibration and conversion of the grayscale value and the actual temperature value of each pixel in the infrared image to obtain a temperature distribution map.

[0072] Here, the calibration conversion between grayscale values ​​and actual temperature values ​​aims to transform the raw grayscale image acquired by the infrared thermal imager into a temperature distribution map that reflects the true temperature of the target surface. Specifically, based on the blackbody radiation law, a mathematical mapping relationship between grayscale values ​​and temperature values ​​can be established. Using parameters such as the gain coefficient, zero drift offset of the infrared thermal imager, and the emissivity of the insulator material, calculations are performed using a radiometric calibration formula. The actual temperature is then solved pixel-by-pixel to obtain the temperature distribution map corresponding to the infrared image. The blackbody radiation law is a physical law describing the relationship between the radiation energy of an ideal blackbody and temperature. It can be used to establish the correspondence between infrared radiation energy and temperature, and in practical applications, it is usually combined with the Stefan-Boltzmann law for radiometric calibration.

[0073] For example, the grayscale calibration formula for an infrared image can be expressed as:

[0074] ;

[0075] Thus, the temperature calculation formula can be expressed as:

[0076] ;

[0077] In the formula, This is represented as the original grayscale value of the pixel located at coordinates (x, y) in the infrared image; It represents the gain coefficient of an infrared thermal imager, used to convert radiant energy into an electrical signal and quantize it into a grayscale value; Represented as the Stefan-Boltzmann constant, it is a physical constant describing the relationship between blackbody radiation energy and temperature; Emissivity is expressed as the emissivity of the insulator material, representing the ratio of the radiant energy of a real object to that of a blackbody at the same temperature; It is represented as the actual Celsius temperature value corresponding to the pixel at coordinates (x, y); This is expressed as the conversion offset between Celsius and Kelvin temperatures; It represents the zero drift offset of the infrared thermal imager and is used to correct background noise caused by sensor dark current, etc.

[0078] In some other embodiments, the calibration conversion between grayscale values ​​and actual temperature values ​​can also be achieved based on other methods such as polynomial fitting, lookup tables, or neural network regression, as long as a correspondence between grayscale values ​​and temperature values ​​can be established, and no specific limitation is made here.

[0079] S302, based on the resolution of the visible light image, the temperature distribution map is upsampled to obtain a high-resolution temperature distribution map with the same resolution as the visible light image.

[0080] Here, to facilitate subsequent pixel-level fusion with the visible light image, the resolution of the temperature distribution map can be adjusted according to the resolution of the visible light image. The low-resolution temperature distribution map is then upsampled to achieve resolution unification, resulting in a high-resolution temperature distribution map with the same height and width as the visible light image. Resolution upsampling refers to increasing the number of pixels in the image through interpolation or learning, thereby enlarging the image size. Specifically, methods such as bilinear interpolation, nearest neighbor interpolation, or transposed convolution can be used to improve the resolution of the temperature distribution map to match that of the visible light image.

[0081] S303, For each pixel in the high-resolution temperature distribution map, calculate the average temperature in the local neighborhood of the pixel.

[0082] Understandably, a local neighborhood refers to a spatial window centered on the current pixel, used to limit the range for calculating the average temperature. The local neighborhood can be set as a rectangular area centered on the pixel with a size of 3×3 or 5×5 pixels. By calculating the average temperature of all pixels in this area, the baseline temperature level of the local area where the pixel is located can be obtained, providing a reference for subsequent judgment on whether the pixel is a temperature anomaly point.

[0083] S304, for each pixel in the high-resolution temperature distribution map, the temperature value of the pixel is compared with the average temperature in the local neighborhood of the pixel. If the temperature value of the pixel is higher than the average temperature in the local neighborhood of the pixel, then the pixel is assigned a first attention weight; if the temperature value of the pixel is not higher than the average temperature in the local neighborhood of the pixel, then the pixel is assigned a second attention weight.

[0084] Furthermore, after obtaining the local average temperature of each pixel, the actual temperature value of each pixel can be compared with its local average temperature. Different attention weights are set based on the comparison results to highlight the feature responses of areas with abnormal temperatures. The first attention weight is greater than the second attention weight, aiming to give higher attention to areas with temperatures above the local average level in subsequent processing. In practice, the first attention weight can be set to 1.0 and the second attention weight to 0.2, or a sigmoid function can be used to map the temperature difference to a continuous weight value between 0 and 1 to more finely differentiate the degree of temperature anomalies.

[0085] S305, Based on the attention weight assigned to each pixel in the high-resolution temperature distribution map, generate the temperature attention feature map.

[0086] Here, after assigning attention weights to each pixel, a weight map of the same size as the high-resolution temperature distribution map can be obtained. At this point, the temperature attention feature map is a feature map obtained after temperature anomaly screening and attention weighting. It has the characteristics of highlighting temperature anomaly areas and suppressing normal temperature areas, and can be directly used for subsequent multimodal fusion processing.

[0087] In some possible embodiments, since visible light images can provide information such as the macroscopic shape, texture details and color distribution of insulators, but are easily affected by uneven lighting, shadow occlusion and specular reflection during actual acquisition, when extracting features from visible light images, it is possible to suppress lighting interference and strengthen the defect edge area to achieve more accurate contour information extraction. Therefore, when extracting contour features from visible light images, the following steps (1) to (3) may be included:

[0088] (1) Convert the visible light image into a grayscale image;

[0089] (2) Perform illumination component separation and suppression processing on the grayscale image to obtain a visible light feature map after illumination suppression that highlights the reflection component of the object;

[0090] (3) Using the polarization gradient feature map as an edge guiding signal, during the downsampling process of the visible light feature map after illumination suppression, the edge guiding signal is multiplied element by element with the visible light feature map to enhance the feature response in the visible light feature map corresponding to the edge region indicated by the edge guiding signal, thereby generating the contour guiding feature map.

[0091] It is understandable that visible light images are the main source of data reflecting the appearance of insulators, but the illumination components they contain often mask the reflection information of the object itself. Therefore, the visible light images can be first converted to grayscale to simplify subsequent calculations. Then, the grayscale images can be processed by illumination component separation and suppression to remove or weaken the influence caused by uneven illumination. Finally, the polarization gradient feature map can be used to perform edge guidance enhancement on the processed visible light feature map to obtain a contour guidance feature map that can accurately reflect the insulator outline and defect edges.

[0092] Illumination component separation and suppression refers to decomposing an image into illumination and reflection components and suppressing the interference of the illumination component on image features. This can be achieved using the Retinex enhancement algorithm. This algorithm assumes that the image is obtained by multiplying the illumination and reflection components. It estimates the illumination component through logarithmic domain transformation and low-pass filtering, and then removes the illumination component from the original image, thus retaining the reflection component that reflects the object's inherent properties. For example, using the single-scale Retinex algorithm, a mesoscale Gaussian filter is used to convolve the grayscale image to obtain the illumination component estimate. Then, the original image and the illumination component are subtracted in the logarithmic domain, and the result is mapped back to the real domain to obtain the illumination-suppressed reflection component image.

[0093] Here, the expression for converting a visible light image to a grayscale image can be represented as:

[0094] ;

[0095] In the formula, This represents the grayscale value of the converted grayscale image at coordinates (x, y); This is represented as the red channel pixel value at coordinates (x, y) of the original visible light image; This is represented as the green channel pixel value at coordinates (x, y) of the original visible light image; This is represented as the blue channel pixel value at coordinates (x, y) of the original visible light image; This is represented by the weighting coefficients of the red channel, corresponding to the human eye's perception weight of the brightness of red light; This is represented by the weighting coefficients of the green channel, corresponding to the human eye's perception weight of the brightness of green light; This is represented by the weighting coefficient of the blue channel, corresponding to the human eye's perception weight of the brightness of blue light.

[0096] For example, when using a single mesoscale Gaussian filter to estimate illumination components while balancing illumination suppression and detail preservation, the process can be represented as follows: First, take the natural logarithm of the grayscale image to obtain a logarithmic domain image; then, convolve the logarithmic domain image using a mesoscale Gaussian kernel to estimate the representation of the illumination components in the logarithmic domain; finally, subtract the original logarithmic domain image from the estimated logarithmic domain illumination components to obtain the logarithmic domain reflectance components, and then restore them to the real domain through exponential operations to obtain the illuminated visible light feature map after illumination suppression. The standard deviation parameter of the Gaussian kernel controls the smoothness of the illumination component estimation; the mesoscale standard deviation can effectively preserve image detail information while suppressing illumination unevenness.

[0097] Furthermore, after obtaining the visible light feature map after illumination suppression, since the polarization gradient feature map accurately reflects the physical properties of the microscopic defect edges on the insulator surface, it can be used as an edge guiding signal during the downsampling process of the visible light feature map. This signal indicates the location of the defect edges. The edge guiding signal is then multiplied element-wise with the visible light feature map to enhance the feature response corresponding to the defect edge region in the visible light feature map, while suppressing the response in non-edge regions, resulting in a contour guiding feature map that incorporates polarization edge information. Here, the contour guiding feature map retains both the macroscopic contour information of the visible light image and enhances the defect edge region indicated by the polarization gradient feature map.

[0098] Furthermore, after obtaining the visible light feature map after illumination suppression, before generating the contour-guided feature map, the visible light feature map after illumination suppression can be subjected to grayscale normalization processing to map its numerical range to the same interval as the polarization gradient feature map, so as to eliminate the fusion mismatch problem caused by the difference in numerical distribution between different modal features.

[0099] In some other embodiments, the illumination component separation and suppression processing of grayscale images can also be performed using the multi-scale Retinex algorithm, homomorphic filtering, or deep learning-based illumination estimation methods, as long as effective separation of the illumination component and the reflection component can be achieved, without any specific limitations.

[0100] In this way, by performing grayscale conversion and illumination component separation and suppression on the visible light image, the interference of factors such as uneven illumination and shadow occlusion on image features is weakened, highlighting the reflection component information of the insulator itself. Furthermore, the polarization gradient feature map is used as an edge guiding signal and multiplied element-wise during the downsampling process of the visible light feature map, so that the defect edge region indicated by the polarization feature is enhanced in the visible light feature map, while the background region is suppressed. This generates a contour guiding feature map that retains both the macroscopic contour information of visible light and the microscopic edge information of polarization, thereby improving the ability of the visible light mode to represent the defect edge.

[0101] S103, based on the polarization gradient feature map, perform pixel-level spatial alignment of the temperature attention feature map and the contour guidance feature map to obtain the aligned infrared feature map and the aligned visible light feature map.

[0102] It is understandable that polarization, infrared, and visible light images acquired for the same target insulator may suffer from pixel-level spatial misalignment due to physical deviations in the mounting positions of the polarization camera, infrared thermal imager, and visible light camera on the drone gimbal, as well as vibrations and attitude changes during drone flight. Furthermore, the feature responses of the same physical location may differ across modal images; for example, temperature anomaly regions in the infrared image may not perfectly coincide with texture edges in the visible light image. This spatial misalignment and semantic differences severely impact the effectiveness of subsequent multimodal fusion. Therefore, after obtaining the polarization gradient feature map, temperature attention feature map, and contour guidance feature map, the polarization gradient feature map can be used as a spatial alignment benchmark. Leveraging its physical properties reflecting the microstructure of defect edges, pixel-level spatial alignment can be performed on the temperature attention feature map and contour guidance feature map, resulting in infrared and visible light feature maps aligned with the polarization gradient feature map as the benchmark.

[0103] Specifically, when performing pixel-level spatial alignment, the similarity between each modal feature map and the baseline feature map can be calculated first, then the spatial offset can be predicted based on the similarity, and finally deformation correction can be performed through deformable convolution, referring to... Figure 4 As shown, the specific steps may include S401~S405:

[0104] S401, using the polarization gradient feature map as a reference for pixel-level spatial alignment, calculate the first pixel-level similarity between the temperature attention feature map and the polarization gradient feature map, and the second pixel-level similarity between the contour guidance feature map and the polarization gradient feature map.

[0105] Here, the polarization gradient feature map is used as the alignment reference because it directly reflects the physical characteristics of microscopic defects on the insulator surface and has the most accurate positioning capability for defect edges. Using this feature map as a reference, the similarity between each pixel position in the temperature attention feature map and its corresponding position in the polarization gradient feature map, as well as the similarity between each pixel position in the contour guidance feature map and its corresponding position in the polarization gradient feature map, can be calculated. This information is then used to determine the direction and distance that each mode feature map needs to be offset from.

[0106] In some possible embodiments, in order to improve the accuracy and stability of similarity calculation, before using the polarization gradient feature map as the alignment benchmark, it can also be processed by a CBR module, which is a combination of convolutional layer, batch normalization layer and ReLU activation function, to obtain a benchmark feature map with stronger feature representation ability for similarity calculation.

[0107] For example, pixel-level similarity calculation can be achieved by calculating the normalized dot product between feature vectors, and its expression can be represented as:

[0108] ;

[0109] ;

[0110] In the formula, This represents the element value in the first pixel-level similarity weight matrix of the temperature attention feature map and the polarization gradient feature map at pixel position (i,j), reflecting the degree of matching between the infrared feature and the polarization prior feature at that position. This is represented as the feature vector of the temperature attention feature map at pixel position (i,j), with a dimension of 256. It is represented as the feature vector of the polarization gradient feature map at pixel position (i,j), with a dimension of 256. This is represented as the feature vector of the physical prior signal at pixel position (i,j) obtained after the polarization gradient feature map is processed by the CBR module, used to enhance the expressive power of polarization prior; This is a small constant used to prevent the denominator from being zero; It represents the element value in the second pixel-level similarity weight matrix of the contour-guided feature map and the polarization gradient feature map at pixel position (i,j), reflecting the degree of matching between the visible light feature and the polarization prior feature at this position; It is represented as the feature vector of the contour-guided feature map at pixel position (i,j), with a dimension of 256.

[0111] S402, based on the first pixel-level similarity, predict the first spatial offset of the temperature attention feature map relative to the polarization gradient feature map.

[0112] Understandably, after obtaining the pixel-level similarity between the temperature attention feature map and the polarization gradient feature map, the degree of matching between the two modalities at each pixel location can be determined. Regions with low similarity indicate spatial misalignment, requiring the calculation of the horizontal and vertical offset distances of this region. This initial spatial offset can guide subsequent deformation correction of the temperature attention feature map. Here, predicting the initial spatial offset can be achieved through regression analysis of the similarity map or by predicting the offset vector of each pixel using a lightweight convolutional network.

[0113] S403, based on the second pixel-level similarity, predict the second spatial offset of the contour-guided feature map relative to the polarization gradient feature map.

[0114] Here, the processing logic in step S403 is the same as that in step S402 above, except that the object being processed is the contour-guided feature map. Therefore, the specific implementation method can be referred to step S402 above, and will not be repeated here.

[0115] S404, based on the first spatial offset, deformable convolution is used to perform deformation correction on the temperature attention feature map, and the aligned infrared feature map is output.

[0116] Furthermore, after obtaining the first spatial offset of the temperature attention feature map relative to the polarization gradient feature map, deformation correction can be performed on the temperature attention feature map using deformable convolution based on this spatial offset. Here, deformable convolution introduces a learnable offset parameter on the basis of standard convolution, which can dynamically adjust the sampling position of the convolution kernel according to the predicted offset at each position in the input feature map, thereby achieving non-rigid deformation correction of the feature map. Specifically, this can be expressed as follows: at each pixel position of the temperature attention feature map, the coordinates of the sampling point are adjusted according to the first spatial offset, so that the convolution kernel can extract features from the corrected position, and then output an infrared feature map that is spatially aligned with the polarization gradient feature map.

[0117] S405, based on the second spatial offset, deformable convolution is used to perform deformation correction on the contour guiding feature map, and the aligned visible light feature map is output.

[0118] Here, the processing logic in step S405 is the same as that in step S404 above, except that the object being processed is the contour-guided feature map. Therefore, the specific implementation method can be referred to step S404 above, and will not be repeated here.

[0119] S104, Based on the polarization gradient feature map, the aligned infrared feature map, and the aligned visible light feature map, determine the spatial enhancement fusion feature map.

[0120] Understandably, after completing pixel-level spatial alignment of the infrared and visible light feature maps, it is necessary to further perform deep fusion of the feature maps from the three modalities to obtain a spatially enhanced fused feature map that contains multimodal semantic information and is spatially focused on the defect region. Here, when performing multimodal fusion, the semantic differences between different modal features and the spatial distribution characteristics of the defect region can be considered to achieve better feature complementarity and defect enhancement effects.

[0121] Specifically, since the polarization gradient feature map reflects the physical prior information of microscopic defects on the insulator surface, the aligned infrared feature map carries temperature anomaly semantics, and the aligned visible light feature map provides texture contour semantics, the three feature maps differ in their expression methods and information dimensions. Direct splicing or simple weighted fusion may lead to feature redundancy or mutual interference. Therefore, this disclosure proposes a hierarchical cross-modal fusion mechanism, which gradually fuses the three modal features through semantic interaction and spatial reinforcement, referring to... Figure 5 As shown, determining the spatial enhancement fusion feature map may include the following steps S501~S506:

[0122] S501, the aligned infrared feature map and the aligned visible light feature map are spliced ​​along the spatial dimension to construct a cross-modal semantic retrieval space.

[0123] Here, the aligned infrared feature map contains semantic information about temperature anomaly regions, indicating the location of temperature rise on the insulator surface; while the aligned visible light feature map contains semantic information about texture contours, providing the macroscopic morphology and edge details of the insulator. To facilitate subsequent semantic retrieval guided by the polarization gradient feature map, the infrared and visible light feature maps can be concatenated along the channel dimension during fusion to construct a joint feature space containing semantic information from both modalities. This space can be considered a cross-modal semantic retrieval database for subsequent query operations.

[0124] In some possible embodiments, in order to improve the accuracy and efficiency of subsequent semantic retrieval, the infrared feature map and the visible light feature map can be refined and normalized by convolution before splicing. The number of channels can be adjusted and the semantic features can be enhanced by 1×1 convolution. Then, the feature distribution can be stabilized by layer normalization or batch normalization, so that the spliced ​​feature space is more compact and easier to retrieve.

[0125] S502, using the polarization gradient feature map as the query vector, perform multi-head self-attention calculation in the cross-modal semantic retrieval space to obtain the initial semantic fusion feature map.

[0126] Understandably, after constructing a cross-modal semantic retrieval space, polarization gradient feature maps can be used as guides to retrieve semantic information related to defects from this space. Specifically, the polarization gradient feature map can be used as a query vector, and then multi-head self-attention computation can be performed on it with the keys and values ​​in the cross-modal semantic retrieval space. Temperature anomaly semantics related to the defect region can be retrieved from the aligned infrared feature map, and texture contour semantics related to the defect region can be retrieved from the aligned visible light feature map.

[0127] Here, multi-head attention computation is a deep learning method based on attention mechanism. It uses multiple parallel attention heads, each of which calculates the similarity between the query and the key in different feature subspaces. Then, the values ​​are weighted and summed according to the similarity. Finally, the outputs of multiple heads are concatenated or fused, which can capture semantic associations in different dimensions and further enrich and improve the accuracy of the retrieved semantic information.

[0128] For example, the multi-head self-attention computation may include the following steps: First, the polarization gradient feature map is transformed linearly to generate a query matrix Q. The concatenated cross-modal semantic retrieval space is transformed linearly to generate a key matrix K and a value matrix V. Then, the dot product of Q and K is divided by a scaling factor and input into the Softmax function to obtain attention weights. The attention weights are then multiplied by V to obtain the weighted features. The above process is executed in parallel in multiple heads, each head using different linear transformation parameters. Finally, the outputs of all heads are concatenated and transformed linearly to obtain the initial semantic fusion feature map.

[0129] Here, taking 8-head multi-head self-attention computation as an example, the core idea is to map features to multiple different subspaces and compute attention in parallel, then fuse the outputs of each head to capture the correlation information at different semantic scales. In specific implementation, the features obtained by adding the polarization gradient feature map to the physical prior signal can be used as the common input for query, key, and value. Then, the multi-head attention mechanism retrieves relevant semantic information from the cross-modal semantic retrieval space. The formula can be expanded as follows:

[0130] ;

[0131] ;

[0132] In the formula, This is represented as the initial semantic fusion feature map output after multi-head self-attention computation; This is represented as the output feature of the k-th attention head; It is represented as a multi-head output fusion matrix, which is used to perform a linear transformation on the features after concatenating the outputs of the 8 heads, and to fuse the information of each head; Represented as a polarization gradient feature map; Represented as a physical prior signal, it is obtained by processing the polarization gradient feature map through the CBR module, and is added to the polarization gradient feature map as the input of the attention mechanism; It is represented as the transpose of the query projection matrix of the k-th head, used to map the input features to the query subspace of that head; It is represented as the transpose of the key projection matrix of the k-th head, used to map the input features to the key subspace of that head; It is represented as the transpose of the value projection matrix of the k-th head, used to map input features to the value subspace of that head; It is represented as a scaling factor, corresponding to the dimension of each head (256÷8=32), and is used to stabilize gradient training.

[0133] S503, Generate a semantic gated mask based on the polarization gradient feature map, and multiply the semantic gated mask with the initial semantic fusion feature map to obtain the target semantic fusion feature map.

[0134] It is understandable that a semantic gating mask refers to a weighted map generated using a polarization gradient feature map to control the flow of semantic information. It can be used to suppress invalid semantic responses in the background region and retain valid semantic information in the defective region. By multiplying the semantic gating mask element-wise with the initial semantic fusion feature map obtained in the above steps, a target semantic fusion feature map can be obtained. In this feature map, the responses in the background region are suppressed, while the semantic features of the defective region are preserved and enhanced.

[0135] Specifically, the semantic gating mask can be generated by applying a Sigmoid activation function to the polarization gradient feature map, mapping the value of each pixel to between 0 and 1, which serves as the gating weight for that pixel location. In the polarization gradient feature map, the values ​​in the defect edge region are higher, and after Sigmoid, they approach 1, indicating that the semantic information at that location should be preserved; the values ​​in the background region are lower, and after Sigmoid, they approach 0, indicating that the semantic information at that location should be suppressed.

[0136] S504, perform multi-scale gradient enhancement on the polarization gradient feature map to generate a spatial attention mask; and use the spatial attention mask to weight the target semantic fusion feature map to obtain a spatially enhanced feature map.

[0137] Here, to further highlight the feature responses of the defect region in the spatial dimension, when spatially enhancing the target semantic fusion feature map, multi-scale gradient enhancement can be performed on the polarization gradient feature map first to generate an attention mask that can indicate the spatial location and contour of the defect region. Multi-scale gradient enhancement refers to processing the polarization gradient feature map with gradient operators or Gaussian difference operators at different scales, fusing gradient responses at multiple scales, so that the generated attention mask can simultaneously capture the spatial distribution of both subtle and large-scale defects. Further element-wise weighting of the target semantic fusion feature map using this spatial attention mask can highlight the feature responses of the defect region and suppress the feature responses of the background region, resulting in a spatially enhanced feature map. Compared to the target semantic fusion feature map, the spatially enhanced feature map is more focused on the spatial location of the defect region and has a stronger defect localization capability.

[0138] S505, the spatially enhanced feature map is processed in parallel by dilated convolution with different dilation rates to extract defect detail features under different receptive fields, and channel fusion is performed on the defect detail features under different receptive fields to obtain multi-scale detail information.

[0139] Specifically, due to the significant scale differences in insulator defects, micro-defects such as microcracks require smaller receptive fields to capture fine edges, while larger defects such as contamination buildup require larger receptive fields to cover the entire area. Therefore, spatially enhanced feature maps can be processed in parallel using dilated convolutions with different dilation rates to extract multi-scale defect details. Here, the dilation rate is a parameter in dilated convolution that determines the interval between sampling points of the convolution kernel. It can be used to expand the receptive field without increasing the number of parameters and computational cost. By setting different dilation rates (such as 2, 4, and 6), defect detail features under small, medium, and large receptive fields can be extracted, corresponding to the spatial details of micro-defects, medium-scale defects, and large-scale defects, respectively.

[0140] Furthermore, after obtaining the defect detail features under different receptive fields, these features can be channel spliced ​​or weighted fused to obtain comprehensive features containing multi-scale detail information, which can fully reflect the structural features of defects at different scales.

[0141] S506, the multi-scale detail information is added to the target semantic fusion feature map to obtain the spatial enhancement fusion feature map.

[0142] Here, by adding multi-scale detail information to the target semantic fusion feature map, the final spatial enhancement fusion feature map can be determined, realizing the complementary fusion of semantic information and detail information, and obtaining a fusion feature map that retains the original semantic features and supplements multi-scale spatial details, providing richer and more accurate feature input for subsequent defect detection.

[0143] S105, Based on the polarization gradient feature map and the spatial reinforcement fusion feature map, determine the defect detection result of the target insulator.

[0144] Understandably, after obtaining the spatially enhanced fusion feature map, the polarization gradient feature map, as a physical prior signal, provides precise geometric information about the defect edges, while the spatially enhanced fusion feature map provides comprehensive features that fuse multimodal semantics and spatial details. By combining the two and performing morphological enhancement and segmentation detection, the final defect detection result for the target insulator can be determined. The defect detection result is a pixel-level classification output, which can include the spatial distribution and location information of defect types such as microcracks, early contamination accumulation, spontaneous breakage, and flashover precursors in the insulator. For example, the defect category to which each pixel belongs can be presented in the form of a segmentation mask map.

[0145] Here, since noise interference or minor defect edge breaks may still exist in the spatially enhanced fused feature map, direct classification may affect the detection accuracy. Therefore, this disclosure also proposes a morphological enhancement method based on polarization gradient guidance. By generating an adaptive structural kernel that matches the local geometric features of the defect, differentiable morphological processing is performed on the fused features to optimize the representation of the defect region. This method may include the following steps (a) to (g):

[0146] (a) On the polarization gradient feature map, for each pixel, calculate the gradient intensity, gradient direction and local variance of the gradient in the local neighborhood of the pixel;

[0147] (b) Determine the principal axis direction of the adaptive structural kernel based on the gradient direction;

[0148] (c) Determine the kernel size of the adaptive structure kernel based on the local variance of the gradient, wherein the kernel size is positively correlated with the local variance of the gradient;

[0149] (d) Determine the kernel weights at each position within the kernel of the adaptive structure based on the gradient intensity. The kernel weight distribution conforms to a Gaussian distribution with the kernel center as the mean, and the greater the gradient intensity, the higher the peak value of the Gaussian distribution.

[0150] (e) Generate the adaptive structural kernel based on the principal axis direction, the kernel size, and the kernel weight distribution;

[0151] (f) Perform differential erosion and differential dilation operations on the spatial enhancement fusion feature map using the adaptive structural kernel to obtain a refined feature map after morphological enhancement;

[0152] (g) Input the refined feature map into the segmentation detection head and output the defect detection result of the target insulator.

[0153] Understandably, calculating the gradient intensity, gradient direction, and local gradient variance within the local neighborhood of each pixel on the polarization gradient feature map is to obtain the local geometric features of the defect region in terms of microstructure, providing a basis for subsequently generating an adaptive structural kernel that matches the defect shape. Specifically, gradient intensity refers to the degree of grayscale change within the local neighborhood of the pixel, which can be obtained by calculating the gradient magnitude from the polarization gradient feature map. It reflects the salience of the defect edge; a larger gradient intensity indicates a higher probability of a distinct edge at that location. Gradient direction refers to the direction of the fastest grayscale change within the local neighborhood of the pixel, which can be calculated from the angle of the gradient vector. It indicates the direction of the defect edge, providing a reference for subsequently determining the principal axis direction of the structural kernel. Local gradient variance represents the dispersion of the gradient values ​​within the local neighborhood of the pixel, which can be obtained by calculating the variance of the gradient values ​​within the local window. It is used to determine the scale range of the defect; a large variance indicates a wider defect area, possibly corresponding to larger-scale defects such as dirt accumulation or breakage, while a small variance indicates a small-scale defect such as a microcrack.

[0154] Here, the adaptive structural kernel refers to a convolutional kernel dynamically generated based on the local geometric features of the defect, used for morphological operations. It can employ different kernel parameters for defects of different locations and shapes to achieve precise enhancement of the defect region. After obtaining the gradient intensity, gradient direction, and local variance of the gradient in the local neighborhood of each pixel on the polarization gradient feature map, the principal axis direction of the adaptive structural kernel can be determined based on the gradient direction. The principal axis direction is mainly used to ensure that the long axis of the structural kernel is consistent with the direction of the defect edge, thus better conforming to the actual shape of the defect in morphological operations. The kernel size of the adaptive structural kernel can be determined based on the local variance of the gradient. The kernel size refers to the spatial coverage of the adaptive structural kernel, usually expressed in pixels as the width and height of the kernel, and is positively correlated with the local variance of the gradient. The kernel weights at each position within the adaptive structural kernel can be determined based on the gradient intensity. The kernel weights are used to control the contribution of different positions within the structural kernel to morphological operations. Their weight distribution conforms to a Gaussian distribution with the kernel center as the mean, giving higher weights to pixels near the kernel center and gradually decreasing weights at the kernel edges. Furthermore, the greater the gradient intensity, the higher the peak value of the Gaussian distribution, thus making the structural kernel more focused on strong edge regions.

[0155] For example, when determining the principal axis direction of the adaptive kernel based on the gradient direction, it can generally be set to align the major axis direction of the adaptive kernel with the gradient direction, so that the principal axis of the kernel extends along the direction of the fastest gray-level change in the local neighborhood of the pixel, thereby enabling the kernel to better fit the actual direction of the defect edge; when determining the kernel size, the following rules can be referenced to divide the defect into different scales according to the magnitude of the local variance of the gradient, and assign corresponding kernel sizes to defects of different scales:

[0156] ;

[0157] Where m represents the kernel size of the adaptive structure kernel, that is, the side length of the structure kernel, in pixels; It is represented as the local variance of the gradient at pixel position (i,j), which reflects the degree of dispersion of the gradient value in the local region at that position and indirectly indicates the size of the defect; This is represented as the first preset threshold, a critical value used to distinguish between minor and moderate defects; This is represented as the second preset threshold, a critical value used to distinguish between medium and large defects. Greater than ;

[0158] When determining the kernel weights, the weight values ​​at each position within the kernel of the adaptive structure can be calculated using the following formula:

[0159] ;

[0160] in, This is represented as the weight value at coordinate (x, y) within the kernel of the adaptive structure; It is represented as the gradient intensity at pixel position (i,j), and is used to control the peak height of the structure kernel; It is represented by a Gaussian distribution function with the mean at the kernel center, used to make the weights gradually decrease from the kernel center to the edge; It is expressed as the square of the Euclidean distance from the kernel coordinates (x,y) to the kernel center (c,c); c represents the coordinate value of the kernel center. For a structural kernel with a side length of m, c is usually taken as (m-1) / 2. It is represented as the standard deviation of a Gaussian distribution and is used to control the rate of weight decay. The larger the value, the smoother the weight distribution.

[0161] Furthermore, after determining the principal axis direction, kernel size, and kernel weight distribution of the adaptive structural kernel based on the gradient intensity, gradient direction, and local gradient variance within the local neighborhood of each pixel on the polarization gradient feature map, an adaptive structural kernel for each pixel location can be generated based on these three factors. Specifically, the basic Gaussian kernel can be rotated according to the principal axis direction so that its major axis aligns with the defect edge direction; then, the rotated Gaussian kernel can be scaled according to its kernel size to match its coverage area with the defect scale; finally, the peak intensity of the Gaussian kernel can be adjusted according to the gradient intensity. Combining these three factors generates an adaptive structural kernel that can dynamically adapt to the local geometric features of the defect. This adaptive structural kernel possesses the characteristics of orientation adaptation, scale adaptation, and intensity adaptation, enabling the generation of optimal morphological processing kernels for defects of different locations and shapes.

[0162] Understandingly, differentiable erosion refers to a differentiable operation achieved by taking the minimum pixel value weighted by the structural kernel within a local region or by approximating it with the Softmin function. This operation can support gradient backpropagation while retaining the denoising effect of traditional erosion. Differentiable dilation refers to a differentiable operation achieved by taking the maximum pixel value weighted by the structural kernel within a local region or by approximating it with the Softmax function. This operation can support gradient backpropagation while retaining the fracture-filling effect of traditional dilation. By sequentially performing differentiable erosion and differentiable dilation operations, opening operations can be implemented to remove isolated noise points and small pseudo-defects in the feature map. By sequentially performing differentiable dilation and differentiable erosion operations, closing operations can be implemented to fill small fractures within the defect region, making the defect edges more continuous and complete. By using the generated adaptive structural kernel to sequentially perform differentiable erosion and differentiable dilation operations on the spatially enhanced fused feature map, morphological enhancement can be adaptively performed according to the local geometric features of the defect. This removes noise interference while enhancing the structural integrity of the defect region, resulting in a refined feature map after morphological enhancement. In this feature map, the defect region is more prominent, the edges are clearer and more continuous, and background noise is effectively suppressed.

[0163] Here, since the refined feature map has undergone morphological enhancement, the representation of the defect region is more accurate and complete. Therefore, it can be input into the segmentation detection head to output the final pixel-level defect detection result. The segmentation detection head is a neural network module for pixel-level classification, typically composed of several convolutional layers, upsampling layers, and classification layers. It can be built based on semantic segmentation models such as encoder-decoder architectures or feature pyramid networks, and optimized through end-to-end training. The segmentation detection head upsamples the input refined feature map to gradually restore resolution, and during the upsampling process, it fuses low-level features from the encoder to supplement spatial details. Finally, it classifies each pixel through convolutional layers and the Softmax activation function, thereby outputting the defect category to which each pixel belongs.

[0164] Specifically, the segmentation detection head is responsible for converting the morphologically enhanced refined feature map into pixel-level defect classification results, thereby achieving accurate location and type identification of insulator defects. After inputting the refined feature map into the segmentation detection head, the spatial resolution of the feature map can be gradually restored through upsampling. During the upsampling process, low-level features from the encoder are fused to supplement spatial detail information, thus obtaining the defect detection results of the target insulator. Figure 6 As shown, the specific steps may include the following steps S601~S605:

[0165] S601, the refined feature map is upsampled by the first transposed convolution to obtain the first upsampled feature map.

[0166] Here, the first transposed convolution refers to a convolution operation that can increase the resolution of the feature map. By upsampling the input feature map through learnable convolution kernel parameters, the low-resolution refined feature map can be enlarged to a higher spatial resolution, which can restore some spatial details lost during the encoding process, such as enlarging from 512×512 to 1024×1024, and thus obtain the first upsampled feature map.

[0167] S602, the first upsampled feature map is fused with a feature map generated during the feature extraction process of each image in the image set, which has the same resolution as the first upsampled feature map, to obtain a fused feature map.

[0168] For example, to further improve segmentation accuracy and make defect edges clearer and more accurate, the first upsampled feature map can be fused across scales with a low-level feature map of the same resolution generated by the encoder during contour feature extraction of the visible light image. This low-level feature map typically comes from an intermediate layer in the encoder's downsampling path and retains rich spatial details and edge information. Alternatively, a feature map of the same resolution generated during temperature feature extraction of the infrared image, or a feature map of the same resolution generated during polarization feature extraction of the polarization image, can be used. Alternatively, multiple low-level feature maps can be fused together and then fused with the first upsampled feature map.

[0169] In this way, by merging the low-level feature map with the upsampled feature map through feature splicing or element-wise addition, the detailed information that may be lost during the upsampling process can be supplemented, so that the final fused feature map contains both high-level semantic information and retains low-level spatial details, thereby improving the positioning accuracy of the segmentation detection head for defect edges and the ability to identify small defects.

[0170] S603, the fused feature map is upsampled by a second transposed convolution to obtain a second upsampled feature map with the same resolution as the visible light image.

[0171] Understandably, after obtaining the fused feature map, although the resolution has been improved, it still does not reach the size of the original input image. Therefore, a second transposed convolution is needed to further upsample the fused feature map, resulting in a second upsampled feature map with the same resolution as the visible light image, for example, scaling up from 1024×1024 to 2048×2048. Here, the parameter settings for the second transposed convolution are similar to those for the first transposed convolution, but because the input feature map resolution is already high, its kernel size and stride need to be adjusted accordingly to achieve the final resolution restoration.

[0172] S604 performs pixel-by-pixel classification on the second upsampled feature map using convolutional layers and the Softmax activation function, and outputs an initial probability map.

[0173] Furthermore, the number of channels in the second upsampled feature map can be compressed to the same number of defect categories using a 1×1 convolutional layer. Then, the classification probability of each pixel is calculated using the Softmax activation function, converting the feature vector of each pixel into a probability value belonging to each defect category. For example, defect categories may include background, microcracks, early dirt accumulation, spontaneous breakage, flashover precursors, etc., thereby outputting an initial probability map with the same resolution as the input image, with each pixel corresponding to a probability vector.

[0174] S605, Post-process the initial probability map and output the defect detection result of the target insulator.

[0175] Here, post-processing refers to a series of optimization operations performed on the initial probability map to improve the accuracy and visual consistency of the final detection results. This can include threshold filtering, connected component removal, and edge smoothing. Post-processing the initial probability map can remove isolated noise points, fill small holes, and smooth defect edges. For example, a probability threshold of 0.5 is set, and pixels with probabilities below this threshold are classified as background; connected components with an area less than 5 pixels are removed to suppress false defects; a 3×3 Gaussian filter is used to smooth the edges of the segmentation results. The final defect detection results are presented as segmentation mask images, where the grayscale value of each pixel represents the defect category to which that pixel belongs. For example, the defect detection results for a target insulator can include segmentation masks for microcrack areas, early contamination accumulation areas, spontaneous breakage areas, and flashover precursor areas. These segmentation masks have the same resolution as the original visible light image, enabling precise location and distribution of various defects on the insulator.

[0176] It should be noted that the insulator defect detection method proposed in this disclosure can be mounted on a transmission line inspection drone using an edge computing unit to achieve real-time processing at the end side. The edge computing unit is responsible for performing all computational tasks from image acquisition to defect detection, eliminating the need to transmit data to the cloud for processing. This significantly reduces transmission latency and ensures data security, thereby meeting the real-time and lightweight requirements of drone inspection. The edge computing unit can be an NVIDIA Jetson AGX Orin edge GPU, with a single-frame image processing time of no more than 80 milliseconds. It can support real-time output for drone inspection and is suitable for detecting various defects in 110kV to 1000kV high-voltage transmission line insulators, including microcracks, early contamination accumulation, spontaneous breakage, and flashover precursors. The detection accuracy decrease is controlled within 5% even in complex environments. Furthermore, the detection method proposed in this disclosure can also be deployed on other types of edge computing devices or applied to other industrial scenarios such as railway contact network insulator inspection and substation equipment inspection, without specific limitations.

[0177] The insulator defect detection method, apparatus, storage medium, and computer equipment provided in this disclosure construct a "polarization-thermal-visible light" three-modal sensing system and design a polarization-guided multimodal image feature fusion method. This achieves complementary fusion of multimodal information, correction of spatial consistency, robust fusion guided by physical characteristics, and accurate detection of defect areas, effectively improving the coverage, spatial consistency, robustness, and accuracy of insulator defect detection.

[0178] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0179] Based on the same inventive concept, this disclosure also provides an insulator defect detection device corresponding to the insulator defect detection method. Since the principle of the device in this disclosure is similar to that of the insulator defect detection method described above, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0180] Reference Figure 7 The diagram shown is a schematic representation of an insulator defect detection device 700 provided in an embodiment of this disclosure. The device includes:

[0181] The image acquisition module 701 is used to acquire an image set of the target insulator through an image extraction device mounted on a UAV; wherein the image set includes polarization images, infrared images and visible light images;

[0182] The feature extraction module 702 is used to extract polarization features from the polarization image to obtain a polarization gradient feature map; and to extract temperature features from the infrared image to obtain a temperature attention feature map; and to extract contour features from the visible light image to obtain a contour guidance feature map.

[0183] The feature alignment module 703 is used to perform pixel-level spatial alignment of the temperature attention feature map and the contour guidance feature map according to the polarization gradient feature map, so as to obtain the aligned infrared feature map and the aligned visible light feature map.

[0184] The feature enhancement module 704 is used to determine a spatial enhancement fusion feature map based on the polarization gradient feature map, the aligned infrared feature map, and the aligned visible light feature map.

[0185] The defect detection module 705 is used to determine the defect detection result of the target insulator based on the polarization gradient feature map and the spatial reinforcement fusion feature map.

[0186] In some possible embodiments, the polarization image includes polarization angle images with respect to the target insulator in four directions: 0 degrees, 45 degrees, 90 degrees, and 135 degrees; the feature extraction module 702 is specifically used for:

[0187] Based on the polarization angle images of the target insulator in four directions (0°, 45°, 90°, and 135°), a linear polarization degree feature map is calculated.

[0188] Perform a convolution operation on the linear polarization degree feature map to extract the regions in the linear polarization degree feature map whose gray-level changes meet preset conditions, and obtain an initial gradient feature map;

[0189] The initial gradient feature map is subjected to multi-scale enhancement and normalization processing to generate the polarization gradient feature map;

[0190] Accordingly, the feature extraction module 702 is specifically used for:

[0191] The grayscale value of each pixel in the infrared image is calibrated and converted to the actual temperature value to obtain a temperature distribution map.

[0192] Based on the resolution of the visible light image, the temperature distribution map is upsampled to obtain a high-resolution temperature distribution map with the same resolution as the visible light image;

[0193] For each pixel in the high-resolution temperature distribution map, calculate the average temperature in the local neighborhood of that pixel.

[0194] For each pixel in the high-resolution temperature distribution map, the temperature value of the pixel is compared with the average temperature in its local neighborhood. If the temperature value of the pixel is higher than the average temperature in its local neighborhood, a first attention weight is assigned to the pixel; if the temperature value of the pixel is not higher than the average temperature in its local neighborhood, a second attention weight is assigned to the pixel. The first attention weight is greater than the second attention weight.

[0195] The temperature attention feature map is generated based on the attention weight assigned to each pixel in the high-resolution temperature distribution map.

[0196] In some possible embodiments, the feature extraction module 702 is specifically used for:

[0197] Convert the visible light image into a grayscale image;

[0198] The grayscale image is subjected to illumination component separation and suppression processing to obtain a visible light feature map that highlights the reflection component of the object after illumination suppression;

[0199] Using the polarization gradient feature map as an edge guiding signal, during the downsampling process of the visible light feature map after illumination suppression, the edge guiding signal is multiplied element-wise with the visible light feature map to enhance the feature response in the visible light feature map corresponding to the edge region indicated by the edge guiding signal, thereby generating the contour guiding feature map.

[0200] In some possible embodiments, the feature alignment module 703 is specifically used for:

[0201] Using the polarization gradient feature map as a reference for pixel-level spatial alignment, the first pixel-level similarity between the temperature attention feature map and the polarization gradient feature map, and the second pixel-level similarity between the contour guidance feature map and the polarization gradient feature map are calculated respectively.

[0202] Based on the first pixel-level similarity, predict the first spatial offset of the temperature attention feature map relative to the polarization gradient feature map;

[0203] Based on the second pixel-level similarity, predict the second spatial offset of the contour-guided feature map relative to the polarization gradient feature map;

[0204] Based on the first spatial offset, the temperature attention feature map is deformed and corrected by deformable convolution, and the aligned infrared feature map is output.

[0205] Based on the second spatial offset, the contour guiding feature map is deformed and corrected by deformable convolution, and the aligned visible light feature map is output.

[0206] In some possible embodiments, the feature enhancement module 704 is specifically used for:

[0207] The aligned infrared feature map and the aligned visible light feature map are spliced ​​along the spatial dimension to construct a cross-modal semantic retrieval space;

[0208] Using the polarization gradient feature map as a query vector, multi-head self-attention calculation is performed in the cross-modal semantic retrieval space to obtain an initial semantic fusion feature map; wherein, using the polarization gradient feature map as a query vector to perform multi-head self-attention calculation in the cross-modal semantic retrieval space includes: retrieving temperature anomaly semantics from the aligned infrared feature map, and retrieving texture contour semantics from the aligned visible light feature map.

[0209] A semantic gated mask is generated based on the polarization gradient feature map, and the semantic gated mask is multiplied with the initial semantic fusion feature map to obtain the target semantic fusion feature map;

[0210] Multi-scale gradient enhancement is performed on the polarization gradient feature map to generate a spatial attention mask; and the spatial attention mask is used to weight the target semantic fusion feature map to obtain a spatially enhanced feature map.

[0211] The spatially enhanced feature map is processed in parallel by dilated convolutions with different dilation rates to extract defect detail features under different receptive fields, and channel fusion is performed on the defect detail features under different receptive fields to obtain multi-scale detail information.

[0212] The multi-scale detail information is added to the target semantic fusion feature map to obtain the spatial enhancement fusion feature map.

[0213] In some possible embodiments, the defect detection module 705 is specifically used for:

[0214] On the polarization gradient feature map, for each pixel, the gradient intensity, gradient direction, and local gradient variance in the local neighborhood of the pixel are calculated.

[0215] The principal axis direction of the adaptive structure kernel is determined based on the gradient direction.

[0216] The kernel size of the adaptive structure kernel is determined based on the local variance of the gradient, and the kernel size is positively correlated with the local variance of the gradient.

[0217] The kernel weights at each position within the adaptive structure kernel are determined based on the gradient intensity. The kernel weight distribution conforms to a Gaussian distribution with the kernel center as the mean, and the greater the gradient intensity, the higher the peak value of the Gaussian distribution.

[0218] The adaptive structure kernel is generated based on the principal axis direction, the kernel size, and the kernel weight distribution;

[0219] The adaptive structural kernel is used to perform differential erosion and differential dilation operations on the spatial enhanced fusion feature map to obtain a refined feature map after morphological enhancement.

[0220] The refined feature map is input into the segmentation detection head, and the defect detection result of the target insulator is output.

[0221] In some possible embodiments, the defect detection module 705 is specifically used for:

[0222] The refined feature map is upsampled by the first transposed convolution to obtain the first upsampled feature map;

[0223] The first upsampled feature map is fused with a feature map generated during feature extraction of each image in the image set, which has the same resolution as the first upsampled feature map, to obtain a fused feature map.

[0224] The fused feature map is upsampled by a second transposed convolution to obtain a second upsampled feature map with the same resolution as the visible light image;

[0225] The second upsampled feature map is classified pixel by pixel using convolutional layers and the Softmax activation function to output an initial probability map.

[0226] The initial probability map is post-processed to output the defect detection results of the target insulator.

[0227] Based on the same technical concept, this disclosure also provides a computer device. (See also...) Figure 8 The diagram shows the structure of a computer device 800 provided in this embodiment of the present disclosure, including a processor 801, a memory 802, and a bus 803. The memory 802 stores execution instructions and includes a main memory 8021 and an external memory 8022. The main memory 8021, also called internal memory, is used to temporarily store computational data in the processor 801 and data exchanged with external memory 8022 such as a hard disk. The processor 801 exchanges data with the external memory 8022 through the main memory 8021.

[0228] In this embodiment, the memory 802 is specifically used to store application code that executes the solution of this application, and its execution is controlled by the processor 801. That is, when the computer device 800 is running, the processor 801 communicates with the memory 802 through the bus 803, so that the processor 801 executes the application code stored in the memory 802, and then executes the method described in any of the foregoing embodiments.

[0229] The memory 802 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0230] Processor 801 may be an integrated circuit chip with signal processing capabilities. The aforementioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0231] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the computer device 800. In other embodiments of this application, the computer device 800 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0232] This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the insulator defect detection method described in the above-described method embodiments. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0233] This disclosure also provides a computer program product carrying program code. The program code includes instructions that can be used to execute the steps of the insulator defect detection method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0234] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0235] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this disclosure, it should be understood that the disclosed systems and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0236] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0237] In addition, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0238] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0239] Finally, it should be noted that the above-described embodiments are merely specific implementations of this disclosure, used to illustrate the technical solutions of this disclosure, and not to limit it. The protection scope of this disclosure is not limited thereto. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this disclosure. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be covered within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. A method for detecting defects in insulators, characterized in that, include: An image set of the target insulator is acquired using an image extraction device mounted on a drone; wherein the image set includes polarization images, infrared images, and visible light images; Polarization features are extracted from the polarization image to obtain a polarization gradient feature map; temperature features are extracted from the infrared image to obtain a temperature attention feature map; and contour features are extracted from the visible light image to obtain a contour guidance feature map. Based on the polarization gradient feature map, the temperature attention feature map and the contour guidance feature map are spatially aligned at the pixel level to obtain the aligned infrared feature map and the aligned visible light feature map. Based on the polarization gradient feature map, the aligned infrared feature map, and the aligned visible light feature map, a spatial enhancement fusion feature map is determined; Based on the polarization gradient feature map and the spatial reinforcement fusion feature map, the defect detection result of the target insulator is determined; The step of determining the spatial enhancement fusion feature map based on the polarization gradient feature map, the aligned infrared feature map, and the aligned visible light feature map includes: The aligned infrared feature map and the aligned visible light feature map are spliced ​​along the spatial dimension to construct a cross-modal semantic retrieval space; Using the polarization gradient feature map as a query vector, multi-head self-attention calculation is performed in the cross-modal semantic retrieval space to obtain an initial semantic fusion feature map; wherein, using the polarization gradient feature map as a query vector to perform multi-head self-attention calculation in the cross-modal semantic retrieval space includes: retrieving temperature anomaly semantics from the aligned infrared feature map, and retrieving texture contour semantics from the aligned visible light feature map. A semantic gated mask is generated based on the polarization gradient feature map, and the semantic gated mask is multiplied with the initial semantic fusion feature map to obtain the target semantic fusion feature map; Multi-scale gradient enhancement is performed on the polarization gradient feature map to generate a spatial attention mask; and the spatial attention mask is used to weight the target semantic fusion feature map to obtain a spatially enhanced feature map. The spatially enhanced feature map is processed in parallel by dilated convolutions with different dilation rates to extract defect detail features under different receptive fields, and channel fusion is performed on the defect detail features under different receptive fields to obtain multi-scale detail information. The multi-scale detail information is added to the target semantic fusion feature map to obtain the spatial enhancement fusion feature map.

2. The method according to claim 1, characterized in that, The polarization image includes polarization angle images in four directions: 0 degrees, 45 degrees, 90 degrees, and 135 degrees, relating to the target insulator; the polarization feature extraction of the polarization image includes: Based on the polarization angle images of the target insulator in four directions (0°, 45°, 90°, and 135°), a linear polarization degree feature map is calculated. Perform a convolution operation on the linear polarization degree feature map to extract the regions in the linear polarization degree feature map whose gray-level changes meet preset conditions, and obtain an initial gradient feature map; The initial gradient feature map is subjected to multi-scale enhancement and normalization processing to generate the polarization gradient feature map; Accordingly, the extraction of temperature features from the infrared image includes: The grayscale value of each pixel in the infrared image is calibrated and converted to the actual temperature value to obtain a temperature distribution map. Based on the resolution of the visible light image, the temperature distribution map is upsampled to obtain a high-resolution temperature distribution map with the same resolution as the visible light image; For each pixel in the high-resolution temperature distribution map, calculate the average temperature in the local neighborhood of that pixel. For each pixel in the high-resolution temperature distribution map, the temperature value of the pixel is compared with the average temperature in its local neighborhood. If the temperature value of the pixel is higher than the average temperature in its local neighborhood, a first attention weight is assigned to the pixel; if the temperature value of the pixel is not higher than the average temperature in its local neighborhood, a second attention weight is assigned to the pixel. The first attention weight is greater than the second attention weight. The temperature attention feature map is generated based on the attention weight assigned to each pixel in the high-resolution temperature distribution map.

3. The method according to claim 1, characterized in that, The extraction of contour features from the visible light image includes: Convert the visible light image into a grayscale image; The grayscale image is subjected to illumination component separation and suppression processing to obtain a visible light feature map that highlights the reflection component of the object after illumination suppression; Using the polarization gradient feature map as an edge guiding signal, during the downsampling process of the visible light feature map after illumination suppression, the edge guiding signal is multiplied element-wise with the visible light feature map to enhance the feature response in the visible light feature map corresponding to the edge region indicated by the edge guiding signal, thereby generating the contour guiding feature map.

4. The method according to claim 1, characterized in that, The step of performing pixel-level spatial alignment of the temperature attention feature map and the contour guidance feature map based on the polarization gradient feature map includes: Using the polarization gradient feature map as a reference for pixel-level spatial alignment, the first pixel-level similarity between the temperature attention feature map and the polarization gradient feature map, and the second pixel-level similarity between the contour guidance feature map and the polarization gradient feature map are calculated respectively. Based on the first pixel-level similarity, predict the first spatial offset of the temperature attention feature map relative to the polarization gradient feature map; Based on the second pixel-level similarity, predict the second spatial offset of the contour-guided feature map relative to the polarization gradient feature map; Based on the first spatial offset, the temperature attention feature map is deformed and corrected by deformable convolution, and the aligned infrared feature map is output. Based on the second spatial offset, the contour guiding feature map is deformed and corrected by deformable convolution, and the aligned visible light feature map is output.

5. The method according to claim 1, characterized in that, The step of determining the defect detection result of the target insulator based on the polarization gradient feature map and the spatial reinforcement fusion feature map includes: On the polarization gradient feature map, for each pixel, the gradient intensity, gradient direction, and local gradient variance in the local neighborhood of the pixel are calculated. The principal axis direction of the adaptive structure kernel is determined based on the gradient direction. The kernel size of the adaptive structure kernel is determined based on the local variance of the gradient, and the kernel size is positively correlated with the local variance of the gradient. The kernel weights at each position within the adaptive structure kernel are determined based on the gradient intensity. The kernel weight distribution conforms to a Gaussian distribution with the kernel center as the mean, and the greater the gradient intensity, the higher the peak value of the Gaussian distribution. The adaptive structure kernel is generated based on the principal axis direction, the kernel size, and the kernel weight distribution; The adaptive structural kernel is used to perform differential erosion and differential dilation operations on the spatial enhanced fusion feature map to obtain a refined feature map after morphological enhancement. The refined feature map is input into the segmentation detection head, and the defect detection result of the target insulator is output.

6. The method according to claim 5, characterized in that, The step of inputting the refined feature map into the segmentation detection head and outputting the defect detection result of the target insulator includes: The refined feature map is upsampled by the first transposed convolution to obtain the first upsampled feature map; The first upsampled feature map is fused with a feature map generated during feature extraction of each image in the image set, which has the same resolution as the first upsampled feature map, to obtain a fused feature map. The fused feature map is upsampled by a second transposed convolution to obtain a second upsampled feature map with the same resolution as the visible light image; The second upsampled feature map is classified pixel by pixel using convolutional layers and the Softmax activation function to output an initial probability map. The initial probability map is post-processed to output the defect detection results of the target insulator.

7. An insulator defect detection device, characterized in that, include: The image acquisition module is used to acquire a set of images of the target insulator using an image extraction device mounted on a UAV; wherein the set of images includes polarization images, infrared images, and visible light images; The feature extraction module is used to extract polarization features from the polarization image to obtain a polarization gradient feature map; and to extract temperature features from the infrared image to obtain a temperature attention feature map; and to extract contour features from the visible light image to obtain a contour guidance feature map. The feature alignment module is used to perform pixel-level spatial alignment of the temperature attention feature map and the contour guidance feature map based on the polarization gradient feature map, so as to obtain an aligned infrared feature map and an aligned visible light feature map. The feature enhancement module is used to determine a spatial enhancement fusion feature map based on the polarization gradient feature map, the aligned infrared feature map, and the aligned visible light feature map. The defect detection module is used to determine the defect detection result of the target insulator based on the polarization gradient feature map and the spatial reinforcement fusion feature map; Specifically, the feature enhancement module is used for: The aligned infrared feature map and the aligned visible light feature map are spliced ​​along the spatial dimension to construct a cross-modal semantic retrieval space; Using the polarization gradient feature map as a query vector, multi-head self-attention calculation is performed in the cross-modal semantic retrieval space to obtain an initial semantic fusion feature map; wherein, using the polarization gradient feature map as a query vector to perform multi-head self-attention calculation in the cross-modal semantic retrieval space includes: retrieving temperature anomaly semantics from the aligned infrared feature map, and retrieving texture contour semantics from the aligned visible light feature map. A semantic gated mask is generated based on the polarization gradient feature map, and the semantic gated mask is multiplied with the initial semantic fusion feature map to obtain the target semantic fusion feature map; Multi-scale gradient enhancement is performed on the polarization gradient feature map to generate a spatial attention mask; and the spatial attention mask is used to weight the target semantic fusion feature map to obtain a spatially enhanced feature map. The spatially enhanced feature map is processed in parallel by dilated convolutions with different dilation rates to extract defect detail features under different receptive fields, and channel fusion is performed on the defect detail features under different receptive fields to obtain multi-scale detail information. The multi-scale detail information is added to the target semantic fusion feature map to obtain the spatial enhancement fusion feature map.

8. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

9. A computer device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 6.

Images