Insulator Defect Detection Method Based on Dual-Mode Frequency-Assisted Segmentation and Intra-String Heterogeneous Sequence Analysis
By performing frequency decomposition and intra-string heterogeneous sequence analysis on the dual-mode images of insulators, the problems of information interference and insufficient defect detection capability in existing technologies are solved, and high-precision insulator defect detection is achieved.
Patent Information
- Application Number
- CN202611527520.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-29
- Publication Date
- 2026-11-10
AI Technical Summary
In existing dual-mode insulator detection methods, the imaging mechanisms of infrared images and visible light images are different, and the feature distributions are quite different, which makes the information easy to interfere with each other. The overall structural information and local detail information lack frequency division modeling, and the periodic arrangement relationship of the insulator pieces is not fully utilized, resulting in insufficient defect detection capability in complex inspection scenarios.
A dual-modal frequency-assisted segmentation and intra-string heterogeneous sequence analysis method is adopted to perform frequency decomposition on visible light and infrared images. By low-frequency global structure modeling and multi-directional high-frequency detail modeling, combined with registration reliability information for feature interaction, the images are divided into intra-string units arranged in physical order, and heterogeneous sequences are constructed for defect detection.
It improves the positioning accuracy and identification capability of defects such as missing insulator discs, broken sheds, cracks, and local thermal anomalies, reduces the impact of complex backgrounds and local registration errors, and enhances the stability and interpretability of the test results.
Smart Images

Figure CN123071328A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of insulator defect detection technology, specifically to an insulator defect detection method based on dual-mode frequency-assisted segmentation and intra-string heterogeneous sequence analysis. Background Technology
[0002] Insulators are critical components in power transmission lines, providing electrical insulation and mechanical support. They are typically installed in strings between towers and conductors. Operating outdoors for extended periods, insulators are susceptible to contamination, temperature variations, lightning strikes, wind and rain, mechanical impacts, and material aging. This can lead to defects such as missing insulator discs, missing or damaged sheds, broken insulator bodies, cracks, foreign matter adhesion, and localized thermal anomalies. Failure to detect and address these defects promptly will degrade the insulator's insulation performance, potentially causing flashovers, string failures, and other accidents, thereby jeopardizing the safe and stable operation of the transmission line. Therefore, regular inspections and defect detection of insulators are crucial for ensuring power grid safety.
[0003] With the development of UAV inspection technology and dual-light imaging equipment, visible light images and infrared images of the same insulator string can be obtained in the same inspection process. Visible light images have the characteristics of clear outline, rich texture and high spatial resolution, which are suitable for characterizing the edge, shape, surface texture and arrangement structure of insulator sheets; infrared images can reflect the thermal radiation distribution of the insulator body and its connection parts, which are suitable for detecting local thermal anomalies and temperature distribution anomalies. However, visible light images are easily affected by factors such as low illumination, shadows, haze and complex backgrounds, and it is difficult to reflect the internal electrical anomalies of the insulator; infrared images usually have problems such as low spatial resolution, blurred target boundaries and insufficient texture information. It can be seen that relying solely on images of any one mode is difficult to fully meet the needs of insulator defect detection in complex inspection scenarios; there are also existing technologies that use only infrared images for insulator defect detection, such as the one described in reference [1], which is also limited by the information expression capability of a single mode.
[0004] To comprehensively utilize the information from visible light images and infrared images, existing technologies have proposed various dual-mode insulator detection methods, mainly including two categories. The first category is the image-level fusion method, which first fuses the infrared image and the visible light image into a fused image, and then inputs the fused image into a target detection network for insulator identification or defect detection. For example, reference [2] discloses a deep learning identification method for UHV line insulator strings that fuses infrared and visible light, which fuses the luminance component of the visible light image with the infrared image, and then combines the chromaticity component of the visible light image to obtain a fused image, and uses a target detection network to extract insulator strings from the fused image; reference [3] proposes to use a generative adversarial network to fuse the infrared image and the visible light image of the insulator, and then uses a target detection network to identify insulator faults in the fused image. The second category is the modal processing method, which performs detection independently in the visible light image and the infrared image, and then associates the detection results of the two modes through geometric mapping. For example, reference [4] discloses an insulator defect detection method based on UAV dual-light fusion, which performs insulator target detection in visible light image, affines the detected insulator marking information into infrared image, and then performs insulator defect detection based on visible light image and determines whether the insulator is overheated based on infrared image by relative temperature difference judgment method.
[0005] However, the aforementioned existing technologies still have the following shortcomings: First, infrared and visible light images have different imaging mechanisms and significantly different feature distributions. Image-level fusion methods directly perform pixel-level weighting, transformation, or feature stitching, which can easily cause interference between edge texture information in visible light images and thermal response information in infrared images. This can weaken small-area thermal anomalies in the fused image, or cause the loss of local structural details such as crack contours and damaged boundaries during the fusion reconstruction process. On the other hand, modal processing methods lack feature-level interaction between the two modes. Their cross-modal correlation relies entirely on geometric mapping, and local registration errors are directly transmitted to the defect judgment results. Furthermore, the visible light detection results and infrared thermal analysis results are independent of each other, making it difficult to detect composite defects where structural and thermal anomalies coexist.
[0006] Secondly, existing methods typically fail to distinguish between low-frequency overall structural information and high-frequency local detail information in insulator images. The main outline of the insulator string, the arrangement of the skirts, and the overall thermal response distribution belong to low-frequency overall structural information, while the edges of the skirts, crack outlines, damage boundaries, and edges of thermal anomaly areas belong to high-frequency local detail information. Inputting the entire image into a unified detection network makes it difficult to simultaneously achieve a complete representation of the overall structure of the insulator string and an enhanced representation of fine-grained defect features.
[0007] Third, existing methods mostly use the entire insulator string or a single target frame as the detection granularity, failing to fully utilize the periodic arrangement relationship of insulator pieces along the insulator string direction and the correlation relationship between adjacent insulator pieces. Therefore, under conditions of complex background, inconsistent modal image quality, or local registration deviation, the ability to locate defects such as missing insulator pieces or missing sheds, which manifest as interruptions in the periodic structure within the string, is insufficient, and the interpretability of the detection results needs to be improved.
[0008] In summary, how to achieve anti-interference feature-level collaborative utilization between visible light images and infrared images in complex inspection scenarios, perform frequency division modeling of the overall structural information and local detail information of the insulator string, and utilize the periodic arrangement relationship of the insulator pieces to realize the defect location and identification of each insulator piece are technical problems that urgently need to be solved by those skilled in the art.
[0009] Cited references: Reference [1]: Fan Liang, Tang Jian, Wang Qiumei, et al. Method and device for detecting internal defects in insulators: China, CN117470859A[P]. 2024-01-30. (Application No.: CN202311804168.3, Application Date: 2023-12-25, Applicant: Guangzhou Zhongke Zhixun Technology Co., Ltd.) Reference [2]: He Bing, Zhou Yunjie, Yuan Qi, et al. Deep learning identification method for UHV line insulator strings integrating infrared and visible light: China, CN107507172A[P]. 2017-12-22. (Application No.: CN201710671781.0, Application Date: 2017-08-08, Applicant: State Grid Shanghai Electric Power Company); Reference [3]: Wang Yong, Yang Yu, Liu Jinshuo, et al. Insulator fault identification based on infrared and visible light image fusion [J]. Infrared Technology, 2025, 47(5): 648-655. 1; Reference [4]: Xiao Suzhi, Zhou Xinlei, Zheng Wei, et al. Method and system for detecting insulator defects based on dual-light fusion of UAV: China, CN115049663B[P]. 2022-12-09. (Application No.: CN202210977736.9, Application Date: 2022-08-16, Patentee: Xingluo Intelligent Technology (Suzhou) Co., Ltd.) Summary of the Invention To address the technical problems of existing dual-modal insulator detection methods, such as the easy interference between structural details and thermal anomaly information, the lack of frequency-division modeling between overall structural information and local defect information, and the failure to utilize the periodic arrangement relationship of insulator pieces, resulting in insufficient defect detection capability in complex inspection scenarios, this invention provides an insulator defect detection method based on dual-modal frequency-assisted segmentation and intra-string heterogeneous sequence analysis. This method performs frequency decomposition on spatially registered visible light and infrared images, achieves insulator string segmentation through low-frequency global structural modeling and multi-directional high-frequency detail modeling, and combines registration reliability information for dual-modal feature interaction. Then, based on the central axis of the insulator string and the periodic arrangement relationship within the string, the insulator string is divided into multiple intra-string units arranged in physical order. Multi-attribute features of each intra-string unit are extracted to construct heterogeneous sequences. Through independent sequence encoding and reliability-based selective interaction, abnormal intra-string units are identified, and their defect locations and defect categories are output. This method can reduce the impact of complex backgrounds, modal differences, and local registration errors on the detection results, thereby achieving the technical objective of improving the positioning accuracy and identification capability of defects such as missing insulator discs, broken sheds, cracks, and local thermal anomalies.
[0010] The technical solution adopted in this invention is as follows: The insulator defect detection method based on dual-mode frequency-assisted segmentation and intra-string heterogeneous sequence analysis includes the following steps: S1: Obtain visible light and infrared images corresponding to the same insulator string, perform spatial registration on the visible light and infrared images, and obtain registration reliability information; S2: Perform frequency decomposition on the visible light image and infrared image respectively to obtain their respective low-frequency components and high-frequency components in multiple directions; S3: Perform global structural feature modeling on the low-frequency components to obtain low-frequency structural features; perform directional detail feature modeling on the high-frequency components in the multiple directions to obtain directional high-frequency features; S4: Perform dual-mode feature interaction based on the low-frequency structural features, directional high-frequency features, and registration reliability information to obtain the insulator string segmentation result; S5: Determine the central axis of the insulator string based on the insulator string segmentation result, and divide the insulator string into multiple intra-string units distributed in physical arrangement order based on the central response of the unit within the string, the high-frequency periodic response along the central axis of the insulator string, and the consistency of the spacing between adjacent intra-string units. S6: Extract the visible light structural features, infrared thermal response features, directional frequency features, and reliability features of each unit in the string, and construct the visible light structural sequence, infrared thermal response sequence, directional frequency sequence, and reliability sequence according to the physical arrangement order. S7: Sequence encoding is performed on the visible light structure sequence, infrared thermal response sequence, and directional frequency sequence respectively, and the encoded structure state, thermal response state, and frequency state are selectively interacted according to the reliability sequence to obtain an intra-string fused state sequence; S8: Determine the abnormal intra-string unit based on the intra-string fusion state sequence, and output the defect location and defect category corresponding to the abnormal intra-string unit.
[0011] S1 includes: S1.1: Obtain the visible light image corresponding to the same insulator string. and infrared images ,in, Represents a visible light image. Represents an infrared image; for the visible light image and infrared images Spatial registration is performed to establish a spatial correspondence between the insulator regions in the two images; S1.2: Calculate any spatial location based on the spatial registration results. Registration reliability at the location: ; in, Indicates spatial location Registration reliability at the location Indicates spatial location Cross-modal edge consistency at the location Indicates spatial location Local phase consistency at that point. Indicates spatial location Local registration residuals at the location, , and These represent the weighting coefficients corresponding to cross-modal edge consistency, local phase consistency, and local registration residuals, respectively. This represents the normalized mapping function; S1.3: Generate a registration reliability map based on the registration reliability at each spatial location. The registration reliability map It is used to characterize the reliability of the correspondence between the infrared image and the visible light image at different spatial locations, and as the registration reliability information to control the intensity of the bimodal feature interaction.
[0012] S2 includes: S2.1: The visible light images are processed using discrete wavelet transform. and infrared images Perform frequency decomposition: ; ; in, This represents the discrete wavelet transform, which employs one or more levels of wavelet decomposition, using the Haar wavelet, Daubechies wavelet, or Symlet wavelet as the wavelet basis. and These represent the low-frequency components of the visible light image and the infrared image, respectively; , and These represent the high-frequency components of the visible light image in the first direction, the second direction, and the diagonal direction, respectively. , and These represent the high-frequency components of the infrared image in the first direction, the second direction, and the diagonal direction, respectively. S2.2: The low-frequency component and The high-frequency components in each direction are used to characterize the main outline, skirt arrangement, and overall thermal response distribution of the insulator string. The high-frequency components in each direction are used to characterize the edges of the insulator skirts, crack outlines, damage boundaries, and edges of thermal anomaly regions, so as to input the corresponding frequency division feature modeling process.
[0013] S3 includes: S3.1: The low-frequency component and Input the corresponding low-frequency global structural feature modeling modules to perform global structural feature modeling: ; in, and These represent the visible light low-frequency global structural feature modeling module and the infrared low-frequency global structural feature modeling module, respectively. and These represent the visible light low-frequency structural features and the infrared low-frequency structural features, respectively; the low-frequency global structural feature modeling module is used to extract the main outline of the insulator string, the skirt arrangement, the overall continuity, and the infrared response distribution trend; S3.2: For any high-frequency direction modality In direction High-frequency components The high-frequency detail feature modeling module for input direction performs direction detail feature modeling: ; in, For modal indexing, Represents visible light modes, Indicates infrared mode; These are directional indices, representing the high-frequency sub-bands in the first direction, the second direction, and the diagonal direction, respectively; Representing modes In direction High-frequency components, Representing modes In direction High-frequency detail feature modeling module in the direction of the direction, Representing modes In direction The high-frequency features in the direction of the insulator; the high-frequency detail feature modeling module is used to enhance the feature representation of the edges of the insulator skirts, cracks, damage boundaries and the edges of thermal anomaly regions; S3.3: The visible light low-frequency structural features The infrared low-frequency structural features and the high-frequency features of the directions in multiple directions As visible light frequency features and infrared frequency features, they are output to S4 for dual-modal feature interaction.
[0014] S4 includes: S4.1: Represent the registration reliability information as a registration reliability map. Based on the aforementioned visible light low-frequency structural features The infrared low-frequency structural features and the registration reliability map Generate low-frequency mode weights: ; in, Represents the normalized exponential function, This indicates a feature concatenation operation. This represents the low-frequency weighted prediction function. and Let represent the visible light low-frequency weights and the infrared low-frequency weights, respectively, and for any spatial location... ,satisfy , and Representing spatial location Visible light low-frequency weighting and infrared low-frequency weighting at the location; S4.2: Based on the low-frequency mode weights, perform weighted interaction on the visible light low-frequency structural features and the infrared low-frequency structural features to obtain low-frequency interaction features: ; in, Indicates low-frequency interaction characteristics, This represents element-wise multiplication; S4.3: For any high-frequency direction According to direction High-frequency characteristics of visible light direction on Infrared high-frequency characteristics and the registration reliability map Generate high-frequency mode weights in the direction of generation: ; The direction is obtained by weighted interaction based on the high-frequency mode weights of the stated direction. High-frequency interaction features in the direction of upward movement: ; in, Indicates direction High-frequency weighted prediction function in the direction of the direction, and They represent directions respectively. Visible light high-frequency weighting and infrared high-frequency weighting, Indicates direction High-frequency interaction features in the direction of the direction, ; S4.4: The low-frequency interaction features and the high-frequency interaction features of the directions in multiple directions The input segmentation and decoding module performs decoding to obtain the segmentation result of the insulator string. The segmentation result of the insulator string includes at least an insulator string region mask. and the center response diagram of the unit within the string ;in, Indicates the masking of the insulator string area. The segmentation and decoding module also outputs at least one of the following: the insulator string center axis probability map, the insulator string boundary probability map, and the insulator string endpoint probability map.
[0015] (12) in, This indicates the segmentation and decoding module.
[0016] S5 includes: S5.1: According to the insulator string region mask Extract the insulator string skeleton, and perform linear fitting, curve fitting, or piecewise spline fitting on the insulator string skeleton to obtain the central axis of the insulator string: ; in, This indicates the central axis of the insulator string. This indicates the position of the normalized arc length along the central axis of the insulator string. and These represent the positions of the normalized arc lengths, respectively. The corresponding x-coordinate and y-coordinate of the image at that location; S5.2: Response diagram of the insulator string's central axis to the center of its neighboring unit. Sampling was performed to obtain the center response curve of the unit within the string: ; The high-frequency interaction features in each direction are sampled to obtain the high-frequency response curves in each direction: ; in, Indicates the position of the normalized arc length The central response of the intra-string cell at the location, Indicates direction Normalized arc length position High-frequency response at that direction Indicates the position on the central axis of the insulator string The sampling neighborhood at that location, This indicates a pooling or statistical sampling operation. Indicates direction The amplitude of the high-frequency interaction characteristics in the direction of the direction; S5.3: Construct the synthesized unit response based on the center response curve of the intra-string unit and the high-frequency response curves in multiple directions: ; in, Indicates the position of the normalized arc length The integrated unit response at the location, , and These represent the directional high-frequency response curves in the first direction, the second direction, and the diagonal direction at the normalized arc length positions, respectively. The value at that location, , , and These represent the response weights corresponding to the center response of the unit within the string and the high-frequency responses in the three directions, respectively. The response weights are preset values or learnable parameters. S5.4: Based on the response of the integrated unit The local extrema determine the candidate center position of the unit within the string, and the positions of adjacent candidate centers are calculated. and Spacing between ,in, and They represent the first The and the first The normalized arc length position corresponding to the candidate center position of each unit within the string. Indicates the first A plurality of adjacent spacings; and a periodic reference spacing is calculated based on the plurality of said adjacent spacings: ; in, Indicates the periodic reference interval, This represents the median value operation. This represents the distance between each adjacent element involved in the calculation. Indicates the number of adjacent spacings involved in the calculation; S5.5: When any of the adjacent spacings is greater than a preset multiple of the periodic reference spacing, candidate center positions of intra-string units are supplemented in the corresponding interval according to the periodic arrangement relationship of the insulator strings, and the spatial range of each intra-string unit is determined according to the adjacent candidate center positions, thereby obtaining multiple intra-string units distributed in physical arrangement order. ,in, Represents the set of units within a string. Indicates the first Units within a string, Indicates the number of units within the string.
[0017] In step S6, the visible light structural features and infrared thermal response features are extracted, including: S6.1: For the first Units within a string Extract visible light structural features: ; in, Indicates the first Visible light structural features of units within a string, Represents the area of a cell within a string. Indicates the width of a unit within a string. Indicates the completeness of the outline. Indicates edge or texture intensity. Indicating shape symmetry, Indicates the difference in spacing or shape between units in adjacent strings; S6.2: For the first Units within a string Extracting infrared thermal response features: ; in, Indicates the first Infrared thermal response characteristics of each unit within a string. Indicates the first The average infrared response of each unit within a string Indicates the first The maximum infrared response of each unit within a string. Indicates the first The degree of dispersion of the infrared response of each unit within a string, Indicates the first Infrared response gradient of each unit within a string, Indicates the first The proportion of hot spot area within each unit of a string. Indicates the first The deviation of the infrared response of each cell within a string from the baseline of the entire string's infrared response, and: ; in, This represents the median value operation. This represents the average infrared response of each cell within the string. Indicates the number of units within the string; when the infrared image contains radiation temperature data, the infrared response is the actual temperature value; when the infrared image does not contain radiation temperature data, the infrared response is the infrared grayscale value or the thermal radiation response value. S6.3: Construct the visible light structure sequence based on the visible light structural features of each unit within the string, according to the physical arrangement order. The infrared thermal response sequence is constructed from the infrared thermal response characteristics of each unit within the string. ,in, Represents the visible light structure sequence. This represents the infrared thermal response sequence.
[0018] In step S6, extracting the directional frequency feature and the reliability feature includes: S6.4: For the first Units within a string The high-frequency response of the unit in multiple directions of the visible light image and infrared image is statistically analyzed: ; in, Representing modes In direction Upper The high-frequency response of each unit within a string. For modal indexing, Represents visible light modes, Indicates infrared mode, For direction index, Indicates the first The number of pixels within a single unit in a string Indicates the first Pixel positions within a unit of a string Indicates pixel position Mode In direction The amplitude of the high-frequency characteristics in the direction of the direction; S6.5: Based on the high-frequency response of the unit in multiple directions, construct the first... Directional frequency characteristics of units within a string: ; in, Indicates the first The directional frequency characteristics of units within a string , , These represent the unit high-frequency responses of the visible light modes in the first direction, the second direction, and the diagonal direction, respectively. , , These represent the unit high-frequency responses of the infrared mode in the first direction, the second direction, and the diagonal direction, respectively; S6.6: Based on the registration reliability information, segmentation confidence, visible light sharpness, and infrared signal-to-noise ratio, generate the first... Reliability characteristics of units within a string: ; in, Indicates the first Reliability characteristics of units within a string, Indicates the first Registration reliability of units within a string Indicates the first The segmentation confidence of each unit within a string. Indicates the first Visible light image quality of each unit within a string, Indicates the first Infrared image quality of each unit within a string, Represents the reliability feature generation function; S6.7: Construct a directional frequency sequence based on the directional frequency characteristics of each unit within the string, according to the physical arrangement order described. A reliability sequence is constructed from the reliability characteristics of each unit within the string. ,in, Represents a directional frequency sequence. Represents a reliability sequence. Indicates the number of units within the string.
[0019] S7 includes: S7.1: Employs a first sequence encoder For the visible light structure sequence Perform sequence encoding to obtain the structure state sequence ; Using a second sequence encoder For the infrared thermal response sequence Sequence encoding is performed to obtain the thermal response state sequence. ; Using a third sequence encoder For the directional frequency sequence Perform sequence encoding to obtain the frequency state sequence. ; in, , and These represent the first sequence encoder, the second sequence encoder, and the third sequence encoder, respectively. , and These represent the structural state sequence, thermal response state sequence, and frequency state sequence, respectively. , and These respectively represent the visible light structure sequence, the infrared thermal response sequence, and the directional frequency sequence; At least two of the first, second, and third sequence encoders have different model parameters or different sequence modeling ranges; The structural state sequence The thermal response state sequence and the frequency state sequence Each by the first The structural state corresponding to each unit within a string Thermal response status and frequency state constitute; S7.2: For the first Each unit within a string, according to its structural state The thermal response state The frequency state and reliability characteristics Generate state interaction gating coefficients: ; in, , and They represent the first The interaction weights corresponding to the structural state, thermal response state, and frequency state of each unit within a string. Indicates the first The reliability characteristics of the unit within the string, i.e., the first unit in the reliability sequence. The elements corresponding to the units within each string Indicates the gating parameters, Represents the normalized exponential function, Indicates feature concatenation operation; S7.3: Selectively interact with each state according to the state interaction gating coefficient to obtain the first state. The fusion state of units within each string: ; The intra-string fusion state sequence is composed of the fusion states of all intra-string units. ,in, Indicates the first The fusion state of units within a string, This represents element-wise multiplication. Represents the sequence of fused states within the string. Indicates the number of units within the string; S7.4: When the registration reliability, segmentation confidence, or modal image quality of any unit within a string is lower than the preset conditions, reduce the interaction weight of the corresponding state of that unit within the string to reduce the impact of erroneous cross-modal information on the defect determination result.
[0020] S8 includes: S8.1: According to the first The states of adjacent units within the same string and the overall statistical state of the insulator string are used to generate the first... Reference states of units within each string: ; in, Indicates the first Reference state of each unit within a string, Indicates the first Each unit within a string before and after The fusion state of units within adjacent strings, Indicates the reference range of cells within adjacent strings. This indicates the overall statistical status of the insulator string. This represents the reference state generation function; S8.2: Calculate the first The fusion state of units within a string With the reference state State deviation between: ; in, Indicates the first Anomaly scoring of units within a string, This represents a distance calculation function or a learnable anomaly scoring function. S8.3: When the abnormal score is... When the value exceeds the preset abnormal threshold, the first... Each string element is identified as an abnormal string element, and the corresponding defect category is output according to the defect classification function: ; in, This represents the defect classification function. Indicates the first The defect categories of each unit within a string; the defect categories include at least one of the following: missing insulator disc, missing shed, damaged shed, damaged body, crack, foreign matter attachment, and localized thermal anomaly; S8.4: The abnormal intra-string unit includes a structural abnormal unit, a thermal response abnormal unit, and a combined structural and thermal response abnormal unit; when the structural abnormal result and thermal response abnormal result of any intra-string unit are inconsistent and its registration reliability is lower than the preset condition, the defect confidence of the intra-string unit is reduced or a verification flag is output.
[0021] This invention provides an insulator defect detection method based on dual-mode frequency-assisted segmentation and intra-string heterogeneous sequence analysis, with the following technical advantages: 1) Addressing the issue that existing dual-mode insulator detection methods fail to differentiate between low-frequency overall structural information and high-frequency local detail information in their modeling, this invention performs frequency decomposition on visible light and infrared images separately. It achieves frequency-division feature representation through low-frequency global structural feature modeling and multi-directional high-frequency detail feature modeling. This enhances the representation of fine-grained defect features such as skirt edges, crack outlines, damage boundaries, and edges of thermal anomaly regions while maintaining the continuity of the insulator string's main outline, skirt arrangement, and overall thermal response distribution. This approach thus meets the representation requirements for both the overall structural information and local defect information of the insulator string.
[0022] 2) Addressing the issue that image-level fusion methods can easily lead to interference between visible light structural details and infrared thermal anomaly information, this invention eliminates the need for fused image reconstruction. Instead, it directly incorporates frequency features into insulator string segmentation, insulator string center axis determination, and intra-string unit division. Furthermore, it combines registration reliability information to perform dual-modal feature interaction at the feature level. This avoids the weakening of small-area thermal anomalies and loss of local structural details during pixel-level fusion reconstruction, while also reducing the risk of local registration errors directly transmitting to defect determination results.
[0023] 3) Addressing the issue that existing methods do not fully utilize the periodic arrangement relationship of insulator discs, this invention combines the center response of the string unit with the high-frequency periodic response in multiple directions to determine the candidate positions of the string unit. Furthermore, it compensates for any missed candidate positions based on the periodic arrangement relationship of the insulator string. This ensures that defects such as missing insulator discs, which manifest as interruptions in the periodic structure of the string, are still retained as independent string units for subsequent analysis. This improves the ability to locate defects such as missing insulator discs, missing sheds, and interruptions in the periodic structure, and reduces the risk of missed detection under complex backgrounds and obstructed conditions.
[0024] 4) This invention constructs visible light structure sequence, infrared thermal response sequence, directional frequency sequence and reliability sequence according to the physical arrangement order of the insulator pieces, so that features with different information sources and different physical meanings are expressed independently. Through heterogeneous sequence modeling, it realizes the collaborative analysis of structural anomalies, thermal response anomalies and composite anomalies of structure and thermal response, taking into account the identification ability of single-mode defects and cross-mode composite defects, and overcoming the shortcomings of the independent processing method of sub-mode that makes it difficult to find composite defects.
[0025] 5) Addressing the issues of inconsistent image quality and local registration deviations in dual-modal images. This invention selectively interacts with the encoded structural state, thermal response state, and frequency state based on reliability information. When the registration reliability, segmentation confidence, or modal image quality of intra-string units is lower than preset conditions, the interaction intensity of the corresponding state is reduced, suppressing the interference of erroneous cross-modal information on the intra-string fusion state, thereby improving the stability and reliability of insulator defect detection results under complex inspection environments.
[0026] 6) This invention generates a reference state based on the state of adjacent intra-string units and the overall statistical state of the insulator string, and uses the state deviation between the fused state of intra-string units and the reference state to determine abnormal intra-string units. It fully utilizes the periodic arrangement relationship and adjacent correlation relationship between insulator discs, so that the defect location result can be interpreted to a specific intra-string unit, and can further distinguish structural abnormal units, thermal response abnormal units, and composite structural and thermal response abnormal units. When the cross-modal judgment results are inconsistent and the registration reliability is insufficient, the defect confidence level can be reduced or a verification flag can be output, thereby reducing the false alarm rate and improving the interpretability and engineering application value of the defect location result. Attached Figure Description
[0027] The present invention will be further described below with reference to the accompanying drawings and examples; Figure 1 This is a schematic diagram of the overall process of the insulator defect detection method based on dual-modal frequency-assisted segmentation and intra-string heterogeneous sequence analysis provided in an embodiment of the present invention.
[0028] Figure 2 This is a schematic diagram of the spatial registration of visible light images and infrared images and the generation of a registration reliability map in step S1 of an embodiment of the present invention.
[0029] Figure 3 This is a schematic diagram illustrating the process of performing frequency decomposition, low-frequency global structural feature modeling, and directional detail feature modeling on visible light and infrared images in steps S2 to S3 of the embodiments of the present invention.
[0030] Figure 4 This is a schematic diagram of the process of generating dual-modal feature interaction and insulator string segmentation results in step S4 of the present invention.
[0031] Figure 5 This is a schematic diagram of the process of determining the central axis of the insulator string and dividing the string into units in step S5 of an embodiment of the present invention.
[0032] Figure 6 This is a schematic diagram illustrating the extraction of visible light structural features, infrared thermal response features, directional frequency features, and reliability features of intra-string units in step S6 of this embodiment of the invention, as well as the construction process of visible light structural sequences, infrared thermal response sequences, directional frequency sequences, and reliability sequences.
[0033] Figure 7 This is a schematic diagram illustrating the process of sequential encoding of the visible light structure sequence, infrared thermal response sequence, and directional frequency sequence, and selectively interacting with them based on the reliability sequence in step S7 of this embodiment of the invention.
[0034] Figure 8 This is a schematic diagram illustrating the process of determining the unit within the abnormal string and outputting the defect location and defect category in step S8 of this embodiment of the invention. Detailed Implementation
[0035] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the invention and to fully convey the scope of the invention to those skilled in the art.
[0036] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the meaning consistent with their meaning in the context of the prior art and should not be interpreted in an idealized or overly formal sense unless specifically defined.
[0037] like Figure 1 As shown, this embodiment provides an insulator defect detection method based on dual-modal frequency-assisted segmentation and intra-string heterogeneous sequence analysis, including the following steps: S1, acquiring and registering visible light and infrared images; S2, performing frequency decomposition on the visible light and infrared images; S3, performing low-frequency global structural feature modeling and directional detail feature modeling; S4, performing dual-modal feature interaction and obtaining insulator string segmentation results; S5, determining the central axis of the insulator string and dividing the intra-string units; S6, extracting multi-attribute features within the string and constructing heterogeneous sequences; S7, performing sequence encoding and reliability-based selective interaction; S8, determining abnormal intra-string units and outputting defect results.
[0038] The core concept of the above steps lies in two aspects: First, frequency decomposition is used to separate the overall structural information and local detail information of the insulator string into different frequency channels, allowing the dual-modal information to interact in a controlled manner at the feature level according to the frequency channels, avoiding information attenuation caused by pixel-level fusion reconstruction. Second, by utilizing the periodic arrangement and adjacent relationships of the insulator sheets along the direction of the insulator string, the insulator string is divided into multiple intra-string units distributed in physical order. Heterogeneous sequences are organized at the granularity of intra-string units and anomaly analysis is performed, thereby reducing the impact of complex backgrounds, modal differences, and local registration errors on the detection results. The following is a combination of... Figures 2-8 Each step is explained in detail.
[0039] S1: Acquire and register visible light and infrared images: like Figure 2 As shown, visible light images of the same insulator string can be acquired during the same inspection using a dual-light imaging device or other image acquisition equipment mounted on a drone. and infrared images ,in, Represents a visible light image. Represents an infrared image. The visible light image... The infrared image has high spatial resolution and is used to render the edges, surface textures, and arrangement of insulator sheets; It is used to present the thermal radiation distribution of the insulator string body and its connection parts. The two modes of images are complementary in terms of information expression: visible light images are suitable for depicting the geometric structure of the insulator, while infrared images are suitable for detecting local thermal anomalies; however, the imaging mechanisms of the two modes are different and the feature distributions are quite different. This is the technical motivation for the subsequent steps to adopt frequency channel controlled interaction rather than direct pixel fusion.
[0040] Because visible light imaging sensors and infrared imaging sensors differ in installation position, field of view, focal length, and pixel resolution, the same insulator string in the two images may exhibit translation, rotation, scaling, and even perspective distortion. Therefore, it is necessary to perform image processing on the visible light images. and the infrared image Spatial registration is performed to establish a pixel-by-pixel spatial correspondence between the insulator string regions in the two images. Specifically, the spatial registration includes: respectively in the visible light images and the infrared image Extract cross-modal matchable features such as edge features, corner features, or phase features to establish feature correspondence between two images; estimate a geometric transformation model based on the feature correspondence, wherein the geometric transformation model adopts an affine transformation model or a projection transformation model; and apply the geometric transformation model to the infrared image. Perform coordinate transformation and resampling. After spatial registration, any spatial location... In the visible light image and the infrared image The middle corresponds to the same physical location on the insulator string.
[0041] It should be noted that, due to the influence of weak texture areas on the insulator surface, occlusion, and the difference in edge representation capabilities between the two modes, the accuracy of spatial registration is not uniform at different spatial locations, and the globally uniform registration assumption will introduce corresponding errors locally. Therefore, this embodiment further calculates the accuracy of any spatial location according to equation (1) based on the spatial registration results. Registration reliability at the location: (1); In equation (1), Indicates spatial location Registration reliability at the location Indicates spatial location Cross-modal edge consistency at the location Indicates spatial location Local phase consistency at that point. Indicates spatial location Local registration residuals at the location, , and These represent the weighting coefficients corresponding to cross-modal edge consistency, local phase consistency, and local registration residuals, respectively. This represents the normalized mapping function.
[0042] Functionally, the cross-modal edge consistency By comparing the two modal images at the position The degree of overlap of edge responses within the neighborhood is obtained and used to measure the reliability of the corresponding structural edges; the local phase consistency... By comparing the two modal images at the position Phase information within the neighborhood is obtained, which is insensitive to differences in grayscale distribution and is suitable for assessing the stability of local structures in cross-modal attitude parameters; the local registration residual... According to the geometric transformation model at position The transformation error within the neighborhood is obtained and used to directly measure the degree of deviation in geometric correspondence. The three indicators jointly characterize the registration quality from the perspectives of edge, phase, and geometric residuals, making the estimation of registration reliability more robust.
[0043] Based on the registration reliability calculated according to formula (1) at each spatial location, a registration reliability map is generated. The registration reliability map With the registered visible light image Infrared images Having the same spatial dimensions, their values are obtained through the normalized mapping function. Mapping to interval The normalized mapping function The weighting coefficients are obtained by using the Sigmoid function. , and All are preset positive values, and For the local registration residual A negative constraint is applied, meaning that the larger the local registration residual, the lower the registration reliability of the corresponding spatial location.
[0044] The registration reliability map This is used to characterize the reliability of the correspondence between the infrared image and the visible light image at different spatial locations, and plays a dual role in subsequent steps: firstly, in the dual-modal feature interaction of S4, the registration reliability map... The first part involves generating modal weights, which automatically reduces the dependence of bimodal feature interactions on unreliable modal correspondences in spatial locations with low registration reliability; the second part refers to the registration reliability map described in S6. The registration reliability of each insulator string is statistically generated based on the insulator string unit, thereby controlling the cross-modal state interaction intensity at the insulator string unit level. As a result, local registration errors are confined to local regions with low registration reliability, preventing them from propagating along the insulator string and affecting defect determination results.
[0045] S2: Frequency decomposition of visible light and infrared images: like Figure 3 As shown, the visible light images are respectively... and the infrared image Frequency decomposition is performed to obtain the corresponding low-frequency components and high-frequency components in multiple directions. Discrete wavelet transform is used for frequency decomposition. The frequency decomposition results of the visible light image are shown in Equation (2), and the frequency decomposition results of the infrared image are shown in Equation (3). (2); (3); In equations (2) and (3), Represents the discrete wavelet transform; and These represent the low-frequency components of the visible light image and the infrared image, respectively. , and These represent the high-frequency components of the visible light image in the first direction, the second direction, and the diagonal direction, respectively. , and These represent the high-frequency components of the infrared image in the first direction, the second direction, and the diagonal direction, respectively.
[0046] The discrete wavelet transform employs one-level or multi-level wavelet decomposition, and the wavelet basis used is Haar wavelet, Daubechies wavelet, Symlet wavelet, or other wavelet basis that can separate low-frequency structural information and high-frequency directional information.
[0047] From the perspective of information distribution principles, different information components in an insulator image are distributed in different frequency channels: the low-frequency component... and Preserve the gradient components in the image, including the low-frequency components of visible light. Characterizing the main outline, skirt arrangement, and overall brightness distribution of the insulator string, the infrared low-frequency component. The overall thermal response distribution and temperature change trend of the insulator string are characterized. The low-frequency components are robust to noise, local occlusion, and complex backgrounds, making them suitable for extracting the overall structural information of the insulator string. The high-frequency components in each direction retain transient components in the image: the first and second high-frequency components are sensitive to edge structures that change along two orthogonal directions, respectively, while the diagonal high-frequency components are sensitive to oblique edges and corner structures. Specifically, the visible light high-frequency components are used to characterize the edges of the insulator skirts, surface texture, crack contours, damage boundaries, and spacing between adjacent insulator sheets, while the infrared high-frequency components are used to characterize the edges of thermal anomaly regions and the contours of temperature abrupt changes.
[0048] The technical significance of performing the above frequency decomposition lies in the following aspects: Firstly, after separating the low-frequency overall structural information from the high-frequency local detail information, modeling methods matching their information scales can be adopted separately, avoiding the mutual constraint between the overall structural expression and the fine-grained defect expression during unified modeling. Secondly, fine-grained defects such as cracks and damage boundaries usually have clear directions, and the high-frequency components separated by direction can specifically enhance the expression of defect features in the corresponding directions. Furthermore, since the insulator sheets are arranged periodically along the insulator string direction, the edges of the sheds form a periodic directional high-frequency response in the high-frequency components, which is used for the location and division of units within the string in S5. When the discrete wavelet transform adopts multi-level wavelet decomposition, the above applications correspond to the low-frequency components and high-frequency components in multiple directions obtained from each level of decomposition.
[0049] S3: Perform low-frequency global structural feature modeling and directional detail feature modeling. Continue as Figure 3 As shown, the low-frequency component and Input the corresponding low-frequency global structural feature modeling module to perform global structural feature modeling. The visible light low-frequency structural features are obtained according to equation (4), and the infrared low-frequency structural features are obtained according to equation (5): (4); (5); In equations (4) and (5), and These represent the visible light low-frequency global structural feature modeling module and the infrared low-frequency global structural feature modeling module, respectively. and These represent the visible light low-frequency structural features and the infrared low-frequency structural features, respectively. The low-frequency global structural feature modeling module is used to extract the main outline of the insulator string, the skirt arrangement, the overall continuity, and the infrared response distribution trend. Since the low-frequency components carry the overall structural information of the insulator string, the low-frequency global structural feature modeling module needs to have a large receptive field or global modeling capability to capture the continuous structural prior of the insulator string along the string direction.
[0050] For any high frequency direction modality In direction High-frequency components The high-frequency detail feature modeling module for input direction performs direction detail feature modeling according to equation (6): (6); In equation (6), For modal indexing, Represents visible light modes, Indicates infrared mode; These are directional indices, representing the high-frequency sub-bands in the first direction, the second direction, and the diagonal direction, respectively; Representing modes In direction High-frequency components, Representing modes In direction High-frequency detail feature modeling module in the direction of the direction, Representing modes In direction The high-frequency directional features are used to enhance the feature representation of insulator skirt edges, cracks, damage boundaries, local texture anomalies, and thermal anomaly region edges. Since high-frequency components carry local detail information, the high-frequency directional features modeling module focuses on detail enhancement within the local receptive field, complementing the global modeling of the low-frequency global structural features modeling module.
[0051] Furthermore, the low-frequency global structural feature modeling module employs a convolutional neural network, an attention network, a state-space network, or a combination of the above networks; the high-frequency detail feature modeling modules in multiple directions employ the same structure and share parameters, employ the same structure but do not share parameters, or employ different network structures for modeling. The visible light low-frequency structural features obtained according to equations (4) to (6) The infrared low-frequency structural features and the high-frequency features of the directions in multiple directions As visible light frequency features and infrared frequency features, they are output to S4 for dual-modal feature interaction.
[0052] S4: Perform dual-modal feature interaction and obtain insulator string segmentation results: like Figure 4 As shown, the registration reliability information is represented as a registration reliability map. Based on the aforementioned visible light low-frequency structural features The infrared low-frequency structural features and the registration reliability map The low-frequency mode weights are generated according to equation (7): (7); In equation (7), Represents the normalized exponential function, This indicates a feature concatenation operation. This represents the low-frequency weighted prediction function. and These represent the visible light low-frequency weights and the infrared low-frequency weights, respectively. For any spatial location The visible light low-frequency weights and the infrared low-frequency weights satisfy the normalization constraint shown in equation (8): (8); In equation (8), and Representing spatial location The visible light low-frequency weight and infrared low-frequency weight are defined. The above normalization constraint enables the two modes to compete for a choice at each spatial location: the weights of the visible light mode and the infrared mode increase or decrease inversely, and the mode weights are adaptively allocated according to the local credibility of the two mode features, rather than being a fixed-proportion weighted sum.
[0053] Based on the low-frequency mode weights, the visible light low-frequency structural features and the infrared low-frequency structural features are weighted and interacted according to Equation (9) to obtain the low-frequency interaction features: (9); In equation (9), Indicates low-frequency interaction characteristics, This indicates element-wise multiplication.
[0054] For any high frequency direction According to direction High-frequency characteristics of visible light direction on Infrared high-frequency characteristics and the registration reliability map Generate the directional high-frequency mode weights according to equation (10): (10); And based on the high-frequency mode weights of the stated direction, a weighted interaction is performed according to equation (11) to obtain the direction. High-frequency interaction features in the direction of upward movement: (11); In equations (10) and (11), Indicates direction High-frequency weighted prediction function in the direction of the direction, and They represent directions respectively. Visible light high-frequency weighting and infrared high-frequency weighting, Indicates direction High-frequency interaction features in the direction of the direction, .
[0055] It should be noted that the weights generated in equations (7) and (10) are based on the registration reliability map. As one of the inputs, the principle is as follows: when the registration reliability at a certain spatial location is low, the correspondence between the two modes at that location is unreliable. By constraining the weight prediction function, the proportion of unreliable modal features in the interaction result can be reduced, thereby suppressing the impact of local registration errors on the segmentation result. At the same time, since the interaction is performed separately at the feature level according to the frequency channel, the edge texture information in the visible light image and the thermal response information in the infrared image interact within their respective matching frequency bands. There is no need to perform pixel-level fusion image reconstruction, which avoids the problem of small-area thermal anomalies being weakened or local structural details being lost during the fusion reconstruction process.
[0056] The low-frequency interaction features obtained according to equation (9) and the high-frequency interaction features of the directions obtained according to equation (11) Input the segmentation and decoding module, and decode according to equation (12) to obtain the segmentation result of the insulator string: (12); In equation (12), Indicates the segmentation and decoding module. Indicates the masking of the insulator string area. This represents the center response diagram of the unit within the string. , and These represent the high-frequency interaction characteristics in the first direction, the second direction, and the diagonal direction, respectively; the insulator string region mask The intra-string cell center response map is used to indicate the spatial region of the insulator string in the image. The response intensity is used to indicate the center position of each insulator disc. Since the decoding input contains both overall structural information and directional details, the insulator string region mask can maintain the continuity of the skirt contour, and the intra-string unit center response map can reflect the periodic arrangement structure of the insulator discs. Furthermore, the segmentation decoding module also outputs at least one of the following: an insulator string center axis probability map, an insulator string boundary probability map, and an insulator string endpoint probability map, to provide more constraint information for determining the center axis of S5.
[0057] S5: Determine the central axis of the insulator string and divide the string into units: like Figure 5 As shown, according to the insulator string region mask Extract the insulator string skeleton, and perform linear fitting, curve fitting, or piecewise spline fitting on the insulator string skeleton to obtain the central axis of the insulator string according to equation (13): (13); In equation (13), This indicates the central axis of the insulator string. This indicates the position of the normalized arc length along the central axis of the insulator string. , and These represent the positions of the normalized arc lengths, respectively. The corresponding x-coordinate and y-coordinate of the image are shown. Linear fitting, curve fitting, or piecewise spline fitting can be used to adapt to the morphology of suspension strings, tension strings, and insulator strings with sag and bending, respectively.
[0058] Response diagram of the insulator string's central axis to the center of the unit within its neighborhood. Sampling is performed, and the center response curve of the unit within the string is obtained according to equation (14): (14); The high-frequency interaction characteristics of each direction are sampled, and the high-frequency response curves of each direction are obtained according to equation (15): (15); In equations (14) and (15), Indicates the position of the normalized arc length The central response of the intra-string cell at the location, Indicates direction Normalized arc length position High-frequency response at that direction Indicates the position on the central axis of the insulator string The sampling neighborhood at that location, This indicates a pooling or statistical sampling operation. Indicates direction The amplitude of the high-frequency interaction features in the direction of the up, Neighborhood sampling along the central axis compresses two-dimensional image information into a one-dimensional response curve along the string direction, transforming the problem of locating cells within the string into a one-dimensional periodic analysis problem along the axis, thus reducing interference from complex backgrounds.
[0059] Based on the central response curve of the intra-string unit and the high-frequency response curves of the multiple directions, the synthesized unit response is constructed according to equation (16): (16); In equation (16), Indicates the position of the normalized arc length The integrated unit response at the location, , and These represent the directional high-frequency response curves in the first direction, the second direction, and the diagonal direction at the normalized arc length positions, respectively. The value at that location, , , and These represent the response weights corresponding to the center response of the intra-string unit and the directional high-frequency responses in three directions, respectively. The response weights are preset values or learnable parameters. The center response of the intra-string unit provides a direct clue to the center position of the insulator sheet, while the directional high-frequency responses provide an indirect clue to the periodic arrangement of the skirt edges. When occlusion or imaging blur weakens the center response, the periodic directional high-frequency responses can still provide a positioning basis. The complementarity of these two types of clues makes the intra-string unit positioning more robust to complex backgrounds and localized image quality degradation.
[0060] According to the integrated unit response The local extrema determine the candidate center positions of the cells within the string, and the spacing between adjacent candidate center positions is calculated according to equation (17): (17); In equation (17), and They represent the first The and the first The normalized arc length position corresponding to the candidate center position of each unit within the string. Indicates the first Each adjacent spacing. Based on the multiple adjacent spacings, the periodic reference spacing is calculated according to formula (18): (18); In equation (18), Indicates the periodic reference interval, This represents the median value operation. This represents the distance between each adjacent element involved in the calculation. This indicates the number of adjacent spacings involved in the calculation. Using median value calculation to determine the period reference spacing avoids interference from individual abnormal spacings, such as oversized spacings caused by missing insulator discs, ensuring that the period reference spacing robustly reflects the normal arrangement period of the insulator discs.
[0061] When any of the adjacent spacings is greater than a preset multiple of the periodic reference spacing, candidate center positions for units within the string are supplemented in the corresponding interval according to the periodic arrangement of the insulator strings. In one embodiment, the preset multiple is 1.5; when the... adjacent spacing satisfy When a missing candidate center position of a string cell exists within the corresponding interval, the position is determined, and the position is determined according to the periodic reference spacing within that interval. The candidate center positions of the units within the string are supplemented at equal intervals, and the number of supplements is calculated according to formula (19): (19); In equation (19), Indicates the first Each adjacent spacing corresponds to the number of candidate center positions for supplementary intra-string units within the interval. This indicates rounding to the nearest integer; the candidate center positions of the supplemented intra-string units are as follows: ,in, Indicates the supplementary position number. The aforementioned compensation mechanism addresses two typical scenarios: first, situations where the overall unit response of the insulator is affected by shading or blurred imaging of the insulator disc. The first possibility is that the local extrema are not obvious and thus are missed; the second possibility is that the missing insulator disc itself causes the comprehensive unit response at that location to disappear, resulting in an abnormal increase in the adjacent spacing. In the latter case, after compensation, the missing location is still retained as an independent intra-string unit, and its anomaly is identified in the state deviation analysis of S8, so that the missing insulator disc appears as an abnormal intra-string unit at the interruption of the intra-string periodic structure, rather than being directly skipped.
[0062] When determining the spatial range of each unit within a string, the midpoint between adjacent candidate center positions is used as the boundary position between adjacent units within the string, or the minimum position of the directional high-frequency response between adjacent candidate center positions is used as the boundary position; the spatial range of each unit within a string is along the central axis of the insulator string. A strip-shaped region is formed, centered on the center position of the corresponding candidate insulator unit and extending to both sides to the boundary position, covering the complete shed outline of the corresponding insulator sheet in a direction perpendicular to the central axis of the insulator string. This results in multiple insulator units distributed in a physical arrangement order. ,in, Represents the set of units within a string. Indicates the first Units within a string, Indicates the number of units within the string.
[0063] S6: Extract multi-attribute features within the string and construct heterogeneous sequences: like Figure 6 As shown, for the first Units within a string Extract visible light structural features according to formula (20): (20); In equation (20), Indicates the first Visible light structural features of units within a string, Represents the area of a cell within a string. Indicates the width of a unit within a string. Indicates the completeness of the outline. Indicates edge or texture intensity. Indicating shape symmetry, This indicates the difference in spacing or shape between adjacent units within the string. The visible light structural features characterize the structural state of units within the string from three levels: geometric quantity, contour quality, and adjacency consistency. Missing insulator pieces will cause abrupt changes in area and adjacent spacing; damaged sheds will cause a decrease in contour integrity and shape symmetry; and foreign matter attachment will cause abnormal edge or texture intensity.
[0064] Regarding the first Units within a string Infrared thermal response features are extracted according to formula (21): (twenty one); In equation (21), Indicates the first Infrared thermal response characteristics of each unit within a string. Indicates the first The average infrared response of each unit within a string Indicates the first The maximum infrared response of each unit within a string. Indicates the first The degree of dispersion of the infrared response of each unit within a string, Indicates the first Infrared response gradient of each unit within a string, Indicates the first The proportion of hot spot area within each unit of a string. Indicates the first The deviation of the infrared response of each cell within a string from the baseline of the entire string's infrared response. This deviation from the baseline of the entire string's infrared response is calculated according to equation (22): (twenty two); In equation (22), This represents the median value operation. This represents the average infrared response of each cell within the string. This indicates the number of cells within the string. When the infrared image contains radiation temperature data, the infrared response is the actual temperature value; when the infrared image does not contain radiation temperature data, the infrared response is an infrared grayscale value, a thermal radiation response value, or other intensity representations that can reflect heat distribution. Calculating the deviation using the entire string's infrared response baseline as a reference eliminates the influence of ambient temperature and overall heating levels, highlighting the relative thermal anomalies of individual cells within the string.
[0065] Regarding the first Units within a string The unit high-frequency response in multiple directions of the visible light image and the infrared image is statistically analyzed according to equation (23): (twenty three); In equation (23), Representing modes In direction Upper The high-frequency response of each unit within a string. For modal indexing, Represents visible light modes, Indicates infrared mode, For direction index, Indicates the first The number of pixels within a single unit in a string Indicates the first Pixel positions within a unit of a string Indicates pixel position Mode In direction The amplitude of the high-frequency characteristics in the direction of the [unclear]. Based on the high-frequency response of the unit in multiple directions, the [unclear] is constructed according to Equation 24. Directional frequency characteristics of units within a string: (twenty four); In equation (24), Indicates the first The directional frequency characteristics of units within a string , , These represent the unit high-frequency responses of the visible light modes in the first direction, the second direction, and the diagonal direction, respectively. , , These represent the high-frequency responses of the unit cells in the first, second, and diagonal directions of the infrared mode, respectively. The directional frequency characteristics are used to characterize at least one of the following: crack direction, damage boundary, missing skirt, and interruption of the periodic structure within the string; for example, the directionality of the crack will cause the high-frequency response of the unit cells in a specific direction to be significantly higher than that in other directions, and the missing skirt will cause the periodic high-frequency response of the corresponding unit cells within the string to be interrupted.
[0066] Based on the registration reliability information, segmentation confidence, visible light sharpness, and infrared signal-to-noise ratio, the first step is generated according to equation (25). Reliability characteristics of units within a string: (25); In equation (25), Indicates the first Reliability characteristics of units within a string, Indicates the first Registration reliability of units within a string Indicates the first The segmentation confidence of each unit within a string. Indicates the first Visible light image quality of each unit within a string, Indicates the first Infrared image quality of each unit within a string, This represents the reliability feature generation function. The reliability features comprehensively characterize the credibility of the corresponding features of each unit within a string, providing a basis for controlling the intensity of cross-modal state interactions in S7.
[0067] According to the physical arrangement order, construct four intra-string sequences according to equations (26) to (29): (26); (27); (28); (29); In equations (26) to (29), Represents the visible light structure sequence. Represents the infrared thermal response sequence. Represents a directional frequency sequence. Represents a reliability sequence. This indicates the number of units within the string. The visible light structure sequence, the infrared thermal response sequence, and the directional frequency sequence have different information sources, physical meanings, and statistical distributions, collectively constituting a heterogeneous sequence within the string. The elements of each sequence in the heterogeneous sequence are organized according to the physical arrangement order of the insulator sheets, so that the order constraints and adjacent relationships between the insulator sheets are preserved in the sequence modeling. This is the structural basis for subsequently using periodic arrangement relationships to identify abnormal units within the string.
[0068] S7: Perform sequence encoding and reliability-based selective interaction: like Figure 7 As shown, the visible light structure sequence is respectively... The infrared thermal response sequence and the direction frequency sequence Sequence encoding is performed. The structural state sequence is obtained according to equation (30), the thermal response state sequence is obtained according to equation (31), and the frequency state sequence is obtained according to equation (32). (30); (31); (32); In equations (30) to (32), , and These represent the first sequence encoder, the second sequence encoder, and the third sequence encoder, respectively. , and These represent the structural state sequence, thermal response state sequence, and frequency state sequence, respectively; the structural state sequence The thermal response state sequence and the frequency state sequence Each by the first The structural state corresponding to each unit within a string Thermal response status and frequency state constitute.
[0069] The reason for using independent encoding for sequences with different attributes is that the information sources, physical meanings, and statistical distributions of visible light structural features, infrared thermal response features, and directional frequency features are all different. If mixed encoding is used, the statistical characteristics of one attribute may overwhelm the abnormal patterns of other attributes. Independent encoding allows each attribute to complete contextual modeling in its own feature space first, preserving the discrimination patterns of each attribute. Furthermore, the first sequence encoder is used to model the shape association and periodic arrangement relationship between insulator pieces, the second sequence encoder is used to model the thermal response variation trend along the direction of the insulator string, and the third sequence encoder is used to model the directional response of local edges, cracks, damage, and periodic structural interruptions. At least two of the first, second, and third sequence encoders have different model parameters or different sequence modeling ranges to adapt to the statistical characteristics of different attribute sequences.
[0070] For the Each unit within a string, according to its structural state The thermal response state The frequency state and reliability characteristics The state interaction gating coefficients are generated according to equation (33): (33); In equation (33), , and They represent the first The interaction weights corresponding to the structural state, thermal response state, and frequency state of each unit within a string. Indicates the first The reliability characteristics of the unit within the string, i.e., the first unit in the reliability sequence. The elements corresponding to the units within each string Indicates the gating parameters, Represents the normalized exponential function, This indicates a feature splicing operation.
[0071] Based on the state interaction gating coefficients, selective interactions are performed on each state according to equation (34) to obtain the first state. The fusion state of units within each string: (34); The fusion state sequence of all intra-string units is constituted according to equation (35): (35); In equations (34) and (35), Indicates the first The fusion state of units within a string, This represents element-wise multiplication. Represents the sequence of fused states within the string. This represents the number of units within the string. The state interaction gating coefficient takes reliability characteristics as one of the inputs, and its essence is to perform soft selection based on units within the string and attribute states: attribute states with higher reliability receive larger interaction weights, while attribute states with lower reliability are suppressed, thereby retaining reliable information and weakening unreliable information in the fused state.
[0072] Furthermore, when the registration reliability, segmentation confidence, or modal image quality of any intra-string unit is lower than a preset condition, the interaction weight of the corresponding state of that intra-string unit is reduced to decrease the impact of erroneous cross-modal information on the defect determination result. The preset condition includes at least one of the following: registration reliability of intra-string units. The segmentation confidence level is lower than the preset registration reliability threshold. Visible light image quality below the preset segmentation confidence threshold Infrared image quality below the preset visible light quality threshold The infrared quality threshold is lower than the preset threshold. The state interaction gating coefficient is based on reliability characteristics. When gating prediction with one of the inputs, the impact of reliability on interaction weights has been implicitly learned; this step further applies explicit suppression based on this.
[0073] In one implementation, the reduction of the interaction weight of the corresponding state of the unit within the string is achieved through an attenuation coefficient: when the visible light image quality When the visible light quality is below the preset visible light quality threshold, the interaction weights of the structural state are adjusted according to equation (36): (36); When infrared image quality When the infrared quality is below the preset infrared quality threshold, the interaction weights of the thermal response state are corrected according to Equation 37: (37); When registration reliability When the value is below the preset registration reliability threshold, the three interaction weights are uniformly reduced according to equations (38) to (40): (38); (39); (40); In equations (36) to (40), , and These represent the corrected structural state interaction weights, the corrected thermal response state interaction weights, and the corrected frequency state interaction weights, respectively. Represents the structural state attenuation coefficient. Represents the thermal response state decay coefficient. The uniform attenuation coefficient is represented; the normalized interaction weights after modification participate in the fusion state shown in equation (34). The calculation is performed. Therefore, in regions with large local registration errors, degraded single-modal image quality, or low segmentation confidence, the interference of erroneous cross-modal information on the fusion state of each intra-string unit is suppressed, and the reliability of the intra-string fusion state sequence is improved.
[0074] S8: Identify cells within the anomaly string and output the defect results: like Figure 8 As shown, according to the first The states of adjacent units within each string and the overall statistical state of the insulator string are used to generate the state of the first string according to formula (41). Reference states of units within each string: (41); In equation (41), Indicates the first Reference state of each unit within a string, Indicates the first Each unit within a string before and after The fusion state of units within adjacent strings, Indicates the reference range of cells within adjacent strings. This indicates the overall statistical status of the insulator string. This represents the reference state generation function. The reference state generation process utilizes the periodic arrangement relationship and adjacent correlation relationship between insulator discs: normal string units are consistent with their adjacent units and the overall string statistical level in terms of structure, thermal response, and frequency properties, while abnormal string units deviate from this consistency; therefore, the reference state estimated from the state of adjacent string units and the overall string statistical state can serve as a dynamic benchmark for determining whether the current string unit is normal, adapting to insulator strings of different voltage levels, different numbers of discs, and different operating conditions without relying on absolute thresholds.
[0075] Calculate the first according to formula (42) The fusion state of units within a string With the reference state State deviation between: (42); In equation (42), Indicates the first Anomaly scoring of units within a string, This represents a distance calculation function or a learnable anomaly scoring function.
[0076] When the abnormal score When the value exceeds the preset abnormal threshold, the first... Each string cell is identified as an abnormal string cell, and the corresponding defect category is output according to equation (43): (43); In equation (43), This represents the defect classification function. Indicates the first The defect category of each string unit is defined; the defect category includes at least one of the following: missing insulator piece, missing shed, damaged shed, damaged insulator body, crack, foreign object attachment, and localized thermal anomaly; simultaneously, the defect location and defect confidence level corresponding to the abnormal string unit are output, whereby the defect confidence level is used to characterize the defect category. The reliability of the information is assessed. Because the units within the abnormal string are numbered according to their physical arrangement, the location of the defect can be directly mapped to the specific insulator disc position within the insulator string, facilitating on-site verification and handling by maintenance personnel.
[0077] The anomaly string units include structural anomaly units, thermal response anomaly units, and combined structural and thermal response anomaly units. The calculation of the first... The structural state of each unit within a string Relative to reference state Structural state deviation and thermal response status Relative to reference state thermal response state deviation ,in, Indicates the first Structural state deviation of units within a string Indicates the first The thermal response state deviation of each unit within the string; the structural state deviation and the deviation of the thermal response state The types of units within the abnormal string are determined by comparing them with corresponding preset deviation thresholds. The correspondence between the types of units within the abnormal string and typical defect categories is shown in Table 1.
[0078]
[0079] Furthermore, when the first The structural anomaly determination results of individual cells within a string are inconsistent with the thermal response anomaly determination results, and their registration reliability is also inconsistent. If the results fall below the preset threshold, it indicates insufficient reliability of the cross-modal correspondence between the units within the string. Inconsistent judgments may stem from local registration errors rather than actual structural and thermal response anomalies. In this case, the defect confidence level of the unit within the string should be adjusted downwards based on registration reliability, or a verification flag should be output to prompt maintenance personnel to perform manual verification or supplementary inspections of the unit within the string. This avoids outputting high-confidence conflict judgments when local registration errors are large, reduces false alarm rates, and improves the engineering reliability of insulator defect detection results.
[0080] In summary, the method provided in this embodiment takes into account both the overall structural information and local details of the insulator string through frequency decomposition and frequency-division modeling. It avoids information attenuation caused by fusion reconstruction through controlled dual-modal feature interaction based on frequency channels. It improves the ability to locate missing defects through intra-string unit partitioning and compensation based on periodic arrangement relationships. It suppresses the propagation of erroneous cross-modal information through heterogeneous sequence modeling and reliability-based selective interaction. It achieves defect location and classification that can be interpreted down to specific intra-string units through state deviation analysis based on reference state. Thus, it comprehensively improves the accuracy, stability and engineering practicality of insulator defect detection in complex inspection scenarios.
Claims
1. A method for detecting insulator defects based on dual-mode frequency-assisted segmentation and intra-string heterogeneous sequence analysis, characterized in that... Includes the following steps: S1: Obtain visible light and infrared images corresponding to the same insulator string, perform spatial registration on the visible light and infrared images, and obtain registration reliability information; S2: Perform frequency decomposition on the visible light image and the infrared image respectively to obtain their respective low-frequency components and high-frequency components in multiple directions; S3: Perform global structural feature modeling on the low-frequency components to obtain low-frequency structural features; The high-frequency components in the multiple directions are modeled with directional detail features to obtain directional high-frequency features; S4: Perform dual-mode feature interaction based on the low-frequency structural features, directional high-frequency features, and registration reliability information to obtain the insulator string segmentation result; S5: Determine the central axis of the insulator string based on the insulator string segmentation result, and divide the insulator string into multiple intra-string units distributed in physical arrangement order based on the central response of the unit within the string, the high-frequency periodic response along the central axis of the insulator string, and the consistency of the spacing between adjacent intra-string units. S6: Extract the visible light structural features, infrared thermal response features, directional frequency features, and reliability features of each unit in the string, and construct the visible light structural sequence, infrared thermal response sequence, directional frequency sequence, and reliability sequence according to the physical arrangement order. S7: Sequence encoding is performed on the visible light structure sequence, infrared thermal response sequence, and directional frequency sequence respectively, and the encoded structure state, thermal response state, and frequency state are selectively interacted according to the reliability sequence to obtain an intra-string fused state sequence; S8: Determine the abnormal intra-string unit based on the intra-string fusion state sequence, and output the defect location and defect category corresponding to the abnormal intra-string unit.
2. The insulator defect detection method based on dual-mode frequency-assisted segmentation and intra-string heterogeneous sequence analysis according to claim 1, characterized in that: S1 includes: S1.1: Obtain the visible light image corresponding to the same insulator string. and infrared images ,in, Represents a visible light image. Represents an infrared image; for the visible light image and infrared images Spatial registration is performed to establish a spatial correspondence between the insulator regions in the two images; S1.2: Calculate any spatial location based on the spatial registration results. Registration reliability at the location: ; in, Indicates spatial location Registration reliability at the location Indicates spatial location Cross-modal edge consistency at the location Indicates spatial location Local phase consistency at that point. Indicates spatial location Local registration residuals at the location, , and These represent the weighting coefficients corresponding to cross-modal edge consistency, local phase consistency, and local registration residuals, respectively. This represents the normalized mapping function; S1.3: Generate a registration reliability map based on the registration reliability at each spatial location. The registration reliability map It is used to characterize the reliability of the correspondence between the infrared image and the visible light image at different spatial locations, and as the registration reliability information to control the intensity of the bimodal feature interaction.
3. The insulator defect detection method based on dual-mode frequency-assisted segmentation and intra-string heterogeneous sequence analysis according to claim 2, characterized in that: S2 includes: S2.1: The visible light images are processed using discrete wavelet transform. and infrared images Perform frequency decomposition: ; ; in, This represents the discrete wavelet transform, which employs one or more levels of wavelet decomposition, using the Haar wavelet, Daubechies wavelet, or Symlet wavelet as the wavelet basis. and These represent the low-frequency components of the visible light image and the infrared image, respectively; , and These represent the high-frequency components of the visible light image in the first direction, the second direction, and the diagonal direction, respectively. , and These represent the high-frequency components of the infrared image in the first direction, the second direction, and the diagonal direction, respectively. S2.2: The low-frequency component and The high-frequency components in each direction are used to characterize the main outline, skirt arrangement, and overall thermal response distribution of the insulator string. The high-frequency components in each direction are used to characterize the edges of the insulator skirts, crack outlines, damage boundaries, and edges of thermal anomaly regions, so as to input the corresponding frequency division feature modeling process.
4. The insulator defect detection method based on dual-mode frequency-assisted segmentation and intra-string heterogeneous sequence analysis according to claim 3, characterized in that: S3 includes: S3.1: The low-frequency component and Input the corresponding low-frequency global structural feature modeling modules to perform global structural feature modeling: ; in, and These represent the visible light low-frequency global structural feature modeling module and the infrared low-frequency global structural feature modeling module, respectively. and These represent the visible light low-frequency structural features and the infrared low-frequency structural features, respectively; the low-frequency global structural feature modeling module is used to extract the main outline of the insulator string, the skirt arrangement, the overall continuity, and the infrared response distribution trend; S3.2: For any high-frequency direction modality In direction High-frequency components The high-frequency detail feature modeling module for input direction performs direction detail feature modeling: ; in, For modal indexing, Represents visible light modes, Indicates infrared mode; These are directional indices, representing the high-frequency sub-bands in the first direction, the second direction, and the diagonal direction, respectively; Representing modes In direction High-frequency components, Representing modes In direction High-frequency detail feature modeling module in the direction of the direction, Representing modes In direction The high-frequency features in the direction of the insulator; the high-frequency detail feature modeling module is used to enhance the feature representation of the edges of the insulator skirts, cracks, damage boundaries and the edges of thermal anomaly regions; S3.3: The visible light low-frequency structural features The infrared low-frequency structural features and the high-frequency features of the directions in multiple directions As visible light frequency features and infrared frequency features, they are output to S4 for dual-modal feature interaction.
5. The insulator defect detection method based on dual-mode frequency-assisted segmentation and intra-string heterogeneous sequence analysis according to claim 4, characterized in that: S4 includes: S4.1: Represent the registration reliability information as a registration reliability map. Based on the aforementioned visible light low-frequency structural features The infrared low-frequency structural features and the registration reliability map Generate low-frequency mode weights: ; in, Represents the normalized exponential function, This indicates a feature concatenation operation. This represents the low-frequency weighted prediction function. and Let represent the visible light low-frequency weights and the infrared low-frequency weights, respectively, and for any spatial location... ,satisfy , and Representing spatial location Visible light low-frequency weighting and infrared low-frequency weighting at the location; S4.2: Based on the low-frequency mode weights, perform weighted interaction on the visible light low-frequency structural features and the infrared low-frequency structural features to obtain low-frequency interaction features: ; in, Indicates low-frequency interaction characteristics. This represents element-wise multiplication; S4.3: For any high-frequency direction According to direction High-frequency characteristics of visible light direction on Infrared high-frequency characteristics and the registration reliability map Generate high-frequency mode weights in the direction of generation: ; The direction is obtained by weighted interaction based on the high-frequency mode weights of the stated direction. High-frequency interaction features in the direction of upward movement: ; in, Indicates direction High-frequency weighted prediction function in the direction of the direction, and They represent directions respectively. Visible light high-frequency weighting and infrared high-frequency weighting, Indicates direction High-frequency interaction features in the direction of the direction, ; S4.4: The low-frequency interaction features and the high-frequency interaction features of the directions in multiple directions The input segmentation and decoding module performs decoding to obtain the segmentation result of the insulator string. The segmentation result of the insulator string includes at least an insulator string region mask. and the center response diagram of the unit within the string ;in, This represents the masking area of the insulator string. The segmentation and decoding module also outputs at least one of the following: the insulator string center axis probability map, the insulator string boundary probability map, and the insulator string endpoint probability map. ; in, This indicates the segmentation and decoding module.
6. The insulator defect detection method based on dual-mode frequency-assisted segmentation and intra-string heterogeneous sequence analysis according to claim 5, characterized in that: S5 includes: S5.1: According to the insulator string region mask Extract the insulator string skeleton, and perform linear fitting, curve fitting, or piecewise spline fitting on the insulator string skeleton to obtain the central axis of the insulator string: ; in, This indicates the central axis of the insulator string. This indicates the position of the normalized arc length along the central axis of the insulator string. and These represent the positions of the normalized arc lengths, respectively. The corresponding x-coordinate and y-coordinate of the image at that location; S5.2: Response diagram of the insulator string's central axis to the center of its neighboring unit. Sampling was performed to obtain the center response curve of the unit within the string: ; The high-frequency interaction features in each direction are sampled to obtain the high-frequency response curves in each direction: ; in, Indicates the position of the normalized arc length The central response of the intra-string cell at the location, Indicates direction Normalized arc length position High-frequency response at that direction, Indicates the position on the central axis of the insulator string The sampling neighborhood at that location, This indicates a pooling or statistical sampling operation. Indicates direction The amplitude of the high-frequency interaction characteristics in the direction of the direction; S5.3: Construct the synthesized unit response based on the center response curve of the intra-string unit and the high-frequency response curves in multiple directions: ; in, Indicates the position of the normalized arc length The integrated unit response at the location, , and These represent the directional high-frequency response curves in the first direction, the second direction, and the diagonal direction at the normalized arc length positions, respectively. The value at that location, , , and These represent the response weights corresponding to the center response of the unit within the string and the high-frequency responses in the three directions, respectively. The response weights are preset values or learnable parameters. S5.4: Based on the response of the integrated unit The local extrema determine the candidate center position of the unit within the string, and the positions of adjacent candidate centers are calculated. and Spacing between ,in, and They represent the first The and the first The normalized arc length position corresponding to the candidate center position of each unit within the string. Indicates the first A plurality of adjacent spacings; and a periodic reference spacing is calculated based on the plurality of adjacent spacings: ; in, Indicates the periodic reference interval, This represents the median value operation. This represents the distance between each adjacent element involved in the calculation. Indicates the number of adjacent spacings involved in the calculation; S5.5: When any of the adjacent spacings is greater than a preset multiple of the periodic reference spacing, candidate center positions of intra-string units are supplemented in the corresponding interval according to the periodic arrangement relationship of the insulator strings, and the spatial range of each intra-string unit is determined according to the adjacent candidate center positions, thereby obtaining multiple intra-string units distributed in physical arrangement order. ,in, Represents the set of units within a string. Indicates the first Units within a string, Indicates the number of units within the string.
7. The insulator defect detection method based on dual-mode frequency-assisted segmentation and intra-string heterogeneous sequence analysis according to claim 6, characterized in that: In step S6, the visible light structural features and infrared thermal response features are extracted, including: S6.1: For the first Units within a string Extract visible light structural features: ; in, Indicates the first Visible light structural features of units within a string, Represents the area of a cell within a string. Indicates the width of a unit within a string. Indicates the completeness of the outline. Indicates edge or texture intensity. Indicating shape symmetry, Indicates the difference in spacing or shape between units in adjacent strings; S6.2: For the first Units within a string Extracting infrared thermal response features: ; in, Indicates the first Infrared thermal response characteristics of each unit within a string. Indicates the first The average infrared response of each unit within a string Indicates the first The maximum infrared response of each unit within a string. Indicates the first The degree of dispersion of the infrared response of each unit within a string, Indicates the first Infrared response gradient of each unit within a string, Indicates the first The proportion of hot spot area within each unit of a string. Indicates the first The deviation of the infrared response of each cell within a string from the baseline of the entire string's infrared response, and: ; in, This represents the median value operation. This represents the average infrared response of each cell within the string. Indicates the number of units within the string; when the infrared image contains radiation temperature data, the infrared response is the actual temperature value; when the infrared image does not contain radiation temperature data, the infrared response is the infrared grayscale value or the thermal radiation response value. S6.3: Construct the visible light structure sequence based on the visible light structural features of each unit within the string, according to the physical arrangement order. The infrared thermal response sequence is constructed from the infrared thermal response characteristics of each unit within the string. ,in, Represents the visible light structure sequence. This represents the infrared thermal response sequence.
8. The insulator defect detection method based on dual-mode frequency-assisted segmentation and intra-string heterogeneous sequence analysis according to claim 7, characterized in that: In step S6, extracting the directional frequency feature and the reliability feature includes: S6.4: For the first Units within a string The high-frequency response of the unit in multiple directions of the visible light image and infrared image is statistically analyzed: ; in, Representing modes In direction Upper The high-frequency response of each unit within a string. For modal indexing, Represents visible light modes, Indicates infrared mode, For direction index, Indicates the first The number of pixels within a single unit in a string. Indicates the first Pixel positions within a unit of a string Indicates pixel position Mode In direction The amplitude of the high-frequency characteristics in the direction of the direction; S6.5: Based on the high-frequency response of the unit in multiple directions, construct the first... Directional frequency characteristics of units within a string: ; in, Indicates the first The directional frequency characteristics of units within a string , , These represent the unit high-frequency responses of the visible light modes in the first direction, the second direction, and the diagonal direction, respectively. , , These represent the unit high-frequency responses of the infrared mode in the first direction, the second direction, and the diagonal direction, respectively; S6.6: Based on the registration reliability information, segmentation confidence, visible light sharpness, and infrared signal-to-noise ratio, generate the first... Reliability characteristics of units within a string: ; in, Indicates the first Reliability characteristics of units within a string, Indicates the first Registration reliability of units within a string Indicates the first The segmentation confidence of each unit within a string. Indicates the first Visible light image quality of each unit within a string, Indicates the first Infrared image quality of each unit within a string, Represents the reliability feature generation function; S6.7: Construct a directional frequency sequence based on the directional frequency characteristics of each unit within the string, according to the physical arrangement order described. A reliability sequence is constructed from the reliability characteristics of each unit within the string. ,in, Represents the directional frequency sequence. Represents a reliability sequence. Indicates the number of units within the string.
9. The insulator defect detection method based on dual-mode frequency-assisted segmentation and intra-string heterogeneous sequence analysis according to claim 8, characterized in that: S7 includes: S7.1: Employs a first sequence encoder For the visible light structure sequence Perform sequence encoding to obtain the structure state sequence ; Using a second sequence encoder For the infrared thermal response sequence Sequence encoding is performed to obtain the thermal response state sequence. ; Using a third sequence encoder For the directional frequency sequence Perform sequence encoding to obtain the frequency state sequence. ; in, , and These represent the first sequence encoder, the second sequence encoder, and the third sequence encoder, respectively. , and These represent the structural state sequence, thermal response state sequence, and frequency state sequence, respectively. , and These respectively represent the visible light structure sequence, the infrared thermal response sequence, and the directional frequency sequence; At least two of the first, second, and third sequence encoders have different model parameters or different sequence modeling ranges; The structural state sequence The thermal response state sequence and the frequency state sequence Each by the first The structural state corresponding to each unit within a string Thermal response status and frequency state constitute; S7.2: For the first Each unit within a string, according to its structural state The thermal response state The frequency state and reliability characteristics Generate state interaction gating coefficients: ; in, , and They represent the first The interaction weights corresponding to the structural state, thermal response state, and frequency state of each unit within a string. Indicates the first The reliability characteristics of the unit within the string, i.e., the first unit in the reliability sequence. The elements corresponding to the units within each string Indicates the gating parameters, Represents the normalized exponential function, Indicates feature concatenation operation; S7.3: Selectively interact with each state according to the state interaction gating coefficient to obtain the first state. The fusion state of units within each string: ; The intra-string fusion state sequence is composed of the fusion states of all intra-string units. ,in, Indicates the first The fusion state of units within a string, This represents element-wise multiplication. Represents the sequence of fused states within the string. Indicates the number of units within the string; S7.4: When the registration reliability, segmentation confidence, or modal image quality of any unit within a string is lower than the preset conditions, reduce the interaction weight of the corresponding state of that unit within the string to reduce the impact of erroneous cross-modal information on the defect determination result.
10. The insulator defect detection method based on dual-mode frequency-assisted segmentation and intra-string heterogeneous sequence analysis according to claim 9, characterized in that: S8 includes: S8.1: According to the first The states of adjacent units within the same string and the overall statistical state of the insulator string are used to generate the first... Reference states of units within each string: ; in, Indicates the first Reference state of each unit within a string, Indicates the first Each unit within a string before and after The fusion state of units within adjacent strings, Indicates the reference range of cells within adjacent strings. This indicates the overall statistical status of the insulator string. This represents the reference state generation function; S8.2: Calculate the first The fusion state of units within a string With the reference state State deviation between: ; in, Indicates the first Anomaly scoring of units within a string, This represents a distance calculation function or a learnable anomaly scoring function. S8.3: When the abnormal score is... When the value exceeds the preset abnormal threshold, the first... Each string element is identified as an abnormal string element, and the corresponding defect category is output according to the defect classification function: ; in, This represents the defect classification function. Indicates the first The defect categories of each unit within a string; the defect categories include at least one of the following: missing insulator disc, missing shed, damaged shed, damaged body, crack, foreign matter attachment, and localized thermal anomaly; S8.4: The abnormal intra-string unit includes a structural abnormal unit, a thermal response abnormal unit, and a combined structural and thermal response abnormal unit; when the structural abnormal result and thermal response abnormal result of any intra-string unit are inconsistent and its registration reliability is lower than the preset condition, the defect confidence of the intra-string unit is reduced or a verification flag is output.
Citation Information
Patent Citations
Extra-high voltage line insulator string deep learning recognition method fused with infrared visible light
CN107507172A
A method and system for detecting insulator defects based on dual-light fusion of UAVs
CN115049663B
Method and device for detecting internal defects of insulator
CN117470859A
