Insulator detection method and system based on intelligent robot

Abstract

The invention discloses an insulator detection method and system based on an intelligent robot, and relates to the field of image detection, and the method comprises the steps: collecting an image through an inspection robot carrying a visible light and infrared camera, processing a visible light image through employing a dynamic correction algorithm fusing Retinex-HSV, and achieving the registration fusion of a multispectral image through employing an SURF feature point and an RANSAC algorithm; based on an improved Mask R-CNN network, precise segmentation of the insulator is achieved, and then defect features such as surface dirt, structural cracks and core rod deterioration are extracted hierarchically; and finally, generating a defect comprehensive probability by fusing multiple features through a D-S evidence theory, and automatically alarming when the defect comprehensive probability exceeds a threshold value. The method has the advantages that the data quality is improved through multi-spectral image adaptive acquisition, Retinex-HSV dynamic correction and SURF-RANSAC fusion, and efficient and accurate full-automatic defect detection of the power transmission line insulator is realized in combination with lightweight Mask R-CNN segmentation and D-S evidence theory multi-feature decision.

Description

Technical Field

[0001] This invention relates to the field of image detection, and in particular to an insulator detection method and system based on an intelligent robot. Background Technology

[0002] Insulators are crucial components in power transmission lines, and their performance directly impacts the safe and stable operation of the power grid. Traditional insulator inspection mainly relies on manual inspections or drone aerial photography, which suffers from low efficiency, high risk, and a high likelihood of missed inspections. With the rapid development of intelligent robots, computer vision, and artificial intelligence technologies, insulator inspection methods and systems based on intelligent robots have emerged.

[0003] Current insulator testing methods suffer from several significant disadvantages. Manual inspection and early-stage live-line testing are labor-intensive and dangerous, easily affected by subjective factors and environmental conditions. Methods like the short-circuit fork method and spark gap method have low accuracy and rely on personnel climbing poles, making their efficiency far from meeting the needs of large-scale line testing. Single-spectrum detection technologies have significant limitations; ultraviolet or infrared single-band detection is easily affected by weather and lighting conditions, making it difficult to comprehensively capture insulator characteristics and resulting in a high rate of false defect detection. In image processing, traditional methods lack adaptive correction capabilities for lighting conditions, leading to poor image quality under strong light and shadow. Furthermore, they often use simple stitching rather than pixel-level fusion, resulting in incomplete feature retention. Detection models are mostly traditional algorithms or unoptimized deep learning networks, which either have insufficient segmentation accuracy or slow running speed, making it difficult to balance efficiency and accuracy. Defect judgment often relies on single indicators and lacks multi-feature integrated models, making it prone to false alarms or missed alarms due to interference from single factors, and its reliability is far inferior to comprehensive judgment methods based on DS evidence theory. Summary of the Invention

[0004] To improve existing methods and systems, this paper provides an insulator detection method and system based on intelligent robots. This method uses an intelligent robot to autonomously inspect and collect multispectral images, which are then processed by illumination correction and fusion. Combined with improved Mask R-CNN segmentation and localization and DS evidence theory for comprehensive judgment, this method achieves efficient and accurate detection and reliable alarm for insulator defects.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: An insulator detection method based on intelligent robots includes: The inspection robot, equipped with a multispectral vision system, autonomously navigates along the power transmission line track and collects visible light images and infrared thermal images of insulators. Based on a dynamic correction algorithm that integrates Retinex theory and HSV color space, strong light suppression and shadow enhancement are performed on visible light images to construct an adaptive illumination compensation model and output an RGB image with balanced illumination. The SURF feature point extraction algorithm is used to extract feature points from the RGB image with illumination equalization and the infrared thermal image respectively. The RANSAC transformation matrix calculation algorithm is used to solve the spatial transformation matrix between the two types of images, and the spatial coordinates of the visible light and infrared images are aligned to obtain a pixel-level fused multispectral feature map. Based on the improved Mask R-CNN network, the input is a fused feature map, where the backbone network is replaced by a lightweight GhostNet module, and a spatial attention mechanism is embedded in the ROI Align layer to output an insulator skirt segmentation mask with local coordinates. For each insulator obtained from the segmentation, a hierarchical extraction operation of defect features is performed. The hierarchical extraction operation of defect features includes surface contamination detection, structural crack identification, and core rod deterioration diagnosis. A DS evidence theory model is established, integrating three types of features: contamination level, crack confidence, and degradation index, to obtain the comprehensive probability of defects. When the comprehensive probability of defects exceeds a preset probability threshold, a defect alarm is automatically generated.

[0006] Preferably, the inspection robot equipped with a multispectral vision system autonomously navigates along the transmission line track and collects visible light images and infrared thermal images of the insulators, specifically including: The inspection robot moves along the power transmission line track. When an insulator is detected to enter the field of view of the multispectral vision system, an image acquisition command is triggered. If no insulator is detected, the inspection robot will forcibly trigger an acquisition at a preset interval. After the acquisition command is triggered, the high-definition visible light camera and the infrared thermal imager start shooting simultaneously. The visible light camera captures the appearance image of the insulator, and the infrared thermal imager captures the temperature distribution image of the insulator. During the data acquisition process, the camera automatically adjusts its exposure parameters based on real-time changes in illumination, and the thermal imager automatically adjusts its temperature detection range.

[0007] Preferably, the dynamic correction algorithm based on the fusion of Retinex theory and HSV color space, which performs strong light suppression and shadow enhancement processing on visible light images, constructs an adaptive illumination compensation model, and outputs an RGB image with balanced illumination, specifically includes: The visible light RGB image is converted to the HSV color space through color space conversion, and three independent channels are separated: the hue (H) channel, the saturation (S) channel, and the lightness (V) channel. For the brightness V channel, large-scale filtering is used to suppress brightness in bright areas with brightness values ​​above the threshold, while small-scale filtering is used to enhance details in shadow areas with brightness values ​​below the threshold. Based on the actual illumination distribution of the image, weights are assigned to the filtering results of different areas. Obtain the pixel value distribution of the saturation S channel, apply high gain to low saturation areas and low gain to high saturation areas, normalize the processed pixel values, and obtain the optimized saturation S channel. The hue H channel is recombined with the optimized saturation S channel and lightness V channel, and converted back to the RGB color space to obtain a preliminary corrected image. Extract the grayscale mean and contrast variance of the initially corrected image, input them into the illumination adaptive compensation model to generate pixel-by-pixel compensation coefficients for brightness adjustment, and output an RGB image with balanced illumination.

[0008] Preferably, the step of extracting feature points from the illumination-equalized RGB image and the infrared thermal imaging image using the SURF feature point extraction algorithm, solving the spatial transformation matrix between the two types of images using the RANSAC transformation matrix calculation algorithm, aligning the spatial coordinates of the visible light and infrared images, and obtaining a pixel-level fused multispectral feature map specifically includes: The RGB image with equal illumination was converted into a single-channel grayscale image, and then light Gaussian denoising was performed on both the image and the infrared thermal image to remove noise. For the two preprocessed images, edge detection using the SURF algorithm is used to locate potential feature points with drastic gray-level changes. Then, effective feature point sets are obtained through thresholding and non-maximum suppression. For each valid feature point, a fixed-length feature descriptor is constructed, the similarity of the descriptors is calculated, and the descriptors are filtered according to a threshold to obtain valid feature point pairs between two images; Randomly select interior point samples from effective feature point pairs, solve the preliminary spatial transformation matrix, calculate the reprojection error of other point pairs and count the number of interior points, select the matrix with the most interior points as the optimal spatial transformation matrix, and obtain the coordinate mapping relationship between the two images. By adjusting the pixel coordinates of the infrared thermal image using the optimal spatial transformation matrix, the insulator structure in the infrared image is made to completely overlap with the visible light image. The aligned infrared image and the visible light image are then fused at the pixel level to generate a multispectral feature map.

[0009] Preferably, the improved Mask R-CNN network, with input fused feature maps, wherein the backbone network is replaced by a lightweight GhostNet module, a spatial attention mechanism is embedded in the ROI Align layer, and the output insulator skirt segmentation mask body with localization coordinates includes: The backbone network in the Mask R-CNN network is replaced with a lightweight GhostNet module. A spatial attention module is embedded after the ROI Align layer to perform channel pooling and convolution on the pooled feature map to generate a weight matrix, which is then multiplied pixel by pixel with the original feature map to enhance key features. Prepare a dataset containing multispectral images of insulators and label the categories, masks, and bounding boxes to train the improved Mask R-CNN network; After preprocessing the multispectral feature map, it is input into the network, and the output includes insulator category, preliminary mask and bounding box coordinates. The initial mask is binarized, and the complete mask is obtained through dilation, erosion, and filling processes. The positioning coordinates are obtained by calculating the bounding rectangle of the mask, and the positioning insulator skirt segmentation mask is output.

[0010] Preferably, the defect feature hierarchical extraction operation is performed on each of the segmented insulators. This defect feature hierarchical extraction operation includes surface contamination detection, structural crack identification, and core rod deterioration diagnosis, specifically including: Independent umbrella skirt regions are identified through connected component analysis. After removing small noise regions, they are sorted and numbered according to coordinates. The RGB and infrared sub-images of insulators with the same coordinate range in the insulator umbrella skirt segmentation mask are then extracted. Extract the gray-level co-occurrence matrix features of the RGB sub-images, input them into a preset classification model, and combine them with visually identifiable appearance features to determine the level of filth. The edges of the preprocessed RGB sub-image are extracted using the improved Canny algorithm, and morphological dilation is performed to connect the fractured crack edges. The confidence level is calculated by the ratio of the actual area / length of the crack to the maximum crack threshold. Locate the core rod region in the infrared sub-image, calculate the average temperature difference between the core rod and the skirt, and calculate the degradation index based on the temperature difference ratio.

[0011] Preferably, the step of establishing the DS evidence theory model, integrating three types of features—fouling level, crack confidence, and degradation index—to obtain the comprehensive defect probability, and automatically generating a defect alarm when the comprehensive defect probability exceeds a preset probability threshold, specifically includes: The identification framework of the DS evidence theory is defined, specifically divided into no defects, slight defects, moderate defects, and severe defects, and the characteristic parameters of the three types of defects are used as independent sources of evidence. The probabilities of the three types of evidence sources are assigned, the probability of each defect is determined according to the level of contamination, and the confidence level of cracks is allocated to each defect level according to the proportion of the value. Compare the probability distribution of the same defect level among the three sources of evidence, calculate the conflict coefficient, and if the conflict coefficient is ≤0.8, synthesize directly; otherwise, perform weighted processing to reduce the impact of unreliable evidence. For each defect level, the product of the probabilities of the two evidence sources is calculated, and the joint probability is obtained through normalization. The joint probability is then combined with the third evidence source to obtain the comprehensive defect probability. Set the judgment threshold for each defect level, and generate a defect alarm when the overall probability of the defect exceeds the threshold.

[0012] Furthermore, an insulator detection system based on an intelligent robot is proposed, comprising: Autonomous navigation module: This module controls the inspection robot to move along the transmission line track and triggers image acquisition commands within the insulator detection range; Autonomous navigation and multispectral acquisition module: The module controls the inspection robot equipped with a multispectral vision system to move autonomously along the power transmission line track, identify insulators and trigger the high-definition visible light camera and infrared thermal imager to acquire images simultaneously; Image illumination adaptive correction module: The module adopts a dynamic correction algorithm that integrates Retinex theory and HSV color space, separates the visible light image channel and specifically handles strong light, shadow and saturation issues, and outputs an RGB image with balanced illumination by combining the illumination adaptive compensation model. Multispectral image fusion module: The module extracts and matches feature points from the illumination-equalized RGB image and the infrared thermal image using the SURF algorithm, and uses the RANSAC algorithm to solve for the optimal spatial transformation matrix to align the image coordinates and generate a multispectral feature map; Insulator skirt segmentation and localization module: The module is based on an improved Mask R-CNN network with a lightweight GhostNet backbone network and a spatial attention mechanism embedded in the ROI Align layer, which outputs an insulator skirt segmentation mask with localization coordinates; Defect Feature Hierarchical Extraction Module: The module performs hierarchical detection of surface contamination, structural cracks and core rod deterioration on the segmented insulator by determining the contamination level through gray-level co-occurrence matrix, calculating crack confidence through improved Canny algorithm, and calculating core rod deterioration index through infrared temperature difference. Defect Comprehensive Judgment and Alarm Module: The module establishes a DS evidence theory model, uses the contamination level, crack confidence, and deterioration index as evidence sources to calculate the comprehensive probability of defects, compares it with a preset threshold, and automatically generates a defect alarm when the probability exceeds the limit. Processor: The processor is used to handle the calculation process of each formula and the construction calculation process of each model.

[0013] Compared with the prior art, the advantages of the present invention are: From the data acquisition perspective, the inspection robot autonomously navigates along its track, simultaneously acquiring visible light and infrared images using a multispectral vision system. It can also automatically adjust parameters based on lighting and temperature, ensuring both acquisition efficiency and data integrity. In image processing, an algorithm integrating Retinex theory and the HSV color space effectively addresses issues of strong light and shadow, improving image quality. The SURF and RANSAC algorithms are combined to achieve pixel-level image fusion, preserving rich features. Regarding the network model, the improved Mask R-CNN uses a lightweight GhostNet module with an embedded spatial attention mechanism, balancing segmentation accuracy and speed. Defect detection employs hierarchical extraction, integrating multiple features using DS evidence theory to accurately calculate the comprehensive probability of defects, ensuring reliable alarms and providing an efficient and accurate solution for transmission line insulator inspection. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the method proposed in this invention; Figure 2 This is a schematic diagram of image acquisition proposed in this invention; Figure 3 This is a schematic diagram of obtaining an RGB image with equal illumination as proposed in this invention; Figure 4 This is a schematic diagram of the multispectral feature map obtained by pixel-level fusion proposed in this invention; Figure 5 This is a schematic diagram of the insulator skirt segmentation mask with positioning coordinates proposed in this invention; Figure 6 This is a schematic diagram of the hierarchical extraction of execution defect features proposed in this invention; Figure 7 This is a schematic diagram illustrating the acquisition of the comprehensive probability of defects proposed in this invention. Detailed Implementation

[0015] The following description is intended to disclose the invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art.

[0016] An insulator inspection system based on an intelligent robot includes: Autonomous navigation module: This module controls the inspection robot to move along the transmission line track and triggers image acquisition commands within the insulator detection range; Autonomous navigation and multispectral acquisition module: The module controls the inspection robot equipped with a multispectral vision system to move autonomously along the power transmission line track, identify insulators and trigger the high-definition visible light camera and infrared thermal imager to acquire images simultaneously; Image illumination adaptive correction module: The module adopts a dynamic correction algorithm that integrates Retinex theory and HSV color space, separates the visible light image channel and specifically handles strong light, shadow and saturation issues, and outputs an RGB image with balanced illumination by combining the illumination adaptive compensation model. Multispectral image fusion module: The module extracts and matches feature points from the illumination-equalized RGB image and the infrared thermal image using the SURF algorithm, and uses the RANSAC algorithm to solve for the optimal spatial transformation matrix to align the image coordinates and generate a multispectral feature map; Insulator skirt segmentation and localization module: The module is based on an improved Mask R-CNN network with a lightweight GhostNet backbone network and a spatial attention mechanism embedded in the ROI Align layer, which outputs an insulator skirt segmentation mask with localization coordinates; Defect Feature Hierarchical Extraction Module: The module performs hierarchical detection of surface contamination, structural cracks and core rod deterioration on the segmented insulator by determining the contamination level through gray-level co-occurrence matrix, calculating crack confidence through improved Canny algorithm, and calculating core rod deterioration index through infrared temperature difference. Defect Comprehensive Judgment and Alarm Module: The module establishes a DS evidence theory model, uses the contamination level, crack confidence, and deterioration index as evidence sources to calculate the comprehensive probability of defects, compares it with a preset threshold, and automatically generates a defect alarm when the probability exceeds the limit. Processor: The processor is used to handle the calculation process of each formula and the construction calculation process of each model.

[0017] See Figure 1 As shown, an insulator detection method based on intelligent robots includes: Step 1: An inspection robot equipped with a multispectral vision system autonomously navigates along the power transmission line track to collect visible light images and infrared thermal images of the insulators; Step 2: Based on the dynamic correction algorithm that integrates Retinex theory and HSV color space, strong light suppression and shadow enhancement are performed on the visible light image to construct an adaptive illumination compensation model and output an RGB image with balanced illumination. Step 3: Extract feature points from the RGB image and the infrared thermal image using the SURF feature point extraction algorithm. Solve the spatial transformation matrix between the two types of images using the RANSAC transformation matrix calculation algorithm. Align the spatial coordinates of the visible light and infrared images to obtain a pixel-level fused multispectral feature map. Step 4: Based on the improved Mask R-CNN network, input the fused feature map, where the backbone network is replaced with a lightweight GhostNet module, a spatial attention mechanism is embedded in the ROI Align layer, and the output is an insulator skirt segmentation mask with localization coordinates. Step 5: Perform a hierarchical extraction operation on each insulator obtained from the segmentation. The hierarchical extraction operation includes surface contamination detection, structural crack identification, and core rod deterioration diagnosis. Step 6: Establish a DS evidence theory model, integrate three types of features: contamination level, crack confidence, and degradation index, and obtain the comprehensive probability of defects. When the comprehensive probability of defects is greater than the preset probability threshold, a defect alarm is automatically generated.

[0018] See Figure 2 As shown, the inspection robot equipped with a multispectral vision system autonomously navigates along the transmission line track and collects visible light images and infrared thermal images of the insulators, specifically including: The inspection robot moves along the power transmission line track. When an insulator is detected to enter the field of view of the multispectral vision system, an image acquisition command is triggered. If no insulator is detected, the inspection robot will forcibly trigger an acquisition at a preset interval. After the acquisition command is triggered, the high-definition visible light camera and the infrared thermal imager start shooting simultaneously. The visible light camera captures the appearance image of the insulator, and the infrared thermal imager captures the temperature distribution image of the insulator. During the data acquisition process, the camera automatically adjusts its exposure parameters based on real-time changes in illumination, and the thermal imager automatically adjusts its temperature detection range.

[0019] See Figure 3 As shown, based on a dynamic correction algorithm that integrates Retinex theory and the HSV color space, strong light suppression and shadow enhancement are applied to visible light images. An adaptive illumination compensation model is constructed to output a balanced RGB image. Specifically, this includes: The visible light RGB image is converted to the HSV color space through color space conversion, and three independent channels are separated: the hue (H) channel, the saturation (S) channel, and the lightness (V) channel. For the brightness V channel, large-scale filtering is used to suppress brightness in bright areas with brightness values ​​above the threshold, while small-scale filtering is used to enhance details in shadow areas with brightness values ​​below the threshold. Based on the actual illumination distribution of the image, weights are assigned to the filtering results of different areas. Obtain the pixel value distribution of the saturation S channel, apply high gain to low saturation areas and low gain to high saturation areas, normalize the processed pixel values, and obtain the optimized saturation S channel. The hue H channel is recombined with the optimized saturation S channel and lightness V channel, and converted back to the RGB color space to obtain a preliminary corrected image. Extract the grayscale mean and contrast variance of the initially corrected image, input them into the illumination adaptive compensation model to generate pixel-by-pixel compensation coefficients for brightness adjustment, and output an RGB image with balanced illumination.

[0020] Specifically, the brightness feature parameters of the pre-corrected image are extracted, the pre-corrected RGB image is converted into a grayscale image, and the weighted average of the red, green and blue channels is used to calculate the mean of all grayscale pixels, which reflects the overall brightness level of the image. Based on the grayscale image, the variance of the pixel brightness values ​​is calculated, which reflects the degree of difference between the bright and dark areas of the image. The larger the variance, the stronger the contrast between light and dark. The grayscale mean and contrast variance are input into a preset adaptive illumination compensation model. This model is trained on a large number of image samples under different illumination conditions and can determine the optimization direction based on the brightness characteristics of the current image. The model generates corresponding pixel-by-pixel compensation coefficients based on the original brightness value of each pixel and the illumination characteristics of its region. A coefficient greater than 1 indicates that the pixel needs to be brightened, and a coefficient less than 1 indicates that the pixel needs to be darkened. The formula for the adaptive illumination compensation model is as follows:

[0021] in, This is the Gamma correction factor. The pixel average value of the V channel in the HSV color space; The pixel-by-pixel compensation coefficient is multiplied one by one with the RGB channel pixel values ​​of the initially corrected image to achieve targeted brightness adjustment, ensuring that the bright areas are further darkened, the shadow areas are further brightened, and the normal areas remain stable, ultimately outputting a balanced RGB image.

[0022] See Figure 4 As shown, feature points are extracted from both the illumination-equalized RGB image and the infrared thermal imaging image using the SURF feature point extraction algorithm. The spatial transformation matrix between the two types of images is then calculated using the RANSAC transformation matrix calculation algorithm. The spatial coordinates of the visible light and infrared images are aligned to obtain a pixel-level fused multispectral feature map. Specifically, this includes: The RGB image with equal illumination was converted into a single-channel grayscale image, and then light Gaussian denoising was performed on both the image and the infrared thermal image to remove noise. For the two preprocessed images, edge detection using the SURF algorithm is used to locate potential feature points with drastic gray-level changes. Then, effective feature point sets are obtained through thresholding and non-maximum suppression. For each valid feature point, a fixed-length feature descriptor is constructed, the similarity of the descriptors is calculated, and the descriptors are filtered according to a threshold to obtain valid feature point pairs between two images; Randomly select interior point samples from effective feature point pairs, solve the preliminary spatial transformation matrix, calculate the reprojection error of other point pairs and count the number of interior points, select the matrix with the most interior points as the optimal spatial transformation matrix, and obtain the coordinate mapping relationship between the two images. By adjusting the pixel coordinates of the infrared thermal image using the optimal spatial transformation matrix, the insulator structure in the infrared image is made to completely overlap with the visible light image. The aligned infrared image and the visible light image are then fused at the pixel level to generate a multispectral feature map.

[0023] Specifically, the SURF algorithm is used to locate regions with drastic gray-level changes for each pixel in both images, such as the boundary between the insulator and the background, the folded edges of the insulator skirts, and areas of abrupt thermal distribution changes in the infrared image. Pixels in these regions are potential feature points. A gray-level change intensity threshold is set and adjusted according to the actual contrast of the images. For example, the threshold is slightly higher for high contrast in the visible light image and slightly lower for low contrast in the infrared image. Pixels with gray-level change intensity below the threshold are filtered out. These pixels mostly come from flat areas in the background and are considered pseudo-feature points. Retaining them would interfere with subsequent matching, so they need to be removed. For the potential feature points after threshold filtering, they are compared within their local neighborhoods. Only the single feature point with the most drastic gray-level change in the local area is retained, and other redundant feature points in the same neighborhood are deleted. Finally, the effective feature point sets for each of the two images are obtained. For each valid feature point in the two images, a fixed-size local neighborhood is taken around its coordinates to statistically analyze the gray-level gradient direction and gradient intensity distribution of pixels within the neighborhood. This information is then converted into a fixed-length feature descriptor. Each feature point descriptor in the visible light image is traversed, and its similarity to all feature point descriptors in the infrared image is calculated one by one. A small number of point pairs are randomly selected from the effective feature point pairs as interior point samples. Interior points refer to point pairs that conform to the true spatial transformation relationship between the two images. False matching exterior points are excluded. Based on these interior point samples, the preliminary spatial transformation matrix is ​​solved. Using a preliminary transformation matrix, the coordinates of all valid feature point pairs in the infrared image are reprojected onto the visible light image. The difference between the reprojected coordinates and the corresponding feature point coordinates in the visible light image is calculated. An error threshold is set, and point pairs with reprojection errors less than or equal to the threshold are determined as inliers. The total number of inliers corresponding to the preliminary matrix is ​​counted. Repeat the above process. Each iteration will yield a preliminary matrix and the corresponding number of interior points. Compare all iteration results and select the preliminary matrix with the largest number of interior points as the optimal spatial transformation matrix.

[0024] See Figure 5 As shown, based on the improved Mask R-CNN network, the input is a fused feature map, where the backbone network is replaced with a lightweight GhostNet module, and a spatial attention mechanism is embedded in the ROI Align layer. The output is an insulator skirt segmentation mask body with localization coordinates, including: The backbone network in the Mask R-CNN network is replaced with a lightweight GhostNet module. A spatial attention module is embedded after the ROI Align layer to perform channel pooling and convolution on the pooled feature map to generate a weight matrix, which is then multiplied pixel by pixel with the original feature map to enhance key features. Prepare a dataset containing multispectral images of insulators and label the categories, masks, and bounding boxes to train the improved Mask R-CNN network; After preprocessing the multispectral feature map, it is input into the network, and the output includes insulator category, preliminary mask and bounding box coordinates. The initial mask is binarized, and the complete mask is obtained through dilation, erosion, and filling processes. The positioning coordinates are obtained by calculating the bounding rectangle of the mask, and the positioning insulator skirt segmentation mask is output.

[0025] Specifically, the backbone network used for basic feature extraction in the original Mask R-CNN is removed, and a GhostNet module is added to reduce computation by leveraging its lightweight convolutional characteristics. The GhostNet module generates ghost feature maps and sequentially extracts low-, medium-, and high-dimensional features from the multispectral feature maps. Low-dimensional features: capture detailed information such as the edges and textures of the insulator skirts; medium-dimensional features: integrate local structural features; high-dimensional features: associate with global semantic information, providing multi-level feature support for subsequent segmentation. The ROI Align layer embeds a spatial attention mechanism. After the pooled feature map is output by the ROI Align layer of Mask R-CNN, a spatial attention module is inserted to compress the channel dimension of the pooled feature map output by ROI Align, highlighting the feature differences between different channels. For the compressed single-channel feature map, a 1×1 convolutional layer is used to learn the spatial weight distribution and generate a spatial weight matrix with the same size as the original pooling feature map. The generated spatial weight matrix is ​​then multiplied pixel by pixel with the original pooling feature map output by ROI Align to enhance the response values ​​of key features such as the umbrella skirt edge and contour, while weakening the interference features of the background region and improving the accuracy of subsequent segmentation. A mask binarization threshold is set, and pixels with a value ≥ 0.5 in the initial mask are set to 1, while pixels with a value < 0.5 are set to 0, generating a binary mask image. A small-sized structuring element is used to dilate the binary mask, expanding the pixel range of the umbrella skirt region and filling small holes inside the mask. After dilation, a structuring element of the same size is used for erosion, shrinking the pixel range of the umbrella skirt region and removing isolated small noise points at the mask edges. The mask after dilation and erosion is then inspected and filled with any remaining large holes, finally yielding a complete and noise-free insulator umbrella skirt segmentation mask. For each umbrella skirt's complete mask, count the coordinates of all pixels with a value of 1, find the minimum x-coordinate, maximum x-coordinate, minimum y-coordinate, and maximum y-coordinate of these pixels, and determine the final positioning coordinates of the umbrella skirt; associate the complete segmentation mask with the positioning coordinates of the corresponding umbrella skirt and output them to finally obtain the insulator umbrella skirt segmentation mask with positioning coordinates.

[0026] See Figure 6 As shown, a hierarchical defect feature extraction operation is performed on each insulator obtained from the segmentation. This hierarchical defect feature extraction operation includes surface contamination detection, structural crack identification, and core rod deterioration diagnosis, specifically including: Independent umbrella skirt regions are identified through connected component analysis. After removing small noise regions, they are sorted and numbered according to coordinates. The RGB and infrared sub-images of insulators with the same coordinate range in the insulator umbrella skirt segmentation mask are then extracted. Extract the gray-level co-occurrence matrix features of the RGB sub-images, input them into a preset classification model, and combine them with visually identifiable appearance features to determine the level of filth. The edges of the preprocessed RGB sub-image are extracted using the improved Canny algorithm, and morphological dilation is performed to connect the fractured crack edges. The confidence level is calculated by the ratio of the actual area / length of the crack to the maximum crack threshold. Locate the core rod region in the infrared sub-image, calculate the average temperature difference between the core rod and the skirt, and calculate the degradation index based on the temperature difference ratio.

[0027] See Figure 7 As shown, a DS evidence theory model is established, integrating three types of features: contamination level, crack confidence, and degradation index, to obtain the comprehensive defect probability. When the comprehensive defect probability exceeds a preset probability threshold, a defect alarm is automatically generated, specifically including: The identification framework of the DS evidence theory is defined, specifically divided into no defects, slight defects, moderate defects, and severe defects, and the characteristic parameters of the three types of defects are used as independent sources of evidence. The probabilities of the three types of evidence sources are assigned, the probability of each defect is determined according to the level of contamination, and the confidence level of cracks is allocated to each defect level according to the proportion of the value. Compare the probability distribution of the same defect level among the three sources of evidence, calculate the conflict coefficient, and if the conflict coefficient is ≤0.8, synthesize directly; otherwise, perform weighted processing to reduce the impact of unreliable evidence. For each defect level, the product of the probabilities of the two evidence sources is calculated, and the joint probability is obtained through normalization. The joint probability is then combined with the third evidence source to obtain the comprehensive defect probability. Set the judgment threshold for each defect level, and generate a defect alarm when the overall probability of the defect exceeds the threshold.

[0028] Specifically, the basic probability allocation formula for the soiling characteristic is as follows:

[0029] in, To determine the strength of defect evidence based on surface contamination characteristics, This is the sensitivity coefficient. In the Lab color space The standard deviation of the channel reflects the unevenness of the insulator surface color; the more severe the contamination, the better. The larger, Channels are used to describe red and green opposite colors. for The contamination threshold of the channel standard deviation. It is a natural constant; The joint probability is calculated level by level. For each defect level in the identification framework, the probability of evidence source 1 for that level is multiplied by the probability of evidence source 2 for that level to obtain the preliminary joint probability. The sum of the preliminary joint probabilities of the four levels is calculated, and the sum is divided by the preliminary joint probability of each level to obtain the normalized joint probability. The combined probability of evidence source 1 and evidence source 2 is obtained. The level-by-level multiplication and normalization process is followed. The probability of the combined probability of the two pieces of evidence for each level is multiplied by the probability of evidence source 3 for that level to obtain the preliminary comprehensive probability. The sum of the preliminary comprehensive probabilities of the four levels is calculated, and the sum is divided by the preliminary comprehensive probability of each level to obtain the final comprehensive defect probability.

[0030] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this specification. Additionally, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0031] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0032] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An insulator detection method based on intelligent robots, characterized in that, include: The inspection robot, equipped with a multispectral vision system, autonomously navigates along the power transmission line track and collects visible light images and infrared thermal images of insulators. Based on a dynamic correction algorithm that integrates Retinex theory and HSV color space, strong light suppression and shadow enhancement are performed on visible light images to construct an adaptive illumination compensation model and output an RGB image with balanced illumination. The SURF feature point extraction algorithm is used to extract feature points from the RGB image with illumination equalization and the infrared thermal image respectively. The RANSAC transformation matrix calculation algorithm is used to solve the spatial transformation matrix between the two types of images, and the spatial coordinates of the visible light and infrared images are aligned to obtain a pixel-level fused multispectral feature map. Based on the improved Mask R-CNN network, the input is a fused feature map, where the backbone network is replaced by a lightweight GhostNet module, and a spatial attention mechanism is embedded in the ROI Align layer to output an insulator skirt segmentation mask with local coordinates. For each insulator obtained from the segmentation, a hierarchical extraction operation of defect features is performed. The hierarchical extraction operation of defect features includes surface contamination detection, structural crack identification, and core rod deterioration diagnosis. A DS evidence theory model is established, integrating three types of features: contamination level, crack confidence, and degradation index, to obtain the comprehensive probability of defects. When the comprehensive probability of defects exceeds a preset probability threshold, a defect alarm is automatically generated.

2. The insulator detection method based on an intelligent robot according to claim 1, characterized in that, The inspection robot equipped with a multispectral vision system autonomously navigates along the power transmission line track and collects visible light images and infrared thermal images of insulators, specifically including: The inspection robot moves along the power transmission line track. When an insulator is detected to enter the field of view of the multispectral vision system, an image acquisition command is triggered. If no insulator is detected, the inspection robot will forcibly trigger an acquisition at a preset interval. After the acquisition command is triggered, the high-definition visible light camera and the infrared thermal imager start shooting simultaneously. The visible light camera captures the appearance image of the insulator, and the infrared thermal imager captures the temperature distribution image of the insulator. During the data acquisition process, the camera automatically adjusts its exposure parameters based on real-time changes in illumination, and the thermal imager automatically adjusts its temperature detection range.

3. The insulator detection method based on an intelligent robot according to claim 1, characterized in that, The dynamic correction algorithm based on the fusion of Retinex theory and HSV color space performs strong light suppression and shadow enhancement processing on visible light images, constructs an adaptive illumination compensation model, and outputs an RGB image with balanced illumination, specifically including: The visible light RGB image is converted to the HSV color space through color space conversion, and three independent channels are separated: the hue (H) channel, the saturation (S) channel, and the lightness (V) channel. For the brightness V channel, large-scale filtering is used to suppress brightness in bright areas with brightness values ​​above the threshold, while small-scale filtering is used to enhance details in shadow areas with brightness values ​​below the threshold. Based on the actual illumination distribution of the image, weights are assigned to the filtering results of different areas. Obtain the pixel value distribution of the saturation S channel, apply high gain to low saturation areas and low gain to high saturation areas, normalize the processed pixel values, and obtain the optimized saturation S channel. The hue H channel is recombined with the optimized saturation S channel and lightness V channel, and converted back to the RGB color space to obtain a preliminary corrected image. Extract the grayscale mean and contrast variance of the initially corrected image, input them into the illumination adaptive compensation model to generate pixel-by-pixel compensation coefficients for brightness adjustment, and output an RGB image with balanced illumination.

4. The insulator detection method based on an intelligent robot according to claim 1, characterized in that, The step of extracting feature points from the illumination-equalized RGB image and the infrared thermal imaging image using the SURF feature point extraction algorithm, solving the spatial transformation matrix between the two types of images using the RANSAC transformation matrix calculation algorithm, aligning the spatial coordinates of the visible light and infrared images, and obtaining a pixel-level fused multispectral feature map specifically includes: The RGB image with equal illumination was converted into a single-channel grayscale image, and then light Gaussian denoising was performed on both the image and the infrared thermal image to remove noise. For the two preprocessed images, edge detection using the SURF algorithm is used to locate potential feature points with drastic gray-level changes. Then, effective feature point sets are obtained through thresholding and non-maximum suppression. For each valid feature point, a fixed-length feature descriptor is constructed, the similarity of the descriptors is calculated, and the descriptors are filtered according to a threshold to obtain valid feature point pairs between two images; Randomly select interior point samples from effective feature point pairs, solve the preliminary spatial transformation matrix, calculate the reprojection error of other point pairs and count the number of interior points, select the matrix with the most interior points as the optimal spatial transformation matrix, and obtain the coordinate mapping relationship between the two images. By adjusting the pixel coordinates of the infrared thermal image using the optimal spatial transformation matrix, the insulator structure in the infrared image is made to completely overlap with the visible light image. The aligned infrared image and the visible light image are then fused at the pixel level to generate a multispectral feature map.

5. The insulator detection method based on an intelligent robot according to claim 1, characterized in that, The improved Mask R-CNN network takes a fused feature map as input, where the backbone network is replaced by a lightweight GhostNet module, and a spatial attention mechanism is embedded in the ROI Align layer. The output is an insulator skirt segmentation mask with localized coordinates, comprising: The backbone network in the Mask R-CNN network is replaced with a lightweight GhostNet module. A spatial attention module is embedded after the ROI Align layer to perform channel pooling and convolution on the pooled feature map to generate a weight matrix, which is then multiplied pixel by pixel with the original feature map to enhance key features. Prepare a dataset containing multispectral images of insulators and label the categories, masks, and bounding boxes to train the improved Mask R-CNN network; After preprocessing the multispectral feature map, it is input into the network, and the output includes insulator category, preliminary mask and bounding box coordinates. The initial mask is binarized, and the complete mask is obtained through dilation, erosion, and filling processes. The positioning coordinates are obtained by calculating the bounding rectangle of the mask, and the positioning insulator skirt segmentation mask is output.

6. The insulator detection method based on an intelligent robot according to claim 1, characterized in that, The defect feature hierarchical extraction operation is performed on each insulator obtained from the segmentation. The defect feature hierarchical extraction operation includes surface contamination detection, structural crack identification, and core rod deterioration diagnosis, specifically including: Independent umbrella skirt regions are identified through connected component analysis. After removing small noise regions, they are sorted and numbered according to coordinates. The RGB and infrared sub-images of insulators with the same coordinate range in the insulator umbrella skirt segmentation mask are then extracted. Extract the gray-level co-occurrence matrix features of the RGB sub-images, input them into a preset classification model, and combine them with visually identifiable appearance features to determine the level of filth. The edges of the preprocessed RGB sub-image are extracted using the improved Canny algorithm, and morphological dilation is performed to connect the fractured crack edges. The confidence level is calculated by the ratio of the actual area / length of the crack to the maximum crack threshold. Locate the core rod region in the infrared sub-image, calculate the average temperature difference between the core rod and the skirt, and calculate the degradation index based on the temperature difference ratio.

7. The insulator detection method based on an intelligent robot according to claim 1, characterized in that, The establishment of the DS evidence theory model integrates three types of features: contamination level, crack confidence, and degradation index, to obtain the comprehensive defect probability. When the comprehensive defect probability is greater than a preset probability threshold, a defect alarm is automatically generated. Specifically, this includes: The identification framework of the DS evidence theory is defined, specifically divided into no defects, slight defects, moderate defects, and severe defects, and the characteristic parameters of the three types of defects are used as independent sources of evidence. The probabilities of the three types of evidence sources are assigned, the probability of each defect is determined according to the level of contamination, and the confidence level of cracks is allocated to each defect level according to the proportion of the value. Compare the probability distribution of the same defect level among the three sources of evidence, calculate the conflict coefficient, and if the conflict coefficient is ≤0.8, synthesize directly; otherwise, perform weighted processing to reduce the impact of unreliable evidence. For each defect level, the product of the probabilities of the two evidence sources is calculated, and the joint probability is obtained through normalization. The joint probability is then combined with the third evidence source to obtain the comprehensive defect probability. Set the judgment threshold for each defect level, and generate a defect alarm when the overall probability of the defect exceeds the threshold.

8. An insulator inspection system based on an intelligent robot, used to implement the insulator inspection method based on an intelligent robot as described in any one of claims 1-7, characterized in that, include: Autonomous navigation module: This module controls the inspection robot to move along the transmission line track and triggers image acquisition commands within the insulator detection range; Autonomous navigation and multispectral acquisition module: The module controls the inspection robot equipped with a multispectral vision system to move autonomously along the power transmission line track, identify insulators and trigger the high-definition visible light camera and infrared thermal imager to acquire images simultaneously; Image illumination adaptive correction module: The module adopts a dynamic correction algorithm that integrates Retinex theory and HSV color space, separates the visible light image channel and specifically handles strong light, shadow and saturation issues, and outputs an RGB image with balanced illumination by combining the illumination adaptive compensation model. Multispectral image fusion module: The module extracts and matches feature points from the illumination-equalized RGB image and the infrared thermal image using the SURF algorithm, and uses the RANSAC algorithm to solve for the optimal spatial transformation matrix to align the image coordinates and generate a multispectral feature map; Insulator skirt segmentation and localization module: The module is based on an improved Mask R-CNN network with a lightweight GhostNet backbone network and a spatial attention mechanism embedded in the ROI Align layer, which outputs an insulator skirt segmentation mask with localization coordinates; Defect Feature Hierarchical Extraction Module: The module performs hierarchical detection of surface contamination, structural cracks and core rod deterioration on the segmented insulator by determining the contamination level through gray-level co-occurrence matrix, calculating crack confidence through improved Canny algorithm, and calculating core rod deterioration index through infrared temperature difference. Defect Comprehensive Judgment and Alarm Module: The module establishes a DS evidence theory model, uses the contamination level, crack confidence, and deterioration index as evidence sources to calculate the comprehensive probability of defects, compares it with a preset threshold, and automatically generates a defect alarm when the probability exceeds the limit. Processor: The processor is used to handle the calculation process of each formula and the construction calculation process of each model.

Images