A power line defect detection method and system based on visual recognition

Abstract

This invention provides a method and system for detecting power line defects based on visual recognition, belonging to the field of power line detection technology. Firstly, this invention employs dual-light registration and differential computation techniques using visible light and infrared images to generate a fused feature map that simultaneously reflects component thermal anomalies and material differences. Secondly, in the component localization stage, it abandons the black-box target detection model and instead utilizes probabilistic Hough transform combined with prior geometric knowledge specific to power lines for line segment selection and reconstruction. This not only reduces the dependence on large amounts of labeled data but also enhances the interpretability of the localization process, enabling the extraction of precise regions of key components such as conductors and insulators from complex backgrounds. In the core defect identification stage, the concept of a probabilistic saliency map is introduced, enhancing the prominence of defect areas. Subsequently, adaptive threshold segmentation and region growing algorithms are used to determine the complete defect contour, achieving accurate mapping from pixel-level features to object-level defects.

Description

Technical Field

[0001] This invention relates to the field of power line inspection technology, and in particular to a method and system for detecting power line defects based on visual recognition. Background Technology

[0002] Currently, power line operation and maintenance mainly relies on a combination of manual inspection and drone aerial photography. Manual inspection is inefficient and greatly affected by environmental and subjective factors. Automated inspection based on drones equipped with visible light cameras has become an industry trend. However, most existing automated inspection technologies rely on single visible light image analysis. These methods are easily affected by changes in lighting, weather conditions, and complex backgrounds. They are difficult to effectively identify early or minor defects such as micro-cracks in insulators, slight corrosion on conductor surfaces, and internal overheating of grading rings. The detection results often suffer from both low detection rates and high false alarm rates. Some technologies have attempted to introduce infrared thermal imagers to assist in diagnosis, but infrared images are usually used only for independent thermal fault analysis, disconnected from the visible light detection process. This fails to achieve deep information integration, resulting in an inability to effectively correlate the morphological and thermal anomalies of components. Existing methods lack such collaborative analysis mechanisms. Furthermore, in the component location stage, traditional methods often directly use general target detection networks such as YOLO or Faster. R-CNN identifies all components, but this method requires a large amount of labeled data to train the model, and the model is like a black box with poor interpretability. In the defect identification stage, most methods use end-to-end deep learning models to directly classify component images as normal or defective. This method lacks explicit modeling of defect features and an interpretable feature enhancement process, making it difficult to adapt to the diversity and concealment of various defects. At the same time, existing methods usually stop at giving a binary judgment of the presence or absence of defects or a rough bounding box, lacking precise quantitative assessment of defect size and severity, making it difficult to support the needs of precise and digital power grid line operation and maintenance management.

[0003] Therefore, it is necessary to provide a visual recognition-based method and system for detecting power line defects to solve the above-mentioned technical problems. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a method and system for detecting power line defects based on visual recognition, achieving the beneficial effect of reliable and accurate detection of line defects.

[0005] This invention provides a visual recognition-based method for detecting power line defects, comprising:

[0006] S1: Register the acquired visible light image and infrared image of the power line, then perform a difference operation between the registered infrared image (which is then converted to grayscale) and the registered visible light image to generate a difference feature map.

[0007] S2: The registered visible light image and the differential feature map are fused together to obtain a multi-channel fused image. The multi-channel fused image is then subjected to edge-preserving filtering and edge detection to obtain a binary edge map.

[0008] S3: Detect candidate line segments in the binary edge map using probabilistic Hough transform, and filter, connect and group the candidate line segments by combining predefined prior geometric constraint rules for the line to obtain the mask of key components.

[0009] S4: Based on the preset domain window, calculate the color statistical features and infrared material anomaly features of each pixel in the region corresponding to the key component mask in the visible light image and the differential feature map respectively. Based on the color statistical features and infrared material anomaly features, generate a defect probability saliency map of the region corresponding to the key component mask.

[0010] S5: Perform adaptive threshold segmentation based on image statistical characteristics on the defect probability saliency map to obtain candidate defect seed points. Based on the candidate defect seed points, execute the region growing algorithm to obtain the defect target contour.

[0011] S6: Based on the preset weighted evaluation model, analyze and evaluate all detected defect target contours and output a defect detection report.

[0012] Preferably, in step S1, the difference operation is an absolute value difference operation.

[0013] Preferably, in step S2, the edge-preserving filter employs a bilateral filter.

[0014] Preferably, in step S3, the filtering, linking, and grouping specifically include:

[0015] Connect collinear line segments with similar directions and endpoint distances within a preset threshold to form a complete component outline line segment.

[0016] Preferably, in step S4, the step of obtaining the defect probability saliency map includes:

[0017] Based on the mask positions of key components, the visible light component region image is cropped from the registered visible light image, and the differential component region image is cropped from the differential feature map;

[0018] The visible light component region image is converted from the RGB color space to the HSV and Lab color spaces to obtain the component HSV image and the component Lab image.

[0019] Based on a preset neighborhood window, the mean and standard deviation of the neighborhood pixel values ​​of the H and S components of each pixel in the HSV image of the component are calculated, and the mean and standard deviation of the neighborhood pixel values ​​of the a and b components of each pixel in the Lab image of the component are calculated to form color statistical features.

[0020] Based on a preset neighborhood window, the mean and standard deviation of the neighborhood pixel values ​​of each pixel in the differential component region image are calculated to form infrared material anomaly features;

[0021] The color statistical features of each pixel in the component HSV image and component Lab image are stitched together with the infrared material anomaly features of each pixel in the differential component region image to obtain the comprehensive features of each pixel.

[0022] Based on the comprehensive features of each pixel and the preset defect feature response function, the defect probability value of each pixel is calculated, and a defect probability saliency map is constructed based on all the defect probability values.

[0023] Preferably, the preset defect feature response function is a linear weighted function.

[0024] Preferably, in step S5, the step of obtaining candidate defect seed points includes:

[0025] The defect probability value of each pixel in the defect probability saliency map is mapped to discrete gray levels to obtain a defect probability gray map, and a gray histogram is calculated.

[0026] Traverse all gray levels in the defect probability grayscale image, and use each gray level as a candidate segmentation threshold. For each candidate segmentation threshold, calculate the inter-class variance when dividing the defect probability grayscale image into foreground and background based on the grayscale histogram.

[0027] The candidate segmentation threshold corresponding to the maximum value among all inter-class variances is determined as the adaptive segmentation threshold, and the pixels with gray levels greater than the adaptive segmentation threshold in the defect probability grayscale image are used as candidate defect seed points.

[0028] Preferably, mapping the defect probability value of each pixel in the defect probability saliency map to discrete gray levels is done by multiplying the defect probability value by the total number of gray levels minus 1, and then rounding to the nearest integer to obtain discrete gray levels.

[0029] Preferably, in step S5, the step of obtaining the defect target contour includes:

[0030] Use at least one candidate defect seed point as the initial growth region for region growth, and calculate the average defect probability value of the initial growth region.

[0031] Calculate the absolute value of the difference between the defect probability value of each unclassified pixel adjacent to the current growth region and the average defect probability value of all pixels in the current growth region;

[0032] All unclassified pixels with absolute values ​​less than the preset similarity tolerance threshold are merged into the current growth region, and the average defect probability value of the current growth region is updated.

[0033] Repeat the above steps until no new unclassified pixels can be merged into the current growth area. The growth area determined at this point is the defect target outline.

[0034] For the remaining candidate defect seed points that have not yet been assigned to any defect target contour, repeat the above steps to obtain all defect target contours.

[0035] This invention provides a power line defect detection system based on visual recognition, comprising:

[0036] The dual-light registration differential module is used to register the acquired visible light image and infrared image of the power line, and then perform differential operation between the registered infrared image (which is then converted to grayscale) and the registered visible light image to generate a differential feature map.

[0037] The fusion edge extraction module is used to perform channel fusion of the registered visible light image and the differential feature map to obtain a multi-channel fusion map. After edge-preserving filtering, the multi-channel fusion map is then subjected to edge detection to obtain a binary edge map.

[0038] The component geometry analysis module is used to detect candidate line segments in the binary edge map using probabilistic Hough transform, and to filter, connect and group the candidate line segments in combination with predefined prior geometric constraint rules of the line to obtain the key component mask.

[0039] The feature fusion saliency map module is used to calculate the color statistical features and infrared material anomaly features of each pixel in the region corresponding to the key component mask in the visible light image and the differential feature map, respectively, based on a preset neighborhood window. Based on the color statistical features and infrared material anomaly features, a defect probability saliency map of the region corresponding to the key component mask is generated.

[0040] The defect precise localization module is used to perform adaptive threshold segmentation based on image statistical characteristics on the defect probability saliency map, obtain candidate defect seed points, and execute a region growing algorithm based on the candidate defect seed points to obtain the defect target contour.

[0041] The defect quantification and evaluation module is used to analyze and evaluate all detected defect target contours based on a preset weighted evaluation model and output a defect detection report.

[0042] Compared with related technologies, the power line defect detection method and system based on visual recognition provided by this invention has the following advantages:

[0043] This invention first employs dual-light registration and differential computation techniques using visible and infrared images at the data level to generate a fused feature map that simultaneously reflects component thermal anomalies and material differences. This provides a richer and more reliable information foundation for subsequent analysis, thereby improving the sensitivity and anti-interference capability of defect detection. Secondly, in the component localization stage, it abandons the black-box general target detection model and instead utilizes probabilistic Hough transform combined with prior geometric knowledge specific to power lines for line segment selection and reconstruction. This not only reduces the dependence on large amounts of labeled data but also enhances the transparency and interpretability of the localization process. It can extract precise regions of key components such as conductors and insulators in complex backgrounds, laying the foundation for subsequent defect identification. Finally, in the core defect identification stage, multi-dimensional feature fusion and generalization techniques are introduced. The concept of a probability saliency map is used to calculate the statistics of each pixel in multiple color spaces and infrared differential features, and then fuse them to generate a defect probability saliency map, which enhances the prominence of defect areas. Subsequently, the adaptive threshold segmentation and region growing algorithm can intelligently extract defect seed points based on the statistical characteristics of the probability map and grow a complete defect outline. This series of operations achieves accurate mapping from pixel-level features to object-level defects, effectively avoiding false detections and missed detections common in traditional methods. Finally, the detected defect outline is subjected to multi-parameter quantitative analysis through a preset weighted evaluation model, and a detailed detection report including location, size and severity level is output, providing power grid operation and maintenance personnel with direct and usable decision-making basis, improving the automation level and engineering practical value of inspection work. Attached Figure Description

[0044] Figure 1 This is a flowchart of a power line defect detection method based on visual recognition according to the present invention;

[0045] Figure 2 This is a module structure diagram of a power line defect detection system based on visual recognition according to the present invention. Detailed Implementation

[0046] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the drawings, not all structures. Moreover, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0047] It should also be noted that, for ease of description, the accompanying drawings show only the parts relevant to the invention and not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but it may also have additional steps not included in the drawings. The process may correspond to a method, function, procedure, subroutine, subroutine, etc.

[0048] Example 1

[0049] A visual recognition-based method for detecting power line defects, in its specific implementation, such as... Figure 1 As shown, a flowchart of a power line defect detection method based on vision recognition is illustrated, including:

[0050] Step S1: Register the acquired visible light image of the power line with the infrared image, then perform a difference operation between the registered infrared image (which is then converted to grayscale) and the registered visible light image to generate a difference feature map.

[0051] Specifically, in step S1, the difference operation is an absolute value difference operation.

[0052] In the specific implementation process, firstly, visible light and infrared images of power lines are acquired, for example, using drones and fixed monitoring equipment, to ensure that the two images are synchronized in time and correspond in space. Then, pixel-level registration is performed. The registration process employs algorithms including, but not limited to, scale-invariant feature transform (SIFT) and accelerated robust feature transform (SURF) to extract and match key points in the images. The infrared image is precisely aligned to the visible light image by calculating, but not limited to, homography matrices and affine transformation models, to achieve complete consistency between the two in spatial coordinates. After registration, the infrared image is converted to a grayscale image. This process involves mapping the thermal radiation data captured by the infrared sensor into a single-channel grayscale image. The contrast is enhanced by linear stretching, for example, to facilitate subsequent processing. Then, a difference operation is performed, specifically an absolute difference operation. For each corresponding pixel in the registered visible light image and the grayscale infrared image, its grayscale value is obtained, the absolute value of the difference between the two is calculated, and this absolute value is used as the value of the new pixel, thereby generating a difference feature map. This difference feature map can highlight the difference areas between the visible light and infrared images. These differences may originate from thermal anomalies or material changes in power line components. The entire difference operation process ensures that the generation of the feature map does not rely on single spectral information, but integrates multi-source data, providing a richer feature basis for subsequent defect detection.

[0053] Step S2: Perform channel fusion between the registered visible light image and the differential feature map to obtain a multi-channel fused image. Perform edge-preserving filtering on the multi-channel fused image and then perform edge detection to obtain a binary edge map.

[0054] Specifically, in step S2, the edge-preserving filter employs a bilateral filter.

[0055] In the specific implementation process, the registered visible light image and the differential feature map are first fused together. The three-channel visible light image and the single-channel differential feature map are concatenated along the channel dimension to form a four-channel fused image containing richer information. This fused image retains both the spectral texture information of visible light and the thermal and material anomaly information represented by the difference between infrared and visible light, providing a reliable data foundation for subsequent edge detection. Next, edge-preserving filtering is applied to the obtained multi-channel fused image using a bilateral filtering algorithm. Bilateral filtering is a nonlinear filtering technique that considers two weighting coefficients: spatial proximity and pixel value similarity. It uses a Gaussian kernel function to weight neighboring pixels in the spatial domain and performs secondary weighting in the value domain based on pixel value differences. This effectively smooths background noise and fine textures while significantly preserving the edge information of key components. This characteristic supports the subsequent accurate extraction of power line edges. The specific implementation of bilateral filtering includes setting the standard deviation of the spatial domain Gaussian kernel and the standard deviation of the value domain Gaussian kernel. These two parameters can be determined based on image resolution. The filtering rate and noise level are adjusted to achieve the best filtering effect. After edge-preserving filtering, edge detection is performed on the filtered fused image. For example, the Canny edge detection algorithm is used. This algorithm first uses a Gaussian filter to smooth the image to further suppress noise. Then, it calculates the gradient intensity and direction of each pixel in the image. For example, the Sobel operator is used to obtain the gradient values ​​in the horizontal and vertical directions to obtain the gradient magnitude and direction. Next, non-maximum suppression technology is applied to refine the gradient magnitude, retaining only the maximum value points in the local gradient direction, thus making the edge lines thinner. Finally, a dual-threshold detection mechanism is used, setting a high threshold and a low threshold. Pixels with gradient magnitudes higher than the high threshold are identified as strong edge points, while pixels with gradient magnitudes lower than the low threshold are discarded. Pixels in between are only retained as edge points if they are connected to strong edge points. Through this series of operations, a clear and well-connected binary edge map is finally output, where white pixels represent detected edges and black pixels represent the background, providing a data basis for subsequent steps.

[0056] Step S3: Detect candidate line segments in the binary edge map using probabilistic Hough transform, and filter, connect and group the candidate line segments according to predefined prior geometric constraints to obtain the key component mask.

[0057] Specifically, in step S3, filtering, joining, and grouping include:

[0058] Connect collinear line segments with similar directions and endpoint distances within a preset threshold to form a complete component outline line segment.

[0059] In the specific implementation process, the binary edge image is first processed using probabilistic Hough transform. Probabilistic Hough transform detects candidate line segments in the image by randomly sampling edge points and accumulating statistical line segment parameter space. Its output includes the coordinate information of each line segment endpoint, thus obtaining a large number of initial candidate line segments. These segments include real power line component edges and interfering line segments in the background. Next, predefined prior geometric constraint rules are used to filter, connect, and group the candidate line segments. For example, geometric constraint rules are first defined based on prior knowledge of power line components. Examples include, but are not limited to, parallel line constraints, requiring conductor segments to have similar direction angles and spacing within a certain range; collinear constraints, allowing line segments with similar directions and endpoint distances within a preset threshold to be connected into longer complete line segments; and specific angle constraints, used to identify right angles and fixed angle relationships in the tower support structure. Then, line segment filtering is performed, calculating the direction angle and length of each line segment to eliminate obviously noisy line segments with chaotic directions and excessively short lengths. Parallel line constraints are used to group line segments with similar direction angles and spacing that conform to the conductor... The actual distance of line segments is retained as conductor candidates. Specific angular constraints are used to filter line segment groups that form right angles and other specific angles as tower structure candidates. Then, line segment connections are performed. For candidate line segments determined to belong to the same component, for example, different segments of a conductor, the consistency of distance and direction between their endpoints is checked. If the endpoint distance is within a preset threshold and the direction angle deviation is less than the tolerance, these line segments are connected to form a longer continuous line segment. This process involves line segment extension and intersection point calculation to ensure that the connected line segments are natural and smooth. Finally, line segments are grouped, classifying the complete line segments obtained after connection and line segment groups with specific geometric relationships. Each group of line segments corresponds to a potential power line component. After completing the filtering, connection, and grouping, key component masks are generated based on these grouped line segments. For each group of line segments, polygon fitting technology is used to delineate the image area covered by the component. The pixel values ​​within this area are set to white foreground, and the remaining areas are black background, thus generating a binary mask image for each key component, i.e., the key component mask.

[0060] Step S4: Based on the preset domain window, calculate the color statistical features and infrared material anomaly features of each pixel in the region corresponding to the key component mask in the visible light image and the differential feature map, respectively. Based on the color statistical features and infrared material anomaly features, generate a defect probability saliency map of the region corresponding to the key component mask.

[0061] Specifically, in step S4, the steps for obtaining the defect probability saliency map include:

[0062] Based on the mask positions of key components, the visible light component region image is cropped from the registered visible light image, and the differential component region image is cropped from the differential feature map;

[0063] The visible light component region image is converted from the RGB color space to the HSV and Lab color spaces to obtain the component HSV image and the component Lab image.

[0064] Based on a preset neighborhood window, the mean and standard deviation of the neighborhood pixel values ​​of the H and S components of each pixel in the HSV image of the component are calculated, and the mean and standard deviation of the neighborhood pixel values ​​of the a and b components of each pixel in the Lab image of the component are calculated to form color statistical features.

[0065] Based on a preset neighborhood window, the mean and standard deviation of the neighborhood pixel values ​​of each pixel in the differential component region image are calculated to form infrared material anomaly features;

[0066] The color statistical features of each pixel in the component HSV image and component Lab image are stitched together with the infrared material anomaly features of each pixel in the differential component region image to obtain the comprehensive features of each pixel.

[0067] Based on the comprehensive features of each pixel and the preset defect feature response function, the defect probability value of each pixel is calculated, and a defect probability saliency map is constructed based on all the defect probability values.

[0068] Specifically, the preset defect feature response function is a linear weighted function.

[0069] In the specific implementation process, firstly, based on the position information of the key component mask, a visible light component region image containing only the key component is precisely cropped from the registered visible light image. Simultaneously, a corresponding differential component region image is cropped from the differential feature map to ensure that subsequent feature extraction is limited to the component region, thereby eliminating background interference. Next, the cropped visible light component region image is converted from the RGB color space to the HSV and Lab color spaces to obtain the component HSV image and component Lab image. This allows for effective feature representation using the sensitivity of different color spaces to defects. Subsequently, based on a preset neighborhood window, the neighborhood... The window is a rectangular window centered on the current pixel. For example, a neighborhood window of size 5 by 5 pixels is used. Local neighborhood features of each pixel are calculated. For a part HSV image, the mean and standard deviation of all pixel values ​​for the H and S components within its neighborhood window are calculated for each pixel. Similarly, for a part Lab image, the mean and standard deviation of all pixel values ​​for the a and b components within its neighborhood window are calculated for each pixel. These statistics together constitute the color statistical features of that pixel. These statistical features can capture the color consistency, texture variations, and anomalous color shifts on the part surface. Furthermore, for differential part region images… Similarly, based on the same preset neighborhood window, the mean and standard deviation of all pixel values ​​within the neighborhood window are calculated for each pixel. These statistics constitute the infrared material anomaly feature of that pixel, reflecting the degree of thermal anomaly and material difference in a local area of ​​the component. Next, the color statistical features of each pixel are concatenated with the infrared material anomaly feature, that is, all statistics are sequentially connected to form a comprehensive feature vector. This comprehensive feature vector contains complementary information from multiple source data. This comprehensive feature vector represents the comprehensive feature of each pixel. Then, according to the preset defect feature response function, this function adopts... A linear weighting function multiplies each feature value in the comprehensive feature vector by a preset weighting coefficient, and sums all the weighted values ​​to obtain a defect probability value. The weighting coefficient is set based on expert experience to highlight the contribution of key features. Finally, a defect probability value is calculated for each pixel in the region corresponding to the key component mask. All these probability values ​​are combined into a defect probability saliency map of the same size as the visible light image. The pixel values ​​outside the region corresponding to the key component mask in the defect probability saliency map are 0, and the value of each pixel in the region corresponding to the key component mask in the defect probability saliency map is the calculated corresponding defect probability value.

[0070] Step S5: Perform adaptive threshold segmentation based on image statistical characteristics on the defect probability saliency map to obtain candidate defect seed points. Based on the candidate defect seed points, execute the region growing algorithm to obtain the defect target contour.

[0071] Specifically, in step S5, the step of obtaining candidate defect seed points includes:

[0072] The defect probability value of each pixel in the defect probability saliency map is mapped to discrete gray levels to obtain a defect probability gray map, and a gray histogram is calculated.

[0073] Traverse all gray levels in the defect probability grayscale image, and use each gray level as a candidate segmentation threshold. For each candidate segmentation threshold, calculate the inter-class variance when dividing the defect probability grayscale image into foreground and background based on the grayscale histogram.

[0074] The candidate segmentation threshold corresponding to the maximum value among all inter-class variances is determined as the adaptive segmentation threshold, and the pixels with gray levels greater than the adaptive segmentation threshold in the defect probability grayscale image are used as candidate defect seed points.

[0075] Specifically, mapping the defect probability value of each pixel in the defect probability saliency map to discrete gray levels is done by multiplying the defect probability value by the total number of gray levels minus 1, and then rounding to the nearest integer to obtain discrete gray levels.

[0076] Specifically, in step S5, the steps for obtaining the defect target contour include:

[0077] Use at least one candidate defect seed point as the initial growth region for region growth, and calculate the average defect probability value of the initial growth region.

[0078] Calculate the absolute value of the difference between the defect probability value of each unclassified pixel adjacent to the current growth region and the average defect probability value of all pixels in the current growth region;

[0079] All unclassified pixels with absolute values ​​less than the preset similarity tolerance threshold are merged into the current growth region, and the average defect probability value of the current growth region is updated.

[0080] Repeat the above steps until no new unclassified pixels can be merged into the current growth area. The growth area determined at this point is the defect target outline.

[0081] For the remaining candidate defect seed points that have not yet been assigned to any defect target contour, repeat the above steps to obtain all defect target contours.

[0082] In the specific implementation process, the generated defect probability saliency map is first processed. Each pixel value in the defect probability saliency map represents the probability that a defect exists at that point, with a value range of 0 to 1. To facilitate subsequent image segmentation, the probability values ​​need to be mapped to discrete gray levels. This is done by multiplying the defect probability value of each pixel by the total number of gray levels minus 1. The total number of gray levels is 256. Then, the values ​​are rounded to obtain integer gray levels between zero and two hundred and fifty-five, thus converting the defect probability saliency map into a defect probability grayscale map. Next, the grayscale histogram of this grayscale map is calculated, and the frequency of occurrence of each gray level is counted. Then, Otsu's method is used for adaptive threshold segmentation. This involves iterating through all possible gray levels from 0 to 255, using each gray level as a candidate threshold. Pixels are divided into foreground and background classes based on the gray-level histogram, and the inter-class variance is calculated. The formula is: foreground pixel ratio multiplied by background pixel ratio multiplied by the square of the difference between the average gray level of the foreground and the average gray level of the background. By comparing all inter-class variances, the gray level corresponding to the maximum value is determined as the adaptive segmentation threshold. Pixels with gray values ​​greater than or equal to the adaptive segmentation threshold in the defect probability grayscale map are extracted as candidate defect seed points. Then, a region growing algorithm is executed to obtain the defect target contour. First, spatially adjacent candidate defect seed points are merged into an initial growing region. The initial growing region contains at least one pixel, that is, at least one candidate defect seed point. The average defect probability value of all pixels in the candidate region is calculated. Then, the absolute value of the difference between the defect probability value of all unclassified pixels adjacent to the current growing region and the average defect probability value of the current growing region is checked. All adjacent unclassified pixels whose absolute value is less than the preset similarity tolerance threshold are merged into the current growing region. The average defect probability value of the merged current growing region is recalculated to dynamically update the region features. The above checking and merging process is repeated until no new pixels that meet the conditions can be merged. At this time, the current growing region is determined as a defect target contour. Finally, all remaining candidate defect seed points that have not yet been assigned to any defect contour are traversed. The above region growing process is repeated for each independent candidate defect seed point until all seed points are processed, thereby obtaining the precise spatial location and shape information of all defect target contours in the image, providing a data foundation for subsequent steps.

[0083] Step S6: Based on the preset weighted evaluation model, analyze and evaluate all detected defect target contours and output a defect detection report.

[0084] In the specific implementation process, the preset weighted evaluation model is a multi-parameter fusion decision model. Its core principle is to calculate a comprehensive defect severity score by weighted summation of multiple key features of each defect contour, thereby achieving standardized classification of defects. First, it is necessary to extract the evaluation parameters of each defect target contour, mainly including geometric parameters and feature parameters. For example, geometric parameters include, but are not limited to, the pixel area of ​​the defect target contour, the aspect ratio of the bounding rectangle of the defect target contour, and the center coordinates of the defect target contour. Among them, the pixel area of ​​the defect target contour is directly obtained by counting the total number of all pixels belonging to the defect target contour, and this value directly reflects the physical size of the defect target contour; the defect target contour... The aspect ratio of the circumscribed rectangle of the defect is obtained by finding the smallest rectangle that can completely enclose the defect target outline, i.e., the minimum circumscribed rectangle, and then calculating the ratio of the pixel length of the longer side to the pixel length of the shorter side of the rectangle. This ratio is used to characterize whether the shape of the defect target outline tends to be strip-shaped, block-shaped, or circular. The center coordinates of the defect target outline are determined by calculating the centroid of the outline. That is, the x and y coordinates of all pixels within the defect target outline are summed and then divided by the total pixel area of ​​the defect target outline to obtain the horizontal and vertical coordinates of the centroid. These coordinates are used to accurately locate the defect in image space. Feature parameters include, but are not limited to, the average probability value of the defect target outline in the original defect probability saliency map, and the defect target outline in the differential feature map. The average response intensity in the figure is as follows: The average probability value of the defect target contour in the original defect probability saliency map is obtained by overlaying the defect target contour onto the defect probability saliency map, extracting the defect probability values ​​of all pixels within the area covered by the defect target contour, and calculating their arithmetic mean. This average value represents the overall confidence level that the region belongs to the defect target contour. The average response intensity of the defect target contour in the differential feature map is obtained by overlaying the defect target contour onto the differential feature map, extracting the grayscale values ​​of all pixels within the area covered by the defect target contour, and calculating their arithmetic mean. This arithmetic mean quantifies the overall intensity of the infrared and visible light differential response exhibited by the defect target contour region, thus indicating thermal anomalies and material anomalies. The severity is then assessed, and each parameter is assigned a preset weighting coefficient. These coefficients are based on power industry standards and expert experience. For example, the defect area is given a higher weight because size is a key factor in assessing severity. The location parameter is assigned different weights depending on whether it is located in a critical load-bearing area. A linear weighted summation method is then used to calculate the comprehensive score for each defect profile. The comprehensive score equals the sum of each parameter value multiplied by its corresponding weighting coefficient. To ensure that parameters with different dimensions can be calculated collaboratively, all parameters must be normalized before weighting to convert them into dimensionless values ​​within the range of zero to one. Based on the calculated comprehensive score, the severity level of the defect is divided into multiple levels using preset scoring interval thresholds. For example, a score below 0...A score of 3 indicates a minor defect; scores between 0.3 and 0.7 indicate a general defect; and scores above 0.7 indicate a severe defect. The analysis results of all defects are then integrated to generate a structured inspection report. The report includes the inspection time, line section information, and detailed entries for each defect. Each defect record includes the defect number, location coordinates, geometric dimensions, severity level, comprehensive score, and maintenance suggestions generated based on the severity level. This ultimately realizes a vision-based power line defect detection method.

[0085] The working principle of the power line defect detection method based on visual recognition provided by this invention is as follows:

[0086] This invention constructs a multi-stage collaborative progressive analysis process. By performing pixel-level registration of visible light and infrared images and generating differential feature maps, it effectively integrates spectral characteristics and thermal radiation information, providing a richer input data foundation for defect detection. Then, channel fusion and edge-preserving filtering techniques are used to enhance structural features in the image. Furthermore, by combining probabilistic Hough transform with prior geometric knowledge specific to power lines, line segments are intelligently filtered and reconstructed to accurately locate key component regions. This step transforms low-level visual features into high-level structural understanding. Subsequently, within the component region, a local neighborhood analysis method is used to extract statistical features of multiple color spaces and infrared material anomaly features. A linearly weighted defect feature response function is used to fuse multi-dimensional features to generate a defect probability saliency map, thereby quantifying the probability of defects at the pixel level. Following this, adaptive threshold segmentation based on image statistical characteristics extracts high-confidence defect seed points from the probability map, and a region growing algorithm using dynamic similarity criteria is used to gradually outline the complete defect contour. Finally, a pre-set weighted evaluation model is used to perform multi-parameter fusion calculations on the geometric attributes and feature intensity of the defect contour, achieving quantitative grading of defect severity and output of a structured report.

[0087] Example 2

[0088] A vision-based power line defect detection system, in its specific implementation, such as... Figure 2 As shown, it illustrates a modular structure diagram of a vision-based power line defect detection system, including:

[0089] The dual-light registration differential module 100 is used to register the acquired visible light image and infrared image of the power line, and to perform differential operation between the registered infrared image (which is then converted to grayscale) and the registered visible light image to generate a differential feature map.

[0090] The fusion edge extraction module 200 is used to perform channel fusion of the registered visible light image and the differential feature map to obtain a multi-channel fusion map, and then perform edge detection on the multi-channel fusion map after edge-preserving filtering to obtain a binary edge map.

[0091] The component geometry analysis module 300 is used to detect candidate line segments in the binary edge map using probabilistic Hough transform, and to filter, connect and group the candidate line segments in combination with predefined prior geometric constraint rules of the line to obtain the key component mask.

[0092] The feature fusion saliency map module 400 is used to calculate the color statistical features and infrared material anomaly features of each pixel in the region corresponding to the key component mask in the visible light image and the differential feature map based on a preset neighborhood window, and to generate a defect probability saliency map of the region corresponding to the key component mask based on the color statistical features and infrared material anomaly features.

[0093] The defect precise localization module 500 is used to perform adaptive threshold segmentation based on image statistical characteristics on the defect probability saliency map, obtain candidate defect seed points, and execute a region growing algorithm based on the candidate defect seed points to obtain the defect target contour.

[0094] The defect quantification and evaluation module 600 is used to analyze and evaluate all detected defect target contours based on a preset weighted evaluation model and output a defect detection report.

[0095] The working principle of the power line defect detection system based on vision recognition provided by this invention is as follows:

[0096] This invention first uses a dual-light registration and difference module 100 to perform high-precision registration of synchronously acquired visible light and infrared images. The registered infrared image is then grayscaled and subjected to absolute value difference calculation with the visible light image to generate a difference feature map that simultaneously reflects thermal anomalies and material differences. Next, a fusion edge extraction module 200 performs channel-level cascade fusion of the registered visible light image and the difference feature map to form a four-channel fused image containing richer information. Bilateral filtering is then used for edge-preserving denoising to effectively suppress background noise while retaining key edge details. Finally, an edge detection algorithm is used to extract a clear binary edge map. Subsequently, a component geometry analysis module 300 uses probabilistic Hough transform to detect a large number of candidate line segments from the binary edge map. Combined with predefined prior geometric constraints, the line segments are intelligently filtered, connected, and grouped to generate an accurate key component mask, separating targets such as conductors and insulators from the complex background. Finally, a feature fusion saliency map module 400 crops the component region image based on the key component mask and performs HSV fusion on the component region image. The system calculates the local neighborhood color statistics of each pixel using the Lab color space, and extracts local infrared material anomaly features from the difference feature map. These features are then concatenated and the defect probability of each pixel is calculated using a linearly weighted defect feature response function, ultimately generating a defect probability saliency map. The defect precision localization module 500 first quantizes the defect probability saliency map into a grayscale image, uses the Otsu method for adaptive threshold segmentation to find the optimal segmentation threshold, and extracts candidate defect seed points. Starting from these seed points, a region growing algorithm is used to gradually merge adjacent pixels based on the similarity criterion of defect probability values ​​between pixels, dynamically updating the region growing parameters until all defect target outlines are fully delineated. Finally, the defect quantification and evaluation module 600 extracts the geometric parameters of each defect outline and its feature intensity in the probability map and difference map. A multi-parameter fusion calculation is performed using a preset weighted evaluation model to obtain a defect severity score. Based on the score range, the defect level is automatically classified, ultimately generating a structured inspection report containing detailed information such as defect location, size, and level.

[0097] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, are implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0098] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compactdisc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0099] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

Claims

1. A method for detecting defects in power lines based on visual recognition, characterized in that, The detection method includes the following steps: S1: Register the acquired visible light image and infrared image of the power line, then perform a difference operation between the registered infrared image (which is then converted to grayscale) and the registered visible light image to generate a difference feature map. S2: The registered visible light image and the differential feature map are fused together to obtain a multi-channel fused image. The multi-channel fused image is then subjected to edge-preserving filtering and edge detection to obtain a binary edge map. S3: Detect candidate line segments in the binary edge map using probabilistic Hough transform, and filter, connect and group the candidate line segments by combining predefined prior geometric constraint rules for the line to obtain the mask of key components. S4: Based on the preset domain window, calculate the color statistical features and infrared material anomaly features of each pixel in the region corresponding to the key component mask in the visible light image and the differential feature map respectively. Based on the color statistical features and infrared material anomaly features, generate a defect probability saliency map of the region corresponding to the key component mask. S5: Perform adaptive threshold segmentation based on image statistical characteristics on the defect probability saliency map to obtain candidate defect seed points. Based on the candidate defect seed points, execute the region growing algorithm to obtain the defect target contour. S6: Based on the preset weighted evaluation model, analyze and evaluate all detected defect target contours and output a defect detection report; In step S4, the steps for obtaining the defect probability saliency map include: Based on the mask positions of key components, the visible light component region image is cropped from the registered visible light image, and the differential component region image is cropped from the differential feature map; The visible light component region image is converted from the RGB color space to the HSV and Lab color spaces to obtain the component HSV image and the component Lab image. Based on a preset neighborhood window, the mean and standard deviation of the neighborhood pixel values ​​of the H and S components of each pixel in the HSV image of the component are calculated, and the mean and standard deviation of the neighborhood pixel values ​​of the a and b components of each pixel in the Lab image of the component are calculated to form color statistical features. Based on a preset neighborhood window, the mean and standard deviation of the neighborhood pixel values ​​of each pixel in the differential component region image are calculated to form infrared material anomaly features; The color statistical features of each pixel in the component HSV image and component Lab image are stitched together with the infrared material anomaly features of each pixel in the differential component region image to obtain the comprehensive features of each pixel. Based on the comprehensive features of each pixel and the preset defect feature response function, the defect probability value of each pixel is calculated, and a defect probability saliency map is constructed based on all the defect probability values.

2. The method of claim 1, wherein the method further comprises: In step S1, the difference operation is an absolute value difference operation.

3. The method of claim 2, wherein the method further comprises: In step S2, the edge-preserving filter employs a bilateral filter.

4. The method of claim 3, wherein the method further comprises: In step S3, filtering, joining, and grouping specifically include: Connect collinear line segments with similar directions and endpoint distances within a preset threshold to form a complete component outline line segment.

5. The power line defect detection method based on visual recognition according to claim 4, characterized in that, The preset defect feature response function is a linear weighted function.

6. The power line defect detection method based on visual recognition according to claim 5, characterized in that, In step S5, the steps for obtaining candidate defect seed points include: The defect probability value of each pixel in the defect probability saliency map is mapped to discrete gray levels to obtain a defect probability gray map, and a gray histogram is calculated. Traverse all gray levels in the defect probability grayscale image, and use each gray level as a candidate segmentation threshold. For each candidate segmentation threshold, calculate the inter-class variance when dividing the defect probability grayscale image into foreground and background based on the grayscale histogram. The candidate segmentation threshold corresponding to the maximum value among all inter-class variances is determined as the adaptive segmentation threshold, and the pixels with gray levels greater than the adaptive segmentation threshold in the defect probability grayscale image are used as candidate defect seed points.

7. The power line defect detection method based on visual recognition according to claim 6, characterized in that, Mapping the defect probability value of each pixel in the defect probability saliency map to discrete gray levels is done by multiplying the defect probability value by the total number of gray levels minus 1, and then rounding to the nearest integer.

8. The power line defect detection method based on visual recognition according to claim 7, characterized in that, In step S5, the steps for obtaining the defect target contour include: Use at least one candidate defect seed point as the initial growth region for region growth, and calculate the average defect probability value of the initial growth region. Calculate the absolute value of the difference between the defect probability value of each unclassified pixel adjacent to the current growth region and the average defect probability value of all pixels in the current growth region; All unclassified pixels with absolute values ​​less than the preset similarity tolerance threshold are merged into the current growth region, and the average defect probability value of the current growth region is updated. Repeat the above steps until no new unclassified pixels are merged into the current growth region. The growth region determined at this point is the defect target outline. For the remaining candidate defect seed points that have not yet been assigned to any defect target contour, repeat the above steps to obtain all defect target contours.

9. A power line defect detection system based on visual recognition, characterized in that, The detection system, applicable to a vision-based power line defect detection method as described in any one of claims 1 to 8, comprises: The dual-light registration differential module is used to register the acquired visible light image and infrared image of the power line, and then perform differential operation between the registered infrared image (which is then converted to grayscale) and the registered visible light image to generate a differential feature map. The fusion edge extraction module is used to perform channel fusion of the registered visible light image and the differential feature map to obtain a multi-channel fusion map. After edge-preserving filtering, the multi-channel fusion map is then subjected to edge detection to obtain a binary edge map. The component geometry analysis module is used to detect candidate line segments in the binary edge map using probabilistic Hough transform, and to filter, connect and group the candidate line segments in combination with predefined prior geometric constraint rules of the line to obtain the key component mask. The feature fusion saliency map module is used to calculate the color statistical features and infrared material anomaly features of each pixel in the region corresponding to the key component mask in the visible light image and the differential feature map, respectively, based on a preset neighborhood window. Based on the color statistical features and infrared material anomaly features, a defect probability saliency map of the region corresponding to the key component mask is generated. The defect precise localization module is used to perform adaptive threshold segmentation based on image statistical characteristics on the defect probability saliency map, obtain candidate defect seed points, and execute a region growing algorithm based on the candidate defect seed points to obtain the defect target contour. The defect quantification and evaluation module is used to analyze and evaluate all detected defect target contours based on a preset weighted evaluation model and output a defect detection report.

Images