Wire insulator defect detection method and system based on binocular vision

Abstract

The invention discloses a wire insulator defect detection method and system based on binocular vision, and the method comprises the steps: firstly obtaining a multi-view image of an insulator, and carrying out the calculation of the initial three-dimensional point cloud data of the insulator according to the multi-view image; then, performing point cloud denoising and dynamic illumination compensation processing on the initial three-dimensional point cloud data to obtain optimized three-dimensional point cloud data; and then, calling a three-dimensional defect detection model integrated with a plurality of modules such as a multi-scale feature extraction module, a mixed attention module, a feature alignment module and a feature enhancement module, and performing detection according to the three-dimensional point cloud data and the two-dimensional image of the insulator to obtain a defect detection result of the insulator. According to the embodiment of the invention, the binocular vision camera is used for acquiring the three-dimensional image to solve the shooting angle problem, the dynamic illumination compensation is used for coping with the illumination change, and the three-dimensional defect detection model is matched to overcome the small defect detection bottleneck, so that the defect detection accuracy is effectively improved, and the high-precision insulator defect detection is realized.

Description

TECHNICAL FIELD

[0001] The embodiment of the application relates to but is not limited to the technical field of image processing, and in particular to a power line insulator defect detection method and system based on binocular vision. BACKGROUND

[0002] With the continuous expansion of the power grid scale, the safety and stability of high-voltage transmission lines have become the core problem of the power industry. Among them, the insulator, as a key component of the transmission line, mainly functions to fix the conductor and provide insulating support between the conductor and the pole or tower, ensuring the transmission of current within the set path. However, once the insulator has defects, it will lead to a decrease in insulating performance, which may cause serious accidents such as partial discharge, flashover, and pollution flashover, and in extreme cases, even cause line tripping or large-scale power outage, seriously threatening the safe and stable operation of the power grid. Therefore, designing an efficient and accurate insulator defect detection method to improve the state perception and operation efficiency of power equipment is of great significance to ensure the safety and reliability of the power grid.

[0003] Currently, power line insulator inspection mainly relies on two-dimensional computer vision technology for detection, using unmanned aerial vehicles or fixed cameras to capture insulator images, and then using deep learning models for defect recognition. However, the current two-dimensional image detection method has inherent limitations, and its recognition effect is affected by factors such as shooting angle, light change, and obstruction interference, making it difficult to obtain complete spatial information of defects. At the same time, insulator defects are generally small, which comprehensively leads to insufficient detection accuracy and robustness. A new detection method is urgently needed to break through the limitations of two-dimensional vision detection and achieve accurate recognition and spatial positioning of insulator defects. SUMMARY

[0004] The following is a summary of the subject matter described in detail in this document. This summary is not intended to limit the scope of protection of the claims.

[0005] The embodiment of the application provides a power line insulator defect detection method and system based on binocular vision, which effectively improves the accuracy of defect detection by fusing unmanned aerial vehicle binocular vision three-dimensional modeling and deep learning technology, and realizes high-precision insulator defect detection.

[0006] In a first aspect, the embodiments of the present application provide a power line insulator defect detection method based on binocular vision, comprising: acquiring a multi-view image of an insulator, and calculating preliminary three-dimensional point cloud data of the insulator according to the multi-view image; performing point cloud denoising and dynamic light compensation processing on the preliminary three-dimensional point cloud data to obtain optimized three-dimensional point cloud data; calling a pre-trained three-dimensional defect detection model, and performing detection according to the three-dimensional point cloud data and a two-dimensional image of the insulator to obtain a defect detection result of the insulator; the three-dimensional defect detection model comprises a multi-scale feature extraction module, a hybrid attention module, a feature alignment module, a feature enhancement module, a defect candidate generation module and a defect result generation module, the multi-scale feature extraction module performs feature extraction and transformation processing of different scales on the three-dimensional point cloud data to obtain multi-scale geometric features; the hybrid attention module combines the two-dimensional image to perform attention feature extraction on the multi-scale geometric features to obtain attention weighted features; the feature alignment module performs feature alignment processing under different perspectives on the attention weighted features to obtain aligned features; the feature enhancement module performs encoding, feature transformation and feature mapping processing on the aligned features to obtain enhanced features; the defect candidate generation module performs feature extraction according to the enhanced features to obtain a candidate defect region; and the defect result generation module performs defect positioning according to the candidate defect region to obtain a defect detection result.

[0007] In combination with the first aspect, in an embodiment of the present application, the multi-scale feature extraction module comprises a multi-scale neighborhood grouping unit, a pooling layer and a multi-layer perception machine, the multi-scale neighborhood grouping unit comprises a plurality of neighborhood radius layers of different scales, each of the neighborhood radius layers is connected with the pooling layer, and the pooling layer is connected with the multi-layer perception machine.

[0008] In combination with the first aspect, in an embodiment of the present application, the hybrid attention module comprises a channel attention unit and a spatial attention unit, the channel attention unit comprises a connected feature channel layer and a squeeze excitation module, and the spatial attention unit comprises a connected spatial convolution layer and a pooling layer.

[0009] In combination with the first aspect, in an embodiment of the present application, the feature enhancement module comprises a neural implicit function unit and a feature fusion unit connected with each other, the neural implicit function unit comprises an encoding layer and a plurality of fully connected layers and a sine activation function connected with each other.

[0010] With reference to the first aspect, in an embodiment of the present application, the calculating the preliminary three-dimensional point cloud data of the insulator according to the multi-view images comprises: performing pixel intensity difference cost calculation on the multi-view images to obtain disparity values; calculating path cumulative costs of multiple directions according to the disparity values through a multi-direction path cost aggregation strategy; determining target disparity values according to the path cumulative costs of the multiple directions; and calculating the preliminary three-dimensional point cloud data of the insulator according to the target disparity values, a camera focal length and a binocular baseline distance.

[0011] With reference to the first aspect, in an embodiment of the present application, the performing point cloud denoising and dynamic light compensation processing on the preliminary three-dimensional point cloud data to obtain optimized three-dimensional point cloud data comprises: performing statistical filtering processing on the preliminary three-dimensional point cloud data, and removing abnormal points obtained after the processing to obtain denoised point cloud data; performing adaptive histogram equalization processing on the multi-view images according to the denoised point cloud data to obtain multi-view images with uniform light; and mapping gray values of the multi-view images with uniform light to the denoised point cloud data to obtain the optimized three-dimensional point cloud data.

[0012] With reference to the first aspect, in an embodiment of the present application, the multi-view images of the insulator are obtained according to the following steps: calibrating internal parameters and external parameters of a binocular vision camera, setting a synchronization mechanism of the binocular vision camera in combination with external trigger signals and time stamp alignment to obtain a preset binocular vision camera; and performing ring-type image acquisition on the insulator by using the preset binocular vision camera to obtain the multi-view images of the insulator.

[0013] With reference to the first aspect, in an embodiment of the present application, the defect result generation module comprises an image extraction unit, a detection head and a defect positioning unit connected with each other; the defect result generation module is used to perform defect positioning according to the candidate defect area to obtain a defect detection result, comprising: the image extraction unit is used to extract a feature image from the candidate defect area; the detection head is used to detect and classify the feature image to obtain defect probability distributions of multiple defects; and the defect positioning unit is used to calculate confidence degrees of the defects according to the defect probability distributions of the multiple defects and defect areas labeled in the three-dimensional point cloud data, and obtain a defect detection result according to the confidence degrees.

[0014] In a second aspect, the embodiments of the present application provide a power line insulator defect detection system based on binocular vision, characterized in that the system is applied to the power line insulator defect detection method based on binocular vision described above, and the system comprises: a data acquisition module configured to acquire multi-view images of an insulator and calculate preliminary three-dimensional point cloud data of the insulator according to the multi-view images; a data processing module configured to perform point cloud denoising and dynamic light compensation processing on the preliminary three-dimensional point cloud data to obtain optimized three-dimensional point cloud data; and a defect detection module configured to call a pre-trained three-dimensional defect detection model, detect according to the three-dimensional point cloud data and a two-dimensional image of the insulator, and obtain a defect detection result of the insulator. The three-dimensional defect detection model comprises a multi-scale feature extraction module, a hybrid attention module, a feature alignment module, a feature enhancement module, a defect candidate generation module, and a defect result generation module. The multi-scale feature extraction module performs feature extraction and transformation processing of different scales on the three-dimensional point cloud data to obtain multi-scale geometric features. The hybrid attention module combines the two-dimensional image to perform attention feature extraction on the multi-scale geometric features to obtain attention weighted features. The feature alignment module performs feature alignment processing of different views on the attention weighted features to obtain aligned features. The feature enhancement module performs encoding, feature transformation, and feature mapping processing on the aligned features to obtain enhanced features. The defect candidate generation module performs feature extraction according to the enhanced features to obtain a candidate defect region. The defect result generation module performs defect positioning according to the candidate defect region to obtain the defect detection result.

[0015] In combination with the second aspect, in an embodiment of the present application, the data acquisition module comprises a binocular vision camera.

[0016] In the embodiment of the present application, first, the multi-view image of the insulator is acquired, and the preliminary three-dimensional point cloud data of the insulator is calculated according to the multi-view image; then, the preliminary three-dimensional point cloud data is subjected to point cloud denoising and dynamic light compensation processing to obtain optimized three-dimensional point cloud data; then, a pre-trained three-dimensional defect detection model is called to detect according to the three-dimensional point cloud data and the two-dimensional image of the insulator to obtain the defect detection result of the insulator. The three-dimensional defect detection model includes a multi-scale feature extraction module, a mixed attention module, a feature alignment module, a feature enhancement module, a defect candidate generation module and a defect result generation module. The multi-scale feature extraction module performs feature extraction and transformation processing of different scales on the three-dimensional point cloud data to obtain multi-scale geometric features. The mixed attention module extracts attention features from the multi-scale geometric features in combination with the two-dimensional image to obtain attention weighted features. The feature alignment module performs feature alignment processing of the attention weighted features under different perspectives to obtain aligned features. The feature enhancement module performs encoding, feature transformation and feature mapping processing on the aligned features to obtain enhanced features. The defect candidate generation module extracts features from the enhanced features to obtain a candidate defect region. The defect result generation module locates defects according to the candidate defect region to obtain the defect detection result. The embodiment of the present application solves the problem of shooting angle by acquiring multi-view images through a binocular vision camera, uses dynamic light compensation to cope with light changes, acquires three-dimensional point cloud in a surrounding manner to solve the problem of occlusion caused by the elliptical structure of the insulator, and cooperates with the three-dimensional defect detection model to overcome the bottleneck of micro-defect detection. Compared with the traditional method, the embodiment of the present application combines the unmanned aerial vehicle binocular vision three-dimensional modeling and deep learning technology, breaks through the limitation of two-dimensional images, effectively improves the accuracy of defect detection, and realizes high-precision insulator defect detection. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 is a flowchart of a power line insulator defect detection method based on binocular vision provided by an embodiment of the present application; Figure 2 is a specific flowchart of step 110 in Figure 1 Figure 3 is a semi-global matching algorithm flowchart provided by an embodiment of the present application; Figure 4 is a specific flowchart of step 120 in Figure 1 Figure 5 is a structure diagram of a three-dimensional defect detection model provided by an embodiment of the present application; Figure 6 is a multi-scale feature extraction module structure diagram provided by an embodiment of the present application; Figure 7 ​​is a mixed attention module structure diagram provided by an embodiment of the present application; Figure 8 is a neural implicit function unit structure diagram provided by an embodiment of the present application; Figure 9 is a working principle flowchart of a three-dimensional defect detection model provided by an embodiment of the present application; Figure 10 is a structure diagram of a wire insulator defect detection system based on binocular vision provided by an embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0019] It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in an order different from that in the flowchart. The terms "first", "second", and the like in the specification and claims and the above-described drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the structures, proportions, sizes, etc. shown in the drawings of the present specification are only used to cooperate with the content disclosed in the specification, so that those skilled in the art can understand and read, and do not limit the conditions that can be implemented by the present application, so they do not have technical significance. Any modification of structure, change of proportion relationship or adjustment of size, without affecting the effect and purpose that can be achieved by the present application, should still fall within the scope of the technical content disclosed by the present application. At the same time, the terms such as "up", "down", "left", "right", "middle" and "one" used in the present specification are only for the purpose of clear understanding, and are not intended to limit the scope of the present application. The change or adjustment of the relative relationship, without substantially changing the technical content, is also considered as the scope of the present application.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0021] With the continuous expansion of the power grid, the safe and stable operation of high-voltage transmission lines has become the core concern of the power industry. As a key insulating support component in the transmission line, the insulator bears the function of fixing the conductor and isolating the charged body from the tower, and is the basis for ensuring the transmission of current along the designated path. However, the insulator is prone to defects such as surface cracks, umbrella skirt damage, and electric erosion marks during long-term operation due to environmental erosion, corona corrosion, mechanical stress, and other factors. Once a defect occurs, its insulating performance will decrease significantly, which may cause partial discharge, flashover, pollution flashover, and other faults, and in severe cases, it can lead to line tripping and even regional power outage, posing a major threat to the safety of the power grid. Therefore, developing efficient and accurate insulator defect detection technology to improve the state perception and operation efficiency of power equipment is of great significance to ensure the reliability of the power grid.

[0022] Currently, the inspection of wire insulators mainly relies on two-dimensional computer vision technology, which uses unmanned aerial vehicles equipped with cameras or fixed monitoring cameras to obtain insulator images, and combines deep learning models to achieve defect recognition. However, the two-dimensional image detection method has inherent technical bottlenecks: first, image acquisition is easily disturbed by factors such as shooting angle, light intensity, and obstacles, leading to blurred or missing defect features; second, two-dimensional images can only provide planar information and cannot fully present the spatial position and depth information of defects, especially for defects in the inner side and hidden parts of the insulator umbrella skirt; third, insulator defects are often small cracks and minor damage, which are easily affected by background noise in two-dimensional images, leading to missed or false detection. The above problems make it difficult for existing methods to meet the actual needs of power operation in terms of detection accuracy and robustness, and a new detection method is needed to break through the limitations of two-dimensional visual detection and achieve accurate recognition and spatial positioning of insulator defects.

[0023] In view of this, the embodiments of the present application provide a wire insulator defect detection method based on binocular vision and a wire insulator defect detection system based on binocular vision. The method uses a binocular vision camera to collect multi-angle images of the insulator, constructs preliminary three-dimensional point cloud data, and after optimization of point cloud denoising and dynamic light compensation, combines with two-dimensional images to realize defect detection through a three-dimensional defect detection model containing six modules such as multi-scale feature extraction and hybrid attention. The method uses surround collection to cope with occlusion, uses multi-scale feature extraction to overcome the difficulty of small defects, and integrates unmanned aerial vehicle binocular vision three-dimensional modeling and deep learning technology to break through the limitations of two-dimensional vision, effectively improve the accuracy of detection, and realize high-precision insulator defect detection.

[0024] The embodiments of the present application will be further described below with reference to the accompanying drawings.

[0025] Reference Figure 1 , Figure 1is a flowchart of a power line insulator defect detection method based on binocular vision provided by an embodiment of the present application. The flowchart can specifically include but is not limited to steps 110 to 130.

[0026] Step 110: Obtain multi-view images of the insulator, and calculate preliminary three-dimensional point cloud data of the insulator according to the multi-view images; Step 120: Perform point cloud denoising and dynamic light compensation processing on the preliminary three-dimensional point cloud data to obtain optimized three-dimensional point cloud data; Step 130: Call a pre-trained three-dimensional defect detection model to perform detection according to the three-dimensional point cloud data and the two-dimensional image of the insulator to obtain a defect detection result of the insulator.

[0027] The steps 110 to 130 are described in detail below.

[0028] In a feasible embodiment, the multi-view images of the insulator refer to multiple sets of two-dimensional images obtained by shooting or collecting the insulator from different directions, angles and distances. These images cover multiple observation angles of the insulator, such as the front, side, top and bottom surfaces and inclined angles, and can completely capture the surface structure, hidden parts (such as the inner side of the shed skirt and the edge connection) and spatial form information.

[0029] In a feasible embodiment, the multi-view images of the insulator can be obtained by a binocular vision camera. The specific steps include: calibrating the internal and external parameters of the binocular vision camera, and combining the external trigger signal and the timestamp alignment to set the synchronization mechanism of the binocular vision camera to obtain a preset binocular vision camera; and performing ring-type image collection on the insulator by using the preset binocular vision camera to obtain the multi-view images of the insulator. It can be understood that the insulator has obvious cylindrical symmetry, and the multi-view images of the insulator can be collected by using the ring shooting method to ensure complete coverage of the surface and reduce the influence of occlusion on three-dimensional reconstruction.

[0030] Specifically, to ensure high-precision measurement of the binocular vision system, Zhang's calibration method is first used to complete the internal and external parameter calibration of the camera. The internal parameters include focal length, principal point coordinates and distortion coefficients, and the external parameters include the rotation matrix and translation vector between the cameras. During the calibration process, a checkerboard calibration board is shot at multiple angles, and a least squares optimization algorithm is used to improve the parameter estimation accuracy and significantly reduce the re-projection error. The objective function is shown in formula (1):

[0031] wherein, represents the coordinates of the observed points (corner points) in the actual image. represents the theoretical coordinates calculated by using the internal and external parameters of the camera, The sum of square of re-projection error of all points (objective function).

[0032] In addition, in order to ensure that the binocular camera can acquire synchronous images at the same time, a hardware synchronization triggering mechanism can be adopted, combined with an external trigger signal and a timestamp alignment technology. It can be understood that the external trigger signal can be issued by a Field-Programmable Gate Array (FPGA) or a microcontroller, ensuring that the binocular camera is exposed at the same time, avoiding parallax error caused by time difference during acquisition. In the high-speed inspection scene, a combination strategy of high frame rate and short exposure time can be adopted, and an inertial measurement unit is used for time domain correction, so as to further reduce motion blur. In the binocular vision system, the geometric relationship between the two cameras can be represented by the following formula (2):

[0033] wherein, represents the three-dimensional point coordinates in the left camera coordinate system, represents the corresponding point coordinates in the right camera coordinate system, represents a rotation matrix (describing the rotation relationship of the two camera coordinate systems), represents a translation vector (describing the relative position between the two cameras).

[0034] In a feasible embodiment, after obtaining the multi-view images of the insulator, the preliminary three-dimensional point cloud data of the insulator can be further calculated according to the multi-view images. As shown in Figure 2 the execution process of step 110 of calculating the preliminary three-dimensional point cloud data of the insulator according to the multi-view images can include but is not limited to steps 210 to 240.

[0035] Step 210: Calculate the pixel intensity difference cost of the multi-view images to obtain the disparity value; Step 220: Calculate the path cumulative cost of multiple directions according to the disparity value through a multi-direction path cost aggregation strategy; Step 230: Determine the target disparity value according to the path cumulative cost of multiple directions; Step 240: Calculate the preliminary three-dimensional point cloud data of the insulator according to the target disparity value, the camera focal length and the binocular baseline distance.

[0036] It should be noted that the processing flow of steps 210 to 240 can be based on a semi-global matching (SGM) algorithm, and the specific flow includes: first, comparing the light-dark differences of the same name pixels of the multi-view images to calculate the preliminary disparity; then, combining the disparity information of adjacent pixels from multiple directions (such as horizontal, vertical, etc.), the cumulative matching cost in each direction is optimized and calculated; then, the optimal disparity is selected by synthesizing the cumulative cost in multiple directions to form an accurate disparity map; finally, using the disparity value, the focal length of the camera and the distance between the two cameras, the preliminary three-dimensional coordinate point cloud of the insulator is calculated. The SGM algorithm ensures the accuracy of disparity calculation and reduces the false matching through multi-directional cost aggregation. The specific flow will be described below in combination with the following formula. Figure 3 The specific flow will be described below in combination with the following formula.

[0037] First, the semi-global matching algorithm determines the disparity matching quality by calculating the cost based on the pixel intensity difference. For each pixel point of the left and right images , the matching cost is calculated by formula (3) under the assumption that the disparity is .

[0038] wherein, is the pixel value of the left image, is the pixel value of the right image, represents the disparity cost (i.e. the disparity value), and the smaller the value, the better the matching.

[0039] Next, in order to improve the stability of the disparity estimation, the semi-global matching algorithm adopts a multi-directional path cost aggregation strategy to calculate the path cumulative cost of the pixel points in multiple directions (such as horizontal, vertical, diagonal, etc.) to smooth the disparity results and reduce the noise influence. The path cumulative cost is calculated by the following recursive formula (4):

[0040] wherein, represents the path cumulative cost in direction ; controls the smoothness of small disparity changes; controls the smoothness of large disparity changes.

[0041] After completing the multi-directional path aggregation, the cumulative costs in all directions are summed and calculated, and the winner-takes-all strategy (WTA) is adopted, and the calculation formula is shown in formula (5), and the smallest cost disparity value is selected as the final estimation result.

[0042]

[0043] To the initially obtained disparity map, methods such as median filtering and weighted median filtering are used to remove isolated error matching points, smooth the disparity map, and further improve the disparity accuracy, so that the disparity result is more consistent with the actual scene. It can be understood that each pixel point of the disparity map corresponds to a pixel point of the original image, and its value is the disparity value of the point. Under normal circumstances, the disparity map can be regarded as a depth distribution map formed by arranging the disparity values of all pixels according to the spatial position of the original image.

[0044] Further, it is checked whether the disparity of each pixel is unique and reasonable. It is ensured that the matching cost under a certain disparity is significantly smaller than the cost under other disparities. If not, the pixel disparity is re-evaluated or marked as invalid.

[0045] To further improve the disparity accuracy, parabolic fitting and other methods can be used for sub-pixel interpolation based on several cost points near the integer disparity, to improve the integer disparity level to the sub-pixel level.

[0046] The consistency of the left and right images is used for checking. According to the calculated disparity, the left image is projected to the right image, and then the right image is projected back to the left image. By comparing the original left image position, it is checked whether there is a large deviation. The pixels with large deviation are re-calculated for disparity or marked as invalid points.

[0047] Through the principle of triangulation, the reliable disparity map obtained through the above processing is converted into depth information. Combined with the intrinsic parameters (such as focal length, principal point coordinates, etc.) and extrinsic parameters (such as rotation matrix, translation vector) of the camera, the depth information is projected back to the three-dimensional space to generate an insulator point cloud model. The calculation formula is shown in the following formula (6):

[0048] wherein, is the point cloud depth value; is the camera focal length; is the binocular baseline distance; is the target disparity value.

[0049] It can be understood that the insulator point cloud model has stored corresponding coordinate data, which can be directly exported to coordinate information through a tool to obtain the preliminary three-dimensional point cloud data of the insulator.

[0050] In a feasible embodiment, after completing the disparity calculation and generating the preliminary three-dimensional point cloud, the data can also be optimized to improve the reliability and robustness of the point cloud. As shown in formula (7), the execution process of the point cloud denoising and dynamic light compensation processing on the preliminary three-dimensional point cloud data in step 120 can include but is not limited to steps 410 to 430. Figure 4

[0051] ​Step 410: statistical filtering processing is performed on the preliminary three-dimensional point cloud data, and the abnormal points obtained after processing are removed to obtain denoised point cloud data; Step 420: adaptive histogram equalization processing is performed on the multi-view images according to the denoised point cloud data to obtain multi-view images with uniform illumination; Step 430: the gray value of the multi-view images with uniform illumination is mapped to the denoised point cloud data to obtain optimized three-dimensional point cloud data.

[0052] In a feasible embodiment, in step 410, for the problem of outliers in the preliminary three-dimensional point cloud data, a statistical filtering method is used to optimize the point cloud data. Statistical filtering is based on statistical analysis of the local neighborhood of the point cloud. Whether each point belongs to an abnormal point is judged by calculating the Euclidean distance mean of the point to its nearest neighbor point. The specific calculation formula is shown in formula (7):

[0053] wherein, represents the current point, represents the first neighbor point of the current point, and the average Euclidean distance of the point to the neighborhood.

[0054] Further, after the abnormal points are calculated, the abnormal points are removed to obtain denoised point cloud data.

[0055] It can be understood that due to the complex lighting conditions of the power inspection scene in the outdoor environment, there may be high-reflective areas on the surface of the insulator, resulting in loss of point cloud texture information or reduced contrast. Therefore, adaptive histogram equalization is introduced to dynamically compensate for the image. It performs histogram equalization in the local area of the image, avoiding the problem of over-enhancement that may be caused by global equalization. The core idea of adaptive histogram equalization is to perform histogram equalization in the local window of each pixel. That is, the gray histogram statistics and equalization are independently performed in the neighborhood window of each image pixel to adapt to the local lighting changes in complex environments.

[0056] In a feasible embodiment, in steps 420 and 430, for a certain pixel point in the image, a local window centered on the point can be selected, the gray histogram of all pixels in the window is calculated, and the probability density function is further obtained, and the calculation formula is shown in formula (8):

[0057] wherein represents the gray value of the pixel in the window the number of pixels in the window, is the total number of pixels in the window.

[0058] Then the obtained local window-in probability density and the original gray value The cumulative distribution function (CDF) of the local window is calculated as the basis of the gray mapping The formula is shown in equation (9):

[0059] Through the above cumulative distribution function, the original gray value is mapped to the enhanced new gray value The transformation relationship is shown in equation (10):

[0060] wherein, is the number of gray levels of the image. The mapping ensures that in each local area, the gray value can be as evenly distributed as possible in the entire dynamic range, thereby achieving more effective local contrast enhancement.

[0061] Further, by using the camera parameters, each three-dimensional point of the denoised point cloud is projected onto the multi-view image with uniform illumination, the corresponding pixel position is found, and the enhanced new gray value is obtained at the projected pixel position. The new gray value is added to the corresponding three-dimensional point cloud as the color or intensity attribute, so that each point has spatial coordinates and optimized gray information. Filter abnormal gray points and smooth the gray values of neighboring points to eliminate noise effects, and finally form an optimized three-dimensional point cloud data with enhanced gray attributes.

[0062] Through the processing flow of steps 410 to 430, the strategy of combining statistical filtering and adaptive histogram equalization is adopted to perform point cloud denoising and dynamic light compensation respectively, to enhance the quality of the point cloud and improve the accuracy of defect detection. The strategy solves the problem that the insulating sub of the power line is usually in a complex outdoor environment, and its surface may be disturbed by noise points, strong light reflection and other factors.

[0063] Referring to Figure 5 , Figure 5is a structural diagram of a three-dimensional defect detection model provided by an embodiment of the present application. The model includes a multi-scale feature extraction module 510, a hybrid attention module 520, a feature alignment module 530, a feature enhancement module 540, a defect candidate generation module 550, and a defect result generation module 560. Among them, the multi-scale feature extraction module 510 performs multi-scale geometric feature extraction and transformation processing on the three-dimensional point cloud data to obtain multi-scale geometric features; the hybrid attention module 520 combines two-dimensional images to extract attention features from the multi-scale geometric features to obtain attention weighted features; the feature alignment module 530 performs feature alignment processing on the attention weighted features under different viewing angles to obtain aligned features; the feature enhancement module 540 encodes, transforms and maps the aligned features to obtain enhanced features; the defect candidate generation module 550 extracts features according to the enhanced features to obtain candidate defect regions; and the defect result generation module 560 locates defects according to the candidate defect regions to obtain defect detection results.

[0064] In a feasible embodiment, the multi-scale feature extraction module 510 fuses multi-scale neighborhood grouping, rotation invariance enhancement and hybrid attention mechanism, which can comprehensively improve the feature expression ability of point cloud data and significantly enhance the recognition ability of cracks, stains and damage. As shown in the figure, Figure 6 The module includes a multi-scale neighborhood grouping unit, a pooling layer and a multi-layer perception machine: the multi-scale neighborhood grouping unit is composed of multiple neighborhood radius layers of different scales, each layer is connected with the pooling layer, and the pooling layer is further connected with the multi-layer perception machine, forming a hierarchical feature extraction structure.

[0065] In a feasible embodiment, the grouping strategy of the multi-scale neighborhood grouping unit can solve the problem of insufficient sensitivity of traditional geometric feature extraction models to defects of different scales. The strategy sets three neighborhood radii of different scales of 0.1mm, 0.3mm and 0.5mm to capture geometric change features of different scales on the surface of the insulator, and improve the perception ability of details such as cracks, stains and damage. Specifically, for any center point, its neighborhood point set is searched in each scale and features are extracted, and then fused through a multi-layer perception machine (MLP) to finally obtain geometric features of different scales. The calculation method of this process is shown in formula (11).

[0066]

[0067] Among them, is the initial feature of the point cloud (such as coordinates, normal vectors, etc.); represents the neighborhood point set within the radius MLP represents a multi-layer perception machine for feature transformation; Pooling represents a pooling operation for extracting local most significant features.

[0068] Through the multi-scale field grouping structure, the model can capture the geometric characteristics of the small defects on the insulator surface in different scale ranges and adapt to different point cloud densities, thereby improving the accuracy of crack and damage detection.

[0069] It can be understood that in actual application scenarios, complex backgrounds such as electric tower structures, vegetation shielding or strong light interference can cause significant interference to insulator defect detection. The hybrid attention module of the embodiment of the application fuses the hybrid attention mechanism of channel attention and spatial attention, combines channel attention and spatial attention, enhances the attention degree of the model to the defect area, and suppresses background noise.

[0070] As shown in Figure 7 , the hybrid attention module includes a channel attention unit and a spatial attention unit, the channel attention unit includes a connected feature channel layer and a squeeze excitation module, and the spatial attention unit includes a connected spatial convolution layer and a pooling layer.

[0071] Among them, the channel attention is used to learn the importance of different feature channels and adaptively adjust the weight. The SE (Squeeze-and-Excitation) module is used to calculate the channel attention, and the calculation method is as shown in formula (12):

[0072] Among them, represents the channel feature; represents the global average pooling; is a fully connected layer parameter; is a ReLU activation function, is a Sigmoid function.

[0073] The spatial attention captures the spatial relationship on the feature map through convolution operation and focuses on the key defect area. The calculation formula is as shown in formula (13):

[0074] Among them, is a 7x7 convolution kernel; represents the maximum pooling, represents the average pooling; as a spatial weight, is used to enhance the feature expression of the key area Finally, the channel attention and the spatial attention are combined, and the final attention weighted feature is calculated through formula (14):

[0075] This mechanism can effectively enhance the feature expression of the insulator defect area and suppress the background noise, thereby improving the accuracy and robustness of defect detection.

[0076] It can be understood that in binocular vision acquisition, the insulator can have a large rotation change due to attitude difference or installation angle difference, resulting in inconsistent expression of the same defect under different point cloud coordinate systems. To improve the robustness of the model to geometric changes, the feature alignment module of the embodiment of the application introduces a local reference system alignment technology to realize the rotation invariance of the features, and can realize feature alignment processing of the attention weighted features under different viewing angles to obtain aligned features.

[0077] Specifically, for any point and its neighborhood a local coordinate axis is constructed, so that the normal vector minimizes the projection error of all neighborhood points relative to the main direction, and the calculation method is as shown in formula (15):

[0078] Wherein, n represents the normal vector (usually a unit normal vector) of the local coordinate axis to be solved; argmin represents the n value corresponding to the minimum value of the target function with respect to n behind, that is, the normal vector n that minimizes the projection error sum; P refers to the points in the neighborhood; represents the position vector of the i-th point in the neighborhood; represents the position vector of the j-th point in the neighborhood. By transforming the local neighborhood to a unified reference system, feature alignment of various geometric defects under different viewing angles is realized, thereby improving the stable recognition ability of the model to defects such as cracks and breakage.

[0079] In a feasible embodiment, the feature enhancement module includes a neural implicit function unit and a feature fusion unit connected to each other. It should be noted that after completing the point cloud denoising and dynamic light compensation, the contrast of the insulator surface defects still needs to be further enhanced to realize more accurate separation of the crack, stain and breakage area. The neural implicit function unit is optimized by fusing Fourier encoding, high-frequency texture enhancement and double-modal feature mapping to realize deep collaborative expression of surface texture and geometric features; at the same time, based on the shape-guided defect candidate region generation mechanism, the robustness and accuracy of detection are significantly improved by jointly constraining the geometric (SDF) and texture difference features. As shown in Figure 8 , the neural implicit function unit includes an encoding layer and a plurality of fully connected layers connected to each other, and a sine activation function is used between the fully connected layers.

[0080] Specifically, to enhance the expression ability of the model to high-frequency defect textures such as cracks and stains on the surface of the insulator, the neural implicit function unit is combined with Fourier encoding in the embodiment to improve the sensitivity of the model to high-frequency signals. Given the original input coordinates, the process of mapping it to a high-dimensional feature space by Fourier encoding is as shown in formula (16):

[0081] wherein, is a randomly initialized Fourier transform matrix; this mapping can convert low-frequency signals into high-frequency features, making it easier for the model to learn subtle changes in surface texture, such as the subtle differences in crack edges.

[0082] In addition, a sine function is used to activate the weight matrix and bias term instead of the traditional ReLU for feature transformation, to improve the network's fitting ability for high-frequency information, as shown in equation (17):

[0083] This activation mechanism can effectively improve the network's sensitivity to small defects and significantly improve the recognition accuracy in complex texture backgrounds.

[0084] It can be understood that the defects on the surface of the insulator often exhibit joint changes in geometric features (such as surface depressions, cracks) and texture features (such as stains, burn marks). Relying solely on geometric or texture information may lead to false positives (such as mistaking normal surface texture changes for defects), and the feature fusion unit of the embodiment combines geometric and texture information to enhance the contrast of the defect area. The joint implicit space construction is shown in equation (18):

[0085] wherein, is the Fourier-encoded high-frequency texture feature; represents the local geometric feature; represents the color texture information; A fully connected neural network is used for feature fusion.

[0086] In a feasible embodiment, after completing feature fusion, to achieve effective extraction of the candidate defect area (ROI), the defect candidate generation module uses a region generation strategy based on joint shape and texture constraints. First, calculate the SDF value of each point through geometric constraints, and set a threshold When , it is considered that there is a geometric anomaly. Second, in terms of texture processing, calculate the color difference between the current point and the neighborhood points, and set a threshold When , it is considered that there is a color anomaly. Finally, the candidate defect area (ROI) is determined by the intersection of the geometric anomaly and the texture anomaly, and the calculation process is shown in equation (19):

[0087] This process effectively suppresses false positives caused by normal surface texture fluctuations by fusing shape and texture local saliency, significantly improving the accuracy and stability of defect area extraction.

[0088] In an embodiment, the defect result generation module comprises an image extraction unit, a detection head and a defect positioning unit connected to each other. When the defect result generation module is used to perform defect positioning according to the candidate defect region to obtain the defect detection result, the image extraction unit can be used to extract a feature image from the candidate defect region; the detection head can be used to detect and classify the feature image to obtain the defect probability distribution of multiple defects; the defect positioning unit can be used to calculate the confidence of each defect according to the defect probability distribution of multiple defects and the labeled defect region in the three-dimensional point cloud data, and obtain the defect detection result according to the confidence.

[0089] Specifically, the detection head classifies the input as the RGB texture image or the geometric depth map extracted from the candidate defect region, receives the probability distribution of the output three-class defects of the three-dimensional defect detection network 3D-DNet via the Softmax layer, and calculates the probability distribution in the manner shown in equation (20):

[0090] wherein, is the probability distribution of each class of defects, and the class corresponding to the maximum probability value is taken as the final classification result.

[0091] In an embodiment, in order to further support defect visualization analysis and manual review, the application labels the defect region in the three-dimensional point cloud data and generates a three-dimensional bounding box and a confidence heat map. For the classified defect region, the three-dimensional bounding box is generated by clustering combined with point cloud geometric information: the K-Means is used to cluster the defect point cloud, the center region of different class defects is extracted, then the minimum outer package cube is fitted for each clustering region to demarcate the spatial range of cracks, stains and damage; finally, the confidence of each defect region is calculated combined with the defect probability output by the classification network, and visualized by color coding.

[0092] In an embodiment, the overall working principle of the three-dimensional defect detection model is as shown in Figure 9As shown, it realizes high-precision detection of insulator defects through a multi-stage processing flow: first, a multi-scale geometric feature extraction module is used to extract rotation-invariant features from the point cloud data generated from the multi-view images collected by the binocular vision camera, combined with multi-scale neighborhood grouping, local reference system alignment, and hybrid attention mechanism. This design enhances the model's sensitivity to subtle defects such as micro-cracks and damage, effectively overcoming detection interference caused by changes in object posture. Subsequently, a neural implicit function unit is introduced, which integrates the geometric shape information and surface texture features of the point cloud in depth with the help of Fourier encoding technology and dual-modal feature fusion strategy. At the same time, through shape-texture joint constraint, high-confidence defect candidate regions are selected. Finally, the candidate regions are passed into the detection head for defect classification, and combined with three-dimensional clustering algorithm and minimum bounding cube fitting method, the spatial positioning of defects is realized. The final output is a three-dimensional visual detection result with confidence heat map, building a complete closed-loop detection link from feature learning, candidate region generation to decision output. This model combines unmanned aerial vehicle binocular vision three-dimensional modeling and deep learning technology, breaking through the limitations of traditional two-dimensional image detection, effectively improving the accuracy and reliability of insulator defect detection.

[0093] The following is a specific example of the insulator defect detection method based on binocular vision.

[0094] In a power inspection scenario, the insulator on the power transmission line needs to be detected for defects. The three-dimensional defect detection model can be applied according to the following embodiment: first, a drone equipped with a binocular vision camera is used to fly around the insulator on the power transmission line and take pictures of the insulator from multiple angles to obtain multi-view image data. These images cover different sides and rotation angles of the insulator. Then, the multi-view images collected are processed to generate preliminary three-dimensional point cloud data of the insulator points. After the preliminary three-dimensional point cloud data is processed for point cloud denoising and dynamic light compensation, the point cloud data is input into a multi-scale geometric feature extraction module to analyze the geometric structure of the insulator point cloud from different scales through multi-scale neighborhood grouping; the local reference system alignment (feature alignment module) is used to eliminate the influence of different angles (i.e. rotation) of the insulator in space; with the help of the hybrid attention mechanism (hybrid attention module), the area where there may be a small crack or damage is focused on, the rotation-invariant feature is extracted, and the perception ability of subtle defects is enhanced. Then, the extracted features are transmitted to the feature enhancement module, the Fourier coding is used to process the features through the neural implicit function unit, and the geometric shape features of the insulator are combined with the surface texture features through the dual-modal feature fusion. Through shape-texture joint constraint, high-confidence defect candidate regions are selected, such as marking parts suspected to have cracks or damage. Then, the defect candidate regions are input into the detection head for defect classification to determine whether the candidate regions are real defects and the type of the defects. The three-dimensional clustering algorithm is used to cluster the defect points, and the minimum bounding cuboid fitting is used to determine the specific location and range of the defects in the three-dimensional space. Finally, the three-dimensional visualization result with a confidence heat map is output. Through the visualization interface, the operator can directly observe the location, size and confidence level of the defects on the insulator, which facilitates the maintenance and replacement of the insulator with defects.

[0095] Reference Figure 10 , Figure 10 is a structure diagram of a power line insulator defect detection system based on binocular vision provided by an embodiment of the present application. The system can be applied to the power line insulator defect detection method based on binocular vision in the foregoing embodiment. The system comprises: a data acquisition module 1010 configured to acquire multi-view images of the insulator and calculate preliminary three-dimensional point cloud data of the insulator according to the multi-view images; a data processing module 1020 configured to perform point cloud denoising and dynamic light compensation processing on the preliminary three-dimensional point cloud data to obtain optimized three-dimensional point cloud data; and a defect detection module 1030 configured to call a pre-trained three-dimensional defect detection model to perform detection according to the three-dimensional point cloud data and a two-dimensional image of the insulator to obtain a defect detection result of the insulator.

[0096] In a feasible embodiment, the data acquisition module comprises a binocular vision camera.

[0097] It should be noted that the wire insulator defect detection system is fully compatible with the aforementioned wire insulator defect detection method, and the function implementation logic and principles of each module of the system are one-to-one corresponding to the technical solutions of the method. The specific functions of the modules can be referred to the related detailed description in the foregoing, which will not be described here again.

[0098] The above description of disclosed embodiments enables one of ordinary skill in the art to make or use the application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting defects in electric wire insulators based on binocular vision, characterized in that: include: Acquire multi-view images of the insulator, and calculate preliminary three-dimensional point cloud data of the insulator based on the multi-view images; performing point cloud denoising and dynamic illumination compensation processing on the preliminary three-dimensional point cloud data to obtain optimized three-dimensional point cloud data; calling a pre-trained three-dimensional defect detection model, performing detection based on the three-dimensional point cloud data and the two-dimensional image of the insulator, and obtaining a defect detection result of the insulator; The three-dimensional defect detection model includes a multi-scale feature extraction module, a hybrid attention module, a feature alignment module, a feature enhancement module, a defect candidate generation module, and a defect result generation module. The multi-scale feature extraction module extracts and transforms features of different scales on the three-dimensional point cloud data to obtain multi-scale geometric features; the hybrid attention module extracts attention features from the multi-scale geometric features in combination with the two-dimensional image to obtain attention weighted features. The feature alignment module performs feature alignment processing on the attention weighted features under different perspectives to obtain aligned features; The feature enhancement module performs encoding, feature transformation and feature mapping on the alignment features to obtain enhanced features; The defect candidate generation module performs feature extraction based on the enhanced features to obtain a candidate defect area; the defect result generation module performs defect location based on the candidate defect area to obtain a defect detection result.

2. The method for detecting defects in electric wire insulators based on binocular vision according to claim 1, characterized in that: The multi-scale feature extraction module includes a multi-scale neighborhood grouping unit, a pooling layer and a multi-layer perceptron. The multi-scale neighborhood grouping unit includes multiple neighborhood radius layers of different scales. Each of the neighborhood radius layers is connected to the pooling layer, and the pooling layer is connected to the multi-layer perceptron.

3. The method for detecting defects in electric wire insulators based on binocular vision according to claim 1, characterized in that: The hybrid attention module includes a channel attention unit and a spatial attention unit. The channel attention unit includes a connected feature channel layer and a squeeze excitation module, and the spatial attention unit includes a connected spatial convolution layer and a pooling layer.

4. The method for detecting defects in electric wire insulators based on binocular vision according to claim 1, characterized in that: The feature enhancement module includes a neural implicit function unit and a feature fusion unit connected to each other, and the neural implicit function unit includes a coding layer and multiple interconnected fully connected layers and a sine activation function.

5. The method for detecting defects in electric wire insulators based on binocular vision according to claim 1, characterized in that: The calculating and obtaining preliminary three-dimensional point cloud data of the insulator according to the multi-view images includes: Performing pixel intensity difference cost calculation on the multi-view images to obtain a disparity value; Calculating the cumulative path costs in multiple directions according to the disparity values ​​using a multi-directional path cost aggregation strategy; Determining a target disparity value according to the cumulative path costs in the multiple directions; Preliminary three-dimensional point cloud data of the insulator is calculated based on the target parallax value, camera focal length and binocular baseline distance.

6. The method for detecting defects in electric wire insulators based on binocular vision according to claim 1, characterized in that: The performing point cloud denoising and dynamic illumination compensation processing on the preliminary three-dimensional point cloud data to obtain optimized three-dimensional point cloud data includes: Perform statistical filtering on the preliminary 3D point cloud data and remove abnormal points obtained after processing to obtain denoised point cloud data; Performing adaptive histogram equalization processing on the multi-view image according to the denoised point cloud data to obtain a multi-view image with uniform illumination; The grayscale values ​​of the multi-view image with uniform illumination are mapped to the denoised point cloud data to obtain optimized three-dimensional point cloud data.

7. The method for detecting defects in electric wire insulators based on binocular vision according to claim 1, characterized in that: The multi-view images of the insulator are obtained according to the following steps: Calibrate the intrinsic and extrinsic parameters of the binocular vision camera, align the external trigger signal with the timestamp, set the synchronization mechanism of the binocular vision camera, and obtain a preset binocular vision camera; The preset binocular vision camera is used to perform surround image acquisition on the insulator to obtain multi-view images of the insulator.

8. The method for detecting defects in electric wire insulators based on binocular vision according to claim 7, characterized in that: The defect result generation module includes an image extraction unit, a detection head and a defect positioning unit that are connected to each other; Utilizing the defect result generation module to locate defects according to the candidate defect areas to obtain defect detection results includes: extracting a feature image from the candidate defect area using the image extraction unit; Detecting and classifying the feature image using the detection head to obtain a defect probability distribution of multiple defects; The defect localization unit is used to calculate the confidence level of each defect according to the defect probability distribution of the multiple defects and the defect areas marked in the three-dimensional point cloud data, and a defect detection result is obtained according to the confidence level.

9. A wire insulator defect detection system based on binocular vision, characterized in that: The method for detecting defects in electric wire insulators based on binocular vision according to any one of claims 1 to 8 comprises: A data acquisition module is used to acquire multi-view images of the insulator and calculate preliminary three-dimensional point cloud data of the insulator based on the multi-view images; a data processing module, configured to perform point cloud denoising and dynamic illumination compensation processing on the preliminary three-dimensional point cloud data to obtain optimized three-dimensional point cloud data; a defect detection module, configured to call a pre-trained three-dimensional defect detection model, perform detection based on the three-dimensional point cloud data and the two-dimensional image of the insulator, and obtain a defect detection result of the insulator; The three-dimensional defect detection model includes a multi-scale feature extraction module, a hybrid attention module, a feature alignment module, a feature enhancement module, a defect candidate generation module and a defect result generation module. The multi-scale feature extraction module performs feature extraction and transformation processing on the three-dimensional point cloud data at different scales to obtain multi-scale geometric features; the hybrid attention module performs attention feature extraction on the multi-scale geometric features in combination with the two-dimensional image to obtain attention weighted features; the feature alignment module performs feature alignment processing on the attention weighted features under different perspectives to obtain alignment features; the feature enhancement module encodes, transforms and maps the alignment features to obtain enhanced features; the defect candidate generation module performs feature extraction based on the enhanced features to obtain candidate defect areas; the defect result generation module performs defect location based on the candidate defect areas to obtain defect detection results.

10. The wire insulator defect detection system based on binocular vision according to claim 9, characterized in that: The data acquisition module includes a binocular vision camera.

Images