Multi-mode power grid equipment defect detection method and system

Abstract

The invention discloses a multi-modal power grid equipment defect detection method and system. The method comprises the following steps: performing defect detection on power grid equipment by applying a random forest algorithm after training optimization; wherein the training process of the random forest algorithm is as follows: acquiring laser point cloud data, visual camera images and ultrasonic data with power grid equipment defects; preprocessing the laser point cloud data and the visual camera image, and converting the ultrasonic data into an ultrasonic image; performing data fusion on the preprocessed laser point cloud data, the visual camera image and the ultrasonic image, and constructing a multi-modal three-dimensional data information model; a reference defect-free model is set, the multi-modal three-dimensional data information model and the reference defect-free model are compared in the aspects of surface defects and internal defects, and difference results are obtained; and performing power grid equipment defect detection training on a random forest algorithm by using a difference result. According to the invention, the precision and integrity of defect detection are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of power equipment defect detection, and in particular to a multi-modal power grid equipment defect detection method and system. Background Art

[0002] With the development of industry, electricity has become more and more important in people's lives, and power equipment has become more widely used. When people use power equipment, its safety and reliability are key factors to ensure the stable operation of the power system. Therefore, defect detection and defect prediction of power grid equipment are important technical means to ensure the safe operation of the power grid and reduce accidents. Traditionally, the method of detecting whether power equipment has defects mainly relies on power workers to go to the site to detect one by one, or use a single sensor, such as visual cameras, lidar scanners and other monitoring. These methods have defects in cost, real-time accuracy and other aspects.

[0003] Traditionally, the commonly used visual camera method for detecting equipment defects can capture two-dimensional images of the surface of power grid equipment, which is relatively rich in information. However, the inherent defects of visual cameras are that they are very sensitive to light and most cameras cannot have direct depth information. These defects make visual cameras easily affected by weather and day and night changes in outdoor scenes such as power plants, so the single visual sensor method is not very reliable.

[0004] LiDAR scanners are not sensitive to light and can directly provide 3D structural information in defect detection, but they also have many unique defects. First, the information obtained by laser scanners is a discrete point cloud, which has a certain complexity in the later 3D reconstruction; at the same time, laser scanners, like visual cameras, can only capture elements on the surface of objects and cannot directly obtain information inside objects.

[0005] Ultrasonic waves can effectively detect internal defects in materials during defect detection, but they have high requirements on the surface condition of objects, and a rough surface is likely to affect the final detection effect.

[0006] In the prior art, the invention patent with the patent publication number CN118582351A discloses a deep learning-based offshore wind turbine blade defect detection system and method. Although the invention patent integrates multiple detection methods such as computer vision, infrared imaging and ultrasonic flaw detection, the patent is a fusion under one-dimensional vector data. The patent aims to establish a digital twin module to simulate the motion trajectory and deformation of the blade under different working conditions, and plan the optimal detection path and viewing angle of the drone / ground vehicle based on the motion trajectory, deformation and stress of the blade, as well as the pixel-level segmentation results and classification results of the blade defects, so as to achieve the maximum coverage detection of the blade. Therefore, the computer vision, infrared imaging and ultrasonic flaw detection data need to be converted into sequence data. As stated in paragraph

[0028] of the specification, "the infrared encoder and the ultrasonic encoder both adopt a 1D convolution stacking structure to encode the temperature matrix sequence and the A-Scan sequence respectively", that is, the data dimensions are all one-dimensional data, which affects the detection accuracy. Summary of the invention

[0007] The technical problem to be solved by the present invention is to solve the problem of low accuracy of existing defect detection combined with ultrasonic data.

[0008] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0009] A multi-modal power grid equipment defect detection method includes: applying a trained and optimized random forest algorithm to perform defect detection on the power grid equipment; wherein the training process of the random forest algorithm is as follows:

[0010] Acquire laser point cloud data, visual camera images, and ultrasonic data with defects in power grid equipment;

[0011] Preprocess the laser point cloud data and visual camera images, and convert the ultrasonic data into ultrasonic images;

[0012] The pre-processed laser point cloud data, visual camera images, and ultrasonic images are fused to construct a multimodal three-dimensional data information model;

[0013] Setting a baseline defect-free model, comparing the multimodal three-dimensional data information model with the baseline defect-free model in terms of surface defects and internal defects, and obtaining difference results;

[0014] The difference results are applied to train the random forest algorithm for power grid equipment defect detection.

[0015] Advantages: The laser scanner provides a high-precision three-dimensional model, while the visual camera supplements its surface texture and color information. In addition, considering the addition of the ultrasonic sensor, it can effectively measure the internal structure of the equipment, effectively making up for the defect that the first two sensors cannot provide internal detection. And convert the one-dimensional ultrasonic data into a two-dimensional ultrasonic image, and then construct a three-dimensional data information model with the two-dimensional ultrasonic image, laser point cloud, and visual camera image. Based on this, more comprehensive and accurate detection and prediction are achieved, the anti-interference ability is enhanced, and the maintenance efficiency is improved.

[0016] In one embodiment of the present invention, the preprocessing method of laser point cloud data includes: applying the PCA principal component analysis method to calculate the covariance matrix of each laser point cloud, and calculating the three eigenvalues ​​of the covariance matrix; according to the three eigenvalues ​​of the covariance matrix, obtaining the geometric structure feature points of the laser point cloud and performing downsampling; applying the ICP algorithm to the laser point cloud after downsampling and screening to align and generate a preliminary three-dimensional point cloud model.

[0017] In one embodiment of the present invention, the preprocessing method of the visual camera image includes: using a Gaussian filter to denoise the visual camera image and then smoothing it through a convolution operation; using a Sobel operator to calculate the gradient amplitude and direction of the denoised and smoothed visual camera image; applying a non-maximum suppression technique to compare the gradient amplitude of each pixel in the camera image with its neighboring pixels along the gradient direction, and if the current pixel is not a local maximum, it is suppressed to 0 to preliminarily filter non-edge pixels; double threshold processing to divide edge pixels into strong edge pixels, weak edge pixels and non-edge pixels; removing non-edge pixels again, and starting from the strong edge pixels, tracing the connected weak edge pixels to ensure the boundary continuity of the visual camera image.

[0018] In one embodiment of the present invention, converting ultrasonic data into ultrasonic images comprises the following steps:

[0019] Move the ultrasonic probe multiple times along the same direction of the object to be measured. Assuming that the horizontal movement distance of the ultrasonic probe is x and the distance in the depth direction is z, arrange the multiple A-scan data obtained in the two directions along an axis to form an A-scan data matrix; and calculate the grayscale value of each A-scan data in the A-scan data matrix to form an ultrasonic image.

[0020] In one embodiment of the present invention, the grayscale value of each A-scan data in the A-scan data matrix is ​​obtained by the following formula:

[0021]

[0022] In the formula, I S(x,z) represents the grayscale value of the A-scan data, A(t) represents the amplitude of the echo signal, v represents the reflection rate of the ultrasonic wave in the medium, and t represents the reflection reception time of the echo signal.

[0023] In one embodiment of the present invention, constructing a multimodal three-dimensional data information model includes:

[0024] Time synchronization of laser point cloud data, visual camera images and ultrasonic images through timestamps;

[0025] The time-synchronized laser point cloud data, visual camera images and ultrasonic images are converted into coordinate systems and uniformly converted into the world coordinate system;

[0026] The laser point cloud data and visual camera images in the same coordinate system are fused to obtain a three-dimensional point cloud model with image features. The three-dimensional point cloud model with image features is then fused with the ultrasonic image to fill in the internal information of the power grid equipment and construct a multimodal three-dimensional data information model.

[0027] In one embodiment of the present invention, the three-dimensional point cloud model with image features is obtained by the following formula:

[0028]

[0029] In the formula, Represented as a 3D point cloud model with image features, Represented as laser point cloud data in the world coordinate system, It is represented as the visual camera image in the world coordinate system, and ∪ is represented as the union;

[0030] The multimodal three-dimensional data information model is obtained by the following formula:

[0031] in,

[0032] In the formula, A W Represented as a multimodal three-dimensional data information model, Represented as an ultrasonic image in the world coordinate system, represents the ultrasonic image converted from the i-th ultrasonic data, N represents the number of ultrasonic images, ∪ represents the union, T WU Represented as the coordinate transformation matrix of the ultrasound image.

[0033] In one embodiment of the present invention, the multimodal three-dimensional data information model is compared with the reference defect-free model in terms of surface defects and internal defects, and a difference result is obtained, including:

[0034] Calculate the volume of the laser point cloud data in the multimodal three-dimensional data information model, which is recorded as the volume of the point cloud model to be detected; and calculate the volume of the laser point cloud data in the reference defect-free model, which is recorded as the reference point cloud model volume; calculate the difference between the volume of the point cloud model to be detected and the volume of the reference point cloud model to obtain the point cloud difference result;

[0035] Calculate the edge pixels of the visual camera image in the multimodal three-dimensional data information model, recorded as the edge image to be detected; and calculate the edge pixels of the visual camera image in the reference defect-free model, recorded as the reference edge image; calculate the difference between the edge image to be detected and the reference edge image to obtain a pixel difference result;

[0036] The echo signal amplitude of the ultrasonic image in the multimodal three-dimensional data information model is calculated and recorded as the current amplitude. The echo signal amplitude of the ultrasonic image in the reference defect-free model is calculated and recorded as the reference amplitude. The difference between the current amplitude and the reference amplitude is calculated to obtain the amplitude difference result.

[0037] In one embodiment of the present invention, the difference results of surface defects and internal defects are used as the difference feature set Features = {V diff ,D edge ,A diff}; where V diff Expressed as point cloud difference result, D edge Expressed as pixel difference result, A diff It is expressed as the amplitude difference result;

[0038] The difference feature set is used as the input of the random forest algorithm to output the defect type of the power grid equipment. The defect type of the power grid equipment is obtained by the following formula:

[0039]

[0040] In the formula, C final Represented as the defect type of power grid equipment, T j (Features) represents the classification result of the jth decision tree, majority vote Represented as a majority voting mechanism, Expressed as the total number of decision trees.

[0041] The present invention also provides a multi-modal power grid equipment defect detection system, which applies the multi-modal power grid equipment defect detection method described above, and comprises:

[0042] Initial data module, used to obtain laser point cloud data, visual camera images and ultrasonic data with grid equipment defects;

[0043] A preprocessing module is used to preprocess the laser point cloud data and visual camera images and convert the ultrasonic data into ultrasonic images;

[0044] The fusion module is used to fuse the pre-processed laser point cloud data, visual camera images, and ultrasonic images to construct a multi-modal three-dimensional data information model;

[0045] The difference feature module is used to set a baseline defect-free model, compare the multimodal three-dimensional data information model with the baseline defect-free model in terms of surface defects and internal defects, and obtain difference results;

[0046] A training module for applying the difference results to train the random forest algorithm for power grid equipment defect detection;

[0047] The prediction module is used to apply the trained and optimized random forest algorithm to perform defect detection on power grid equipment.

[0048] Compared with the prior art, the present invention has the following beneficial effects: the present invention fuses laser point cloud, visual image and ultrasonic data to construct a comprehensive three-dimensional spatial model containing surface and internal information. This data fusion strategy significantly improves the accuracy and completeness of defect detection.

[0049] The present invention utilizes ultrasonic sensors to supplement the internal structure information that cannot be obtained by laser point clouds and visual images, thereby achieving effective detection of internal defects of the equipment.

[0050] The multimodal data fusion strategy of the present invention breaks through the limitations of traditional single-modal sensors and achieves higher-precision defect detection and prediction by comprehensively acquiring comprehensive information on the surface and interior of the equipment. Existing technologies usually rely on single sensors, such as visual cameras or lidars, which are limited in their ability to detect specific defect types or are sensitive to environmental factors, such as light or surface conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 The figure is a flow chart of the training process of the random forest algorithm according to an embodiment of the present invention.

[0052] Figure 2 A flowchart of constructing a multimodal three-dimensional data information model according to an embodiment of the present invention.

[0053] Figure 3 4 is a block diagram of a prediction method according to an embodiment of the present invention.

[0054] Figure 4 This is a block diagram of a multi-modal power grid equipment defect detection system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to facilitate those skilled in the art to understand the technical solution of the present invention, the technical solution of the present invention is further described in conjunction with the accompanying drawings of the specification.

[0056] The terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0057] See also Figure 1 As shown, the present invention provides a multimodal power grid equipment defect detection method, including applying a trained and optimized random forest algorithm to perform defect detection on the power grid equipment; wherein the training process of the random forest algorithm is as follows:

[0058] S10, obtaining laser point cloud data, visual camera images and ultrasonic data with defects of power grid equipment.

[0059] In this embodiment, laser point cloud data, visual camera images and ultrasonic data are acquired using radar, camera and ultrasonic probe respectively.

[0060] S20, preprocessing the laser point cloud data and the visual camera image, and converting the ultrasonic data into an ultrasonic image.

[0061] In this embodiment, a laser point cloud data scan may contain tens of thousands to millions of point cloud points. Such a huge number is not all necessary. Therefore, after obtaining the original data, we will first remove the points that are too far or too close, and extract the obvious geometric structure points such as line points and surface points, perform a downsampling, and then perform point cloud registration to generate a preliminary 3D model. Specifically, first, the PCA principal component analysis method is used to calculate the covariance matrix of each laser point cloud:

[0062]

[0063] Among them, L is the i-th laser point cloud P i The number of neighboring laser point clouds, is represented as the average 3D coordinate of the laser point cloud, and C is represented as the covariance matrix.

[0064] Next, calculate the three eigenvalues ​​of the covariance matrix: 1 , 2 , 3 , and sort the three eigenvalues ​​from large to small: λ 1 >λ 2 >λ 3, and according to the sorted three eigenvalues, calculate the geometric structure feature points of the laser point cloud, including the flatness of the plane point and the linearity of the linear point. Obtained by the following formula: And, the linearity of the linear point Obtained by the following formula: If you get the flatness and linearity are respectively greater than the set flatness threshold λ pe , linearity threshold λ le If , then take the plane point and the linear point.

[0065] Finally, the downsampled and filtered laser point cloud is registered using the ICP algorithm to generate a preliminary 3D point cloud model. The ICP algorithm is expressed by the following formula:

[0066]

[0067] In the formula, q i Represents the target laser point, R represents the rotation matrix, t represents the translation, and l represents the number of matching point cloud pairs. After minimizing the above objectives and obtaining R and t, the current laser point cloud is added to the 3D point cloud model for supplementation and improvement.

[0068] In this embodiment, the visual camera will capture a real-time two-dimensional image of the surface of the power grid equipment. After the commonly used image enhancement technology, image denoising, feature extraction and other steps, it will be combined with the laser point cloud data to prepare for subsequent defect detection. The visual camera image feature extraction mainly uses the Canny edge detection algorithm to extract features, and its main steps are as follows:

[0069] First, a Gaussian filter is used to denoise the visual camera image. The formula is as follows:

[0070]

[0071] Where σ represents the standard deviation of the Gaussian kernel, and x, y represent the coordinates of the visual camera image.

[0072] Furthermore, the convolution operation smoothes it out using the following formula:

[0073] I smooth (x,y)=I(x,y)G(x,y);

[0074] In the formula, I smooth (x, y) represents the visual camera image after smoothing, and I(x, y) represents the grayscale value of the visual camera image in the horizontal direction x and the vertical direction y.

[0075] Then, the Sobel operator is used to calculate the gradient magnitude and direction of the denoised and smoothed visual camera image. Specifically, the horizontal and vertical gradient calculation formulas of the Sobel operator are as follows:

[0076] Among them, the horizontal gradient G x The formula is:

[0077]

[0078] Vertical gradient G y The formula is:

[0079]

[0080] Calculate the gradient magnitude G Sobel (x,y) and direction θ(x,y):

[0081]

[0082] Furthermore, non-maximum suppression technology is applied to filter out non-edge pixels and make the blurred boundaries clear. This process retains the maximum value of the gradient magnitude at each pixel and filters out other values. The process mainly compares the gradient magnitude of each pixel in the camera image with its neighboring pixels along the gradient direction. If the current pixel is not a local maximum, it is suppressed to 0.

[0083] Then, double threshold processing classifies edge pixels into strong edge, weak edge and non-edge. Set high threshold pixels and low threshold pixels, and classify edge pixels above the high threshold pixels as strong edges, those between the high threshold pixels and the low threshold pixels as weak edges, and those below the low threshold pixels as non-edges.

[0084] Finally, non-edge pixels are removed again, and starting from strong edge pixels, connected weak edge pixels are traced to ensure the boundary continuity of the visual camera image.

[0085] In one embodiment of the present invention, the basic principle of ultrasonic detection is to analyze the reflection and transmission characteristics of the ultrasonic wave inside the device after it contacts the device, and obtain the internal structure information of the device. After the ultrasonic wave signal is denoised, the commonly used waveform analysis methods include one-dimensional A-scan and two-dimensional B-scan. When an abnormal waveform is identified, it means that there may be potential problems such as defects.

[0086] In this embodiment, before the dimensionality conversion is performed, the ultrasonic signal is first denoised by a bandpass filter, as follows:

[0087] S f (t) = F -1 (F(S(t))·H(f));

[0088] Where S(t) represents the input signal, H(f) represents the transfer function of the bandpass filter, and F -1 , F represents positive and negative Fourier transform, S f (t) represents the filtered ultrasonic signal.

[0089] A-scan is a one-dimensional signal analysis method used to represent the reflected waveform along a straight line. The working principle is: the ultrasonic sensor probe emits ultrasonic pulses, which propagate in the object being measured. When the ultrasonic wave encounters discontinuity in the object medium, such as internal defects, it will be reflected and the reflected wave data will be recorded. 0 , the reflection receiving time t, and the reflection rate v of the ultrasonic wave in the medium, the straight-line distance d from the defect position to the probe can be calculated, the formula is as follows:

[0090]

[0091] In the returned data, the amplitude (signal strength) of the reflected signal is also recorded:

[0092] A(t)=A 0 e -αt sin(2πft+φ);

[0093] Where A(t) represents the amplitude of the echo signal, A 0 It represents the initial amplitude, α represents the attenuation exponent, f represents the frequency, φ represents the phase angle, e represents the natural constant, and sin represents the sine function.

[0094] B-scan is a two-dimensional ultrasonic image, which is obtained by moving the ultrasonic probe along one direction of the object being measured and arranging the multiple A-scan data along one axis to form a two-dimensional interface image of the object being measured in that direction. For its specific imaging process, assuming that the distance moved by the ultrasonic probe in the lateral direction is x and the distance in the depth direction is z, the multiple A-scan data obtained in the two directions are arranged along one axis to form an A-scan data matrix; and the grayscale value of each A-scan data in the A-scan data matrix is ​​calculated to form an ultrasonic image. Then the grayscale value I of each point in the ultrasonic image S (x,z) can be expressed as the relationship between the echo signal intensity A(t) of the ultrasonic probe at position x and time t:

[0095]

[0096] In the formula, I S (x,z) represents the grayscale value of the A-scan data, v represents the reflection rate of the ultrasonic wave in the medium, and t represents the reflection reception time of the echo signal.

[0097] S30, fusing the pre-processed laser point cloud data, visual camera images, and ultrasonic images to construct a multimodal three-dimensional data information model.

[0098] See also Figure 2 As shown, in one embodiment of the present invention, after acquiring the three sensor data in the previous part, the next thing to do is to fuse the three-modal data. The first thing to do is time synchronization, ensuring that the data of the three sensors are collected at the same time point, usually matched by timestamps. The second step is to complete the alignment of the three sensor data in space, that is, to unify the coordinate system.

[0099] Assume that the coordinate systems of the laser radar, visual camera, and ultrasonic radar sensor are {C L}、{C V} and {C U}, the three are relative to the world coordinate system {C W The transformation matrices of} are T WL , T WV and T WU Where T WX Indicates transforming sensor X (which can represent three sensors separately) into the world coordinate system.

[0100] Among them, the laser point cloud data and visual camera image in the world coordinate system are expressed by the following formula:

[0101]

[0102] In the formula, Represented as laser point cloud data in the world coordinate system, Represented as a visual camera image in the world coordinate system, P L Represented as laser point cloud data in its own coordinate system, P V Represented as a visual camera image in its own coordinate system.

[0103] Suppose the ultrasonic image converted from the i-th ultrasonic data is Combining the odometer positioning output of the aforementioned laser SLAM and visual SLAM technologies, the volume reconstruction method is used to generate a three-dimensional internal model, and the slice image is projected into three-dimensional space, that is, the ultrasonic image in the world coordinate system, which is obtained by the following formula:

[0104]

[0105] Represented as an ultrasonic image in the world coordinate system, represents the ultrasonic image converted from the i-th ultrasonic data, N represents the number of ultrasonic images, ∪ represents the union, T WU Represented as the coordinate transformation matrix of the ultrasound image.

[0106] In this embodiment, after the same coordinates are obtained, the laser point cloud data and the visual camera image are fused to obtain a three-dimensional point cloud model with image features:

[0107]

[0108] In the formula, It is represented as a 3D point cloud model with image features, and ∪ is represented as a union.

[0109] Then, the 3D point cloud model with image features is fused with the ultrasonic image to fill in the internal information of the power grid equipment and construct a multimodal 3D data information model:

[0110]

[0111] In the formula, A W Represented as a multimodal three-dimensional data information model.

[0112] S40, setting a reference defect-free model, comparing the multimodal three-dimensional data information model with the reference defect-free model in terms of surface defects and internal defects, and obtaining difference results.

[0113] See also Figure 3 As shown, in one embodiment of the present invention, the defect detection part is mainly compared with a given reference defect-free model. The reference defect-free model is generally calibrated manually before leaving the factory. For this part, according to the surface and internal structure of the equipment, the defect detection is divided into two parts: surface defect detection and internal defect detection.

[0114] In the surface defect detection part, the normal and curvature of each point cloud point, as well as the texture and color features in its image features, can be calculated, and then compared with the corresponding point cloud in the baseline defect-free model to detect surface geometric anomalies such as corrosion, wear, cracks, and coating shedding. As for the comparison of image features, the image feature values ​​are extracted using the Canny edge detection algorithm, and then compared with the image in the baseline defect-free model to achieve defect detection. In this internal defect detection part, the ultrasonic data of the current model is compared with the ultrasonic data in the baseline defect-free model in terms of signal features. Signal processing techniques, such as amplitude or peak value, are applied to identify abnormal signal waveforms, and then compared with the baseline signal waveforms to determine the health status of the internal structure, and detect and locate structural defects such as cracks and voids inside the power grid equipment.

[0115] The method for constructing a benchmark defect-free model is the same as the method for constructing a multimodal 3D data information model, except that the data used are defect-free laser point cloud data, visual camera images, and ultrasonic data. The details are as follows:

[0116] Calculate the volume of the laser point cloud data in the multimodal 3D data information model, recorded as the volume of the point cloud model to be detected; and calculate the volume of the laser point cloud data in the reference defect-free model, recorded as the reference point cloud model volume; calculate the difference between the volume of the point cloud model to be detected and the volume of the reference point cloud model, and obtain the point cloud difference result:

[0117] V diff =|V test -V ref |;

[0118] Among them, V diff Expressed as point cloud difference result, V test Represented as the volume of the point cloud model to be detected, V ref Represented as the volume of the benchmark point cloud model.

[0119] Calculate the edge pixels of the visual camera image in the multimodal three-dimensional data information model, recorded as the edge image to be detected; and calculate the edge pixels of the visual camera image in the reference defect-free model, recorded as the reference edge image; calculate the difference between the edge image to be detected and the reference edge image, and obtain the pixel difference result:

[0120] D edge =∑|E test -E ref |;

[0121] Where D edge Expressed as pixel difference result, E test Represented as the edge image to be detected, E ref Denoted as the reference edge image.

[0122] The echo signal amplitude of the ultrasonic image in the multimodal three-dimensional data information model is calculated and recorded as the current amplitude. The echo signal amplitude of the ultrasonic image in the reference defect-free model is calculated and recorded as the reference amplitude. The difference between the current amplitude and the reference amplitude is calculated to obtain the amplitude difference result.

[0123] A diff =|A test -A ref |;

[0124] In the formula, A diff Expressed as the amplitude difference result, A test Represents the current amplitude, A ref Indicated as the reference amplitude.

[0125] S50, applying the difference results to train the random forest algorithm for power grid equipment defect detection.

[0126] In one embodiment of the present invention, we use the difference results obtained from the first two parts: surface defect detection and internal defect detection as input data, and use the random forest algorithm to comprehensively analyze and classify this information. Specifically, the difference features of surface and internal defect detection. That is, the difference results on surface defects and internal defects are used as the difference feature set Features = {V diff ,D edge ,A diff}.

[0127] The random forest algorithm performs classification by constructing multiple decision trees. Each decision tree randomly selects a different subset of features from the input feature vector for training and independently classifies the feature vector. Finally, the classification results of each decision tree are integrated through a majority voting mechanism to determine the final defect category of the input data. This process can combine surface and internal detection features to achieve an accurate assessment of the health status or defect type of the equipment.

[0128] That is, the difference feature set is used as the input of the random forest algorithm to output the defect type of the power grid equipment; the defect type of the power grid equipment is obtained by the following formula:

[0129]

[0130] In the formula, C final Represented as the defect type of power grid equipment, T j (Features) represents the classification result of the jth decision tree, majority vote Represented as a majority voting mechanism, Expressed as the total number of decision trees.

[0131] During training, for example, the loss function is set to minimize the inspection error and other optimizations are performed. Then, based on the inspection results, a visual interface displays the appearance of internal and external defects, providing reference for power engineers and prompts for subsequent operations.

[0132] See also Figures 1 to 4 As shown, the present invention also provides a multi-modal power grid equipment defect detection system, which applies the multi-modal power grid equipment defect detection method described above, including:

[0133] The initial data module 10 is used to obtain laser point cloud data, visual camera images and ultrasonic data with defects in power grid equipment.

[0134] The preprocessing module 20 is used to preprocess the laser point cloud data and the visual camera image, and convert the ultrasonic data into an ultrasonic image.

[0135] The fusion module 30 is used to fuse the pre-processed laser point cloud data, visual camera images, and ultrasonic images to construct a multi-modal three-dimensional data information model.

[0136] The difference feature module 40 is used to set a reference defect-free model, compare the multimodal three-dimensional data information model with the reference defect-free model in terms of surface defects and internal defects, and obtain difference results.

[0137] The training module 50 is used to apply the difference results to train the random forest algorithm for power grid equipment defect detection.

[0138] The prediction module 60 is used to apply the trained and optimized random forest algorithm to perform defect detection on the power grid equipment.

[0139] It is obvious to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, the embodiments should be regarded as exemplary and non-limiting from any point of view, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention, and any reference numerals in the claims should not be regarded as limiting the claims involved.

[0140] The above-described embodiments merely represent implementation methods of the invention. The protection scope of the present invention is not limited to the above-described embodiments. For those skilled in the art, several modifications and improvements may be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A multi-modal power grid equipment defect detection method, characterized in that: include: The random forest algorithm after training optimization is used to detect defects in power grid equipment. The training process of the random forest algorithm is as follows: Acquire laser point cloud data, visual camera images, and ultrasonic data with defects in power grid equipment; Preprocess the laser point cloud data and visual camera images, and convert the ultrasonic data into ultrasonic images; The pre-processed laser point cloud data, visual camera images, and ultrasonic images are fused to construct a multimodal three-dimensional data information model; Setting a baseline defect-free model, comparing the multimodal three-dimensional data information model with the baseline defect-free model in terms of surface defects and internal defects, and obtaining difference results; The difference results are applied to train the random forest algorithm for power grid equipment defect detection.

2. The multi-modal power grid equipment defect detection method according to claim 1, characterized in that: The preprocessing method of laser point cloud data includes: applying the PCA principal component analysis method to calculate the covariance matrix of each laser point cloud, and calculating the three eigenvalues ​​of the covariance matrix; according to the three eigenvalues ​​of the covariance matrix, obtaining the geometric structure feature points of the laser point cloud and performing downsampling; applying the ICP algorithm to align the laser point cloud after downsampling and generating a preliminary three-dimensional point cloud model.

3. The multi-modal power grid equipment defect detection method according to claim 1, characterized in that: The preprocessing method of the visual camera image includes: using a Gaussian filter to denoise the visual camera image and then smoothing it through a convolution operation; using the Sobel operator to calculate the gradient amplitude and direction of the denoised and smoothed visual camera image; applying the non-maximum suppression technology to compare the gradient amplitude of each pixel in the camera image with its neighboring pixels along the gradient direction. If the current pixel is not a local maximum, it is suppressed to 0 to preliminarily filter non-edge pixels; double threshold processing divides edge pixels into strong edge pixels, weak edge pixels and non-edge pixels; again remove non-edge pixels, and starting from the strong edge pixels, track the connected weak edge pixels to ensure the boundary continuity of the visual camera image.

4. The multi-modal power grid equipment defect detection method according to claim 1, characterized in that: The conversion of ultrasound data into ultrasound images includes the following steps: Move the ultrasonic probe multiple times along the same direction of the object to be measured. Assuming that the horizontal movement distance of the ultrasonic probe is x and the distance in the depth direction is z, arrange the multiple A-scan data obtained in the two directions along an axis to form an A-scan data matrix; and calculate the grayscale value of each A-scan data in the A-scan data matrix to form an ultrasonic image.

5. The multi-modal power grid equipment defect detection method according to claim 4, characterized in that: The grayscale value of each A-scan data in the A-scan data matrix is ​​obtained by the following formula: In the formula, I S (x,z) represents the grayscale value of the A-scan data, A(t) represents the amplitude of the echo signal, v represents the reflection rate of the ultrasonic wave in the medium, and t represents the reflection reception time of the echo signal.

6. The multi-modal power grid equipment defect detection method according to claim 1, characterized in that: Construct a multimodal 3D data information model, including: Time synchronization of laser point cloud data, visual camera images and ultrasonic images through timestamps; The time-synchronized laser point cloud data, visual camera images and ultrasonic images are converted into coordinate systems and uniformly converted into the world coordinate system; The laser point cloud data and visual camera images in the same coordinate system are fused to obtain a three-dimensional point cloud model with image features. The three-dimensional point cloud model with image features is then fused with the ultrasonic image to fill in the internal information of the power grid equipment and construct a multimodal three-dimensional data information model.

7. The multi-modal power grid equipment defect detection method according to claim 5, characterized in that: The three-dimensional point cloud model with image features is obtained by the following formula: In the formula, Represented as a 3D point cloud model with image features, Represented as laser point cloud data in the world coordinate system, It is represented as the visual camera image in the world coordinate system, and ∪ is represented as the union; The multimodal three-dimensional data information model is obtained by the following formula: in, In the formula, A W Represented as a multimodal three-dimensional data information model, Represented as an ultrasonic image in the world coordinate system, represents the ultrasonic image converted from the i-th ultrasonic data, N represents the number of ultrasonic images, T WU Represented as the coordinate transformation matrix of the ultrasound image.

8. The multi-modal power grid equipment defect detection method according to claim 1, characterized in that: Compare the multimodal 3D data information model with the benchmark defect-free model in terms of surface defects and internal defects, and obtain difference results, including: Calculate the volume of the laser point cloud data in the multimodal three-dimensional data information model, which is recorded as the volume of the point cloud model to be detected; and calculate the volume of the laser point cloud data in the reference defect-free model, which is recorded as the reference point cloud model volume; calculate the difference between the volume of the point cloud model to be detected and the volume of the reference point cloud model to obtain the point cloud difference result; Calculate the edge pixels of the visual camera image in the multimodal three-dimensional data information model, recorded as the edge image to be detected; and calculate the edge pixels of the visual camera image in the reference defect-free model, recorded as the reference edge image; calculate the difference between the edge image to be detected and the reference edge image to obtain a pixel difference result; The echo signal amplitude of the ultrasonic image in the multimodal three-dimensional data information model is calculated and recorded as the current amplitude. The echo signal amplitude of the ultrasonic image in the reference defect-free model is calculated and recorded as the reference amplitude. The difference between the current amplitude and the reference amplitude is calculated to obtain the amplitude difference result.

9. The multi-modal power grid equipment defect detection method according to claim 1, characterized in that: The difference results of surface defects and internal defects are taken as the difference feature set Features = {V diff ,D edge ,A diff }; Among them, V diff Expressed as point cloud difference result, D edge Expressed as pixel difference result, A diff It is expressed as the amplitude difference result; The difference feature set is used as the input of the random forest algorithm to output the defect type of the power grid equipment. The defect type of the power grid equipment is obtained by the following formula: In the formula, C final Represented as the defect type of power grid equipment, T j (Features) represents the classification result of the jth decision tree, majority vote Represented as a majority voting mechanism, Expressed as the total number of decision trees.

10. A multi-modal power grid equipment defect detection system, characterized in that: The multi-modal power grid equipment defect detection method according to any one of claims 1 to 9 comprises: Initial data module, used to obtain laser point cloud data, visual camera images and ultrasonic data with grid equipment defects; A preprocessing module is used to preprocess the laser point cloud data and visual camera images and convert the ultrasonic data into ultrasonic images; The fusion module is used to fuse the pre-processed laser point cloud data, visual camera images, and ultrasonic images to construct a multi-modal three-dimensional data information model; The difference feature module is used to set a baseline defect-free model, compare the multimodal three-dimensional data information model with the baseline defect-free model in terms of surface defects and internal defects, and obtain difference results; A training module for applying the difference results to train the random forest algorithm for power grid equipment defect detection; The prediction module is used to apply the trained and optimized random forest algorithm to perform defect detection on power grid equipment.

Images