Online monitoring method and system for hidden dangers in power transmission lines based on 3D point clouds

Abstract

This application relates to the field of image data processing technology, specifically to a method and system for online monitoring of potential hazards in power transmission lines based on 3D point clouds. The method includes: analyzing the projection of the gradient directions of all pixels within a local window of each pixel in the grayscale image of the power transmission line to determine the feature vector and its corresponding feature value; determining the gradient direction feature value based on the differences between feature values ​​and the angles between the feature vector, gradient vector, and the horizontal axis; acquiring continuous pixels; determining local transformation feature values ​​based on the distribution of gradient magnitudes between continuous pixels and their neighboring pixels, and acquiring local feature values; assigning spatial range weights of a filtering algorithm to each pixel in the depth image to obtain a 3D point cloud of the power transmission line; and using a neural network to monitor the power transmission line. This application aims to obtain a more accurate 3D point cloud of the power transmission line, improving the accuracy and efficiency of power transmission line monitoring.

Description

Technical Field

[0001] This application relates to the field of image data processing technology, specifically to a method and system for online monitoring of potential hazards in power transmission lines based on three-dimensional point clouds. Background Technology

[0002] Overhead transmission lines are typical power transmission paths and carriers in power systems, and ensuring their safe operation is a crucial aspect of the current power sector. Under normal circumstances, outdoor overhead transmission lines are exposed to complex environments for extended periods, suffering from wind and rain erosion, corona discharge, and chemical corrosion. This can easily lead to surface damage and cracks in the conductors and insulators, potentially causing partial discharge, strand breakage, or even line failure, threatening the safe operation of the power grid. Traditional inspection methods are inefficient, costly, and struggle to detect defects promptly. In recent years, with the development and widespread use of visual technology, online monitoring of transmission line hazards can be significantly improved by using drones to acquire 3D point cloud data, combined with intelligent detection technology based on the fusion of vision and depth perception. This approach enhances the efficiency and accuracy of hazard detection in transmission lines.

[0003] After acquiring images and point cloud data using drones, the images are converted into depth maps and fused with the point cloud data to form a target 3D point cloud. However, depth images often contain significant image sensor noise and are affected by ambient lighting. Therefore, filtering and denoising of the acquired depth maps are typically required. Considering the edge regions in power transmission line scenarios, such as the edges of insulator strings, the connections between conductors and connecting hammers, and the corners of tower angle steel, as well as the edge regions where line damage occurs during power transmission line hazards, these regions exhibit sharp, high-contrast geometric edge features. During filtering and denoising, these edge regions may be smoothed out because they resemble noise features, affecting the accuracy of 3D point cloud construction and subsequent online monitoring of power transmission line hazards. Conventional filtering algorithms cannot adaptively adjust edge sensitivity during denoising, reducing the accuracy of online monitoring of power transmission line hazards. Summary of the Invention

[0004] In view of the above, it is necessary to provide an online monitoring method and system for potential hazards in power transmission lines based on 3D point clouds to solve the above problems.

[0005] The first aspect of this application provides a method for online monitoring of hidden dangers in power transmission lines based on three-dimensional point clouds, the method comprising:

[0006] Acquire point cloud data of transmission lines and depth images corresponding to grayscale images;

[0007] Taking each pixel in the grayscale image as the center, a local window is preset. The projection of the gradient direction of all pixels within the local window of each pixel is analyzed. Combined with the principal component analysis method, all feature vectors and their corresponding feature values ​​of the local window corresponding to each pixel are determined. Based on the differences between the feature values, combined with the angles between the feature vectors, gradient vectors and the horizontal axis, the gradient direction feature value of the local window corresponding to each pixel is determined.

[0008] Based on the gradient magnitude, all pixels within the local window of each pixel are judged to obtain continuous pixels; according to the distribution of gradient magnitude of continuous pixels within the local window of each pixel in the grayscale image and its neighboring pixels, the local transformation feature value of each pixel is determined, and combined with the gradient direction feature value, the local feature value of the local window corresponding to each pixel is obtained.

[0009] Based on the local feature values, a spatial range weight of the filtering algorithm is assigned to each pixel in the depth map corresponding to the grayscale image. The filtered depth map is then projected to obtain point cloud data, which is matched with the point cloud data of the transmission line to obtain the three-dimensional point cloud of the transmission line. A neural network is then used to monitor the transmission line.

[0010] Preferably, the process of determining all feature vectors and their corresponding feature values ​​for each pixel within a local window is as follows:

[0011] The gradient direction of each pixel in the grayscale image is mapped to a two-dimensional vector. The two-dimensional vectors obtained from all pixels in the local window corresponding to each pixel are used as input to principal component analysis. The output is all feature vectors and their corresponding feature values ​​for the local window corresponding to each pixel. The feature vectors include a first feature vector and a second feature vector, and the feature values ​​include a first feature value and a second feature value.

[0012] Preferably, the step of determining the gradient direction feature value of the local window corresponding to each pixel is as follows:

[0013] For each pixel corresponding to a local window, based on the first feature value and the second feature value, the feature deviation within the local window corresponding to each pixel is obtained;

[0014] The first angle is the angle between the main direction of the first feature vector of the local window corresponding to each pixel and the horizontal axis; the second angle is the angle between the gradient vector of each pixel and the horizontal axis.

[0015] Based on the difference between the first included angle and the second included angle, and combined with the feature deviation, the gradient direction feature value of each pixel is obtained.

[0016] Preferably, the step of obtaining the feature deviation within the local window corresponding to each pixel point specifically involves:

[0017] Calculate the difference and sum between the first feature value and the second feature value respectively; then, positively fuse the negative correlation mapping result of the sum value with the difference value to obtain the feature deviation.

[0018] Preferably, the gradient direction feature value is specifically:

[0019] Obtain the sine value of the difference between the first and second included angles of each pixel, and positively fuse the negative correlation mapping result after normalizing the sine value with the feature deviation to obtain the gradient direction feature value of each pixel.

[0020] Preferably, the acquisition of consecutive pixels specifically includes:

[0021] Obtain the gradient magnitude threshold of all pixels within the local window corresponding to each pixel. Take all pixels with gradient magnitude greater than the gradient magnitude threshold as calibrated pixels. If there are calibrated pixels in the eight neighborhoods of each calibrated pixel, record the corresponding calibrated pixels as consecutive pixels.

[0022] Preferably, determining the local transformation feature value of each pixel specifically involves:

[0023] Calculate the absolute value of the difference between the gradient magnitude of each consecutive pixel and each pixel in its eight neighborhoods, and take the mean of the absolute values ​​of all consecutive pixels as the magnitude mean.

[0024] If pixel a is a continuous pixel, obtain the sum of the absolute values ​​of the differences between the mean values ​​of the magnitudes of all pairs of continuous pixels in the local window corresponding to pixel a, and normalize them to use as the local transformation feature value of pixel a.

[0025] If pixel a is not a continuous pixel, obtain the average of the maximum and minimum values ​​of the local transformation feature values ​​of all continuous pixels in the grayscale image, and use it as the local transformation feature value of pixel a.

[0026] Preferably, the process of obtaining the local feature values ​​of the local window corresponding to each pixel is as follows:

[0027] Obtain the normalized result of the gradient direction feature value of each pixel, and use the ratio of the local change feature value of each pixel to the normalized result as the local feature value of each pixel.

[0028] Preferably, the step of assigning spatial range weights for the filtering algorithm to each pixel in the depth map corresponding to the grayscale image is specifically as follows:

[0029] Calculate the sum between the natural number 1 and the local feature value of each pixel, and then positively fuse the obtained sum with the preset initial spatial range weight to obtain the spatial range weight of the corresponding pixel in the depth map.

[0030] Secondly, embodiments of this application also provide an online monitoring system for potential hazards in transmission lines based on three-dimensional point clouds, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of any of the methods described above.

[0031] The beneficial effects of the above scheme are as follows: First, by acquiring point cloud data and depth images of the transmission line, this application can provide basic three-dimensional spatial information for subsequent processing; by analyzing gradient direction and principal component analysis, it can extract local edge and texture features of the image, which can be used to identify the contours of objects and structural information within the region, enhancing the expressive power of the image and improving the accuracy of region recognition in subsequent steps; the analysis of gradient direction feature values ​​can capture the directional and shape features of surface changes of objects in the image, and by comparing the angle between feature values ​​and gradient direction, it can effectively distinguish the surface features of different regions, which helps to accurately analyze the shape and location of the transmission line; by judging continuous pixels by gradient magnitude, it can effectively identify continuous surface regions in the image, which can help determine the actual location and shape of the line. Improving accuracy helps reduce noise and highlight key feature areas. Based on the distribution of gradient magnitudes between consecutive pixels and their neighbors within a local window corresponding to each pixel in the grayscale image, local transformation feature values ​​are determined for each pixel. Combined with these gradient direction feature values, local feature values ​​for the corresponding local window are obtained. Extracting these local transformation feature values ​​enhances the capture of details, especially sensitivity to subtle changes in the image, effectively distinguishing local features of high-contrast edge regions and damaged edge regions. Feature analysis is performed on edge regions and flat regions containing noise. Based on these local feature values, spatial range weights for the filtering algorithm are assigned to each pixel in the depth map corresponding to the grayscale image. The filtering algorithm can denoise and enhance the image, making it smoother and more coherent. Determining weights based on local feature values ​​helps optimize filtering effects and preserves important features in the image, especially when processing depth images. This ensures more accurate preservation of pixel features at high-contrast edges and fault / damage edges, guaranteeing the precise transmission of key information. Finally, projecting the filtered depth map onto the point cloud space yields a clearer and more accurate 3D representation, crucial for 3D modeling and monitoring of transmission lines. This provides more accurate spatial data, supporting subsequent line inspection and maintenance. Employing neural networks for transmission line monitoring further improves the accuracy and efficiency of monitoring. By training neural network models, abnormal conditions, faults, and potential risks in the lines can be effectively identified, enabling automated and intelligent monitoring and improving the safety and stability of transmission line monitoring. Attached Figure Description

[0032] Figure 1 A flowchart illustrating the steps of an online monitoring method for hidden dangers in transmission lines based on three-dimensional point clouds, provided in one embodiment of this application;

[0033] Figure 2 A flowchart illustrating the acquisition of local feature values ​​provided in one embodiment of this application. Detailed Implementation

[0034] In the description of the embodiments in this application, the words "exemplary," "or," and "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary," "or," and "for example" is intended to present the relevant concepts in a specific manner.

[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this application's specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0036] It should also be noted that the terms "first" and "second" in this application and its accompanying drawings are used to distinguish similar objects, rather than to describe a specific order or sequence. The methods disclosed in the embodiments of this application or the methods shown in the flowcharts include one or more steps for implementing the method. Without departing from the scope of protection of this application, the execution order of multiple steps can be interchanged, and some steps can also be deleted.

[0037] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0038] The following description, in conjunction with the accompanying drawings, details the specific scheme of the online monitoring method and system for hidden dangers in transmission lines based on three-dimensional point clouds provided in this application.

[0039] Please see Figure 1 The diagram illustrates a flowchart of an embodiment of the online monitoring method for hidden dangers in transmission lines based on three-dimensional point clouds, which includes the following steps:

[0040] The first step: Obtain the point cloud data of the transmission line and the depth image corresponding to the grayscale image.

[0041] In this application, a high-definition industrial camera and a lidar are used to acquire RGB images and point cloud data of power transmission lines. In this embodiment, a high-definition industrial camera with a 20-megapixel industrial camera is used to acquire RGB images of power transmission lines.

[0042] First, a drone inspection route is set. When the drone flies along the route, a high-definition industrial camera is used to acquire high-definition RGB images. The images acquired by each camera continuously are guaranteed to have more than 30% overlap to avoid missing images of the power transmission line. At the same time, a lidar is used to acquire point cloud data of the power transmission line in real time.

[0043] Preprocessing is performed on the image data acquired by the high-definition industrial camera: Using the acquired image data as input, this embodiment uses a weighted average method to calculate grayscale, outputting grayscale-processed image data; other embodiments may also use the maximum value method, minimum value method, and average value method to perform grayscale processing. Next, using the acquired grayscale image as input, this embodiment uses median filtering for filtering, outputting the filtered grayscale image; other embodiments may also use Gaussian filtering, Wiener filtering, and nonlocal mean filtering to perform image filtering.

[0044] After acquiring the preprocessed images, the camera parameters such as focal length, principal point, and distortion coefficients are used as input to calculate the depth map corresponding to each preprocessed grayscale image using the COLMAP model. The COLMAP model is a well-known technique, and the specific calculation process will not be elaborated upon.

[0045] The second step is to take each pixel in the grayscale image as the center, pre-set a local window, analyze the projection of the gradient direction of all pixels within the local window of each pixel, and combine the principal component analysis method to determine all feature vectors and their corresponding feature values ​​for the local window corresponding to each pixel; based on the differences between the feature values, and combined with the angles between the feature vectors, gradient vectors and the horizontal axis, determine the gradient direction feature value for the local window corresponding to each pixel.

[0046] Considering the sparse data points in 3D point clouds acquired by LiDAR during 3D reconstruction, the depth map, as core data supplementing the details of the 3D point cloud, directly impacts the accuracy and usability of the final model. Therefore, the acquired depth map needs optimization. Bilateral filtering is a commonly used method in depth map optimization because it smooths noise while effectively preserving edge features in the image. This makes it particularly suitable for scenarios like power transmission lines that require handling complex edges and noise.

[0047] First, the grayscale image pixels corresponding to the depth image are analyzed. In transmission lines, there are high-contrast geometric edge regions such as the edges of insulator strings, conductor connections, and the corners of tower angle steel, as well as sharp edge features appearing when the line is damaged. These edge features may resemble noise. When using the same spatial range to calculate spatial and color terms for different pixels, when a larger spatial range is set for calculation, high-contrast edge regions may be affected by distant, unrelated pixels, resulting in blurred edges; while when a smaller spatial range is set, noise in flat areas may not be effectively filtered out because there are fewer usable pixels within a smaller range, making effective noise smoothing difficult. Therefore, the regions where the pixels of the original image corresponding to the depth image are located are analyzed first.

[0048] In grayscale images of transmission lines, high-contrast geometric edge areas such as the edges of insulator strings, the connection points between conductors and connecting hammers, and the corners of tower angle steel, as well as areas of line damage, will appear as regular, continuous, and high-contrast jumps. Noise, on the other hand, will appear as random, isolated, and irregular jumps, while flat areas will appear as smooth transitions. A local window for each pixel is constructed, with the window size set to 5×5 in this embodiment. When the image cannot satisfy the requirement of a 5×5 area centered on the pixel, pixels within that area will be used to reconstruct the local window corresponding to the target pixel. For example, if the target pixel is located at the top left corner of the image, and its corresponding 5×5 area only exists in the lower right 3×3 area of ​​that pixel, then that 3×3 area will be set as the local window corresponding to the target pixel, and the missing area will not be supplemented.

[0049] Taking pixel a in the image as an example, the Sobel operator is first used to obtain the gradient directions of all pixels within the local window of pixel a, and then the gradient direction of each pixel is mapped to a two-dimensional unit vector. , ), Representative angle The projection in the horizontal direction (x-axis direction) reflects the horizontal component. Representative angle The projection in the vertical direction (y-axis direction) reflects the vertical component. Taking the two-dimensional unit vectors of all pixels within the local window as input, the PCA algorithm is used to calculate the feature vectors, outputting the feature vectors and their corresponding eigenvalues. The feature vectors include a first feature vector and a second feature vector, and the eigenvalues ​​include a first eigenvalue and a second eigenvalue. The first feature vector represents the vector corresponding to the direction of the most dramatic gradient change within the local window (i.e., the principal direction angle), and the first eigenvalue represents the variance (dispersion) of the gradient angles within the local window along the principal direction angle. The second feature vector represents the vector of the direction angles orthogonal to the principal direction angle within the local window, corresponding to the remaining gradient change directions, and the second eigenvalue represents the variance of the gradient angles within the local window along the direction corresponding to the second feature vector.

[0050] Based on the first and second feature values, the regional features within the local window are analyzed. Specifically, the difference and sum between the first and second feature values ​​are calculated respectively; the negative correlation mapping result of the sum value is positively fused with the difference value to obtain the feature deviation. In this embodiment, the first feature value is denoted as... The second eigenvalue is denoted as The characteristic deviation is expressed by the formula Obtain, in the formula, This is a preset parameter with a value of 0.01, used to prevent the denominator from being 0.

[0051] It should be understood that when there are high-contrast geometric edges or broken lines within a local window, there is a linear structure with a clear direction, and the first eigenvalue will be much larger than the second eigenvalue, with the obtained feature deviation approaching 1; when the local window is mainly a flat area containing noise, there is no obvious dominant direction in the local area, so the first eigenvalue is similar to the second eigenvalue, and the calculated feature deviation approaches 0.

[0052] Next, the angle between the principal direction of the first feature vector corresponding to pixel a after PCA dimensionality reduction within the local window and the horizontal axis is taken as the first angle, denoted as . Simultaneously, the angle between the gradient vector corresponding to pixel a and the horizontal axis is taken as the second angle, denoted as... The sine value of the difference between the first and second included angles corresponding to pixel a is obtained. The negative correlation mapping result after normalization of the sine value is then positively fused with the feature deviation to obtain the gradient direction feature value of pixel a. In this embodiment, the difference between variables is calculated using the absolute value of the difference; the normalization function is the sigmoid function; the sine value is denoted as A, and the feature deviation is denoted as B. The gradient direction feature value is obtained using the formula... To obtain, in the formula, This is a preset parameter with a value of 0.01, used to prevent the denominator from being 0.

[0053] It should be understood that when a pixel is located in a high-contrast edge or damaged edge region, since the edge line mainly exhibits a consistent high contrast, the principal direction angle of the first feature vector obtained by PCA dimensionality reduction within its local window is perpendicular to the edge direction. Since the gradient vector of the edge pixel is perpendicular to the edge direction, the first and second included angles are similar or differ by approximately 180°, thus making... As the value approaches 0, the gradient direction feature value increases accordingly; however, when the pixel is located in a flat region containing noise, the principal component direction is biased towards the distribution of noise points, and since the feature deviation value approaches 0, the calculated gradient direction feature value will still approach 0.

[0054] The third step: Based on the gradient magnitude, judge all pixels within the local window of each pixel to obtain continuous pixels; according to the distribution of gradient magnitude of continuous pixels within the local window of each pixel in the grayscale image and its neighboring pixels, determine the local transformation feature value of each pixel; and combine the gradient direction feature value to obtain the local feature value of the local window corresponding to each pixel.

[0055] Furthermore, the analysis examines whether each pixel is located at the edge of an image, such as an umbrella or a damaged / cracked area. When a pixel is located at an edge, the high contrast of the edge causes a grayscale jump at the edge, resulting in continuous, high, and similar gradient amplitudes for pixels on both sides. In flat areas, pixels do not have obvious edge features, and the overall gradient amplitude is relatively uniform. However, at the location of noise points, the gradient amplitude fluctuates randomly, and noise points may stick together, resulting in pseudo-edges. In this case, the gradient amplitude around the noise point changes drastically and randomly, without any obvious regularity.

[0056] Therefore, by combining the influence of noise on the judgment of whether a pixel is located at an edge, the edge and pseudo-edge situation of the local region where each pixel is located is analyzed. The spatial range of the pixel is weighted according to different situations. First, the pixels in the local window corresponding to each pixel are obtained: the gradient magnitude of all pixels in the local window corresponding to pixel a is obtained, and the gradient magnitude threshold is obtained using the Otsu thresholding algorithm. All pixels with gradient magnitude greater than the threshold are taken as labeled pixels. For each labeled pixel, if there is a labeled pixel in its eight neighborhood, the current labeled pixel is recorded as a continuous pixel.

[0057] Secondly, further analysis is performed to determine whether pixel a is located at an edge or a pseudo-edge. The absolute value of the difference in gradient magnitude between each consecutive pixel and each pixel in its eight neighborhoods is calculated, and the average of these absolute values ​​for all consecutive pixels is taken as the magnitude mean. If pixel a is a consecutive pixel, the sum of the absolute values ​​of the differences in magnitude mean between all pairs of consecutive pixels in the local window corresponding to pixel a is obtained, normalized, and used as the local transformation feature value of pixel a. If pixel a is not a consecutive pixel, the average of the maximum and minimum values ​​of the local transformation feature values ​​of all consecutive pixels in the grayscale image is obtained and used as the local transformation feature value of pixel a. In this embodiment, the maximum-minimum normalization method is used.

[0058] It should be understood that when pixel a is located in a flat region that is neither an edge nor a pseudo-edge, the local transformation feature value can ensure a more reasonable spatial range planning when filtering noise in the flat region; when pixel a is located at an edge, due to the continuity of the edge and the similarity of the amplitude, the mean amplitude is similar, which leads to the local transformation feature value approaching 0; when pixel a is located at a pseudo-edge formed by noise, due to the chaotic amplitude of the pseudo-edge, the mean amplitude is random, the local transformation feature value is large and approaches 1, and a larger spatial range is needed to filter out the noise point.

[0059] The local window feature transformation corresponding to pixel a is analyzed. Local feature values ​​for pixel a are constructed by combining gradient direction feature values ​​to characterize the local region features of pixel a: the normalized result of the gradient direction feature value of each pixel is obtained, and the ratio of the local change feature value of each pixel to the normalized result is used as the local feature value of each pixel. The flowchart for obtaining the local feature values ​​is as follows: Figure 2 As shown.

[0060] It should be understood that when a pixel is located in a high-contrast edge region or a damaged edge region of the scene, the higher the calculated gradient direction feature value, the lower the local change feature value, and the smaller the obtained local feature value; conversely, when a pixel is located in a flat region containing noise, the lower the calculated gradient direction feature value, the higher the local change feature value, and the larger the obtained local feature value.

[0061] The fourth step: Based on the local feature values, assign spatial range weights of the filtering algorithm to each pixel in the depth map corresponding to the grayscale image, project the filtered depth map to obtain point cloud data, match it with the point cloud data of the transmission line to obtain the three-dimensional point cloud of the transmission line, and use a neural network to monitor the transmission line.

[0062] Considering the depth map corresponding to a grayscale image, when using bilateral filtering for optimization, a smaller filtering range is needed for high-contrast or damaged edge regions to avoid interference from distant pixels on edge pixels, which could lead to blurring. Conversely, a larger filtering range can be used for flat regions containing noise to better remove noise. Therefore, based on the above analysis, the spatial range weights for the spatial and color terms of the bilateral filtering are adaptively set: In the formula, This represents the adaptive spatial range weight of pixel a in the depth map, with a one-to-one correspondence between the pixels in the depth map and the original image; This represents the local feature value of pixel a; This represents the initial spatial range weight in the depth map, with a value range of [5, 10]. In this embodiment, the value is 5.

[0063] After obtaining the adaptive range weights for each pixel, the acquired depth image is used as input, and a bilateral filtering algorithm is applied to filter the depth image. The spatial and color terms are weighted using pixels within the spatial range. Pixels that partially belong to the spatial range are not used. The final output is the filtered depth image. Bilateral filtering is a well-known technique, and its specific process will not be elaborated further.

[0064] Each pixel of each depth map is back-projected into 3D space, and the point cloud is transformed to a unified coordinate system by combining the extrinsic parameters of the UAV camera. Then, using the point cloud data obtained through depth map projection and the point cloud data collected by LiDAR as input, a point cloud fine-registration algorithm is used to fine-tune and align the two, and finally output the matched 3D point cloud of the transmission line. The point cloud fine-registration algorithm used in this embodiment is the ICP algorithm, which is a well-known technology in the field, and the specific operation steps will not be described in detail.

[0065] First, the labelCloud annotation software was used to annotate the acquired 3D point cloud of the transmission line with 3D bounding boxes, marking areas with potential hazards. Then, the annotated 3D point cloud was voxelized using an octree to obtain voxelized 3D point cloud data. Next, the 3D U-Net algorithm was used to train the transmission line hazard detection model. Cross-entropy was used as the loss function, the number of feature channels was set to 16, the AdamW optimizer was selected, the initial learning rate was 3e-4, and the training epochs were set to 100. The transmission line hazard detection model was obtained through model training and then run on a terminal to complete online monitoring of the transmission line based on the acquired voxelized 3D point cloud.

[0066] Based on the same inventive concept as the above methods, this application also provides an online monitoring system for hidden dangers in transmission lines based on three-dimensional point clouds, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of any one of the above-described methods for online monitoring of hidden dangers in transmission lines based on three-dimensional point clouds.

[0067] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description; sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0068] It will be apparent to those skilled in the art that this application is not limited to the details of the exemplary embodiments described above, and that this application can be implemented in other specific forms without departing from its essential characteristics. Therefore, the embodiments described above should be considered exemplary and non-limiting in all respects; modifications to the technical solutions described in the foregoing embodiments, or equivalent substitutions of some technical features, without causing the essence of the corresponding technical solutions to deviate from the scope of the technical solutions in the embodiments of this application, should all be included within the protection scope of this application.

Claims

1. A method for online monitoring of hidden dangers in power transmission lines based on three-dimensional point clouds, characterized in that, The method includes the following steps: Acquire point cloud data of transmission lines and depth images corresponding to grayscale images; Taking each pixel in the grayscale image as the center, a local window is preset. The projection of the gradient direction of all pixels within the local window of each pixel is analyzed. Combined with the principal component analysis method, all feature vectors and their corresponding feature values ​​of the local window corresponding to each pixel are determined. Based on the differences between the feature values, combined with the angles between the feature vectors, gradient vectors and the horizontal axis, the gradient direction feature value of the local window corresponding to each pixel is determined. Based on the gradient magnitude, all pixels within the local window of each pixel are judged to obtain continuous pixels; according to the distribution of gradient magnitude of continuous pixels within the local window of each pixel in the grayscale image and its neighboring pixels, the local transformation feature value of each pixel is determined, and combined with the gradient direction feature value, the local feature value of the local window corresponding to each pixel is obtained. Based on the local feature values, a spatial range weight of the filtering algorithm is assigned to each pixel in the depth map corresponding to the grayscale image. The filtered depth map is then projected to obtain point cloud data, which is matched with the point cloud data of the transmission line to obtain the three-dimensional point cloud of the transmission line. A neural network is then used to monitor the transmission line.

2. The online monitoring method for hidden dangers in transmission lines based on three-dimensional point clouds as described in claim 1, characterized in that, The process of determining all feature vectors and their corresponding feature values ​​for the local window corresponding to each pixel is as follows: The gradient direction of each pixel in the grayscale image is mapped to a two-dimensional vector. The two-dimensional vectors obtained from all pixels in the local window corresponding to each pixel are used as input to principal component analysis. The output is all feature vectors and their corresponding feature values ​​for the local window corresponding to each pixel. The feature vectors include a first feature vector and a second feature vector, and the feature values ​​include a first feature value and a second feature value.

3. The online monitoring method for hidden dangers in transmission lines based on three-dimensional point clouds as described in claim 2, characterized in that, The step of determining the gradient direction feature value of the local window corresponding to each pixel is as follows: For each pixel corresponding to a local window, based on the first feature value and the second feature value, the feature deviation within the local window corresponding to each pixel is obtained; The first angle is the angle between the main direction of the first feature vector of the local window corresponding to each pixel and the horizontal axis; the second angle is the angle between the gradient vector of each pixel and the horizontal axis. Based on the difference between the first included angle and the second included angle, and combined with the feature deviation, the gradient direction feature value of each pixel is obtained.

4. The online monitoring method for hidden dangers in transmission lines based on three-dimensional point clouds as described in claim 3, characterized in that, The specific steps for obtaining the feature deviation within the local window corresponding to each pixel are as follows: Calculate the difference and sum between the first feature value and the second feature value respectively; then, positively fuse the negative correlation mapping result of the sum value with the difference value to obtain the feature deviation.

5. The online monitoring method for hidden dangers in transmission lines based on three-dimensional point clouds as described in claim 3, characterized in that, The gradient direction feature value is specifically: Obtain the sine value of the difference between the first and second included angles of each pixel, and positively fuse the negative correlation mapping result after normalizing the sine value with the feature deviation to obtain the gradient direction feature value of each pixel.

6. The online monitoring method for hidden dangers in transmission lines based on three-dimensional point clouds as described in claim 1, characterized in that, The acquisition of continuous pixels specifically involves: Obtain the gradient magnitude threshold of all pixels within the local window corresponding to each pixel. Take all pixels with gradient magnitude greater than the gradient magnitude threshold as calibrated pixels. If there are calibrated pixels in the eight neighborhoods of each calibrated pixel, record the corresponding calibrated pixels as consecutive pixels.

7. The online monitoring method for hidden dangers in transmission lines based on three-dimensional point clouds as described in claim 1, characterized in that, The determination of the local transformation feature value of each pixel is specifically as follows: Calculate the absolute value of the difference between the gradient magnitude of each consecutive pixel and each pixel in its eight neighborhoods, and take the mean of the absolute values ​​of all consecutive pixels as the magnitude mean. If pixel a is a continuous pixel, obtain the sum of the absolute values ​​of the differences between the mean values ​​of the magnitudes of all pairs of continuous pixels in the local window corresponding to pixel a, and normalize them to use as the local transformation feature value of pixel a. If pixel a is not a continuous pixel, obtain the average of the maximum and minimum values ​​of the local transformation feature values ​​of all continuous pixels in the grayscale image, and use it as the local transformation feature value of pixel a.

8. The online monitoring method for hidden dangers in transmission lines based on three-dimensional point clouds as described in claim 1, characterized in that, The process of obtaining the local feature values ​​of the local window corresponding to each pixel is as follows: Obtain the normalized result of the gradient direction feature value of each pixel, and use the ratio of the local change feature value of each pixel to the normalized result as the local feature value of each pixel.

9. The online monitoring method for hidden dangers in transmission lines based on three-dimensional point clouds as described in claim 1, characterized in that, The process involves assigning spatial range weights to each pixel in the depth map corresponding to the grayscale image, specifically as follows: Calculate the sum between the natural number 1 and the local feature value of each pixel, and then positively fuse the obtained sum with the preset initial spatial range weight to obtain the spatial range weight of the corresponding pixel in the depth map.

10. An online monitoring system for potential hazards in transmission lines based on three-dimensional point clouds, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1-9.

Images