Power transmission line external damage scene safety distance analysis method based on multi-source data

Abstract

This invention discloses a method for analyzing the safe distance in power transmission line external damage scenarios based on multi-source data, relating to the technical field of power transmission line monitoring. The method includes: acquiring data on the location of the power transmission line using a visual camera and a lidar; extracting features from the image set and point cloud dataset to establish image feature sets and point cloud feature sets; establishing multi-class probability distributions corresponding to pixels and lidar points; constructing image sub-conditional random fields and point cloud sub-conditional random fields; performing cross-modal edge fusion to establish a hybrid conditional random field; constructing the spatial locations of the damaged object and the power transmission line, identifying the minimum distance from the damaged object to the power transmission line, and performing safety warning trigger analysis. This method solves the technical problem in existing technologies of accurately and in real-time monitoring the safe distance between the damaged object and the power transmission line in external damage scenarios, achieving the technical effect of improving the accuracy of power transmission line monitoring and the timeliness of safety warnings.

Description

Technical Field

[0001] This application relates to the technical field of transmission line monitoring, and in particular to a method for analyzing the safe distance in external damage scenarios of transmission lines based on multi-source data. Background Technology

[0002] As a critical infrastructure of the power system, the safe and stable operation of transmission lines is of paramount importance. In actual operation, transmission lines face numerous external threats, known as external damage problems. These include threats posed by construction machinery approaching or trees growing and encroaching on the lines, severely impacting the reliability and safety of the power grid. Traditional monitoring methods for external damage risks to transmission lines have many limitations. They struggle to comprehensively and accurately perceive complex and ever-changing external damage scenarios, and their monitoring efficiency is low. Furthermore, they cannot effectively analyze the safe distance between transmission lines and external damage objects in complex scenarios, thus failing to provide timely and accurate safety warnings. This can lead to transmission line faults and tripping, causing large-scale power outages and compromising the safety of transmission line operation.

[0003] Currently, there is a technical problem in the relevant technologies that makes it difficult to accurately and in real time monitor the safe distance between the damaged object and the transmission line in the event of external damage to the transmission line. Summary of the Invention

[0004] This application provides a method for analyzing the safe distance between externally damaged objects and transmission lines in scenarios involving external damage to transmission lines based on multi-source data. This solves the technical problem in the prior art of accurately and in real-time monitoring the safe distance between externally damaged objects and transmission lines in such scenarios, thereby improving the accuracy of transmission line monitoring and the timeliness of safety warnings.

[0005] This application provides a method for analyzing the safety distance in power transmission line damage scenarios based on multi-source data. The method includes: acquiring data on the location of the power transmission line using a visual camera and a lidar, respectively, to establish an image set and a point cloud dataset; performing initial registration of the image set and the point cloud dataset based on the acquisition parameters of the visual camera and the lidar; extracting features from the image set and the point cloud dataset to establish an image feature set and a point cloud feature set; inputting the image feature set and the point cloud feature set into a random forest model to establish multi-class probability distributions corresponding to pixels and lidar points; constructing an image sub-conditional random field based on the multi-class probability distributions of pixels and pixel neighborhood relationships, and constructing a point cloud sub-conditional random field based on the multi-class probability distributions of lidar points and spatial nearest neighbor constraints; performing cross-modal connection edge fusion of the image sub-conditional random field and the point cloud sub-conditional random field under the initial registration constraints to establish a hybrid conditional random field; using the hybrid conditional random field to construct the spatial location of the damaged object and the power transmission line, identifying the minimum distance from the damaged object to the power transmission line, and performing safety early warning trigger analysis based on the minimum distance.

[0006] In a possible implementation, the method for analyzing the safe distance of transmission line external damage scenarios based on multi-source data further performs the following processing: extracting pixel feature vectors for each pixel based on the image feature set, wherein the feature dimensions of the pixel feature vectors include RGB values, gray values, Gabor filter response, Sobel edge strength, SIFT and HOG, and coordinate information; inputting the pixel feature vectors into the image sub-classifier in the random forest model, performing probability distribution prediction for the corresponding pixel points, and establishing multi-class probability distributions for the pixel points.

[0007] In a possible implementation, the method for analyzing the safe distance of transmission line external damage scenarios based on multi-source data further performs the following processing: extracting point feature vectors for each laser point based on the point cloud feature set, wherein the feature dimensions of the point feature vectors include spatial coordinates, normal vector, curvature, reflection intensity, local point density, and point cloud elevation difference; inputting the point feature vectors into the point cloud sub-classifier in the random forest model, performing probability distribution prediction for the corresponding laser points, and establishing multi-class probability distributions for the laser points.

[0008] In a possible implementation, the method for analyzing the safe distance of transmission line external damage scenarios based on multi-source data further performs the following processing: defining single-node potential functions for pixels and laser points according to the multi-class probability distributions of pixels and laser points respectively; configuring adjacent pixel constraints and spatial neighborhood constraints, which are represented by binary potential functions; performing label fusion on the single-node potential functions and binary potential functions, and constructing image sub-conditional random fields and point cloud sub-conditional random fields respectively by minimizing energy functions.

[0009] In a possible implementation, the method for analyzing the safe distance of transmission line external damage scenarios based on multi-source data further performs the following processing: external parameter calibration and alignment of the image and point cloud using the initial registration constraints; mapping each laser point to a pixel location using a projection matrix, and establishing a one-to-many mapping between laser points and pixels through nearest neighbor search, constructing a cross-modal connection edge set; establishing a potential function for the cross-modal connection edges, using the potential function to perform semantic consistency analysis of the bridging edges of the cross-modal connection edge set, retaining cross-modal connection edges whose semantic consistency meets a preset threshold; and fusing the image sub-conditional random field and the point cloud sub-conditional random field based on the retained cross-modal connection edges to establish a hybrid conditional random field.

[0010] In a possible implementation, the method for analyzing the safe distance of transmission line external damage scenarios based on multi-source data further performs the following processing: establishing an alignment bias set for laser points and pixel points in the retained cross-modal bridging edge; using the alignment bias set to make a group bias decision under the influence of distance radiation, wherein the influence of distance radiation is an influence that decays with distance length; and updating the retained cross-modal bridging edge according to the group bias decision.

[0011] In a possible implementation, the method for analyzing the safe distance of a transmission line external damage scenario based on multi-source data further performs the following processing: obtaining the fixed point and the lowest point of sag of the transmission line; performing wind deflection galloping fitting simulation of the transmission line based on the fixed point and the lowest point of sag, and establishing the wind deflection galloping fitting simulation result; using the wind deflection galloping fitting simulation result and the spatial position of the externally damaged object to identify the shortest distance, and taking the shortest distance identification result as the minimum distance.

[0012] In a possible implementation, the method for analyzing the safe distance of transmission line external damage scenarios based on multi-source data also performs the following processing: continuously collecting data on the location of the transmission line to establish a time-series dataset; and using the time-series dataset to fit the motion trend of the externally damaged object to establish an evolution warning.

[0013] In a possible implementation, the method for analyzing the safe distance of transmission line external damage scenarios based on multi-source data also performs the following processing: obtaining the voltage level of the transmission line and simultaneously obtaining the object category of the externally damaged object; calculating the warning level based on the voltage level, object category, and minimum distance to complete the safety warning trigger analysis.

[0014] In a possible implementation, the method for analyzing the safe distance of transmission line external damage scenarios based on multi-source data also performs the following processing: establishing a visual demonstration video based on the object category and the transmission line; issuing an early warning based on the visual demonstration video and the early warning level.

[0015] This application proposes a method for analyzing the safe distance in power transmission line external damage scenarios based on multi-source data. This method utilizes visual cameras and LiDAR to collect data on the location of power transmission lines. Features are extracted from image sets and point cloud datasets to establish image feature sets and point cloud feature sets, respectively. Multi-class probability distributions corresponding to pixels and LiDAR points are established. Image sub-conditional random fields and point cloud sub-conditional random fields are constructed. Cross-modal edge fusion is performed to establish a hybrid conditional random field. The spatial locations of the damaged object and the power transmission line are constructed, and the minimum distance from the damaged object to the power transmission line is identified, followed by safety warning trigger analysis. This method solves the technical problem in existing technologies of accurately and in real-time monitoring the safe distance between the damaged object and the power transmission line in external damage scenarios, achieving the technical effect of improving the accuracy of power transmission line monitoring and the timeliness of safety warnings. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings of the embodiments of the present invention will be briefly described below. Flowcharts are used in this application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed precisely in sequence. Instead, various steps can be processed in reverse order or simultaneously as needed. Furthermore, other operations can be added to these processes, or one or more steps can be removed from these processes.

[0017] Figure 1 This is a flowchart illustrating the safety distance analysis method for power transmission line external damage scenarios based on multi-source data, provided in an embodiment of this application.

[0018] Figure 2 This is a flowchart illustrating the process of constructing a point cloud sub-conditional random field in the method for analyzing the safe distance of transmission line external damage scenarios based on multi-source data provided in this application embodiment. Detailed Implementation

[0019] The above description is merely an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below.

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description of this application will be provided in conjunction with the accompanying drawings. The described embodiments should not be considered as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] In the following description, references to "some embodiments" describe a subset of all possible embodiments. However, it is understood that "some embodiments" can be the same or different subsets of all possible embodiments and can be combined with each other without conflict. The terms "first" and "second" are used merely to distinguish similar objects and do not represent a specific ordering of objects. The terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, product, or server that includes a series of steps is not necessarily limited to those steps explicitly listed, but may include other steps not explicitly listed or inherent to such processes, methods, products, or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only.

[0022] This application provides a method for analyzing the safe distance in power transmission line external damage scenarios based on multi-source data, such as... Figure 1 As shown, the method includes:

[0023] Step S100: Use a visual camera and a lidar to collect data on the location of the power transmission line, establish an image set and a point cloud dataset, and complete the initial registration of the image set and point cloud dataset based on the acquisition parameters of the visual camera and lidar.

[0024] Preferably, the location of the power transmission line is captured by a vision camera, including capturing the shape of the power transmission line and its surrounding environment, namely static targets such as objects crossing or spanning the power transmission line, as well as large machinery such as cranes, tower cranes, and pump trucks, and dynamic targets such as corrugated steel plates, kites, plastic films, and balloons in the power transmission line scene. Multiple images reflecting the power transmission line and its surrounding environment are obtained and composed of an image set. The vision camera focuses light onto an image sensor through a lens. The image sensor is composed of many tiny photosensitive elements, such as charge-coupled devices (CCD) or complementary metal-oxide-semiconductor (CMOS). When light shines on the photosensitive elements, it generates charge or current signals, which are processed and converted to form digital image data.

[0025] Preferably, data acquisition of the transmission line location is performed using LiDAR. This involves emitting a laser beam and measuring the time of reflected light to determine the distance to the target object, scanning the transmission line location, and recording the three-dimensional coordinates of each point in space to obtain multiple point cloud datasets. This dataset accurately presents the spatial location and shape of the transmission line and surrounding objects, such as the specific location and shape of power towers. Then, based on the visual camera's acquisition parameters (focal length, aperture, shooting angle, etc.) and the LiDAR's acquisition parameters (scanning range, scanning frequency, laser beam angle resolution, etc.), initial registration of the image set and point cloud dataset is completed. Specifically, based on the two acquisition parameters, the images in the image set are initially aligned and calibrated with the point clouds in the point cloud dataset in terms of spatial location and direction, ensuring that the same object corresponds in both the image and the point cloud. For example, the location of the transmission tower in the image corresponds to the location of the transmission tower in the point cloud data, ensuring accurate correlation between data from different modalities and more efficiently analyzing external damage scenarios of transmission lines.

[0026] Step S200: Extract features from the image set and the point cloud dataset respectively to establish the image feature set and the point cloud feature set.

[0027] Preferably, feature extraction is performed on the image set. Specifically, this includes converting the images from the common RGB color space to other color spaces more suitable for feature extraction, such as the HSV (Hue, Saturation, Lightness) space or the Lab color space; calculating color histograms for different regions in the image, statistically analyzing the distribution of various colors, and describing the color characteristics of the transmission line and surrounding objects. For example, the metal parts of the transmission line may have specific gray tones, and color features can be used to initially distinguish objects of different materials; and using the Gray-Level Co-occurrence Matrix (GLCM) to analyze the spatial distribution of gray values ​​in the image, i.e., converting the color image to a grayscale image and then calculating the GLCM, where the GLCM describes the spatial distribution of pixel pairs with specific gray values ​​in a given direction. The frequency of occurrence at different distances is calculated, and gray-level co-occurrence matrices are typically calculated in multiple directions (e.g., 0°, 45°, 90°, 135°) and at different distances to obtain texture information, such as the surface texture of transmission lines and the structural texture of towers, which helps to identify different objects and components. Edge detection algorithms, such as the Canny algorithm, are used to perform edge detection on the image to extract the edges of objects in the image. Then, the shapes of transmission lines and external objects are described through contour analysis and other methods, such as the circular cross-section of transmission lines and rectangular towers. The extracted color, texture, shape and other features are integrated to form an image feature set. Each image has a corresponding feature vector representation in this feature set, which is used to comprehensively describe the visual features of the transmission lines and their surrounding environment in the image.

[0028] Preferably, feature extraction is performed on the point cloud dataset. Specifically, this includes calculating the coordinate information of points in the point cloud, including the point's position, distance, and angle, to describe the spatial geometry of transmission lines and damaged objects. For example, point cloud data can be used to obtain information such as the spatial orientation of transmission lines and the height and position of towers. The normal vectors of points in the point cloud are estimated to reflect the local orientation and curvature information of the point cloud surface, which helps to distinguish objects with different shapes and surface features. For example, the surface normal vectors of transmission lines and towers are significantly different. The density distribution of the point cloud is statistically analyzed. The point cloud density may be different for different objects or different parts of an object. For example, the point cloud density of transmission lines is relatively low, while the point cloud density of towers or damaged objects may be higher. The extracted geometric, normal, and density features are integrated to construct a point cloud feature set. Each point in the point cloud dataset has a corresponding feature representation in the feature set, which can accurately reflect the spatial characteristics and physical properties of the point cloud data.

[0029] Step S300: Input the image feature set and point cloud feature set into the random forest model respectively to establish the multi-class probability distributions corresponding to the pixel points and laser points.

[0030] Preferably, the image feature set and point cloud feature set are used as input data for the random forest model to establish multi-class probability distributions corresponding to pixels and laser points. Specifically, for the image feature set, the random forest model learns the relationship between the features of pixels in the image and different categories, and outputs a probability distribution for each pixel belonging to each category. For example, in the analysis of external damage to power transmission lines, the categories may include power transmission lines, background, and damaged objects. The random forest model calculates the probability that the pixel belongs to the power transmission line, the probability that it belongs to the background, and the probability that it belongs to the damaged object based on the features of the pixel. Then, each pixel in the image obtains a multi-class probability distribution, and the pixel is most likely to belong to the category based on the maximum probability. Similarly, for point cloud feature sets, the random forest model establishes multiple probability distributions for each laser point. That is, based on the characteristics of the laser point, the random forest model calculates the probability that the laser point belongs to different categories, such as the probability of belonging to the point cloud of transmission lines, towers, ground, or externally damaged objects. Then, it classifies each laser point and obtains the confidence information of each category, i.e., the probability value.

[0031] Furthermore, step S300 also includes step S310, extracting a pixel feature vector for each pixel based on the image feature set. The feature dimensions of the pixel feature vector include RGB value, gray value, Gabor filter response, Sobel edge strength, SIFT and HOG, and coordinate information. Step S320, inputting the pixel feature vector into the image sub-classifier in the random forest model to perform probability distribution prediction for the corresponding pixel and establish a multi-class probability distribution for the pixel.

[0032] Preferably, for an image in the image feature set, each pixel is traversed, and the corresponding feature value of each pixel is calculated according to feature dimensions such as RGB value, grayscale value, Gabor filter response, Sobel edge strength, SIFT and HOG, and coordinate information. These are then combined into a pixel feature vector. Specifically, RGB value refers to the color values ​​of the three channels (red, green, and blue) in a color image, which directly reflects the color information of the pixel. Different objects have different color representations in the image; for example, power transmission lines may exhibit a specific metallic color, and their RGB values ​​will have a corresponding range. Grayscale value is the brightness value of the pixel after converting the color image to a grayscale image, which can better highlight the light and dark features of the object and help to identify the outline and structure of the object. The Gabor filter is a filter with direction selectivity and scale selectivity. Applying Gabor to the image... After applying the r-filter, filtering responses at different directions and scales can be obtained, which can be used to capture the texture information of the image, such as the texture of the surface of power transmission lines and the structural texture of towers; the Sobel operator is a commonly used edge detection operator. By calculating the gradient of the image in the horizontal and vertical directions, it obtains the Sobel edge strength of the pixel, highlighting the edges of objects in the image and helping to identify the shape and boundaries of objects, such as the boundary between power transmission lines and the surrounding background; SIFT (Scale Invariant Feature Transform) can extract feature points and feature descriptors of the image at different scales and rotation angles; HOG (Histogram of Oriented Gradients) describes the local features of the image by calculating the gradient orientation histogram of local regions of the image; the coordinate information of a pixel in the image can reflect the position of the pixel in the image. Combined with other features, it can better analyze the spatial distribution of objects in the image.

[0033] Preferably, the random forest is a classification model composed of multiple decision trees. The image sub-classifier is the unit that processes image data. Specifically, the image sub-classifier is trained with a large number of pixel feature vectors of known categories and category labels. Then, the pixel feature vector of each extracted pixel is sequentially input into the image sub-classifier. Each decision tree in the image sub-classifier splits and judges nodes based on the feature values ​​of the pixel feature vector, and finally outputs the category prediction result. Then, the random forest model calculates the probability of each pixel belonging to each category by combining the prediction results of multiple decision trees. That is, the model calculates the probability that the pixel belongs to the power transmission line, the probability that it belongs to the background, and the probability that it belongs to the external damaged object, etc., based on the category prediction results of the decision trees. Then, the probabilities of each pixel belonging to each category are combined to generate the multi-class probability distribution corresponding to the pixel, which intuitively reflects which category each pixel is most likely to belong to, and the probability of belonging to other categories. For example, the probability of a pixel belonging to the power transmission line category is 0.7, the probability of belonging to the background category is 0.2, and the probability of belonging to the external damaged object category is 0.1. By establishing multi-class probability distributions for pixels, images can be classified and analyzed in a more detailed manner, which helps in the identification of externally damaged objects and the analysis of safe distances.

[0034] Step S400: Construct an image sub-conditional random field based on the multi-class probability distribution of pixels and pixel neighborhood relationships, and construct a point cloud sub-conditional random field based on the multi-class probability distribution of laser points and spatial nearest neighbor constraints.

[0035] Preferably, pixel neighborhood relationships refer to the spatial relationships between a pixel and its neighboring pixels. The pixels above, below, to the left, right, and diagonally adjacent to a pixel in an image are defined as its neighboring pixels. There is a certain correlation between neighboring pixels; for example, neighboring pixels may have similar colors, textures, or other features. Furthermore, at object boundaries, the categories of neighboring pixels may also have certain constraints. Then, a sub-conditional random field (CRF) is constructed based on the multi-class probability distribution of pixels and the pixel neighborhood relationships. Each pixel is treated as a node, and the neighborhood relationships between pixels are treated as edges. An energy function is defined based on the multi-class probability distribution of pixels and the relationships between neighboring pixels to measure the rationality of the pixel category assignment in the entire image. Here, the Conditional Random Field (CRF) is an undirected graphical model used to handle data with dependencies. For example, if the category of a pixel differs greatly from the categories of its neighboring pixels, it will show a higher energy value in the energy function, and vice versa. By minimizing the energy function, the most reasonable pixel category assignment can be obtained, i.e., more accurate image classification can be achieved.

[0036] Preferably, spatial nearest neighbor constraint refers to the relationship between a laser point and other laser points that are relatively close in the surrounding space. In point cloud data, the nearest neighbor is determined by calculating the distance between laser points. Unlike the pixel neighborhood relationship in an image, the spatial nearest neighbor relationship in point cloud emphasizes the distance and positional relationship in three-dimensional space. For example, in the point cloud data of a power transmission line, laser points belonging to the same power transmission line are often close to each other in space and have a certain spatial distribution pattern. Based on this, each laser point is treated as a node, and the spatial nearest neighbor relationship between laser points is treated as an edge to construct a point cloud sub-conditional random field. Then, based on the multi-class probability distribution of laser points and the constraint relationship between spatial nearest neighbors, and considering the consistency between the class of the laser point and the class of its spatial nearest neighbors, as well as the rationality of the spatial distribution, an energy function is defined. By minimizing the energy function, the laser points in the point cloud data can be classified more accurately, thereby better identifying different objects and scenes such as power transmission lines, towers, and the ground.

[0037] Furthermore, such as Figure 2 As shown, step S400 further includes step S410, defining single-node potential functions for pixels and laser points respectively based on the multi-class probability distributions of pixels and laser points; step S420, configuring adjacent pixel constraints and spatial neighborhood constraints, wherein the adjacent pixel constraints and the spatial neighborhood constraints are represented by binary potential functions; step S430, performing label fusion on the single-node potential functions and binary potential functions, and constructing image sub-conditional random fields and point cloud sub-conditional random fields respectively by minimizing energy functions.

[0038] Preferably, an image sub-conditional random field is constructed using the relationship between pixels and their neighborhoods for inference optimization of pixel labels. Specifically, this is based on the multi-class probability distribution of pixels. Define a single-node potential function for each pixel i, representing the probability that the pixel belongs to each category: ,in, It is the category label of pixel i, such as power line, background, external damage, etc. It is a random variable representing the category label of pixel i. It is the feature vector of pixel i, such as color, texture, brightness, etc. Let be the single-node potential function of pixel i; there are spatial neighborhood relationships between laser points in the point cloud. The similarity of neighboring pixels is measured by Euclidean distance or the angular difference of the point cloud normal vector, and then a binary potential function is defined using the spatial neighborhood relationship: ,in, It is a binary potential function for pixel i and pixel j. It is a pixel. eigenvectors, It is the Euclidean distance between pixel i and pixel j in the feature space. This is the similarity weight, with a value between 0 and 1. This is a hyperparameter used to control the range of influence of feature similarity on label consistency. The larger the value, the larger the denominator of the exponential part, and the closer the exponential function value is to 1. The influence of feature distance on the constraint of label consistency between pixels is weakened. It is a pixel category consistency function, typically designed as follows: The potential function maintains spatial consistency of pixel categories through the local smoothness of the image; the single-node potential function and the binary potential function are fused for labeling, and finally the energy function of the entire image is minimized. To determine the optimal class label assignment, i.e., to find a set of class labels for pixels that minimizes the energy function, a subconditional random field for the image is constructed. It is the label sequence of the entire image.

[0039] Preferably, a sub-conditional random field is constructed in the point cloud by leveraging the constraint relationships between the laser points and their spatial neighbors, thereby optimizing the label distribution of each laser point in the point cloud. Specifically, a single-node potential function is defined based on the classification probability of each laser point. The single-node potential function is ,in, It is a laser point Category tags, such as transmission line points, tower points, ground points, and external damaged object points. These are the feature vectors of the laser point, such as position, normal vector, curvature, etc. It is a random variable representing the category label of laser point n. This is the single-node potential function for each laser point n. Assuming spatial neighborhood relationships exist between laser points in the point cloud, the similarity between neighboring points is measured using Euclidean distance or the angular difference of the point cloud normal vector. A binary potential function is then constructed using spatial neighborhood constraints. ,in, These are laser points in the point cloud. and laser point spatial coordinates, It is a laser point and laser point The binary potential function, This is the spatial proximity weight. The closer two laser points are, the closer this value is to 1, and the stronger the constraint. The farther apart the two laser points are, the closer this value is to 0, and the weaker the constraint. It is the category consistency function of laser points, defined as: Then, the single-node potential function and the binary potential function are fused using labels, and the energy function is minimized. The optimal category label assignment is determined by finding a set of category labels for laser points that minimizes the energy function, thereby achieving accurate segmentation of the point cloud. A sub-conditional random field for the point cloud is then constructed, where the energy function... It is the sum of all potential functions. The label sequence representing all laser points in the entire point cloud is the optimal label configuration to be solved.

[0040] Step S500: Under the initial registration constraints, perform cross-modal connection edge fusion of the image sub-conditional random field and the point cloud sub-conditional random field to establish a hybrid conditional random field.

[0041] Preferably, images and point clouds belong to different modalities of data. Images are two-dimensional visual information, while point clouds are three-dimensional spatial information. Based on the image sub-conditional random field and the point cloud sub-conditional random field, cross-modal connection edge fusion is performed. That is, fusion is achieved by adding cross-modal connection edges. Here, the connection edge represents the relationship between pixels in the image and laser points in the point cloud. According to the initial registration constraints, the laser point corresponding to a certain pixel in the image in the point cloud is found, or a certain spatial mapping relationship is determined. Then, connection edges are established to fuse the information in the image sub-conditional random field and the point cloud sub-conditional random field to obtain a hybrid conditional random field. This hybrid conditional random field can simultaneously utilize information from the image and the point cloud for more accurate reasoning and classification. The nodes of the hybrid conditional random field include both pixels in the image and laser points in the point cloud. The edges include neighborhood edges between pixels in the image, spatial neighborhood edges between laser points in the point cloud, and cross-modal connection edges. The hybrid conditional random field can accurately obtain the optimal classification or labeling results for each point in the image and point cloud data.

[0042] Furthermore, step S500 also includes step S510, using the initial registration constraints to perform extrinsic parameter calibration and alignment of the image and point cloud; step S520, mapping each laser point to a pixel location through a projection matrix, and establishing a one-to-many mapping between laser points and pixels through nearest neighbor search, constructing a cross-modal connection edge set; step S530, establishing the potential function of the cross-modal connection edge, using the potential function to perform semantic consistency analysis of the bridging edges of the cross-modal connection edge set, and retaining cross-modal connection edges whose semantic consistency meets a preset threshold; step S540, fusing the image sub-conditional random field and the point cloud sub-conditional random field based on the retained cross-modal connection edges to establish a hybrid conditional random field.

[0043] Preferably, the extrinsic parameter calibration of the image and point cloud is performed using initial registration constraints. This involves obtaining extrinsic parameters based on feature matching using the initial registration information, and determining the relative position and orientation of the image acquisition device and the point cloud acquisition device. Then, the point cloud data and image data are aligned according to the extrinsic parameters. This involves transforming the point cloud data according to rotation and translation to achieve a precise spatial correspondence with the image data. The projection matrix is ​​used to project laser points in three-dimensional space onto pixels in a two-dimensional image plane. The projection matrix maps each laser point to a pixel location. Due to the distribution characteristics of laser points and pixels, each laser point may correspond to multiple pixels. A nearest neighbor search algorithm is used to find the multiple pixels closest to the projected pixel, thus establishing a one-to-many mapping relationship between laser points and pixels. The connection between each laser point and its corresponding multiple pixels is considered as an edge, thus forming a set of cross-modal connection edges.

[0044] Preferably, a potential function is established by comprehensively considering the feature information of laser points and pixels, such as color, texture, spatial location, and the probability of their respective categories. This function is used to measure the rationality and reliability of cross-modal connection edges. Then, the potential function is used to perform semantic consistency analysis on each edge in the set of cross-modal connection edges, that is, to evaluate the semantic similarity between laser points and pixels, such as whether they belong to the same object, such as power transmission lines or towers. Only when the semantic consistency meets a preset threshold is the cross-modal connection edge reliable. Cross-modal connection edges that do not meet the preset threshold are then removed, and only those that meet the requirements are retained. Next, the reliable cross-modal connection edges are used to connect the image sub-conditional random field and the point cloud sub-conditional random field, so that the data of the two modalities can transmit information, and finally a hybrid conditional random field is obtained, which helps to achieve more accurate and comprehensive classification.

[0045] Furthermore, step S530 also includes step S531, establishing an alignment bias set for laser points and pixel points in the retained cross-modal bridging edge; step S532, using the alignment bias set to make a group bias decision under the influence of distance radiation, wherein the influence of distance radiation is an influence that decays with distance length; and step S533, updating the retained cross-modal bridging edge according to the group bias decision.

[0046] Preferably, for each cross-modal bridging edge, i.e., the edge connecting the laser point and the pixel point, since the laser point and the pixel point differ in spatial position, features, etc., the alignment will have a bias. By calculating the coordinate deviation when the laser point is projected onto the pixel point, the degree of difference in color, texture, geometric features, etc., the alignment bias information between the laser point and the pixel point is determined. The alignment bias information of all points constitutes the alignment bias set, which records the alignment status of the laser point and the pixel point in each cross-modal bridging edge. Distance radiation influence refers to the gradual attenuation of mutual influence as the distance between the laser point and the pixel point increases. Then, the alignment bias set is used to perform a group bias decision on the distance radiation influence of all retained cross-modal bridging edges. That is, considering the alignment bias and spatial distance relationship of each cross-modal bridging edge, the bias decision of the entire group is determined, i.e., the laser points and pixels involved in all retained cross-modal bridging edges, to obtain the group bias decision result. Finally, the retained cross-modal bridging edges are updated based on the group bias decision results. If the decision results indicate that some cross-modal bridging edges have significant alignment problems or low reliability, they are removed from the retention set; bridging edges with good alignment and high reliability are retained.

[0047] Step S600: The spatial location of the damaged object and the transmission line is constructed using the hybrid conditional random field, and the minimum distance from the damaged object to the transmission line is identified. Safety warning trigger analysis is performed based on the minimum distance.

[0048] Preferably, for damaged objects and transmission lines, the hybrid conditional random field (HRF) determines their spatial location in three-dimensional space based on visual features such as color, texture, and shape of the objects in the image, as well as geometric features such as the three-dimensional coordinates and point cloud distribution of the objects. For example, by recognizing the appearance of the transmission line in the image and the corresponding three-dimensional point information in the point cloud, the direction and position of the transmission line in space can be accurately constructed. Similarly, for damaged objects, their specific spatial location is determined using their feature information in the image and point cloud, thus accurately depicting the positional relationship in three-dimensional space. Then, the distance from the damaged object to the transmission line is calculated. For example, the transmission line and the damaged object are approximated with appropriate geometric shapes, and the distance from each point on the damaged object to each point on the transmission line is calculated according to the distance calculation formula, and the minimum distance is found.

[0049] Preferably, based on the safety standards and actual operational requirements of transmission lines, a pre-set safety threshold distance is used to determine whether an externally damaged object poses a safety threat to the transmission line. Then, the minimum distance from the identified externally damaged object to the transmission line is compared with the safety threshold distance. If the minimum distance is less than or equal to the safety threshold distance, it indicates that the externally damaged object has approached or entered the danger zone that could damage the transmission line, triggering a safety warning to indicate potential safety risks and allow for timely intervention. If the minimum distance is greater than the safety threshold distance, it indicates that the current distance between the externally damaged object and the transmission line is within a safe range, and there is no obvious safety hazard at present, but continuous monitoring is still required. This achieves real-time monitoring and early warning of external damage risks to transmission lines, ensuring the safe operation of the transmission lines.

[0050] Furthermore, step S600 also includes step S610, obtaining the fixed point and the lowest point of sag of the transmission line; step S620, performing wind deflection galloping fitting simulation of the transmission line based on the fixed point and the lowest point of sag, and establishing the wind deflection galloping fitting simulation result; step S630, using the wind deflection galloping fitting simulation result and the spatial position of the externally damaged object to identify the shortest distance, and taking the shortest distance identification result as the minimum distance.

[0051] Preferably, the transmission line has fixed connection points at locations such as towers to support and fix the line, limiting its range of movement. Obtaining the coordinates of these fixed points helps determine the approximate direction and boundary conditions of the transmission line. Due to the weight of the transmission line itself, a certain sag will form between two fixed points. The lowest point of the sag is the point where the transmission line is lowest from the ground within that distance. Accurately obtaining the location of the lowest point of the sag is used to analyze the spatial morphology of the transmission line. The distance between the lowest point of the sag and surrounding objects may be the smallest in the entire line, making it more susceptible to the influence of external objects.

[0052] Preferably, under the influence of wind, transmission lines will exhibit wind-induced galloping. Wind deflection refers to the horizontal deviation of the line from its initial position due to wind force, while galloping is a more complex vibration phenomenon, including periodic oscillations in both vertical and horizontal directions. The amplitude and frequency of wind-induced galloping are influenced by various factors, such as wind speed, wind direction, and line tension. Using information obtained from fixed points and the lowest point of sag, combined with the physical characteristics of the transmission line conductors (material, diameter, tension, etc.) and meteorological data such as local wind speed and direction statistics, a simulation of wind-induced galloping is performed. For example, the finite element method is used to simulate the motion of the transmission line under different wind conditions. By continuously adjusting the model parameters and input conditions, the simulation results can accurately reflect the actual wind-induced galloping of the transmission line, ultimately establishing a wind-induced galloping simulation result that includes information on the spatial position changes of the transmission line at different times and under different wind conditions.

[0053] Preferably, by combining the simulation results of wind deflection galloping with the spatial location of the damaged object, the distance relationship between the damaged object and the transmission line is analyzed at different time points or under different wind conditions. Specifically, for each wind condition, considering wind deflection galloping, the distance between points on the damaged object and points on the transmission line is calculated. That is, spatial search is used to quickly locate the transmission line section near the damaged object and accurately calculate its distance, obtaining multiple distances under different wind conditions. Then, the minimum value is selected as the minimum distance from the damaged object to the transmission line, which can more accurately reflect the potential threat level of the damaged object to the transmission line, thereby more scientifically assessing the safe distance between the damaged object and the transmission line and improving the accuracy and reliability of safety early warning.

[0054] Furthermore, step S600 also includes step S640, performing continuous data acquisition of the transmission line location to establish a time-series dataset; and step S650, using the time-series dataset to fit the motion trend of the externally damaged object and establish an evolution warning.

[0055] Preferably, various sensors, such as position sensors and tilt sensors, are installed on the transmission line to acquire its position information in real time, capturing changes in its state at different times. The collected transmission line position data is then organized chronologically to form a time-series dataset, containing the transmission line's position coordinates, tension, vibration frequency, etc., reflecting the changes in the transmission line's position over time. Information related to the externally damaged object is then extracted from the time-series dataset, such as the relative position of the object to the transmission line and the object's own trajectory. The movement trend of the object is then fitted using time-series analysis to predict its future position and direction of movement based on historical data, thus establishing an evolution warning system. For example, when the predicted position of the object approaches the transmission line to a certain extent, or when its movement trend indicates a potential threat to the transmission line, an evolution warning is triggered. This warning may include the object's current position, predicted trajectory, and estimated time of arrival at the transmission line's danger zone, preventing damage and ensuring the safe operation of the transmission line.

[0056] Furthermore, step S600 also includes step S660, obtaining the voltage level of the transmission line and simultaneously obtaining the object category of the externally damaged object; step S670, calculating the warning level based on the voltage level, object category, and minimum distance to complete the safety warning trigger analysis.

[0057] Preferably, transmission lines of different voltage levels have different requirements for safety distances and protective measures. Voltage level information, such as 110kV, 220kV, and 500kV, is obtained through the design data of the transmission lines. External damaging objects refer to external objects that may cause damage to the transmission lines. These may include construction machinery such as cranes and tower cranes, as well as objects such as trees, billboards, kites, plastic film, and balloons. Different voltage levels, object types, and minimum distances have different degrees of impact on the safety of transmission lines. A corresponding weight is assigned to each factor. Generally speaking, the higher the voltage level, the stricter the safety distance requirements and the greater the weight. Large construction machinery poses a greater potential threat to the safety of transmission lines and has a relatively high weight. The minimum distance directly reflects the proximity of the external damaging object to the transmission line. The smaller the distance, the higher the degree of danger and the higher the weight.

[0058] Then, a warning level calculation model is established. Warning values ​​are calculated through weighted summation, and different warning level thresholds are set according to the transmission line safety operation standards. For example, the warning values ​​are divided into four levels: a warning value less than 0.5 indicates a low-risk warning, meaning the external object poses a relatively small threat to the transmission line's safety, requiring routine monitoring; a warning value between 0.5 and 1.0 indicates a medium-risk warning, requiring increased attention to the external object and the implementation of preventative measures; a warning value between 1.0 and 1.5 indicates a high-risk warning, requiring immediate protective measures such as halting related construction activities and temporarily securing the object; and a warning value greater than 1.5 indicates an extremely high-risk warning, necessitating immediate emergency measures such as evacuating personnel and shutting down for maintenance to prevent potential transmission line faults and safety accidents. By comparing the calculated warning value with the set warning level thresholds, the degree of threat posed by the current external object to the transmission line's safety is determined, triggering the corresponding level of safety warning. This achieves a comprehensive assessment and accurate warning of the transmission line's safety status, ensuring the reliable operation of the transmission line.

[0059] Furthermore, step S670 also includes step S671, establishing a visual demonstration video based on the object category and the power transmission line; and step S672, issuing an early warning based on the visual demonstration video and the early warning level.

[0060] Preferably, 3D modeling software is used to construct 3D models of transmission lines and externally damaged objects based on collected data. For example, a line model is built according to the actual route of the transmission line and the location of the towers, and corresponding models of externally damaged objects are built according to object categories, such as cranes, tower cranes, and trees. Then, based on the movement trend of the externally damaged objects and the state of the transmission line, the movement path and actions of the models are set to simulate the process of the externally damaged objects approaching the transmission line and to show their relative position changes. Some auxiliary elements can also be added, such as arrows indicating the direction of movement and distance value displays. Finally, the produced 3D animation is combined with an appropriate background to form a visual demonstration video. Then, the warning level is associated with the visual demonstration video, and different colors and symbols are used in the video. Alternatively, warning messages can be differentiated. For example, a level 1 warning can be indicated with a green symbol, a level 2 warning with a yellow symbol, a level 3 warning with an orange symbol, and a level 4 warning with a red symbol. Then, warnings can be issued based on the warning level and a visual demonstration video. For instance, when the warning level reaches a certain threshold, different types of sound alarms can be issued, and a visual demonstration video can be displayed on the monitoring screen, highlighting the current warning level and related warning messages. Alarm devices, such as alarm lights and horns, can be installed in relevant areas near the transmission lines. When the warning level reaches a certain standard, the on-site alarm devices can be activated to warn surrounding personnel of potential dangers. This allows for a more intuitive and effective communication of external damage risks to the transmission lines, ensuring their safe operation.

[0061] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application. In some cases, the actions or steps described in this application can be performed in a different order than that shown in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A method for analyzing safe distances in power transmission line external damage scenarios based on multi-source data, characterized in that, The method includes: Data on the location of power transmission lines were collected using visual cameras and lidar, respectively, to establish image sets and point cloud datasets. Initial registration of the image sets and point cloud datasets was completed based on the acquisition parameters of the visual cameras and lidar. Feature extraction is performed on the image set and the point cloud dataset respectively to establish image feature set and point cloud feature set; The image feature set and point cloud feature set are respectively input into the random forest model to establish multi-class probability distributions corresponding to pixels and laser points; Image subconditional random fields are constructed based on the multi-class probability distribution of pixels and pixel neighborhood relationships, and point cloud subconditional random fields are constructed based on the multi-class probability distribution of laser points and spatial nearest neighbor constraints. Under initial registration constraints, cross-modal connectivity edge fusion of image sub-conditional random fields and point cloud sub-conditional random fields is performed to establish a hybrid conditional random field, including: The initial registration constraints are used to perform extrinsic parameter calibration and alignment of the image and point cloud; Each laser point is mapped to a pixel location using a projection matrix, and a one-to-many mapping between laser points and pixels is established through nearest neighbor search, thus constructing a set of cross-modal connection edges; Establish the potential function of cross-modal connection edges, and use the potential function to perform semantic consistency analysis of the bridging edges of the cross-modal connection edge set, retaining cross-modal connection edges whose semantic consistency meets the preset threshold; A hybrid conditional random field is established based on the retained cross-modal connection edges fused image sub-conditional random field and point cloud sub-conditional random field. The spatial locations of externally damaged objects and transmission lines are constructed using the hybrid conditional random field, and the minimum distance from the externally damaged object to the transmission line is identified. Safety early warning trigger analysis is then performed based on the minimum distance.

2. The method for analyzing the safe distance in transmission line external damage scenarios based on multi-source data as described in claim 1, characterized in that, The step of inputting the image feature set and point cloud feature set into a random forest model to establish multi-class probability distributions corresponding to pixels and laser points includes: Pixel feature vectors are extracted for each pixel based on the image feature set. The feature dimensions of the pixel feature vectors include RGB values, gray values, Gabor filter response, Sobel edge strength, SIFT and HOG, and coordinate information. The pixel feature vector is input into the image subclassifier in the random forest model to perform probability distribution prediction for the corresponding pixel and establish the multi-class probability distribution of the pixel.

3. The method for analyzing the safe distance in transmission line external damage scenarios based on multi-source data as described in claim 2, characterized in that, The step of inputting the image feature set and point cloud feature set into a random forest model to establish multi-class probability distributions corresponding to pixels and laser points also includes: Based on the point cloud feature set, a point feature vector is extracted for each laser point. The feature dimensions of the point feature vector include spatial coordinates, normal vector, curvature, reflection intensity, local point density, and point cloud elevation difference. The point feature vectors are input into the point cloud sub-classifier in the random forest model to perform probability distribution prediction of the corresponding laser points and establish multi-class probability distributions of the laser points.

4. The method for analyzing the safe distance in a transmission line external damage scenario based on multi-source data as described in claim 1, characterized in that, The construction of an image sub-conditional random field based on the multi-class probability distribution of pixels and pixel neighborhood relationships, and the construction of a point cloud sub-conditional random field based on the multi-class probability distribution of laser points and spatial nearest neighbor constraints, include: The single-node potential functions of pixels and laser points are defined based on the multi-class probability distributions of pixels and laser points, respectively. Configure adjacent pixel constraints and spatial neighborhood constraints, which are characterized by binary potential functions; Label fusion is performed on the single-node potential function and the binary potential function, and the image subconditional random field and the point cloud subconditional random field are constructed by minimizing the energy function, respectively.

5. The method for analyzing the safe distance in a transmission line external damage scenario based on multi-source data as described in claim 4, characterized in that, The step of performing semantic consistency analysis on bridging edges of the cross-modal connection edge set using the potential function, and retaining cross-modal connection edges whose semantic consistency meets a preset threshold, includes: Establish an alignment bias set that preserves the laser points and pixel points in the cross-modal bridging edge; The alignment bias set is used to make a group bias decision under the influence of distance radiation, wherein the influence of distance radiation is an effect that decays with distance length. The cross-modal bridge edge is updated based on the group bias decision.

6. The method for analyzing the safe distance in a transmission line external damage scenario based on multi-source data as described in claim 1, characterized in that, The method of constructing the spatial location of the damaged object and the transmission line using the hybrid conditional random field, and identifying the minimum distance from the damaged object to the transmission line, further includes: Obtain the fixed points and the lowest point of sag of the transmission line; Based on the fixed point and the lowest point of sag, a wind deflection galloping fitting simulation of the transmission line is performed, and the wind deflection galloping fitting simulation results are established. Using the simulation results of wind deflection and galloping, and the spatial position of the externally damaged object, the shortest distance is identified, and the shortest distance identification result is taken as the minimum distance.

7. The method for analyzing the safe distance in external damage scenarios of transmission lines based on multi-source data as described in claim 1, characterized in that, After performing the security warning trigger analysis based on the minimum distance, the method further includes: Perform continuous data acquisition on the location of transmission lines and establish a time-series dataset; The motion trend of externally damaged objects is fitted using the aforementioned time-series dataset to establish an evolution warning system.

8. The method for analyzing the safe distance in a transmission line external damage scenario based on multi-source data as described in claim 1, characterized in that, The analysis for triggering a safety warning based on the minimum distance includes: Obtain the voltage level of the transmission line and simultaneously obtain the object category of the externally damaged object; The warning level is calculated based on the voltage level, object type, and minimum distance to complete the safety warning trigger analysis.

9. The method for analyzing the safe distance in a transmission line external damage scenario based on multi-source data as described in claim 8, characterized in that, The calculation of the warning level based on the voltage level, object category, and minimum distance to complete the safety warning trigger analysis includes: A visual demonstration video will be created based on the object categories and power transmission lines. Early warnings will be issued based on the visual demonstration video and the warning level.

Images