Power transmission channel invader labeling method and device, equipment and storage medium

Abstract

The invention discloses a power transmission channel invader labeling method and device, equipment and a storage medium. The method comprises the steps of calculating a relation conversion matrix of environment point cloud data and environment picture data according to the collected environment point cloud data and environment picture data of a power transmission line; inputting the environment picture data into an invader detection model for image detection, and identifying target invader information; determining the shortest distance between the invader and the power transmission line based on the target invader information, the environmental point cloud data and the relation conversion matrix; and according to the environment point cloud data, the target intrusion information and the shortest distance between the invader and the power transmission line, labeling the invader in the environment picture data. According to the invention, the invading object in the surrounding environment of the power transmission line is detected, the shortest distance between the invading object and the power transmission line is calculated, and the condition of the invading object in the surrounding environment of the power transmission line can be accurately identified to protect the safety of the power transmission line.

Description

Technical Field

[0001] The present application relates to the field of intrusion detection technology, and more specifically, to a method, device, equipment and storage medium for marking intrusion objects in power transmission channels. Background Art

[0002] As an important part of the power system, the safety of transmission lines is of vital importance. In actual scenarios, there are often intrusions such as intruders around transmission lines, which may damage the transmission lines and cause safety accidents. Therefore, measuring and displaying the location of the intrusion and its distance to the transmission line is conducive to early warning of the staff, thereby avoiding accidents in time. LiDAR, as a laser ranging sensor, can quickly and accurately obtain the three-dimensional coordinate information of the environment, and based on this information, the distance from the intrusion to the transmission line can be calculated.

[0003] Existing methods for detecting intrusions on power lines are mainly divided into two types: methods based on single image data or point cloud data and methods based on deep learning. The methods based on single image data or point cloud data use radar devices to transmit radar waves, receive radar waves reflected by objects and analyze and process them. The methods based on deep learning collect image information within the range of power lines, capture image content features, analyze the application scenario type, determine the alarm level, upload the alarm signal to the client and on-site staff, and determine the upload path of the alarm signal based on the alarm level.

[0004] However, methods based on single image data or point cloud data only consider image information or point cloud information, which may be inaccurate or unintuitive, and not convenient for workers to observe on-site conditions. Methods based on deep learning use the acquired image information within the range of marked transmission lines for model training. Existing methods mostly use neural networks to extract features during detection, and detect intruder targets on certain layers of output. In addition, some methods use feature fusion to enhance the feature information of intruders, but these methods do not consider the importance of different features to the detection task well, so that the feature information of intruders will be overwhelmed by other unimportant feature information in the feature information extracted by the neural network. Summary of the invention

[0005] In response to at least one defect or improvement need in the prior art, the present invention provides a method, device, equipment and storage medium for marking intruders in power transmission channels, which are used to solve the problem that the prior art method for detecting intruders in power lines only considers image information or point cloud information, and does not consider the importance of different features for the detection task, so that the feature information of the intruder in the feature information extracted by the neural network will be submerged by other unimportant feature information, resulting in inaccurate or unintuitive detection results.

[0006] To achieve the above object, according to a first aspect of the present invention, a method for marking intrusions in a power transmission channel is provided, comprising:

[0007] Calculate the relationship conversion matrix between the environmental point cloud data and the environmental image data according to the collected environmental point cloud data and environmental image data of the transmission line;

[0008] Input the environmental image data into the intrusion detection model for image detection to identify the target intrusion information;

[0009] Determine the shortest distance between the intruder and the transmission line based on the target intrusion information, environmental point cloud data and relationship transformation matrix;

[0010] The intrusion objects are marked in the environmental image data according to the environmental point cloud data, the target intrusion information and the shortest distance between the intrusion object and the transmission line.

[0011] In a possible implementation, the target intrusion object information includes an intrusion object annotation box; determining the shortest distance between the intrusion object and the transmission line based on the target intrusion object information, the environmental point cloud data and the relationship transformation matrix also includes:

[0012] Determine the annotation box channel point cloud according to the intruder annotation box, environmental point cloud data and the relationship transformation matrix;

[0013] Perform Euclidean clustering and center focusing processing on the point cloud of the labeled frame channel to obtain several point cloud clusters in the labeled frame channel;

[0014] The shortest distance between the intruder and the transmission line is determined based on several point cloud clusters and environmental point cloud data.

[0015] In a possible implementation, the point cloud of the annotated frame channel is subjected to Euclidean clustering and center focusing processing to obtain a plurality of point cloud clusters within the annotated frame channel, and further includes:

[0016] Perform Euclidean clustering on the channel point cloud of the labeled box to obtain the channel point cloud cluster, and perform denoising on the channel point cloud cluster;

[0017] Calculate the geometric center of the annotation box channel point cloud and the geometric center of each denoised channel point cloud cluster respectively;

[0018] The channel point cloud clusters whose distance from the geometric center of the channel point cloud cluster to the geometric center of the annotation frame channel point cloud is greater than the preset distance are removed to obtain several point cloud clusters in the annotation frame channel.

[0019] In a possible implementation, determining the shortest distance between an intruder and a transmission line based on a plurality of point cloud clusters and environmental point cloud data further includes:

[0020] Remove the point clouds that overlap with the environment point cloud data in the point cloud clustering clusters;

[0021] Calculate the Euclidean distance between each point cloud cluster and the transmission line after deduplication.

[0022] The minimum point cloud Euclidean distance is recorded, and the point cloud of the intrusion endpoint and the point cloud of the transmission line endpoint corresponding to the minimum point cloud Euclidean distance are recorded.

[0023] In a possible implementation, the intrusion object is marked in the transmission line image data according to the transmission line point cloud data, the target intrusion information and the shortest distance between the intrusion object and the transmission line, and further includes:

[0024] Generate a discrete point cloud between the point cloud of the intrusion endpoint and the point cloud of the transmission line endpoint with a preset step size;

[0025] Project the discrete point cloud to the environment image data based on the relational transformation matrix;

[0026] Mark intrusion boxes and minimum point cloud Euclidean distance in transmission line image data.

[0027] In a possible implementation, generating a discrete point cloud between the point cloud of the intruder endpoint and the point cloud of the transmission line endpoint with a preset step size also includes:

[0028] Based on the fitting of the endpoint point cloud of the intruder and the endpoint point cloud of the transmission line, the endpoint point cloud straight line equation is obtained;

[0029] Determine the number of discrete point clouds to be generated according to the endpoint point cloud straight line equation, the minimum point cloud Euclidean distance and the preset step size;

[0030] The coordinates of the discrete point clouds are determined according to the number of discrete point clouds, the endpoint point clouds of the intruding objects, the endpoint point clouds of the transmission lines and the straight line equations of the endpoint point clouds.

[0031] In a possible implementation, calculating the relationship conversion matrix between the environmental point cloud data and the environmental image data according to the collected environmental point cloud data and environmental image data of the transmission line also includes:

[0032] Calibrate the camera based on the environmental image data to determine the camera's intrinsic parameters;

[0033] Set feature points in the environmental point cloud data and environmental image data of the same environment, and determine the external parameters between the laser radar and the camera based on the feature points;

[0034] The relationship transformation matrix between the environmental point cloud data and the environmental image data is calculated through the external parameters between the camera's internal laser radar and the camera.

[0035] According to a second aspect of the present invention, there is also provided a device for marking intrusions in a power transmission channel, comprising:

[0036] A conversion matrix module, which is configured to calculate a relationship conversion matrix between the environmental point cloud data and the environmental image data according to the environmental point cloud data and the environmental image data collected from the power transmission line;

[0037] An image recognition module is configured to input environmental image data into an intrusion detection model for image detection to identify target intrusion information;

[0038] A distance calculation module, which is configured to determine the shortest distance between the intruder and the transmission line based on the target intruder information, the environmental point cloud data and the relationship transformation matrix;

[0039] The intrusion marking module is configured to mark the intrusion object in the environmental image data according to the environmental point cloud data, the target intrusion information and the shortest distance between the intrusion object and the power line.

[0040] According to the third aspect of the present invention, there is also provided a device for marking intrusions in a power transmission channel, which includes at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the computer program is executed by the processing unit, the processing unit executes the steps of any one of the above-mentioned methods for marking intrusions in a power transmission channel.

[0041] According to a fourth aspect of the present invention, there is also provided a storage medium storing a computer program executable by an access authentication device, which, when the computer program runs on the access authentication device, enables the access authentication device to execute any of the steps of the above-mentioned method for marking intrusions in power transmission channels.

[0042] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0043] The present invention provides a method for marking intrusions in power transmission channels. By combining environmental point cloud data and environmental image data, it is possible to more accurately identify intrusions near power transmission lines, especially those objects whose distance or height is difficult to accurately judge in two-dimensional images. The establishment of a relationship conversion matrix enables the two data types to be seamlessly integrated, thereby improving the overall accuracy and efficiency of power transmission line intrusion detection. The real-time shortest distance between the intrusion and the power transmission line is calculated, and the intrusion and its shortest distance information to the power transmission line are directly marked in the environmental image data, so that management personnel can intuitively understand the on-site conditions of the environment around the power transmission line, quickly locate the problem area, and facilitate the adoption of countermeasures, thereby improving the emergency response speed and decision-making efficiency of intrusions around the power transmission line. This solution also reduces the reliance on manual inspections through automated and intelligent monitoring and labeling, reduces operation and maintenance costs, and improves the stability and reliability of power transmission line grid operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 A schematic diagram of a flow chart of an embodiment of a method for marking intrusions in a power transmission channel provided by the present invention;

[0046] Figure 2 The present invention provides Figure 1 A flowchart of an embodiment of step S300;

[0047] Figure 3 The present invention provides Figure 2 A flowchart of an embodiment of step S320;

[0048] Figure 4 A corresponding diagram of the relationship between an image coordinate system and a point cloud coordinate system according to an embodiment of the present invention;

[0049] Figure 5 The present invention provides Figure 2 A flowchart of an embodiment of step S330;

[0050] Figure 6 The present invention provides Figure 1 A flowchart of an embodiment of step S400;

[0051] Figure 7 The present invention provides Figure 6 A schematic diagram of a flow chart of an embodiment of step S410;

[0052] Figure 8 The present invention provides Figure 1 A schematic flow chart of an embodiment of step S100;

[0053] Fig. 9 A schematic structural diagram of an embodiment of a device for marking intrusions in a power transmission channel provided by the present invention;

[0054] Fig.10 A schematic diagram of the structure of a device for marking intrusions in a power transmission channel provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0056] The terms "first", "second", "third", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0057] The present invention provides a method, device, equipment and storage medium for marking intruders in a power transmission channel, which are described below respectively.

[0058] See also Figure 1 , Figure 1 A flow chart of an embodiment of a method for marking an intrusion in a power transmission channel provided by the present invention. In a specific embodiment of the present invention, a method for marking an intrusion in a power transmission channel is disclosed, comprising:

[0059] S100, calculating a relationship conversion matrix between the environmental point cloud data and the environmental image data according to the collected environmental point cloud data and environmental image data of the transmission line;

[0060] S200, inputting the environmental image data into the intrusion detection model to perform image detection and identify the target intrusion information;

[0061] S300, determining the shortest distance between the intruder and the transmission line based on the target intruder information, the environmental point cloud data and the relationship conversion matrix;

[0062] S400: marking the intrusion object in the environmental image data according to the environmental point cloud data, the target intrusion information and the shortest distance between the intrusion object and the transmission line.

[0063] In the above embodiment, environmental point cloud data can be collected using LiDAR, and environmental image data can be collected using a high-definition camera. It should be noted that for the same environment, at least one set of environmental point cloud data and one set of environmental image data are required respectively, and then the correspondence between the environmental point cloud data and the environmental image data is calculated based on the environmental point cloud data and the environmental image data under the same environment.

[0064] It is necessary to preprocess the environmental image data to avoid interference with the identification of intruders. The preprocessed environmental image data is input into a pre-trained intruder detection model. The model is built based on deep learning technology (such as convolutional neural network CNN) and can automatically identify the target intruder information in the image. The detection results usually include the location of the intruder (given in the form of a marked box. As a preferred embodiment, the marked box in the present invention is a rectangle, and the pixel coordinate values ​​of the four vertices of the marked box need to be given), category, and possible confidence level, etc.

[0065] After obtaining the target intrusion information and environmental point cloud data, the relational transformation matrix is ​​used to map the intrusion annotation box in the image to the three-dimensional point cloud space to form a three-dimensional annotation box channel. The corresponding point cloud data is screened in the annotation box channel. These point cloud data represent the position and shape of the intrusion in the three-dimensional space. The point cloud data of the transmission line is identified from the environmental point cloud data, and the geometric algorithm in the three-dimensional space (such as the point-to-line distance formula, the nearest point query, etc.) is used to calculate the shortest distance between the intrusion point cloud (or its representative point, such as the cluster center) and the transmission line.

[0066] It should be noted that the location of the intrusion is marked on the picture with eye-catching colors and shapes, and different intrusions are distinguished. The specific value of the shortest distance is displayed next to it and the shortest distance line between the two is marked. A text description is added next to the annotation box to indicate the type of intrusion, the probability of identification, and whether emergency measures need to be taken. The annotated picture is displayed to the management personnel or the monitoring system through the user interface so that they can quickly understand the on-site situation and make corresponding decisions.

[0067] Compared with the prior art, the method for marking intrusions in a power transmission channel provided in this embodiment can more accurately identify intrusions near the power transmission line, especially those objects whose distance or height is difficult to accurately judge in a two-dimensional image, by combining environmental point cloud data and environmental image data. The establishment of a relationship conversion matrix enables the two data types to be seamlessly integrated, thereby improving the overall accuracy and efficiency of intrusion detection on the power transmission line. The real-time shortest distance between the intrusion and the power transmission line is calculated, and the intrusion and its shortest distance information with the power transmission line are directly marked in the environmental image data, so that the management personnel can intuitively understand the on-site conditions of the environment around the power transmission line, quickly locate the problem area, and facilitate the adoption of countermeasures, thereby improving the emergency response speed and decision-making efficiency of intrusions around the power transmission line. This solution also reduces the reliance on manual inspections through automated and intelligent monitoring and labeling, reduces operation and maintenance costs, and improves the stability and reliability of power transmission line grid operation.

[0068] See also Figure 2 , Figure 2 The present invention provides Figure 1 In some embodiments of the present invention, the target intrusion information includes an intrusion annotation box; determining the shortest distance between the intrusion and the transmission line based on the target intrusion information, the environmental point cloud data and the relationship conversion matrix also includes:

[0069] S310, determining a labeling frame channel point cloud according to the intruder labeling frame, the environmental point cloud data and the relationship conversion matrix;

[0070] S320, performing Euclidean clustering and center focusing processing on the point cloud of the annotation frame channel to obtain a plurality of point cloud clusters within the annotation frame channel;

[0071] S330: Determine the shortest distance between the intruder and the transmission line based on a plurality of point cloud clusters and environmental point cloud data.

[0072] In the above embodiment, spatial alignment is performed based on the pixel coordinates of the four vertices of the annotation box of the intruder and the three-dimensional point cloud data of the environment. The pixel coordinates of the four vertices of the annotation box of the intruder are converted from the image coordinate system to the three-dimensional space coordinate system through a relational transformation matrix (usually including transformation parameters such as translation and rotation, which are used to unify different coordinate systems), thereby determining a three-dimensional "annotation box channel", which is an extended area around the intruder annotation box in the three-dimensional space, and is used for subsequent point cloud screening and processing.

[0073] The Euclidean distance clustering analysis of the point cloud data in the annotation box channel needs to calculate the Euclidean distance between each point in the point cloud, and classify the points with similar distances into the same cluster. In the clustering process, an appropriate distance threshold can be set to distinguish different objects or different parts of objects, effectively reducing the impact of noise and outliers on the results. In each cluster, its center point (such as the centroid) or representative point is calculated. These points will serve as a generalized representation of the cluster. By focusing on the center, the data can be further simplified, which is convenient for subsequent distance calculation and analysis. In addition, for large or complex clusters, shape analysis or feature extraction is also required to more accurately reflect its spatial form.

[0074] According to the specific characteristics of the transmission line (such as linear structure, height range, material reflection characteristics, etc.) and the existing geographic information or prior knowledge, the point cloud data of the transmission line is identified from the overall environmental point cloud data. After determining the point cloud clusters of the intrusion and the point cloud of the transmission line, the geometric calculation methods in three-dimensional space (such as the nearest point query, the least squares fitting straight line and the point-to-straight line distance formula, etc.) are used to calculate the shortest distance from the center point of each cluster to the transmission line. Considering that the transmission line may be composed of multiple parallel lines, it is necessary to traverse all possible transmission line segments to find the global shortest distance.

[0075] It should be noted that according to the calculated shortest distance and in combination with the safety distance standard of transmission facilities, the potential threat level of the intrusion object to the transmission line is evaluated. The video is recorded in real time by the camera and framed, and at the same time, the point cloud is collected in real time by the radar and framed. After frame fusion, it is displayed, and the moving process of the intrusion target in the transmission channel scene and the change process of the distance from the conductor can be played in real time. The warning level is divided in stages for different distances. A feasible division scheme is as follows:

[0076] L > 15m (no warning), 10m < L ≤ 15m (P4 level), 7m < L ≤ 10m (P3 level), 4m < L ≤ 7m (P2 level), 1m < L ≤ 4m (P1 level), L < 1m (P0 level, red warning).

[0077] Users can choose to view the results they want, and according to the importance of the data obtained from the warning levels at different distances, intrusion objects that are too close to the transmission line can be detected and warned in a timely manner, thus ensuring the safety of the power transmission system.

[0078] Please refer to Figure 3 , Figure 3 for an embodiment flow diagram of step S320 provided by the present invention. In some embodiments of the present invention, after performing Euclidean clustering and center focusing processing on the labeled box channel point cloud to obtain several point cloud clustering clusters in the labeled box channel, it further includes: Figure 2

[0079] S321. Perform Euclidean clustering processing on the labeled box channel point cloud to obtain channel point cloud clusters, and perform denoising processing on the channel point cloud clusters;

[0080] S322. Calculate the geometric center of the labeled box channel point cloud and the geometric center of each denoised channel point cloud cluster respectively;

[0081] S323. Remove the channel point cloud clusters whose distance from the geometric center of the channel point cloud cluster to the geometric center of the labeled box channel point cloud is greater than a preset distance, and obtain several point cloud clustering clusters in the labeled box channel.

[0082] In the above embodiments, a coordinate system needs to be established for both the environmental image data and the environmental point cloud data to describe the spatial positions of the pixel points or point clouds therein. Please refer to Figure 4 , Figure 4 for a corresponding relationship diagram of an embodiment of the image coordinate system and the point cloud coordinate system provided by the present invention. X′O′Y′ is the image coordinate system, and OXYZ is the point cloud coordinate system. After mapping according to the external parameters, the two-dimensional rectangular box can form the entire channel in the three-dimensional point cloud, and the X coordinate range of the point cloud in the channel is (0, ∞).

[0083] ​In order to reduce the amount of calculation and improve the real-time performance of the system, the point cloud data in the annotation box is divided into multiple independent clusters by setting a threshold distance, that is, if the distance between two points is less than this threshold, they are considered to be part of the same cluster. After clustering, each cluster may still contain some abnormal points or noise points, which may be caused by measurement errors, occlusions or other environmental factors. The denoising process aims to remove these points and improve the accuracy and efficiency of subsequent processing. In an embodiment of the present invention, the denoising method can be statistical filtering, radius filtering, or a method based on machine learning.

[0084] The average position of all point cloud data in the annotation box is calculated as the overall geometric center. The coordinates of all points in the point cloud are weighted averaged to obtain the geometric center of the overall channel point cloud. Then, for each denoised cluster, the average position of all points inside is calculated to obtain their respective geometric centers, and then the average distance from all cluster centers to the overall channel center is obtained. As a preferred embodiment, the preset distance in the present invention is the average distance from all cluster centers to the overall channel center. Clusters whose distance from the overall channel center is greater than the average distance are removed to achieve center focusing on the overall channel point cloud.

[0085] See also Figure 5 , Figure 5 The present invention provides Figure 2 In some embodiments of the present invention, determining the shortest distance between the intruder and the transmission line based on a plurality of point cloud clusters and environmental point cloud data further includes:

[0086] S331, removing the point cloud that overlaps with the environmental point cloud data in the point cloud cluster overlap;

[0087] S332, respectively calculating the point cloud Euclidean distance from each point cloud cluster to the transmission line after deduplication;

[0088] S333. Record the minimum point cloud Euclidean distance, and record the intrusion object endpoint point cloud and the transmission line endpoint point cloud corresponding to the minimum point cloud Euclidean distance.

[0089] In the above embodiment, the acquired point cloud data is filtered, and voxel grid downsampling can be used to reduce the amount of data while retaining the main features to reduce the computational complexity. The three-dimensional model is constructed using environmental point cloud data, and the spatial search efficiency can be optimized through data structures such as spatial partitioning (such as octrees) and KD trees. For each point cloud cluster, all points in it are traversed, and the environmental model is used to check whether these points are within a preset safety distance range or highly overlap with known environmental structures (such as the ground, buildings, trees, etc.). Spatial indexing is used to accelerate the query process, and only possible intersecting point cloud areas are compared. For the detected overlapping point clouds, choose to directly delete or mark them as invalid according to the actual application requirements so that they can be ignored in subsequent analysis.

[0090] Through curve fitting, least squares method and other methods, based on the characteristic points in the point cloud data, such as the location of the tower and the cable hanging form, an accurate model of the transmission line is constructed according to the characteristics of the transmission line. For each point cloud cluster after deduplication, the minimum Euclidean distance from each point inside it to the transmission line model is calculated. The projection method can be used, that is, the points in the point cloud are projected onto the transmission line, and the distance between the projection point and the original point is calculated. To speed up the calculation process, it is possible to consider dividing the transmission line model into multiple segments and constructing a suitable approximate expression (such as line segments, arcs, etc.) for each segment. Then, the points in each point cloud cluster are calculated in a segmented manner, and the closest line segment or arc segment is selected for distance calculation.

[0091] After traversing all point cloud clusters and completing the distance calculation, find and record the minimum Euclidean distance value. At the same time, it is necessary to record the specific point (or point set, if the distance is very close, it may be an area) in the intruder cluster when the minimum distance is reached as the endpoint point cloud of the intruder, and the corresponding nearest point on the transmission line as the endpoint point cloud of the transmission line.

[0092] See also Figure 6 , Figure 6 The present invention provides Figure 1 In some embodiments of the present invention, the intrusion object is marked in the transmission line image data according to the transmission line point cloud data, the target intrusion information and the shortest distance between the intrusion object and the transmission line, and further includes:

[0093] S410, generating a discrete point cloud between the point cloud of the intrusion object endpoint and the point cloud of the transmission line endpoint with a preset step size;

[0094] S420, projecting the discrete point cloud onto the environment image data based on the relationship transformation matrix;

[0095] S430, marking the intrusion object labeling box and the minimum point cloud Euclidean distance in the transmission line image data.

[0096] In the above embodiment, the straight line between the two points is determined based on the point cloud of the intruder endpoint and the point cloud of the transmission line endpoint corresponding to the minimum distance. According to actual needs and image resolution, a suitable step size (such as millimeter level or pixel level) is set to generate a discrete point cloud on the line. The step size should be selected to ensure that the generated discrete point cloud can accurately reflect the shape of the line and will not increase unnecessary calculations due to being too dense. Along the line, a series of discrete points are generated at intervals of a preset step size to form a discrete point cloud. These discrete points will be used for subsequent projection and annotation on the image.

[0097] Each point in the discrete point cloud generated in step S410 is projected onto the two-dimensional plane of the environmental image data through the relational transformation matrix. It may be necessary to process factors such as perspective distortion and lens distortion to ensure the accuracy of the projection. The projected two-dimensional coordinates are converted into pixel coordinates on the image to prepare for subsequent annotation.

[0098] The minimum point cloud Euclidean distance value is displayed at a suitable position next to the annotation box on the image. In order to enhance readability, different colors, fonts or icons can be used to highlight this information. The annotated image is integrated with relevant data (such as the type of intruder, timestamp, location information, etc.) to form a complete monitoring report or early warning information, which can be used for subsequent analysis, processing or archiving.

[0099] See also Figure 7 , Figure 7 The present invention provides Figure 6 In some embodiments of the present invention, generating a discrete point cloud between the point cloud of the intruder endpoint and the point cloud of the power line endpoint with a preset step size also includes:

[0100] S411, obtaining the endpoint point cloud straight line equation based on the fitting of the endpoint point cloud of the intruder and the endpoint point cloud of the transmission line;

[0101] S412, determining the number of discrete point clouds to be generated according to the endpoint point cloud straight line equation, the minimum point cloud Euclidean distance and the preset step size;

[0102] S413, determining the coordinates of the discrete point clouds according to the number of discrete point clouds, the endpoint point clouds of the intruding object, the endpoint point clouds of the transmission line, and the straight line equations of the endpoint point clouds.

[0103] In the above embodiment, it is assumed that the two points are P 1 (x 1 ,y 1 ,z 1 ), P 2 (x 2 ,y 2 ,z 2). The equation of the straight line is fitted based on the coordinates of the two points as follows:

[0104]

[0105] Where t is a parameter, which takes all real values. The equation represents the 1 To P 2 The direction of the point (x, y, z) is determined by t. d is the preset step size. 1 and P 2 A number of discrete point clouds are generated with intervals of d, and a total of n points need to be generated. In the point cloud coordinate system, a line segment S consisting of n point clouds is obtained, and the length of the line segment S is the closest distance L between the intruding target object and the wire. The point cloud line segment S with a length of L is mapped to the two-dimensional image according to the internal and external parameters obtained in step 1 to obtain the pixel points after the point cloud projection, and a line segment intersecting with the target rectangular frame can be obtained in the two-dimensional image. One end of the line segment points to the top of the target vehicle, and the other end points to the wire.

[0106] Create an empty list or array to store the coordinates of discrete points. 1 Start by converting the parametric form of the line equation into an incremental form or using vector operations, moving along the direction of the line equation, each time with a step length d, calculating the coordinates of the new discrete point and adding it to the list. For example, in two-dimensional space, you can first calculate the unit direction vector, then use this vector and the step length to calculate the new coordinate point, and stop the calculation when the number of generated discrete points reaches n, and the last point will be close to or exactly point P. 2 .

[0107] See also Figure 8 , Figure 8 The present invention provides Figure 1 In some embodiments of the present invention, the relationship conversion matrix between the environmental point cloud data and the environmental image data is calculated based on the collected environmental point cloud data and environmental image data of the transmission line, and further includes:

[0108] S110, calibrating the camera based on the environmental image data to determine the camera internal parameters;

[0109] S120, setting feature points in the environmental point cloud data and environmental image data of the same environment, and determining the external parameters between the laser radar and the camera according to the feature points;

[0110] S130, calculating the relationship conversion matrix between the environmental point cloud data and the environmental image data through the external parameters between the camera's internal laser radar and the camera.

[0111] In the above embodiment, a calibration plate with known geometric features (such as a checkerboard calibration plate) is used, and the position and spacing of the feature points on the calibration plate are precisely known. The calibration plate is placed in the field of view of the camera, and multiple pictures are taken from different angles and distances to ensure that the entire field of view of the camera is covered. Image processing technology (such as corner detection) is used to automatically extract the feature points on the calibration plate from each calibration picture. Using the extracted feature point coordinates and the known coordinates on the calibration plate, the internal parameters of the camera (focal length, optical center coordinates, distortion coefficient, etc.) are calculated through optimization algorithms such as minimizing reprojection errors.

[0112] Make sure that the lidar and camera scan and shoot the same environment in the same time period so that their data have overlapping areas. Select obvious and easily identifiable feature points (such as wall corners, ground markings, special signs, etc.) in the overlapping area. These feature points should be clearly identifiable in both the environmental point cloud data and the environmental image data. Use feature matching algorithms (such as SIFT, SURF, ORB, etc.) to identify and match the corresponding feature points in the environmental point cloud data in the environmental image data. Based on the matched feature point pairs, use the iterative closest point (ICP) algorithm or other optimization methods to calculate the transformation matrix (i.e., external parameters) from the lidar coordinate system to the camera coordinate system. This matrix describes the relative position and posture between the lidar and the camera.

[0113] The camera intrinsic parameter matrix and the extrinsic parameter matrix between the lidar and the camera are combined to form a complete transformation matrix, which can transform the points in the environmental point cloud data into the camera coordinate system, and then project the three-dimensional points onto the two-dimensional image plane through the camera's projection model. Use a set of known three-dimensional points (such as corner points on a calibration plate) for testing, project these points onto the image plane through the transformation matrix, and compare them with the actual feature points on the image to verify the accuracy of the transformation matrix. Once the accuracy of the transformation matrix is ​​verified, it can be applied to the entire environmental point cloud data, transforming each point in the point cloud into the camera coordinate system and projecting it onto the environmental image, thereby achieving accurate correspondence and fusion between the point cloud and the image.

[0114] In order to better implement the method for marking intrusions in the power transmission channel in the embodiment of the present invention, based on the method for marking intrusions in the power transmission channel, correspondingly, please refer to Fig. 9 , Fig. 9 This is a schematic diagram of the structure of an embodiment of a device for marking intrusions in a power transmission channel provided by the present invention. The embodiment of the present invention provides a device 900 for marking intrusions in a power transmission channel, including:

[0115] A conversion matrix module 910, which is configured to calculate a relationship conversion matrix between the environmental point cloud data and the environmental image data according to the collected environmental point cloud data and environmental image data of the power transmission line;

[0116] An image recognition module 920 is configured to input the environmental image data into the intrusion detection model for image detection and identify target intrusion information;

[0117] A distance calculation module 930, which is configured to determine the shortest distance between the intruder and the transmission line based on the target intruder information, the environmental point cloud data and the relationship transformation matrix;

[0118] The intrusion marking module 940 is configured to mark the intrusion object in the environmental image data according to the environmental point cloud data, the target intrusion information and the shortest distance between the intrusion object and the power line.

[0119] It should be noted here that: the device 900 provided in the above embodiment can implement the technical solutions described in the above method embodiments. The specific implementation principles of the above modules or units can refer to the corresponding contents in the above method embodiments, which will not be repeated here.

[0120] See also Fig.10 , Fig.10 A schematic diagram of the structure of a transmission channel intrusion marking device provided by an embodiment of the present invention. Based on the above-mentioned transmission channel intrusion marking method, the present invention also provides a transmission channel intrusion marking device, which can be a computing device such as a mobile terminal, a desktop computer, a notebook, a PDA, and a server. The transmission channel intrusion marking device 1000 includes a processor 1010, a memory 1020, and a display 1030. Fig.10 Only some components of the synchronous tracking flying welding equipment for real-time measurement of battery height are shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

[0121] In some embodiments, the memory 1020 may be an internal storage unit of the power transmission channel intrusion marking device 1000, such as a hard disk or memory of the power transmission channel intrusion marking device 1000. In other embodiments, the memory 1020 may also be an external storage device of the power transmission channel intrusion marking device 1000, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the power transmission channel intrusion marking device 1000. Further, the memory 1020 may also include both the internal storage unit of the power transmission channel intrusion marking device 1000 and an external storage device. The memory 1020 is used to store application software and various types of data installed in the power transmission channel intrusion marking device 1000, such as the program code of the power transmission channel intrusion marking device 1000. The memory 1020 can also be used to temporarily store data that has been output or is to be output. In one embodiment, a power transmission channel intrusion marking program 1040 is stored in the memory 1020 , and the power transmission channel intrusion marking program 1040 can be executed by the processor 1010 , thereby implementing the power transmission channel intrusion marking method of each embodiment of the present application.

[0122] In some embodiments, the processor 1010 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run program codes or process data stored in the memory 1020, such as executing a method for marking intruders in power transmission channels.

[0123] In some embodiments, the display 1030 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 1030 is used to display information of the device 1000 for marking intrusions in the power transmission channel and to display a visual user interface. The components 1010-1030 of the device 1000 for marking intrusions in the power transmission channel communicate with each other via a system bus.

[0124] In one embodiment, when the processor 1010 executes the power transmission channel intrusion marking program 1040 in the memory 1020 , the steps in the power transmission channel intrusion marking method described above are implemented.

[0125] This embodiment further provides a computer-readable storage medium, on which a program for marking intrusions into a power transmission channel is stored. When the program for marking intrusions into a power transmission channel is executed by a processor, the following steps are implemented:

[0126] Calculate the relationship conversion matrix between the environmental point cloud data and the environmental image data according to the collected environmental point cloud data and environmental image data of the transmission line;

[0127] Input the environmental image data into the intrusion detection model for image detection to identify the target intrusion information;

[0128] Determine the shortest distance between the intruder and the transmission line based on the target intrusion information, environmental point cloud data and relationship transformation matrix;

[0129] The intrusion objects are marked in the environmental image data according to the environmental point cloud data, the target intrusion information and the shortest distance between the intrusion object and the transmission line.

[0130] In summary, a method for labeling intrusions in power transmission channels can more accurately identify intrusions near power transmission lines, especially those objects whose distance or height is difficult to accurately judge in two-dimensional images, by combining environmental point cloud data and environmental image data. The establishment of the relationship conversion matrix enables the two data types to be seamlessly integrated, thereby improving the overall accuracy and efficiency of intrusion detection on power transmission lines. The real-time shortest distance between the intrusion and the power transmission line is calculated, and the intrusion and its shortest distance information with the power transmission line are directly labeled in the environmental image data, so that managers can intuitively understand the on-site situation of the environment around the power transmission line, quickly locate the problem area, and facilitate the adoption of countermeasures, thereby improving the emergency response speed and decision-making efficiency of intrusions around the power transmission line. This solution also reduces the reliance on manual inspections through automated and intelligent monitoring and labeling, reduces operation and maintenance costs, and improves the stability and reliability of power transmission line grid operation.

[0131] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of the above method are implemented. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a micro drive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.

[0132] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0133] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0134] In the several embodiments provided in the present application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0135] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0136] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0137] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, disk or optical disk and other media that can store program code.

[0138] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, which can include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0139] The above is only an exemplary embodiment of the present disclosure, and the scope of the present disclosure cannot be limited thereto. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the specification and practicing the disclosure here, those skilled in the art will easily think of the implementation scheme of the present disclosure. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the technical field not recorded in the present disclosure. The description and examples are regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

[0140] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0141] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for marking intrusions in a power transmission channel, characterized in that: include: Calculate the relationship conversion matrix between the environmental point cloud data and the environmental image data according to the collected environmental point cloud data and environmental image data of the power transmission line; Inputting the environmental image data into an intrusion detection model for image detection to identify target intrusion information; Determine the shortest distance between the intruder and the transmission line based on the target intruder information, the environmental point cloud data and the relationship transformation matrix; The intruding object is marked in the environmental image data according to the environmental point cloud data, the target intrusion information and the shortest distance between the intruding object and the power line.

2. The method for marking intrusions in a power transmission channel according to claim 1, characterized in that: The target intrusion information includes an intrusion marking frame; The determining the shortest distance between the intruder and the transmission line based on the target intruder information, the environmental point cloud data and the relationship conversion matrix further includes: Determine a labeling frame channel point cloud according to the intruder labeling frame, the environmental point cloud data and the relationship conversion matrix; Performing Euclidean clustering and center focusing processing on the point cloud of the annotation frame channel to obtain a plurality of point cloud clusters within the annotation frame channel; The shortest distance between the intruder and the power transmission line is determined based on the plurality of point cloud clusters and the environmental point cloud data.

3. The method for marking intrusions in a power transmission channel according to claim 2, characterized in that: The step of performing Euclidean clustering and center focusing processing on the point cloud of the annotation frame channel to obtain a plurality of point cloud clusters in the annotation frame channel further includes: Performing Euclidean clustering on the channel point cloud of the annotation box to obtain a channel point cloud cluster, and performing denoising on the channel point cloud cluster; Calculating the geometric center of the channel point cloud of the annotation box and the geometric center of each denoised channel point cloud cluster respectively; The channel point cloud clusters whose distance from the geometric center of the channel point cloud cluster to the geometric center of the marked frame channel point cloud is greater than a preset distance are removed to obtain a plurality of point cloud clusters in the marked frame channel.

4. The method for marking intrusions in a power transmission channel according to claim 2, characterized in that: The determining the shortest distance between the intruder and the transmission line based on the plurality of point cloud clusters and the environmental point cloud data further includes: Removing point clouds that overlap with the environmental point cloud data from the point cloud clusters; Calculate the Euclidean distance between each point cloud cluster and the transmission line after deduplication. The minimum point cloud Euclidean distance is recorded, and the point cloud of the intrusion endpoint and the point cloud of the transmission line endpoint corresponding to the minimum point cloud Euclidean distance are recorded.

5. The method for marking intrusions in a power transmission channel according to claim 4, characterized in that: The step of marking the intruder in the transmission line image data according to the transmission line point cloud data, the target intrusion information and the shortest distance between the intruder and the transmission line also includes: Generate a discrete point cloud between the point cloud of the endpoint of the intruder and the point cloud of the endpoint of the transmission line with a preset step size; Projecting the discrete point cloud onto the environment image data based on the relationship transformation matrix; The intruder annotation box and the minimum point cloud Euclidean distance are marked in the transmission line image data.

6. The method for marking intrusions in a power transmission channel according to claim 5, characterized in that: The step of generating a discrete point cloud between the point cloud of the intrusion endpoint and the point cloud of the transmission line endpoint with a preset step size also includes: Based on the point cloud of the intruder endpoint and the point cloud of the transmission line endpoint, a straight line equation of the endpoint point cloud is obtained by fitting; Determine the number of discrete point clouds to be generated according to the endpoint point cloud straight line equation, the minimum point cloud Euclidean distance and the preset step size; The coordinates of the discrete point clouds are determined according to the number of the discrete point clouds, the endpoint point clouds of the intruding object, the endpoint point clouds of the transmission line and the straight line equations of the endpoint point clouds.

7. The method for marking intrusions in a power transmission channel according to claim 1, characterized in that: The step of calculating the relationship conversion matrix between the environmental point cloud data and the environmental image data according to the collected environmental point cloud data and environmental image data of the power transmission line also includes: Calibrate the camera based on the environmental image data to determine the camera intrinsic parameters; Setting feature points in environmental point cloud data and environmental image data of the same environment, and determining external parameters between the laser radar and the camera according to the feature points; The relationship conversion matrix between the environmental point cloud data and the environmental image data is calculated by using the external parameters between the laser radar and the camera through the internal parameters of the camera.

8. A device for marking intrusions in power transmission channels, characterized in that: include: A conversion matrix module, which is configured to calculate a relationship conversion matrix between the environmental point cloud data and the environmental image data according to the collected environmental point cloud data and environmental image data of the power transmission line; An image recognition module is configured to input the environmental image data into an intrusion detection model for image detection to identify target intrusion information; A distance calculation module, configured to determine the shortest distance between the intruder and the transmission line based on the target intruder information, the environmental point cloud data and the relationship transformation matrix; The intrusion marking module is configured to mark the intrusion object in the environmental image data according to the environmental point cloud data, the target intrusion information and the shortest distance between the intrusion object and the power line.

9. A device for marking intrusions in power transmission channels, characterized in that: It comprises at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the computer program is executed by the processing unit, the processing unit executes the steps of the method for marking intrusions in a power transmission channel as claimed in any one of claims 1 to 7.

10. A storage medium, characterized in that: It stores a computer program executable by an access authentication device. When the computer program runs on the access authentication device, the access authentication device executes the steps of the method for marking intruders in a power transmission channel as described in any one of claims 1 to 7.

Images