Methods, devices, electronic equipment, and storage media for generating protection zones of power transmission channels and locating potential mechanical damage hazards.

Abstract

This invention belongs to the technical field of intelligent operation and maintenance of power transmission lines, specifically relating to a method and device for generating protection zones and locating potential mechanical damage hazards in power transmission channels. The method includes: acquiring image data and point cloud data of the power transmission channel; registering the image data and point cloud data to obtain a depth map and coordinate system transformation parameters; acquiring protection zone parameters based on the point cloud data; determining the range of the power transmission channel protection zone by combining the protection zone parameters and a range expansion threshold; acquiring target bounding box information of potential mechanical damage hazards based on the image data using a target detection algorithm; obtaining the position of the hazard target in the local spatial coordinate system based on the target bounding box information and the depth map, and then obtaining its position in the world coordinate system; finally, visually labeling the protection zone and the hazard target position based on the range of the power transmission channel protection zone and the position of the hazard target in the world coordinate system. This invention solves the problems of low efficiency, low safety, and high cost associated with traditional manual inspections.

Description

Technical Field

[0001] This invention belongs to the technical field of intelligent operation and maintenance of power transmission lines, and more specifically, relates to a method, device, electronic equipment and storage medium for generating protection zones of power transmission channels and locating potential mechanical damage hazards. Background Technology

[0002] In this crucial area of ​​power transmission, the safety and reliability of transmission channels are paramount. However, because transmission channels traverse diverse geographical environments, they are susceptible to various impacts, including natural disasters and human-caused damage, leading to frequent hidden dangers that seriously threaten the safe operation of the power system. Traditional manual inspections are not only inefficient and unable to meet the ever-increasing safety demands of transmission lines, but also incur high costs.

[0003] Therefore, a pressing issue is how to achieve automated and intelligent inspections to effectively monitor the status of power transmission channels, promptly detect potential hazards, and accurately determine the severity and location of these hazards. With the rapid development of computer vision technology, integrating image-based target detection technology with spatial positioning technology has become a solution. This integration enables effective alarm and accurate location of hazards, significantly reducing patrol and investigation time.

[0004] Chinese patent document CN116545122A discloses a device and method for monitoring external damage to power transmission lines, comprising: a camera for capturing monitoring images of the power transmission line and its surrounding environment; a point cloud processing module for calculating the plane containing the conductor in the power transmission line using plane equations based on the point cloud data; a data fusion module for fusing the monitoring images and point cloud data; a positioning and ranging module for identifying construction machinery from the monitoring images and determining the position of the construction machinery based on the fused point cloud data; and determining the distance between the construction machinery and the conductor based on the plane containing the conductor and the position of the construction machinery. This invention achieves precise positioning of the construction machinery by registering the point cloud data and the monitoring images, thereby realizing quantitative distance measurement between the construction machinery and the conductor, and thus enabling monitoring of external damage to power transmission lines.

[0005] Chinese patent document CN115561769A discloses a power transmission line tree obstruction identification system based on laser point cloud and image data fusion. The system includes: a central processing unit; a detection system for collecting data on paths or obstacles on the power transmission line; a data acquisition system for sampling data on tree obstructions and the power transmission line; a positioning system for locating the power transmission line and facilitating data fusion with the sampling data; an adjustment system for adjusting the system after tree obstruction identification; a hazard identification system for monitoring the environment surrounding the power transmission line; and a clearing system for clearing obstructions. This invention calculates the distance between the power transmission line and trees along the passageway based on image information, eliminating the need to carry numerous instruments and effectively improving the convenience of inspection personnel conducting tree obstruction inspections along power transmission line passageways.

[0006] In view of this, the present invention designs a method for generating protection zones and locating mechanical damage hazards in power transmission channels. This method enables real-time and accurate acquisition of the coordinates of mechanical damage hazards in power transmission channels. By delineating protection zones below the line, it can effectively assist maintenance personnel in judging the level of hazards, greatly improving work efficiency. Summary of the Invention

[0007] The present invention aims to overcome at least one of the defects of the prior art and provide a method for generating protection zones and locating potential mechanical damage hazards in power transmission channels.

[0008] The present invention also discloses an apparatus equipped with a method for generating a protection zone for power transmission channels and locating potential mechanical damage hazards.

[0009] The detailed technical solution of this invention is as follows:

[0010] A method for generating protection zones and locating potential mechanical damage hazards in power transmission channels, the method comprising:

[0011] S1. Acquire image data and point cloud data of the power transmission channel, and register the image data and point cloud data to obtain the registered depth map and the coordinate system transformation parameters generated during the registration process;

[0012] S2. Obtain the protection zone parameters based on the point cloud data, and determine the range of the power transmission channel protection zone by combining the protection zone parameters and the given range expansion threshold, and combine the coordinate system transformation parameters, protection zone parameters and depth map to construct a positioning model;

[0013] S3. Based on the image data, use a target detection algorithm to obtain the target box information of the mechanical external damage hazard, determine the bottom center point of the hazard according to the target box information of the mechanical external damage hazard, and obtain the depth value of the bottom center point of the hazard in the image coordinate system in combination with the depth map, and obtain the position of the mechanical external damage hazard target in the local spatial coordinate system according to the depth value and the coordinate system transformation parameters.

[0014] S4. Based on the position of the mechanical external damage hazard target in the local spatial coordinate system and the coordinate system transformation parameters, obtain the position of the mechanical external damage hazard target in the world coordinate system;

[0015] S5. Based on the range of the power transmission channel protection zone and the position of the mechanical external damage hazard target in the world coordinate system, the power transmission channel protection zone and the mechanical external damage hazard target position are visually marked respectively.

[0016] According to a preferred embodiment of the present invention, in step S1, a camera is used to acquire image data of the power transmission channel. The camera is fixedly installed on any tower used to mount the power transmission conductor. The image data is an RGB two-dimensional image. It means, and ;

[0017] Point cloud data of the power transmission channel is acquired using a drone equipped with a lidar system. The point cloud data consists of n three-dimensional arrays. It means, and , .

[0018] According to a preferred embodiment of the present invention, in step S1, before registering the image data and the point cloud data, the point cloud data is first preprocessed, specifically including:

[0019] The point cloud data is filtered using a cloth filtering algorithm, wherein the grid size of the cloth filtering algorithm is 0.5m, the number of filtering iterations is 500, and the classification threshold is 0.2m;

[0020] Based on the image data of the transmission channel, two towers of the overhead conductor to be segmented are selected. The filtered point cloud data is then segmented according to the coordinates of the two towers in the point cloud data to obtain point cloud data that matches the image data. The coordinates of the two towers are as follows: and And the coordinates of the tower where the camera is located are ;

[0021] The point cloud data is transformed from the world coordinate system to a local spatial coordinate system, which is based on the coordinates of the tower where the camera is located. The origin is [the point where the origin is located].

[0022] According to a preferred embodiment of the present invention, in step S1, the registered depth map is a two-dimensional image with the distance between the camera position and the corresponding position in the point cloud data of the power transmission scene as the pixel value;

[0023] The coordinate system transformation parameters generated during the registration process include camera intrinsic parameters M and camera extrinsic parameters R and T. The camera intrinsic parameter M is used for the transformation from the image coordinate system to the camera coordinate system, and the camera intrinsic parameter M is:

[0024] (1);

[0025] In equation (1), This is the length of the camera's focal length along the x-axis. This is the length of the camera's focal length along the y-axis. The position of the center point along the x-axis of the image coordinate system. This represents the position of the center point along the y-axis of the image coordinate system.

[0026] The camera extrinsic parameters R and T are a 3×3 rotation matrix R and a 3×1 translation matrix T, respectively, used for the transformation from the camera coordinate system to the local space coordinate system, and are as follows:

[0027] (2);

[0028] (3);

[0029] In equations (2) and (3), This represents the projection of a unit vector in the camera coordinate system onto the local space coordinate system. This represents the distance translated along the x-axis when a unit vector in the camera coordinate system is transformed to the local space coordinate system. This represents the distance a unit vector in the camera coordinate system is translated along the y-axis when transformed to the local space coordinate system. This represents the distance a unit vector in the camera coordinate system is translated along the z-axis when it is transformed to the local space coordinate system.

[0030] According to a preferred embodiment of the present invention, step S2 specifically includes:

[0031] Extract the tower coordinates, traverse point coordinates, and ground point coordinates from the point cloud data;

[0032] Based on the tower coordinates, the required traverse point coordinates are extracted, and the maximum and minimum values ​​of the horizontal and vertical coordinates of the traverse points are calculated to obtain the coordinates of the four corner points of the traverse.

[0033] Extract the coordinates of the ground points projected onto the ground from the four corner points of the conductor, and use these ground point coordinates to delineate the initial range of the protection zone;

[0034] The initial range of the protection zone is expanded to both sides of the conductor according to the given range expansion threshold, thus obtaining the final range of the transmission channel protection zone.

[0035] According to a preferred embodiment of the present invention, step S3 specifically includes:

[0036] The image data is input into the target detection algorithm to generate target bounding boxes for potential mechanical damage hazards. ,in, These represent the horizontal and vertical coordinates of the upper left corner of the target frame for potential mechanical damage hazards, respectively. These represent the width and height of the target frame for potential mechanical damage hazards, respectively.

[0037] Determine the center point of the bottom of the hazard based on the target frame information of the mechanical external damage hazard. ,and ,in, Let x and y represent the horizontal and vertical coordinates of the bottom center point of the target frame for potential mechanical damage hazards, respectively. ;

[0038] The center point of the hazard's bottom in the image coordinate system is obtained based on the depth map. depth value ;

[0039] Based on the center point of the bottom of the hazard Depth value And the camera's intrinsic parameter M, the camera coordinate system position of the target with potential mechanical damage is obtained as follows:

[0040] (4);

[0041] In equation (4), The camera coordinate system position of the target indicating potential mechanical damage is indicated. These represent the x and y coordinates of the center point at the bottom of the hazard, respectively. For depth value, The position of the center point along the x-axis of the image coordinate system. This represents the position of the center point along the y-axis of the image coordinate system.

[0042] Based on the camera coordinate system position of the target with potential mechanical damage. Including the camera's extrinsic parameters R and T, the local spatial coordinate system position of the target with potential mechanical damage is obtained as follows:

[0043] (5);

[0044] In equation (5), This indicates the local spatial coordinate system position of the target with potential mechanical damage.

[0045] According to a preferred embodiment of the present invention, in step S4, the world coordinate system is the UTM coordinate system with the epsg designation. Based on the position of the mechanical external damage hazard target in the local spatial coordinate system and the coordinate system transformation parameters, the position of the mechanical external damage hazard target in the world coordinate system is obtained as follows:

[0046] (6);

[0047] In equation (6), This indicates the location of the potential mechanical damage target in the world coordinate system. , These represent the rotation and translation matrices from the local space coordinate system to the world coordinate system in the coordinate system transformation parameters;

[0048] in, , Let represent the rotation and translation matrices from the local coordinate system to the world coordinate system, respectively, and let the rotation matrix... Translation matrix Represented as:

[0049] (7);

[0050] (8);

[0051] (9);

[0052] In equations (7)-(9), This represents the angle between two towers along the x-axis in the local spatial coordinate system. This represents the difference in the x-coordinates of two towers in a local spatial coordinate system. This represents the difference in the ordinates of two towers in a local spatial coordinate system. This represents the coordinates of the tower where the camera is located, and is the origin of the local spatial coordinate system.

[0053] In another aspect of the present invention, an apparatus is provided for implementing the method of generating a protection zone for a power transmission channel and locating potential mechanical damage hazards, the apparatus comprising:

[0054] The data acquisition module is used to acquire image data and point cloud data of the power transmission channel, and to register the image data and point cloud data to obtain a registered depth map and coordinate system transformation parameters generated during the registration process.

[0055] The protection zone generation module is used to obtain protection zone parameters based on the point cloud data, and determine the range of the power transmission channel protection zone by combining the protection zone parameters and a given range expansion threshold, and combine the coordinate system transformation parameters, protection zone parameters and depth map to construct a positioning model;

[0056] The first hazard target positioning module is used to obtain the target box information of the mechanical external damage hazard based on the image data using a target detection algorithm, determine the bottom center point of the hazard based on the mechanical external damage hazard target box information, obtain the depth value of the bottom center point of the hazard in the image coordinate system in combination with the depth map, and obtain the position of the mechanical external damage hazard target in the local spatial coordinate system based on the depth value and coordinate system transformation parameters.

[0057] The second positioning module for the hidden danger target is used to obtain the position of the mechanical external damage hidden danger target in the world coordinate system based on the position of the mechanical external damage hidden danger target in the local spatial coordinate system and the coordinate system transformation parameters;

[0058] The visualization annotation module is used to visualize and annotate the location of the power transmission channel protection zone and the mechanical external damage hazard target in the world coordinate system, respectively.

[0059] In another aspect of the invention, an electronic device is also provided, comprising:

[0060] At least one processor; and

[0061] The memory stores instructions that, when executed by the at least one processor, cause the at least one processor to perform the transmission channel protection zone generation and mechanical damage hazard location method as described above.

[0062] In another aspect of the invention, a machine-readable storage medium is also provided, which stores executable instructions that, when executed, cause the machine to perform the transmission channel protection zone generation and mechanical damage hazard location method as described above.

[0063] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0064] (1) The method for generating protection zones and locating mechanical external damage hazards in power transmission channels provided by the present invention generates depth maps and coordinate system transformation parameters by registering image data and point cloud data of power transmission channels, and establishes the transformation relationship from image coordinate system to world coordinate system. When a mechanical external damage hazard target is identified based on the target detection algorithm, the location information of the mechanical external damage hazard target can be further obtained, so as to realize the real-time and accurate acquisition of the coordinate position of mechanical external damage hazards in power transmission channels, and improve the timeliness of power transmission channel hazard discovery and rescue.

[0065] (2) Based on point cloud data information, the present invention generates a controllable protection area in the conductor area of ​​the power transmission channel, which can effectively assist maintenance personnel in judging the level of hidden dangers and greatly improve work efficiency.

[0066] (3) The method of the present invention is simple and efficient, and has high precision. It can assist the operation and maintenance personnel to judge the hidden danger level and obtain the hidden danger location information in the first time, greatly improving the timeliness of hidden danger assessment and maintenance rescue, significantly reducing the line patrol time and troubleshooting time, and realizing automatic and intelligent patrol inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 is the execution flowchart of the transmission line corridor protection area generation and mechanical external damage hidden danger location method of the present invention.

[0068] Figure 2 is the application effect diagram of the method of the present invention.

[0069] Figure 3 is the coordinate system involved in the method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0070] The following further describes the present disclosure in conjunction with the drawings and embodiments.

[0071] It should be noted that the following detailed description is exemplary and intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present disclosure belongs. ]

[0072] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0073] Without conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other.

[0074] Embodiment 1

[0075] Refer Figure 1 , this embodiment provides a transmission line corridor protection area generation and mechanical external damage hidden danger location method, and the method includes:

[0076] S1. Obtain the image data and point cloud data of the transmission line corridor, and register the image data and point cloud data to obtain the registered depth map and the coordinate system conversion parameters generated during the registration process.

[0077] Specifically, image data of the power transmission channel can be acquired using visual monitoring equipment. In this embodiment, a camera is used to acquire image data. The camera is fixedly installed on a tower at either end of the conductor. The acquired image data is in the form of an RGB two-dimensional image. Indicates, that is .

[0078] Point cloud data of power transmission channels can be acquired by scanning and collecting data using a drone equipped with a lidar system. The data will be in the form of n three-dimensional arrays. Indicates, that is , .

[0079] The acquired point cloud data includes point cloud coordinate information for the conductor, the towers supporting the conductor, the camera, and ground points. The coordinates of the two towers supporting the conductor are as follows: and .

[0080] Furthermore, before registering the image data and point cloud data, preprocessing operations are performed on both data separately. The image data preprocessing primarily involves standardizing the dimensions of all acquired image data. This can be done using image cropping tools or similar methods to resize and unify the dimensions of all acquired image data. This is a standard procedure and will not be elaborated upon here.

[0081] The preprocessing operations for point cloud data mainly include filtering and segmentation of the point cloud. The specific operation process is as follows:

[0082] The cloth filtering algorithm is used to downsample point cloud data and extract ground point cloud data. The grid size of the cloth filtering algorithm is 0.5m, the number of filtering iterations is 500, and the classification threshold is 0.2m.

[0083] Based on the image data of the power transmission channel, two towers of the overhead conductor to be segmented are selected. Then, the filtered point cloud data is segmented according to the coordinates of the two towers in the point cloud data to obtain point cloud data that matches the image data. Each segmented point cloud file contains two towers, each extending a fixed distance.

[0084] Simultaneously, all point cloud coordinates are transformed from the world coordinate system to a local spatial coordinate system with the tower where the camera is located as the origin. In this embodiment, the coordinates of the tower where the camera is located are... That is, the origin of the local spatial coordinate system is .

[0085] Next, the preprocessed image data and point cloud data are registered. AnnotationTools software can be used for this registration. During registration, the image data of the power transmission channel is used as a reference, and the point cloud is located in a local spatial coordinate system. By rotating and translating the point cloud, it is matched with the power transmission channel image, ultimately obtaining the matched depth map and the coordinate system transformation parameters generated during the registration process.

[0086] Furthermore, bilinear interpolation can be used to fill in local null areas in the depth map, improving the accuracy of hazard location. This operation is existing technology and will not be elaborated here.

[0087] The coordinate system transformation parameters generated during the registration process are described as camera intrinsic parameters M and camera extrinsic parameters R and T. The camera intrinsic parameters M are inherent camera properties, including the optical center and focal length, and are used for the transformation from the image coordinate system to the camera coordinate system. Specifically:

[0088] (1);

[0089] In equation (1), This is the length of the camera's focal length along the x-axis (horizontal) direction; This is the length of the camera's focal length along the y-axis (vertical) direction; The position of the center point along the x-axis (horizontal) of the image coordinate system. This represents the position of the center point of the image coordinate system along the y-axis (vertical) direction.

[0090] The camera's extrinsic parameters R and T are defined by a 3×3 rotation matrix R and a 3×1 translation matrix T, with the coordinates of the tower where the camera is located as the origin of the local spatial coordinate system. These are used for the transformation from the camera coordinate system to the local spatial coordinate system, specifically:

[0091] (2);

[0092] (3);

[0093] In equations (2) and (3), This represents the projection of a unit vector in the camera coordinate system onto the local space coordinate system. This represents the distance translated along the x-axis when a unit vector in the camera coordinate system is transformed to the local space coordinate system. This represents the distance a unit vector in the camera coordinate system is translated along the y-axis when transformed to the local space coordinate system. This represents the distance a unit vector in the camera coordinate system is translated along the z-axis when it is transformed to the local space coordinate system.

[0094] S2. Obtain the protection zone parameters based on the point cloud data, and determine the range of the power transmission channel protection zone by combining the protection zone parameters and the given range expansion threshold. Combine the coordinate system transformation parameters, protection zone parameters and depth map to construct a positioning model.

[0095] Specifically, the configuration of the protection zone parameters can be determined based on the tower coordinates in the point cloud data and the actual situation in the power transmission scenario. In this embodiment, the parameters are used... Indicates the parameters of the protected area, and ,in( , )~( , The numbers represent the coordinates of the four points in the protected area: the upper left, lower left, upper right, and lower right.

[0096] The specific process for obtaining the protected area parameters based on the point cloud data is as follows:

[0097] Extract the tower coordinates, traverse point coordinates, and ground point coordinates from the point cloud data;

[0098] Based on the tower coordinates, the required traverse point coordinates are extracted, and the maximum and minimum values ​​of the horizontal and vertical coordinates of the traverse points are calculated. Middle left top point The minimum value on the x-axis. Maximum value of the y-axis, bottom left point The minimum value on the x-axis. The minimum value of the ordinate is used to obtain the coordinates of the four corner points of the conductor, thereby obtaining the parameters of the protected area.

[0099] Then, the range of the transmission channel protection zone is determined by combining the protection zone parameters and the given range expansion threshold, specifically as follows:

[0100] Extract the coordinates of the ground points projected onto the ground from the four corner points of the conductor, and use these ground point coordinates to delineate the initial range of the protection zone;

[0101] The initial range of the protection zone is expanded to both sides of the conductor according to the given range expansion threshold, thus obtaining the final range of the transmission channel protection zone.

[0102] S3. Based on the image data, use a target detection algorithm to obtain the target bounding box information of the mechanical external damage hazard. Determine the bottom center point of the hazard according to the target bounding box information of the mechanical external damage hazard, and obtain the depth value of the bottom center point of the hazard in the image coordinate system in combination with the depth map. Obtain the position of the mechanical external damage hazard target in the local spatial coordinate system according to the depth value and the coordinate system transformation parameters.

[0103] In this embodiment, the target detection algorithm can use a pre-trained YOLOv5 algorithm model to identify potential mechanical damage targets in images (image data of the power transmission channel) captured by visible light equipment, which facilitates the subsequent location and labeling of potential hazards.

[0104] Specifically, the visible light image is input into the target detection algorithm, and the algorithm outputs the target bounding box information of the mechanical external damage hazard. This target bounding box information is described as a rectangular box. ,in, These represent the horizontal and vertical coordinates of the upper left corner of the target frame for potential mechanical damage hazards, respectively. These represent the width and height of the target frame for potential mechanical damage hazards, respectively.

[0105] Then, the center point at the bottom of the hazard can be determined based on the target frame information of the mechanical external damage hazard. ,and ,in, Let x and y represent the horizontal and vertical coordinates of the bottom center point of the target frame for potential mechanical damage hazards, respectively. .

[0106] Subsequently, the center point of the hazard's bottom in the image coordinate system was obtained by combining the depth map. depth value Specifically, this involves querying depth information based on the depth map in the positioning model, and identifying the center point at the bottom of the potential hazard. Since no depth information is available, the average depth of all depth maps is used as the depth information to finally obtain the center point of the hazard's bottom in the image coordinate system. depth value .

[0107] Finally, based on the depth value And coordinate system transformation parameters are used to obtain the position information of the potential mechanical damage target in the local spatial coordinate system, specifically:

[0108] First, based on the center point at the bottom of the hazard... Depth value In addition to the camera's intrinsic parameter M, the camera coordinate system position of the target with potential mechanical damage is obtained. ,Right now:

[0109] (4);

[0110] In equation (4), The camera coordinate system position of the target indicating potential mechanical damage is indicated. These represent the x and y coordinates of the center point at the bottom of the hazard, respectively. For depth value, The position of the center point along the x-axis of the image coordinate system. This represents the position of the center point along the y-axis of the image coordinate system.

[0111] Then, based on the camera coordinate system position of the target with potential mechanical damage hazards. In addition to the camera's extrinsic parameters R and T, the local spatial coordinate system position of the target with potential mechanical damage is obtained. ,Right now:

[0112] (5);

[0113] In equation (5), This indicates the local spatial coordinate system position of the target with potential mechanical damage.

[0114] S4. Based on the position of the mechanical external damage hazard target in the local spatial coordinate system and the coordinate system transformation parameters, obtain the position of the mechanical external damage hazard target in the world coordinate system.

[0115] In this embodiment, the position information of the potential mechanical damage target in the world coordinate system can be based on the local spatial coordinate system position of the potential mechanical damage target. and the rotation matrix in the coordinate system transformation parameters Translation matrix To obtain the position of the target with potential mechanical damage in the world coordinate system. ,and:

[0116] (6);

[0117] In equation (6), This indicates the location of the potential mechanical damage target in the world coordinate system. , represents the rotation and translation matrices from the local space coordinate system to the world coordinate system in the coordinate system transformation parameters; where the world coordinate system is the UTM coordinate system with the zone number epsg.

[0118] Furthermore, the rotation matrix from the local spatial coordinate system to the world coordinate system can be obtained based on the tower information. Translation matrix Two of the towers are located in the positive y-axis direction of the local spatial coordinate system, and the rotation matrix is... Translation matrix It can be represented as:

[0119] (7);

[0120] (8);

[0121] (9);

[0122] In equations (7)-(9), This represents the angle between two towers along the x-axis in the local spatial coordinate system. This represents the difference in the x-coordinates of two towers in a local spatial coordinate system. This represents the difference in the ordinates of two towers in a local spatial coordinate system. This represents the coordinates of the tower where the camera is located, and is the origin of the local spatial coordinate system.

[0123] The coordinate system used in the method of this embodiment is as follows: Figure 3 As shown, it includes the world coordinate system Ow, the camera coordinate system Oc, the image coordinate system O-xy, and the pixel coordinate system O-uv.

[0124] S5. Based on the range of the power transmission channel protection zone and the position of the mechanical external damage hazard target in the world coordinate system, the power transmission channel protection zone and the mechanical external damage hazard target position are visually marked respectively.

[0125] If the target location of the potential mechanical damage hazard is described as its position in the WGS84 world coordinate system, then:

[0126] (9);

[0127] In equation (9), Longitude As a dimension, Specify the input coordinate system as UTM, and zone as the zone number of the UTM coordinate system. Specify the output coordinate system as .

[0128] The effectiveness of the method in this embodiment is verified by the following example:

[0129] The images were taken by a monitoring device on a power transmission channel in a certain area and the image size was 1408*1408.

[0130] 1) Image data is acquired based on the visual monitoring equipment (i.e., camera) of the power transmission channel: Simultaneously, point cloud data is acquired by scanning and collecting data using a lidar-equipped drone. The current point cloud data collected within the channel has a range of 350×80m. 2 .

[0131] After obtaining the point cloud data, the point cloud is first preprocessed using a cloth filtering algorithm with a grid size of 0.5m, a filtering iteration count of 500, and a classification threshold of 0.2m. Then, the point cloud is segmented according to the tower coordinates to obtain point cloud data that matches the image data.

[0132] The coordinates of the two towers are respectively and ,in The tower where the camera is located is the origin of the local spatial coordinate system.

[0133] ;

[0134] .

[0135] Then based on the tower where the camera is located The coordinate system is transformed using the origin, converting the point cloud data from the world coordinate system to the local spatial coordinate system.

[0136] 2) Use Annotation Tools software to register the transmission channel image data and point cloud data, and obtain the registered depth map and coordinate system transformation parameters generated during the registration process:

[0137] The camera intrinsic parameter M, from the image coordinate system to the camera coordinate system, is:

[0138] ;

[0139] The camera extrinsic parameters, rotation matrix R and translation matrix T, from the camera coordinate system to the local space coordinate system are as follows:

[0140] ;

[0141] .

[0142] 3) Fill in the obtained depth map and calculate the parameters of the protected area; combine the coordinate system transformation parameters, protected area parameters, depth map and other data obtained in 1) and 2) to make a positioning model for subsequent hazard positioning tasks.

[0143] The parameters for the protected area, obtained from the tower information, are as follows:

[0144] The area under the conductor is obtained by calculating the projection points from the conductor to the ground. It is described by four corner points, and the size of the protected area is controlled by a threshold 'a', which can be set from 1 to 3. This helps to determine the level of hazard. The parameters of the protected area are:

[0145] .

[0146] 4) Obtain the target bounding box of potential mechanical damage hazards in the image based on the target detection algorithm, such as... Figure 2 The excavator in the image is the target of the mechanical external damage hazard mentioned in this article, and its target box is:

[0147] .

[0148] 5) Based on the image coordinate system location information of the potential hazard target and the coordinate system transformation parameters in the localization model, obtain the location information of the potential hazard target in the local spatial coordinate system:

[0149] First, obtain the coordinates of the center point at the bottom of the potential hazard: And obtain the center point of the bottom of the hidden danger in the depth map. Corresponding depth value =166.24;

[0150] Then, according to the formula Obtain the coordinates of the camera in the hazard control system: ;

[0151] Finally, based on the rotation matrix R and translation matrix T mentioned above, the local spatial coordinate system position of the potential hazard target is obtained:

[0152] .

[0153] 6) Based on the local spatial coordinate system location information of the potential hazard target and the relevant parameters in the positioning model, obtain the location information of the mechanical external damage hazard target in the world coordinate system:

[0154] First, based on the tower coordinates obtained in 1), and Obtain the angle between two towers along the x-axis in the local spatial coordinate system. =-0.0862308;

[0155] Then, the rotation matrix from the local coordinate system to the world coordinate system is obtained using the following formula. Translation matrix :

[0156] = ;

[0157] ;

[0158] Finally, the world coordinate system position of the potential hazard target was calculated. .

[0159] 7) Visualize and annotate the location information of the protected area and the hazard target based on the world coordinate system location information of the hazard target and the parameters of the protected area:

[0160] First, obtain the WGS84 coordinate system position of the potential hazard target using the following function;

[0161] ;

[0162] ;

[0163] Then, the obtained WGS coordinate system position and protected area parameters are visualized and annotated on the original image, as shown in the following figure. Figure 2 As shown, Figure 2 The dark area in the middle is the marked protection zone, the dashed box is the maximum range of the protection zone, and the solid box is the marked target location of potential mechanical damage hazards.

[0164] Example 2

[0165] This embodiment provides an apparatus for generating protection zones and locating potential mechanical damage hazards in power transmission channels. The apparatus includes:

[0166] The data acquisition module is used to acquire image data and point cloud data of the power transmission channel, and to register the image data and point cloud data to obtain a registered depth map and coordinate system transformation parameters generated during the registration process.

[0167] The protection zone generation module is used to obtain protection zone parameters based on the point cloud data, and determine the range of the power transmission channel protection zone by combining the protection zone parameters and a given range expansion threshold, and combine the coordinate system transformation parameters, protection zone parameters and depth map to construct a positioning model;

[0168] The first hazard target positioning module is used to obtain the target box information of the mechanical external damage hazard based on the image data using a target detection algorithm, determine the bottom center point of the hazard based on the mechanical external damage hazard target box information, obtain the depth value of the bottom center point of the hazard in the image coordinate system in combination with the depth map, and obtain the position of the mechanical external damage hazard target in the local spatial coordinate system based on the depth value and coordinate system transformation parameters.

[0169] The second positioning module for the hidden danger target is used to obtain the position of the mechanical external damage hidden danger target in the world coordinate system based on the position of the mechanical external damage hidden danger target in the local spatial coordinate system and the coordinate system transformation parameters;

[0170] The visualization annotation module is used to visualize and annotate the location of the power transmission channel protection zone and the mechanical external damage hazard target in the world coordinate system, respectively.

[0171] Example 3

[0172] This embodiment also provides an electronic device, including:

[0173] At least one processor; and

[0174] The memory stores instructions that, when executed by the at least one processor, cause the at least one processor to perform the transmission channel protection zone generation and mechanical damage hazard location method as described above.

[0175] In this embodiment, the electronic device may include, but is not limited to: personal computer, server computer, workstation, desktop computer, laptop computer, notebook computer, mobile computing device, smartphone, tablet computer, cellular phone, personal digital assistant (PDA), handheld device, messaging device, wearable computing device, consumer electronic device, etc.

[0176] Example 4

[0177] This embodiment also provides a machine-readable storage medium storing executable instructions, which, when executed, cause the machine to perform the transmission channel protection zone generation and mechanical damage hazard location method as described above.

[0178] Specifically, a system or apparatus equipped with a readable storage medium may be provided, on which software program code implementing the functions of any of the embodiments described above is stored, and the computer or processor of the system or apparatus can read and execute the instructions stored in the readable storage medium.

[0179] In this case, the program code read from the readable medium itself can perform the functions of any of the above embodiments, and therefore the machine-readable code and the readable storage medium storing the machine-readable code constitute a part of this specification.

[0180] Examples of readable storage media include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer or the cloud via a communication network.

[0181] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0182] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0183] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0184] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0185] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the technical solutions of the present invention, and are not intended to limit the specific implementation of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the claims of the present invention should be included within the protection scope of the claims of the present invention.

Claims

1. A method for generating protection zones and locating potential mechanical damage hazards in power transmission channels, characterized in that, The method includes: S1. Acquire image data and point cloud data of the power transmission channel, and register the image data and point cloud data to obtain the registered depth map and the coordinate system transformation parameters generated during the registration process; S2. Obtain the protection zone parameters based on the point cloud data, and determine the range of the power transmission channel protection zone by combining the protection zone parameters and the given range expansion threshold, and combine the coordinate system transformation parameters, protection zone parameters and depth map to construct a positioning model; S3. Based on the image data, use a target detection algorithm to obtain the target box information of the mechanical external damage hazard, determine the bottom center point of the hazard according to the target box information of the mechanical external damage hazard, and obtain the depth value of the bottom center point of the hazard in the image coordinate system in combination with the depth map, and obtain the position of the mechanical external damage hazard target in the local spatial coordinate system according to the depth value and the coordinate system transformation parameters. S4. Based on the position of the mechanical external damage hazard target in the local spatial coordinate system and the coordinate system transformation parameters, obtain the position of the mechanical external damage hazard target in the world coordinate system; S5. Based on the range of the power transmission channel protection zone and the position of the mechanical external damage hazard target in the world coordinate system, the power transmission channel protection zone and the position of the mechanical external damage hazard target are visually marked respectively; Step S2 specifically includes: Extract the tower coordinates, traverse point coordinates, and ground point coordinates from the point cloud data; Based on the tower coordinates, the required traverse point coordinates are extracted, and the maximum and minimum values ​​of the horizontal and vertical coordinates of the traverse points are calculated to obtain the coordinates of the four corner points of the traverse. Extract the coordinates of the ground points projected onto the ground from the four corner points of the conductor, and use these ground point coordinates to delineate the initial range of the protection zone; The initial range of the protection zone is expanded to both sides of the conductor according to the given range expansion threshold, thus obtaining the final range of the transmission channel protection zone. Step S3 specifically includes: The image data is input into the target detection algorithm to generate target bounding boxes for potential mechanical damage hazards. ,in, These represent the horizontal and vertical coordinates of the upper left corner of the target frame for potential mechanical damage hazards, respectively. These represent the width and height of the target frame for potential mechanical damage hazards, respectively. Determine the center point of the bottom of the hazard based on the target frame information of the mechanical external damage hazard. ,and ,in, Let x and y represent the horizontal and vertical coordinates of the bottom center point of the target frame for potential mechanical damage hazards, respectively. ; The center point of the hazard's bottom in the image coordinate system is obtained based on the depth map. depth value ; Based on the center point of the bottom of the hazard Depth value And the camera's intrinsic parameter M, the camera coordinate system position of the target with potential mechanical damage is obtained as follows: （4）； In equation (4), The camera coordinate system position of the target indicating potential mechanical damage is indicated. These represent the x and y coordinates of the center point at the bottom of the hazard, respectively. For depth value, The position of the center point along the x-axis of the image coordinate system. This represents the position of the center point along the y-axis of the image coordinate system. Based on the camera coordinate system position of the target with potential mechanical damage. In addition to the camera's extrinsic parameters R and T, the local spatial coordinate system position of the target with potential mechanical damage is obtained as follows: （5）； In equation (5), The local spatial coordinate system position of the target posing a potential mechanical damage hazard; In step S4, the world coordinate system is the UTM coordinate system with the epsg designation. Based on the position of the potential mechanical damage target in the local spatial coordinate system and the coordinate system transformation parameters, the position of the potential mechanical damage target in the world coordinate system is obtained as follows: （6）； In equation (6), This indicates the location of the potential mechanical damage target in the world coordinate system. , These represent the rotation and translation matrices from the local space coordinate system to the world coordinate system in the coordinate system transformation parameters; in, , Let represent the rotation and translation matrices from the local coordinate system to the world coordinate system, respectively, and let the rotation matrix... Translation matrix Represented as: （7）； （8）； （9）； In equations (7)-(9), This represents the angle between two towers along the x-axis in the local spatial coordinate system. This represents the difference in the x-coordinates of two towers in a local spatial coordinate system. This represents the difference in the ordinates of two towers in a local spatial coordinate system. This represents the coordinates of the tower where the camera is located, and is the origin of the local spatial coordinate system.

2. The method for generating protection zones and locating potential mechanical damage hazards in power transmission channels according to claim 1, characterized in that, In step S1, a camera is used to acquire image data of the power transmission channel. The camera is fixedly installed on any tower used to mount the power transmission conductor. The image data is an RGB two-dimensional image. It means, and ; Point cloud data of the power transmission channel is acquired using a drone equipped with a lidar system. The point cloud data consists of n three-dimensional arrays. It means, and , .

3. The method for generating protection zones and locating potential mechanical damage hazards in power transmission channels according to claim 1, characterized in that, In step S1, before registering the image data and point cloud data, the point cloud data is first preprocessed, specifically including: The point cloud data is filtered using a cloth filtering algorithm, wherein the grid size of the cloth filtering algorithm is 0.5m, the number of filtering iterations is 500, and the classification threshold is 0.2m; Based on the image data of the transmission channel, two towers of the overhead conductor to be segmented are selected. The filtered point cloud data is then segmented according to the coordinates of the two towers in the point cloud data to obtain point cloud data that matches the image data. The coordinates of the two towers are as follows: and And the coordinates of the tower where the camera is located are ; The point cloud data is transformed from the world coordinate system to a local spatial coordinate system, which is based on the coordinates of the tower where the camera is located. The origin is [the point where the origin is located].

4. The method for generating protection zones and locating potential mechanical damage hazards in power transmission channels according to claim 1, characterized in that, In step S1, the registered depth map is a two-dimensional image with the distance between the camera position and the corresponding position of the point cloud data in the power transmission scene as the pixel value. The coordinate system transformation parameters generated during the registration process include camera intrinsic parameters M and camera extrinsic parameters R and T. The camera intrinsic parameter M is used for the transformation from the image coordinate system to the camera coordinate system, and the camera intrinsic parameter M is: （1）； In equation (1), This is the length of the camera's focal length along the x-axis. This is the length of the camera's focal length along the y-axis. The position of the center point along the x-axis of the image coordinate system. This represents the position of the center point along the y-axis of the image coordinate system. The camera extrinsic parameters R and T are a 3×3 rotation matrix R and a 3×1 translation matrix T, respectively, used for the transformation from the camera coordinate system to the local space coordinate system, and are as follows: （2）； （3）； In equations (2) and (3), This represents the projection of a unit vector in the camera coordinate system onto the local space coordinate system. This represents the distance translated along the x-axis when a unit vector in the camera coordinate system is transformed to the local space coordinate system. This represents the distance a unit vector in the camera coordinate system is translated along the y-axis when transformed to the local space coordinate system. This represents the distance a unit vector in the camera coordinate system is translated along the z-axis when it is transformed to the local space coordinate system.

5. An apparatus for implementing the method for generating a protection zone and locating potential mechanical damage hazards in a power transmission channel as described in any one of claims 1-4, characterized in that, The device includes: The data acquisition module is used to acquire image data and point cloud data of the power transmission channel, and to register the image data and point cloud data to obtain a registered depth map and coordinate system transformation parameters generated during the registration process. The protection zone generation module is used to obtain protection zone parameters based on the point cloud data, and determine the range of the power transmission channel protection zone by combining the protection zone parameters and a given range expansion threshold, and combine the coordinate system transformation parameters, protection zone parameters and depth map to construct a positioning model; The first hazard target positioning module is used to obtain the target box information of the mechanical external damage hazard based on the image data using a target detection algorithm, determine the bottom center point of the hazard based on the mechanical external damage hazard target box information, obtain the depth value of the bottom center point of the hazard in the image coordinate system in combination with the depth map, and obtain the position of the mechanical external damage hazard target in the local spatial coordinate system based on the depth value and coordinate system transformation parameters. The second positioning module for the hidden danger target is used to obtain the position of the mechanical external damage hidden danger target in the world coordinate system based on the position of the mechanical external damage hidden danger target in the local spatial coordinate system and the coordinate system transformation parameters; The visualization annotation module is used to visualize and annotate the location of the power transmission channel protection zone and the mechanical external damage hazard target in the world coordinate system, respectively.

6. An electronic device, characterized in that, The electronic device includes: At least one processor; and The memory stores instructions that, when executed by the at least one processor, cause the at least one processor to perform the method for generating a power transmission channel protection zone and locating potential mechanical damage hazards as described in any one of claims 1 to 4.

7. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores executable instructions, which, when executed, cause the machine to perform the method for generating a power transmission channel protection zone and locating potential mechanical damage hazards as described in any one of claims 1 to 4.

Images