Method for measuring safe distance of external broken object of power transmission conductor based on multi-modal data fusion
Through multimodal data fusion technology, the problems of high hardware cost and limited range in the safety distance monitoring of external objects in transmission conductors are solved, and the precise location of external objects is achieved, reducing the risk of accidents on transmission lines.
Patent Information
- Application Number
- CN202510633315.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-16
- Publication Date
- 2025-08-12
AI Technical Summary
The prior art has problems such as high additional hardware costs, limited measurement range and separation of measurement information from basic data in monitoring of external breakage safety distances of transmission conductors.
The multimodal data fusion method is adopted, through the fusion of point clouds and images, the data is aligned using the PNP algorithm, combined with the camera's internal and external parameter matrix calibration, the PCL algorithm is used to separate the wire data and generate a depth map, and combined with the yolov5 deep learning model to identify external objects and calculate the safe distance.
It reduces hardware costs, expands the measurement range, realizes absolute positioning of the external broken object position, improves monitoring accuracy, and reduces the risk of transmission line accidents.
Smart Images
Figure CN120471897A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of power facility safety monitoring, and in particular to a method for measuring the safety distance of objects outside a transmission line based on multimodal data fusion. Background Art
[0002] Currently, the safety distance monitoring of objects outside transmission lines mainly uses real-time radar data or long-baseline binocular ranging to measure the safety distance. However, these methods may have the following drawbacks: (1) Additional hardware costs: Additional high-precision radars and cameras need to be deployed; (2) Limited measurement range: Since the distance between towers in transmission lines is usually more than 500 meters, it is difficult for sensors to accurately measure data at this distance; (3) Separation of measurement information and basic data: The measurement information is not integrated with the existing point cloud data of the transmission line, and the measurement results cannot be reflected in the transmission line point cloud big data.
[0003] Based on this, we propose a method for measuring the safe distance of damaged objects outside transmission lines based on multimodal data fusion. Summary of the Invention
[0004] The purpose of the present invention is to provide a method for measuring the safe distance of damaged objects outside the transmission line based on multimodal data fusion, which can improve the measurement range and accuracy of the point cloud and image fusion online detection device.
[0005] The purpose of the present invention can be achieved through the following technical solutions: A method for measuring the safe distance of objects outside a transmission line based on multimodal data fusion, the method specifically comprising the following steps: S1. Data preparation; S2, fusion of point cloud and image; S3, separating the wire data and the depth map; S4. Calculation of safety distance.
[0006] As a further solution of the present invention, the data preparation in step S1 includes the following steps: S11, historical point cloud data between two transmission towers in a certain section of transmission line; S12. Pictures taken by the installed cameras in this section of the transmission line, and the camera's internal parameter matrix.
[0007] As a further solution of the present invention, the fusion of the point cloud and the image in step S2 includes the following steps: S21, by marking the points at the same position in the point cloud and the image, aligning the point cloud and the image using the PNP algorithm; S22. Use the camera extrinsic manual calibration tool to calibrate the point cloud and image pose matrix to achieve fusion alignment of the image and point cloud.
[0008] As a further solution of the present invention, the separation of the wire data and the depth map in step S3 includes the following steps: S31, separating the wires from the complete model data using the PCL algorithm and saving the data as a binary file; S32. Generate a depth map based on the point cloud map based on the calibrated extrinsic parameter matrix and save the data as a binary file.
[0009] As a further solution of the present invention, the safety distance calculation in step S4 includes the following steps: S41. Importing the separated wires and depth map into an edge device; S42, external broken object recognition: Use the external broken object recognition model of yolov5 deep learning to identify the rectangular box of the external broken object in the image; External object positioning: Calculate the distance between the external object and the camera through the rectangular frame and depth map in the image, and inversely calculate the height of the external object and the three-dimensional coordinate line segment of the top of the external object through the camera intrinsic parameter matrix; S43. Distance calculation: Calculate the Euclidean distance between the external object and the conductor using the three-dimensional coordinate line segment at the top of the external object and the three-dimensional data of the conductor, and find the closest point pair.
[0010] Beneficial effects of the present invention: (1) Reduce costs: No need to deploy real-time high-precision sensors other than cameras; (2) Range improvement: The measurement range of historical point cloud data depends only on whether the point cloud model is complete and the actual distance between towers; (3) Intelligent decision-making: Because the location of the external damage object calculated by this method is the absolute position in the entire transmission line, it is convenient for subsequent statistics, optimization, and decision-making.
[0011] (4) Reduce the risk of power transmission accidents caused by collisions with external objects, such as cranes, on power transmission lines. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] The present invention will be further described below with reference to the accompanying drawings.
[0013] Figure 1 It is a flowchart of the method of the present invention. DETAILED DESCRIPTION
[0014] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.
[0015] See also Figure 1 As shown, the present invention is a method for measuring the safe distance of objects outside the transmission line based on multimodal data fusion, and the measurement method specifically includes the following steps: S1. Data preparation; S2, fusion of point cloud and image; S3, separating the wire data and the depth map; S4. Calculation of safety distance.
[0016] The data preparation in step S1 includes the following steps: S11, historical point cloud data between two transmission towers in a certain section of transmission line; S12. Pictures taken by the installed cameras in this section of the transmission line, and the camera's internal parameter matrix.
[0017] The fusion of the point cloud and the image in step S2 includes the following steps: S21, by marking the points at the same position in the point cloud and the image, aligning the point cloud and the image using the PNP algorithm; S22. Use the camera extrinsic manual calibration tool to calibrate the point cloud and image pose matrix to achieve better fusion alignment of the image and point cloud.
[0018] Separating the wire data and the depth map in step S3 includes the following steps: S31, separating the wires from the complete model data using the PCL algorithm and saving the data as a binary file; S32. Generate a depth map based on the point cloud map based on the calibrated extrinsic parameter matrix and save the data as a binary file.
[0019] The safety distance calculation in step S4 includes the following steps: S41. Importing the separated wires and depth map into an edge device; S42, external broken object recognition: Use the external broken object recognition model of yolov5 deep learning to identify the rectangular box of the external broken object in the image; External object positioning: Calculate the distance between the external object and the camera through the rectangular frame and depth map in the image, and inversely calculate the height of the external object and the three-dimensional coordinate line segment of the top of the external object through the camera intrinsic parameter matrix; S43. Distance calculation: Calculate the Euclidean distance between the external object and the conductor using the three-dimensional coordinate line segment at the top of the external object and the three-dimensional data of the conductor, and find the closest point pair.
[0020] In this example, a transmission line simulation laboratory was built by ourselves. A handheld modeling tool was used to scan the laboratory point cloud model as historical point cloud data, and the conductor data was separated from the point cloud data. Then, an ordinary camera was used to take pictures, and the internal reference was provided by the camera. Use a self-developed annotation tool to annotate five point pairs in the image and point cloud, calculate the camera extrinsics using the PNP algorithm, and then use a self-developed calibration tool to calibrate the camera extrinsics. The fusion data of point cloud and image, namely depth map, is calculated through camera intrinsic and extrinsic parameters.
[0021] Use the deep learning model yolov5 to identify the rectangular box of the external object in the picture, and use the data corresponding to the lower edge of the rectangular box in the depth map as the distance data of the external object.
[0022] Combining the external object distance data, camera internal parameters and external parameters, the height of the external object and the three-dimensional line segment data of the top in the point cloud space are calculated.
[0023] Combining the separated wire data and 3D line segment data, the Euclidean distance between the two is calculated to be the distance between the current external object and the wire.
[0024] The present invention does not require the deployment of real-time high-precision sensors other than cameras. The measurement range of historical point cloud data only depends on whether the point cloud model is complete and the actual spacing between towers. The calculated position of the external damage is the absolute position in the entire transmission line, which is convenient for subsequent statistics, optimization, and decision-making, and can reduce the damage to the transmission line caused by external damage.
[0025] The above is a detailed description of an embodiment of the present invention. However, the content described is only a preferred embodiment of the present invention and should not be considered to limit the scope of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.
Claims
1. A method for measuring the safe distance of objects outside transmission lines based on multimodal data fusion, characterized in that: The measuring method specifically comprises the following steps: S1. Data preparation; S2, fusion of point cloud and image; S3, separating wire data and depth map; S4. Calculation of safety distance.
2. The method for measuring the safe distance of objects outside a power transmission line based on multimodal data fusion according to claim 1 is characterized in that: The data preparation in step S1 includes the following steps: S11, historical point cloud data between two transmission towers in a certain section of transmission line; S12. Pictures taken by the installed cameras in this section of the transmission line, and the camera's internal parameter matrix.
3. The method for measuring the safe distance of objects outside the power transmission line based on multimodal data fusion according to claim 1 is characterized in that: The fusion of the point cloud and the image in step S2 includes the following steps: S21, by marking the points at the same position in the point cloud and the image, aligning the point cloud and the image using the PNP algorithm; S22. Use the camera extrinsic manual calibration tool to calibrate the point cloud and image pose matrix to achieve fusion alignment of the image and point cloud.
4. The method for measuring the safe distance of objects outside a power transmission line based on multimodal data fusion according to claim 1 is characterized in that: Separating the wire data and the depth map in step S3 includes the following steps: S31. Separate the wires from the complete model data using the PCL algorithm and save the data as a binary file; S32. Generate a depth map based on the point cloud map based on the calibrated extrinsic parameter matrix and save the data as a binary file.
5. The method for measuring the safe distance of objects outside the power transmission line based on multimodal data fusion according to claim 1 is characterized in that: The safety distance calculation in step S4 includes the following steps: S41. Importing the separated wires and depth map into an edge device; S42, external broken object recognition: Use the external broken object recognition model of yolov5 deep learning to identify the rectangular box of the external broken object in the image; External object positioning: Calculate the distance between the external object and the camera through the rectangular frame and depth map in the image, and inversely calculate the height of the external object and the three-dimensional coordinate line segment of the top of the external object through the camera intrinsic parameter matrix; S43. Distance calculation: Calculate the Euclidean distance between the external object and the conductor using the three-dimensional coordinate line segment at the top of the external object and the three-dimensional data of the conductor, and find the closest point pair.