A method for real-time monitoring video fusion of multi-source data from integrated power transmission lines

Abstract

This invention relates to a real-time monitoring video fusion method for integrating multi-source data from power transmission lines. The method integrates and displays laser point cloud data of the transmission corridor, oblique photography data of the transmission corridor, and 3D model data of the transmission line on a power transmission digital twin platform to construct a realistic 3D scene of the transmission corridor. Then, the platform loads the locations of potential hazards along the transmission corridor based on geographical coordinates. Next, prefabricated camera positions are constructed for the locations of these potential hazards in the realistic 3D scene. Finally, the corner points of the calibrated objects in the laser point cloud data of the transmission corridor in the realistic 3D scene are registered with the corresponding points in the video monitoring images obtained by the video cameras at the prefabricated camera positions, ensuring that the video monitoring images are closely aligned with the images determined by the oblique photography data of the transmission corridor and the 3D model data of the transmission line. This invention provides a real-time realistic 3D scene and facilitates rapid determination of the true geographical location of potential hazards along the transmission line.

Description

Technical Field

[0001] This invention relates to the field of power monitoring technology, specifically to a method for real-time monitoring video fusion of multi-source data from integrated power transmission lines. Background Technology

[0002] In the technology for monitoring power transmission lines, the use of Geographic Information Systems (GIS) to create realistic 3D scenes based on the fusion of laser point cloud technology, oblique photogrammetry, and GIM modeling technology for transmission lines offers several advantages. It not only completely recreates the topography within the transmission corridor but also boasts high-precision measurability, enabling full-angle 3D measurements. However, realistic 3D scenes represent measurement results at a specific point in time and are essentially static maps, making data timeliness issues unavoidable.

[0003] Using video surveillance equipment to monitor power transmission lines can solve the problem of data timeliness. However, the video surveillance images acquired by the equipment are isolated from the surrounding environment, and it is not possible to intuitively understand the exact location of the video surveillance images in the real world through the screen. It is usually necessary to watch multiple split-screen images at the same time to know the true geographical location of the power transmission lines in the images, which is extremely inconvenient.

[0004] Therefore, when monitoring transmission lines, it is urgent to solve the problem of data timeliness while quickly determining the true geographical location of transmission lines, especially potential hazards. Summary of the Invention

[0005] The technical problem to be solved by the embodiments of the present invention is to provide a real-time monitoring video fusion method for integrating multi-source data of transmission lines, which solves the problem of data timeliness and facilitates the rapid determination of the true geographical location of transmission lines, especially the hidden danger points of transmission lines.

[0006] To address the aforementioned technical problems, embodiments of the present invention provide a method for real-time monitoring video fusion of multi-source data from integrated power transmission lines, the method comprising the following steps:

[0007] S11. Acquire data, which includes the location coordinates and attribute information of potential hazards in the power transmission corridor, GIM 3D model data of the power transmission line, LAS point cloud data of the power transmission corridor, OSGB oblique photography data of the power transmission corridor, and the coordinates of video cameras, video surveillance remote control interface and video surveillance streaming media service address provided by the Internet of Things platform.

[0008] S12. Lightweight processing is performed on the laser point cloud (LAS) data of the power transmission corridor, the oblique photogrammetry (OSGB) data of the power transmission corridor, and the GIM 3D model data of the power transmission line, converting them into a tile-type dataset with a hierarchical data structure to obtain lightweight laser point cloud data of the power transmission corridor, oblique photogrammetry data of the power transmission corridor, and 3D model data of the power transmission line.

[0009] S13. By integrating and displaying the lightweight laser point cloud data of the power transmission corridor, the oblique photography data of the power transmission corridor, and the three-dimensional model data of the power transmission line on the existing power transmission digital twin platform, a real-world three-dimensional scene of the power transmission corridor is constructed.

[0010] S14. Based on the location coordinates of the potential hazards in the power transmission corridor, the locations of the potential hazards in the power transmission corridor are loaded into the real-world 3D scene on the power transmission digital twin platform according to the geographical coordinates.

[0011] S15. Based on the coordinates of the video camera, the video surveillance remote control interface, and the video surveillance streaming media service address, in the real-world 3D scene, construct prefabricated camera positions for the potential hazards of the power transmission corridor.

[0012] S16. Register the corner points of the calibration objects in the laser point cloud data of the power transmission corridor in the real-world 3D scene with the corresponding points in the video monitoring images obtained by the video camera at the prefabricated location, and adjust the video monitoring images at the corresponding points to make the video monitoring images at the corresponding points fit the images determined by the oblique photography data of the power transmission corridor and the images determined by the 3D model data of the power transmission line.

[0013] Further, step S15 specifically includes:

[0014] Connect the video surveillance remote control interface and the video surveillance body service address to the power transmission digital twin platform;

[0015] Based on the coordinates of the video camera, a 3D model of the video camera is set at the same coordinate position in the real-world 3D scene;

[0016] Call the video surveillance remote control interface to obtain the parameters of the video camera corresponding to the coordinates of the video camera;

[0017] The video camera is remotely controlled to focus on the location of the potential hazard point in the power transmission corridor, and the focused camera parameters are written into a JSON registration file, thereby constructing a prefabricated camera position for the potential hazard point in the power transmission corridor.

[0018] Furthermore, the parameters of the video camera include orientation, tilt angle, rotation angle, field of view, aspect ratio, focal length, and shooting distance.

[0019] Furthermore, in step S16, the video monitoring images at the registered locations are adjusted using a video ground-fitting editing tool so that the video monitoring images at the registered locations are aligned with the images determined by the oblique photography data of the power transmission corridor and the images determined by the three-dimensional model data of the power transmission line.

[0020] Furthermore, after step S16, the method further includes:

[0021] S17. The video surveillance images that are fitted with the calibrated ground features in the corresponding images determined by the oblique photography data of the power transmission corridor are converted into matrix parameters and stored in the database.

[0022] Furthermore, the video surveillance streaming media service address conforms to the RTSP protocol.

[0023] Implementing the embodiments of the present invention has the following beneficial effects: The embodiments of the present invention construct a real-world 3D scene of the power transmission corridor on a power transmission digital twin platform using laser point cloud data, oblique photography data of the power transmission corridor, and 3D model data of the power transmission line. Then, based on geographical coordinates, pre-set camera positions for video cameras targeting potential hazards in the power transmission corridor are constructed within the real-world 3D scene. Through adjustments, the video monitoring images obtained by the video cameras at the pre-set positions are aligned with the images determined by the oblique photography data of the power transmission corridor and the 3D model data of the power transmission line. Thus, the embodiments of the present invention imbue the static real-world 3D scene data with real-time image information acquired by the video cameras, thereby transforming the two-dimensional monitoring image into a three-dimensional image, providing a real-time real-world 3D scene, and solving the problem of data timeliness in static real-world 3D scenes. Furthermore, this invention provides a geographic coordinate-based matching method for modeling hidden danger points in power transmission corridors from real-time monitoring images acquired by video cameras, particularly those related to power transmission corridor inspections. This solves the problem that conventional video surveillance images cannot determine the true geographical location of objects within the image, facilitating the rapid determination of the true geographical location of power transmission lines, especially hidden danger points, in video surveillance images. Therefore, this invention provides technical support for real-time intelligent identification of changes in hidden danger points in power transmission corridors and intelligent early warning of these points, significantly improving the efficiency of maintenance personnel in investigating hidden danger points within power transmission corridors. Through intuitive and clear real-time 3D scenes, maintenance personnel can quickly determine the location of hidden danger points in long-distance power transmission lines, saving time from problem discovery to developing solutions. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart of a real-time monitoring video fusion method for multi-source data of integrated power transmission lines, according to an embodiment of the present invention. Detailed Implementation

[0026] The following description of the embodiments is taken with reference to the accompanying drawings, which illustrate specific embodiments in which the invention can be implemented.

[0027] like Figure 1 As shown in the figure, this invention provides a real-time monitoring video fusion method for integrating multi-source data of power transmission lines, the method comprising the following steps:

[0028] S11. Acquire data, which includes the location coordinates and attribute information of potential hazards in the power transmission corridor, 3D model data of the power transmission line GIM (Grid Information Model), laser point cloud (LAS) data of the power transmission corridor, oblique photography (OSGB) data of the power transmission corridor, and the coordinates of video cameras, video surveillance remote control interface, and video surveillance streaming media service address provided by the Internet of Things platform.

[0029] Specifically, the data can be categorized into aerial survey data, GIM data, coordinate data, IoT platform data, and other data. The aerial survey data includes laser point cloud (LAS) data and oblique photogrammetry (OSGB) data of the power transmission corridor. The GIM data includes 3D GIM model data of the power transmission line. The coordinate data includes the location coordinates of potential hazards along the power transmission corridor and the coordinates of video cameras. The IoT platform data includes the video surveillance remote control interface and the video surveillance streaming media service address, wherein the video surveillance streaming media service address conforms to the RTSP protocol. The other data includes hazard attribute information of potential hazards along the power transmission corridor. The aerial survey data and the GIM data are primarily used to construct a realistic 3D scene of the power transmission corridor. The coordinate data and the IoT platform data are primarily used to construct prefabricated camera positions within the realistic 3D scene and generate JSON registration files. The hazard attribute information of potential hazards along the power transmission corridor is used to assist in understanding the hazard information of these points.

[0030] S12. Lightweight processing is performed on the laser point cloud (LAS) data of the power transmission corridor, the oblique photogrammetry (OSGB) data of the power transmission corridor, and the GIM 3D model data of the power transmission line, converting them into a tile-style dataset with a hierarchical data structure to obtain lightweight laser point cloud data of the power transmission corridor, oblique photogrammetry data of the power transmission corridor, and 3D model data of the power transmission line.

[0031] Specifically, the tile-based dataset with a hierarchical data structure has a hierarchical data structure that satisfies the requirements for transmission and rendering and a data structure that conforms to the tile format set requirements.

[0032] Lightweight processing of the laser point cloud (LAS) data of the power transmission corridor can specifically include: segmenting the LAS data of the power transmission corridor using a non-uniform octree algorithm; expanding the quadtree by dividing the overall data block into eight sub-blocks using three orthogonal segmentation planes; and adopting segmentation strategies such as irregular segmentation, tight bounding volume, and overlapping segmentation by configuring segmentation parameters.

[0033] Lightweight processing of the oblique photogrammetry OSGB data of the power transmission corridor can specifically include: segmenting the oblique photogrammetry OSGB data of the power transmission corridor using a non-uniform octree algorithm, and converting it into batched 3D model data (B3DM) by configuring block parameters and merging root nodes.

[0034] The lightweight processing of the transmission line GIM 3D model data can specifically include: segmenting the transmission line GIM 3D model data using a non-uniform quadtree algorithm; by configuring the segmentation parameters, segmentation strategies such as irregular segmentation, tight bounding volume, and overlapping segmentation can be adopted; and a grid segmentation algorithm can be used to perform non-uniform and covered segmentation of the strip-shaped 3D model into any number of blocks.

[0035] The process of lightweighting the data described above is well known to those skilled in the art, and therefore will not be repeated here.

[0036] S13. By integrating and displaying the lightweight laser point cloud data of the power transmission corridor, the oblique photography data of the power transmission corridor, and the three-dimensional model data of the power transmission line on the existing power transmission digital twin platform, a real-world three-dimensional scene of the power transmission corridor is constructed.

[0037] The power transmission digital twin platform mentioned is an existing power transmission digital twin platform, so it will not be described in detail here.

[0038] S14. Based on the location coordinates of the potential hazards in the power transmission corridor, the location of the potential hazards in the power transmission corridor is loaded into the real-world 3D scene on the power transmission digital twin platform according to the geographical coordinates.

[0039] S15. Based on the coordinates of the video camera, the video surveillance remote control interface, and the video surveillance streaming media service address, in the real-world 3D scene, construct prefabricated camera positions for the potential hazards of the power transmission corridor.

[0040] Furthermore, step S15 may specifically include the following steps:

[0041] S151. Connect the video surveillance remote control interface and the video surveillance body service address to the power transmission digital twin platform.

[0042] S152. Based on the coordinates (geographic coordinates) of the video camera, set up a 3D model of the video camera at the same coordinate position in the real-world 3D scene.

[0043] S153. Call the video surveillance remote control interface to obtain the parameters of the video camera corresponding to the coordinates of the video camera.

[0044] Specifically, the parameters of the video camera may include orientation, tilt angle, rotation angle, field of view, aspect ratio, focal length, and shooting distance.

[0045] S154. Remotely control the video camera to focus on the location of the potential hazard point in the power transmission corridor. After adjusting to a reasonable camera position and focal length, write the focused camera parameters into a JSON registration file, thereby constructing a pre-fabricated camera position for the potential hazard point in the power transmission corridor.

[0046] After constructing the prefabricated locations for the potential hazards along the power transmission corridor, the real-time monitoring video fusion method of this embodiment further includes:

[0047] S16. Register the corner point of the calibration object in the laser point cloud data of the power transmission corridor in the real-world 3D scene with the same point in the video monitoring image obtained by the video camera at the prefabricated position, and adjust the video monitoring image at the same point determined by the registration so that the video monitoring image at the same point is consistent with the image determined by the oblique photography data of the power transmission corridor and the image determined by the 3D model data of the power transmission line.

[0048] In step S16, the control points in the real-world 3D scene are first associated with their corresponding points in the video surveillance images obtained by the video camera. Then, the video surveillance images at the corresponding points are adjusted so that the video surveillance images at the corresponding points are aligned with the images determined by the oblique photography data of the power transmission corridor and the images determined by the 3D model data of the power transmission line.

[0049] Adjustments to the video surveillance footage can be made using a video ground-alignment editing tool. This tool adjusts the video surveillance footage at the registered location, and specific adjustments can include local stretching, twisting, and deformation, so that the video surveillance footage at the registered location aligns with the footage determined by the oblique photography data of the power transmission corridor and the footage determined by the three-dimensional model data of the power transmission line.

[0050] After overlaying the video surveillance footage at the same location with the footage determined by the oblique photography data of the power transmission corridor and the footage determined by the three-dimensional model data of the power transmission line, the real-time monitoring video fusion method of this embodiment of the invention may further include:

[0051] S17. The video surveillance images that are fitted with the calibrated ground features in the corresponding images determined by the oblique photography data of the power transmission corridor are converted into matrix parameters and stored in the database.

[0052] In summary, this embodiment of the invention first constructs a real-world 3D scene of the power transmission corridor on a power transmission digital twin platform using laser point cloud data, oblique photography data, and 3D model data of the power transmission corridor. Then, based on geographic coordinates, it constructs pre-set camera positions for video cameras targeting potential hazards in the power transmission corridor within the real-world 3D scene. By adjusting these positions, the video monitoring images obtained by the video cameras at the pre-set positions are aligned with the images determined by the oblique photography data and the 3D model data of the power transmission corridor. Thus, this embodiment of the invention imbues static real-world 3D scene data with real-time image information acquired by video cameras, thereby transforming the 2D monitoring images into 3D images and providing a real-time real-world 3D scene, solving the problem of data timeliness in static real-world 3D scenes.

[0053] Furthermore, this invention provides a geographic coordinate-based matching method for modeling real-time monitoring images acquired by video cameras, particularly the hidden danger points in power transmission corridor inspections. This solves the problem that conventional video surveillance images cannot determine the true geographical location of objects within the image. In other words, this invention uses geometric projection to fit the video surveillance image onto the ground and integrates it into a real-world 3D scene. This solves the problems of conventional video surveillance images being isolated from their surroundings, not being able to intuitively understand the exact location of the video surveillance image in the real world through the screen, and needing to view multiple split-screen images simultaneously. It can quickly determine the true geographical location of power transmission lines, especially hidden danger points on power transmission lines, in video surveillance images.

[0054] Therefore, the embodiments of the present invention provide technical support for real-time intelligent identification of changes in hidden danger points in power transmission corridors and intelligent early warning of hidden danger points, which greatly improves the efficiency of operation and maintenance personnel in investigating hidden danger points in power transmission corridors. Through intuitive and clear real-time three-dimensional scenes, operation and maintenance personnel can quickly determine the location of hidden danger points in long-distance power transmission lines, saving the process time from discovering problems to formulating handling measures.

[0055] The above description discloses only preferred embodiments of the present invention and should not be construed as limiting the scope of the present invention. Therefore, equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.

Claims

1. A method for real-time monitoring video fusion of multi-source data from integrated power transmission lines, characterized in that, The method includes the following steps: S11. Acquire data, including the location coordinates and attribute information of potential hazards in the power transmission corridor, GIM 3D model data of the power transmission line, LAS point cloud data of the power transmission corridor, OSGB oblique photography data of the power transmission corridor, and the coordinates of video cameras, video surveillance remote control interface and video surveillance streaming media service address provided by the Internet of Things platform. S12. Lightweight processing is performed on the laser point cloud (LAS) data of the power transmission corridor, the oblique photogrammetry (OSGB) data of the power transmission corridor, and the GIM 3D model data of the power transmission line, converting them into a tile-type dataset with a hierarchical data structure to obtain lightweight laser point cloud data of the power transmission corridor, oblique photogrammetry data of the power transmission corridor, and 3D model data of the power transmission line. S13. By integrating and displaying the lightweight laser point cloud data of the power transmission corridor, the oblique photography data of the power transmission corridor, and the three-dimensional model data of the power transmission line on the existing power transmission digital twin platform, a real-world three-dimensional scene of the power transmission corridor is constructed. S14. Based on the location coordinates of the potential hazards in the power transmission corridor, the locations of the potential hazards in the power transmission corridor are loaded into the real-world 3D scene on the power transmission digital twin platform according to the geographical coordinates. S15. Based on the coordinates of the video camera, the video surveillance remote control interface, and the video surveillance streaming media service address, in the real-world 3D scene, construct prefabricated camera positions for the potential hazards of the power transmission corridor. S16. Register the corner point of the calibration object in the laser point cloud data of the power transmission corridor in the real-world 3D scene with the same point in the video monitoring image obtained by the video camera at the prefabricated position, and adjust the video monitoring image at the same point determined by the registration so that the video monitoring image at the same point is consistent with the image determined by the oblique photography data of the power transmission corridor and the image determined by the 3D model data of the power transmission line. Step S15 specifically includes: Connect the video surveillance remote control interface and the video surveillance body service address to the power transmission digital twin platform; Based on the coordinates of the video camera, a 3D model of the video camera is set at the same coordinate position in the real-world 3D scene; Call the video surveillance remote control interface to obtain the parameters of the video camera corresponding to the coordinates of the video camera; The video camera is remotely controlled to focus on the location of the potential hazard point in the power transmission corridor, and the focused camera parameters are written into a JSON registration file, thereby constructing a prefabricated camera position for the potential hazard point in the power transmission corridor.

2. The real-time monitoring video fusion method according to claim 1, characterized in that, The parameters of the video camera include orientation, tilt angle, rotation angle, field of view, aspect ratio, focal length, and shooting distance.

3. The real-time monitoring video fusion method according to claim 1, characterized in that, In step S16, the video monitoring images at the registered locations are adjusted using a video ground-fitting editing tool so that the video monitoring images at the registered locations are aligned with the images determined by the oblique photography data of the power transmission corridor and the images determined by the three-dimensional model data of the power transmission line.

4. The real-time monitoring video fusion method according to claim 1, characterized in that, The method, after step S16, further includes: S17. The video surveillance images that are fitted with the calibrated ground features in the corresponding images determined by the oblique photography data of the power transmission corridor are converted into matrix parameters and stored in the database.

5. The real-time monitoring video fusion method according to claim 1, characterized in that, The video surveillance streaming media service address conforms to the RTSP protocol.

Images