A method, device, and computer-readable storage medium for predicting ground-based movable hazard risks based on power transmission channel visualization.

Abstract

This invention belongs to the technical field of intelligent operation and maintenance of transmission lines, and more specifically, relates to a method for predicting the risk of mobile ground hazards based on the visualization of transmission channels. The method includes: acquiring three-dimensional point cloud data and two-dimensional image data of the transmission line towers; determining the extrinsic and intrinsic parameters of the camera; establishing a mapping relationship between the two-dimensional image and the three-dimensional point cloud; capturing at least three images at fixed time intervals using a monitoring device; identifying mobile hazards in each image and mapping them into three-dimensional point cloud data; calculating the center coordinates of the mobile hazard point cloud; constructing a motion estimation model based on the center point coordinates of the same hazard at different times, and predicting the next location of the hazard's center point; comparing the Euclidean nearest neighbor distance from the last observed hazard center point location to the conductor with the predicted center point location, and determining the risk level of the mobile hazard. This invention solves the problems of high hazard generation rate but low closed-loop efficiency in handling transmission lines, as well as the problem of predicting the risk of mobile hazards.

Description

Technical Field

[0001] This invention belongs to the field of intelligent operation and maintenance of power transmission lines, and more specifically, relates to a method, device and computer-readable storage medium for predicting ground-based movable hidden danger risks based on power transmission channel visualization. Background Technology

[0002] With the deep application of artificial intelligence technology, intelligent monitoring systems for power transmission lines based on intelligent image recognition and on-site hazard analysis algorithms have become an important technical means to promote the intelligent transformation of power inspection work. Against the backdrop of continuously upgrading demands for electricity quality in people's lives, the power inspection system is gradually evolving towards multi-dimensional monitoring, with the accurate identification and risk assessment of complex hazards becoming a key direction in this technological evolution.

[0003] Chinese patent document CN115995058A discloses an artificial intelligence-based online monitoring method for power transmission channel safety. The method involves acquiring surveillance images and laser point cloud data of the power transmission channel, preprocessing the images, obtaining coordinate transformation relationships using camera calibration and attitude estimation algorithms, constructing a single-stage detector, and training and adjusting the model. Using the target detection results and coordinate transformation relationships, the minimum distance from the target object to the power line is calculated. Finally, based on the safety distance regulations for overhead lines in the power grid company's enterprise standards, the safety level is determined, and timely warnings are issued for the target object.

[0004] It is worth noting that current mainstream systems primarily rely on two-dimensional image analysis for hazard identification. While this significantly improves power grid safety monitoring capabilities, the generation of massive amounts of visualized alarm data has revealed a significant discrepancy between the efficiency of hazard handling and the scale of data output. This leads to increased maintenance costs and an imbalance in resource allocation. Further analysis reveals that some movable hazards within the field of view, such as construction machinery, do not pose a substantial risk due to their movement trajectory or spatial position not interacting with transmission lines. Therefore, their safety threat level requires dynamic assessment. However, in the field of transmission line safety assessment, systematic research on dynamic risk prediction for movable hazards is still lacking.

[0005] In view of this, how to provide an efficient and reliable method for predicting the risk level of movable hazards in transmission lines, so as to achieve quantitative prediction and graded early warning of the risk level of movable hazards, effectively improve the effectiveness of alarm information and optimize the allocation of operation and maintenance resources, has become a technical bottleneck that the industry urgently needs to overcome. Summary of the Invention

[0006] This invention aims to overcome at least one of the defects of the prior art and provides a method for predicting the risk level of ground-based movable hazards based on power transmission channel visualization. By mapping the movable hazards from multiple consecutive two-dimensional images to coordinates in three-dimensional space, a three-dimensional spatial motion estimation model is constructed. At the same time, combined with the three-dimensional coordinates of the towers and lines, the problem of predicting the risk level of movable hazards is solved.

[0007] The detailed technical solution of this invention is as follows: A method for predicting the risk level of ground-based movable hazards based on power transmission channel visualization, the method comprising: S1: Acquire 3D point cloud data and 2D image data of the power transmission channel, determine camera intrinsic and extrinsic parameters through calibration, and determine the mapping relationship from 3D point cloud to 2D image; S2: After the monitoring device captures an image, it identifies movable hazards in the image and uses the least squares method to fit the plane equation to establish ground plane constraints. The center point at the bottom of the movable hazard frame is taken as the pixel reference point. The depth value is obtained based on the intersection of the ray direction of the pixel reference point and the ground plane constraint equation, and the three-dimensional coordinates of the pixel reference point are obtained. Depth values ​​for non-pixel reference points are obtained using a deep learning-based monocular depth estimation model and corrected using pixel reference points; The final result is three-dimensional point cloud data of the movable hazard; S3: Calculate the coordinates of the center point of the hazard using the bounding box method based on principal component analysis for the 3D point cloud data of the movable hazard according to images from different times; S4: Construct a motion estimation model based on the center point coordinates of the same hidden danger at different times; S5: Calculate the location of the next movable hazard based on the motion prediction model, calculate the three-dimensional distance from the last observed location of the movable hazard and the next predicted location to the tower conductor point cloud, and determine the risk level of the movable hazard accordingly.

[0008] Furthermore, S1 specifically includes: S11: Collect three-dimensional point cloud data of power transmission lines by using a drone equipped with lidar, and install monitoring devices on the power transmission towers to obtain two-dimensional image data of the power transmission lines. The two-dimensional image data includes the power transmission lines, the opposite towers, and the ground. S12: Taking the current tower as the origin and the direction towards the tower as the positive y-axis, for points in the world coordinate system ( Transformed into camera coordinate system through rotation and translation. The conversion formula is: (1); In formula (1), Let i be the position of the i-th point in the 3D point cloud in the world coordinate system after rotation and translation in the camera coordinate system. There are n 3D points in the 3D point cloud. Let be the position of the i-th point in the 3D point cloud of the camera coordinate system along the X, Y, and Z axes; Let R be the position of the i-th point in the 3D point cloud in the world coordinate system along the X, Y, and Z axes; R is the rotation matrix, and T is the translation matrix. Let i be the position of the i-th point in the 3D point cloud in the world coordinate system after rotation and translation in the camera coordinate system; S13: The 3D point cloud data is then converted from the camera coordinate system to a 2D image, i.e., the pixel coordinate system. The formula after the conversion is: (2); In formula (2), It is the pixel coordinate in the horizontal direction of the i-th point in the 3D point cloud after it is projected onto the pixel coordinate system; is the pixel coordinate in the vertical direction of the i-th point in the 3D point cloud after projection onto the pixel coordinate system; W is the camera intrinsic parameter matrix. and The focal lengths of the camera are in the x and y directions; and These are the coordinates of the center point of the image coordinate system.

[0009] Furthermore, S2 specifically includes: S21: The power transmission visualization monitoring device continuously captures multiple images, capturing three or more images at fixed time intervals. The captured images are then used to identify movable hazards through a detection-based instance segmentation model, where the detection-based instance segmentation model is an existing model. S22: Extract ground point clouds from 3D point cloud data and transform them to the camera coordinate system. Use the least squares method to construct a ground plane constraint optimization. The objective equation is: Where (a, b, c) represents the normal vector of the plane, and d represents the intercept of the plane. The optimization objective is: (3); In formula (3), X ic Y ic and Z ic Let N be the position of the i-th point in the 3D point cloud of the camera coordinate system in the X, Y and Z axes, and N be the number of ground point clouds; S23: Take the bottom midpoint of the identified movable hazard bounding box as the pixel reference point, and back-project the reference point to the ground plane. Specifically, construct the ray direction from the pixel to the camera coordinates: (4); In formula (4), m is the number of three-dimensional point clouds of movable hazards; This represents the ray direction of pixel i in the camera coordinate system. Multiply depth This allows you to determine a point in the camera coordinate system; and These are the focal lengths along the camera's x and y axes, respectively. and These represent the positions of the center points in the vertical and horizontal directions of the image coordinate system, respectively. The intersection of the ray direction and the ground plane needs to meet the following requirements. The depth value of the intersection point between the ray and the ground is obtained by combining the ground plane constraint equation. λ That is, the pixel reference point depth value: (5); In formula (5), X ic Y ic Let i be the position of pixel i in the camera coordinate system; the ray direction vector multiplied by the depth value gives the coordinates of the reference pixel in three-dimensional space. The depth values ​​of non-pixel reference points are calculated using a deep learning-based monocular depth estimation model and corrected using pixel reference points; The final result is three-dimensional point cloud data of the movable hazard.

[0010] Furthermore, the correction using pixel reference points specifically involves: calculating the ratio of the pixel reference point depth value to the pixel reference point depth value output by the monocular depth estimation model, and then multiplying it by the depth value of the non-pixel reference point for absolute distance correction, ultimately obtaining the three-dimensional point cloud data of the movable hazard.

[0011] Furthermore, the process of calculating the coordinates of the center point of the movable hazard using the bounding box centering method based on principal component analysis based on the three-dimensional point cloud data of S3 includes: S31: Calculate the center C of the 3D point cloud of a movable hazard; (6); In formula (6), C represents the center of the three-dimensional point cloud of the movable hazard; the new coordinates are obtained by subtracting the center from the coordinates of each point. ,in, The coordinates of the 3D point in the camera coordinate system; S32: The covariance matrix is ​​used to reflect the anisotropy of the point cloud distribution. The formula for calculating the covariance matrix is: (7); Then, perform eigenvalue decomposition on the covariance matrix E to obtain the eigenvalues ​​and eigenvectors of the covariance matrix. These eigenvectors are the principal component directions, forming the principal component coordinate system; S33: Project the decentralized point cloud onto the principal component coordinate system In the principal component coordinate system, calculate the extreme values ​​of each axis, including the maximum and minimum values: (8); In formula (8), , , These are the values ​​of the i-th point in the camera coordinate system on the x-axis, y-axis, and z-axis components of the principal component coordinate system, respectively. and These are the minimum and maximum values ​​of all points in the camera coordinate system on the x-axis component of the principal component coordinate system, respectively. and These are the minimum and maximum values ​​of all points in the camera coordinate system on the y-axis component of the principal component coordinate system, respectively. and These are the minimum and maximum values ​​of all points in the camera coordinate system on the z-axis component of the principal component coordinate system, respectively. In the principal component coordinate system, the center point The midpoint of the extreme values ​​of each axis: (9); Furthermore, , where OBB is the position of the center point in the principal component coordinate system in the original camera coordinate system.

[0012] The motion estimation model described in step S4 is to model the movement trend of the OBB center point calculated in step S3 for the same movable hazard at different times. S41: In real-world scenarios, the speed and direction of motion of movable hazards are usually relatively stable in a short period of time. Therefore, a three-dimensional motion estimation model is established by using polynomial fitting. Based on the center point coordinates of the same hidden danger at different times, a motion estimation model is constructed. Specifically, quadratic functions with time as the variable are constructed for the motion in the x, y, and z axes respectively. (10); In formula (10), t is the capture time, and the first capture time is... The initial value is 1, and subsequent times are based on the initial time plus 1 unit of time; All are coefficients to be determined; S42: Furthermore, the time of the three snapshots... Substituting the x-coordinate of the corresponding OBB center point into formula (10), we get: (11); In formula (11), These represent the x-coordinate components of the hazard center at the three capture times; Similarly, for and Establish this system of equations and use matrix methods to solve for the coefficients of each coordinate axis. For the x-component, we have: (12); Solving the above matrix yields the following results. Similarly, regarding and It can be solved and ; Finally, a motion estimation model for movable hazards can be obtained. .

[0013] Specifically, the risk level determination process for movable hazards described in step S5 is as follows: Calculate the center point coordinates of the last observed location of the movable hazard and the predicted location at the next time point, and use the Euclidean nearest neighbor distance with the conductor as the final hazard distance. If the predicted hazard distance is less than the last observed hazard distance, it means that the movable hazard is approaching the tower line and is a high-risk hazard; if the predicted hazard distance is greater than or equal to the last observed hazard distance, it means that the movable hazard is away from the tower line and is a low-risk hazard.

[0014] In another aspect of the present invention, an apparatus is provided for a method to predict the risk level of ground-based movable hazards based on power transmission channel visualization, the apparatus comprising: At least one processor; and The memory stores instructions that, when executed by the at least one processor, cause the at least one processor to perform the above-described method for predicting the risk level of ground-movable hazards based on power transmission channel visualization.

[0015] In another aspect of the invention, a computer-readable storage medium is also provided, which stores executable instructions that, when executed, cause the machine to perform the method described above for predicting the risk level of ground-based movable hazards based on power transmission channel visualization.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) This invention provides a method, device, and computer-readable storage medium for predicting the risk of mobile hazards on the ground based on the visualization of power transmission channels. This method can construct a mapping from two-dimensional images to three-dimensional spatial coordinates. Based on multiple captured two-dimensional images, it can accurately analyze the movement trajectory of mobile hazards in three-dimensional space, and then scientifically predict the next location of the mobile hazards, assisting in the risk assessment of mobile hazards. This pioneering data application not only greatly reduces the ineffective output of hazard alarms, but also helps operation and maintenance personnel to make timely and effective response strategies, thereby ensuring the safe and stable operation of power transmission lines.

[0017] (2) The present invention provides a method, device and computer-readable storage medium for predicting the risk of ground-based mobile hazards based on the visualization of power transmission channels. It designs a motion estimation model for three-dimensional mobile hazards. In response to the challenges of large differences in the shape and posture of mobile hazards of power transmission lines and the difficulty in determining the three-dimensional center point, it cleverly adopts the bounding box center method based on principal component analysis. The bounding box generated by this method can rotate according to the shape of the object, so that it can more tightly surround the object, thereby ensuring the accuracy and reliability of the calculation of the three-dimensional center point of the mobile hazard, and providing a solid data foundation for subsequent motion estimation analysis. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the movable hidden danger risk prediction method in this invention; Figure 2 This is a schematic diagram of a movable hidden danger at time t1 provided in Embodiment 1 of the present invention; Figure 3 This is a schematic diagram of the movable hidden danger at time t2 provided in Embodiment 1 of the present invention; Figure 4 This is a schematic diagram of a movable hidden danger at time t3 provided in Embodiment 1 of the present invention. Detailed Implementation

[0019] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0020] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0021] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0022] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0023] Example 1 This embodiment provides a method for predicting the risk of ground-based movable hazards based on power transmission channel visualization, such as... Figure 1 As shown, the method includes: Step S1: Acquire 3D point cloud data and 2D image data, and establish a mapping relationship from 3D point cloud to 2D image; The radar acquires three-dimensional point cloud data of the power transmission channel, and the monitoring device acquires two-dimensional image data of the power transmission channel. The camera intrinsic and extrinsic parameter matrices are determined through calibration, and the mapping relationship from the three-dimensional point cloud to the two-dimensional image is determined.

[0024] Preferably, S1 specifically includes: S11: Collect 3D point cloud data of power transmission lines using drones equipped with lidar; install monitoring devices on power transmission towers to acquire 2D image data of power transmission lines, including power transmission lines, opposite towers, and the ground; Preferably, a drone equipped with a lidar is used to perform multi-angle and multi-height flight scans of the target tower line. The lidar emits laser beams and receives reflected signals to generate high-density raw three-dimensional point cloud data. A monitoring device, such as a visible light camera, is installed on the transmission tower to capture two-dimensional images containing the transmission line, the opposite tower, and the ground.

[0025] S12: First, in this example, the camera is installed in the world coordinate system. With a pitch angle of -10° and no yaw or roll, obtain the camera extrinsic parameters, including the rotation matrix R and translation matrix T: (13); (14); Taking the current tower as the origin, and the direction towards the tower as the positive y-axis, for a point in the world coordinate system ( Transformed into camera coordinate system through rotation and translation. The conversion formula is: (1); In formula (1), Let R be the position of the i-th point in the 3D point cloud in the world coordinate system after rotation and translation in the camera coordinate system; R is the rotation matrix; T is the translation matrix. Let i be the position of the i-th point in the 3D point cloud of the world coordinate system in the horizontal direction. Let i be the position of the i-th point in the vertical direction in the 3D point cloud of the world coordinate system; Let i be the position of the i-th point in the 3D point cloud of the world coordinate system in the direction perpendicular to the XY plane; Let be the position of the i-th point in the 3D point cloud of the camera coordinate system in the horizontal direction; Let be the position of the i-th point in the 3D point cloud of the camera coordinate system in the vertical direction; Let i be the position of the i-th point in the 3D point cloud of the world coordinate system in the direction perpendicular to the XY plane; Let i be the position of the i-th point in the 3D point cloud in the world coordinate system after rotation and translation in the camera coordinate system; S13: Preferably, the camera intrinsic parameter matrix W of the power transmission monitoring device after calibration is: (15); This embodiment uses camera intrinsic parameters to convert the parameters of the camera coordinate system to the pixel coordinate system. Specifically, the camera coordinate system is first converted to the image coordinate system, and then the image coordinate system is converted to the pixel coordinate system. The final conversion formula is: (2); In formula (2), It is the pixel coordinate in the horizontal direction of the i-th point in the 3D point cloud after it is projected onto the pixel coordinate system; It is the pixel coordinate in the vertical direction of the i-th point in the 3D point cloud after it is projected onto the pixel coordinate system; S2: Identify movable hazards in images and map them into 3D point cloud data; Using a power transmission line monitoring device, three or more images are captured continuously at fixed time intervals. Movable hazards in each image are identified and mapped into 3D point cloud data. The least squares method is used to fit the plane equation, establishing ground plane constraints. A detection-based instance segmentation model is used to identify movable hazards in the 2D images, such as... Figure 2 , Figure 3 and Figure 4 These are schematic diagrams of movable hazards at time t1, time t2, and time t3, respectively. The diagrams include hazard frames and hazard masks. The center point at the bottom of the hazard frame of the movable hazard is taken as the pixel reference point. The depth value is obtained based on the intersection of the ray direction of the pixel reference point and the ground plane constraint equation, and the three-dimensional coordinates of the pixel reference point are obtained. The depth values ​​of non-pixel reference points are calculated using a deep learning-based monocular depth estimation model. The ratio of the pixel reference point depth value to the pixel reference point depth value output by the monocular depth estimation model is calculated, and this ratio is multiplied by the depth value of the non-pixel reference point for absolute distance correction, ultimately yielding the 3D point cloud data of the movable hazard.

[0026] The principle formula is as follows: Calculated pixel reference point depth value / pixel reference point depth value output by monocular depth estimation model = corrected non-pixel reference point depth value / non-pixel reference point depth value output by monocular depth estimation model.

[0027] S2 specifically includes: S21: The monitoring device captures at least 3 images at fixed 10-minute intervals, such as... Figures 2 to 4 As shown; identify movable hazards in the image and map them into 3D point cloud data; Preferably, a detection-based instance segmentation model is used to identify movable hazard pixels, and the three-dimensional world coordinates of the hazard are calculated in reverse according to the mapping relationship in step S1.

[0028] S22: In this embodiment, ground point cloud is extracted from the point cloud data in step S1. The ground point cloud coordinates are as follows: (16); Transform it to the camera coordinate system, construct ground plane constraints using the least squares method, and the objective equation is: Where (a, b, c) represents the normal vector of the plane, and d represents the intercept of the plane. The optimization objective is: (3); In formula (3), Xic, Yic and Zic are the positions of the i-th point in the 3D point cloud of the camera coordinate system in the X, Y and Z axis directions, and N is the number of ground point clouds.

[0029] Finally, after fitting the ground point cloud data, the ground plane constraint equation is obtained: (17).

[0030] S23: In this embodiment, the center point (1912, 215) at the bottom of the movable hazard frame in the image acquired at time t1 is taken as the pixel reference point, and the ray direction from the pixel to the camera coordinates is: (4); In formula (4), and These are the focal lengths along the camera's x and y axes, respectively. and These represent the positions of the center points in the vertical and horizontal directions of the image coordinate system, respectively.

[0031] The intersection of the ray direction and the ground plane needs to meet the following requirements. By combining the ground plane constraint equation, the depth value of the intersection point of the ray and the ground is obtained, which is the pixel reference point depth value: (5); Therefore, the corresponding position in the camera coordinate system can be obtained as (11.866, -19.235, 73.5). The ray direction vector multiplied by the depth value is the coordinate of the reference pixel in three-dimensional space. The depth values ​​of non-pixel reference points are calculated using a deep learning-based monocular depth estimation model. Then, the ratio of the pixel reference point depth value to the pixel reference point depth value output by the monocular depth estimation model is calculated, and this ratio is multiplied by the depth value of the non-pixel reference point for absolute distance correction, i.e., the correction of the depth value of the non-pixel reference point. The deep learning-based monocular depth estimation model includes models such as Depth Anything and ZoeDepth. The final result is three-dimensional point cloud data of the movable hazard.

[0032] In this embodiment, the movable hazard pixels of the two-dimensional images at time points t1, t2, and t3 are calculated in the camera coordinate system as follows: (18); (19); (20).

[0033] S3: Calculate the center coordinates of the movable hazard point cloud using the bounding box of principal component analysis; The coordinates of the center point of the movable hazard were calculated using the bounding box centering method based on principal component analysis for the 3D point cloud data of the hazard based on images from different time periods; specifically including: S31: Calculate the center coordinates of the movable hazard point cloud using the bounding box of principal component analysis; The center of the 3D point cloud for calculating movable hazards is: (6); (twenty one); (twenty two); (twenty three); Subtract the center from the coordinates of each point to get the new coordinates: (twenty four); (25); (26).

[0034] S32: The covariance matrix is ​​used to reflect the anisotropy of the point cloud distribution. The formula for calculating the covariance matrix is: (7); Specifically, the covariance matrices are obtained as follows: (27); (28); (29).

[0035] Furthermore, eigenvalues ​​and corresponding eigenvectors are obtained by performing eigendecomposition on the covariance matrix E. These eigenvectors, which are the principal component directions, constitute a new coordinate system. Taking E1 as an example, the eigenvectors of the E1 matrix ,in: (30); Feature vector These represent the directions of the x, y, and z axes in the principal component coordinate system, respectively; similarly, the coordinate axes in the principal component coordinate systems of E2 and E3 can be calculated.

[0036] S33: Map the coordinates of the decentered point cloud to the principal component coordinate system. The transformation formula is as follows: Therefore, there is (31); (32); (33); In the principal component coordinate system, calculate the extreme values ​​(maximum and minimum values) of each axis of the point cloud data: (8); In formula (8), , , These are the values ​​of the i-th point in the camera coordinate system on the x-axis, y-axis, and z-axis components of the principal component coordinate system, respectively. and These are the minimum and maximum values ​​of all points in the camera coordinate system on the x-axis component of the principal component coordinate system, respectively. and These are the minimum and maximum values ​​of all points in the camera coordinate system on the y-axis component of the principal component coordinate system, respectively. and These are the minimum and maximum values ​​of all points in the camera coordinate system on the z-axis component of the principal component coordinate system, respectively. In the principal component coordinate system, the midpoint of the extreme values ​​of each axis is taken as the center point of the hidden danger, denoted as . : (9); Furthermore, through The calculated center coordinates of the movable hazard in the principal component coordinate system are: (34); (35); (36); C represents the center of the three-dimensional point cloud of movable potential hazards.

[0037] S4: Construct a motion estimation model based on the coordinates of the center point of the hidden danger at different times calculated in step S3; S41: Construct a motion estimation model based on the center point coordinates of the same hidden danger at different times: (10); In formula (10), t is the capture time, and the first capture time is... The initial value is 1, and subsequent times are incremented by 1 based on the initial value; All are coefficients to be determined; Specifically, taking three images at times t1, t2, and t3 as examples, quadratic functions with time as the variable are constructed for the motion in the x, y, and z axes respectively: (37); (38); (39); S42: Furthermore, the time of the three snapshots... Substituting the x-coordinate of the corresponding OBB center point into the above equation (10): (11) In formula (11), These represent the x-coordinate components of the hazard center at the three capture times.

[0038] Solving the above matrix yields the following results. Similarly, for and Establish this system of equations and use matrix methods to solve for the coefficients of each coordinate axis. For the x-component, we have: (12); Similarly, regarding and It can be solved and ; Finally, a motion estimation model for movable hazards can be obtained: .

[0039] S5: Determine the risk level of movable hazards; The location of the next movable hazard is calculated based on the motion prediction model. The three-dimensional distance from the last observed location of the movable hazard and the next predicted location to the tower conductor point cloud is calculated, and the risk level of the movable hazard is determined accordingly.

[0040] In step S5, the next location of the hazard center point is predicted based on the motion estimation model; the Euclidean nearest neighbor distance from the last center point location and the predicted center point location to the conductor is compared to determine the risk level of the movable hazard.

[0041] Specifically, based on the motion estimation model, the predicted center location of the movable hazard at the next time step is: ; Calculate separately and The Euclidean nearest neighbor distances to the conductor's three-dimensional coordinates are 26.23 and 25.34. The movement trajectory of the movable hazard is closer to the tower line, so it should be classified as a high-risk hazard; otherwise, it should be classified as a low-risk hazard.

[0042] Specifically, the risk level determination process for movable hazards described in step S5 is as follows: Calculate the center point coordinates of the last observed location of the movable hazard and the predicted location at the next time point, and use the Euclidean nearest neighbor distance with the conductor as the final hazard distance. If the predicted hazard distance is less than the hazard distance observed in the last time, it means that the movable hazard is approaching the tower and line, which is a high-risk hazard; conversely, if the distance increases, it means that the movable hazard is moving away from the tower and line, which is a low-risk hazard.

[0043] Example 2 This embodiment provides an apparatus for implementing a method for predicting ground-based movable hazard risks based on power transmission channel visualization. The apparatus includes: At least one processor; and The memory stores instructions that, when executed by the at least one processor, cause the at least one processor to perform a ground-based mobile hazard risk prediction method based on power transmission channel visualization as described above.

[0044] In this embodiment, electronic devices include, but are not limited to: personal computers, server computers, workstations, desktop computers, laptop computers, notebook computers, mobile computing devices, smartphones, tablet computers, cellular phones, personal digital assistants (PDAs), handheld devices, messaging devices, wearable computing devices, consumer electronic devices, etc.

[0045] Example 3 This embodiment also provides a computer-readable storage medium storing executable instructions, which, when executed, cause the machine to perform the above-described method for predicting ground-based mobile hazard risks based on power transmission channel visualization.

[0046] Specifically, a system or apparatus equipped with a readable storage medium may be provided, on which software program code implementing the functions of any of the embodiments described above is stored, and the computer or processor of the system or apparatus can read and execute the instructions stored in the readable storage medium.

[0047] In this case, the program code itself, which can be read from the readable medium, can perform the functions of any of the above embodiments, and therefore the computer-readable code and the readable storage medium storing the computer-readable code constitute a part of this specification.

[0048] Examples of readable storage media include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer or the cloud via a communication network.

[0049] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0050] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the technical solutions of the present invention, and are not intended to limit the specific implementation of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the claims of the present invention should be included within the protection scope of the claims of the present invention.

Claims

1. A method for predicting the risk of ground-based movable hazards based on power transmission channel visualization, characterized in that, The method includes: S1: Acquire 3D point cloud data and 2D image data of the power transmission channel, determine the camera intrinsic parameter matrix and camera extrinsic parameter matrix through calibration, and determine the mapping relationship from 3D point cloud to 2D image; S2: After the monitoring device captures an image, it identifies movable hazards in the image and uses the least squares method to fit the plane equation to establish ground plane constraints. The center point of the bottom of the hazard frame of the movable hazard is taken as the pixel reference point. The depth value is obtained based on the intersection of the ray direction of the pixel reference point and the ground plane constraint equation, and the three-dimensional coordinates of the pixel reference point are obtained. Depth values ​​for non-pixel reference points are obtained using a deep learning-based monocular depth estimation model and corrected using pixel reference points; The final result is three-dimensional point cloud data of the movable hazard; S2 specifically includes: S21: The power transmission visualization monitoring device continuously captures multiple images, capturing three or more images at fixed time intervals, and uses a detection-based instance segmentation model to identify movable hidden dangers from the captured images. S22: Extract ground point clouds from 3D point cloud data and transform them to the camera coordinate system. Use the least squares method to construct a ground plane constraint optimization. The objective equation is: Where (a, b, c) represents the normal vector of the plane, and d represents the intercept of the plane. The optimization objective is: （3）； In formula (3), X ic Y ic and Z ic Let N be the position of the i-th point in the 3D point cloud of the camera coordinate system in the X, Y and Z axes, and N be the number of ground point clouds; S23: Take the bottom midpoint of the identified movable hazard bounding box as the pixel reference point, and back-project the reference point to the ground plane. Specifically, construct the ray direction from the pixel to the camera coordinates: （4）； In formula (4), m is the number of three-dimensional point clouds of movable hazards; Represents the ray direction of pixel i in the camera coordinate system; and These are the focal lengths along the camera's x and y axes, respectively. and These represent the positions of the center points in the vertical and horizontal directions of the image coordinate system, respectively. The intersection of the ray direction and the ground plane needs to meet the following requirements. The depth value of the intersection point between the ray and the ground is obtained by combining the ground plane constraint equation. λ : （5）； In formula (5), X ic Y ic Let i be the position of pixel i in the camera coordinate system. The ray direction vector multiplied by the depth value is the coordinate of the reference pixel in three-dimensional space. The depth values ​​of non-pixel reference points are calculated using a deep learning-based monocular depth estimation model and corrected using pixel reference points; The final result is three-dimensional point cloud data of the movable hazard; S3: Calculate the coordinates of the center point of the hazard using the bounding box centering method based on principal component analysis for the 3D point cloud data of movable hazards; S4: Construct a motion estimation model based on the center point coordinates of the same hidden danger at different times; S5: Predict the location of the next hazard center point based on the motion estimation model, compare the Euclidean nearest neighbor distance from the last observed hazard center point location to the predicted center point location to determine the risk level of the movable hazard; The method for determining the risk level of movable hazards includes: Calculate the center point coordinates of the last observed location of the movable hazard and the predicted location at the next time point, and use the Euclidean nearest neighbor distance with the conductor as the final hazard distance. If the predicted hazard distance is less than the hazard distance observed in the last time, it indicates that the movable hazard is approaching the tower line and is a high-risk hazard. When the predicted distance to a potential hazard is greater than or equal to the distance of the last observed hazard, it indicates that the movable hazard is located far from the pole or line and is considered a low-risk hazard.

2. The method for predicting ground-based movable hazard risks based on power transmission channel visualization according to claim 1, characterized in that, S1 specifically includes: S11: Collect three-dimensional point cloud data of power transmission lines by using a drone equipped with lidar, and install monitoring devices on the power transmission towers to obtain two-dimensional image data of the power transmission lines. The two-dimensional image data includes the power transmission lines, the opposite towers, and the ground. S12: Taking the current tower as the origin and the direction towards the tower as the positive y-axis, for points in the world coordinate system ( Transformed into camera coordinate system through rotation and translation. The conversion formula is: （1）； In formula (1), Let i be the position of the i-th point in the 3D point cloud in the world coordinate system after rotation and translation in the camera coordinate system. There are n 3D points in the 3D point cloud. Let be the position of the i-th point in the 3D point cloud of the camera coordinate system along the X, Y, and Z axes; Let R be the position of the i-th point in the 3D point cloud in the world coordinate system along the X, Y, and Z axes; R is the rotation matrix; and T is the translation matrix. Let i be the position of the i-th point in the 3D point cloud in the world coordinate system after rotation and translation in the camera coordinate system; S13: The 3D point cloud data is then converted from the camera coordinate system to a 2D image, i.e., the pixel coordinate system. The formula after the conversion is: （2）； In formula (2), It is the pixel coordinate in the horizontal direction of the i-th point in the 3D point cloud after it is projected onto the pixel coordinate system; is the pixel coordinate in the vertical direction of the i-th point in the 3D point cloud after projection onto the pixel coordinate system; W is the camera intrinsic parameter matrix. and The focal lengths of the camera are in the x and y directions; and These are the coordinates of the center point of the image coordinate system.

3. The method for predicting ground-based movable hazard risks based on power transmission channel visualization according to claim 1, characterized in that, The bounding box center method based on principal component analysis is used to calculate the coordinates of the hazard center point, which includes: S31: Calculate the center C of the 3D point cloud of a movable hazard; （6）； In formula (6), the new coordinates of each point are obtained by subtracting the center from the original coordinates. ,in, The coordinates of the 3D point in the camera coordinate system; S32: The covariance matrix is ​​used to reflect the anisotropy of the point cloud distribution. The formula for calculating the covariance matrix is: （7）； Perform eigenvalue decomposition on the covariance matrix E to obtain eigenvalues ​​and corresponding eigenvectors. These eigenvectors are the principal component directions, forming a new coordinate system; S33: Project the decentralized point cloud onto the principal component coordinate system In the principal component coordinate system, calculate the extreme values ​​of each axis: （8）； In formula (8), , , These are the values ​​of the i-th point in the camera coordinate system on the x-axis, y-axis, and z-axis components of the principal component coordinate system, respectively. and These are the minimum and maximum values ​​of all points in the camera coordinate system on the x-axis component of the principal component coordinate system, respectively. and These are the minimum and maximum values ​​of all points in the camera coordinate system on the y-axis component of the principal component coordinate system, respectively. and These are the minimum and maximum values ​​of all points in the camera coordinate system on the z-axis component of the principal component coordinate system, respectively. In the principal component coordinate system, the center point The midpoint of the extreme values ​​of each axis: （9）； Transform the center point in the principal component coordinate system to the original camera coordinate system. OBB is the position of the center point in the principal component coordinate system in the original camera coordinate system. 。 4. The method for predicting ground-based movable hazard risks based on power transmission channel visualization according to claim 1, characterized in that, The method of constructing a motion estimation model based on the center point coordinates of the same hidden danger at different times includes: S41: A three-dimensional motion estimation model is established using polynomial fitting, constructing quadratic functions with time as the variable for the motion in the x, y, and z axes respectively: （10）； In formula (10), t is the capture time, and the first capture time is... The initial value is 1, and subsequent times are based on the initial time plus 1 unit of time. All are coefficients to be determined; S42: The time of the three snapshots Substituting the x-value of the corresponding OBB center point coordinates into formula (10), we get: （11）； In formula (11), These represent the x-coordinate components of the hazard center at the three capture times; Similarly, for and Establish a system of similar equations and use matrix methods to solve for the coefficients of each coordinate axis. For the x-component, we have: （12）； Solving the above matrix yields the following results. Similarly, regarding and It can be solved and ; Finally, a motion estimation model for movable hazards can be obtained. .

5. The method for predicting ground-based movable hazard risks based on power transmission channel visualization according to claim 1, characterized in that, The specific method for correction using pixel reference points is as follows: calculate the ratio of the pixel reference point depth value to the pixel reference point depth value output by the monocular depth estimation model, and then multiply it by the depth value of a non-pixel reference point to perform absolute distance correction.

6. A ground-based mobile hazard risk prediction device based on power transmission channel visualization, characterized in that, The device includes: processor; A memory on which computer programs that can run on the processor are stored; When the computer program is executed by the processor, it implements the steps of the method for predicting ground-based mobile hazard risks based on power transmission channel visualization as described in any one of claims 1 to 5.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 5.

Images