Ground movable hidden danger risk prediction method and device based on power transmission channel visualization and computer readable storage medium

Abstract

The invention belongs to the technical field of intelligent operation inspection of power transmission lines, and particularly relates to a ground movable hidden danger risk prediction method based on visualization of a power transmission channel. The method comprises the following steps: acquiring three-dimensional point cloud data and two-dimensional image data of a tower line; determining external parameters and internal parameters of the camera; establishing a mapping relation between the two-dimensional image and the three-dimensional point cloud; the monitoring and shooting device captures more than three images at a fixed time interval; identifying movable hidden dangers in each image and mapping the movable hidden dangers into three-dimensional point cloud data; calculating a center coordinate of the movable hidden danger point cloud; constructing a motion estimation model according to the center point coordinates of the same hidden danger at different moments, and predicting the center point position of the next hidden danger; and comparing the Euclidean nearest neighbor distance from the hidden danger center point position observed at the last time and the predicted center point position to the lead, and determining the movable hidden danger risk level. The problems of high hidden danger output rate of the power transmission line, low closed loop disposal efficiency and movable hidden danger risk prediction are solved.

Description

Technical Field

[0001] This invention belongs to the field of intelligent operation and maintenance of power transmission lines, and more specifically, relates to a method, device and computer-readable storage medium for predicting ground-based movable hidden danger risks based on power transmission channel visualization. Background Technology

[0002] With the deep application of artificial intelligence technology, intelligent monitoring systems for power transmission lines based on intelligent image recognition and on-site hazard analysis algorithms have become an important technical means to promote the intelligent transformation of power inspection work. Against the backdrop of continuously upgrading demands for electricity quality in people's lives, the power inspection system is gradually evolving towards multi-dimensional monitoring, with the accurate identification and risk assessment of complex hazards becoming a key direction in this technological evolution.

[0003] Chinese patent document CN115995058A discloses an artificial intelligence-based online monitoring method for power transmission channel safety. The method involves acquiring surveillance images and laser point cloud data of the power transmission channel, preprocessing the images, obtaining coordinate transformation relationships using camera calibration and attitude estimation algorithms, constructing a single-stage detector, and training and adjusting the model. Using the target detection results and coordinate transformation relationships, the minimum distance from the target object to the power line is calculated. Finally, based on the safety distance regulations for overhead lines in the power grid company's enterprise standards, the safety level is determined, and timely warnings are issued for the target object.

[0004] It is worth noting that current mainstream systems primarily rely on two-dimensional image analysis for hazard identification. While this significantly improves power grid safety monitoring capabilities, the generation of massive amounts of visualized alarm data has revealed a significant discrepancy between the efficiency of hazard handling and the scale of data output. This leads to increased maintenance costs and an imbalance in resource allocation. Further analysis reveals that some movable hazards within the field of view, such as construction machinery, do not pose a substantial risk due to their movement trajectory or spatial position not interacting with transmission lines. Therefore, their safety threat level requires dynamic assessment. However, in the field of transmission line safety assessment, systematic research on dynamic risk prediction for movable hazards is still lacking.

[0005] In view of this, how to provide an efficient and reliable method for predicting the risk level of movable hazards in transmission lines, so as to achieve quantitative prediction and graded early warning of the risk level of movable hazards, effectively improve the effectiveness of alarm information and optimize the allocation of operation and maintenance resources, has become a technical bottleneck that the industry urgently needs to overcome. Summary of the Invention

[0006] This invention aims to overcome at least one of the defects of the prior art and provides a method for predicting the risk level of ground-based movable hazards based on power transmission channel visualization. By mapping the movable hazards from multiple consecutive two-dimensional images to coordinates in three-dimensional space, a three-dimensional spatial motion estimation model is constructed. At the same time, combined with the three-dimensional coordinates of the towers and lines, the problem of predicting the risk level of movable hazards is solved.

[0007] The detailed technical solution of this invention is as follows: A method for predicting the risk level of ground-based movable hazards based on power transmission channel visualization, the method comprising: S1: Acquire 3D point cloud data and 2D image data of the power transmission channel, determine camera intrinsic and extrinsic parameters through calibration, and determine the mapping relationship from 3D point cloud to 2D image; S2: After the monitoring device captures an image, it identifies movable hazards in the image and uses the least squares method to fit the plane equation to establish ground plane constraints. The center point at the bottom of the movable hazard frame is taken as the pixel reference point. The depth value is obtained based on the intersection of the ray direction of the pixel reference point and the ground plane constraint equation, and the three-dimensional coordinates of the pixel reference point are obtained. Depth values ​​for non-pixel reference points are obtained using a deep learning-based monocular depth estimation model and corrected using pixel reference points; Finally, three-dimensional point cloud data of the movable hazard was obtained; S3: Calculate the coordinates of the center point of the hazard using the bounding box method based on principal component analysis for the 3D point cloud data of the movable hazard according to images from different times; S4: Construct a motion estimation model based on the center point coordinates of the same hidden danger at different times; S5: Calculate the location of the next movable hazard based on the motion prediction model, calculate the three-dimensional distance from the last observed location of the movable hazard and the next predicted location to the tower conductor point cloud, and determine the risk level of the movable hazard accordingly.

[0008] Furthermore, S1 specifically includes: S11: Collect three-dimensional point cloud data of power transmission lines by using a drone equipped with lidar, and install monitoring devices on the power transmission towers to obtain two-dimensional image data of the power transmission lines. The two-dimensional image data includes the power transmission lines, the opposite towers, and the ground. S12: Taking the current tower as the origin and the direction towards the tower as the positive y-axis, for points in the world coordinate system ( Transformed into camera coordinate system through rotation and translation. The conversion formula is: (1); In formula (1), Let i be the position of the i-th point in the 3D point cloud in the world coordinate system after rotation and translation in the camera coordinate system. There are n 3D points in the 3D point cloud. Let be the position of the i-th point in the 3D point cloud of the camera coordinate system along the X, Y, and Z axes; Let R be the position of the i-th point in the 3D point cloud in the world coordinate system along the X, Y, and Z axes; R is the rotation matrix, and T is the translation matrix. Let i be the position of the i-th point in the 3D point cloud in the world coordinate system after rotation and translation in the camera coordinate system; S13: The 3D point cloud data is then converted from the camera coordinate system to a 2D image, i.e., the pixel coordinate system. The formula after the conversion is: (2); In formula (2), It is the pixel coordinate in the horizontal direction of the i-th point in the 3D point cloud after it is projected onto the pixel coordinate system; is the pixel coordinate in the vertical direction of the i-th point in the 3D point cloud after projection onto the pixel coordinate system; W is the camera intrinsic parameter matrix. and Let x be the focal length of the camera in the x and y directions; and These are the coordinates of the center point of the image coordinate system.

[0009] Furthermore, S2 specifically includes: S21: The power transmission visualization monitoring device continuously captures multiple images, capturing three or more images at fixed time intervals. The captured images are then used to identify movable hazards through a detection-based instance segmentation model, where the detection-based instance segmentation model is an existing model. S22: Extract ground point clouds from 3D point cloud data and transform them to the camera coordinate system. Use the least squares method to construct a ground plane constraint optimization. The objective equation is: Where (a, b, c) represents the normal vector of the plane, and d represents the intercept of the plane. The optimization objective is: (3); In formula (3), X ic Y ic and Z ic Let N be the position of the i-th point in the 3D point cloud of the camera coordinate system in the X, Y and Z axes, and N be the number of ground point clouds; S23: Take the bottom midpoint of the identified movable hazard bounding box as the pixel reference point, and back-project the reference point to the ground plane. Specifically, construct the ray direction from the pixel to the camera coordinates: (4); In formula (4), m is the number of three-dimensional point clouds of movable hazards; This represents the ray direction of pixel i in the camera coordinate system. Multiply depth This allows you to determine a point in the camera coordinate system; and These are the focal lengths along the camera's x and y axes, respectively. and These represent the positions of the center points in the vertical and horizontal directions of the image coordinate system, respectively. The intersection of the ray direction and the ground plane needs to meet the following requirements. The depth value of the intersection point between the ray and the ground is obtained by combining the ground plane constraint equation. λ That is, the pixel reference point depth value: (5); In formula (5), X ic Y ic Let i be the position of pixel i in the camera coordinate system; the ray direction vector multiplied by the depth value gives the coordinates of the reference pixel in three-dimensional space. The depth values ​​of non-pixel reference points are calculated using a deep learning-based monocular depth estimation model and corrected using pixel reference points; The final result is three-dimensional point cloud data of the movable hazard.

[0010] Furthermore, the correction using pixel reference points specifically involves: calculating the ratio of the pixel reference point depth value to the pixel reference point depth value output by the monocular depth estimation model, and then multiplying it by the depth value of the non-pixel reference point for absolute distance correction, ultimately obtaining the three-dimensional point cloud data of the movable hazard.

[0011] Furthermore, the process of calculating the coordinates of the center point of the movable hazard using the bounding box centering method based on principal component analysis based on the three-dimensional point cloud data of S3 includes: S31: Calculate the center C of the 3D point cloud of a movable hazard; (6); In formula (6), C represents the center of the three-dimensional point cloud of the movable hazard; the new coordinates are obtained by subtracting the center from the coordinates of each point. ,in, The coordinates of the 3D point in the camera coordinate system; S32: The covariance matrix is ​​used to reflect the anisotropy of the point cloud distribution. The formula for calculating the covariance matrix is: (7); Then, perform eigenvalue decomposition on the covariance matrix E to obtain the eigenvalues ​​and eigenvectors of the covariance matrix. These eigenvectors are the principal component directions, forming the principal component coordinate system; S33: Project the decentralized point cloud onto the principal component coordinate system In the principal component coordinate system, calculate the extreme values ​​of each axis, including the maximum and minimum values: (8); In formula (8), , , These are the values ​​of the i-th point in the camera coordinate system on the x-axis, y-axis, and z-axis components of the principal component coordinate system, respectively. and These are the minimum and maximum values ​​of all points in the camera coordinate system on the x-axis component of the principal component coordinate system, respectively. and These are the minimum and maximum values ​​of all points in the camera coordinate system on the y-axis component of the principal component coordinate system, respectively. and These are the minimum and maximum values ​​of all points in the camera coordinate system on the z-axis component of the principal component coordinate system, respectively. In the principal component coordinate system, the center point The midpoint of the extreme values ​​of each axis: (9); Furthermore, , where OBB is the position of the center point in the principal component coordinate system in the original camera coordinate system.

[0012] The motion estimation model described in step S4 is to model the movement trend of the OBB center point calculated in step S3 for the same movable hazard at different times. S41: In real-world scenarios, the speed and direction of motion of movable hazards are usually relatively stable in a short period of time. Therefore, a three-dimensional motion estimation model is established by using polynomial fitting. Based on the center point coordinates of the same hidden danger at different times, a motion estimation model is constructed. Specifically, quadratic functions with time as the variable are constructed for the motion in the x, y, and z axes respectively. (10); In formula (10), t is the capture time, and the first capture time is... The initial value is 1, and subsequent times are based on the initial time plus 1 unit of time; All are coefficients to be determined; S42: Furthermore, the time of the three snapshots... Substituting the x-coordinate of the corresponding OBB center point into formula (10), we get: (11); In formula (11), These represent the x-coordinate components of the hazard center at the three capture times; Similarly, for and Establish this system of equations and use matrix methods to solve for the coefficients of each coordinate axis. For the x-component, we have: (12); Solving the above matrix yields the following results. Similarly, regarding and It can be solved and ; Finally, a motion estimation model for movable hazards can be obtained. .

[0013] Specifically, the risk level determination process for movable hazards described in step S5 is as follows: Calculate the center point coordinates of the last observed location of the movable hazard and the predicted location at the next time point, and use the Euclidean nearest neighbor distance with the conductor as the final hazard distance. If the predicted hazard distance is less than the last observed hazard distance, it means that the movable hazard is approaching the tower line and is a high-risk hazard; if the predicted hazard distance is greater than or equal to the last observed hazard distance, it means that the movable hazard is away from the tower line and is a low-risk hazard.

[0014] In another aspect of the present invention, an apparatus is provided for a method to predict the risk level of ground-based movable hazards based on power transmission channel visualization, the apparatus comprising: At least one processor; and The memory stores instructions that, when executed by the at least one processor, cause the at least one processor to perform the above-described method for predicting the risk level of ground-movable hazards based on power transmission channel visualization.

[0015] In another aspect of the invention, a computer-readable storage medium is also provided, which stores executable instructions that, when executed, cause the machine to perform the method described above for predicting the risk level of ground-based movable hazards based on power transmission channel visualization.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) This invention provides a method, device, and computer-readable storage medium for predicting the risk of mobile hazards on the ground based on the visualization of power transmission channels. This method can construct a mapping from two-dimensional images to three-dimensional spatial coordinates. Based on multiple captured two-dimensional images, it can accurately analyze the movement trajectory of mobile hazards in three-dimensional space, and then scientifically predict the next location of the mobile hazards, assisting in the risk assessment of mobile hazards. This pioneering data application not only greatly reduces the ineffective output of hazard alarms, but also helps operation and maintenance personnel to make timely and effective response strategies, thereby ensuring the safe and stable operation of power transmission lines.

[0017] (2) The present invention provides a method, device and computer-readable storage medium for predicting the risk of ground-based mobile hazards based on the visualization of power transmission channels. It designs a motion estimation model for three-dimensional mobile hazards. In response to the challenges of large differences in the shape and posture of mobile hazards of power transmission lines and the difficulty in determining the three-dimensional center point, it cleverly adopts the bounding box center method based on principal component analysis. The bounding box generated by this method can rotate according to the shape of the object, so that it can more tightly surround the object, thereby ensuring the accuracy and reliability of the calculation of the three-dimensional center point of the mobile hazard, and providing a solid data foundation for subsequent motion estimation analysis. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the movable hidden danger risk prediction method in this invention; Figure 2 This is a schematic diagram of a movable hidden danger at time t1 provided in Embodiment 1 of the present invention; Figure 3 This is a schematic diagram of the movable hidden danger at time t2 provided in Embodiment 1 of the present invention; Figure 4 This is a schematic diagram of a movable hidden danger at time t3 provided in Embodiment 1 of the present invention. Detailed Implementation

[0019] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0020] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0021] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0022] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0023] Example 1 This embodiment provides a method for predicting the risk of ground-based movable hazards based on power transmission channel visualization, such as... Figure 1 As shown, the method includes: Step S1: Acquire 3D point cloud data and 2D image data, and establish a mapping relationship from 3D point cloud to 2D image; The radar acquires three-dimensional point cloud data of the power transmission channel, and the monitoring device acquires two-dimensional image data of the power transmission channel. The camera intrinsic and extrinsic parameter matrices are determined through calibration, and the mapping relationship from the three-dimensional point cloud to the two-dimensional image is determined.

[0024] Preferably, S1 specifically includes: S11: Collect 3D point cloud data of power transmission lines using drones equipped with lidar; install monitoring devices on power transmission towers to acquire 2D image data of power transmission lines, including power transmission lines, opposite towers, and the ground; Preferably, a drone equipped with a lidar is used to perform multi-angle and multi-height flight scans of the target tower line. The lidar emits laser beams and receives reflected signals to generate high-density raw three-dimensional point cloud data. A monitoring device, such as a visible light camera, is installed on the transmission tower to capture two-dimensional images containing the transmission line, the opposite tower, and the ground.

[0025] S12: First, in this example, the camera is installed in the world coordinate system. With a pitch angle of -10° and no yaw or roll, obtain the camera extrinsic matrix, including the rotation matrix R and the translation matrix T: (13); (14); Taking the current tower as the origin, and the direction towards the tower as the positive y-axis, for a point in the world coordinate system ( Transformed into camera coordinate system through rotation and translation. The conversion formula is: (1); In formula (1), Let R be the position of the i-th point in the 3D point cloud in the world coordinate system after rotation and translation in the camera coordinate system; R is the rotation matrix; T is the translation matrix. Let i be the position of the i-th point in the 3D point cloud of the world coordinate system in the horizontal direction. Let i be the position of the i-th point in the vertical direction in the 3D point cloud of the world coordinate system; Let i be the position of the i-th point in the 3D point cloud of the world coordinate system in the direction perpendicular to the XY plane; Let be the position of the i-th point in the 3D point cloud of the camera coordinate system in the horizontal direction; Let be the position of the i-th point in the 3D point cloud of the camera coordinate system in the vertical direction; Let i be the position of the i-th point in the 3D point cloud of the world coordinate system in the direction perpendicular to the XY plane; Let i be the position of the i-th point in the 3D point cloud in the world coordinate system after rotation and translation in the camera coordinate system; S13: Preferably, the camera intrinsic parameter matrix W of the power transmission monitoring device after calibration is: (15); This embodiment uses camera intrinsic parameters to convert the parameters of the camera coordinate system to the pixel coordinate system. Specifically, the camera coordinate system is first converted to the image coordinate system, and then the image coordinate system is converted to the pixel coordinate system. The final conversion formula is: (2); In formula (2), It is the pixel coordinate in the horizontal direction of the i-th point in the 3D point cloud after it is projected onto the pixel coordinate system; It is the pixel coordinate in the vertical direction of the i-th point in the 3D point cloud after it is projected onto the pixel coordinate system; S2: Identify movable hazards in images and map them into 3D point cloud data; Using a power transmission line monitoring device, three or more images are captured continuously at fixed time intervals. Movable hazards in each image are identified and mapped into 3D point cloud data. The least squares method is used to fit the plane equation, establishing ground plane constraints. A detection-based instance segmentation model is used to identify movable hazards in the 2D images, such as... Figure 2 , Figure 3 and Figure 4 These are schematic diagrams of movable hazards at time t1, time t2, and time t3, respectively. The diagrams include hazard frames and hazard masks. The center point at the bottom of the hazard frame of the movable hazard is taken as the pixel reference point. The depth value is obtained based on the intersection of the ray direction of the pixel reference point and the ground plane constraint equation, and the three-dimensional coordinates of the pixel reference point are obtained. The depth values ​​of non-pixel reference points are calculated using a deep learning-based monocular depth estimation model. The ratio of the pixel reference point depth value to the pixel reference point depth value output by the monocular depth estimation model is calculated, and this ratio is multiplied by the depth value of the non-pixel reference point for absolute distance correction, ultimately yielding the 3D point cloud data of the movable hazard.

[0026] The principle formula is as follows: Calculated pixel reference point depth value / pixel reference point depth value output by monocular depth estimation model = corrected non-pixel reference point depth value / non-pixel reference point depth value output by monocular depth estimation model.

[0027] S2 specifically includes: S21: The monitoring device captures at least 3 images at fixed 10-minute intervals, such as... Figures 2 to 4 As shown; identify movable hazards in the image and map them into 3D point cloud data; Preferably, a detection-based instance segmentation model is used to identify movable hazard pixels, and the three-dimensional world coordinates of the hazard are calculated in reverse according to the mapping relationship in step S1.

[0028] S22: In this embodiment, ground point cloud is extracted from the point cloud data in step S1. The ground point cloud coordinates are as follows: (16); Transform it to the camera coordinate system, construct ground plane constraints using the least squares method, and the objective equation is: Where (a, b, c) represents the normal vector of the plane, and d represents the intercept of the plane. The optimization objective is: (3); In formula (3), Xic, Yic and Zic are the positions of the i-th point in the 3D point cloud of the camera coordinate system in the X, Y and Z axis directions, and N is the number of ground point clouds.

[0029] Finally, after fitting the ground point cloud data, the ground plane constraint equation is obtained: (17).

[0030] S23: In this embodiment, the center point (1912, 215) at the bottom of the movable hazard frame in the image acquired at time t1 is taken as the pixel reference point, and the ray direction from the pixel to the camera coordinates is: (4); In formula (4), and These are the focal lengths along the camera's x and y axes, respectively. and These represent the positions of the center points in the vertical and horizontal directions of the image coordinate system, respectively.

[0031] The intersection of the ray direction and the ground plane needs to meet the following requirements. By combining the ground plane constraint equation, the depth value of the intersection point of the ray and the ground is obtained, which is the pixel reference point depth value: (5); Therefore, the corresponding position in the camera coordinate system can be obtained as (11.866, -19.235, 73.5). The ray direction vector multiplied by the depth value is the coordinate of the reference pixel in three-dimensional space. The depth values ​​of non-pixel reference points are calculated using a deep learning-based monocular depth estimation model. Then, the ratio of the pixel reference point depth value to the pixel reference point depth value output by the monocular depth estimation model is calculated, and this ratio is multiplied by the depth value of the non-pixel reference point for absolute distance correction, i.e., the correction of the depth value of the non-pixel reference point. The deep learning-based monocular depth estimation model includes models such as Depth Anything and ZoeDepth. The final result is three-dimensional point cloud data of the movable hazard.

[0032] In this embodiment, the movable hazard pixels of the two-dimensional images at time points t1, t2, and t3 are calculated in the camera coordinate system as follows: (18); (19); (20).

[0033] S3: Calculate the center coordinates of the movable hazard point cloud using the bounding box of principal component analysis; The coordinates of the center point of the movable hazard were calculated using the bounding box centering method based on principal component analysis for the 3D point cloud data of the hazard based on images from different time periods; specifically including: S31: Calculate the center coordinates of the movable hazard point cloud using the bounding box of principal component analysis; The center of the 3D point cloud for calculating movable hazards is: (6); (twenty one); (twenty two); (twenty three); Subtract the center from the coordinates of each point to get the new coordinates: (twenty four); (25); (26).

[0034] S32: The covariance matrix is ​​used to reflect the anisotropy of the point cloud distribution. The formula for calculating the covariance matrix is: (7); Specifically, the covariance matrices are obtained as follows: (27); (28); (29).

[0035] Furthermore, eigenvalues ​​and corresponding eigenvectors are obtained by performing eigendecomposition on the covariance matrix E. These eigenvectors, which are the principal component directions, constitute a new coordinate system. Taking E1 as an example, the eigenvectors of the E1 matrix ,in: (30); Feature vector These represent the directions of the x, y, and z axes in the principal component coordinate system, respectively; similarly, the coordinate axes in the principal component coordinate systems of E2 and E3 can be calculated.

[0036] S33: Map the coordinates of the decentered point cloud to the principal component coordinate system. The transformation formula is as follows: Therefore, there is (31); (32); (33); In the principal component coordinate system, calculate the extreme values ​​(maximum and minimum values) of each axis of the point cloud data: (8); In formula (8), , , These are the values ​​of the i-th point in the camera coordinate system on the x-axis, y-axis, and z-axis components of the principal component coordinate system, respectively. and These are the minimum and maximum values ​​of all points in the camera coordinate system on the x-axis component of the principal component coordinate system, respectively. and These are the minimum and maximum values ​​of all points in the camera coordinate system on the y-axis component of the principal component coordinate system, respectively. and These are the minimum and maximum values ​​of all points in the camera coordinate system on the z-axis component of the principal component coordinate system, respectively. In the principal component coordinate system, the midpoint of the extreme values ​​of each axis is taken as the center point of the hidden danger, denoted as . : (9); Furthermore, through The calculated center coordinates of the movable hazard in the principal component coordinate system are: (34); (35); (36); C represents the center of the three-dimensional point cloud of movable potential hazards.

[0037] S4: Construct a motion estimation model based on the coordinates of the center point of the hidden danger at different times calculated in step S3; S41: Construct a motion estimation model based on the center point coordinates of the same hidden danger at different times: (10); In formula (10), t is the capture time, and the first capture time is... The initial value is 1, and subsequent times are incremented by 1 based on the initial value; All are coefficients to be determined; Specifically, taking three images at times t1, t2, and t3 as examples, quadratic functions with time as the variable are constructed for the motion in the x, y, and z axes respectively: (37); (38); (39); S42: Furthermore, the time of the three snapshots... Substituting the x-coordinate of the corresponding OBB center point into the above equation (10): (11) In formula (11), These represent the x-coordinate components of the hazard center at the three capture times.

[0038] Solving the above matrix yields the following results. Similarly, for and Establish this system of equations and use matrix methods to solve for the coefficients of each coordinate axis. For the x-component, we have: (12); Similarly, for and It can be solved and ; Finally, a motion estimation model for movable hazards can be obtained: .

[0039] S5: Determine the risk level of movable hazards; The location of the next movable hazard is calculated based on the motion prediction model. The three-dimensional distance from the last observed location of the movable hazard and the next predicted location to the tower conductor point cloud is calculated, and the risk level of the movable hazard is determined accordingly.

[0040] In step S5, the next location of the hazard center point is predicted based on the motion estimation model; the Euclidean nearest neighbor distance from the last center point location and the predicted center point location to the conductor is compared to determine the risk level of the movable hazard.

[0041] Specifically, based on the motion estimation model, the predicted center location of the movable hazard at the next time step is: ; Calculate separately and The Euclidean nearest neighbor distances to the conductor's three-dimensional coordinates are 26.23 and 25.34. The movement trajectory of the movable hazard is closer to the tower line, so it should be classified as a high-risk hazard; otherwise, it should be classified as a low-risk hazard.

[0042] Specifically, the risk level determination process for movable hazards described in step S5 is as follows: Calculate the center point coordinates of the last observed location of the movable hazard and the predicted location at the next time point, and use the Euclidean nearest neighbor distance with the conductor as the final hazard distance. If the predicted hazard distance is less than the hazard distance observed in the last time, it means that the movable hazard is approaching the tower line and is a high-risk hazard; conversely, if the distance increases, it means that the movable hazard is moving away from the tower line and is a low-risk hazard.

[0043] Example 2 This embodiment provides an apparatus for implementing a method for predicting ground-based movable hazard risks based on power transmission channel visualization. The apparatus includes: At least one processor; and The memory stores instructions that, when executed by the at least one processor, cause the at least one processor to perform a ground-based mobile hazard risk prediction method based on power transmission channel visualization as described above.

[0044] In this embodiment, electronic devices include, but are not limited to: personal computers, server computers, workstations, desktop computers, laptop computers, notebook computers, mobile computing devices, smartphones, tablet computers, cellular phones, personal digital assistants (PDAs), handheld devices, messaging devices, wearable computing devices, consumer electronic devices, etc.

[0045] Example 3 This embodiment also provides a computer-readable storage medium storing executable instructions, which, when executed, cause the machine to perform the above-described method for predicting ground-based mobile hazard risks based on power transmission channel visualization.

[0046] Specifically, a system or apparatus equipped with a readable storage medium may be provided, on which software program code implementing the functions of any of the embodiments described above is stored, and the computer or processor of the system or apparatus can read and execute the instructions stored in the readable storage medium.

[0047] In this case, the program code itself, which can be read from the readable medium, can perform the functions of any of the above embodiments, and therefore the computer-readable code and the readable storage medium storing the computer-readable code constitute a part of this specification.

[0048] Examples of readable storage media include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer or the cloud via a communication network.

[0049] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0050] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the technical solutions of the present invention, and are not intended to limit the specific implementation of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the claims of the present invention should be included within the protection scope of the claims of the present invention.

Claims

1. A method for ground movable hazard risk prediction based on power transmission corridor visualization, characterized in that, The method comprises: S1: obtaining three-dimensional point cloud data and two-dimensional image data of a power transmission channel, determining a camera intrinsic matrix and a camera extrinsic matrix through calibration, and determining a mapping relationship from three-dimensional point cloud to two-dimensional image; S2: identifying a movable hidden danger in the image after the image is captured by the monitoring and shooting device, using a least square method to fit a plane equation to establish a ground plane constraint; taking a center point at the bottom of a hidden danger frame of the movable hidden danger as a pixel reference point, obtaining a depth value based on an intersection of a pixel reference point ray direction and the ground plane constraint equation, and obtaining three-dimensional coordinates of the pixel reference point; The depth value of the non-pixel reference point is obtained by using a monocular depth estimation model based on deep learning, and is corrected by using the pixel reference point; Finally, three-dimensional point cloud data of the movable hidden danger is obtained; S3: calculating a hidden danger center point coordinate by using an oriented bounding box center method based on principal component analysis on the three-dimensional point cloud data of the movable hidden danger; S4: constructing a motion estimation model according to the center point coordinates of the same hidden danger at different time points; S5: predicting a next hidden danger center point position according to the motion estimation model, comparing a Euclidean nearest neighbor distance of the last observed hidden danger center point position and the predicted center point position to a guide line, and determining a movable hidden danger risk level.

2. The method of claim 1, wherein, The S1 specifically comprises: S11: collecting three-dimensional point cloud data of a tower line by a UAV carrying a laser radar, and obtaining two-dimensional image data of the tower line by a monitoring and shooting device arranged on a power transmission tower, wherein the two-dimensional image data comprises a power transmission line, an opposite tower and a ground; S12: take the current tower as the origin, and the direction towards the tower as the positive direction of the y-axis. The point (x, y) in the world coordinate system is converted into the camera coordinate system (x', y') through rotation and translation, and the conversion formula is as follows: x' = x * cos (a) + y * sin (a) y' = -x * sin (a) + y * cos (a) （1）； In formula (1), is the position of the i-th point in the three-dimensional point cloud in the world coordinate system after rotation and translation in the camera coordinate system, and there are n three-dimensional points in the three-dimensional point cloud; is the position of the i-th point in the three-dimensional point cloud in the camera coordinate system in the X, Y and Z axis directions; is the position of the i-th point in the three-dimensional point cloud in the world coordinate system in the X, Y and Z axis directions; R is a rotation matrix; and T is a translation matrix; is the position of the i-th point in the three-dimensional point cloud in the world coordinate system after rotation and translation in the camera coordinate system. The S2 specifically comprises: （2）； In formula (2), is the pixel coordinate of the i-th point in the three-dimensional point cloud in the horizontal direction after projection to the pixel coordinate system; is the pixel coordinate of the i-th point in the three-dimensional point cloud in the vertical direction after projection to the pixel coordinate system; W is the camera intrinsic matrix, and are the focal lengths of the x and y directions of the camera; and are the center point coordinates of the image coordinate system.

3. The method of claim 2, wherein, S21: continuously capturing multiple times by a power transmission visual monitoring and shooting device, continuously capturing 3 or more images at fixed time intervals, and determining a movable hidden danger by using a detection-based instance segmentation model on the captured images; S23: taking a midpoint at the bottom of a hidden danger frame of the identified movable hidden danger as a pixel reference point, and back-projecting the reference point to a ground plane, specifically, constructing a ray direction from a pixel to a camera coordinate: S22: Extract the ground point cloud from the three-dimensional point cloud data, and transform it to the camera coordinate system, use the least square method to construct the ground plane constraint optimization, the target equation is wherein (a, b, c) represents the normal vector of the plane, and d represents the intercept of the plane, and the optimization target is: （3）； In formula (3), X ic , Y ic , and Z ic are positions of the i-th point in the X, Y, and Z axis directions in the three-dimensional point cloud of the camera coordinate system, and N is the number of ground point clouds. The depth value of the non-pixel reference point is calculated by using a monocular depth estimation model based on deep learning, and is corrected by using the pixel reference point; （4）； In formula (4), m is the number of movable hidden danger three-dimensional point clouds; represents the ray direction of pixel point i in the camera coordinate system; and are the focal length lengths of the x-axis and y-axis directions of the camera respectively, and are the center point positions in the vertical and horizontal directions of the image coordinate system respectively; According to the intersection of the ray direction and the ground plane needs to meet , combined with the ground plane constraint equation to get the ray and the ground intersection depth value Finally, three-dimensional point cloud data of the movable hidden danger is obtained. : （5）； In formula (5), X ic , Y ic are positions of the pixel point i in the camera coordinate system, and the ray direction vector multiplied by the depth value is the coordinate of the reference pixel point in the three-dimensional space. The calculation of the hidden danger center point coordinate by using the oriented bounding box center method based on the principal component analysis comprises: S31: calculating a three-dimensional point cloud center C of the movable hidden danger; 4. The method of claim 1, wherein, S32: using a covariance matrix to reflect anisotropy of point cloud distribution, and a calculation formula of the covariance matrix is: Converting the center point in the principal component coordinate system to the original camera coordinate system, and OBB is a position of the center point in the principal component coordinate system in the original camera coordinate system: （6）； In equation (6), the new coordinates of each point after subtracting the center wherein, are the coordinates of the three-dimensional point in the camera coordinate system; The construction of the motion estimation model according to the center point coordinates of the same hidden danger at different time points comprises: （7）； Eigen-decomposition of the covariance matrix E gives the eigenvalues and the corresponding eigenvectors The eigenvectors are the principal component directions and form the new coordinate system. S33: Project the decentralized point cloud to the principal component coordinate system Under the principal component coordinate system, calculate the extreme value of each axis: （8）； In equation (8), , , are the values of the i-th point in the camera coordinate system in the principal component coordinate system x, y and z axis components, respectively; and are the minimum and maximum values of all points in the camera coordinate system in the principal component coordinate system x axis component, respectively; and are the minimum and maximum values of all points in the camera coordinate system in the principal component coordinate system y axis component, respectively; and are the minimum and maximum values of all points in the camera coordinate system in the principal component coordinate system z axis component, respectively. In the principal component coordinate system, the center point is the midpoint of the extreme values of each axis （9）； S41: establishing a three-dimensional motion estimation model by using a polynomial fitting, and constructing a quadratic function with time as a variable for motion in x, y and z axis directions respectively: 。 5. The method of claim 1, wherein, The correction by using the pixel reference point specifically comprises: calculating a ratio of a pixel reference point depth value and a pixel reference point depth value output by the monocular depth estimation model, and then multiplying the ratio with a depth value of the non-pixel reference point to perform absolute distance correction. ​ （10）； In formula (10), t is the time of the snapshot, and the initial value of the first snapshot time is 1, and the subsequent time is 1 unit of time based on the initial time, are all coefficients to be determined; S42: The time of the three snapshots Substituting the x-value of the corresponding OBB center point coordinates into formula (10), we get: （11）； In formula (11), respectively represent the horizontal coordinate components of the hidden danger center at the three snapshot time points. By the same token, for and Setting up a similar system of equations, using matrix methods to solve for the coefficients of each coordinate axis, for the x component, we have: （12）； Solving the above matrix can obtain , similarly, for and can be solved and ; A final mobile hazard motion estimation model can be obtained .

6. The method of claim 3, wherein, ​ 7. The method of claim 1, wherein, The method for determining the risk level of the movable hidden danger comprises: The center point coordinates of the last observed movable hidden danger position and the next time point predicted position are calculated respectively, and the Euclidean nearest neighbor distance is taken as the final hidden danger distance. When the predicted hidden danger distance is less than the last observed hidden danger distance, it indicates that the movable hidden danger is approaching the tower line, and it is a high-risk hidden danger. When the predicted hidden danger distance is greater than or equal to the last observed hidden danger distance, it indicates that the movable hidden danger is away from the tower line, and it is a low-risk hidden danger.

8. A ground movable hazard risk prediction device based on power transmission corridor visualization, characterized by, The device comprises: a processor; a memory having a computer program stored thereon and executable on the processor; When the computer program is executed by the processor, the steps of the method for predicting the risk of the movable hidden danger on the ground based on the visualization of the power transmission channel are implemented.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. When the computer program is executed by the processor, the steps of the method according to any one of claims 1-7 are implemented.

Images