A method and system for power transmission channel safety early warning based on monitoring images and point cloud data

Abstract

This invention relates to the field of power operation and maintenance technology, and discloses a method and system for early warning of transmission channel safety based on monitoring images and point cloud data. The method includes: acquiring monitoring images and three-dimensional point cloud data of the transmission channel; establishing a mapping relationship between image pixels and spatial coordinates through calibration; identifying external damage targets in the images using a model and generating pixel marker boxes; calculating the spatial distance between the external damage target and the transmission line by combining the mapping relationship and marker box information, and determining whether an early warning is triggered; when the distance is greater than a safety threshold and the target is mobile, simulating its trajectory and predicting the distance change trend, further determining whether an early warning is triggered. This solution can effectively distinguish between static obstacles and dynamic threats, improve the timeliness and accuracy of transmission channel safety early warning, reduce the false alarm rate, and ensure the safe operation of the power grid.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of power operation and maintenance, and particularly relates to a power transmission channel safety early warning method and system based on monitoring images and point cloud data. BACKGROUND

[0002] In order to discover external damage hidden danger risks in the power transmission channel in time, the power grid has installed visual monitoring devices, which can discover the hidden danger risks in the channel construction in time through high-precision front-end sensing, intelligent hidden danger identification and real-time alarm technology.

[0003] However, the prior art still has the following problems: first, the existing power transmission anti-external damage algorithm is mainly based on object recognition of static pictures, and the current inventory visual monitoring device lacks ranging capability, resulting in an alarm efficiency of less than 10%, a high invalidity rate of the visual device alarm, a huge workload of manual review of the alarm information, and an impact on the quality and efficiency of centralized monitoring, second, the small-scale deployed visual monitoring device with ranging function needs unmanned aerial vehicle laser radar data for offline data calibration and registration. In addition, due to the performance of the front-end laser radar, the ranging coverage is limited, and due to the high price of equipment procurement, it is impossible to achieve the goal of reducing the invalidity of the alarm and the pressure of the alarm rate through large-scale replacement and installation coverage in the short term, third, the visual monitoring device has a shooting interval of 15 minutes, which is relatively long, and the process of the external damage target entering the power transmission channel may not be shot, resulting in that the external damage hidden danger cannot be discovered in time.

[0004] Therefore, it is urgent to develop a power transmission channel safety early warning method and system based on monitoring images and point cloud data, which can effectively distinguish static obstacles and dynamic threats, significantly improve the timeliness and accuracy of power transmission channel safety early warning, reduce the false alarm rate, and ensure the safe operation of the power grid. SUMMARY

[0005] In order to solve the above technical problems, the present application provides a power transmission channel safety early warning method and system based on monitoring images and point cloud data, which can effectively distinguish static obstacles and dynamic threats, significantly improve the timeliness and accuracy of power transmission channel safety early warning, reduce the false alarm rate, and ensure the safe operation of the power grid.

[0006] The present application provides a power transmission channel safety early warning method based on monitoring images and point cloud data, which comprises the following steps:

[0007] S1, acquiring visual monitoring images and three-dimensional point cloud data of a target power transmission channel, calibrating the visual monitoring images and the three-dimensional point cloud data, establishing a mapping relationship, and generating a model index file;

[0008] S2, identifying an external damage target in the visual monitoring images through model intelligent recognition, and outputting a pixel marking box of the external damage target; wherein the external damage target includes a mobile external damage target and a non-mobile external damage target;

[0009] S3, calculating the distance between the external damage target and the power transmission line in the target power transmission channel according to the model index file and the pixel marking box of the external damage target;

[0010] S4, if the distance between the external damage target and the power transmission line in the target power transmission channel is less than or equal to the safety threshold, safety warning is performed;

[0011] If the distance between the external damage target and the power transmission line in the target power transmission channel is greater than the safety threshold, the type of the external damage target is judged: if the external damage target is a non-mobile external damage target, no safety warning is performed; if the external damage target is a mobile external damage target, S5 is entered;

[0012] S5, simulating the driving track of the mobile external damage target, and analyzing the distance change between the mobile external damage target and the power transmission line in the target power transmission channel according to the driving track;

[0013] S6, if the distance between the external damage target and the power transmission line in the target power transmission channel exists less than or equal to the safety threshold in the simulation process, safety warning is performed; otherwise, no safety warning is performed.

[0014] Further, in S3, the distance between the external damage target and the power transmission line in the target power transmission channel is calculated according to the model index file and the pixel marking box of the external damage target, comprising:

[0015] S31, mapping the pixel coordinates of the bottom corner points on both sides of the pixel marking box of the external damage target to three-dimensional space coordinates according to the mapping relationship in the model index file;

[0016] S32, calculating the space size corresponding to each pixel in the pixel marking box of the external damage target according to the pixel coordinates of the bottom corner points on both sides and the corresponding three-dimensional space coordinates;

[0017] S33, calculating the space distance from the top corner points on both sides of the pixel marking box of the external damage target to the bottom corner points on the same side according to the pixel distance between the top corner points on both sides and the bottom corner points on both sides of the pixel marking box of the external damage target and the space size corresponding to each pixel;

[0018] S34, obtaining the three-dimensional space coordinates of the top corner points on both sides of the pixel marking box of the external damage target according to the three-dimensional space coordinates of the bottom corner points on both sides and the space distance from the top corner points on both sides to the bottom corner points on the same side;

[0019] S35, calculating the space distance from the top corner points on both sides to the nearest power transmission line according to the three-dimensional space coordinates of the top corner points on both sides of the pixel marking box of the external damage target and the coordinates of the power transmission line in the target power transmission channel in the model index file, and taking the minimum value of the two space distances as the distance between the external damage target and the power transmission line in the target power transmission channel.

[0020] Further, in S32, the spatial size corresponding to each pixel in the pixel marking box of the external damage target is calculated according to the pixel coordinates of the two bottom corner points and the corresponding three-dimensional space coordinates, and the calculation formula is as follows:

[0021] ;

[0022] wherein, S p represents the spatial size corresponding to each pixel in the pixel marking box of the external damage target, d s represents the spatial distance between the two bottom corner points of the pixel marking box, (x0, y0, z0) represents the three-dimensional space coordinates corresponding to the left bottom corner point of the pixel marking box, (x1, y1, z1) represents the three-dimensional space coordinates corresponding to the right bottom corner point of the pixel marking box, d p represents the pixel distance between the two bottom corner points of the pixel marking box, (u0, v0) represents the pixel coordinates corresponding to the left bottom corner point of the pixel marking box, and (u1, v1) represents the pixel coordinates corresponding to the right bottom corner point of the pixel marking box.

[0023] Further, in S5, the driving track of the moving external damage target is simulated, and the distance change between the moving external damage target and the power transmission line in the target power transmission channel is analyzed according to the driving track, including:

[0024] S51, simulating the driving track of the moving external damage target;

[0025] S52, simulating the driving of the moving external damage target along the driving track, and generating the pixel marking box of the moving external damage target in the driving process;

[0026] If the driving track of the moving external damage target is perpendicular to the target power transmission channel, the width and height of the pixel marking box of the moving external damage target remain unchanged;

[0027] If the driving track of the moving external damage target is not perpendicular to the target power transmission channel, the width and height of the pixel marking box of the moving external damage target are corrected in real time according to the driving track;

[0028] S53, analyzing the distance change between the moving external damage target and the power transmission line in the target power transmission channel according to the pixel marking box of the moving external damage target.

[0029] Further, in S52, the width and height of the pixel marking box of the moving external damage target are corrected in real time according to the driving track, including:

[0030] Sa, determining the driving direction according to the pixel coordinates of the driving track;

[0031] Sb, determining the width and height of the pixel marking box according to the pixel marking box of the initial position of the moving external breaking target, and the three-dimensional space coordinates corresponding to the bottom right corner point of the pixel marking box;

[0032] Sc, when the moving external breaking target moves to the next position along the driving track, calculating the pixel coordinates corresponding to the bottom right corner point of the pixel marking box of the moving external breaking target after moving according to the driving direction, the pixel coordinates of the bottom right corner point of the pixel marking box of the initial position of the moving external breaking target, and the width and height of the pixel marking box;

[0033] Sd, obtaining the three-dimensional space coordinates corresponding to the bottom right corner point of the pixel marking box of the moving external breaking target after moving according to the mapping relationship and the pixel coordinates corresponding to the bottom right corner point of the pixel marking box of the moving external breaking target after moving;

[0034] Se, calculating the width and height of the pixel marking box of the moving external breaking target after moving according to the three-dimensional space coordinates corresponding to the bottom right corner point of the pixel marking box of the moving external breaking target after moving, and the internal and external parameters of the camera used for shooting the visual monitoring image;

[0035] Sf, repeating steps Sb-Se until the simulated moving external breaking target leaves the target power transmission channel along the driving track.

[0036] Further, in Sc, the calculation formula of the pixel coordinates corresponding to the bottom right corner point of the pixel marking box of the moving external breaking target after moving is as follows:

[0037] ;

[0038] ;

[0039] Where (u1, v1) represents the pixel coordinates of the bottom right corner point of the pixel marking box of the initial position of the moving external breaking target, (u1', v1') represents the pixel coordinates corresponding to the bottom right corner point of the pixel marking box of the moving external breaking target after moving, w0 represents the width of the pixel marking box of the initial position of the moving external breaking target, h0 represents the height of the pixel marking box of the initial position of the moving external breaking target, (dirx1, diry1) represents the direction vector of the driving direction of the moving external breaking target from the initial position to the next position along the driving track, dirx1 represents the horizontal displacement component, and diry1 represents the vertical displacement component.

[0040] Further, in Se, the calculation formula of the width and height of the pixel marking box of the moving external breaking target after moving is as follows:

[0041] ;

[0042] ;

[0043] wherein w' represents the width of the pixel marking frame of the moving external damage target after moving, h' represents the height of the pixel marking frame of the moving external damage target after moving, R represents the rotation matrix of the camera used for shooting the visual monitoring image, P1' represents the three-dimensional space coordinates corresponding to the right bottom corner point of the pixel marking frame of the moving external damage target after moving, represents the intermediate parameter, Z1 represents the depth parameter of the camera used for shooting the visual monitoring image, f x , f y represents the focal length of the camera used for shooting the visual monitoring image, (c x , c y ) represents the optical center of the camera used for shooting the visual monitoring image, t represents the translation vector of the camera used for shooting the visual monitoring image, and T represents the transposed matrix. x , (·) y , (·) z respectively represent the three components of the vector in the camera coordinate system. x represents the X-axis component, i.e. the horizontal direction of the camera coordinate system. y represents the Y-axis component, i.e. the vertical direction of the camera coordinate system. z represents the Z-axis component, i.e. the optical axis direction of the camera coordinate system, i.e. the depth.

[0044] The application also provides a power transmission channel safety early warning system based on monitoring images and point cloud data, which is used for executing the power transmission channel safety early warning method based on monitoring images and point cloud data.

[0045] The data acquisition module is used for acquiring the visual monitoring image and the three-dimensional point cloud data of the target power transmission channel, calibrating the visual monitoring image and the three-dimensional point cloud data, establishing a mapping relationship, and generating a model index file.

[0046] The external damage target recognition module is used for intelligently recognizing the external damage target in the visual monitoring image through the model, and outputting the pixel marking frame of the external damage target.

[0047] The distance calculation module is used for calculating the distance between the external damage target and the power transmission line in the target power transmission channel according to the model index file and the pixel marking frame of the external damage target.

[0048] The early warning module is further used for performing safety early warning if the distance between the external damage target and the power transmission line in the target power transmission channel is less than or equal to the safety threshold in the simulation process; otherwise, no safety early warning is performed.

[0049] The trajectory prediction module is used for simulating the driving trajectory of the mobile external damage target, and analyzing the distance change between the mobile external damage target and the power transmission line in the target power transmission channel according to the driving trajectory.

[0050] The early warning module is further used for performing safety early warning if the distance between the external damage target and the power transmission line in the target power transmission channel is less than or equal to the safety threshold in the simulation process; otherwise, no safety early warning is performed.

[0051] The embodiment of the present application has the following technical effects:

[0052] The present application fuses the monitoring image and the three-dimensional point cloud data, constructs the mapping relationship between the visual image and the real space, realizes the accurate spatial positioning and dynamic behavior prediction of the external damage target around the power transmission channel, uses the multi-modal data fusion and the camera imaging geometric principle, accurately converts the pixel marking box in the image into the three-dimensional space coordinates, and then calculates the actual spatial distance between the external damage target and the power transmission line, improves the ranging accuracy, for the mobile target, combines the motion direction and the camera view angle relationship, dynamically corrects the scale change of the pixel box in the moving process, simulates the driving trajectory and predicts the safety distance change trend, realizes the forward-looking risk early warning, the method overcomes the limitations of single image or point cloud data in spatial positioning accuracy and dynamic analysis, effectively distinguishes static obstacles and dynamic threats, significantly improves the timeliness and accuracy of the power transmission channel safety early warning, reduces the false alarm rate, and guarantees the safe operation of the power grid. BRIEF DESCRIPTION OF DRAWINGS

[0053] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the specific embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0054] Figure 1 is a flow chart of the power transmission channel safety early warning method based on monitoring image and point cloud data provided by the embodiment of the present application;

[0055] Figure 2 is a schematic diagram of the pixel marking box of the external damage target output by the model provided by the embodiment of the present application;

[0056] Figure 3 is a schematic diagram of simulating the driving track of a moving external breaking target provided by an embodiment of the present application;

[0057] Figure 4 is a schematic diagram of generating a pixel marking box along the driving track of a moving external breaking target provided by an embodiment of the present application;

[0058] Figure 5 is a structural schematic diagram of a system for power transmission channel safety early warning based on monitoring images and point cloud data provided by an embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described clearly and completely below. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

[0060] The embodiments of the present application provide a power transmission channel safety early warning method based on monitoring images and point cloud data, Figure 1 is a flowchart of the power transmission channel safety early warning method based on monitoring images and point cloud data provided by an embodiment of the present application, referring to Figure 1 , the method comprises the following steps:

[0061] S1, acquiring the visual monitoring image and the three-dimensional point cloud data of a target power transmission channel, calibrating the visual monitoring image and the three-dimensional point cloud data, establishing a mapping relationship, and generating a model index file.

[0062] Exemplarily, the calibration method can be realized according to the following method:

[0063] Define the world coordinate system (point cloud coordinate system), the camera coordinate system (the origin is the optical center of the camera), and the image plane coordinate system (two-dimensional pixel coordinate system of the image);

[0064] The projection equation is as follows:

[0065] ;

[0066] (u,v) represents the image principal point coordinates, that is, the intersection of the camera optical axis and the image plane;

[0067] (X,Y) is the coordinate of the feature point in the image plane coordinate system;

[0068] (X A ,Y A ,Z A ) is the three-dimensional coordinate of the feature point in the world coordinate system;

[0069] (X S ,Y S Z S () represents the three-dimensional coordinates of the camera's optical center in the world coordinate system;

[0070] a i b i c i Let i be the elements of the rotation matrix R, i = 1, 2, 3, which correspond to the rotation parameters of the X, Y, and Z axes, respectively, and f represents the camera focal length.

[0071] In some embodiments, calibration may also be performed using other methods, which are not limited here.

[0072] After completing the calibration of the visualized monitoring images and 3D point cloud data and generating the model index file, the system has the ability to convert image pixel coordinates to real space coordinates.

[0073] S2. The model intelligently identifies external damage targets in the visual monitoring image and outputs the pixel marker box of the external damage target.

[0074] For example, Figure 2 This is a schematic diagram of a pixel marker box for an externally damaged target output by a model, provided in an embodiment of the present invention. See also... Figure 2 Among them, external damage targets include mobile external damage targets and non-mobile external damage targets. For example, wildfires and smoke are non-mobile external damage targets, while construction vehicles and ordinary vehicles are mobile external damage targets.

[0075] The model used in this step is not limited.

[0076] S3. Based on the model index file and the pixel bounding box of the externally damaged target, calculate the distance between the externally damaged target and the power transmission line in the target power transmission channel.

[0077] In some embodiments, S3 includes the following sub-steps:

[0078] S31. Based on the mapping relationship in the model index file, map the pixel coordinates of the bottom corner points on both sides of the pixel marker box of the externally damaged target to three-dimensional spatial coordinates.

[0079] S32. Calculate the spatial size of each pixel in the pixel marker box of the externally damaged target based on the pixel coordinates of the bottom corners on both sides and the corresponding three-dimensional spatial coordinates.

[0080] Specifically, the formula for calculating the space size corresponding to each pixel is as follows:

[0081] ;

[0082] Among them, Sp d represents the spatial size of each pixel in the pixel marker box of the externally damaged target. s This represents the spatial distance between the two bottom corner points of the pixel bounding box, where (x0, y0, z0) represents the three-dimensional spatial coordinates corresponding to the left bottom corner point of the pixel bounding box, and (x1, y1, z1) represents the three-dimensional spatial coordinates corresponding to the right bottom corner point of the pixel bounding box. p This represents the pixel distance between the two bottom corners of the pixel marker box, where (u0, v0) represents the pixel coordinates corresponding to the left bottom corner of the pixel marker box, and (u1, v1) represents the pixel coordinates corresponding to the right bottom corner of the pixel marker box.

[0083] S33. Based on the pixel distance between the top and bottom corners of the pixel marker box on both sides of the externally damaged target and the spatial size corresponding to each pixel, calculate the spatial distance from the top corners of the pixel marker box on both sides to the bottom corners on the same side.

[0084] S34. Based on the three-dimensional spatial coordinates of the bottom corners on both sides and the spatial distance from the top corners on both sides of the pixel marker frame of the externally damaged target to the bottom corners on the same side, obtain the three-dimensional spatial coordinates of the top corners on both sides of the pixel marker frame of the externally damaged target.

[0085] S35. Based on the three-dimensional spatial coordinates of the top two corner points of the pixel marker box of the externally damaged target and the coordinates of the power transmission line in the target power transmission channel in the model index file, calculate the spatial distance from the top two corner points to the nearest power transmission line, and take the minimum of the two spatial distances as the distance between the externally damaged target and the power transmission line in the target power transmission channel.

[0086] After the target detection model outputs the pixel bounding box of the breached target, the bottom corners on both sides of the rectangle are selected as reference points. Using the mapping relationship stored in the model index file, these two pixel coordinates are converted into corresponding 3D spatial coordinates, serving as the starting point for subsequent spatial calculations. To achieve the conversion from pixel scale to spatial scale, the pixel distance between the left and right bottom corners in the image is calculated. Combined with their actual distance in 3D space, the spatial size ratio represented by each pixel within the bounding box is derived. This ratio reflects the correspondence between the image resolution and the actual physical distance at the current viewpoint. Based on this, the vertical pixel distance between the top corner and the bottom corner on the same side of the pixel bounding box is measured. Combined with the spatial size corresponding to each pixel, the vertical height difference between the top and bottom corners is calculated. Using the 3D spatial coordinates of the bottom corners and the calculated spatial height difference, the 3D spatial coordinates of the left and right top corners are further derived, thus constructing the three-dimensional position contour of the breached target in 3D space. Subsequently, based on the three-dimensional coordinate information of the transmission line pre-stored in the model index file, the spatial distances from the left and right top corners to the nearest transmission line are calculated, and the smaller value is selected as the minimum safe distance between the external damage target and the transmission line. This method improves the accuracy of distance measurement by analyzing the mapping relationship between pixels and space step by step, making full use of image geometric features and point cloud spatial information. It is particularly suitable for complex monitoring scenarios with varying distances and tilted viewing angles, thus enhancing the reliability of early warning judgments.

[0087] S4. If the distance between the externally damaged target and the power transmission line in the target power transmission channel is less than or equal to the safety threshold, a safety warning will be issued.

[0088] If the distance between the external damage target and the power transmission line in the target power transmission channel is greater than the safety threshold, determine the type of external damage target: if the external damage target is a non-moving external damage target, no safety warning will be issued; if the external damage target is a moving external damage target, proceed to S5.

[0089] S5. Simulate the trajectory of the moving externally damaged target and analyze the distance changes between the moving externally damaged target and the power transmission line in the target power transmission channel based on the trajectory.

[0090] In some embodiments, S5 specifically includes the following sub-steps:

[0091] S51, Simulate the trajectory of a moving externally damaged target.

[0092] The process of simulating driving trajectories involves simulating all possible driving trajectories, rather than predicting a single driving trajectory. This step can be achieved by manually calibrating the trajectory or by generating it through model prediction, and no specific limitations are imposed here.

[0093] S52. Simulate the movement of the externally damaged target along the driving trajectory and generate pixel marker boxes of the moving externally damaged target during the driving process.

[0094] If the trajectory of the moving externally damaged target is perpendicular to the target power transmission channel, the width and height of the pixel marker box of the moving externally damaged target remain unchanged. For example, Figure 3 This is a schematic diagram illustrating the trajectory of a simulated moving externally damaged target according to an embodiment of the present invention. See also... Figure 3 The simulated driving trajectory in the figure is perpendicular to the target power transmission channel.

[0095] If the trajectory of the moving external damage target is not perpendicular to the target power transmission channel, the width and height of the pixel marker box of the moving external damage target will be corrected in real time according to the trajectory.

[0096] Furthermore, the width and height of the pixel marker box of the moving externally damaged target are corrected in real time based on the driving trajectory, specifically including:

[0097] Sa: Determine the driving direction based on the pixel coordinates of the driving trajectory.

[0098] Sb. Determine the width and height of the pixel marker box based on the pixel marker box of the initial position of the moving externally damaged target, as well as the three-dimensional spatial coordinates corresponding to the bottom right corner of the pixel marker box.

[0099] Sc. After the moving external damage target moves to the next position along the driving trajectory, calculate the pixel coordinates corresponding to the right bottom corner of the pixel marker frame after the moving external damage target moves, based on the driving direction, the pixel coordinates of the bottom right corner of the pixel marker frame of the initial position of the moving external damage target, and the width and height of the pixel marker frame.

[0100] Specifically, the calculation formula is as follows:

[0101] ;

[0102] ;

[0103] ;

[0104] ;

[0105] Where (u1,v1) represents the pixel coordinates of the bottom right corner of the pixel marker box at the initial position of the moving external target, (u1',v1') represents the pixel coordinates of the bottom right corner of the pixel marker box after the moving external target has moved, w0 represents the width of the pixel marker box at the initial position of the moving external target, w0=u1−u0, h0 represents the height of the pixel marker box at the initial position of the moving external target, h0=v1−v0, where (u0,v0) represents the pixel coordinates of the top left corner of the pixel marker box at the initial position of the moving external target, (dirx1,diry1) represents the direction vector of the moving external target moving from the initial position to the next position along the driving trajectory, dirx1 represents the lateral displacement component, diry1 represents the longitudinal displacement component, and (rx0,ry0) and (rx1,ry1) are the pixel coordinates of the initial position and the next moving position on the driving trajectory, respectively.

[0106] Specifically, the x-coordinate of the new position is superimposed with the initial width value on the original x-coordinate to simulate the target's forward movement in the direction of travel; the y-coordinate is adjusted according to the height value and the ratio of the vertical and horizontal components of the direction vector to ensure that the vertical displacement of the corner points is consistent with the direction of movement. This calculation method fully considers the non-linear behavior of target movement in the image under perspective projection, avoiding deformation distortion caused by simple translation. The pixel coordinates obtained by this method can realistically reflect the changes in the bottom contact point of the target at different positions, providing accurate input for subsequent spatial coordinate mapping and correction of the bounding box size. This process is repeated with each position update, forming a continuous corner trajectory, supporting the visualization simulation of the entire driving process and dynamic assessment of safe distance, improving the visual consistency and spatial rationality of trajectory prediction.

[0107] Sd. Based on the mapping relationship and the pixel coordinates corresponding to the bottom right corner of the pixel marker frame after the moving external target has moved, the three-dimensional spatial coordinates corresponding to the bottom right corner of the pixel marker frame after the moving external target has moved are obtained.

[0108] Se, based on the three-dimensional spatial coordinates corresponding to the bottom right corner of the pixel marker frame after the moving external damage target has moved, and the internal and external parameters of the camera used to capture the visual monitoring image, calculate the width and height of the pixel marker frame after the moving external damage target has moved.

[0109] Specifically, the calculation formula is as follows:

[0110] ;

[0111] ;

[0112] Where w' represents the width of the pixel marker box after the moving external damage target has moved, h' represents the height of the pixel marker box after the moving external damage target has moved, R represents the rotation matrix of the camera used to capture the visual monitoring image, and P1' represents the three-dimensional spatial coordinates corresponding to the bottom right corner of the pixel marker box after the moving external damage target has moved. Z1 represents the depth-of-field parameter of the camera used to capture the visual surveillance images, and f represents the intermediate parameter. x f y This indicates the focal length of the camera used to capture visual surveillance images, (c x c y f represents the optical center of the camera used to capture visual surveillance images. x f y c x and c y All parameters are derived from the camera's intrinsic parameter matrix K. t represents the translation vector of the camera used to capture the visual surveillance images, and T represents the transpose matrix; (·) x 、(·) y 、(·) z Let represent the three components of the vector in the camera coordinate system, (·) x This represents the X-axis component, i.e., the horizontal direction of the camera coordinate system, (·). y This represents the Y-axis component, which is the vertical direction of the camera coordinate system. (·) z The Z-axis component represents the direction of the optical axis in the camera coordinate system, i.e., the depth. x and y represent the horizontal and vertical positions of the moving target in the camera coordinate system, which directly affect the pixel coordinates (u,v). z represents the depth of the moving target, which determines the scaling ratio of the perspective projection (smaller for farther objects and larger for closer ones).

[0113] This step involves using rotation and translation vectors to compensate for the impact of camera pose on target projection, and determining the precise scaling ratio of the target at the new position using focal length and optical center position. The resulting width and height accurately represent the true scale of the target at a specific location in the 2D image, ensuring accurate capture of actual size changes even under complex viewing conditions. This correction mechanism is continuously iterated throughout the simulation, ensuring that the target's pixel bounding box accurately reflects its actual position and size at each moment, thus providing reliable data support for subsequent distance calculations and improving the accuracy and timeliness of the entire system's external threat identification.

[0114] Sf, repeat steps Sb-Se until the simulated moving external damage target leaves the target power transmission channel along the travel trajectory.

[0115] When the system determines that the externally damaged target is moving and its current position is more than a safe distance from the power transmission line, the trajectory prediction mechanism is activated. First, based on the target's positional change trend in consecutive image frames, combined with the road direction, obstacle distribution, and target's direction of movement in the scene, its possible travel path is constructed. This path is expressed as a sequence of pixel coordinates on the monitoring image plane to simulate the target's future movement. During the simulation, a sequence of pixel bounding boxes is generated corresponding to the target's movement along the trajectory. If the target's direction of travel is perpendicular to the direction of the power transmission line, its projection scale in the image is considered to change little, and the width and height of the initial bounding box remain unchanged. If the direction of travel has an angle with the power transmission line, due to perspective effects, the target's visual size in the image will change with distance, requiring dynamic adjustment of the width and height of the pixel bounding box. By analyzing the direction vector of the travel path and combining it with the bounding box parameters of the target's initial position, the coordinates of its bottom corner at the new position are calculated in real time. The corrected bounding box size is then calculated using the camera imaging model and spatial mapping relationships. This process ensures that the target's representation in the image accurately reflects its spatial location and outline range, whether the target is moving away from or near the camera. Subsequently, based on the corrected pixel bounding boxes at each time step, the spatial coordinate transformation and distance calculation process is repeated to continuously evaluate the trend of minimum distance change between the target and the transmission line.

[0116] Furthermore, in some embodiments, the largest possible pixel bounding box of the moving external damage target can be used as a simulation object for analysis. For example, when the moving external damage target is a crane, the boundary of the pixel bounding box is expanded based on the maximum extension length of the crane boom, and the pixel bounding box at the maximum extension length of the crane boom is used as the pixel bounding box of the moving external damage target to simulate the moving external damage target traveling along the trajectory and perform subsequent analysis.

[0117] S53. Analyze the distance change between the moving externally damaged target and the power transmission line in the target power transmission channel based on the pixel marker box of the moving externally damaged target.

[0118] By simulating the entire movement of the target, the system can identify potential intrusion risks in advance, achieve proactive early warning of dynamic threats, and improve the initiative and accuracy of security protection.

[0119] S6. If, during the simulation, the distance between the externally damaged target and the power transmission line in the target's power transmission channel is less than or equal to the safety threshold, a safety warning will be issued; otherwise, no safety warning will be issued.

[0120] For example, Figure 4 This is a schematic diagram illustrating the generation of pixel marker boxes along the trajectory of a simulated moving externally damaged target, as provided in an embodiment of the present invention. See also... Figure 4In the image, the crane was 33.964 meters away from the conductor when the photo was taken, which is greater than the safe operating distance of the conductor and is therefore safe. As the crane continued to move along the simulated line, the distance between it and the conductor decreased, reaching a minimum of 4.776 meters, which is significantly less than the safe operating distance of the conductor and affects the safe operation of the conductor, requiring timely warning.

[0121] This invention integrates monitoring images and 3D point cloud data to construct a mapping relationship between visualized images and real space, enabling precise spatial positioning and dynamic behavior prediction of external damage targets around power transmission channels. Utilizing multimodal data fusion and camera imaging geometry principles, it accurately converts pixel bounding boxes in images into 3D spatial coordinates, thereby calculating the actual spatial distance between the external damage target and the transmission line, improving ranging accuracy. For moving targets, it dynamically corrects the scale changes of pixel bounding boxes during movement by combining their motion direction with the camera's viewing angle, simulating their trajectory and predicting the trend of safe distance changes, achieving proactive risk warning. This method overcomes the limitations of single images or point cloud data in spatial positioning accuracy and dynamic analysis, effectively distinguishing between static obstacles and dynamic threats, significantly improving the timeliness and accuracy of power transmission channel safety warnings, reducing false alarm rates, and ensuring the safe operation of the power grid.

[0122] This invention also provides a power transmission channel safety early warning system based on monitoring images and point cloud data, used to execute the aforementioned power transmission channel safety early warning method based on monitoring images and point cloud data. Figure 5 This is a schematic diagram of the power transmission channel safety early warning system based on monitoring images and point cloud data provided in an embodiment of the present invention. See also... Figure 5 The system includes the following modules:

[0123] The data acquisition module is used to acquire visual monitoring images and 3D point cloud data of the target power transmission channel, calibrate the visual monitoring images and 3D point cloud data, establish a mapping relationship, and generate a model index file.

[0124] The external damage target identification module is used to intelligently identify external damage targets in the visual monitoring image through a model and output the pixel bounding box of the external damage target; wherein, the external damage target includes moving external damage targets and non-moving external damage targets;

[0125] The distance calculation module is used to calculate the distance between the externally damaged target and the power transmission line in the target power transmission channel based on the model index file and the pixel marker box of the externally damaged target;

[0126] The early warning module is used to issue a safety warning if the distance between the external damage target and the transmission line in the target transmission channel is less than or equal to a safety threshold; if the distance between the external damage target and the transmission line in the target transmission channel is greater than the safety threshold, it determines the type of the external damage target: if the external damage target is a non-moving external damage target, no safety warning is issued; if the external damage target is a moving external damage target, the trajectory prediction module is invoked.

[0127] The trajectory prediction module is used to simulate the travel trajectory of a moving externally damaged target and analyze the changes in distance between the moving externally damaged target and the power transmission line in the target power transmission channel based on the travel trajectory.

[0128] The early warning module is also used to issue a safety warning if the distance between the external target and the power transmission line in the target power transmission channel is less than or equal to the safety threshold during the simulation process; otherwise, no safety warning is issued.

[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. A power transmission passage safety early warning method based on monitoring images and point cloud data, characterized in that, The method comprises the following steps: S1, acquiring a visual monitoring image and three-dimensional point cloud data of a target power transmission channel, calibrating the visual monitoring image and the three-dimensional point cloud data, establishing a mapping relationship, and generating a model index file; S2, intelligently identifying an external damage target in the visual monitoring image through a model, and outputting a pixel marking box of the external damage target; wherein the external damage target comprises a mobile external damage target and a non-mobile external damage target; S3, calculating a distance between the external damage target and a power transmission line in the target power transmission channel according to the model index file and the pixel marking box of the external damage target; S4, if the distance between the external damage target and the power transmission line in the target power transmission channel is less than or equal to a safety threshold, a safety warning is given; if the distance between the external damage target and the power transmission line in the target power transmission channel is greater than the safety threshold, the type of the external damage target is determined: if the external damage target is a non-mobile external damage target, no safety warning is given; if the external damage target is a mobile external damage target, S5 is entered; S5, simulating a driving track of the mobile external damage target, and analyzing a distance change between the mobile external damage target and the power transmission line in the target power transmission channel according to the driving track; S6, if the distance between the external damage target and the power transmission line in the target power transmission channel is less than or equal to the safety threshold during simulation, a safety warning is given; otherwise, no safety warning is given; in S5, simulating a driving track of the mobile external damage target, and analyzing a distance change between the mobile external damage target and the power transmission line in the target power transmission channel according to the driving track, comprising: S51, simulating a driving track of the mobile external damage target; S52, simulating driving of the mobile external damage target along the driving track, and generating a pixel marking box of the mobile external damage target in the driving process; if the driving track of the mobile external damage target is perpendicular to the target power transmission channel, the width and height of the pixel marking box of the mobile external damage target remain unchanged; if the driving track of the mobile external damage target is not perpendicular to the target power transmission channel, the width and height of the pixel marking box of the mobile external damage target are corrected in real time according to the driving track; S53, analyzing a distance change between the mobile external damage target and the power transmission line in the target power transmission channel according to the pixel marking box of the mobile external damage target; in S52, the width and height of the pixel marking box of the mobile external damage target are corrected in real time according to the driving track, comprising: Sa, determining a driving direction according to pixel coordinates of the driving track; Sb, determining the width and height of the pixel marking box of the mobile external damage target according to a pixel marking box of an initial position of the mobile external damage target, and determining three-dimensional space coordinates corresponding to a right bottom corner point of the pixel marking box; Sc, when the mobile external damage target moves to a next position along the driving track, calculating pixel coordinates corresponding to a right bottom corner point of a pixel marking frame of the mobile external damage target after movement according to the driving direction, pixel coordinates of the right bottom corner point of the pixel marking frame of the initial position of the mobile external damage target, and the width and height of the pixel marking frame; Sd, obtaining three-dimensional space coordinates corresponding to the right bottom corner point of the pixel marking frame of the mobile external damage target after movement according to the mapping relationship and the pixel coordinates corresponding to the right bottom corner point of the pixel marking frame of the mobile external damage target after movement; Se, calculating the width and height of the pixel marking frame of the mobile external damage target after movement according to the three-dimensional space coordinates corresponding to the right bottom corner point of the pixel marking frame of the mobile external damage target after movement and internal and external parameters of a camera used for shooting the visual monitoring image; Sf, repeatedly performing steps Sb-Se until the mobile external damage target leaves the target power transmission channel along the driving track. 2.The power transmission passage safety warning method based on monitoring image and point cloud data according to claim 1, characterized in that, In the S3, the distance between the external damage target and the power transmission line in the target power transmission channel is calculated according to the model index file and the pixel marking frame of the external damage target, including: S31, mapping pixel coordinates of two side bottom corner points of the pixel marking frame of the external damage target into three-dimensional space coordinates according to the mapping relationship in the model index file; S32, calculating a space size corresponding to each pixel in the pixel marking frame of the external damage target according to the pixel coordinates of the two side bottom corner points and the corresponding three-dimensional space coordinates; S33, calculating a space distance from a same side bottom corner point to a same side top corner point of the pixel marking frame of the external damage target according to pixel distances between the two side top corner points and the two side bottom corner points and the space size corresponding to each pixel; S34, obtaining three-dimensional space coordinates of the two side top corner points of the pixel marking frame of the external damage target according to the three-dimensional space coordinates of the two side bottom corner points and the space distance from the same side bottom corner point to the same side top corner point; S35, calculating a space distance from each of the two side top corner points to the nearest power transmission line according to the three-dimensional space coordinates of the two side top corner points of the pixel marking frame of the external damage target and coordinates of the power transmission line in the target power transmission channel in the model index file, and taking a minimum value of the two space distances as the distance between the external damage target and the power transmission line in the target power transmission channel.

3. The method of claim 2, wherein the method further comprises: In the S32, the space size corresponding to each pixel in the pixel marking frame of the external damage target is calculated according to the pixel coordinates of the two side bottom corner points and the corresponding three-dimensional space coordinates, and the calculation formula is as follows: ； wherein S p represents a spatial size corresponding to each pixel in the pixel marker frame of the external breaking target, d s represents a spatial distance between two bottom corner points of the pixel marker frame, (x0, y0, z0) represents a three-dimensional spatial coordinate corresponding to the left bottom corner point of the pixel marker frame, and (x1, y1, z1) represents a three-dimensional spatial coordinate corresponding to the right bottom corner point of the pixel marker frame, d p represents a pixel distance between two bottom corner points of the pixel marker frame, (u0, v0) represents a pixel coordinate corresponding to the left bottom corner point of the pixel marker frame, and (u1, v1) represents a pixel coordinate corresponding to the right bottom corner point of the pixel marker frame.

4. The method of claim 1, wherein the method further comprises: In the Sc, the calculation formula of the pixel coordinates corresponding to the right bottom corner point of the pixel marking frame of the mobile external damage target after movement is as follows: ； ； Wherein, (u1, v1) represents the pixel coordinates of the right bottom corner point of the pixel marking box of the initial position of the moving external damage target, (u1', v1') represents the pixel coordinates corresponding to the right bottom corner point of the pixel marking box after the moving external damage target moves, w0 represents the width of the pixel marking box of the initial position of the moving external damage target, h0 represents the height of the pixel marking box of the initial position of the moving external damage target, (dirx1, diry1) represents the direction vector of the driving direction of the moving external damage target from the initial position to the next position along the driving track, dirx1 represents the horizontal displacement component, and diry1 represents the vertical displacement component.

5. The method of claim 4, wherein the method further comprises: In the Se, the calculation formula of the width and height of the pixel marking box after the moving external damage target moves is as follows: ； ； wherein w' represents the width of the pixel marking frame of the moved outer breaking target, h' represents the height of the pixel marking frame of the moved outer breaking target, R represents the rotation matrix of the camera used for shooting the visual monitoring image, P1' represents the three-dimensional space coordinates corresponding to the right bottom corner point of the pixel marking frame of the moved outer breaking target, represents the intermediate parameter, Z1 represents the depth of field parameter of the camera used for shooting the visual monitoring image, f x , f y represents the focal length of the camera used for shooting the visual monitoring image, (c x , c y ) represents the optical center of the camera used for shooting the visual monitoring image, t represents the translation vector of the camera used for shooting the visual monitoring image, T represents the transposed matrix; x , (·) y , (·) z respectively represent the three components of the vector in the camera coordinate system, (·) x represents the X-axis component, i.e. the horizontal direction of the camera coordinate system, (·) y represents the Y-axis component, i.e. the vertical direction of the camera coordinate system, (·) z represents the Z-axis component, i.e. the optical axis direction of the camera coordinate system, i.e. the depth.

6. A power transmission passage safety warning system based on monitoring images and point cloud data, configured to perform the power transmission passage safety warning method based on monitoring images and point cloud data according to any one of claims 1-5, characterized in that, The system comprises the following modules: A data acquisition module is configured to acquire a visual monitoring image and three-dimensional point cloud data of a target power transmission channel, calibrate the visual monitoring image and the three-dimensional point cloud data, establish a mapping relationship, and generate a model index file; An external damage target recognition module is configured to intelligently recognize an external damage target in the visual monitoring image by using a model, and output a pixel marking box of the external damage target; wherein the external damage target comprises a moving external damage target and a non-moving external damage target; A distance calculation module is configured to calculate a distance between the external damage target and a power transmission line in the target power transmission channel according to the model index file and the pixel marking box of the external damage target; An early warning module is configured to perform safety early warning if the distance between the external damage target and the power transmission line in the target power transmission channel is less than or equal to a safety threshold; and if the distance between the external damage target and the power transmission line in the target power transmission channel is greater than the safety threshold, determine the type of the external damage target: if the external damage target is a non-moving external damage target, no safety early warning is performed; and if the external damage target is a moving external damage target, a trajectory prediction module is called; The trajectory prediction module is configured to simulate a driving track of the moving external damage target, and analyze a distance change between the moving external damage target and the power transmission line in the target power transmission channel according to the driving track; The early warning module is further configured to perform safety early warning if, in the simulation process, the distance between the external damage target and the power transmission line in the target power transmission channel is less than or equal to the safety threshold; otherwise, no safety early warning is performed; The simulation of the driving track of the moving external damage target and the analysis of the distance change between the moving external damage target and the power transmission line in the target power transmission channel according to the driving track comprise: S51, simulating a driving track of the moving external damage target; S52, simulating driving of the moving external damage target along the driving track, and generating a pixel marking box of the moving external damage target in the driving process; If the driving track of the moving external damage target is perpendicular to the target power transmission channel, the width and height of the pixel marking box of the moving external damage target remain unchanged; If the driving track of the moving external damage target is not perpendicular to the target power transmission channel, the width and height of the pixel marking box of the moving external damage target are corrected in real time according to the driving track. S53, analyzing distance variation between the mobile external damage target and the power transmission line in the target power transmission channel according to the pixel marking box of the mobile external damage target; In the S52, the width and height of the pixel marking box of the mobile external damage target are corrected in real time according to the driving track, including: Sa, determining driving direction according to pixel coordinates of the driving track; Sb, determining the width and height of the pixel marking box of the mobile external damage target according to the pixel marking box of the initial position of the mobile external damage target, and determining three-dimensional space coordinates corresponding to the right bottom corner point of the pixel marking box; Sc, after the mobile external damage target moves to the next position along the driving track, calculating pixel coordinates corresponding to the right bottom corner point of the pixel marking box of the mobile external damage target after moving according to the driving direction, the pixel coordinates of the right bottom corner point of the pixel marking box of the initial position of the mobile external damage target, and the width and height of the pixel marking box; Sd, obtaining three-dimensional space coordinates corresponding to the right bottom corner point of the pixel marking box of the mobile external damage target after moving according to the mapping relationship and the pixel coordinates corresponding to the right bottom corner point of the pixel marking box of the mobile external damage target after moving; Se, calculating the width and height of the pixel marking box of the mobile external damage target after moving according to the three-dimensional space coordinates corresponding to the right bottom corner point of the pixel marking box of the mobile external damage target after moving, and internal and external parameters of a camera used for shooting the visual monitoring image; Sf, repeatedly performing steps Sb-Se until the mobile external damage target leaves the target power transmission channel along the driving track.

Images