Abnormal target three-dimensional monitoring method and system based on double-channel iron tower video

Abstract

The embodiment of the invention discloses an abnormal target three-dimensional monitoring method and system based on a double-channel iron tower video, and the method comprises the steps: obtaining first video data of a first video device, and carrying out the real-time monitoring of a target at the periphery of an iron tower based on the first video data; based on the target discovery of the first video device, guiding a second video device to perform pan-tilt control; a binocular stereo positioning system is established based on the first video device and the second video device, and three-dimensional positioning and warning of a target are achieved; three-dimensional positioning of abnormal targets around the power transmission tower can be realized through a double-vision three-dimensional collaborative architecture, potential threats can be accurately found and positioned in real time, early warning can be performed, and safe operation of a power system is guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of power facility monitoring, and particularly to a three-dimensional monitoring method and system for abnormal targets based on dual-path tower videos. Background Art

[0002] Currently, video monitoring systems for transmission towers generally use monocular high-definition cameras in combination with edge computing devices, and relevant algorithms are used to detect targets such as kites and drones. The common technical routes usually include:

[0003] 1) Real-time transmission of video streams through the RTSP protocol; 2) Deployment of lightweight models (accelerated by TensorRT); 3) GPS / IMU-assisted pan-tilt attitude calibration; 4) A monitoring mode combining timed polling and motion detection triggering.

[0004] However, the information obtained by such methods is not accurate and comprehensive enough, and the false alarm rate is high and the target continuous tracking rate is insufficient under complex backgrounds (such as clouds and tree shadows). It can be seen that the current video monitoring of transmission towers cannot meet the monitoring requirements of high precision, real-time performance, and multi-dimensions. Summary of the Invention

[0005] The main purpose of the present invention is to provide a three-dimensional monitoring method and system for abnormal targets based on dual-path tower videos, which can achieve high-precision three-dimensional positioning and real-time monitoring of abnormal targets around transmission towers.

[0006] To achieve the above object, in the first aspect of the present application, a three-dimensional monitoring method for abnormal targets based on dual-path tower videos is provided, and the method includes:

[0007] Obtain the first video data of the first video device, and perform real-time monitoring on the targets around the tower based on the first video data;

[0008] Based on the target discovery of the first video device, guide the second video device to perform pan-tilt control;

[0009] Establish a binocular stereo positioning system based on the first video device and the second video device to perform three-dimensional positioning and warning of the target.

[0010] In the second aspect of the present application, a three-dimensional monitoring system for abnormal targets based on dual-path tower videos is provided, including a single-path video abnormal target detection module, a cross-device collaborative control module, and a binocular stereo positioning module, wherein:

[0011] The single-path video abnormal target detection module is used to obtain the first video data of the first video device, and perform real-time monitoring on the targets around the tower based on the first video data;

[0012] The cross-device collaborative control module is used to guide the second video device to perform pan-tilt control based on the target discovery of the first video device;

[0013] The binocular stereo positioning module is used to establish a binocular stereo positioning system based on the first video device and the second video device, and perform three-dimensional positioning and warning of the target.

[0014] A third aspect of the present application provides a computer-readable storage medium storing a computer program, which when executed by a processor causes the processor to execute each step in the method described in the first aspect.

[0015] The present application provides a three-dimensional monitoring method for abnormal targets based on dual-path tower videos. By obtaining the first video data of the first video device, the targets around the tower are monitored in real time based on the first video data; based on the target discovery of the first video device, the second video device is guided to perform pan-tilt control; based on the first video device and the second video device, a binocular stereo positioning system is established to perform three-dimensional positioning and warning of the target; the three-dimensional positioning of abnormal targets around the transmission tower can be realized through a binocular stereo collaborative architecture, and potential threats can be discovered, located, and warned in real time and accurately, ensuring the safe operation of the power system. Description of the Drawings

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0017] Among them:

[0018] Figure 1 It is a schematic flowchart of a three-dimensional monitoring method for abnormal targets based on dual-path tower videos provided by an embodiment of the present application;

[0019] Figure 2 It is a schematic structural diagram of a three-dimensional monitoring system for abnormal targets based on dual-path tower videos provided by an embodiment of the present application. Detailed Embodiments

[0020] To enable those skilled in the art to better understand the solution of this application, the following will clearly and completely describe the technical solution in the embodiments of this application in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without creative efforts belong to the scope of protection of this application.

[0021] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products, or devices.

[0022] Referring to "embodiments" in this context means that the specific features, structures, or characteristics described in connection with the embodiments can be included in at least one embodiment of this application. The appearance of this phrase at various positions in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0023] The non-maximum suppression (NMS) involved in the embodiments of this application: a screening mechanism for eliminating duplicate markings. When the algorithm generates multiple recognition frames for the same target (such as simultaneously marking 3 kite frames), it automatically retains the most accurate one.

[0024] The PnP algorithm involved in the embodiments of this application: a mathematical method for inferring the position from a photo. By the position of an object in the photo and the tilt angle of the camera itself, the actual spatial coordinates of the target object are calculated, similar to calculating the height of a flagpole based on the length of its shadow.

[0025] The epipolar constraint correction involved in the embodiments of this application: a technology for "aligning perspectives" of dual cameras, which makes the images captured by the left and right cameras parallel like human eyes, ensuring that the same abnormal target seen is in a matchable position in both images.

[0026] The disparity calculation involved in the embodiments of this application: the key principle of binocular vision ranging. By comparing the horizontal position difference of the target in the left and right camera images (similar to the image differences seen by the left and right eyes), the actual distance of the target from the camera is calculated.

[0027] The embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application.

[0028] Please refer to Figure 1 , which is a schematic flow chart of an abnormal target three-dimensional monitoring method based on dual-path tower video provided by an embodiment of the present application. As Figure 1 shown, the method includes:

[0029] 101. Obtain the first video data of the first video device, and perform real-time monitoring on the targets around the tower based on the above first video data.

[0030] In the embodiments of the present application, the execution subject of the method can be a three-dimensional monitoring system for abnormal targets based on dual-path tower video, which will be further introduced later.

[0031] The three-dimensional monitoring method for abnormal targets based on dual-path tower video in the embodiments of the present application mainly aims at the core defects of single-eye perception limitation and three-dimensional positioning lack, studies the binocular stereo three-dimensional reconstruction technology based on the dual-path video of the tower pole, and realizes the meter-level positioning of potential hazard targets; based on the pan-tilt linkage, realizes cross-device collaborative response, constructs a three-dimensional perception and intelligent collaboration technology system, and solves the deficiency of the existing single monitoring dimension.

[0032] In an optional implementation manner, the above first video device adopts an industrial-grade camera, and is built with a GPS / IMU module to record the longitude, latitude and attitude angle of the device;

[0033] The above step 101 includes:

[0034] Use the improved YOLOv5s model to perform target detection processing on the above first video data to obtain the target pixel coordinates, bounding box size and class confidence.

[0035] The real-time detection of abnormal targets in a single video in the present application specifically includes:

[0036] Hardware deployment:

[0037] Camera: The video device can adopt an industrial-grade camera with more than 2 million pixels, and the frame rate ≥ 25fps to ensure the clarity and smoothness of the video stream. The camera is built with a GPS / IMU module to record the longitude, latitude and attitude angle of the device, providing basic data for subsequent three-dimensional positioning;

[0038] Edge computing device: Deploy an NVIDIA Jetson Xavier edge computing device, which has powerful computing capabilities, can process video stream data in real time, and supports the efficient operation of deep learning models;

[0039] Video transmission can obtain a 1080P@25fps video stream through the RTSP protocol to ensure the real-time transmission and low-latency processing of video data;

[0040] Pan-tilt control: A PTZ pan-tilt (continuous 360° horizontal rotation, ±90° vertical) can be used to achieve full coverage of the monitoring area and support remote control and automatic tracking functions.

[0041] Algorithm implementation:

[0042] The YOLOv5s model is selected as the basic architecture because it achieves a good balance between real-time performance and detection accuracy.

[0043] Specifically, the model can be improved by optimizing the network structure to reduce the computational amount while maintaining high detection accuracy. In one implementation, the input size of the improved model is 640×640 pixels, which is suitable for the resource limitations of edge computing devices.

[0044] Furthermore, a custom dataset containing multiple types of hazard targets can be constructed. For example, it can be: kites (more than 5000 annotations), drones (more than 8000 annotations), floating objects (more than 3000 annotations), and engineering vehicles (more than 6000 annotations). The diversity and richness of the dataset ensure that the model can adapt to different scenarios and target forms.

[0045] Optionally, data augmentation techniques (such as random cropping, flipping, color jittering, etc.) are used to expand the dataset and improve the generalization ability of the model.

[0046] Detection algorithm optimization:

[0047] Non-maximum suppression (NMS): Set the NMS threshold to 0.5 to eliminate duplicate detection boxes and retain the most accurate detection results.

[0048] Confidence threshold: Set the confidence threshold to 0.6 to filter out low-confidence detection results and ensure that the output detection results have high reliability.

[0049] TensorRT acceleration: Use TensorRT to optimize the model to achieve an inference speed of about 5ms / frame, meeting the requirements of real-time monitoring.

[0050] The algorithm can output the pixel coordinates (u, v) of the target, the bounding box size (width and height), and the class confidence. This information provides the basic data for subsequent 3D positioning and early warning.

[0051] 102. Based on the target discovery of the above first video device, guide the second video device for pan-tilt control.

[0052] In this application, cross-device visual axis collaborative control can be mainly based on the target discovery of the first video device to guide the second video device for pan-tilt control.

[0053] In an alternative embodiment, step 102 includes:

[0054] Establish the conversion relationship from the coordinate system of the first video device to the WGS84 world coordinate system, and solve the initial value of the target world coordinate corresponding to the target pixel coordinate through the PnP algorithm;

[0055] Based on the initial value of the target world coordinate, calculate the required yaw angle θ and pitch angle φ of the second video device, and control the PTZ pan-tilt of the second video device according to the calculated yaw angle θ and pitch angle φ.

[0056] The specific steps of the above method include:

[0057] 1. The video device that discovers the target is defined as the first video device (Device 1), and establish the conversion of the camera coordinate system of Device 1 → WGS84 world coordinate system:

[0058] X_world = R1 * (X_cam1) + T1

[0059] where R1 is the rotation matrix of Device 1 and T1 is the translation vector of Device 1;

[0060] 2. Solve the initial value of the target world coordinate through the PnP algorithm:

[0061] Based on the target coordinates of Device 1, calculate the required yaw angle θ and pitch angle φ of Device 2:

[0062] θ = arctan2(Δy / Δx)

[0063] φ = arctan2(Δz / √(Δx 2 + Δy2))

[0064] Convert the calculated pan-tilt control parameters into PTZ commands and send them to Device 2 to achieve fast coordinated response.

[0065] 103. Based on the first video device and the second video device, establish a binocular stereo positioning system to achieve three-dimensional positioning and warning of the target.

[0066] In the embodiment of the present application, the first video device and the second video device are jointly calibrated to form a binocular stereo positioning system, and three-dimensional positioning of the abnormal target is achieved through binocular vision technology, and the precise distance between the target and the tower is calculated for early warning.

[0067] In an alternative embodiment, step 103 includes:

[0068] 31. Pre-completed the joint calibration of the first video device and the second video device, and obtain the camera parameters of the first video device and the second video device;

[0069] 32. Correct the images of the first video device and the second video device above to obtain a corrected binocular image;

[0070] 33. Calculate the disparity of the corrected binocular image above to obtain a disparity map;

[0071] 34. Calculate the three-dimensional coordinates of the target above according to the disparity map and the camera parameters above;

[0072] 35. Calculate the distance between the target and the pole tower according to the three-dimensional coordinates of the target above and issue a warning.

[0073] In the camera calibration in the embodiments of the present application, mainly the camera parameters of the two cameras are obtained, including the internal parameters and the relative external parameters, providing a basis for subsequent three-dimensional reconstruction and positioning. Specifically, it may include:

[0074] Internal parameter calibration: Calibrate the two cameras separately to obtain their respective internal parameter matrices K1 and K2, including the focal lengths fx, fy and the principal points cx, cy.

[0075] Relative external parameter calibration: Through the joint calibration of the two cameras, obtain the relative external parameter [R∣T] between the two cameras, that is, the rotation matrix R and the translation vector T.

[0076] In an alternative embodiment, the above step 32 includes:

[0077] Perform distortion correction on the images of the first video device and the second video device above;

[0078] Adjust the images of the first video device and the second video device above so that the positions of the same target in the two images meet the epipolar constraint condition, and obtain a corrected binocular image.

[0079] Specifically, the above step 32 is mainly to eliminate image distortion and ensure that the binocular image meets the epipolar constraint condition. Among them, the images of the two cameras can be distorted corrected based on the Bouguet algorithm to eliminate the influence of lens distortion on the images.

[0080] Epipolar constraint correction: Adjust the images of the two cameras so that the positions of the same target in the two images meet the epipolar constraint condition to ensure the alignment of the binocular images.

[0081] Furthermore, the disparity calculation in the embodiments of the present application is mainly to calculate the disparity map through binocular image matching, providing a basis for three-dimensional reconstruction.

[0082] Specifically, the Semi-Global Matching (SGM) algorithm can be used to calculate the disparity of the rectified binocular images and output a disparity map. Each pixel value in the disparity map represents the horizontal displacement (disparity value d) of the corresponding point in the left and right images.

[0083] Then, the three-dimensional coordinates of the target point can be calculated based on the disparity map and camera parameters.

[0084] Further optionally, the above step 34 includes:

[0085] The three-dimensional coordinates of the above target are calculated through the following formula:

[0086] Z = (f * B) / d

[0087] X = (u - cx) * Z / f

[0088] Y = (v - cy) * Z / f

[0089] Wherein, f represents the focal length, B represents the baseline distance, d represents the disparity value; u is the horizontal pixel coordinate of the target point in the image, cx is the horizontal coordinate of the principal point of the camera, Z is the depth of the target point, v is the vertical pixel coordinate of the target point in the image, and cy is the vertical coordinate of the principal point of the camera.

[0090] Specifically, in the embodiment of the present application, the depth Z of the target point can be calculated according to the disparity value d and camera parameters (focal length f and baseline distance B), and then the three-dimensional coordinates (X, Y, Z) of the target point can be calculated according to the depth Z and pixel coordinates (u, v).

[0091] Further optionally, the above step 35 includes:

[0092] Calculate the three-dimensional Euclidean distance between the target and the tower pole according to the three-dimensional coordinates of the above target;

[0093] Obtain a distance threshold, and judge the three-dimensional Euclidean distance between the target and the tower pole based on the above distance threshold to determine whether to trigger an alarm.

[0094] After determining the three-dimensional coordinates of the target and according to the known three-dimensional coordinates of the tower pole, the three-dimensional Euclidean distance between the target and the tower pole can be calculated.

[0095] Specifically, the distance threshold can be set as needed. When the target distance is lower than the threshold, an alarm signal is triggered.

[0096] For example, a dynamic threshold can be set:

[0097] Horizontal direction: An alarm is triggered when the distance is less than 50 meters.

[0098] Vertical direction: An alarm is triggered when the distance is less than 30 meters.

[0099] Current video monitoring of transmission towers mainly relies on monocular cameras combined with traditional image processing algorithms, which have significant defects: 1) high false alarm rate under complex backgrounds (such as clouds and tree shadows); 2) the monocular system cannot obtain three-dimensional information of the target and can only provide two-dimensional early warnings; 3) most systems adopt a timed round-robin mode, and the target continuous tracking rate is insufficient;

[0100] This solution improves the three-dimensional positioning accuracy to the meter level through a binocular stereo collaborative architecture, filling the gap that real-time three-dimensional monitoring cannot be carried out compared with the monocular system; optimizing the edge computing algorithm to meet the real-time early warning requirements; the multi-device collaborative response significantly improves the target continuous tracking rate. In particular, pure vision three-dimensional monitoring can be realized, avoiding dependence on high-cost devices such as radars, and reducing the comprehensive operation and maintenance cost by about 40%.

[0101] Based on the description of the foregoing method embodiments, the embodiments of the present application further provide an abnormal target three-dimensional monitoring system based on dual-path tower videos.

[0102] Figure 2 It is a schematic structural diagram of an abnormal target three-dimensional monitoring system based on dual-path tower videos provided by the embodiments of the present application. As Figure 2 shown, the abnormal target three-dimensional monitoring system 200 based on dual-path tower videos includes a single-path video abnormal target detection module 210, a cross-device collaborative control module 220, and a binocular stereo positioning module 230, where:

[0103] The above single-path video abnormal target detection module 210 is used to obtain the first video data of the first video device and perform real-time monitoring on the targets around the tower based on the first video data;

[0104] The above cross-device collaborative control module 220 is used to guide the second video device to perform pan-tilt control based on the target discovery of the first video device;

[0105] The above binocular stereo positioning module 230 is used to establish a binocular stereo positioning system based on the first video device and the second video device to perform three-dimensional positioning and warning of the target.

[0106] Specifically, an abnormal target three-dimensional monitoring system based on dual-path tower videos in the embodiments of the present application mainly consists of two core components: a single-path video abnormal target detection module 210 and a binocular stereo positioning module 230. In addition, it also includes hardware devices (such as cameras, edge computing devices, pan-tilts, etc.) and software algorithms (such as target detection algorithms, three-dimensional reconstruction algorithms, etc.). In one implementation, the system architecture is as follows:

[0107] (1) Hardware devices

[0108] Dual-channel industrial cameras: Installed on transmission towers respectively to collect video streams in real time. Each camera is equipped with over 2 million pixels, a frame rate ≥ 25fps, and supports high-definition video acquisition.

[0109] Edge computing device: Such as NVIDIA Jetson Xavier, used to process video streams in real time and run object detection and 3D positioning algorithms.

[0110] PTZ pan-tilt head: Supports continuous horizontal rotation of 360° and vertical rotation of ±90°, used to adjust the camera direction and track targets.

[0111] -GPS / IMU module: Records the longitude, latitude and attitude angles of the camera, providing auxiliary information for 3D positioning.

[0112] (2) Single-channel video abnormal target detection module 210

[0113] It can monitor abnormal targets around the transmission tower in real time, such as kites, drones, floating objects and engineering vehicles. Specifically, an improved YOLOv5s model can be used, with an input size of 640×640 pixels, accelerated by TensorRT, to achieve an inference speed of about 5ms / frame.

[0114] Based on a custom dataset (including four types of targets: kites, drones, floating objects and engineering vehicles) for training to ensure that the model can accurately identify various abnormal targets.

[0115] Set the non-maximum suppression (NMS) threshold to 0.5 and the confidence threshold to 0.6 to filter duplicate detection boxes and low-confidence results.

[0116] Output the pixel coordinates (u, v) of the target, the bounding box size and the class confidence.

[0117] (3) Binocular stereo positioning module 230

[0118] Realize the 3D positioning of abnormal targets through binocular vision technology and calculate the precise distance between the target and the tower. The specific implementation steps can include

[0119] Functions such as camera calibration, image preprocessing and epipolar constraint correction, disparity calculation, 3D coordinate solution and distance warning are not elaborated here.

[0120] (4) Cross-device collaborative control module 220

[0121] When the first video device detects an abnormal target, it quickly guides the second video device to aim at the target to achieve dual-device collaborative response.

[0122] It can implement steps such as coordinate conversion and pan-tilt head control, which are not elaborated here.

[0123] Based on the description of the foregoing system architecture, the system working process may include:

[0124] 1. Video acquisition: Dual cameras collect the video stream around the transmission tower in real time.

[0125] 2. Target detection: The single-channel video abnormal target detection module detects abnormal targets in real time and outputs the pixel coordinates and category information of the targets.

[0126] 3. Cooperative control: When an abnormal target is detected, the pan-tilt direction of the second video device is quickly adjusted through the cross-device cooperative control module to align it with the target.

[0127] 4. Three-dimensional positioning: The binocular stereo positioning module obtains the accurate three-dimensional position of the target through parallax calculation and three-dimensional coordinate solution.

[0128] 5. Warning trigger: According to the distance between the target and the tower, a warning signal is triggered to remind the monitoring personnel to take measures.

[0129] In the system of the embodiments of the present application, through binocular stereo vision technology, three-dimensional positioning with meter-level accuracy is achieved, filling the gap that a monocular system cannot obtain depth information; combined with edge computing devices and optimized algorithms, the requirements of real-time monitoring and warning are met; through pan-tilt linkage and cross-device cooperative control, the continuity and reliability of target tracking are significantly improved; the pure vision solution avoids the dependence on high-cost devices such as radars and reduces the system construction and operation and maintenance costs. This system is applicable to the monitoring of abnormal targets around transmission towers, and can discover and locate potential threats in real time and accurately, ensuring the safe operation of the power system.

[0130] It can be understood that the relevant content related to Figure 2 each module in has been described in detail in the foregoing method embodiments, and specifically, reference can be made to the content in the method embodiments; that is Figure 2 the provided three-dimensional monitoring system 200 for abnormal targets based on dual-channel tower videos can execute any step in the Figure 1 embodiments shown, and details are not described here.

[0131] In one embodiment, a computer-readable storage medium is also proposed. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the processor executes any step in the foregoing method embodiments.

[0132] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0133] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0134] The above-described embodiments merely represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A three-dimensional monitoring method for abnormal targets based on dual-path iron tower videos, characterized in that, The method includes: Obtaining the first video data of the first video device, and performing real-time monitoring on the targets around the iron tower based on the first video data; Based on the target discovery of the first video device, guiding the second video device to perform pan-tilt control; Based on the first video device and the second video device, establishing a binocular stereo positioning system to perform three-dimensional positioning and warning of the target.

2. The abnormal target three-dimensional monitoring method based on the dual-path iron tower video according to claim 1, characterized in that The first video device uses an industrial-grade camera, and is built with a GPS / IMU module to record the longitude, latitude and attitude angle of the device; The real-time monitoring of the targets around the iron tower based on the first video data includes: Using an improved YOLOv5s model to perform target detection processing on the first video data to obtain target pixel coordinates, bounding box sizes and class confidence levels.

3. The abnormal target three-dimensional monitoring method based on the dual-path iron tower video according to claim 2, characterized in that, The guiding the second video device to perform pan-tilt control based on the target discovery of the first video device includes: Establishing the conversion relationship from the coordinate system of the first video device to the WGS84 world coordinate system, and solving the initial value of the target world coordinate corresponding to the target pixel coordinate through the PnP algorithm; Calculating the required yaw angle θ and pitch angle φ of the second video device based on the initial value of the target world coordinate, and controlling the PTZ pan-tilt of the second video device according to the calculated yaw angle θ and pitch angle φ.

4. The abnormal target three-dimensional monitoring method based on the dual-path iron tower video according to claim 3, wherein, The establishing a binocular stereo positioning system based on the first video device and the second video device to achieve three-dimensional positioning and warning of the target includes: Pre-completing the joint calibration of the first video device and the second video device to obtain the camera parameters of the first video device and the second video device; Correcting the images of the first video device and the second video device to obtain corrected binocular images; Performing disparity calculation on the corrected binocular images to obtain a disparity map; Calculating the three-dimensional coordinates of the target according to the disparity map and the camera parameters; Calculating the distance between the target and the tower pole according to the three-dimensional coordinates of the target, and giving an early warning.

5. The abnormal target three-dimensional monitoring method based on dual-path iron tower video according to claim 4, characterized in that, The correcting the images of the first video device and the second video device to obtain corrected binocular images includes: Performing distortion correction on the images of the first video device and the second video device; Adjusting the images of the first video device and the second video device so that the positions of the same target in the two images satisfy the epipolar constraint condition to obtain corrected binocular images.

6. The abnormal target three-dimensional monitoring method based on dual-path tower video according to claim 4, wherein, The calculating the three-dimensional coordinates of the target according to the disparity map and the camera parameters includes: Calculating the three-dimensional coordinates of the target through the following formula: Z = (f * B) / d; X = (u - cx) * Z / f; Y = (v - cy) * Z / f; Wherein, f represents the focal length, B represents the baseline distance, d represents the disparity value; u is the horizontal pixel coordinate of the target point in the image, cx is the horizontal coordinate of the principal point of the camera, Z is the depth of the target point, v is the vertical pixel coordinate of the target point in the image, and cy is the vertical coordinate of the principal point of the camera.

7. The three-dimensional monitoring method for abnormal targets based on dual-path tower videos according to claim 4, wherein, The calculating the distance between the target and the tower pole according to the three-dimensional coordinates of the target, and giving an early warning includes: Calculating the three-dimensional Euclidean distance between the target and the tower pole according to the three-dimensional coordinates of the target; Obtain a distance threshold, and based on the distance threshold, judge the three-dimensional Euclidean distance between the target and the pole tower to determine whether to trigger an alarm.

8. An abnormal target three-dimensional monitoring system based on dual-path tower video, characterized in that, It includes a single-channel video abnormal target detection module, a cross-device collaborative control module, and a binocular stereo positioning module, where: The single-channel video abnormal target detection module is used to obtain the first video data of the first video device and perform real-time monitoring on the targets around the iron tower based on the first video data; The cross-device collaborative control module is used to guide the second video device to perform pan-tilt control based on the target discovery of the first video device; The binocular stereo positioning module is used to establish a binocular stereo positioning system based on the first video device and the second video device to perform three-dimensional positioning and warning of the target.

9. The abnormal target three-dimensional monitoring system based on the dual-path iron tower video according to claim 4, wherein, It further includes the first video device and the second video device, which are respectively installed on the transmission pole tower and used to collect video streams in real time; The edge computing device is used to process the video stream in real time and run target detection and three-dimensional positioning algorithms; The PTZ pan-tilt is used to adjust the camera directions of the first video device and the second video device to track the target; The GPS / IMU module is used to record the longitude, latitude, and attitude angles of the camera and provide auxiliary information for three-dimensional positioning.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, the processor is caused to execute the steps of the method according to any one of claims 1-7.

Images