Electronic fence system based on unmanned aerial vehicle video and implementation method thereof

Abstract

The invention discloses an electronic fence system based on an unmanned aerial vehicle video and an implementation method thereof. The method comprises the following steps: receiving a flight route drawn by a user and an electronic fence geographic area; controlling the unmanned aerial vehicle to fly according to the flight route, collecting video streams, and obtaining space state information of the unmanned aerial vehicle; based on a camera projection model and a coordinate conversion algorithm, mapping the geographic boundary of the electronic fence to a video image pixel coordinate system in real time; an electronic fence boundary is drawn and displayed in the video picture; performing real-time target detection and identification on the video stream, and judging the position relationship between the position of the identified target and the boundary of the electronic fence; and when the target is identified to enter the area enclosed by the boundary of the electronic fence, triggering an early warning signal. According to the invention, the dynamic and flexible deployment of the electronic fence is realized, the monitoring precision and intuition are improved, the real-time intrusion detection and early warning are supported, and the method is suitable for various security and monitoring scenes.

Description

Technical Field

[0001] This invention relates to the field of electronic fence technology, and in particular to an electronic fence system based on drone video and its implementation method. Background Technology

[0002] Electronic fence technology, as an important perimeter security measure, has been widely used in scenarios such as restricted area management, property protection, and area monitoring. Traditional electronic fence technologies mainly rely on fixedly installed sensors (such as infrared beam sensors and vibration fiber optics), surveillance cameras, or GNSS-based terminal equipment. These solutions have significant limitations: fixed systems have high deployment costs and poor flexibility, making it difficult to cover complex terrain or large areas; GNSS solutions suffer from decreased accuracy in signal-blocked environments and lack the ability to perceive three-dimensional space, making them ineffective against intrusions from the air. Furthermore, existing technologies largely rely on manual monitoring or simple rule-based judgments, resulting in low levels of intelligence and risks of response delays and high false negative rates.

[0003] In recent years, with the rapid development of the drone industry and computer vision technology, utilizing drones as aerial mobile sensing platforms has become an effective way to solve the aforementioned problems. Drones possess a high field of view, excellent maneuverability, and flexible deployment capabilities, enabling them to quickly reach target areas and acquire high-definition video data. Simultaneously, advancements in image processing, target detection, and geographic information technology (GIS) have made it possible to extract and analyze geographic information from video streams in real time.

[0004] Therefore, developing a dynamic electronic fence system that integrates real-time UAV video, precise geospatial calibration, and intelligent video analysis technologies to overcome the inherent defects of traditional technologies has become an important and valuable development direction in this field. Summary of the Invention

[0005] To address the above problems, this invention provides an electronic fence system based on UAV video and its implementation method, aiming to solve the problems of inflexible deployment, poor terrain adaptability, and lack of three-dimensional monitoring capabilities of traditional electronic fence systems, and to achieve dynamic, accurate, and intelligent aerial perimeter security.

[0006] This invention provides a method for implementing an electronic fence system based on drone video, comprising: S1 receives the flight path drawn by the user on the front-end interface and the defined geofence geographic information area. S2 sends flight commands to the drone, controls the camera equipment on the drone to fly along the flight path and collect real-time video streams, and at the same time obtains the drone's spatial status information; S3, based on the camera projection model and coordinate transformation algorithm, maps the boundary coordinates of the geographic information area of ​​the electronic fence to the image pixel coordinate system of the video stream in real time; S4, in the video stream, draw and display the electronic fence boundary based on the mapped pixel coordinates; S5 performs real-time target detection and recognition on the video stream, and predicts the target's trajectory based on the target detection results and the UAV's spatial state information.

[0007] Furthermore, the spatial status information includes the latitude and longitude of the UAV, its flight altitude, camera yaw angle, pitch angle, and zoom level.

[0008] By acquiring the precise spatial attitude and camera parameters of the drone, the system can more accurately establish the mapping relationship between video images and geographic coordinates, improve the mapping accuracy of the electronic fence in the video footage, and enhance the system's adaptability to dynamic environments.

[0009] Furthermore, S3 specifically includes: S31. Based on the difference in latitude and longitude between the UAV and the target point, and combined with the Earth ellipsoid model, calculate the local planar coordinates (ΔE, ΔN) of the target point relative to the current position of the UAV. S32, based on the real-time yaw angle of the camera device, perform a rotation transformation on the local plane coordinates (ΔE, ΔN) to obtain image plane coordinates (x', y') aligned with the camera's viewpoint; S33, based on the drone's flight altitude and the camera equipment's internal parameters, map the image plane coordinates (x', y') to the video image's pixel coordinates (u, v).

[0010] By breaking down the complex coordinate mapping process into three logically clear and progressively advancing calculation steps, a high-precision conversion from the geographic coordinate system to the image pixel coordinate system is achieved. This ensures the real-time and accurate display of the electronic fence boundary in the video stream, enhances the system's robustness to changes in drone attitude, and improves user experience and monitoring effectiveness.

[0011] Furthermore, the local planar coordinates of the target point relative to the current position of the UAV are calculated using the following formula: ; ; in, Indicates the difference in latitude; Indicates the difference in longitude; Indicates the latitude value of the current location; R earth This represents the average radius of the Earth.

[0012] By introducing Earth curvature correction, the accuracy of long-distance coordinate transformation is improved, avoiding errors caused by planar approximation, making it suitable for large-scale, high-precision electronic fence applications.

[0013] Furthermore, the local planar coordinates are rotated using the following formula: ; Where x' is the rotated eastward component; y' is the rotated northward component; θ is the camera yaw angle.

[0014] It effectively solves the problem of misalignment between the image coordinate system and the geographic coordinate system caused by changes in the drone's flight path, ensuring that the direction of the electronic fence in the video is consistent with the actual geographic direction, thus improving visual intuitiveness and operational accuracy.

[0015] Furthermore, the image plane coordinates are mapped to pixel coordinates using the following formula: ; ; in,( , () represents the center coordinates of the pixel; z represents the flight altitude of the drone; , These are the focal length parameters along the x and y axes of the camera intrinsic parameter matrix, respectively.

[0016] By combining the camera's internal parameters with the drone's altitude information, a precise mapping from physical space to image pixels is achieved, ensuring stable display of the electronic fence under different flight altitudes and lens zoom conditions.

[0017] Furthermore, the method also includes: S5 performs real-time target detection and recognition on the video stream, and determines the positional relationship between the identified target and the electronic fence boundary. When a target is detected entering the area enclosed by the electronic fence boundary, an early warning signal is triggered.

[0018] It enables automatic identification and real-time alarm of intrusion behavior within the electronic fence area, significantly improving the system's intelligence level and security effectiveness, reducing manual monitoring costs, and enhancing emergency response capabilities.

[0019] Furthermore, S5 specifically includes: Real-time target detection is performed on the video stream, and the geometric center coordinates of the target's bounding rectangle are extracted as the target pixel coordinates; The ray casting method is used to determine the positional relationship between the target pixel coordinates and the mapped area of ​​the electronic fence in the image pixel coordinate system; When a target is determined to have entered the electronic fence area by ray casting, and the same determination result is obtained in multiple subsequent frames, an early warning signal is triggered.

[0020] By introducing a triple approach of geometric center coordinate positioning, ray casting for position determination, and multi-frame continuous verification, a synergistic improvement in positioning accuracy, system reliability, and computational efficiency is achieved: by extracting the geometric center coordinates of the target's circumscribed rectangle and combining them with the ray casting method to accurately determine its positional relationship with the electronic fence, the accuracy of judgment in complex scenarios is significantly improved; the multi-frame continuous verification mechanism effectively filters out instantaneous false detections, enhancing the system's anti-interference capability and early warning reliability; at the same time, the simplified calculation model ensures processing efficiency, enabling the system to issue timely and accurate early warnings even under stringent real-time requirements.

[0021] The present invention also provides an electronic fence system based on drone video, comprising: The front-end interaction module is used by users to draw flight routes, define electronic fence geographic information areas, and display monitoring screens. The drone flight platform is equipped with camera equipment, GPS module and inertial measurement unit, and is used to perform flight missions, collect video streams and collect its own spatial status information; The communication module is used to establish a data transmission link between the front-end interaction module, the back-end service module and the UAV flight platform; The backend service module is used to process flight commands, video streams, and status information; The video analytics module is used to perform target detection and recognition, positional relationship determination, and early warning triggering operations; The cloud storage module is used to store flight route files, video data, and system configuration information.

[0022] A complete and collaborative hardware and software system has been built, realizing full-process automation from task planning, data acquisition, processing and analysis to storage and early warning. It has good scalability, stability and practicality, and is suitable for electronic fence applications in a variety of complex scenarios.

[0023] Furthermore, the process of predicting the motion trajectory includes: collecting a synchronous dataset containing target coordinates, timestamps, and UAV status; Extract the motion features of the target within a preset number of historical frames. The motion features include instantaneous velocity change rate, trajectory curvature change coefficient, and acceleration variance. The extracted motion features are compared with a preset dynamic judgment threshold. When all the motion features meet the linear motion condition, it is judged as linear motion; otherwise, it is judged as nonlinear motion. If the motion is determined to be linear, an adaptive Kalman filter algorithm is used to predict the target's future position. If the motion is determined to be nonlinear, then particle filtering or extended Kalman filtering algorithms are used to predict the future position of the target. Based on the spatial state information of the UAV, a comprehensive error model is constructed that includes GPS positioning error, IMU attitude drift error and camera jitter error; An extended Kalman filter (EKF) is used, and during the update phase of the filtering algorithm, the comprehensive error model is used to correct the predicted trajectory of the target. The compensated target coordinates are back-mapped to the image pixel coordinate system through the camera projection model to verify the compensation accuracy, and the filtering parameters are dynamically adjusted based on the verification results. Predictive analysis is performed based on the compensated trajectory data, and the prediction results are used for early warning and visual monitoring.

[0024] By synchronously collecting target coordinates and UAV status data, the reliability of the data foundation for trajectory analysis is ensured; the algorithm selection mechanism based on motion characteristics takes into account both prediction accuracy and computational efficiency in linear and nonlinear motion scenarios; the introduction of UAV motion compensation effectively eliminates apparent motion error, making trajectory prediction truly reflect the target's ground motion status; finally, the prediction results are deeply integrated into early warning judgment and visual monitoring, significantly improving the system's active defense capability and situational awareness level, and realizing a technological leap from passive alarm to active early warning.

[0025] Compared with existing technologies, the advantages of this invention are as follows: By receiving user-defined flight paths and electronic fence areas, the UAV can quickly fly to the designated airspace, enabling temporary or long-term monitoring of any shape and area, greatly improving the flexibility and adaptability of the system application. Through camera projection models and coordinate transformation algorithms, the boundary coordinates of the geographic information system are mapped in real-time and accurately to the image pixel coordinate system of the video stream, ensuring that the electronic fence boundary highly matches the real geographical environment in the video footage. Drawing and displaying the electronic fence boundary in the real-time video stream allows monitoring personnel to intuitively see the superposition effect of the virtual fence and the real scene, clearly grasping the scope of the monitored area and the relative position of targets, greatly reducing the cognitive load of monitoring personnel and improving situational awareness and decision-making efficiency. The invention organically combines the UAV's aerial maneuvering vision, the front-end human-machine interface, and the back-end intelligent processing capabilities to form a complete solution. It achieves a closed-loop process from task planning, data acquisition, real-time processing to visual feedback, providing efficient and intelligent technical means for applications such as regional security, boundary patrol, and facility protection. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this drawing or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this drawing. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0027] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments provided by this invention without inventive effort are within the scope of protection of this invention.

[0029] This invention provides a method for implementing an electronic fence system based on drone video, such as... Figure 1 As shown, the specific steps include: S1 receives the flight path drawn by the user on the front-end interface and the defined geofence geographic information area.

[0030] Specifically, users can manually plot the drone's flight path through the GIS map interface on the application's front end, and then automatically upload the plotted flight path to OSS cloud storage.

[0031] S2, send flight commands to the drone, control the camera equipment on the drone to fly along the flight path and collect real-time video streams, and at the same time obtain the spatial status information of the drone, including the latitude and longitude of the drone, flight altitude, camera yaw angle, pitch angle and zoom magnification.

[0032] Specifically, the user selects a stored flight path on the front end and sends a flight command to the application backend. After verifying the legality of the command, the application backend forwards the command to the target drone through the communication module. After receiving the command, the drone reads the corresponding flight path file from the OSS cloud storage and starts the takeoff procedure. After takeoff, the drone automatically turns on the high-definition camera, pushes a real-time video stream to the streaming media service, and sends its own spatial status information to the application backend, including the drone's latitude and longitude, flight altitude, camera yaw angle, pitch angle, and zoom magnification.

[0033] During drone flight, users can select one of two fence modes through the front-end fence configuration function: Mode 1: Call a preset GIS area, such as a stored protected area boundary or controlled area; Mode 2: Real-time plotting, where multiple points are clicked on in the front-end video screen to form a polygonal fence.

[0034] After the user confirms the fence area, they can click "Enable Fence," and the configuration information (including fence coordinate nodes and alert triggering conditions) will be synchronized to the application backend.

[0035] S3, based on the camera projection model and coordinate transformation algorithm, maps the boundary coordinates of the electronic fence geographic information area to the image pixel coordinate system of the video stream in real time.

[0036] Specifically, after receiving the fence configuration information, the application backend initiates the pixel-to-geographic coordinate conversion process to accurately plot the fence in the video frame. This includes the following steps: S30 associates the corresponding UAV code with the task ID; based on the UAV code, it obtains the corresponding UAV spatial status information.

[0037] S31. Based on the difference in latitude and longitude between the UAV and the target point, and combined with the Earth ellipsoid model, calculate the local planar coordinates (ΔE, ΔN) of the target point relative to the current position of the UAV.

[0038] Specifically, the difference between the latitude and longitude of the drone's current location and the latitude and longitude of the target point (fence node) is calculated and converted into radians.

[0039] Based on the Earth ellipsoid model, the formula for calculating the horizontal distance ΔE in the longitude direction (east-west) is: ; Among them, R earth Indicates the average radius of the Earth; This represents the difference in longitude after conversion to radians; φ0 represents the latitude value of the current location of the drone.

[0040] The formula for calculating the horizontal distance ΔN in the latitudinal direction (north-south direction) is: ; in, This represents the difference in latitude after conversion to radians.

[0041] Specific examples are as follows: If the drone's current latitude φ0 = 39.9042° and longitude λ0 = 116.4074°, and the target point's latitude φ = 39.9045° and longitude λ = 116.4078°, and the Earth's average radius R... earth =6371000m.

[0042] Calculate the latitudinal distance ΔN: ; ΔN=6371000×0.00005235≈33.36m.

[0043] Calculate the distance ΔE in the longitude direction: ; ; ΔE=6371000×0.00000698×0.767≈34.1m; The target point is located 33.36 meters north of the drone and 34.1 meters east of it.

[0044] S32, based on the real-time yaw angle of the camera device, perform a rotation transformation on the local plane coordinates (ΔE, ΔN) to obtain the image plane coordinates (x', y') aligned with the camera's viewpoint.

[0045] The direction of the target is determined by rotation transformation. The purpose is to transform the local planar coordinates (ΔE, ΔN) from the world coordinate system to the image coordinate system, thereby aligning with the actual shooting angle of the drone camera and ensuring that the electronic fence plotting position is completely matched with the real-time view of the camera.

[0046] The yaw angle of the drone camera directly determines its shooting direction. If this rotation transformation is not performed, although the local plane coordinates (ΔE, ΔN) still take the current position of the drone as the origin, its direction will deviate from the image coordinate system (i.e. the coordinate system under the camera's view), which will eventually lead to the incorrect plotting position of the electronic fence in the video.

[0047] Specifically, the rotation transformation is accomplished using a rotation matrix. The core principle is to utilize the camera's yaw angle θ to convert the eastward component ΔE and northward component ΔN in the world coordinate system into new components (x', y') in the image coordinate system. The specific steps are as follows: The yaw angle output by the camera is usually in degrees, but trigonometric function calculations need to be based on radians (rad), so unit conversion is required first: ; in, This is the angle value of the camera's yaw angle; This is the converted radian value.

[0048] Based on the mathematical principles of planar coordinate rotation, a rotation matrix R(θ) is constructed to characterize the rotational mapping relationship between the world coordinate system and the image coordinate system: ; Geometrically, the rotation matrix rotates the world coordinate system around the origin (the drone's current position) by the camera yaw angle θ, thus aligning it perfectly with the direction of the image coordinate system.

[0049] Transform the local plane coordinates (ΔE, ΔN) in the world coordinate system into new coordinates (x', y') in the image coordinate system using the rotation matrix R(θ): ; After unfolding, we get: .

[0050] Specific examples are as follows: When the yaw angle is 45° and the target point's UTM coordinates (i.e., local plane coordinates) are (100, 100), convert the yaw angle to radians: ; Calculate the rotation matrix: ; Applying the rotation formula: .

[0051] The results x′=141.42 m and y′=0 m indicate that the target point is located at the center of the right side of the image.

[0052] S33, based on the drone's flight altitude and the camera equipment's internal parameters, map the image plane coordinates (x', y') to the video image's pixel coordinates (u, v).

[0053] Calculating the horizontal distance Δu from the target point to the image center is a crucial step in quadrant determination within an electronic fence system. This is achieved by mapping the rotated coordinates (x', y') to pixel coordinates (u, v) and calculating the distance relative to the image center coordinates (u, v). , The difference between the horizontal and vertical distances (Δv) can determine the lateral position of the target point. This process, combined with the calculation of the vertical distance Δv, can accurately determine the quadrant position of the target point, providing a reliable basis for the dynamic mapping and intrusion detection of electronic fences.

[0054] Calculating the horizontal pixel offset Δu from the target point to the image center is a crucial step in accurately determining the target's position in an electronic fence system. Specifically, this involves first mapping the rotated spatial coordinates (x', y') to the pixel coordinates (u, v) of the video image, and then comparing this with the pixel coordinates of the image center (u, v). , Perform the difference operation (i.e.) This yields the horizontal position of the target point relative to the center of the image; combined with the vertical offset... This allows for precise determination of the target point's location within a quadrant (or region) of the video frame. This process provides accurate information for the dynamic mapping of electronic fences on the video and for subsequent intelligent detection of whether a target has intruded into the fence.

[0055] Specifically, the spatial coordinates of the image plane are converted into pixel coordinates of the video frame using the following formula: ; ; Where z is the flight altitude of the drone; , These are the focal length parameters along the x and y axes of the camera intrinsic parameter matrix, respectively.

[0056] After obtaining the pixel coordinates (u, v), it is necessary to further calculate the pixel offset, which is the difference between the pixel coordinates and the image center coordinates. This clarifies the position of the target point relative to the center of the image. The calculation formula is as follows: ; ; By using horizontal offset Δu (to determine if the target is to the left or right) and vertical offset Δv (to determine if the target is to the top or bottom), the relative position of the target in the video frame can be accurately located, providing core location information for subsequent pixel-level mapping and intrusion detection of electronic fences.

[0057] Specific examples are as follows: If the coordinates after rotation by S32 are: x′=10 m, y′=5 m; Flight altitude: z = 100m; The camera's intrinsic parameters are: ; but: ; Δu = 1010 - 960 = 50px; Δv = 565 - 540 = 25px; Therefore, the target point is located 50 pixels to the right and 25 pixels below the center of the image.

[0058] S4, in the video stream, draws and displays the electronic fence boundary based on the mapped pixel coordinates.

[0059] Specifically, the pixel coordinates of multiple fence nodes are connected using OpenCV tools to generate a polygonal electronic fence, and the plotted video stream is then pushed back to the streaming media service.

[0060] S5, perform real-time target detection and recognition on the video stream, and determine the positional relationship between the identified target and the electronic fence boundary.

[0061] The specific implementation process is as follows: S51, Real-time Video Stream Access and Frame Extraction: The system receives real-time video streams pushed by the UAV flight platform through the communication module. The video analysis module pulls video frames from the streaming media service module at a fixed frame rate and converts them into RGB or grayscale image formats for subsequent processing. To ensure real-time performance, the system adopts a "frame buffering and parallel processing" mechanism to effectively alleviate video stream transmission latency and avoid lag in target detection.

[0062] S52, Target Detection Algorithm Selection and Execution: To balance real-time performance and detection accuracy in security scenarios, the system selects lightweight target detection algorithms that support small target recognition and have a single-frame processing time of less than 40ms, such as YOLOv5s or YOLOv8n, ensuring smooth processing in a 30fps video stream while maintaining a high average accuracy (mAP ≥ 0.85). Specific execution steps include: Scale the video frames to the model input size (e.g., 640×640), perform pixel normalization and necessary color gamut conversion; With the help of CPU or GPU acceleration, output the class label, confidence score and bounding rectangle coordinates of the target in the image.

[0063] S53, Target Pixel Coordinate Extraction: The system uses the geometric center of the target's bounding rectangle as its representative position in the image. The pixel coordinate calculation formula is as follows: u obj =(x1+x2) / 2; v obj =(y1+y2) / 2; Among them, (u obj v obj () represents the coordinates of the target's center point; (x1, y1) represents the coordinates of the top-left vertex of the target's bounding rectangle; (x2, y2) represents the coordinates of the lower right corner vertex of the target's bounding rectangle.

[0064] To suppress coordinate jumps caused by video jitter or detection fluctuations, the center coordinates of the same target in multiple consecutive frames are smoothed and filtered to improve the stability and reliability of the location data.

[0065] S54, Geographic Fence Pixel Region Determination: After coordinate transformation, the geographic boundary of the electronic fence is represented as a closed polygon region in the image pixel coordinate system. The video analysis module obtains this polygon data from the backend service and caches it locally to reduce latency caused by repeated requests. The system uses the Ray Casting Algorithm to determine the positional relationship between the target pixel and the fence polygon: if the number of intersections between a horizontal ray drawn to the right from the target point and the polygon boundary is odd, then the target is determined to be inside the electronic fence.

[0066] S55, Boundary and Special Case Handling: To enhance the robustness of the system in real-world scenarios, the following handling mechanisms are implemented: Boundary handling: If the target point is located at the fence boundary, it will be judged as "intrusion" or "boundary alert" according to preset rules; Multi-target processing: If multiple targets are detected in a single frame image, the above ray method is performed on the center coordinates of each target to determine the position status of each target, i.e., whether the target is inside or outside the electronic fence; False alarm filtering: For targets initially identified as entering the fence, the system will continue to verify them in the following 2-3 frames. Only when multiple consecutive frames confirm that the target is inside will the final intrusion alarm be triggered, effectively reducing false alarms caused by false detection in a single frame.

[0067] S6 triggers an early warning signal when a target is detected entering the area enclosed by the electronic fence boundary.

[0068] The accuracy and reliability of this invention rely on the following core theories to ensure the accuracy of coordinate transformation, spatial positioning, and video analysis: (1) Theories related to geographic information and spatial positioning Latitude and longitude to geocentric coordinate system conversion (WGS-84): Converting the latitude and longitude (φ, λ, h) of the UAV / target point to geocentric rectangular coordinates (x, y, z) provides a basis for cross-coordinate system calculations, as shown in the following formula: x = (N + h) cos(φ) cos(λ); y = (N + h) cos(φ) sin(λ); z = [(1 - e²)N + h] sin(φ); Where φ is latitude; λ represents longitude; h represents altitude; N is the radius of curvature of the zonal loop; 'a' represents the Earth's semi-major axis; e² is the square of the first eccentricity of the Earth.

[0069] Geodetic distance formula (Haversine): The formula for quickly calculating the spherical distance d between two latitude and longitude points to verify coordinate transformation accuracy is as follows: ; Where r represents the Earth's average radius.

[0070] UTM coordinate transformation (Gauss-Kruger projection): Latitude and longitude coordinates are transformed into Cartesian coordinates (X, Y) using Gauss-Kruger projection for drawing and managing electronic fence paths.

[0071] (2) Theory of video projection and viewpoint calculation Relationship between field of view (FOV) and lens focal length: To determine the monitoring coverage area of ​​a drone camera and prevent the fence from extending beyond the field of view, the formula is as follows: ; ; Where w is the horizontal resolution of the image; p is the vertical resolution of the image; f is the camera's focal length.

[0072] Pin-hole camera projection model: Establishing the mapping relationship between three-dimensional spatial points and two-dimensional pixels is the core of pixel-to-geographic coordinate transformation, and the formula is as follows: ; Where K is the camera intrinsic parameter matrix; O is the rotation matrix; t is the translation matrix; (X, Y, Z) are spatial points in world coordinates; s is the scale factor.

[0073] Triangulation principle: 3D positioning is achieved by calculating the target depth (distance) using parallax, as shown in the following formula: ; ; Where B is the baseline distance of the binocular camera; d is the disparity value; x This represents the lateral displacement on the image.

[0074] The present invention also provides an electronic fence system based on drone video, comprising: The front-end interaction module allows users to draw flight routes, define geofence geographic information areas, and display monitoring screens.

[0075] The drone flight platform is equipped with camera equipment, GPS module and inertial measurement unit, and is used to perform flight missions, collect video streams and collect its own spatial status information.

[0076] The communication module is used to establish a data transmission link between the front-end interaction module, the back-end service module and the UAV flight platform.

[0077] The backend service module is used to process flight commands, video streams, and status information.

[0078] The streaming media service module is used to receive and distribute video streams pushed by drones.

[0079] The video analytics module is used to perform target detection and recognition, motion trajectory prediction, positional relationship determination, and early warning triggering.

[0080] The implementation process of motion trajectory prediction is as follows: The motion trajectory prediction function relies on the collaborative data supply from multiple modules within the system to ensure data consistency and timeliness. The specific implementation includes the following steps: Data Synchronization Acquisition: To maintain real-time consistency with target detection, the system acquires data at the same frame rate (e.g., 30fps), obtaining a complete set of data containing "target coordinates - timestamp - UAV status" every 1 / 30th of a second (approximately 33 milliseconds) to form a historical dataset for trajectory analysis. This dataset needs to be correlated with the UAV's spatial status information (latitude and longitude, flight altitude H, camera yaw angle θ, frame interval Δt) and the target's geographic coordinates. The timestamp synchronization accuracy is ≤10ms to avoid errors caused by asynchronous data.

[0081] Motion pattern recognition and algorithm adaptation: Based on the target motion characteristics, it is categorized into two types: linear motion and nonlinear motion. To balance real-time performance and prediction accuracy, the system selects appropriate lightweight prediction algorithms for different motion patterns. For linear moving targets, Kalman filtering is used for prediction, which has excellent estimation effect and computational efficiency in uniform or uniformly accelerated motion models. For nonlinear moving targets, extended Kalman filtering or lightweight particle filtering is selected to fit the complex motion trend of the target through nonlinear state equations, so as to meet the real-time processing requirements of the system while ensuring a certain level of accuracy.

[0082] Specifically, three core features of the target's historical trajectory over the past 10 frames are extracted: Instantaneous velocity change rate η: Based on the difference in geographic coordinates of the target between adjacent frames (ΔX=ΔE) n -ΔEn-1 ΔY=ΔN n -ΔN n-1 Calculation speed η=|v n -v n-1 | / v n-1 (v) n-1 ≠0); The trajectory curvature variation coefficient κ: The radius of curvature R is calculated by taking the geographic coordinates of three consecutive frames and using the three-point circle method. n R n-1 , κ=|1 / R n - 1 / R n-1 |; Acceleration variance σ²: Acceleration a is calculated based on the velocity of the last 10 frames. k =(v k – v k-1 ) / Δt, σ²=Σ(a k - ā)² / 9 (ā is the mean acceleration; v k Let a be the instantaneous velocity of the k-th frame; k (This is the instantaneous acceleration in the k-th frame).

[0083] Set dynamic thresholds (which can be adapted to different scenarios via the front-end interaction module): η≤0.2m / s², κ≤0.1rad / m, σ²≤0.05m 2 / s 4 If the motion is linear, it is considered linear; otherwise, it is considered nonlinear. The recognition accuracy is ≥92%.

[0084] The linear motion employs an adaptive Kalman filter, dynamically adjusting parameters by the UAV's yaw rate of change (increasing the process noise covariance by 20% when it is greater than 0.5° / s, and decreasing it by 10% otherwise) and flight altitude (increasing the observation noise covariance by 15% when it is greater than 100m, and decreasing it by 10% otherwise), with the position prediction error being less than or equal to 0.2m.

[0085] Specifically, a 4-dimensional state vector X1 is defined to describe the target's motion state, including position and velocity: ; in, and These are the eastward and northward speeds, respectively.

[0086] The uniform motion model is as follows: ; ; Where F is the state transition matrix; Assuming it is process noise, set to zero-mean Gaussian noise; Δt is the frame interval time; The predicted state at the current moment; This represents the optimal state at the previous moment.

[0087] The actual observed value at the current moment is obtained using the following equation. : ; in, To observe noise; G is the observation matrix.

[0088] The adjustment formula for the process noise covariance Q is as follows: ; Where Q0 is the initial process noise covariance, which can be set as a diagonal matrix; This represents the rate of change of yaw angle.

[0089] The formula for adjusting the observation noise covariance R is as follows: ; Where R0 is the initial process noise covariance, which can be set as a diagonal matrix.

[0090] Predict the current frame state based on a uniform motion model: ; ; in, Let be the optimal state covariance matrix of the previous time step; Let be the covariance matrix of the predicted state at the current moment; Let F be the transpose of F.

[0091] Status Update: ; ; ; in, The Kalman gain at the current moment; Let G be the transpose of G; I is the identity matrix.

[0092] Through the aforementioned adaptive adjustment mechanism, the system can maintain high prediction accuracy even when the attitude and altitude of the UAV change, achieving stable and accurate estimation of the trajectory of linear moving targets.

[0093] Nonlinear motion employs lightweight particle filtering, reducing the number of particles from 500 to 200 through Latin hypercube sampling. Dynamic weights are assigned to particles based on the drone's yaw angle (particles with a yaw angle deviation of less than or equal to 10° are multiplied by 1.2, and those with a deviation greater than 30° are multiplied by 0.8). The single-frame processing time is less than or equal to 30ms, meeting the requirements of 30fps real-time video streaming.

[0094] Specifically, for a target undergoing nonlinear motion, a 2D state vector X2 is used to describe its position state: .

[0095] The particle filtering steps are as follows: D particles are sampled from the prior distribution, where D=200, and each particle has a weight of 1 / D; The particles propagate according to the motion model. For nonlinear motion, a uniform motion model with random perturbations is used: ; in, The velocity is obtained through differential estimation of the target's historical trajectory. ; For the random perturbation of the i-th particle; Let i be the state of the i-th particle at the current moment; Let be the state vector of the i-th particle at the previous time step.

[0096] Update particle weights based on observations: ; in, Let be the weight of the i-th particle at the current moment; Let be the weight of the i-th particle at the previous time step; The observation likelihood function represents the state of the particle. The following observations were obtained The probability density function is usually taken as a Gaussian distribution. .

[0097] The weights are adjusted based on the deviation between the target and the UAV's yaw angle: 1) Calculate the angle of the target relative to the UAV ; 2) Calculate the angle deviation ; 3) Adjust the weights using the following formula: ; 4) Normalize the weights: .

[0098] Whether to resample is determined based on the number of effective particles. When the number of effective particles is less than a preset threshold (usually set to D / 2 or D / 3), resampling is performed. The resampling method is system resampling.

[0099] State estimation Weighted average of the particle set: .

[0100] Drone motion error compensation: Since the drone's own flight will cause the target's "apparent motion" (not the target's own motion) to appear in the video footage, this system integrates the drone's real-time spatial state data (such as position, attitude, etc.) to perform coordinate system one and motion compensation on the detected target coordinates, thereby eliminating the observation error introduced by the drone's own motion and ensuring that the trajectory prediction results reflect the target's true motion trajectory relative to the ground.

[0101] Specifically, based on the spatial state information of the UAV, a comprehensive error model is constructed that includes GPS positioning error, IMU attitude drift error, and camera jitter error.

[0102] GPS positioning error ΔGPS = ±0.3 × sinθ G (θ) G The angle of GPS signal obstruction, θ G ∈[0,90°]); IMU Attitude Drift Error (α is the attitude drift angle); Camera shake error ΔPixel = ±1×(fx / H) (fx is the camera intrinsic parameter x-axis focal length).

[0103] An extended Kalman filter (EKF) is used, and during the update phase of the filtering algorithm, the predicted trajectory of the target is corrected using the comprehensive error model.

[0104] Specifically, extended Kalman filter (EKF) fusion compensation is adopted. In the prediction stage, the target coordinates are predicted based on the UAV flight speed v_drone and yaw angle θ (X_pred=X_prev+v_drone×Δt×cosθ, Y_pred=Y_prev+v_drone×Δt×sinθ), where (X_prev, Y_prev) are the coordinates of the previous time step. During the update phase, the predicted values ​​are corrected using the error model parameters. The compensated target coordinates are then mapped back to the image pixel coordinate system through the camera projection model to verify the compensation accuracy. Based on the verification results, the filtering parameters are dynamically adjusted.

[0105] Prediction Results and System Collaboration: The trajectory prediction results directly serve the system's needs for "early warning" and "visual monitoring." By predicting the future location of the target, the system can anticipate potential intrusion behaviors and make forward-looking positional judgments with the electronic fence boundary. This enables the early warning triggering mechanism to be implemented in advance and the predicted trajectory to be visualized in the monitoring screen, enhancing the entire system's proactive defense and situational awareness capabilities.

[0106] The cloud storage module is used to store flight route files, video data, and system configuration information.

[0107] It should be noted that the present invention is not limited to the above-described embodiments. The above embodiments are merely examples, and any embodiments that have the same structure and perform the same effects as the technical concept within the scope of the present invention are included within the scope of the present invention. Furthermore, various modifications that can be conceived by those skilled in the art to the embodiments, and other ways of constructing by combining some of the constituent elements of the embodiments, without departing from the spirit of the present invention, are also included within the scope of the present invention.

Claims

1. A method for implementing an electronic fence system based on drone video, characterized in that, include: S1 receives the flight path drawn by the user on the front-end interface and the defined geofence geographic information area. S2 sends flight commands to the drone, controls the camera equipment on the drone to fly along the flight path and collect real-time video streams, and at the same time obtains the drone's spatial status information; S3, based on the camera projection model and coordinate transformation algorithm, maps the boundary coordinates of the geographic information area of ​​the electronic fence to the image pixel coordinate system of the video stream in real time; S4, in the video stream, draw and display the electronic fence boundary based on the mapped pixel coordinates; S5 performs real-time target detection and recognition on the video stream, and predicts the target's trajectory based on the target detection results and the UAV's spatial state information.

2. The implementation method of an electronic fence system based on UAV video as described in claim 1, characterized in that, The spatial status information includes the latitude and longitude of the UAV, flight altitude, camera yaw angle, pitch angle, and zoom level.

3. The implementation method of an electronic fence system based on UAV video as described in claim 2, characterized in that, S3 specifically includes: S31. Based on the difference in latitude and longitude between the UAV and the target point, and combined with the Earth ellipsoid model, calculate the local planar coordinates (ΔE, ΔN) of the target point relative to the current position of the UAV. S32, based on the real-time yaw angle of the camera device, perform a rotation transformation on the local plane coordinates (ΔE, ΔN) to obtain image plane coordinates (x', y') aligned with the camera's viewpoint; S33, based on the drone's flight altitude and the camera equipment's internal parameters, map the image plane coordinates (x', y') to the video image's pixel coordinates (u, v).

4. The implementation method of an electronic fence system based on UAV video as described in claim 3, characterized in that, The local planar coordinates of the target point relative to the current position of the UAV are calculated using the following formula: ； ； in, Indicates the difference in latitude; Indicates the difference in longitude; Indicates the latitude value of the current location; R earth This represents the average radius of the Earth.

5. The implementation method of an electronic fence system based on UAV video as described in claim 3, characterized in that, The following formula is used to perform a rotation transformation on the local planar coordinates: ； Where x' is the rotated eastward component; y' is the rotated northward component; θ is the camera yaw angle; The following formula is used to map image plane coordinates to pixel coordinates: ； ； in,( , () represents the center coordinates of the pixel; z represents the flight altitude of the drone; , These are the focal length parameters along the x and y axes of the camera intrinsic parameter matrix, respectively.

6. The implementation method of an electronic fence system based on UAV video as described in claim 1, characterized in that, The real-time target detection and recognition of the video stream specifically includes: Real-time target detection is performed on the video stream, and the geometric center coordinates of the target's bounding rectangle are extracted as the target pixel coordinates; The ray casting method is used to determine the positional relationship between the target pixel coordinates and the mapped area of ​​the electronic fence in the image pixel coordinate system; When a target is determined to have entered the electronic fence area by ray casting, and the same determination result is obtained in multiple subsequent frames, an early warning signal is triggered.

7. The implementation method of an electronic fence system based on UAV video as described in claim 1, characterized in that, The motion trajectory prediction process includes: Motion features are extracted from the target's historical trajectory data, and the target's motion patterns are identified as linear and nonlinear motions based on preset thresholds. The target's future position is predicted by selecting the corresponding prediction algorithm based on the identified motion pattern. A motion error model is established based on the spatial state information of the UAV to compensate for the predicted trajectory of the target.

8. The method for implementing an electronic fence system based on UAV video as described in claim 7, characterized in that, The process of extracting motion features from the target's historical trajectory data, identifying the target's motion patterns as linear or nonlinear based on preset thresholds, and selecting a corresponding prediction algorithm according to the identified motion patterns to predict the target's future position specifically includes: Extract the motion features of the target within a preset number of historical frames. The motion features include instantaneous velocity change rate, trajectory curvature change coefficient, and acceleration variance. The extracted motion features are compared with a preset dynamic judgment threshold. When all the motion features meet the linear motion condition, it is judged as linear motion; otherwise, it is judged as nonlinear motion. If the motion is determined to be linear, an adaptive Kalman filter algorithm is used to predict the target's future position. If the motion is determined to be nonlinear, then particle filtering or extended Kalman filtering algorithms are used to predict the target's future position.

9. The method for implementing an electronic fence system based on UAV video as described in claim 7, characterized in that, The establishment of a motion error model based on the spatial state information of the UAV to compensate for the predicted trajectory of the target specifically includes: Based on the spatial state information of the UAV, a comprehensive error model is constructed that includes GPS positioning error, IMU attitude drift error and camera jitter error; An extended Kalman filter (EKF) is used, and during the update phase of the filtering algorithm, the comprehensive error model is used to correct the predicted trajectory of the target. The compensated target coordinates are back-mapped to the image pixel coordinate system through the camera projection model to verify the compensation accuracy, and the filtering parameters are dynamically adjusted based on the verification results.

10. An electronic fence system based on drone video, characterized in that, include: The front-end interaction module is used by users to draw flight routes, define electronic fence geographic information areas, and display monitoring screens. The drone flight platform is equipped with camera equipment, GPS module and inertial measurement unit, and is used to perform flight missions, collect video streams and collect its own spatial status information; The communication module is used to establish a data transmission link between the front-end interaction module, the back-end service module and the UAV flight platform; The backend service module is used to process flight commands, video streams, and status information; The streaming media service module is used to receive and distribute video streams pushed by drones; The video analytics module is used to perform target detection and recognition, motion trajectory prediction, positional relationship determination, and early warning triggering operations. The cloud storage module is used to store flight route files, video data, and system configuration information.

Images