A monocular unmanned aerial vehicle motion target visual velocity measurement method based on optical flow field analysis

Abstract

The application discloses a monocular unmanned aerial vehicle (UAV) moving target visual velocity measurement method based on an optical flow field analysis, relates to a video measurement technical field, and comprises the following steps: vehicle detection and tracking are performed on video sequence frames, a history track queue is maintained for each vehicle target, a dynamic region mask is constructed, background feature points are extracted in an effective static region outside the mask, and adjacent frame pixel displacement vectors are calculated; space masking, statistical outlier rejection and direction consistency screening are performed on the displacement vectors, a first-order affine optical flow field is fitted, and a robust optical flow field function is obtained through time sequence smoothing; the average cumulative pixel displacement of four corner points is taken as the total pixel displacement of the vehicle, background displacement is accumulated along the corner point track according to the optical flow field, and the real pixel displacement and a relative speed ratio with a direction are obtained by subtraction; a history parameter queue is maintained, a reference background amplitude is calculated based on a high confidence interval history optical flow amplitude, a calibration coefficient is determined by combining a preset maximum cruising speed, an absolute speed of the vehicle is estimated by multiplying the UAV ground speed by the ratio, and the absolute speed of the vehicle is output.

Description

Technical Field

[0001] This invention relates to the field of video measurement technology, specifically to a visual velocity measurement method for moving targets using a monocular unmanned aerial vehicle (UAV) based on optical flow field analysis. Background Technology

[0002] Against the backdrop of the rapid development of intelligent transportation systems and smart highways globally, drones, with their advantages of flexible deployment, wide field of view, and rapid response, have gradually become an important aerial platform for highway inspection, traffic flow monitoring, and emergency event perception. Among these, non-contact speed estimation of moving vehicles on the road based on drone-borne monocular vision is a key technological means to achieve large-scale, low-cost traffic condition perception, which is of great significance for improving the level of highway operation and management and ensuring driving safety.

[0003] However, existing monocular vision-based vehicle speed measurement methods typically assume a static background or rely on inter-frame differencing and static background modeling to extract moving regions, making it difficult to effectively model the global background parallax caused by the drone's own flight. When the drone is in continuous motion, its translation, rotation, and attitude changes create a large-scale background displacement on the image plane. This background motion is superimposed and coupled with the vehicle's actual motion, making it difficult for existing methods to accurately distinguish between "background changes caused by the drone's own motion" and "the vehicle's actual motion." This is especially true when the vehicle is traveling at low speed or temporarily parked, as the vehicle's motion signal is easily overwhelmed by the background parallax motion, leading to misjudgments of speed or direction of motion.

[0004] Furthermore, existing methods largely rely on external prior information such as GPS, IMU, road markings, or ground control points to achieve the mapping from pixel scale to physical speed. In GPS-denied environments such as bridges and tunnels, or in open road sections without ground control points, the applicability of such methods is significantly limited, making it difficult to meet the practical needs of continuous and stable vehicle speed estimation in high-speed UAV inspection scenarios.

[0005] If the self-motion of the drone and the relative motion of the vehicle cannot be effectively decoupled, the reliability of the speed measurement results cannot be guaranteed. This will not only mislead the judgment of traffic situation, but may also lead to missed or false reports of low-speed or stationary vehicles, weaken the actual effectiveness of the intelligent inspection system, increase the cost of manual verification, and even trigger unnecessary control measures due to false traffic event warnings, affecting the efficiency of highway traffic and driving safety. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a visual velocity measurement method for moving targets in a monocular UAV based on optical flow field analysis, thus solving the problems mentioned in the background technology.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a visual velocity measurement method for moving targets in a monocular UAV based on optical flow field analysis, comprising the following steps:

[0008] S1, perform vehicle detection on the input video sequence frames to obtain a set of vehicle detection boxes; perform multi-target tracking on the set of vehicle detection boxes, maintain a historical trajectory queue for each vehicle target, the historical trajectory queue records the geometric state of multiple consecutive frames within a preset time window, the preset time window covers multiple consecutive frames from the current frame back a preset number of frames to the current frame, the geometric state includes the bounding box coordinates, center point coordinates and four corner point coordinates;

[0009] S2, construct a dynamic region mask using the vehicle detection box set, take the pixel region outside the dynamic region mask as the effective static region, extract background feature points in the effective static region, and use the Lucas-Kanade sparse optical flow algorithm to calculate the pixel displacement vector of the background feature points between adjacent frames.

[0010] S3. The pixel displacement vectors are subjected to multi-level filtering, including spatial masking, statistical outlier removal, and directional consistency constraints. The pixel displacement vectors retained after multi-level filtering are used as high-confidence displacement vectors. A first-order affine optical flow field model representing global background motion is fitted using the high-confidence displacement vectors to obtain an optical flow field function that describes the background motion vector at any position in the image. The historical parameter sequence of the most recent preset smoothed frames is maintained, and the coefficients of the optical flow field function are temporally smoothed to obtain a robust optical flow field function.

[0011] S4, for each vehicle target, within the preset time window, calculate the total pixel displacement of the vehicle target and the corresponding cumulative background displacement. The total pixel displacement of the vehicle target is obtained by averaging the cumulative pixel displacements of the four corner points of the vehicle target between the start and end frames of the preset time window. The corresponding cumulative background displacement is obtained by querying the robust optical flow field function frame by frame along the trajectory of the four corner points of the vehicle target and accumulating the inter-frame background displacement. Subtract the total pixel displacement of the vehicle target from the corresponding cumulative background displacement to obtain the true pixel displacement of the vehicle target relative to the background. Determine the relative velocity ratio in the direction of the actual pixel displacement and the background displacement in each frame.

[0012] S5. Maintain a historical parameter queue containing historical optical flow field parameters. The historical parameter queue is a sliding window queue. Calculate the reference background amplitude based on the historical optical flow amplitude in the high-confidence interval of the historical parameter queue. Determine the calibration coefficients based on the UAV's preset maximum cruising speed and the reference background amplitude. Estimate the current UAV's ground speed using the calibration coefficients and the magnitude of the optical flow field constant term in the current frame. Multiply the directional relative speed ratio with the UAV's ground speed to calculate and output the vehicle's absolute speed.

[0013] Further, in step S1, the YOLOv8n object detection model is used for vehicle detection. For each frame of image, the YOLOv8n object detection model outputs a set of vehicle detection boxes, which includes axis-aligned bounding boxes and corresponding confidence scores. The ByteTrack tracker is used to achieve target association and globally unique ID allocation through a dual-threshold association mechanism and a Kalman filter prediction-matching framework. The length of the historical trajectory queue is the preset number of frames plus one frame. The coordinates of the four corner points are the coordinates of the top left, top right, bottom left, and bottom right vertices of the axis-aligned bounding boxes.

[0014] Furthermore, in step S2, the dynamic region mask marks the pixel areas covered by all vehicle detection boxes in the vehicle detection box set as invalid regions; the effective static region is the pixel area in the image that does not belong to the invalid region; the Shi-Tomasi corner detector is used to extract background feature points in the effective static region, and the extracted background feature points meet the condition that the minimum distance between them and the neighboring selected points is not less than 10 pixels; 500 to 1500 background feature points are retained.

[0015] Furthermore, in step S3, the spatial masking filtering is: removing pixel displacement vectors whose endpoints fall within the vehicle detection frame;

[0016] The statistical outlier removal is as follows: calculate the mean and standard deviation of the remaining pixel displacement vectors after spatial masking filtering, and retain pixel displacement vectors whose displacement deviates from the mean by no more than a preset multiple of the standard deviation.

[0017] The directional consistency constraint is as follows: calculate the cosine of the angle between the remaining pixel displacement vector after statistical outlier removal and the global average optical flow direction, and retain pixel displacement vectors whose cosine of the angle is not less than a preset threshold.

[0018] Further, in step S3, the first-order affine optical flow field model includes translational components, horizontal scaling components, vertical scaling components, rotational components, and shearing components, totaling six degrees of freedom; the coefficients of the optical flow field function are obtained by solving the high-confidence displacement vector using the linear least squares method; the temporal smoothing includes: independently calculating the mean and standard deviation of each coefficient component within the most recent preset smoothing frame number, removing abnormal frames that deviate from the mean by more than 1.5 times the standard deviation, and taking the arithmetic mean of the coefficient values ​​of the remaining valid frames to obtain the smoothed coefficients.

[0019] Furthermore, in step S4, the cumulative pixel displacement and background cumulative displacement are calculated for the four corner points of the vehicle target, and the average of the calculated real pixel displacements of the four corner points is used to obtain the overall real pixel displacement. The direction of the relative velocity ratio with direction is determined by the dot product sign of the overall real pixel displacement vector and the background cumulative displacement vector, and the ratio of the amplitude of the overall real pixel displacement to the amplitude of the background cumulative displacement is used as the ratio value.

[0020] Further, in step S4, the total number of vehicle pixels is removed to obtain the average vehicle displacement per frame by the preset number of frames, and the corresponding cumulative background pixels are removed to obtain the average background displacement per frame by the preset number of frames; the difference between the average vehicle displacement per frame and the average background displacement per frame is taken as the average real pixel displacement per frame; the directional relative velocity ratio is the ratio of the amplitude of the average real pixel displacement per frame to the amplitude of the background displacement per frame in the current frame, and the amplitude of the background displacement per frame in the current frame is the magnitude of the optical flow field constant term in the current frame.

[0021] Furthermore, in step S5, the high confidence interval is determined by performing quantile statistics on the magnitude of the optical flow field constant term in the historical parameter queue, selecting a sample subset corresponding to the 85th to 98th quantiles, and calculating the average amplitude of the sample subset as the reference background amplitude.

[0022] Furthermore, in step S5, the calibration coefficient is the ratio of the preset maximum cruising speed of the UAV to the amplitude of the reference background; the current UAV ground speed is the smaller value among the product of the magnitude of the current frame optical flow field constant term and the calibration coefficient, and the preset maximum cruising speed of the UAV.

[0023] This invention provides a visual velocity measurement method for moving targets using a monocular UAV based on optical flow field analysis, which has the following advantages:

[0024] This invention relies solely on a monocular camera video stream. Without using GPS, IMU, road markings, or ground control points, it constructs a dynamic region mask to eliminate vehicle dynamic interference, extracts background feature points, and calculates pixel displacements. After multi-level filtering, a first-order affine optical flow field is fitted and temporally smoothed to obtain a robust optical flow field function. Based on this, the cumulative pixel displacements of the vehicle's four corners are subtracted from the cumulative background displacement to decouple the vehicle's true pixel displacement relative to the background and the directional relative velocity ratio. Furthermore, a sliding window of historical optical flow parameters is used to statistically analyze the reference background amplitude in high-confidence intervals. Combined with the UAV's preset maximum cruising speed, the UAV's ground velocity is self-calibrated, ultimately achieving continuous output of the vehicle's absolute speed. This scheme effectively solves the problem of separating the actual vehicle motion from the UAV's self-moving background, significantly improving the reliability of identifying low-speed and stationary vehicles. It can complete the mapping from pixel scale to physical velocity without external sensors, and has beneficial effects such as low cost, easy deployment, strong environmental adaptability, and real-time stable speed measurement. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating the steps of a visual velocity measurement method for moving targets using a monocular UAV based on optical flow field analysis, as described in this invention.

[0026] Figure 2 This is a flowchart illustrating the velocity calculation process of a visual velocity measurement method for moving targets using a monocular UAV based on optical flow field analysis, as described in this invention.

[0027] Figure 3 This image shows the speed measurement effect of a monocular UAV moving target visual speed measurement method based on optical flow field analysis according to the present invention. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] Example 1

[0030] Please see Figures 1 to 3 This invention provides a visual velocity measurement method for moving targets using a monocular UAV based on optical flow field analysis. This method relies solely on the video stream input from a monocular camera and achieves continuous estimation of the vehicle's absolute speed by decoupling the self-motion of the UAV platform from the relative motion of the vehicle, without using GPS, IMU, road markings, or ground control points. Figure 1The flowchart for velocity calculation is as follows, which includes dual-path parallel processing: the left path performs vehicle detection, multi-target tracking, and historical trajectory construction, while the right path performs dynamic region masking, background feature point extraction, background optical flow field fitting, and robust optical flow field construction. The two paths converge at the vehicle-to-background motion decoupling step, and then, after background velocity self-calibration and vehicle absolute velocity output, the final vehicle velocity is output. This method includes the following steps:

[0031] S1, perform vehicle detection on the input video sequence frames to obtain a set of vehicle detection boxes; perform multi-target tracking on the set of vehicle detection boxes, maintain a historical trajectory queue for each vehicle target, the historical trajectory queue records the geometric state of multiple consecutive frames within a preset time window, the preset time window covers multiple consecutive frames from the current frame back a preset number of frames to the current frame, the geometric state includes bounding box coordinates, center point coordinates and four corner point coordinates;

[0032] As a preferred embodiment, in step S1, the YOLOv8n object detection model is used for vehicle detection. For each frame of image, the YOLOv8n object detection model outputs a set of vehicle detection boxes, which includes axis-aligned bounding boxes and corresponding confidence scores. The ByteTrack tracker is used to achieve target association and globally unique ID allocation through a dual-threshold association mechanism and a Kalman filter prediction-matching framework. The length of the historical trajectory queue is the preset number of frames plus one frame. The coordinates of the four corner points are the coordinates of the top left, top right, bottom left, and bottom right vertices of the axis-aligned bounding boxes.

[0033] In this embodiment, step S1 includes vehicle detection, multi-target tracking, and historical trajectory queue maintenance, as detailed below:

[0034] The input video sequence frames are from an RTMP video stream from a monocular camera mounted on an UAV. In this embodiment, the video frame rate is 25 frames per second, and the image resolution is 1920×1080 pixels.

[0035] Vehicle detection is performed using the YOLOv8n target detection model. The YOLOv8n target detection model has approximately 3.2M parameters and has been validated on aerial datasets such as VisDrone for its adaptability to small high-altitude targets and complex backgrounds. It also supports ONNX / TensorRT deployment and is suitable for real-time operation on UAVs.

[0036] For each frame of the image, the YOLOv8n object detection model outputs a set of vehicle detection boxes. The set of vehicle detection boxes includes one or more vehicle detection boxes, each of which includes an axis-aligned bounding box and a corresponding confidence score. The axis-aligned bounding box refers to a rectangle parallel to the image coordinate axes. In this embodiment, the axis-aligned bounding box is represented by four parameters: the x-coordinate of the top-left corner, the y-coordinate of the top-left corner, the width, and the height.

[0037] The set of vehicle detection boxes is input into the ByteTrack tracker. The ByteTrack tracker achieves target association and globally unique ID assignment through a dual-threshold association mechanism and a Kalman filter prediction-matching framework. The dual-threshold association mechanism uses both high-confidence and low-confidence detection boxes for target association; the Kalman filter prediction-matching framework predicts the target motion state and matches it with the detection boxes in the current frame. Under challenges such as occlusion and blurring, the ByteTrack tracker can still maintain trajectory continuity and assign a globally unique ID to each vehicle.

[0038] The ByteTrack tracker outputs a globally unique ID and corresponding bounding box for each vehicle target. The bounding box is an axis-aligned bounding box.

[0039] Based on the vehicle detection results in S101 and the multi-target tracking results in S102, the system maintains a historical trajectory queue for each vehicle target. The length of the historical trajectory queue is the preset number of frames plus one frame. The preset time window covers multiple consecutive frames of images from the current frame backwards a preset number of frames to the current frame.

[0040] The preset frame rate can be set according to the UAV's flight speed, video frame rate, and target movement speed. The preset frame rate is between 5 and 20. The selection criteria are as follows: at a frame rate of 25 frames per second, 5 frames correspond to 0.2 seconds, and 20 frames correspond to 0.8 seconds. Too few preset frame rates will result in high noise in vehicle displacement estimation and insufficient background optical flow integration samples; too many preset frame rates will result in increased motion decoupling delay and slower response to motor vehicles. Therefore, in this embodiment, the preset frame rate is 10, and the length of the historical trajectory queue is 11 frames, recording the geometric state of each frame in the 11 consecutive frames from the current frame t back 10 frames to the current frame t. The preferred preset frame rate is 10, corresponding to a 0.4-second time window, balancing short-term motion smoothness and dynamic response capability.

[0041] The geometric state of the i-th target in frame r includes the bounding box coordinates, center point coordinates, and coordinates of the four corner points, as detailed below:

[0042] Bounding box coordinates:

[0043]

[0044] in, This represents the bounding box coordinates of the i-th target in the r-th frame; This represents the x-coordinate of the top-left corner of the bounding box of the i-th target in the r-th frame, in pixels; This represents the top-left ordinate of the bounding box of the i-th target in the r-th frame, in pixels. This represents the bounding box width of the i-th target in the r-th frame, in pixels; This represents the bounding box height of the i-th target in the r-th frame, in pixels.

[0045] Center point coordinates:

[0046]

[0047] in, This represents the coordinates of the center point of the i-th target in the r-th frame; , The meaning is the same as above.

[0048] Corner coordinates:

[0049]

[0050] in, This represents the coordinates of the k-th corner point of the i-th target in the r-th frame; k=1,2,3,4 correspond to the top-left, top-right, bottom-left, and bottom-right corners of the detection box, respectively. In this embodiment, the coordinates of the four corner points are as follows:

[0051] Top left corner: ;

[0052] Top right corner: ;

[0053] Bottom left corner: ;

[0054] Bottom right corner: .

[0055] This historical trajectory queue serves a dual purpose: firstly, the coordinate sequence of the four corner points is used to calculate the cumulative pixel displacement of the vehicle within the image plane, suppressing single-point tracking errors through multi-corner averaging; secondly, the coordinates of the four corner points in each frame are used as sampling points to query the background optical flow field and integrate background motion, achieving motion compensation. This step constructs a continuous, geometrically complete vehicle trajectory using the YOLOv8n object detection model and the ByteTrack tracker, laying the foundation for subsequently decoupling the pure vehicle motion from complex visual motion.

[0056] S2, construct a dynamic region mask using the vehicle detection box set, take the pixel region outside the dynamic region mask as the effective static region, extract background feature points in the effective static region, and use the Lucas-Kanade sparse optical flow algorithm to calculate the pixel displacement vector of the background feature points between adjacent frames.

[0057] As a preferred option, in step S2, the dynamic region mask marks the pixel areas covered by all vehicle detection boxes in the vehicle detection box set as invalid regions; the effective static region is the pixel area in the image that does not belong to the invalid region; the Shi-Tomasi corner detector is used to extract background feature points in the effective static region, and the extracted background feature points meet the condition that the minimum distance between them and the neighboring selected points is not less than 10 pixels; 500 to 1500 background feature points are retained.

[0058] As a preferred embodiment, in step S2, the pixel area covered by the vehicle detection frame is expanded to construct a dynamic region mask. The width and height of the vehicle detection frame are expanded outward by no more than 15% of the original width and height, respectively, and the area of ​​the expanded mask area does not exceed 132.25% of the original vehicle detection frame area; or, the pixel area covered by the vehicle detection frame is not further expanded.

[0059] As a preferred option, in step S2, the pixel displacement vector is calculated using the Lucas-Kanade sparse optical flow algorithm, with a window size of 15×15 pixels, a pyramid layer of 3 layers, and a maximum number of iterations of 30.

[0060] As a preferred embodiment, in step S2, if the number of background feature points extracted within the effective static area is less than a preset minimum number, the quality level factor of the Shi-Tomasi corner detector is reduced or the minimum spacing is decreased for re-extraction; if it is still less than the preset minimum number, the background optical flow field update of the current frame is skipped, and the robust optical flow field function of the previous frame is used.

[0061] Specifically, this step includes mask construction, effective static region definition, background feature point extraction, pixel displacement vector calculation, and degradation strategy, as follows:

[0062] To accurately model the background optical flow changes caused by the motion of the UAV platform, dynamic foreground interference such as vehicle motion needs to be eliminated. A dynamic region mask is constructed using the vehicle detection box set output in step S1. The dynamic region mask marks the pixel areas covered by all vehicle detection boxes in the vehicle detection box set as invalid regions. In this embodiment, the pixel areas covered by the vehicle detection boxes are not further expanded to avoid over-eliminating potential static texture regions. In another embodiment, the pixel areas covered by the vehicle detection boxes are expanded to construct the dynamic region mask, specifically by expanding the width and height of the vehicle detection boxes outward by no more than 15% of the original width and height, respectively, and the area of ​​the expanded mask area does not exceed 132.25% of the original vehicle detection box area. The area of ​​the vehicle detection box is defined as the product of its width and height. The purpose of the expansion is to cover the areas that vehicle motion may occupy, while avoiding over-eliminating static texture regions.

[0063] The effective static region is the pixel region in the image that does not belong to the invalid region, that is, the pixel region remaining after removing the coverage areas of all vehicle detection boxes. The effective static region is used to extract background feature points to avoid dynamic foregrounds such as vehicle movement from contaminating the background optical flow estimation.

[0064] Within the effective static region, the Shi-Tomasi corner detector is used to extract background feature points. The Shi-Tomasi corner detector selects stable corner points with significant gradient changes by analyzing the minimum eigenvalue of the local autocorrelation matrix of the image, and has greater geometric stability and repeatability compared to simple response functions such as FAST.

[0065] In this embodiment, the maximum number of corner points in the Shi-Tomasi corner detector is set to 1500, the quality level factor is set to 0.01, and the minimum Euclidean distance is set to 10 pixels. Typically, 500 to 1500 background feature points are retained. The following filtering conditions are applied to the extracted background feature points: First, the background feature points are located within the effective static area; second, the background feature points maintain a minimum spacing of no less than 10 pixels with their neighboring selected points to avoid spatial clustering and improve the spatial coverage uniformity of optical flow points.

[0066] Subsequently, the Lucas-Kanade sparse optical flow algorithm is used to calculate the pixel displacement vector of the selected background feature points from frame t-1 to frame t:

[0067]

[0068] in, This represents the pixel displacement vector of the j-th background feature point; This represents the pixel displacement of the j-th background feature point in the horizontal direction, in pixels. This represents the pixel displacement of the j-th background feature point in the vertical direction, in pixels.

[0069] In this embodiment, the Lucas-Kanade sparse optical flow algorithm uses a window size of 15×15 pixels, a pyramid of 3 layers, and a maximum of 30 iterations to obtain sub-pixel-level motion estimation. Based on the assumptions of constant local brightness and small displacement, the algorithm obtains the displacement vector by solving the linear optical flow equations within the local neighborhood. It is computationally efficient and highly accurate, making it suitable for static background areas with rich textures and gentle field-of-view changes in high-altitude aerial photography scenarios.

[0070] Degradation strategy: If the number of background feature points extracted within the effective static region is less than a preset minimum number, the quality level factor of the Shi-Tomasi corner detector is reduced or the minimum spacing is decreased for re-extraction; if it is still less than the preset minimum number, the background optical flow field update of the current frame is skipped, the robust optical flow field function of the previous frame is used, and the current frame is marked as low confidence. In this embodiment, the preset minimum number is 50 background feature points.

[0071] S3. The pixel displacement vectors are subjected to multi-level filtering, including spatial masking, statistical outlier removal, and directional consistency constraints. The pixel displacement vectors retained after multi-level filtering are used as high-confidence displacement vectors. The high-confidence displacement vectors are used to fit a first-order affine optical flow field model that represents the global background motion to obtain an optical flow field function that describes the background motion vector at any position in the image. The historical parameter sequence of the most recent preset smoothed frames is maintained, and the coefficients of the optical flow field function are temporally smoothed to obtain a robust optical flow field function.

[0072] As a preferred embodiment, in step S3, the spatial masking filtering is: removing pixel displacement vectors whose endpoints fall into the vehicle detection frame;

[0073] The statistical outlier removal is as follows: calculate the mean and standard deviation of the remaining pixel displacement vectors after spatial masking filtering, and retain pixel displacement vectors whose displacement deviates from the mean by no more than a preset multiple of the standard deviation.

[0074] The directional consistency constraint is as follows: calculate the cosine of the angle between the remaining pixel displacement vector after statistical outlier removal and the global average optical flow direction, and retain pixel displacement vectors whose cosine of the angle is not less than a preset threshold.

[0075] As a preferred embodiment, the preset multiple is 1.5 and the preset threshold is 0.95.

[0076] As a preferred embodiment, in step S3, the first-order affine optical flow field model includes translational components, horizontal scaling components, vertical scaling components, rotational components, and shearing components, totaling six degrees of freedom; the coefficients of the optical flow field function are obtained by solving the high-confidence displacement vector using the linear least squares method; the temporal smoothing includes: independently calculating the mean and standard deviation of each coefficient component within the most recent preset smoothing frame number, removing abnormal frames that deviate from the mean by more than 1.5 times the standard deviation, and taking the arithmetic mean of the coefficient values ​​of the remaining valid frames to obtain the smoothed coefficients.

[0077] The preset smoothing frame number is 10; if the number of valid frames after removing abnormal frames is less than the preset minimum valid frame number, the coefficients after smoothing the previous frame will be used or the update will be skipped.

[0078] In step S3, after obtaining the robust optical flow field function, the fitting residual between the high-confidence displacement vector and the robust optical flow field function is calculated. If the fitting residual is greater than a preset residual threshold, the confidence of the background optical flow field of the current frame is reduced or the robust optical flow field function of the previous frame is used.

[0079] This step includes multi-level screening, first-order affine optical flow field fitting, temporal parameter smoothing, robust optical flow field output, and residual evaluation, as detailed below:

[0080] In this embodiment, the original Lucas-Kanade optical flow vector is susceptible to noise, local texture loss, or residual dynamic interference, requiring a multi-level filtering mechanism to extract high-confidence background motion samples. The pixel displacement vector output in step S2 is subjected to the following three-level filtering:

[0081] Spatial masking filtering: Calculate the endpoint of the displacement vector for each pixel. If the endpoint falls within any vehicle detection box, the pixel displacement vector is removed to prevent residual dynamic targets from contaminating the background estimation.

[0082] Statistical outlier removal: Calculate the mean of the remaining pixel displacement vectors in the horizontal and vertical directions after spatial masking filtering. with standard deviation Retain the pixel displacement vector that simultaneously satisfies the following conditions:

[0083]

[0084]

[0085] in, This represents the mean of the remaining pixel displacement vectors in the horizontal direction; This represents the mean of the remaining pixel displacement vectors in the vertical direction; This represents the standard deviation of horizontal displacement. This represents the standard deviation of displacement in the vertical direction; Indicates the outlier determination factor in the horizontal direction; This represents the outlier determination factor in the vertical direction, usually taken as... ;

[0086] Orientation consistency constraint: Calculate the displacement vector of the remaining pixels after statistical outlier removal and the global average optical flow. The cosine of the included angle;

[0087]

[0088] in,

[0089] This represents the angle between the displacement vector of the j-th pixel and the global average optical flow; This represents the displacement vector of the j-th pixel; Represents the global average optical flow vector; This represents the magnitude of the displacement vector of the j-th pixel; This represents the magnitude of the global average optical flow vector. Only retain [the specified value]. ≥ , where This represents the directional consistency threshold. In this embodiment, it is taken as... The corresponding angular deviation is less than about 18°, ensuring that the retained pixel displacement vectors are highly consistent in the direction of motion.

[0090] The pixel displacement vectors retained after the above three-level screening constitute a set of high-confidence displacement vectors.

[0091] First-order affine background optical flow field fitting: High-confidence displacement vectors, after multi-level screening, constitute a sparse but reliable background motion sample set. To accurately characterize the spatial non-uniform motion caused by perspective projection under the large field of view of the UAV at high altitude, i.e., the background displacement is large at near distances and small at far distances in the image, a first-order affine optical flow model is used for global fitting:

[0092]

[0093]

[0094] in For image plane coordinates, This represents the horizontal background optical flow component at image location (x, y); This represents the vertical background optical flow component at image location (x, y). These represent the six coefficients of a first-order affine optical flow model. This model contains six degrees of freedom and can describe basic affine transformations such as translation, scaling, shearing, and rotation, sufficient to approximate the background parallax gradient from a high-altitude perspective.

[0095] Given M high-confidence displacement vectors and their corresponding feature point coordinates:

[0096]

[0097] in, This represents the image coordinates of the j-th background feature point; Let represent the pixel displacement vector of the j-th background feature point; M represents the number of high-confidence displacement vectors, and M is much greater than 6. M must be no less than 20 to ensure the stability of the least squares solution.

[0098] The horizontal direction coefficient vector is obtained by solving the overdetermined system of equations using the linear least squares method:

[0099]

[0100] And, the vertical direction coefficient vector:

[0101]

[0102] in, b represents the horizontal direction coefficient vector; b represents the vertical direction coefficient vector; superscript This represents the vector transpose. Substituting the coefficients obtained from the solution into the first-order affine optical flow model above, we obtain the optical flow field function.

[0103] Temporal parameter smoothing: To suppress inter-frame parameter jumps caused by transient occlusion, texture degradation, or tracking jitter, short-term temporal smoothing is performed on the optical flow coefficients. Specifically, a historical parameter sequence of the most recent N frames is maintained:

[0104]

[0105] in, This represents the horizontal direction coefficient vector of the nth frame; This represents the vertical direction coefficient vector of frame tn; N represents the preset smoothing frame number; and n represents the backtracking frame number. In this embodiment, N = 10 frames. It should be noted that N is the length of the optical flow field coefficient temporal smoothing window, which is set independently of the preset frame number in step S1, and the two can be the same or different.

[0106] Mean filtering is performed independently for each coefficient component. Taking any coefficient component c as an example, its original estimated sequence within the time window is: The smoothing process is as follows:

[0107] The first step is to calculate the mean and standard deviation of this coefficient over the past N frames:

[0108]

[0109]

[0110] Where μ represents the mean of the coefficient over the most recent N frames; σ represents the standard deviation of the coefficient over the most recent N frames; This represents the value of the coefficient in frame tn; N represents the preset number of smoothing frames.

[0111] The second step is to remove outlier frames that deviate from the mean by more than a preset multiple, thus forming a valid index set:

[0112]

[0113] Among them, among them, Represents the set of valid frame indices; This represents the value of the coefficient in the nth frame; This represents the mean; The value represents the standard deviation; 1.5 is the anomaly detection factor. In this embodiment, the anomaly detection factor is 1.5. If the number of valid frames is less than the preset minimum number of valid frames, the smoothing value of the previous frame is maintained or the current update is skipped. The preset minimum number of valid frames is 3.

[0114] The third step, where M = |Ω| is the number of valid frames, is used to calculate the arithmetic mean of the coefficient values ​​of the remaining valid frames, which is then used as the smoothed output of the current frame.

[0115]

[0116] in, The value of the smoothing coefficient for the current frame t is represented by M; the number of valid frames is represented by Ω; and the set of valid frame indices is represented by Ω. This represents the value of the coefficient in frame tn.

[0117] After applying the smoothing process described above to all six coefficients, the robust optical flow field function is obtained:

[0118]

[0119] in, Represents the robust optical flow field function; This represents the smoothed horizontal background optical flow component; This represents the smoothed vertical background optical flow component. The background optical flow vector information at any image coordinate (x, y) can be obtained.

[0120] The robust optical flow field function is output to step S4, which is used to query the background motion frame by frame and integrate along the trajectory of the four corner points of the vehicle target.

[0121] The first-order affine optical flow model obtained in this step is suitable for scenarios where the UAV is flying at a fixed altitude, with stable attitude, and mainly performing translational flight. When the UAV experiences drastic attitude changes or continuous altitude changes, the fitting accuracy of the first-order affine model decreases. In this case, the confidence level of the background optical flow field in the current frame can be reduced or the self-calibration update can be paused.

[0122] In another embodiment, after obtaining the robust optical flow field function, the fitting residual between the high-confidence displacement vector and the robust optical flow field function is calculated. If the fitting residual is greater than a preset residual threshold, the confidence of the background optical flow field in the current frame is reduced, or the robust optical flow field function of the previous frame is used. The preset residual threshold is set according to a preset proportion of the mean of the amplitude of the high-confidence displacement vector. This strategy effectively filters out high-frequency noise and outlier interference while retaining the ability to respond to the dynamic motion of the real platform, ensuring the smoothness and reliability of the background motion estimation in the time dimension, and avoiding abrupt jumps or drifts in velocity estimation.

[0123] S4, for each vehicle target, within the preset time window, calculate the total pixel displacement of the vehicle target and the corresponding cumulative background displacement; the total pixel displacement of the vehicle target is obtained by averaging the cumulative pixel displacements of the four corner points of the vehicle target between the start and end frames of the preset time window; the corresponding cumulative background displacement is obtained by querying the robust optical flow field function frame by frame along the trajectory of the four corner points of the vehicle target and accumulating the inter-frame background displacement; subtract the total pixel displacement of the vehicle target from the corresponding cumulative background displacement to obtain the true pixel displacement of the vehicle target relative to the background; determine the relative velocity ratio in the direction of the actual pixel displacement and the background displacement in each frame based on the actual pixel displacement.

[0124] As a preferred embodiment, in step S4, the cumulative pixel displacement and background cumulative displacement are calculated for the four corner points of the vehicle target, and the average of the calculated real pixel displacements at the four corner points is taken to obtain the overall real pixel displacement. The direction of the relative velocity ratio with direction is determined by the dot product sign of the overall real pixel displacement vector and the background cumulative displacement vector, and the ratio of the amplitude of the overall real pixel displacement to the amplitude of the background cumulative displacement is used as the ratio value.

[0125] As a preferred embodiment, in step S4, the total number of vehicle pixels is removed to obtain the average vehicle displacement per frame by the preset number of frames, and the corresponding cumulative background pixels are removed to obtain the average background displacement per frame by the preset number of frames; the difference between the average vehicle displacement per frame and the average background displacement per frame is taken as the average real pixel displacement per frame; the directional relative velocity ratio is the ratio of the amplitude of the average real pixel displacement per frame to the amplitude of the background displacement per frame in the current frame, and the amplitude of the background displacement per frame in the current frame is the magnitude of the optical flow field constant term in the current frame.

[0126] In step S4, when the cumulative background displacement amplitude or the current frame background displacement amplitude per frame is less than the preset lower limit of background displacement, the directional relative velocity ratio is set to zero or marked as low confidence, or the denominator is replaced with the larger value between the preset lower limit of background displacement and a very small positive number; the directional relative velocity ratio is set to an upper limit saturation value, and when the calculated ratio exceeds the upper limit saturation value, the upper limit saturation value is taken.

[0127] In step S4, the direction is determined by the sign of the dot product between the overall real pixel displacement vector and the background cumulative displacement vector: a dot product greater than zero indicates that the vehicle and the drone are moving in the same direction, a dot product less than zero indicates that the vehicle and the drone are moving in opposite directions, and a dot product equal to zero indicates that the vehicle is stationary relative to the ground or the direction is undetermined.

[0128] In this embodiment, this step includes calculating the total pixel displacement of the vehicle, calculating the cumulative background displacement, determining the actual pixel displacement and direction, calculating the relative velocity ratio with direction, and low-speed protection, as detailed below:

[0129] Vehicle total pixel displacement calculation: For a vehicle target, within the preset time window, i.e., the time interval from frame tL to frame t, where L represents the preset number of frames, the historical trajectory queue provides the coordinates of each of the four corner points of the vehicle target in each frame. For the k-th corner point, calculate the cumulative pixel displacement from frame tL to frame t:

[0130]

[0131] in, This represents the cumulative pixel displacement vector of the k-th corner point of the i-th vehicle target from frame tL to frame t; This represents the coordinates of the k-th corner point of the i-th vehicle target in frame t; This represents the coordinates of the k-th corner point of the i-th vehicle target in frame tL; L represents the preset frame number; k=1,2,3,4 correspond to the top left, top right, bottom left, and bottom right corners of the vehicle detection box, respectively.

[0132] The total pixel displacement of the vehicle is obtained by averaging the cumulative pixel displacements of the four corner points.

[0133]

[0134] in, Represents the total pixel displacement vector of the i-th vehicle target; This represents the cumulative pixel displacement vector of the k-th corner point of the i-th vehicle target; k=1,2,3,4 correspond to the four corner points respectively.

[0135] To further ensure consistency between the time reference and subsequent UAV ground velocity estimation, the total number of vehicle pixels was removed by a preset number of frames to obtain the average vehicle displacement per frame:

[0136]

[0137] in, This represents the average vehicle displacement vector per frame for the i-th vehicle target; L represents the total pixel displacement vector of the i-th vehicle target; L represents the preset number of frames.

[0138] Background cumulative displacement: The robust optical flow field function is queried frame by frame along the trajectory of the four corner points of the vehicle target, and the inter-frame background displacement is accumulated to obtain the corresponding cumulative background displacement. To ensure alignment with the time interval of the total pixel displacement of the vehicle, the background displacement is accumulated frame by frame from frame tL to frame t-1:

[0139]

[0140] in, This represents the cumulative background displacement vector corresponding to the k-th corner point of the i-th vehicle target; Indicates the corner position in the r-th frame. The inter-frame background optical flow vector is given by the robust optical flow field function, which is the background displacement from frame r to frame (r+1); r represents the frame number, which is accumulated from tL to t-1; L represents the preset number of frames. This represents the coordinates of the k-th corner point of the i-th vehicle target in the r-th frame.

[0141] It should be noted that the length N of the temporal smoothing window for the robust optical flow field function coefficients is set independently of the preset number of frames L. N represents the length of the historical parameter sequence used to smooth the optical flow field coefficients, preferably N=10; L represents the length of the trajectory window used to calculate the cumulative displacement of the vehicle and the background, preferably L=10. The two can be the same or different.

[0142] The average cumulative background displacement is obtained by averaging the cumulative background displacements at the four corner points.

[0143]

[0144] in, This represents the average cumulative background displacement vector of the i-th vehicle target; This represents the background cumulative displacement vector of the i-th vehicle target at the k-th corner point; k=1,2,3,4 correspond to the four corner points respectively.

[0145] In this embodiment, to further ensure time reference consistency, the average background cumulative bits are removed by a preset number of frames to obtain the average background displacement per frame:

[0146]

[0147] in, This represents the average background displacement vector per frame for the i-th vehicle target; L represents the average cumulative background displacement vector of the i-th vehicle target; L represents the preset number of frames.

[0148] Actual pixel displacement and orientation determination:

[0149] The actual pixel displacement of the vehicle target relative to the background is:

[0150]

[0151] in, This represents the overall true pixel displacement vector of the i-th vehicle target; Represents the total pixel displacement vector of the i-th vehicle target; Let represent the average cumulative background displacement vector of the i-th vehicle target.

[0152] In another embodiment, the average true pixel displacement per frame is calculated using the average displacement per frame:

[0153]

[0154] in, This represents the average real pixel displacement vector per frame for the i-th vehicle target; This represents the average vehicle displacement vector per frame for the i-th vehicle target; Let represent the average background displacement vector per frame for the i-th vehicle target.

[0155] Calculate the overall true pixel displacement magnitude and the average cumulative background displacement magnitude:

[0156]

[0157]

[0158] in, This represents the overall true pixel displacement magnitude of the i-th vehicle target; This represents the average cumulative background displacement amplitude of the i-th vehicle target; This represents the Euclidean norm.

[0159] Determine direction: Calculate the dot product of the overall true pixel displacement vector and the average background cumulative displacement vector. If the dot product is greater than zero, the vehicle and drone are moving in the same direction, and the direction sign is positive; if the dot product is less than zero, the vehicle and drone are moving in opposite directions, and the direction sign is negative; if the dot product is equal to zero, the vehicle is stationary relative to the ground or its direction is undetermined. The direction sign is represented as:

[0160]

[0161] in, The sign of the i-th vehicle target is indicated by sign(⋅); sign(⋅) represents the sign function. This represents the dot product of the overall true pixel displacement vector and the average cumulative background displacement vector.

[0162] In embodiments employing average per-frame displacement, the direction sign is determined by the dot product of the average per-frame true pixel displacement vector and the average per-frame background displacement vector:

[0163]

[0164] in, Indicates direction symbol; This represents the average true pixel displacement vector per frame. This represents the average background displacement vector per frame.

[0165] Calculation of relative speed ratio in both directions and low-speed protection:

[0166] The directional relative velocity ratio is used to reflect the ratio of the vehicle's motion intensity to the drone's motion intensity, and carries information on whether the motion is in the same or opposite direction.

[0167] In the embodiment employing cumulative displacement, the directional relative velocity ratio is:

[0168]

[0169]

[0170] in, This represents the relative velocity ratio with direction for the i-th vehicle target; Indicates direction symbol; This represents the overall true pixel displacement magnitude; This represents the average cumulative background displacement amplitude; This represents a very small positive number that is not divisible by zero. In this embodiment, we take... .

[0171] In the preferred embodiment employing average displacement per frame, the directional relative velocity ratio is:

[0172]

[0173] in, Indicates the ratio of relative velocities with direction; Indicates direction symbol; This represents the average actual pixel displacement magnitude per frame; This represents the displacement amplitude of the background in each frame of the current frame, which is the magnitude of the optical flow field constant term in the current frame.

[0174] The magnitude of the optical flow field constant term in the current frame is:

[0175]

[0176] in, This indicates the displacement magnitude of the background in each frame of the current frame; This represents the horizontal constant term of the optical flow field in the current frame; This represents the vertical constant term of the optical flow field in the current frame.

[0177] Low speed protection: When Less than the preset background displacement lower limit When the drone is stationary or flying at low speed, the background displacement per frame is too small, and directly calculating the velocity ratio will lead to numerical instability. In this case, at least one of the following protection methods should be used:

[0178] Set the relative velocity ratio with direction to zero;

[0179] The relative velocity ratio with direction is marked as low confidence;

[0180] Replace the denominator with and the smallest positive number The larger value in, that is:

[0181]

[0182] in, ε represents the preset lower limit of background displacement; ε represents a very small positive number; max(⋅) represents taking the maximum value.

[0183] Set the upper limit saturation value of the ratio. When the calculated ratio exceeds At that time, take :

[0184]

[0185] in, This indicates the upper limit saturation value of the ratio; clip(⋅) restricts the value to a given range. =10.

[0186] When using the cumulative displacement ratio, the lower limit of the background displacement and the upper limit of the ratio saturation value can also be set. When the relative velocity ratio is less than the preset lower limit of the background displacement, the ratio with direction is set to zero or marked as low confidence.

[0187] The ratio of relative velocities in the direction of the belt The output is sent to step S5, where it is used to calculate the vehicle's absolute speed in conjunction with the UAV's ground-to-ground speed. When... When positive, it indicates that the vehicle and the drone are moving in the same direction; when... When negative, it indicates that the vehicle and the drone are moving in opposite directions; when... When the value is zero, it indicates that the vehicle is stationary relative to the ground or the background displacement is too small to be determined.

[0188] S5, maintain a historical parameter queue containing historical optical flow field parameters, the historical parameter queue being a sliding window queue; calculate a reference background amplitude based on the historical optical flow amplitude in the high-confidence interval of the historical parameter queue; determine calibration coefficients based on the preset maximum cruising speed of the UAV and the reference background amplitude; estimate the current UAV's ground velocity using the calibration coefficients and the magnitude of the optical flow field constant term in the current frame; multiply the directional relative velocity ratio obtained in step S4 with the UAV's ground velocity to calculate and output the vehicle's absolute speed.

[0189] As a preferred embodiment, in step S5, the high confidence interval is determined by performing quantile statistics on the magnitude of the optical flow field constant term in the historical parameter queue, selecting a sample subset corresponding to the 85% to 98% quantiles, and calculating the average amplitude of the sample subset as the reference background amplitude.

[0190] As a preferred embodiment, in step S5, a minimum number of samples for the subset and a lower limit for the reference background amplitude are set; when the number of samples in the sample subset is less than the minimum number of samples for the subset, or the reference background amplitude is less than the lower limit for the reference background amplitude, the estimated value of the UAV's ground velocity in the previous frame or the default value is used, and the confidence level of the vehicle's absolute velocity in the current frame is reduced.

[0191] As a preferred embodiment, in step S5, the calibration coefficient is the ratio of the preset maximum cruising speed of the UAV to the amplitude of the reference background; the current UAV ground speed is taken as the smaller value among the product of the magnitude of the current frame optical flow field constant term and the calibration coefficient, and the preset maximum cruising speed of the UAV.

[0192] As a preferred embodiment, in step S5, when the magnitude of the current frame optical flow field constant term is less than the preset background displacement lower limit, the directional relative velocity ratio is set to zero or marked as low confidence, or the denominator in the UAV ground velocity estimation is replaced with the larger value between the preset background displacement lower limit and the minimum positive number; and an upper limit saturation value is set for the directional relative velocity ratio.

[0193] As a preferred option, in step S5, before outputting the absolute vehicle speed, the absolute vehicle speeds calculated in multiple consecutive frames are subjected to mean filtering. Abnormal speed values ​​that deviate from the mean by more than a preset multiple are removed, and the average is taken as the final absolute vehicle speed output for the current frame.

[0194] As a preferred embodiment, in step S5, the length of the sliding window queue is configured according to the calibration response requirements, and the value range is 300 to 900 frames; the maximum cruise speed preset by the UAV is preset by the mission planning, and the value range is 10 to 20 meters per second.

[0195] In this embodiment, this step includes maintaining the historical parameter queue, filtering high-confidence intervals and calculating the reference background amplitude, estimating calibration coefficients and UAV ground velocity, calculating vehicle absolute speed, mean filtering, and output, as detailed below:

[0196] Historical parameter queue maintenance: Due to the lack of external sensors, the drone's ground velocity cannot be directly obtained. This invention employs a self-calibration mechanism based on historical optical flow statistics. This self-calibration mechanism is suitable for scenarios where the drone flies at a fixed altitude, the ground is flat, the attitude is stable, and the flight is mainly translational; when the drone experiences drastic attitude changes or continuous altitude changes, the estimation confidence level can be reduced or the self-calibration update can be paused.

[0197] Maintain a historical parameter queue with a sliding window length of K:

[0198]

[0199] in, Represents the historical parameter queue; This represents the horizontal constant term of the optical flow field in the tr-th frame; denoted by tr, where K represents the vertical constant term of the optical flow field in frame tr; K represents the sliding window length; and r represents the backtracking frame number. In this embodiment, K is set to 500 frames, which is approximately 20 seconds at 25 frames per second. K can be configured according to the calibration response requirements, with a value ranging from 300 to 900 frames. When the drone speed changes rapidly, a smaller K value can be used to improve the response speed; when the drone speed is stable, a larger K value can be used to improve statistical stability.

[0200] It should be noted that K is the sliding window length of the historical parameter queue, used to statistically analyze the reference background amplitude; the preset smoothing frame number N in step S3 is the temporal smoothing window length of the optical flow field coefficients; the two are set independently. Preferably, N=10, K=500.

[0201] and The optical flow constant term represents the background motion components in the horizontal and vertical directions of the image center region. In typical forward-flying scenarios of UAVs, the magnitude of the optical flow constant term usually dominates the background motion intensity. To avoid instability in a single-direction component due to UAV sideslip or lateral maneuvers, this step does not use a fixed directional component, but instead uses the magnitude of the optical flow constant term as the statistical object of historical optical flow amplitude.

[0202] High confidence interval screening and reference background amplitude calculation:

[0203] For each historical optical flow field parameter in the historical parameter queue, calculate the magnitude of its optical flow field constant term:

[0204]

[0205] in, This represents the magnitude of the optical flow field constant term in the tr-th frame; This represents the horizontal constant term of the optical flow field in the tr-th frame; This represents the vertical constant term of the optical flow field in the tr-th frame.

[0206] Historical optical flow amplitude Perform quantile statistics and select the sample subset corresponding to the high confidence interval. In this embodiment, a sample subset corresponding to the 85th to 98th percentiles was selected, while the 98th to 100th percentiles were not selected to avoid sudden abnormal optical flow fields. These high-confidence samples correspond to the UAV being in a high-speed and stable flight state, representing the maximum flight speed of the UAV at the waypoint settings.

[0207] Set the minimum number of samples in the subset, for example, to require Not less than 30; if less, extend the sliding window or use the default reference background amplitude. Calculate the reference background amplitude corresponding to the maximum velocity:

[0208]

[0209] in, Indicates the reference background amplitude; This represents a subset of historical parameters with high confidence. Indicates the number of samples in the subset; , This represents the optical flow field constant term in the subset. It sets the lower limit of the reference background amplitude when... When the value is below the lower limit, the estimated ground velocity of the UAV in the previous frame or the default value is used, and the confidence level is reduced. In this embodiment, the lower limit of the reference background amplitude is set according to a preset ratio of the historical reference background amplitude.

[0210] Calibration coefficients and UAV ground speed estimation:

[0211]

[0212] in, Indicates the calibration coefficient; This indicates the drone's preset maximum cruising speed; This represents the reference background amplitude. The calibration coefficient represents the drone's ground velocity corresponding to a unit pixel of background amplitude.

[0213] Combined with the magnitude of the optical flow field constant term in the current frame:

[0214]

[0215] in, This represents the magnitude of the optical flow field constant term in the current frame, i.e., the displacement amplitude of the background in each frame of the current frame; This represents the horizontal constant term of the optical flow field in the current frame; This represents the vertical constant term of the optical flow field in the current frame.

[0216] The current ground speed of the drone can be estimated:

[0217]

[0218] in, This represents the estimated ground velocity of the UAV in the current frame; k represents the calibration coefficient. This indicates the magnitude of the optical flow field constant term in the current frame; This represents the preset maximum cruising speed of the drone. In this embodiment, it is taken as... =13m / s is the preset maximum cruising speed. Additionally, The value can be preset according to the UAV model and mission configuration, with a typical range of 10 to 20 m / s. This parameter comes from the UAV mission planning system and is not measured in real time by a visual sensor. Therefore, this invention still relies solely on monocular video streams for speed measurement and does not use external sensors such as GPS or IMU.

[0219] when Greater than At that time, take Used to limit overspeed estimation and prevent Too small a value leads to unstable values.

[0220] Vehicle absolute speed calculation: Multiply the directional relative speed ratio obtained in step S4 by the UAV's ground speed to calculate the vehicle's absolute speed.

[0221]

[0222] in, 3 represents the absolute speed of the i-th vehicle target in frame t, in km / h; 3.6 represents the unit conversion factor from m / s to km / h. This represents the relative speed ratio of the i-th vehicle in the target direction; This indicates the drone's ground speed in the current frame, in m / s.

[0223] when When positive, it indicates that the vehicle and the drone are moving in the same direction; when... When negative, it indicates that the vehicle and the drone are moving in opposite directions; when... A value of zero indicates that the vehicle is stationary relative to the ground or the background displacement is too small to be determined. During output, the velocity amplitude and direction of motion can be output simultaneously, or the sign of the velocity can be retained.

[0224] Mean filtering and output: Before outputting the absolute vehicle speed, mean filtering is performed on the absolute vehicle speeds calculated over multiple consecutive frames. In this embodiment, the absolute vehicle speed sequence of the most recent 5 frames is maintained. Abnormal speed values ​​deviating from the mean by more than a preset multiple are removed, and the arithmetic mean is taken as the final output speed for the current frame. The preset multiple is 1.5 times the standard deviation. If the number of valid frames is insufficient, the current calculated value is output directly.

[0225] The final output is the vehicle speed in km / h, and can include the vehicle target ID, detection box, direction of motion, and confidence level information.

[0226] This solution completely eliminates the reliance on external sensors and prior scene knowledge. Through a triple mechanism of background modeling, motion decoupling, and statistical self-calibration, it achieves pure vision-based absolute velocity estimation in complex airborne dynamic scenarios, providing scalable basic perception capabilities for UAV intelligent traffic inspection.

[0227] Some of the data in the above formulas are numerical calculations with dimensions removed, and the contents not described in detail in this specification are all prior art known to those skilled in the art.

[0228] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A visual velocity measurement method for moving targets using a monocular UAV based on optical flow field analysis, characterized in that: Includes the following steps: S1, perform vehicle detection on the input video sequence frames to obtain a set of vehicle detection boxes; Multi-target tracking is performed on the vehicle detection box set. A historical trajectory queue is maintained for each vehicle target. The historical trajectory queue records the geometric state of multiple consecutive frames within a preset time window. The preset time window covers multiple consecutive frames of images from the current frame back a preset number of frames to the current frame. The geometric state includes the bounding box coordinates, center point coordinates, and four corner point coordinates. S2, construct a dynamic region mask using the vehicle detection box set, take the pixel region outside the dynamic region mask as the effective static region, extract background feature points in the effective static region, and use the Lucas-Kanade sparse optical flow algorithm to calculate the pixel displacement vector of the background feature points between adjacent frames. S3. The pixel displacement vectors are subjected to multi-level filtering, including spatial masking, statistical outlier removal, and directional consistency constraints. The pixel displacement vectors retained after multi-level filtering are used as high-confidence displacement vectors. A first-order affine optical flow field model representing global background motion is fitted using the high-confidence displacement vectors to obtain an optical flow field function that describes the background motion vector at any position in the image. The historical parameter sequence of the most recent preset smoothed frames is maintained, and the coefficients of the optical flow field function are temporally smoothed to obtain a robust optical flow field function. S4. For each vehicle target, within the preset time window, calculate the total pixel displacement of the vehicle target and the corresponding cumulative background displacement. The total pixel displacement of the vehicle is obtained by averaging the cumulative pixel displacements of the four corner points of the vehicle target between the start and end frames of the preset time window; The corresponding cumulative background displacement is obtained by querying the robust optical flow field function frame by frame along the trajectory of the four corner points of the vehicle target and accumulating the inter-frame background displacement; the total pixel displacement of the vehicle is subtracted from the corresponding cumulative background displacement to obtain the true pixel displacement of the vehicle target relative to the background; the relative velocity ratio in the direction is determined based on the true pixel displacement and the background displacement in each frame. S5. Maintain a historical parameter queue containing historical optical flow field parameters. The historical parameter queue is a sliding window queue. Calculate the reference background amplitude based on the historical optical flow amplitude in the high-confidence interval of the historical parameter queue. Determine the calibration coefficients based on the UAV's preset maximum cruising speed and the reference background amplitude. Estimate the current UAV's ground speed using the calibration coefficients and the magnitude of the optical flow field constant term in the current frame. Multiply the directional relative speed ratio with the UAV's ground speed to calculate and output the vehicle's absolute speed.

2. The method for visual velocity measurement of moving targets by a monocular UAV based on optical flow field analysis according to claim 1, characterized in that: In step S1, the YOLOv8n object detection model is used for vehicle detection. For each frame of image, the YOLOv8n object detection model outputs a set of vehicle detection boxes, which includes axis-aligned bounding boxes and corresponding confidence scores. The ByteTrack tracker is used to achieve target association and globally unique ID allocation through a dual-threshold association mechanism and a Kalman filter prediction-matching framework. The length of the historical trajectory queue is the preset number of frames plus one frame. The coordinates of the four corner points are the coordinates of the top left, top right, bottom left, and bottom right vertices of the axis-aligned bounding boxes.

3. The method for visual velocity measurement of moving targets by a monocular UAV based on optical flow field analysis according to claim 1, characterized in that: In step S2, the dynamic region mask marks the pixel areas covered by all vehicle detection boxes in the vehicle detection box set as invalid regions; the valid static region is the pixel area in the image that does not belong to the invalid region. The Shi-Tomasi corner detector is used to extract background feature points in the effective static area. The extracted background feature points meet the condition that the minimum distance between them and the neighboring selected points is not less than 10 pixels; 500 to 1500 background feature points are retained.

4. The method for visual velocity measurement of moving targets by a monocular UAV based on optical flow field analysis according to claim 1, characterized in that: In step S3, the spatial masking filtering is: removing pixel displacement vectors whose endpoints fall within the vehicle detection frame; The statistical outlier removal is as follows: calculate the mean and standard deviation of the remaining pixel displacement vectors after spatial masking filtering, and retain pixel displacement vectors whose displacement deviates from the mean by no more than a preset multiple of the standard deviation. The directional consistency constraint is as follows: calculate the cosine of the angle between the remaining pixel displacement vector after statistical outlier removal and the global average optical flow direction, and retain pixel displacement vectors whose cosine of the angle is not less than a preset threshold.

5. The method for visual velocity measurement of moving targets by a monocular UAV based on optical flow field analysis according to claim 1, characterized in that: In step S3, the first-order affine optical flow field model includes translational components, horizontal scaling components, vertical scaling components, rotational components, and shearing components, totaling six degrees of freedom. The coefficients of the optical flow field function are obtained by solving the high-confidence displacement vector using the linear least squares method. The temporal smoothing includes: independently calculating the mean and standard deviation of each coefficient component within the most recent preset smoothing frame number, removing abnormal frames that deviate from the mean by more than 1.5 times the standard deviation, and taking the arithmetic mean of the coefficient values ​​of the remaining valid frames to obtain the smoothed coefficients.

6. The method for visual velocity measurement of moving targets by a monocular UAV based on optical flow field analysis according to claim 1, characterized in that: In step S4, the cumulative pixel displacement and background cumulative displacement are calculated for the four corner points of the vehicle target, and the average of the calculated real pixel displacements of the four corner points is taken to obtain the overall real pixel displacement. The direction of the relative velocity ratio with direction is determined by the dot product sign of the overall real pixel displacement vector and the background cumulative displacement vector, and the ratio of the amplitude of the overall real pixel displacement to the amplitude of the background cumulative displacement is used as the ratio value.

7. The method for visual velocity measurement of moving targets by a monocular UAV based on optical flow field analysis according to claim 1, characterized in that: In step S4, the total number of vehicle pixels is removed to obtain the average vehicle displacement per frame by the preset number of frames, and the corresponding cumulative background pixels are removed to obtain the average background displacement per frame by the preset number of frames. The difference between the average vehicle displacement per frame and the average background displacement per frame is taken as the average real pixel displacement per frame; the directional relative velocity ratio is the ratio of the average real pixel displacement amplitude per frame to the current frame background displacement amplitude per frame, and the current frame background displacement amplitude is the magnitude of the current frame optical flow field constant term.

8. The method for visual velocity measurement of moving targets by a monocular UAV based on optical flow field analysis according to claim 1, characterized in that: In step S5, the high confidence interval is determined by performing quantile statistics on the magnitude of the optical flow field constant term in the historical parameter queue, selecting a sample subset corresponding to the 85th to 98th quantiles, and calculating the average amplitude of the sample subset as the reference background amplitude.

9. The method for visual velocity measurement of moving targets by a monocular UAV based on optical flow field analysis according to claim 1, characterized in that: In step S5, the calibration coefficient is the ratio of the preset maximum cruising speed of the UAV to the amplitude of the reference background; the current UAV ground speed is the smaller value among the product of the magnitude of the current frame optical flow field constant term and the calibration coefficient, and the preset maximum cruising speed of the UAV.

Images