Multi-parameter collaborative optimization system of unmanned aerial vehicle optical flow algorithm

Abstract

The invention discloses a multi-parameter collaborative optimization system for an unmanned aerial vehicle optical flow algorithm, and relates to the field of unmanned aerial vehicles. The multi-parameter collaborative optimization system for the unmanned aerial vehicle optical flow algorithm comprises an image preprocessing module, a feature point management module and an optical flow verification module, and the image preprocessing module, the feature point management module and the optical flow verification module work cooperatively to optimize optical flow displacement vector calculation. According to the multi-parameter collaborative optimization system of the unmanned aerial vehicle optical flow algorithm, the contrast of a low-texture region is enhanced through the image preprocessing module, the feature point identification degree is improved, a more reliable basis is provided for optical flow calculation, and a hierarchical threshold strategy adapts to the requirements of the unmanned aerial vehicle for the number of feature points in different flight stages; periodic detection avoids calculation failure caused by loss of feature points, and a dynamic weight distribution mechanism optimizes distribution uniformity of the feature points, so that optical flow calculation is more accurate, and adaptability of an optical flow algorithm to different flight states is improved.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) technology, specifically to a multi-parameter collaborative optimization system for UAV optical flow algorithms. Background Technology

[0002] With the widespread application of drone technology, the demand for stability and precise control in various complex environments is increasing.

[0003] In low-altitude flight, especially in low-texture scenarios such as large bodies of water, smooth surfaces, and snowy areas, UAV visual positioning faces numerous challenges. Traditional optical flow algorithms struggle with feature point detection and tracking in these scenarios due to the scarcity of feature points in low-texture regions. This leads to inaccurate optical flow displacement vector calculations, severely impacting the hovering stability and flight safety of UAVs.

[0004] Meanwhile, during drone flight, feature points may be lost due to occlusion, rapid movement, or other reasons, further reducing the reliability of optical flow algorithms. Therefore, developing a system that can effectively improve the performance of optical flow algorithms in low-texture scenes is of significant practical importance. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a multi-parameter collaborative optimization system for UAV optical flow algorithms, solving the problems of low feature point recognition in low-texture regions, lack of an effective dynamic management mechanism for feature points, and the potential generation of abnormal displacement vectors during optical flow calculation. To achieve the above objectives, the present invention provides the following technical solution: a multi-parameter collaborative optimization system for UAV optical flow algorithms, comprising an image preprocessing module, a feature point management module, and an optical flow verification module, wherein the image preprocessing module, the feature point management module, and the optical flow verification module work together to optimize the calculation of optical flow displacement vector; The image preprocessing module processes the input image frames to improve the contrast of low-texture areas; the feature point management module dynamically manages feature point detection and tracking; the optical flow verification module filters the displacement vector to remove outliers, and finally outputs the effective displacement vector to the UAV flight control system to improve the hovering stability of the UAV in low-texture scenes.

[0006] Preferably, the image preprocessing module performs histogram equalization on the input image frame to enhance the feature point recognition of low-texture areas.

[0007] Preferably, the feature point management module adopts a hierarchical threshold strategy: the minimum feature point threshold during takeoff is set to 10, and the minimum feature point threshold during hovering is set to 30.

[0008] Preferably, the feature point management module performs periodic detection: feature point detection is forcibly triggered once every 30 frames to avoid calculation failure caused by feature point loss.

[0009] Preferably, the feature point management module adopts a dynamic weight allocation mechanism: the current frame weight Wc=0.7, the historical frame weight Wh=0.3, and the distribution uniformity is optimized by fusing the feature point information of the current frame and the historical frame.

[0010] Preferably, the optical flow verification module performs forward optical flow calculation: obtaining the displacement vector Vf from the feature point of the current frame t to the tracking point of the next frame t+1.

[0011] Preferably, the optical flow verification module performs reverse optical flow tracing: tracing back from the tracking point in frame t+1 to frame t to obtain the backtracking point.

[0012] Preferably, the optical flow verification module generates an error mask: if the coordinate difference between the original feature point and the backtracking point of frame t is... If epsilon=0.5px, then the feature point is marked as invalid.

[0013] Preferably, the 30-frame cycle detection of the feature point management module is synchronized with the clock of the UAV flight control system to ensure the accuracy of the detection cycle.

[0014] Preferably, the error mask of the optical flow verification module is output in bitmap format for the flight control system to eliminate abnormal displacement vectors in real time.

[0015] Its beneficial effects are as follows: 1. This multi-parameter collaborative optimization system for the UAV optical flow algorithm enhances the contrast of low-texture areas and improves feature point recognition through an image preprocessing module, providing a more reliable foundation for optical flow calculation. This improves the hovering stability of the UAV in low-texture scenes and reduces shaking and drift caused by inaccurate positioning. A hierarchical threshold strategy adapts to the feature point quantity requirements of different flight stages of the UAV, periodic detection avoids computational failure due to feature point loss, and a dynamic weight allocation mechanism optimizes the uniformity of feature point distribution, making optical flow calculation more accurate and improving the adaptability of the optical flow algorithm to different flight states. The optical flow verification module effectively eliminates abnormal displacement vectors through forward optical flow calculation, reverse optical flow tracing, and error mask generation, providing more accurate optical flow information to the UAV flight control system. This helps the flight control system to more accurately control the UAV's flight attitude and position, improving flight safety and reliability.

[0016] 2. The multi-parameter collaborative optimization system of the UAV optical flow algorithm synchronizes the periodic detection of the feature point management module with the UAV flight control system clock, ensuring the accuracy of the feature point detection cycle and further improving the stability and reliability of the optical flow algorithm. The error mask of the optical flow verification module is output in bitmap format, which facilitates the flight control system to quickly identify and eliminate abnormal displacement vectors, improving the system's real-time processing capability. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a system architecture diagram of the present invention; Figure 2 This is a flowchart of the optical flow algorithm of the present invention; Figure 3 This is a flowchart of the drone_ive_st_lk_proc algorithm of this invention; Figure 4 This is a distribution diagram of feature points on a marble surface processed by histogram equalization according to the present invention; Figure 5 This is a visual diagram of the optical flow processed by histogram equalization on a marble floor according to the present invention; Figure 6 This is a distribution diagram of feature points on a wooden floor processed by histogram equalization according to the present invention; Figure 7 This is a visual diagram of the optical flow processed by histogram equalization on a wooden floor according to the present invention; Figure 8 This is a distribution diagram of the overall feature points of the present invention; Figure 9 This is a comparison chart showing the average motion amplitude during optical flow algorithm testing using a grid carpet dataset, as presented in this invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are described clearly and completely. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0021] This invention discloses a multi-parameter collaborative optimization system for UAV optical flow algorithms, according to the appendix. Figure 1-3 As shown, it includes an image preprocessing module, a feature point management module, and an optical flow verification module. The image preprocessing module, feature point management module, and optical flow verification module work together to optimize the optical flow displacement vector calculation. The image preprocessing module processes the input image frames to improve the contrast of low-texture areas; the feature point management module dynamically manages feature point detection and tracking; the optical flow verification module filters the displacement vector to remove outliers, and finally outputs the effective displacement vector to the UAV flight control system to improve the hovering stability of the UAV in low-texture scenes.

[0022] The image preprocessing module enhances the contrast of low-texture areas, improving feature point recognition and providing a more reliable foundation for optical flow calculation. This improves the hovering stability of the UAV in low-texture scenes and reduces shaking and drift caused by inaccurate positioning. A tiered threshold strategy adapts to the feature point quantity requirements of different flight stages of the UAV. Periodic detection prevents feature point loss that could lead to computational failure, and a dynamic weight allocation mechanism optimizes the uniformity of feature point distribution, making optical flow calculation more accurate and improving the adaptability of the optical flow algorithm to different flight states. The optical flow verification module effectively eliminates abnormal displacement vectors through forward optical flow calculation, reverse optical flow tracing, and error mask generation, providing more accurate optical flow information to the UAV flight control system. This helps the flight control system to more accurately control the UAV's flight attitude and position, improving flight safety and reliability.

[0023] The feature point management module's periodic detection is synchronized with the UAV flight control system's clock, ensuring the accuracy of the feature point detection cycle and further improving the stability and reliability of the optical flow algorithm. The error mask of the optical flow verification module is output in bitmap format, facilitating the flight control system to quickly identify and eliminate abnormal displacement vectors, thus improving the system's real-time processing capabilities.

[0024] The image preprocessing module performs histogram equalization on the input image frame to enhance the feature point recognition of low-texture areas.

[0025] The feature point management module adopts a tiered threshold strategy: the minimum feature point threshold is set to 10 during takeoff and 30 during hovering.

[0026] The feature point management module performs periodic detection: feature point detection is forcibly triggered once every 30 frames to avoid calculation failure caused by the loss of feature points.

[0027] The feature point management module adopts a dynamic weight allocation mechanism: the current frame weight Wc=0.7, the historical frame weight Wh=0.3, and the distribution uniformity is optimized by fusing the feature point information of the current frame and the historical frame.

[0028] The optical flow verification module performs forward optical flow calculation: obtains the displacement vector Vf from the feature point of the current frame t to the tracking point of the next frame t+1.

[0029] The optical flow verification module performs reverse optical flow tracing: tracing back from the tracking point in frame t+1 to frame t to obtain the backtracking point.

[0030] The optical flow verification module generates an error mask: if the coordinate difference between the original feature points and the backtracking points of frame t is... If epsilon=0.5px, then the feature point is marked as invalid.

[0031] The feature point management module's 30-frame cycle detection is synchronized with the UAV flight control system clock to ensure the accuracy of the detection cycle.

[0032] The error mask of the optical flow verification module is output in bitmap format for the flight control system to eliminate abnormal displacement vectors in real time.

[0033] The image preprocessing module enhances the contrast of low-texture areas, improving feature point recognition and providing a more reliable foundation for optical flow calculation. This improves the hovering stability of the UAV in low-texture scenes and reduces shaking and drift caused by inaccurate positioning. A tiered threshold strategy adapts to the feature point quantity requirements of different flight stages of the UAV. Periodic detection prevents feature point loss that could lead to computational failure, and a dynamic weight allocation mechanism optimizes the uniformity of feature point distribution, making optical flow calculation more accurate and improving the adaptability of the optical flow algorithm to different flight states. The optical flow verification module effectively eliminates abnormal displacement vectors through forward optical flow calculation, reverse optical flow tracing, and error mask generation, providing more accurate optical flow information to the UAV flight control system. This helps the flight control system to more accurately control the UAV's flight attitude and position, improving flight safety and reliability.

[0034] The feature point management module's periodic detection is synchronized with the UAV flight control system's clock, ensuring the accuracy of the feature point detection cycle and further improving the stability and reliability of the optical flow algorithm. The error mask of the optical flow verification module is output in bitmap format, facilitating the flight control system to quickly identify and eliminate abnormal displacement vectors, thus improving the system's real-time processing capabilities.

[0035] The overall process includes the following steps: S1. Image input and preprocessing; S11. Raw image acquisition: The UAV's visual sensor acquires continuous frame images (frame t, frame t+1), which are then converted into grayscale images after being input into the system. This reduces the amount of computation while retaining the grayscale features required for optical flow calculation.

[0036] S12, Histogram Equalization Enhancement: The image preprocessing module performs histogram equalization on the grayscale image. By stretching the grayscale range of low-texture areas (such as marble and cement), the contrast is improved, providing richer texture information for subsequent feature point detection and solving the problem of insufficient feature points in low-texture scenes in the traditional Lucas-Kanade algorithm.

[0037] S2, Feature point detection and tracking; S21. Pyramid Construction and Corner Detection: Based on the drone_ive_st_lk_proc algorithm, an image pyramid is constructed from the preprocessed grayscale image. Multi-scale hierarchical processing improves the robustness of feature point tracking and adapts to scene changes at different UAV flight altitudes. The Shi-Tomasi corner detection algorithm (feature point basis of the Lucas-Kanade algorithm) is adopted, combined with the hierarchical threshold strategy of the feature point management module: the minimum feature point threshold is set to 10 during takeoff and 30 during hovering, ensuring that there are sufficient feature points for calculation in different flight stages.

[0038] S22, 30-frame periodic detection and dynamic weight allocation: The feature point management module forcibly triggers feature point detection every 30 frames (synchronized with the flight control system clock) to avoid tracking interruptions caused by feature point loss and to supplement new feature points to maintain computational continuity. Feature point information from the current frame and historical frames is fused, with the current frame weight Wc=0.7 and the historical frame weight Wh=0.3, optimizing the uniformity of feature point distribution and reducing motion vector drift accumulation caused by uneven distribution (solving the problem of uneven feature point distribution in the traditional Lucas-Kanade algorithm).

[0039] S3, Optical Flow Calculation and Anomaly Filtering (integrating Lucas-Kanade optical flow core and bidirectional verification mechanism). S31, Lucas-Kanade optical flow vector calculation: Based on the pyramid hierarchy, the forward optical flow is calculated using the Lucas-Kanade algorithm. By solving the optical flow equation under the assumption of grayscale invariance, the displacement vector Vf from the feature point in frame t to the tracking point in frame t+1 is obtained, thus completing the preliminary motion vector estimation.

[0040] S32. Bidirectional optical flow verification, reverse optical flow tracing: Apply the Lucas-Kanade algorithm backward from the tracking point in frame t+1 to obtain the coordinates in frame t. Compare the original feature point coordinates of frame t. Error mask generation: If `epsilon=0.5px` marks it as an invalid point, generates a bitmap format error mask, and solves the problem of insufficient filtering of mismatched points by traditional standard deviation and angle threshold methods.

[0041] S33. Outlier Filtering and Quality Assessment: Outlier displacement vectors are removed based on the error mask, and valid vectors are retained. At the same time, based on the outlier filtering step of the drone_ive_st_lk_proc algorithm, outlier vectors are filtered using the standard deviation of length and the mean of angle ±30°. The remaining vectors are then subjected to quality assessment to ensure the accuracy of the output displacement vectors.

[0042] S4, displacement conversion and flight control output; S41. Physical displacement conversion: Process the filtered effective displacement vector by dividing the pixel offset by the pixel focal length and multiplying by the scaling factor to convert it into the actual physical displacement (unit: meters), thus realizing the mapping from image pixel motion to the actual motion of the UAV.

[0043] S42. Low-pass filtering and data encapsulation: Low-pass filtering is applied to the physical displacement data to smooth high-frequency noise interference and improve data stability. The displacement vector, encapsulated according to the MAVLINK protocol, is input into the flight control system to provide precise displacement feedback for the UAV's hovering attitude adjustment, ultimately improving hovering stability in low-texture scenes.

[0044] drone_ive_st_lk_proc is specifically the processing flow of the Lucas-Kanade optical flow sparse tracking algorithm for drones. The Lucas-Kanade algorithm, or LK algorithm for short, is a classic optical flow estimation algorithm in the field of computer vision, used to calculate the motion information of objects in an image. This embodiment will not elaborate on it here.

[0045] Technical features description: The periodic detection unit of the feature point management module is synchronized with the flight control system clock to ensure that detection is triggered every 30 frames; the error mask of the bidirectional optical flow verification module is output in bitmap format for the flight control system to eliminate abnormal displacement vectors in real time.

[0046] Technical solution implementation data: Test environment: D-OMF20 drone (EVT stage), concrete floor / marble / checkered carpet scene; result: 1. Regarding the image preprocessing module, the following comparison shows the before and after results of histogram equalization on the marble floor dataset, examining the feature point distribution and optical flow visualization. Figure 4 and attached Figure 5 As shown; Regarding the image preprocessing module, the following is a comparison of the feature point distribution and optical flow visualization results before and after histogram equalization on the wooden floor dataset. Figure 6 and attached Figure 7 As shown; Conclusion: Previously, the wooden board and marble scenes had very few feature points. After histogram equalization preprocessing, the number of feature points increased significantly, and the uniformity of distribution was noticeably improved. The results show that the problem of insufficient feature points has been solved, which is beneficial for optical flow calculation.

[0047] 2. Regarding the feature point management module, the addition of dynamic feature point supplementation and 30-frame periodic feature point distribution detection yields better results than using feature point detection every 30 frames and dynamic feature point supplementation separately, respectively. The feature point distribution is significantly more uniform. A comparison chart is attached below. Figure 8 As shown; 3. Regarding the optical flow verification module: Evaluation metric: flow_magnitude (average motion amplitude) represents the average motion intensity of feature points in the scene (unit: pixels / frame).

[0048] It can be used to determine the dynamics of a scene: An amplitude close to 0 indicates that the feature point is almost stationary (static scene or stable hovering).

[0049] Larger values ​​(e.g., >5px): rapid movement or drastic camera movement.

[0050] The following is a comparison chart of the average motion amplitude tested using the plaid carpet dataset for optical flow algorithms. Figure 9 As shown; The blue line represents the scenario with two-way verification, where the average motion is more balanced. Without two-way verification, the motion fluctuates more, and sudden drifts occur. The results show that adding two-way verification is beneficial for stable hovering.

[0051] The image preprocessing module enhances the contrast of low-texture areas, improving feature point recognition and providing a more reliable foundation for optical flow calculation. This improves the hovering stability of the UAV in low-texture scenes and reduces shaking and drift caused by inaccurate positioning. A tiered threshold strategy adapts to the feature point quantity requirements of different flight stages of the UAV. Periodic detection prevents feature point loss that could lead to computational failure, and a dynamic weight allocation mechanism optimizes the uniformity of feature point distribution, making optical flow calculation more accurate and improving the adaptability of the optical flow algorithm to different flight states. The optical flow verification module effectively eliminates abnormal displacement vectors through forward optical flow calculation, reverse optical flow tracing, and error mask generation, providing more accurate optical flow information to the UAV flight control system. This helps the flight control system to more accurately control the UAV's flight attitude and position, improving flight safety and reliability.

[0052] The feature point management module's periodic detection is synchronized with the UAV flight control system's clock, ensuring the accuracy of the feature point detection cycle and further improving the stability and reliability of the optical flow algorithm. The error mask of the optical flow verification module is output in bitmap format, facilitating the flight control system to quickly identify and eliminate abnormal displacement vectors, thus improving the system's real-time processing capabilities.

[0053] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0054] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A multi-parameter collaborative optimization system for UAV optical flow algorithms, characterized in that, It includes an image preprocessing module, a feature point management module, and an optical flow verification module, which work together to optimize the optical flow displacement vector calculation; The image preprocessing module processes the input image frames to improve the contrast of low-texture areas; the feature point management module dynamically manages feature point detection and tracking; the optical flow verification module filters the displacement vector to remove outliers, and finally outputs the effective displacement vector to the UAV flight control system to improve the hovering stability of the UAV in low-texture scenes.

2. The multi-parameter collaborative optimization system for UAV optical flow algorithm according to claim 1, characterized in that, The image preprocessing module performs histogram equalization on the input image frames.

3. The multi-parameter collaborative optimization system for UAV optical flow algorithm according to claim 1, characterized in that, The feature point management module adopts a tiered threshold strategy: the minimum feature point threshold is set to 10 during takeoff and 30 during hovering.

4. The multi-parameter collaborative optimization system for UAV optical flow algorithm according to claim 1, characterized in that, The feature point management module performs periodic detection: feature point detection is forcibly triggered once every 30 frames.

5. The multi-parameter collaborative optimization system for UAV optical flow algorithm according to claim 1, characterized in that, The feature point management module adopts a dynamic weight allocation mechanism: the current frame weight Wc=0.7, the historical frame weight Wh=0.3, and the distribution uniformity is optimized by fusing the feature point information of the current frame and the historical frame.

6. The multi-parameter collaborative optimization system for UAV optical flow algorithm according to claim 1, characterized in that, The optical flow verification module performs forward optical flow calculation: obtains the displacement vector Vf from the feature point of the current frame t to the tracking point of the next frame t+1.

7. The multi-parameter collaborative optimization system for UAV optical flow algorithm according to claim 1, characterized in that, The optical flow verification module performs reverse optical flow tracing: tracing back from the tracking point in frame t+1 to frame t to obtain the backtracking point.

8. The multi-parameter collaborative optimization system for UAV optical flow algorithm according to claim 1, characterized in that, The optical flow verification module generates an error mask: if the coordinate difference between the original feature point and the backtracking point in frame t is... If epsilon=0.5px, then the feature point is marked as invalid.

9. The multi-parameter collaborative optimization system for UAV optical flow algorithm according to claim 1, characterized in that, The feature point management module performs 30-frame periodic detection and synchronizes with the UAV flight control system clock.

10. The multi-parameter collaborative optimization system for UAV optical flow algorithm according to claim 8, characterized in that, The error mask of the optical flow verification module is output in bitmap format, which is used by the flight control system to eliminate abnormal displacement vectors in real time.

Images