Improved unmanned aerial vehicle horizon real-time detection method and system based on YOLO segmentation

Abstract

The application discloses an unmanned aerial vehicle horizon real-time detection method and system based on improved YOLO segmentation, and relates to the technical field of horizon detection, and comprises the following steps: S1, acquiring a front view image or a video frame sequence of an unmanned aerial vehicle; S2, performing sky region segmentation by using a lightweight sky segmentation network; S3, extracting a low sky boundary point set from a sky mask; S4, fitting a current frame horizon parameter based on the low sky boundary point set; S5, performing adaptive time sequence smoothing processing based on boundary integrity; and S6, outputting the horizon parameter after time sequence smoothing. The application adopts the unmanned aerial vehicle horizon real-time detection method and system based on improved YOLO segmentation, extracts a sky region by using a lightweight sky segmentation network, reduces the horizon search range from a complex original image to a real demarcation region between the sky and the non-sky, and effectively reduces the interference of buildings, ground textures, cloud changes and complex background edges on the detection result.

Description

Technical Field

[0001] This invention relates to the field of horizon detection technology, and in particular to an improved YOLO segmentation method and system for real-time horizon detection of unmanned aerial vehicles. Background Technology

[0002] Existing UAV horizon detection technologies mainly include traditional methods based on edges or grayscale changes, methods based on geometric constraints, semantic segmentation methods based on deep learning, and methods based on video temporal filtering. While traditional edge or Hough transform methods are simple to implement, they are susceptible to false edges in complex backgrounds such as buildings, clouds, and ground textures. Geometric constraint methods are effective for specific structured scenes but rely heavily on scene priors, lacking versatility. Deep learning segmentation methods improve semantic recognition capabilities, but existing models are mostly designed for general segmentation tasks, resulting in large parameter counts, high computational costs, and insufficient boundary geometric continuity, making it difficult to balance real-time airborne deployment with stable horizon extraction. Conventional temporal smoothing methods typically do not adaptively adjust based on the current frame's boundary observation quality, leading to fluctuating output results in scenarios with occlusion, jitter, or missing boundaries. Therefore, a UAV horizon detection technology solution that balances lightweight design, boundary stability, geometric fitting accuracy, and temporal robustness is urgently needed. Summary of the Invention

[0003] The purpose of this invention is to provide an improved YOLO segmentation method and system for real-time horizon detection in UAVs, overcoming the problems of traditional horizon detection methods in the prior art, such as susceptibility to interference from complex backgrounds, unstable single-frame fitting, high airborne deployment costs, and jitter in timing output. This invention achieves high robustness, low latency, and real-time deployment of horizon detection for UAV application scenarios.

[0004] To achieve the above objectives, this invention provides an improved method for real-time UAV horizon detection using YOLO segmentation, comprising the following steps: S1. Image acquisition: Acquire forward-view images or video frame sequences during flight using the UAV's onboard visual acquisition equipment; S2, Sky Segmentation: Sky region segmentation is performed using a lightweight sky segmentation network; S3. Low sky boundary extraction: Extract the low sky boundary point set from the sky mask; S4, horizon fitting; S3, obtaining the boundary point set. P Then, a straight line is fitted to the boundary point set to obtain the geometric parameters of the horizon in the current frame; S5, Timing Smoothing: Adaptive timing smoothing processing based on boundary integrity; S6. Output the results. Output the time-smoothed horizon parameters, including the horizon tilt angle, horizon position, and the corresponding visualization overlay results.

[0005] Preferably, the scenes collected in S1 include unobstructed clear horizon scenes, scenes where buildings partially obstruct the horizon, scenes where buildings completely obstruct the horizon, and complex scenes under different weather, lighting, and flight perspective conditions.

[0006] Preferably, the sky segmentation network in S2 includes a backbone feature extraction module, a feature fusion module, and a segmentation output module. The backbone feature extraction module adopts a lightweight feature extraction unit, introduces a boundary-aware attention unit in the shallow feature stage, and introduces a context enhancement unit in the deep feature stage.

[0007] Preferably, the feature fusion module is used to integrate features at different scales and enhance boundary details, including a top-down upsampling fusion path, a high-resolution boundary enhancement branch, and a bottom-up aggregation backflow path.

[0008] Preferably, the sky mask in S3 is set to The image width is For each column of x-coordinates in the image The search satisfies The position of the pixel with the largest y-coordinate among the pixels is denoted as: in, This indicates that the corresponding pixel is identified as a sky region, and the lowest sky pixels extracted from each column can form a set of boundary points: If a column contains no valid sky pixels, that column is marked as invalid and skipped from processing. Let the number of valid boundary points be... Then define the boundary integrity index. for: in, Used to characterize the observability, continuity, and integrity of the low sky boundary in the current frame.

[0009] Preferably, in S4, a two-dimensional straight line fitting method is used to fit the boundary points to obtain the parameters of the fitted straight line: in, This represents the direction vector of the fitted line. This represents a point on the fitted line.

[0010] Preferably, the horizon observation in the current frame of the image in S4: in, It reflects the degree of tilt of the current horizon relative to the horizontal axis of the image. It reflects the vertical position of the horizon at the center of the image.

[0011] Preferably, a time-series smoothing mechanism is introduced in S5 to construct a state vector containing the horizon tilt angle, the horizon longitudinal position, and the rate of change: in, Indicates the first Frame horizon tilt angle, Indicates the first The vertical coordinate of the horizon at the center of the frame image. and These represent the rates of change of the corresponding parameters; The observation vector for the current frame is: in, and These are the horizon observation parameters for the current frame obtained from S4.

[0012] Preferably, the boundary integrity index obtained in S5 based on S3 is... Adaptive adjustment of the measurement noise covariance matrix, measurement noise covariance matrix It can be represented as: in, The preset basic measurement noise covariance matrix is ​​used.

[0013] An improved YOLO-based real-time horizon detection method for UAVs includes an image acquisition module, a sky segmentation module, a boundary extraction module, a horizon fitting module, a temporal smoothing module, and a result output module. The image acquisition module acquires front-view images or video frames of the UAV; the sky segmentation module generates a sky mask using a lightweight sky segmentation network; the boundary extraction module extracts low-sky boundary points from the sky mask and calculates the boundary integrity index; the horizon fitting module fits the horizon geometric parameters based on the boundary point set; the temporal smoothing module performs adaptive filtering based on boundary integrity; and the result output module outputs the horizon detection results and their visualization information.

[0014] Therefore, this invention employs the aforementioned improved YOLO segmentation-based real-time drone horizon detection method and system. By first performing semantic sky segmentation and then extracting the low boundary between the sky and non-sky areas, it can more accurately pinpoint the region where the true horizon is located, thereby improving detection reliability in complex scenes. The lightweight feature extraction structure reduces the number of model parameters and computational load, making it more suitable for deployment on resource-constrained platforms such as drones and embedded vision terminals. Through shallow boundary awareness attention, deep context enhancement, and high-resolution boundary enhancement branches, the continuity, integrity, and geometric stability of the low sky boundary are improved, making the segmentation results more suitable as input for subsequent horizon fitting.

[0015] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the process for the improved YOLO segmentation method for real-time detection of the horizon of unmanned aerial vehicles according to the present invention. Detailed Implementation

[0017] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0018] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0019] Example Please see Figure 1 This invention provides an improved method and system for real-time detection of UAV horizon using YOLO segmentation, comprising the following steps: S1. Acquire forward-view images or video frame sequences from the UAV. This is done using the UAV's onboard visual acquisition equipment. Image data can originate from real-time cameras, onboard video streams, or offline stored video files. Acquisition scenarios include unobstructed, clear horizon scenes; scenes with partially obscured horizons by buildings; scenes with completely obscured horizons by buildings; and complex scenes under different weather, lighting, and flight perspective conditions. To ensure consistency in subsequent processing, the input images can be scaled, normalized, and converted in format according to actual deployment requirements to meet the input requirements of the sky segmentation network.

[0020] S2. Sky region segmentation using a lightweight sky segmentation network; the image obtained in S1 is input into the lightweight sky segmentation network, which outputs the corresponding sky region segmentation mask. The lightweight sky segmentation network extracts the sky region, narrowing the horizon search range from the complex original image to the true boundary between the sky and non-sky areas, effectively reducing the interference of buildings, ground textures, cloud variations, and complex background edges on the detection results. The lightweight feature extraction structure reduces model complexity, a shallow boundary awareness mechanism enhances the spatial response capability of the low sky boundary, deep context enhancement improves the semantic representation capability of the sky in complex scenes, and a high-resolution boundary enhancement branch improves the continuity, integrity, and geometric stability of the low sky boundary, making the segmentation results more suitable for subsequent horizon fitting.

[0021] The Sky Segmentation Network is a single-class segmentation network optimized for UAV horizon detection tasks, identifying only the sky category. The network includes a backbone feature extraction module, a feature fusion module, and a segmentation output module. The backbone feature extraction module extracts multi-scale features from the input image. Lightweight feature extraction units replace conventional high-complexity convolutional stacking structures in the backbone network to reduce the number of parameters and computational cost. Boundary-aware attention units are introduced in the shallow feature stage to enhance the spatial location information representation of the sky-non-sky boundary region and improve the response capability of low sky boundaries. Context enhancement units are introduced in the deep feature stage to strengthen the semantic representation of the sky under complex backgrounds, distant horizons, and occlusion conditions.

[0022] The backbone feature extraction module includes: an initial convolutional downsampling unit, used to convert the input image into a basic feature map; a multi-level lightweight feature extraction unit, preferably using a Ghost-based bottleneck structure instead of a conventional heavy residual / stacked convolutional structure to reduce the number of parameters and computational cost; a shallow boundary-aware attention unit, set in the shallow feature stage, used to enhance the representation of positional information in the horizontal and vertical directions and improve the feature response near the sky boundary; and a deep context enhancement unit, set in the deep feature stage, used to fuse local and global contextual information.

[0023] The feature fusion module integrates features at different scales and enhances boundary details. This module includes: a top-down upsampling fusion path to combine deep semantic features with mid-to-shallow high-resolution features; a high-resolution boundary enhancement branch to fuse shallow boundary details with high-level semantic features at a higher resolution scale; and a bottom-up aggregation backflow path to feed the high-resolution boundary enhancement results back to mid-to-high-level features, thereby simultaneously improving the semantic correctness and geometric continuity of the segmentation output.

[0024] In the feature fusion stage, a high-resolution boundary enhancement branch is set up to progressively upsample high-level semantic features to a higher resolution scale and fuse them with shallow boundary detail features. Then, the data flows back to mid-to-high-level features via a bottom-up aggregation path to enhance the continuity and geometric integrity of the low-level sky boundary. Finally, the sky segmentation result is output by the segmentation output module based on the multi-scale fused features, which can output a binary mask, a probability map, or an instance segmentation mask. Since this invention only performs single-class detection on the sky, the output head can be simplified to a single-class segmentation head to further reduce the computational burden. Through the above approach, the model complexity can be significantly reduced while maintaining sky segmentation accuracy, improving its real-time deployment capability on UAV platforms with limited onboard computing power.

[0025] S3. Extract the low sky boundary point set from the sky mask. After obtaining the sky segmentation mask, further extract the low sky boundary for horizon estimation from the sky mask.

[0026] Let the sky mask be The image width is For each column of x-coordinates in the image The search satisfies The position of the pixel with the largest y-coordinate among the pixels is denoted as: in, This indicates that the corresponding pixel is identified as part of the sky region. The lowest sky pixels extracted from each column form the boundary point set: If a column contains no valid sky pixels, that column is marked as invalid and skipped from processing. Let the number of valid boundary points be... Then define the boundary integrity index. for: in, Used to characterize the observability, continuity, and completeness of the low sky boundary in the current frame. When A larger value indicates that there is sufficient boundary information available for fitting in the current frame; when... A smaller value indicates that the current frame may have issues such as building occlusion, missing boundaries, jitter blurring, or discontinuous segmentation. Compared to directly performing edge extraction and Hough transform on the original image, this invention constructs a boundary point set through sky segmentation results, which can more accurately locate the "sky-non-sky" boundary region, thereby reducing the interference of building textures, high-frequency ground edges, and complex backgrounds on horizon detection.

[0027] S4. Based on the low sky boundary point set, the horizon parameters of the current frame are fitted. Instead of directly outputting a segmentation mask for the sky segmentation results, the lowest-ranking sky pixels are extracted column-by-column to form a low sky boundary point set. Linear fitting is then performed based on this boundary point set to obtain tilt and position parameters that characterize the horizon state of the current frame. This allows the sky semantic segmentation results to be further transformed into structured geometric information suitable for UAV attitude-assisted perception and visual stabilization control, thus distinguishing it from existing methods that only output semantic segmentation results.

[0028] In obtaining the boundary point set Then, a straight line is fitted to the boundary point set to obtain the geometric parameters of the horizon in the current frame. A two-dimensional straight line fitting method can be used to fit the boundary points to obtain the fitted straight line parameters: in, This represents the direction vector of the fitted line. This represents a point on the fitted line. Further, the horizon inclination angle is calculated based on the fitted parameters. : Simultaneously, the horizon coordinate at a certain reference x-coordinate in the image is selected as the position parameter; preferably, the x-coordinate of the image center is used. As a reference position, the ordinate of the horizon at the center of the image It can be represented as: This constitutes the horizon observation for the current frame: in, It reflects the degree of tilt of the current horizon relative to the horizontal axis of the image. It reflects the vertical position of the horizon at the center of the image. By representing the horizon as the tilt angle and the ordinate of the reference position, it facilitates subsequent video temporal modeling, UAV attitude-assisted estimation, and visual stabilization control.

[0029] S5. Adaptive temporal smoothing processing based on boundary integrity: The proportion of effective boundary points to the image width represents the completeness of the horizon observation in the current frame. The observation noise weight is adaptively adjusted according to this index, so that the credibility of the current frame observation is improved when the boundary information is relatively complete, while the impact of the current frame observation on the final result is reduced when there is building occlusion, jitter, boundary loss or local segmentation anomaly.

[0030] For horizon observations in consecutive frames of a video sequence, this invention further introduces a temporal smoothing mechanism to improve the stability of the horizon output. A state vector is constructed containing the horizon tilt angle, the horizon longitudinal position, and its rate of change. in, Indicates the first Frame horizon tilt angle, Indicates the first The vertical coordinate of the horizon at the center of the frame image. and These represent the rates of change of the corresponding parameters. The state transition process can be represented using a constant velocity model as follows: in, This represents the time interval between adjacent video frames. The observation vector for the current frame is: in, and The horizon observation parameters for the current frame are obtained from S4. Unlike existing fixed-parameter filtering methods, this invention uses the boundary integrity index obtained from S3. Adaptive adjustment of the measurement noise covariance matrix. Measurement noise covariance matrix. It can be represented as: in, The preset basic measurement noise covariance matrix is ​​used. When the boundary integrity index... A larger value indicates that the low sky boundary observation in the current frame is relatively complete, and the observation results have high reliability. Therefore, the filter assigns a larger weight to the current observation; when the boundary integrity index... When the value is small, it indicates that there are insufficient boundary points in the current frame or that the boundary is significantly affected by factors such as occlusion, jitter, and segmentation errors. In this case, the weight of the current observation is reduced, making the output result more dependent on the predicted values ​​of historical states. Through this adaptive mechanism, the horizon parameter jumps caused by instantaneous false detections, local occlusion, and flight jitter can be effectively suppressed, thereby improving the temporal stability of the horizon in consecutive video frames.

[0031] S6. Output the horizon detection results. Output the time-smoothed horizon parameters, including the horizon tilt angle, horizon position, and the corresponding visualization overlay result. The smoothed horizon line can be drawn on the current video frame image, simultaneously displaying the horizon angle information; alternatively, the horizon parameters can be sent to the UAV flight control module, attitude-assisted perception module, visual navigation module, trajectory correction module, or environmental perception module for subsequent control decisions. Through the above processing, this invention not only outputs stable horizon visualization results but also provides reliable input for UAV flight state estimation and visual-assisted control.

[0032] An improved YOLO-based real-time drone horizon detection system includes an image acquisition module, a sky segmentation module, a boundary extraction module, a horizon fitting module, a temporal smoothing module, and a result output module. The image acquisition module acquires front-view images or video frames of the drone; the sky segmentation module generates a sky mask using a lightweight sky segmentation network; the boundary extraction module extracts low-level sky boundary points from the sky mask and calculates boundary integrity indices; the horizon fitting module fits the horizon geometric parameters based on the boundary point set; the temporal smoothing module performs adaptive filtering based on boundary integrity; and the result output module outputs the horizon detection results and their visualization information.

[0033] A dataset was constructed based on approximately 1000 image frames collected from real-world drone scenarios. These scenarios included situations with the horizon completely obscured by buildings, partially obscured by buildings, clear horizons without obstruction, different weather conditions, different lighting conditions, and different flight perspectives. The dataset was augmented to approximately 6000 images, incorporating flipping and contrast adjustments. The lightweight sky segmentation network of this invention was trained and validated using this dataset. Results showed that the complete model of this invention has approximately 2.757M parameters, approximately 10.1 GFLOPs of computation, and an inference speed of approximately 29.29 FPS. The sky segmentation metrics achieved high levels, indicating that this invention significantly reduces the number of parameters and computation while maintaining high segmentation performance, making it suitable for real-time deployment on drone platforms. Furthermore, this invention can still obtain relatively stable horizon estimation results under occlusion, complex backgrounds, and jittery scenarios, demonstrating good engineering feasibility and application value.

[0034] The implementation results show that, while maintaining high sky segmentation accuracy, the present invention has approximately 2.757M model parameters, approximately 10.1 GFLOPs of computation, and an inference speed of approximately 29.29 FPS, demonstrating good real-time processing capabilities and engineering feasibility. For various complex scenarios, including complete building occlusion, partial occlusion, no occlusion, and different weather conditions, lighting conditions, and flight perspectives, the present invention can output relatively stable horizon detection results.

[0035] Therefore, this invention employs the improved YOLO segmentation-based real-time UAV horizon detection method and system described above. It extracts the sky region from the UAV's forward-view image using a lightweight sky segmentation network, then constructs a low-sky boundary point set based on a sky mask and performs geometric fitting of the horizon. This avoids the background interference problem caused by traditional methods directly searching for the horizon in the original image. By performing semantic sky segmentation first and then extracting the low boundary between the sky and non-sky areas, the true horizon region can be more accurately located, thereby improving detection reliability in complex scenes. This invention does not simply pursue high sky region segmentation accuracy, but rather optimizes the network structure specifically for the horizon detection task. It maintains high sky segmentation performance while balancing low latency and real-time performance, meeting the online perception requirements during UAV flight. This invention proposes a horizon parameterization fitting method based on a low-sky boundary point set, representing the current frame's horizon as the tilt angle and the ordinate at the reference position, facilitating subsequent temporal modeling and attitude-assisted analysis. This invention introduces an adaptive temporal smoothing method based on boundary integrity, which dynamically adjusts the observation noise weight according to the integrity of the effective boundary points in the current frame. This can significantly improve the stability of the horizon output in continuous video, reduce jumps and jitter, and is more suitable for application scenarios such as UAV attitude-assisted perception, visual stabilization control, and trajectory correction.

[0036] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. An improved method for real-time horizon detection of UAVs using YOLO segmentation, characterized in that, Includes the following steps: S1. Image acquisition: Acquire forward-view images or video frame sequences during flight using the UAV's onboard visual acquisition equipment; S2, Sky Segmentation: Sky region segmentation is performed using a lightweight sky segmentation network; S3. Low sky boundary extraction: Extract the low sky boundary point set from the sky mask; S4, horizon fitting; S3, obtaining the boundary point set. P Then, a straight line is fitted to the boundary point set to obtain the geometric parameters of the horizon in the current frame; S5, Timing Smoothing: Adaptive timing smoothing processing based on boundary integrity; S6. Output the results. Output the time-smoothed horizon parameters, including the horizon tilt angle, horizon position, and the corresponding visualization overlay results.

2. The improved YOLO segmentation method for real-time UAV horizon detection according to claim 1, characterized in that: The scenes captured in S1 include unobstructed clear horizon scenes, scenes with buildings partially obscuring the horizon, scenes with buildings completely obscuring the horizon, and complex scenes under different weather, lighting, and flight perspective conditions.

3. The improved YOLO segmentation method for real-time UAV horizon detection according to claim 2, characterized in that: The sky segmentation network in S2 includes a backbone feature extraction module, a feature fusion module, and a segmentation output module. The backbone feature extraction module uses a lightweight feature extraction unit, introduces a boundary-aware attention unit in the shallow feature stage, and introduces a context enhancement unit in the deep feature stage.

4. The improved YOLO segmentation method for real-time UAV horizon detection according to claim 3, characterized in that: The feature fusion module is used to integrate features at different scales and enhance boundary details, including a top-down upsampling fusion path, a high-resolution boundary enhancement branch, and a bottom-up aggregation backflow path.

5. The improved YOLO segmentation method for real-time UAV horizon detection according to claim 4, characterized in that: S3 sets the sky mask as follows: The image width is For each column of x-coordinates in the image The search satisfies The position of the pixel with the largest y-coordinate among the pixels is denoted as: in, This indicates that the corresponding pixel is identified as a sky region, and the lowest sky pixels extracted from each column can form a set of boundary points: If a column contains no valid sky pixels, that column is marked as invalid and skipped from processing. Let the number of valid boundary points be... Then define the boundary integrity index. for: in, Used to characterize the observability, continuity, and integrity of the low sky boundary in the current frame.

6. The improved YOLO segmentation method for real-time UAV horizon detection according to claim 5, characterized in that: In S4, a two-dimensional line fitting method is used to fit the boundary points to obtain the parameters of the fitted line: in, This represents the direction vector of the fitted line. This represents a point on the fitted line.

7. The improved YOLO segmentation method for real-time UAV horizon detection according to claim 6, characterized in that: Horizon observation in the current frame of the image in S4: in, It reflects the degree of tilt of the current horizon relative to the horizontal axis of the image. It reflects the vertical position of the horizon at the center of the image.

8. The improved YOLO segmentation method for real-time UAV horizon detection according to claim 7, characterized in that: S5 introduces a time-series smoothing mechanism to construct a state vector containing the horizon tilt angle, the horizon longitudinal position, and the rate of change: in, Indicates the first Frame horizon tilt angle, Indicates the first The vertical coordinate of the horizon at the center of the frame image. and These represent the rates of change of the corresponding parameters; The observation vector for the current frame is: in, and These are the horizon observation parameters for the current frame obtained from S4.

9. The improved YOLO segmentation method for real-time UAV horizon detection according to claim 8, characterized in that: Boundary integrity index obtained from S3 in S5 Adaptive adjustment of the measurement noise covariance matrix, measurement noise covariance matrix It can be represented as: in, The preset basic measurement noise covariance matrix is ​​used.

10. An improved YOLO segmentation-based real-time drone horizon detection method system, applied to the improved YOLO segmentation-based real-time drone horizon detection method according to any one of claims 1-9, characterized in that: The system includes an image acquisition module, a sky segmentation module, a boundary extraction module, a horizon fitting module, a temporal smoothing module, and a result output module. The image acquisition module is used to acquire forward-view images or video frames of the UAV; the sky segmentation module is used to generate a sky mask using a lightweight sky segmentation network; the boundary extraction module is used to extract low-sky boundary points from the sky mask and calculate the boundary integrity index; the horizon fitting module is used to fit the horizon geometric parameters based on the boundary point set; the temporal smoothing module is used to perform adaptive filtering based on boundary integrity; and the result output module is used to output the horizon detection results and their visualization information.

Images