Fragmented sky polarization orientation correction method and system for drone navigation

Abstract

This invention provides a method and system for correcting the polarization of fragmented sky light for UAV navigation, relating to the fields of UAV navigation and multi-sensor fusion technology. The method includes: during UAV flight, acquiring a raw polarization image containing sky regions in real time and performing distortion correction to obtain a distortion-free polarization image; using a lightweight sky segmentation model to identify the distortion-free polarization image and extract effective sky fragment regions; extracting the polarization information of the effective sky fragment regions based on the distortion-free polarization image covered by the effective sky fragment regions; calculating the absolute geographic heading angle of the UAV at the current moment using a fragmented polarization heading solution model; obtaining the predicted heading angle at the current moment and correcting it using an unscented Kalman filter method; calculating the deviation between the corrected heading angle and the desired heading angle to control the UAV to fly along a predetermined route. This invention effectively achieves stable and reliable control of the UAV's positioning trajectory.

Description

Technical Field

[0001] This invention relates to the field of UAV navigation and multi-sensor fusion technology, and in particular to a method and system for correcting the orientation of fragmented sky polarized light for UAV navigation. Background Technology

[0002] Drones are increasingly used in smart cities, logistics, and emergency rescue. However, in the "urban canyon" environment filled with tall buildings, Global Navigation Satellite System (GNSS) signals are severely obstructed, reflected, and interfered with by multipath effects, leading to signal failure or a sharp decline in accuracy. In such GNSS-denied environments, drones typically rely on Inertial Navigation Systems (INS) for dead reckoning. However, low-cost Micro-Electro-Mechanical Systems (MEMS) Inertial Measurement Units (IMUs) exhibit significant bias and noise, causing their calculated heading angles to drift rapidly over time. This results in a quadratic increase in positioning error, leading to significant deviations from the intended flight path and even posing a collision risk.

[0003] To address the drift problem of atmospheric polarization navigation systems (INS), researchers have introduced various auxiliary sensors. Among them, orientation technology based on atmospheric polarization patterns, as a passive and interference-resistant absolute orientation method, has attracted widespread attention. This technology mimics the navigation mechanism of organisms such as sand ants, calculating the vehicle's heading by detecting the distribution pattern of polarized light in the sky. However, traditional polarization navigation methods typically assume operation under conditions of open sky or mostly visible sky, and their mathematical models rely on a complete (or large-area) distribution of sky polarization angles. In urban canyons, the sky is fragmented into scattered, narrow, and irregular "pieces" (such as building gaps) by buildings. Traditional methods fail because they cannot obtain a sufficiently large and shaped area of ​​sky, resulting in poor or inaccurate calculations.

[0004] Therefore, there is an urgent need for a technical solution that can adapt to extremely complex urban environments and achieve highly reliable heading correction using only scattered and fragmented sky information. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention proposes a fragmented sky polarization orientation correction method and system for UAV navigation based on a lightweight sky segmentation model. The aim is to utilize fragmented sky information captured in real-time by a polarization camera in "urban canyon" environments where GNSS signals are unavailable to calculate the absolute heading angle. This angle is then used as a powerful observation to correct INS heading drift, thereby achieving stable and reliable control of the UAV's positioning trajectory. This invention is particularly suitable for complex urban scenarios where GNSS signals are denied.

[0006] On the one hand, this invention proposes a method for correcting the orientation of fragmented sky polarized light for UAV navigation, which includes the following process:

[0007] During the flight of the UAV, the original polarization image containing the sky area is acquired in real time and the distortion is corrected to obtain a distortion-free polarization image.

[0008] A lightweight sky segmentation model is used to identify distortion-free polarization images, obtain a probability distribution map of the sky region, and extract multiple effective sky fragment regions from the probability distribution map of the sky region.

[0009] For each valid sky debris region, the polarization information of the valid sky debris region is extracted based on the distortion-free polarization image covered by the valid sky debris region.

[0010] Based on the polarization information of all valid sky debris regions, the absolute geographic heading angle of the UAV at the current moment is calculated using the fragmented polarization heading solution model.

[0011] The predicted heading angle at the current moment is obtained, and the predicted heading angle is corrected by using an unscented Kalman filter method based on the absolute geographic heading angle of the UAV at the current moment, so as to obtain the corrected heading angle.

[0012] The deviation between the corrected heading angle and the UAV's desired heading angle is calculated, and the UAV is controlled to fly along the predetermined route based on this deviation.

[0013] Furthermore, the specific method for acquiring raw polarization images containing sky regions in real time during UAV flight and performing distortion correction to obtain distortion-free polarization images is as follows:

[0014] During the flight of the drone, the polarization camera on board the drone is used to collect raw polarization images containing the sky area in real time.

[0015] The original polarization image includes: original polarization component images of at least three different known polarization directions at the same time;

[0016] Obtain the intrinsic parameter matrix K and distortion coefficient D of the polarization camera;

[0017] For any original polarization component image in the original polarization image, the distortion of the original polarization component image is corrected using the intrinsic parameter matrix K and the distortion coefficient D to obtain the corrected polarization component image.

[0018] The original polarization component image is resampled using the corrected polarization component image to obtain a distortion-free polarization component image. Then, all the obtained distortion-free polarization component images are used as the distortion-free polarization image.

[0019] Furthermore, the lightweight sky segmentation model consists of an encoder and a decoder;

[0020] The encoder uses MobileNetV3 to extract features from the input image layer by layer, resulting in a multi-scale feature map.

[0021] The decoder performs step-by-step fusion of multi-scale feature maps to obtain the final feature map. The specific process of mapping the final feature map to a probability distribution map of the sky region is as follows: the multi-scale feature maps are arranged in order of increasing resolution, denoted as... ;in Represents multi-scale feature maps; The number of feature maps representing a multi-scale feature map; This represents the first-level feature map; This represents the second-level feature map; Indicates the first Level feature map;

[0022] The lowest resolution level 1 feature map is obtained through bilinear upsampling. Restore to the previous level feature map After achieving the same resolution, then compare with the feature map. Channel splicing is performed to obtain the first fused feature map. ;

[0023] From the first fusion feature map To begin, repeat the following process:

[0024] The first... Fusion Feature Map Restore to feature map After achieving the same resolution, then compare with the feature map. Perform channel splicing to obtain the first... Fusion Feature Map ;in ;

[0025] Until the first Fusion Feature Map Until then, and will the Fusion Feature Map As the final feature map;

[0026] After mapping the final feature map to a single-channel feature map using two-dimensional convolution, the probability distribution map of the sky region is obtained by passing the Sigmoid activation function; the value of each pixel in the probability distribution map of the sky region is the probability that the pixel belongs to the sky category, and the value ranges from 0 to 1.

[0027] Furthermore, the specific method for using a lightweight sky segmentation model to identify distortion-free polarization images, obtaining a probability distribution map of the sky region, and extracting multiple effective sky debris regions from the probability distribution map of the sky region is as follows:

[0028] Input any distortion-free polarization component image from the distortion-free polarization image into the lightweight sky segmentation model to obtain the probability distribution map of the sky region;

[0029] The probability distribution map of the sky region is binarized according to a preset threshold to obtain a sky segmentation mask; each pixel in the sky segmentation mask is marked as either sky or non-sky.

[0030] Connectivity labeling is performed on the sky segmentation mask, marking all adjacent sky pixels as the same connected region, and each connected region as a sky fragment region;

[0031] For each sky fragment region, calculate the pixel area of ​​the sky fragment region. If the calculated pixel area is less than the preset area threshold, the sky fragment region is regarded as an invalid region and is removed.

[0032] A sky fragment region whose area per pixel is not less than a preset area threshold is considered a valid sky fragment region.

[0033] Furthermore, the specific method for extracting the polarization information of each valid sky debris region based on the distortion-free polarization image covered by that valid sky debris region is as follows:

[0034] The polarization information includes: representative polarization angle, average degree of polarization, and observation direction vector;

[0035] For any valid sky debris region According to the effective sky debris area Calculate the Stokes parameter for each pixel covered on the undistorted polarization component image in the undistorted polarization image.

[0036] Calculate the polarization angle of each pixel based on the Stokes parameters. With polarization degree ;

[0037] Take valid sky debris area Polarization angle of all pixels The median value is used as the effective sky debris region. The representative polarization angle ;

[0038] Calculate the effective sky debris region polarization degree of all pixels The average value is used as the effective sky debris area. average polarization ;

[0039] Calculate the effective sky debris region using any distortion-free polarization component image from a distortion-free polarization image. The average coordinates of all pixels within the area are used as the effective sky fragment region. centroid coordinates ;in Indicates the effective sky debris area The horizontal coordinate of the centroid in the pixel coordinate system; Indicates the effective sky debris area The ordinate of the centroid in the pixel coordinate system;

[0040] The effective sky debris region is determined using the intrinsic parameter matrix K. centroid coordinates By back-projecting onto the camera coordinate system, the effective sky debris region can be obtained. Observation direction vector .

[0041] Furthermore, the specific method for calculating the absolute geographic heading angle of the UAV at the current moment using the fragmented polarization heading solution model based on the polarization information of all valid sky debris regions is as follows:

[0042] Obtain the current UTC timestamp and the latitude and longitude of the UAV, and use the Solar Position Algorithm (SPA) to calculate the solar altitude angle at the current moment based on the UTC timestamp and the UAV's latitude and longitude. and solar azimuth This leads to the construction of solar position parameters. ;

[0043] Based on polarization information and solar position parameters of all valid sky debris regions A nonlinear least squares optimization problem is constructed and used as a solution model for fragmented polarization heading.

[0044] ;

[0045] in, Indicates the absolute geographic heading angle of the drone; Indicates to Find the minimum value; The number of effective sky debris areas; This represents the theoretical calculation function for the polarization angle based on the single-scattering Rayleigh model. It is used to calculate the theoretical polarization angle of the effective sky debris region based on the UAV's absolute geographic heading angle, solar position parameters, and the observation direction vector of the effective sky debris region.

[0046] Based on the fragmented polarization heading calculation model, the Random Sample Consensus (RANSAC) algorithm is used to generate the RANSAC in-point set and the initial heading estimate of the UAV.

[0047] Using the initial estimate of the UAV's heading as the initial value, a nonlinear optimization algorithm is employed based on the RANSAC interior point set to solve the fragmented polarization heading calculation model, thereby obtaining the UAV's absolute geographic heading angle. .

[0048] Furthermore, the calculation process of the polarization angle calculation function based on the single-scattering Rayleigh model is as follows:

[0049] For any valid sky debris region By utilizing the pre-calibrated extrinsic rotation matrix of the polarization camera relative to the aircraft coordinate system, the effective sky debris region is captured. Observation direction vector Transform to body coordinate system;

[0050] Get the current drone roll angle and drone pitch angle ;

[0051] Utilizing the current drone roll angle UAV pitch angle and the absolute geographical heading angle of the drone Construct a rotation matrix from the body coordinate system to the geographic coordinate system. ;

[0052] Using the rotation matrix Effective sky debris area Observation direction vector Transform to the geographic coordinate system to obtain the observation direction vector in the geographic coordinate system. ;

[0053] Based on the solar position parameters, construct the solar direction vector in the geographic coordinate system. ;

[0054] Calculate the solar direction vector in the geographic coordinate system. Observation direction vector in geographic coordinate system The vector product yields the effective sky debris region. Theoretical polarization direction vector in geographic coordinate system ;

[0055] Effective Sky Fragmentation Area Theoretical polarization direction vector in geographic coordinate system Transform to camera coordinate system to obtain effective sky debris region Theoretical polarization direction vector in camera coordinate system ;

[0056] According to the effective sky debris area Theoretical polarization direction vector in camera coordinate system Calculate the effective sky debris region Theoretical polarization angle .

[0057] Furthermore, the specific method for generating the RANSAC in-point set and the initial heading estimate of the UAV based on the fragmented polarization heading solution model and using the Random Sample Consensus (RANSAC) algorithm is as follows:

[0058] Set the maximum number of iterations, start the iteration, and perform the following operations during each iteration:

[0059] Initialize an empty hypothetical interior point set;

[0060] The representative polarization angle of each valid sky debris region and observation direction vector As a sample, from The smallest sample set is randomly selected from the samples; the number of samples in the smallest sample set is... ;

[0061] Based on the minimum sample set, the assumed heading angle value is calculated in reverse using the polarization angle calculation function based on the single-scattering Rayleigh model. ;

[0062] for For each of the samples, based on the assumed heading angle value Given the solar position parameters and the observed direction vector of the sample, the theoretical polarization angle of the sample is calculated using a polarization angle calculation function based on the Rayleigh model of single scattering. Then, the theoretical polarization angle and the representative polarization angle of the sample are calculated. The residual;

[0063] All samples with residuals less than a preset threshold are included in the hypothesis interior set of the current iteration.

[0064] When the maximum number of iterations is reached, the iteration ends. By comparing the number of samples contained in the hypothetical interior point sets of all iteration rounds, the hypothetical interior point set with the largest number of samples is selected as the RANSAC interior point set, and the heading angle hypothesis value corresponding to this hypothetical interior point set is set. As an initial estimate of the drone's heading .

[0065] Furthermore, the specific method for obtaining the predicted heading angle at the current moment, and correcting the predicted heading angle using an unscented Kalman filter based on the UAV's absolute geographic heading angle calculated at the current moment, to obtain the corrected heading angle, is as follows:

[0066] Define nominal state ;in, The attitude of the drone; For the speed of the drone; Location of the drone; Zero bias for the gyroscope; To achieve zero bias in the accelerometer;

[0067] Define error state ;in, Represents the three-dimensional attitude error vector, and , These represent roll error, pitch error, and heading error, respectively. Indicates speed error; Indicates positional error; This indicates the zero bias error of the gyroscope; This indicates the zero bias error of the accelerometer;

[0068] Obtain the current angular velocity measurement value and acceleration measurement values The attitude, velocity, and position in the nominal state of the previous moment are integrated forward to obtain the predicted nominal state value at the current moment.

[0069] The state transition matrix is ​​determined based on the error state dynamic equation, and the process noise covariance matrix is ​​determined based on the angular random walk, acceleration random walk, random walk noise of gyroscope zero bias, and random walk noise of accelerometer zero bias.

[0070] Using the state transition matrix and process noise covariance matrix, the error state covariance matrix is ​​predicted;

[0071] For the UAV's absolute geographic heading angle calculated at the current moment, an unscented Kalman filter update process is performed as follows:

[0072] Extract the predicted heading angle at the current moment from the nominal state prediction value at the current moment. The absolute geographic heading angle of the UAV calculated at the current moment. With predicted heading angle The difference is used as the observed new information. ;

[0073] Based on the mean error state at the current moment and the predicted error state covariance matrix, a symmetric sampling strategy is used to generate... There are Sigma points; among them. Error state dimensionality;

[0074] Propagate each Sigma point through the observation equation to obtain the mean of the predicted observations. Observation covariance and the cross-covariance between error state and observation ;

[0075] Based on the observed covariance and the cross-covariance between error state and observation Calculate the Kalman gain;

[0076] Calculate observational information Compared with the predicted observation mean The difference is used to update the error state and the predicted error state covariance matrix with the Kalman gain, so as to obtain the error state estimate and error state covariance matrix updated by observation at the current time.

[0077] The error state estimate updated by observation at the current time is injected into the nominal state prediction value at the current time to obtain the nominal state at the current time, and the corrected heading angle is extracted from the nominal state at the current time.

[0078] Set the error state to the zero vector to prepare for correction at the next time step.

[0079] On the other hand, the present invention proposes a fragmented sky polarization orientation correction system for UAV navigation, the system comprising:

[0080] The image acquisition and preprocessing module is used to acquire raw polarization images containing sky areas in real time during the flight of the UAV and perform distortion correction to obtain distortion-free polarization images.

[0081] The fragmented sky segmentation module is used to identify distortion-free polarization images using a lightweight sky segmentation model, obtain a probability distribution map of the sky region, and extract multiple effective sky fragment regions from the probability distribution map of the sky region.

[0082] The polarization information extraction module is used to extract the polarization information of each valid sky debris region based on the distortion-free polarization image covered by the valid sky debris region.

[0083] The heading angle calculation module is used to calculate the absolute geographic heading angle of the UAV at the current moment based on the polarization information of all valid sky debris regions and the fragmented polarization heading solution model.

[0084] The filtering and correction module is used to obtain the predicted heading angle at the current moment, and correct the predicted heading angle using an unscented Kalman filter method based on the absolute geographic heading angle of the UAV at the current moment, so as to obtain the corrected heading angle.

[0085] The flight control module is used to calculate the deviation between the corrected heading angle and the UAV's desired heading angle, and to control the UAV to fly along a predetermined route based on this deviation.

[0086] The beneficial effects of adopting the above technical solution are as follows:

[0087] (1) Strong environmental adaptability: The method of this invention systematically proposes a polarization orientation solution for "fragmented sky" scenarios, breaking through the bottleneck of its application in urban environments. Specifically, the method of this invention uses a lightweight sky segmentation model to identify distortion-free polarized images, obtains a probability distribution map of the sky region, and extracts several effective sky fragment regions from the probability distribution map through connected component labeling. On this basis, a fragmented polarization heading calculation model specifically for non-full sky scenarios is further constructed, and reliable fragments are screened through the RANSAC algorithm, so that the heading angle can still be stably calculated even when the sky is severely obscured by buildings, forming a fragmented scenario.

[0088] (2) Strong anti-interference capability: The method of the present invention utilizes polarized light navigation, which is not subject to electromagnetic interference, and does not rely on magnetometers that are easily affected by urban ferromagnetic materials, providing a pure absolute heading reference. Specifically, the method of the present invention acquires original polarized images with three different polarization directions (0°, 45°, and 90°) using a polarized light camera, and calculates the absolute geographic heading angle of the UAV using a polarization angle theory calculation function based on the single scattering Rayleigh model. This heading angle is calculated entirely based on the optical characteristics of the atmospheric polarization mode and is not affected by the abnormal magnetic fields of ferromagnetic materials such as high-voltage lines, metal buildings, and underground pipelines in the urban environment. Compared with the traditional magnetometer heading measurement scheme, it has higher reliability and stability.

[0089] (3) Significant correction effect: The method of the present invention directly injects the absolute heading angle as the "anchor point" into the filter, which fundamentally curbs the divergence trend of the INS heading angle and suppresses the trajectory error from "quadratic growth" to "bounded oscillation", significantly improving the long-term navigation accuracy. Specifically, the method of the present invention adopts the unscented Kalman filtering method, uses the calculated absolute geographic heading angle of the UAV as the observation value, and periodically resets and corrects the heading state estimate.

[0090] (4) Combining Real-Time Performance and Practicality: The method of this invention achieves real-time processing with limited airborne computing power through optimized image segmentation algorithms and fast analytical models, meeting the high dynamic requirements of actual UAV flight. Specifically, the method of this invention uses MobileNetV3 as the encoder and a feature pyramid structure as the decoder to construct a lightweight sky segmentation model, and significantly reduces the number of parameters and computational load through depthwise separable convolution and inverse residual structures. The entire solution process (including solar position calculation, RANSAC robust estimation, and nonlinear optimization) can be completed within 100 milliseconds on the airborne embedded computing unit, meeting the real-time navigation frequency requirements of UAVs from 1Hz to 10Hz, and supporting high-dynamic flight tasks such as urban logistics and inspection. Attached Figure Description

[0091] Figure 1 This is a flowchart of the fragmented sky polarization orientation correction method for UAV navigation in this embodiment;

[0092] Figure 2 This is a structural diagram of the lightweight sky segmentation model in this embodiment;

[0093] Figure 3 This is a schematic diagram illustrating the polarization information extraction process for the sky debris region in this embodiment;

[0094] Figure 4 This is a schematic diagram of the unscented Kalman filter update and correction process in this embodiment;

[0095] Figure 5 This is a schematic diagram illustrating the process of controlling the drone to fly along a predetermined route in this embodiment;

[0096] Figure 6 This is a structural diagram of the fragmented sky polarization orientation correction system for UAV navigation in this embodiment. Detailed Implementation

[0097] To facilitate understanding of this application, specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and embodiments. The following embodiments are illustrative of the invention but are not intended to limit its scope. Rather, these embodiments are provided to provide a more thorough and complete understanding of the disclosure of this application.

[0098] Example 1:

[0099] This embodiment uses a quadcopter drone performing urban logistics tasks as an example. During the mission, the quadcopter drone is equipped with a polarized light camera, a GNSS receiver, an onboard inertial measurement unit (IMU), an onboard computing unit, and a flight control module. The polarized light camera, GNSS receiver, and IMU are all communicatively connected to the onboard computing unit, which in turn is communicatively connected to the flight control module. The IMU includes a three-axis gyroscope and a three-axis accelerometer. The onboard computing unit has built-in functional modules including a lightweight sky segmentation model, a polarization information processing algorithm, a fragmented polarization heading solution model, and an unscented Kalman filter method. These modules are used to effectively correct the heading drift of the inertial navigation system by constructing a fragmented sky segmentation model, a polarization information solution model, and a heading angle calculation and multi-sensor fusion algorithm, thereby improving the navigation accuracy and stability of the drone in environments where global navigation satellite system signals are denied.

[0100] In this embodiment, the quadcopter drone can be the Matrice series drone manufactured by DJI, or a drone platform based on an open-source flight control architecture; the polarization camera can be an industrial camera with an integrated multi-directional polarization filter array, such as one based on Sony's IMX series polarization image sensor or other camera devices with imaging capabilities in multiple polarization directions such as 0°, 45°, 90°, and 135°; the GNSS receiver can be a high-performance receiver module supporting multiple satellites and frequencies (such as GPS, BeiDou, GLONASS, Galileo), such as one based on u-blox's NEO-M9N or ZED-F9P series or equivalent. The high-precision positioning module provides the UAV with initial position, initial heading (dual-antenna direction finding), and a final effective positioning reference before entering areas with dense high-rise buildings and severely attenuated or failed GNSS signals, such as urban canyons. The airborne inertial measurement unit can be a MEMS inertial device integrating a three-axis gyroscope and a three-axis accelerometer. The airborne computing unit can be a computing platform with embedded processing capabilities, such as an embedded computing module based on NVIDIA's Jetson series or other processing devices with equivalent computing power. The flight control module can be an open-source or commercial flight control system, such as the PX4 or ArduPilot flight control system. It should be noted that the flight platform, polarized light camera, airborne inertial measurement unit, airborne computing unit, and flight control module can all be implemented using existing mature commercial equipment. Those skilled in the art can select or substitute equivalent components according to specific application requirements. For example, in high-precision application scenarios, a high-performance inertial measurement unit and a high-computing-power airborne computing platform can be selected; in lightweight or low-cost application scenarios, a low-power embedded computing platform and lightweight sensors can be selected. This invention does not limit this.

[0101] When a drone enters an area with dense high-rise buildings and severely attenuated or failed GNSS signals, the fragmented sky polarization orientation correction method for drone navigation described in this embodiment is automatically executed, such as... Figure 1 As shown, the method includes the following steps:

[0102] During the flight of the UAV, the original polarization image containing the sky area is acquired in real time and the distortion is corrected to obtain a distortion-free polarization image.

[0103] The specific method for acquiring raw polarization images containing sky regions in real time during UAV flight and performing distortion correction to obtain distortion-free polarization images is as follows:

[0104] During the flight of the drone, the onboard polarization camera collects raw polarized images of the sky region in real time.

[0105] The original polarization image includes: original polarization component images of at least three different known polarization directions at the same time.

[0106] In this embodiment, the UAV flies along a predetermined route, and its onboard polarized light camera (i.e., a camera capable of acquiring images of light intensity in different polarization directions) continuously captures images towards the upper hemisphere sky at a preset frequency, such as 10Hz, to obtain a raw polarized image containing the sky region. The polarized light camera has at least three filters with different polarization directions, such as 0°, 45°, and 90°, for acquiring the raw polarized image.

[0107] Obtain the intrinsic parameter matrix K and distortion coefficient D of the polarization camera.

[0108] In this embodiment, the polarized light camera needs to be calibrated before use to obtain the intrinsic parameter matrix K and distortion coefficients D. The calibration can be performed using conventional camera calibration methods in the art, including but not limited to offline calibration methods based on calibration boards or online automatic calibration methods. Those skilled in the art can choose a suitable calibration method according to actual needs, and this invention does not limit this choice.

[0109] For any original polarization component image in the original polarization image, the distortion of the original polarization component image is corrected using the intrinsic parameter matrix K and the distortion coefficient D to obtain the corrected polarization component image.

[0110] The original polarization component image is resampled using the corrected polarization component image to obtain a distortion-free polarization component image. Then, all the obtained distortion-free polarization component images are used as the distortion-free polarization image.

[0111] In real-time processing, the acquired original polarization component image is first distorted using the intrinsic parameter matrix K and distortion coefficient D to eliminate the influence of lens distortion on geometric measurements. The specific process includes: mapping the pixel coordinates in the original polarization component image to the normalized imaging plane based on the camera imaging model obtained during calibration; and compensating for radial and tangential distortion using the distortion coefficient D to obtain the compensated pixel coordinates. Subsequently, the original polarization component image is resampled (e.g., using bilinear interpolation) based on the compensated pixel coordinates to generate a distortion-free polarization component image.

[0112] It should be noted that, to improve real-time processing efficiency, this embodiment can also pre-calculate the pixel mapping relationship using the intrinsic parameter matrix K and distortion coefficients D to generate a distortion correction lookup table. In real-time processing, the original polarization component image is rapidly mapped and resampled using the distortion correction lookup table, thereby achieving real-time distortion correction of the image to eliminate the influence of lens distortion on geometric measurements.

[0113] A lightweight sky segmentation model is used to identify distortion-free polarization images, obtain a probability distribution map of the sky region, and extract multiple effective sky fragment regions from the probability distribution map of the sky region.

[0114] The lightweight sky segmentation model consists of an encoder and a decoder.

[0115] The encoder uses MobileNetV3 to extract features from the input image layer by layer, resulting in a multi-scale feature map.

[0116] In this embodiment, as Figure 2 As shown, a lightweight sky segmentation model is built based on the MobileNetV3 architecture and specifically optimized for training on images with complex urban backgrounds such as broken skies, glass curtain walls, and gray buildings. This model can quickly and accurately segment all pixel regions belonging to the "sky" in the input image. Specifically, MobileNetV3 is used as the encoder. Through MobileNetV3's depthwise separable convolutional layers and inverse residual structure, semantic features of the input image are extracted layer by layer, and feature maps at multiple scales are output, such as... Figure 2 As shown, this embodiment outputs four feature maps at different scales, from shallow (edges, textures) to deep (global semantics), representing the features respectively. Figure 4 ,feature Figure 3 ,feature Figure 2 and characteristics Figure 1 .

[0117] The decoder is implemented using a feature pyramid structure, which is used to fuse multi-scale feature maps step by step to obtain the final feature map, and then maps the final feature map to a probability distribution map of the sky region.

[0118] The decoder performs step-by-step fusion of multi-scale feature maps to obtain the final feature map. The specific process of mapping the final feature map to a probability distribution map of the sky region is as follows: the multi-scale feature maps are arranged in order of increasing resolution, denoted as... ;in Represents multi-scale feature maps; The number of feature maps representing a multi-scale feature map; This represents the first-level feature map; This represents the second-level feature map; Indicates the first Level feature map.

[0119] The lowest resolution level 1 feature map is obtained through bilinear upsampling. Restore to the previous level feature map After achieving the same resolution, then compare with the feature map. Channel splicing is performed to obtain the first fused feature map. .

[0120] From the first fusion feature map To begin, repeat the following process:

[0121] The first... Fusion Feature Map Restore to feature map After achieving the same resolution, then compare with the feature map. Perform channel splicing to obtain the first... Fusion Feature Map ;in .

[0122] Until the first Fusion Feature Map Until then, and will the Fusion Feature Map As the final feature map.

[0123] In this embodiment, a decoder branch is added to the MobileNetV3 architecture, implemented using a feature pyramid structure. Feature maps of different scales output from the encoder are bilinearly upsampled to the same resolution and then concatenated. This is achieved through skip connections, directly concatenating shallow features (containing rich edge and texture information) from the encoder with the upsampled deep features from the decoder to better recover the boundaries of small-scale sky fragments. The shallow features are preferably the outputs of intermediate layers in the encoder with higher spatial resolution. In this embodiment, for example... Figure 2 As shown, Features are obtained through bilinear upsampling Figure 1 Restore to features Figure 2 After achieving the same resolution, then compare with features Figure 2 Channel splicing is performed to obtain the first fused feature map. The first fused feature map is obtained by bilinear upsampling. Restore to features Figure 3 After achieving the same resolution, then compare with features Figure 3 Channel splicing is performed to obtain the second fused feature map. The second fused feature map is obtained through bilinear upsampling. Restore to features Figure 4 After achieving the same resolution, then compare with features Figure 4 Channel splicing is performed to obtain the third fused feature map. ; the third fusion feature map As the final feature map, it retains rich edge and texture information, thereby improving the ability to recover the boundaries of small-scale sky debris.

[0124] After mapping the final feature map to a single-channel feature map using two-dimensional convolution, the probability distribution map of the sky region is obtained by passing the Sigmoid activation function; the value of each pixel in the probability distribution map of the sky region is the probability that the pixel belongs to the sky category, and the value ranges from 0 to 1.

[0125] In this embodiment, as Figure 2 As shown, the final feature map obtained in the decoder after stepwise upsampling and fusing multi-scale features through skip connections is processed. A 1×1 convolutional layer is used to map the final feature map into a single-channel output, and a probability distribution map of the sky region is obtained through a Sigmoid activation function. The probability distribution map is output in the form of a binary mask, with each pixel value ranging from 0 to 1. Due to occlusion by obstacles such as buildings and bridges, the segmented sky region usually presents multiple disconnected, small, and irregularly shaped "fragments".

[0126] It should be noted that the lightweight sky segmentation model used in this embodiment was trained as follows: During training, the training set used was a dataset consisting of polarization images and their pixel-level annotations. A weighted binary cross-entropy loss function was used to optimize the lightweight sky segmentation model. This loss function addresses the class imbalance problem caused by the small proportion of sky fragment pixels by assigning higher weights to sky category pixels. The trained lightweight sky segmentation model can quickly and accurately identify the probability of each pixel in the image belonging to the sky category.

[0127] The specific method for identifying distortion-free polarization images using a lightweight sky segmentation model, obtaining a probability distribution map of the sky region, and extracting multiple effective sky debris regions from the probability distribution map of the sky region is as follows:

[0128] By inputting any distortion-free polarization component image from the distortion-free polarization image into the lightweight sky segmentation model, a probability distribution map of the sky region is obtained.

[0129] In this embodiment, the distortion-free polarization component image corresponding to the 0° polarization direction is input into the lightweight sky segmentation model to obtain the probability distribution map of the sky region.

[0130] The probability distribution map of the sky region is binarized according to a preset threshold to obtain a sky segmentation mask; each pixel in the sky segmentation mask is marked as either sky or non-sky.

[0131] In this embodiment, pixels with a probability greater than 0.5 are marked as sky, and pixels with a probability less than 0.5 are marked as non-sky, thus obtaining a sky segmentation mask; in the sky segmentation mask, a pixel value of 1 represents sky, and a pixel value of 0 represents non-sky.

[0132] The sky segmentation mask is labeled with connected components, and all adjacent sky pixels are labeled as the same connected region, and each connected region is treated as a sky fragment region.

[0133] For each sky fragment region, calculate the pixel area of ​​that sky fragment region. If the calculated pixel area is less than a preset area threshold, then the sky fragment region is considered an invalid region and is removed.

[0134] A sky fragment region whose area per pixel is not less than a preset area threshold is considered a valid sky fragment region.

[0135] In this embodiment, in the sky segmentation mask, sets of pixels with a pixel value of 1 that are adjacent to each other are marked as the same connected region, and each connected region corresponds to a sky fragment region. The extracted sky fragment regions are then filtered for validity: the pixel area of ​​each sky fragment region is calculated, and if the calculated pixel area is less than a preset area threshold, the sky fragment region is considered an invalid region formed by noise or imaging defects and is discarded. Statistical analysis of polarized images acquired in typical urban canyon scenes shows that the pixel area of ​​noise regions is generally less than 100 pixels; therefore, this embodiment sets the area threshold to 100 pixels.

[0136] For each valid sky debris region, the polarization information of that valid sky debris region is extracted based on the distortion-free polarization image it covers.

[0137] The specific method for extracting the polarization information of each valid sky debris region based on the distortion-free polarization image covered by that valid sky debris region is as follows:

[0138] The polarization information includes: representative polarization angle, average degree of polarization, and observation direction vector.

[0139] For each segmented valid sky debris region, its polarization information is extracted, including: the representative polarization angle (AoP), the confidence index of the degree of polarization (DoP), and the observation direction vector of the sky debris region. In this embodiment, the average degree of polarization is used as an optional confidence index of the degree of polarization.

[0140] For any valid sky debris region According to the effective sky debris area Calculate the Stokes parameter for each pixel covering the undistorted polarization component image in the undistorted polarization image.

[0141] In this embodiment, as Figure 3As shown, for an effective sky debris region, the polarization information of that region is calculated based on the pixels it covers in the distortion-free polarized image. Specifically, using the light intensity images (i.e., polarization component images) acquired by the polarization camera at 0°, 45°, and 90°, the Stokes parameter of each pixel is calculated. , is represented as:

[0142] ;

[0143] ;

[0144] ;

[0145] in, Total light intensity; and All are linearly polarized components; These represent the light intensity values ​​of the polarization component images of pixels in the four polarization directions of 0°, 45°, 90°, and 135°, respectively. For the polarization component images of the three polarization directions of 0°, 45°, and 90° acquired in this embodiment, in order to obtain more stable observation values ​​sensitive to Rayleigh scattering skylight, when the polarization camera is a multi-band imaging device, it is preferable to select the light intensity data of the blue light band (e.g., a center wavelength of approximately 450 nm and a bandwidth of approximately 20 nm). The blue light band is selected for calculation because Rayleigh scattering intensity is inversely proportional to the fourth power of the incident light wavelength. Shorter wavelength light has a stronger scattering effect and higher polarization information contrast, which is beneficial to improving the stability and observability of sky polarization characteristics. When the polarization camera is a single-band imaging device, it is preferable to use short-band imaging data sensitive to Rayleigh scattering as the light intensity data.

[0146] It should be noted that, It is possible Estimate or use the light intensity value in the known polarization direction, and calculate the light intensity value in the 135° polarization direction by interpolation using a polarization model.

[0147] Calculate the polarization angle of each pixel based on the Stokes parameters. With polarization degree .

[0148] ;

[0149] ;

[0150] in, This represents the square root operation.

[0151] Take valid sky debris area Polarization angle of all pixels The median value is used as the effective sky debris region. The representative polarization angle .

[0152] Calculate the effective sky debris region polarization degree of all pixels The average value is used as the effective sky debris area. average polarization .

[0153] In this embodiment, considering that the sky debris region may contain interference from edge-mixed pixels, thin clouds, or sensor noise, it is necessary to adjust the polarization parameters (i.e., polarization angles) of the pixels within the region. With polarization degree Statistical integration is performed to obtain a robust regional characteristic. Specifically, to avoid the influence of outliers, valid sky debris regions are selected. Polarization angle of all pixels (Example: Effective sky debris region) The polarization angles of the individual pixels are 15°, 16°, 14°, 30°, and 15°, with 30° being a clear outlier. The median value of these values ​​is used as the effective sky debris region. The representative polarization angle Calculate the effective sky debris region. The average polarization degree (DoP) of all pixels within the region is used as the effective sky debris region. average polarization The average polarization degree can then be used as the effective sky debris region. Weighting indicators for the reliability of observations.

[0154] Calculate the effective sky debris region using any distortion-free polarization component image from a distortion-free polarization image. The average coordinates of all pixels within the area are used as the effective sky fragment region. centroid coordinates ;in Indicates the effective sky debris area The horizontal coordinate of the centroid in the pixel coordinate system; Indicates the effective sky debris area The ordinate of the centroid in the pixel coordinate system.

[0155] In this embodiment, the top left corner of the distortion-free polarization component image is taken as the origin of the pixel coordinate system. The axis runs horizontally to the right along the image. The axis runs vertically downwards along the image.

[0156] The effective sky debris region is determined using the intrinsic parameter matrix K. centroid coordinates By back-projecting onto the camera coordinate system, the effective sky debris region can be obtained. Observation direction vector .

[0157] In this embodiment, it should be noted that before performing back projection using the pre-calibrated intrinsic parameter matrix, it is essential to ensure that the input centroid coordinates are accurate. This point was obtained from an image that had already undergone lens distortion correction. Using the intrinsic parameter matrix K, this point was back-projected onto the normalized imaging plane in the camera coordinate system to obtain the effective sky debris region. Observation direction vector The camera coordinate system is a three-dimensional rectangular coordinate system established with the camera's optical center as the origin. In the camera coordinate system, The axis is along the camera's optical axis and usually points in the direction of the subject. shaft and The axes are parallel to the horizontal and vertical axes of the image plane, respectively. The normalized image plane is an imaginary plane located in the camera coordinate system. A two-dimensional plane on a plane is used to establish the ideal projection relationship between three-dimensional spatial points and image pixels (without considering lens distortion).

[0158] The specific process of the back projection is as follows:

[0159] ;

[0160] ;

[0161] ;

[0162] ;

[0163] in, Represents the horizontal coordinates on the normalized imaging plane; This represents the horizontal coordinate of the intersection point of the camera's optical axis and the normalized imaging plane. This represents the horizontal equivalent focal length in pixels, and , Indicates the physical focal length; This indicates the pixel size of a single pixel in the horizontal direction; Represents the vertical coordinates on the normalized imaging plane; This represents the vertical coordinate of the intersection point of the camera's optical axis and the normalized imaging plane. This represents the equivalent focal length in the vertical direction, expressed in pixels. , This indicates the cell size of a single pixel in the vertical direction; This represents the optical axis direction coordinates on the normalized imaging plane, with a constant value of 1. , This represents the square root operation.

[0164] Observation direction vector It represents the actual sky direction from the camera's optical center to the center of the sky debris region, and is a key geometric input connecting image observation and the physical model of sky polarization.

[0165] Effective Sky Fragmentation Area The polarization information is output as a data structure containing polarization and geometric features, such as a triplet. Wherein, represents the polarization angle. and observation direction vector It is a necessary input for subsequent heading calculations; average degree of polarization As a quality weighting coefficient.

[0166] Based on the polarization information of all valid sky debris regions, the absolute geographic heading angle of the UAV at the current moment is calculated using the fragmented polarization heading solution model.

[0167] In this embodiment, the polarization information of multiple effective sky debris regions is extracted and input into a mathematical analytical model specifically designed for fragmented sky scene optimization, namely the fragmented polarization heading calculation model, to calculate the absolute geographic heading angle of the UAV at the current moment. The core of this model is to establish a geometric-physical relationship between the observed values ​​and the UAV's heading based on Rayleigh scattering theory, and obtain the absolute geographic heading angle by solving an optimization problem. Unlike traditional methods that rely on complete and symmetrical sky polarization patterns, this model abandons the dependence on sky integrity, utilizing only scattered and irregular sky debris information. It constructs a set of constraint equations relating the fragmented polarization angle observations to the relative geometric relationship between the sun and the carrier, and employs a robust estimation algorithm to quickly calculate the absolute geographic heading angle of the UAV at the current moment. This model has stronger adaptability to the shape, number, and arbitrary distribution of debris regions.

[0168] The specific method for calculating the absolute geographic heading angle of the UAV at the current moment using the fragmented polarization heading solution model based on the polarization information of all valid sky debris regions is as follows:

[0169] Obtain the current UTC timestamp and the latitude and longitude of the UAV, and use the Solar Position Algorithm (SPA) to calculate the solar altitude angle at the current moment based on the UTC timestamp and the UAV's latitude and longitude. and solar azimuth This leads to the construction of solar position parameters. .

[0170] In this embodiment, the current UTC timestamp and the latitude and longitude of the UAV can be obtained through the GNSS receiver on the UAV. Preferably, before the UAV enters the urban canyon environment, the last valid GNSS positioning result is obtained, and this positioning result must meet a preset signal quality threshold. The signal quality threshold can be determined by one or more of the following indicators: positioning status, number of visible satellites, position accuracy factor, and positioning result stability; the last valid positioning result that meets the above threshold conditions is used as the current UTC timestamp and the latitude and longitude of the UAV.

[0171] Then, using the Solar Position Algorithm (SPA), the altitude angle of the sun at this moment is calculated. and geographical azimuth These two angles constitute the solar position parameters. It should be noted that the solar position algorithm used in this embodiment is an existing standard algorithm, and those skilled in the art know how it is implemented. Therefore, the specific formula will not be described in detail in this invention.

[0172] Based on the polarization information of all valid sky debris regions and the solar position parameters, a nonlinear least squares optimization problem is constructed and used as a solution model for fragmented polarization heading.

[0173] ;

[0174] in, The absolute geographic heading angle of the UAV is used as the value to be solved. Indicates to Find the minimum value; The number of effective sky debris areas; This represents the function for calculating the polarization angle based on the Rayleigh model of single scattering.

[0175] The polarization angle calculation function based on the single-scattering Rayleigh model is used to calculate the theoretical polarization angle of the effective sky debris region based on the UAV's absolute geographic heading angle, solar position parameters, and the observation direction vector of the effective sky debris region.

[0176] In this embodiment, the physical basis of the polarization angle calculation function based on the Rayleigh single scattering model is Rayleigh's single scattering law, which states that the polarization direction of the scattered light is perpendicular to the plane determined by the sun's direction, the observer (drone), and the sky scattering point.

[0177] The calculation process of the polarization angle calculation function based on the single-scattering Rayleigh model is as follows:

[0178] For any valid sky debris region By utilizing the pre-calibrated extrinsic rotation matrix of the polarization camera relative to the aircraft coordinate system, the effective sky debris region is captured. Observation direction vector Transform to the body coordinate system.

[0179] In this embodiment, the body coordinate system is a three-dimensional Cartesian coordinate system fixed to the UAV body. Typically, the UAV's center of gravity is used as the origin, and the direction pointing towards the nose is used as the body coordinate system. The axis direction is used as the coordinate system for the aircraft, pointing towards the right side of the fuselage. The axis direction is used as the coordinate system for the aircraft, pointing downwards from the fuselage. The axis direction satisfies the right-hand rule. The extrinsic rotation matrix of the polarization camera relative to the body coordinate system can be obtained through the same calibration process as obtaining the intrinsic parameter matrix K and distortion coefficient D, or it can be obtained by using an independent hand-eye calibration method or by directly calculating using precise installation dimensions. This extrinsic rotation matrix is ​​used to transform the observation direction vector from the camera coordinate system to the body coordinate system;

[0180] Get the current drone roll angle and drone pitch angle .

[0181] Utilizing the current drone roll angle UAV pitch angle and the absolute geographical heading angle of the drone Construct a rotation matrix from the body coordinate system to the geographic coordinate system. .

[0182] In this embodiment, the geographic coordinate system is a three-dimensional Cartesian coordinate system used to describe the absolute direction and position of an object on Earth. This embodiment adopts the Northeast-Northeast coordinate system (ENU), defining the heading angle as 0° northward, increasing clockwise, i.e., 90° eastward, 180° southward, and 270° westward, conforming to navigation conventions. The rotation matrix from the body coordinate system to the geographic coordinate system is constructed from the UAV's three attitude angles: roll, pitch, and heading. The construction process is as follows: first, rotate the heading angle around the Z-axis (sky axis) of the geographic coordinate system; then, rotate the pitch angle around the Y-axis (north axis) of the geographic coordinate system; finally, rotate the roll angle around the X-axis (east axis) of the geographic coordinate system. The current UAV roll angle is... and drone pitch angle Measured by the onboard inertial measurement unit (IMU); the heading angle is expressed as the absolute geographic heading angle of the UAV to be solved.

[0183] Using the rotation matrix Effective sky debris area Observation direction vector Transform to the geographic coordinate system to obtain the observation direction vector in the geographic coordinate system. .

[0184] Based on the solar position parameters, construct the solar direction vector in the geographic coordinate system. .

[0185] In this embodiment, the standard celestial coordinate transformation method is used to convert the solar altitude angle. and solar azimuth Convert to a unit vector pointing towards the sun in a geographic coordinate system, i.e., the sun's direction vector in a geographic coordinate system. .

[0186] Calculate the solar direction vector in the geographic coordinate system. Observation direction vector in geographic coordinate system The vector product yields the effective sky debris region. Theoretical polarization direction vector in geographic coordinate system .

[0187] In this embodiment, according to Rayleigh's law of single scattering, since the polarization direction of the scattered light is perpendicular to the plane determined by the solar direction and the observation direction, for the effective sky debris region... , will along The theoretical polarization direction vector corresponding to the atmospheric scattering point in the direction is used as the effective sky debris region. Theoretical three-dimensional polarization direction vector , is represented as:

[0188] ;

[0189] in, This represents the vector product operation.

[0190] Effective Sky Fragmentation Area Theoretical polarization direction vector in geographic coordinate system Transform to camera coordinate system to obtain effective sky debris region Theoretical polarization direction vector in camera coordinate system .

[0191] According to the effective sky debris area Theoretical polarization direction vector in camera coordinate system Calculate the effective sky debris region Theoretical polarization angle .

[0192] In this embodiment, Transform back to the camera coordinate system to obtain the theoretical polarization direction vector in the camera coordinate system. And then calculate The projection orientation angle on the normalized imaging plane (XY plane) in the camera coordinate system is used as the effective sky debris region. Theoretical polarization angle , is represented as:

[0193] ;

[0194] in, and They represent the theoretical polarization direction vectors, respectively. The components of the Y-axis and X-axis in the normalized imaging plane in the camera coordinate system; the obtained theoretical polarization angle. That is The output value.

[0195] Because of the asymmetrical distribution of sky debris and the potential inclusion of outliers (outsiders) from non-sky regions (such as glass reflections), directly using the standard least squares method would lead to solution failure. Therefore, this embodiment employs a robust process combining the Random Sample Consensus (RANSAC) algorithm with nonlinear least squares optimization for initial estimation and precise optimization of the absolute geographic heading angle.

[0196] Based on the fragmented polarization heading solution model, the Random Sample Consensus (RANSAC) algorithm is used to generate the RANSAC in-point set and the initial heading estimate of the UAV.

[0197] The specific method for generating the RANSAC in-point set and the initial heading estimate of the UAV based on the fragmented polarization heading solution model and using the Random Sample Consensus (RANSAC) algorithm is as follows:

[0198] Set the maximum number of iterations, start the iteration, and perform the following operations during each iteration:

[0199] In this embodiment, the maximum number of iterations is set to 200.

[0200] Initialize an empty set of hypothetical interior points.

[0201] The representative polarization angle of each valid sky debris region and observation direction vector As a sample, from The smallest sample set is randomly selected from the samples; the number of samples in the smallest sample set is... .

[0202] In this embodiment, the number of samples in the minimum sample set It is the minimum number of observation equations required to generate a candidate heading hypothesis, and The principle for determining the value is: the absolute geographic heading angle to be solved is... For a parameter with one degree of freedom, according to parameter estimation theory, at least one effective observation equation is needed to constrain this parameter, i.e. However, to combat observation noise, satisfy RANSAC model validation logic, and ensure the sufficiency of geometric constraints, a single observation (i.e., ) insufficient, two observations (i.e. The system constructed from these components is relatively fragile. Therefore, this embodiment selects... As the minimum sample size, this value achieves the optimal balance between computational efficiency and solution robustness, and is the smallest practical unit capable of using fragmented sky information for stable heading calculation.

[0203] By using the minimum sample set to reverse-engineer the fragmented polarization heading solution model, the assumed heading angle values ​​corresponding to the minimum sample set can be obtained. .

[0204] In this embodiment, based on a randomly selected minimum sample set, a heading angle hypothesis is obtained by jointly reversing the fragmented polarization heading solution model using three samples. Specifically, the representative polarization angles and observation direction vectors of the three samples are substituted into a nonlinear least squares optimization problem, i.e., substituted into... Find the heading angle that minimizes the sum of squared residuals, and use it as the assumed heading angle value for the current smallest sample set. . This is not the final solution, but rather a hypothetical heading angle to be verified. Subsequently, by comparing it with the residuals representing all polarization angles, the RANSAC algorithm will select the heading angle hypothesis that supports the most observation points as the final initial heading estimate.

[0205] for For each of the samples, based on the assumed heading angle value Given the solar position parameters and the observed direction vector of the sample, the theoretical polarization angle of the sample is calculated using a polarization angle calculation function based on the Rayleigh model of single scattering. Then, the theoretical polarization angle and the representative polarization angle of the sample are calculated. The residual.

[0206] All samples whose residuals are less than a preset residual threshold are included in the hypothesis interior set of the current iteration.

[0207] In this embodiment, based on the assumed value of the heading angle Consistency checks are performed. Statistical analysis is conducted on multiple sets of data collected from typical urban canyon scenarios, with a residual threshold set to 5° to ensure a good balance between accuracy and recall in inlier selection. If the residual for a sample is less than 5°, that sample is included in the hypothetical inlier set for the current iteration. The hypothetical inlier set is defined as the heading hypothesis generated for the smallest randomly sampled set in the RANSAC algorithm. The set of all samples that satisfy the condition that the deviation between the observed value (i.e., the representative polarization angle) and the theoretical value (i.e., the theoretical polarization angle) is less than a preset threshold.

[0208] When the maximum number of iterations is reached, the iteration ends. By comparing the number of samples contained in the hypothetical interior point sets of all iteration rounds, the hypothetical interior point set with the largest number of samples is selected as the RANSAC interior point set, and the heading angle hypothesis value corresponding to this hypothetical interior point set is set. As an initial estimate of the drone's heading .

[0209] In this embodiment, after repeating the iteration process 200 times, the hypothetical interior point set containing the largest number of samples is selected as the RANSAC interior point set, and the heading angle hypothesis value corresponding to this hypothetical interior point set is set. As an initial estimate of the drone's heading The above process effectively filters out outliers, resulting in an initial heading estimate supported by reliable inliers. .

[0210] Using the initial estimate of the UAV's heading as the initial value, a nonlinear optimization algorithm is employed based on the RANSAC interior point set to solve the fragmented polarization heading calculation model, thereby obtaining the UAV's absolute geographic heading angle. .

[0211] In this embodiment, based on the RANSAC in-point set and the initial heading estimate of the UAV, an accurate solution for the fragmented polarization heading calculation model is achieved through nonlinear optimization. Specifically, the process involves using the initial heading estimate of the UAV... Using the initial values, the nonlinear least squares optimization problem is reconstructed and solved using the RANSAC interior point set, as follows:

[0212] ;

[0213] in, Let be the number of samples contained in the RANSAC inset, and ; For the first Each sample represents a polarization angle; For the first The observation direction vector corresponding to each sample.

[0214] The nonlinear optimization employs an efficient nonlinear optimization algorithm (such as the existing Levenberg-Marquardt algorithm) iteratively until convergence, obtaining the final high-precision absolute heading angle solution, i.e., the absolute geographic heading angle of the UAV. .

[0215] It should be noted that the above calculation process (including solar position calculation, RANSAC robust estimation, and nonlinear optimization) can be completed within 100 milliseconds on the onboard computing unit (such as an embedded GPU or high-performance microprocessor) mounted on the UAV, meeting the frequency requirements for real-time navigation of the UAV (typically 1-10Hz). Its final output... The accuracy can reach 0.5°~1.5° under typical clear weather conditions, providing highly reliable absolute heading observations for subsequent sensor fusion.

[0216] The predicted heading angle at the current moment is obtained, and the predicted heading angle is corrected by using an unscented Kalman filter method based on the absolute geographic heading angle of the UAV at the current moment, so as to obtain the corrected heading angle.

[0217] In this embodiment, the absolute geographic heading angle of the UAV is a direct observation obtained through periodic calculation, characterized by high precision and low frequency. Its calculation period is determined by the acquisition frequency of the polarized light camera and the actual processing speed of the heading calculation process, and is approximately 0.5 to 1 second.

[0218] The calculated absolute geographic heading angle of the UAV is input into a multi-sensor fusion filter centered on the onboard inertial measurement unit (IMU). This multi-sensor fusion filter handles the nonlinear characteristics of the inertial navigation system and can be a variant of the nonlinear Kalman filter, such as the Error-State Extended Kalman Filter (ES-EKF) or the Unscented Kalman Filter (UKF). The absolute heading information provided by polarized light can periodically and powerfully reset and correct the heading angle estimate in the filter that rapidly diverges due to the accumulation of IMU errors. This significantly suppresses the exponential growth of heading errors caused by the accumulation of IMU errors.

[0219] To further enhance the adaptability to the nonlinear characteristics of inertial navigation systems (especially the nonlinear characteristics of UAV maneuvering and polarization observation models), this embodiment employs an unscented Kalman filter method for heading angle correction. This method performs intuitive IMU mechanical integration in the nominal state space, while simultaneously performing Kalman filter estimation and updating in the error state space, effectively separating high-frequency dynamics from low-frequency error estimation. It is particularly suitable for handling high-speed integration problems of IMU data. Details are as follows:

[0220] The specific method for obtaining the predicted heading angle at the current moment, and correcting the predicted heading angle using an unscented Kalman filter based on the absolute geographic heading angle of the UAV at the current moment, to obtain the corrected heading angle is as follows:

[0221] Define nominal state ;in, The attitude of the drone; For the speed of the drone; Location of the drone; Zero bias for the gyroscope; This is for zero bias of the accelerometer.

[0222] In this embodiment, nominal state This is used for direct physical integration. The attitude, velocity, and position of the UAV can all be obtained by inertial navigation mechanized integration of the angular velocity and acceleration output by the onboard inertial measurement unit. The inertial navigation mechanized integration is the process of converting measurements from inertial sensors (such as accelerometers and gyroscopes) into velocity, position, and attitude through integration operations in the inertial navigation system. Specifically, the acceleration value measured by the accelerometer is integrated once to obtain the velocity, and then integrated again to obtain the position; the angular velocity value measured by the gyroscope is integrated once to obtain the angular increment (attitude change), the angular increment is converted into attitude, and then the incremental quaternion is multiplied by the attitude quaternion from the previous moment using quaternion multiplication to obtain the attitude quaternion at the current moment, thus determining the UAV's attitude. The gyroscope zero bias and accelerometer zero bias are used as error states, which are estimated online during the filtering process. That is, in each observation update stage, the filter recursively corrects the error states based on the observation residuals using Kalman gain, thereby achieving dynamic estimation and compensation of the zero bias.

[0223] Define error state ;in, Represents the three-dimensional attitude error vector, and , These represent roll error, pitch error, and heading error, respectively. Indicates speed error; Indicates positional error; This indicates the zero bias error of the gyroscope; This indicates the zero bias error of the accelerometer.

[0224] In this embodiment, the error state This is used to describe the deviation between the nominal state and the true state, and is estimated and corrected during the filtering process. It should be noted that, in this embodiment, the nominal state... The attitude is represented by attitude quaternions, while the error state is represented by attitude quaternions. The attitude is represented using Euler angles (small angle approximation), which is the standard practice for error-state Kalman filtering. Under the small angle approximation, the conversion relationship between quaternions and Euler angle errors is as follows:

[0225] ;

[0226] in, This is a quaternion estimate, obtained recursively from the integral of the IMU angular velocity; This is quaternion multiplication.

[0227] Obtain the current angular velocity measurement value and acceleration measurement values The nominal state prediction value for the current moment is obtained by forward integration of the attitude, velocity and position in the nominal state of the previous moment.

[0228] In this embodiment, the onboard inertial measurement unit carried by the UAV is used to collect the angular velocity measurement value at the current moment at a high frequency (e.g., 200Hz). and acceleration measurement values The nominal state at the previous time step is then integrated forward to obtain the predicted nominal state at the current time step. The nominal state at the current time step is obtained through initialization at the initial time step, and in subsequent steps, it is derived from the nominal state obtained after updating it with an unscented Kalman filter from the previous time step.

[0229] The specific process of the forward integration includes:

[0230] Posture update:

[0231] ;

[0232] in, express The drone's posture at all times ; express The drone's attitude at all times; This is the quaternion corresponding to the angle increment, used to convert the angle increment vector into a quaternion.

[0233] Speed ​​updates:

[0234] ;

[0235] in, express The speed of the drone at all times; express The speed of the drone at all times; The attitude matrix; It is the gravity vector; This indicates the sampling interval of the IMU.

[0236] Location update:

[0237] ;

[0238] in, express Real-time drone location; express The location of the drone at all times.

[0239] Using the updated attitude, velocity, and position, the nominal state prediction value for the current moment is obtained.

[0240] The state transition matrix is ​​determined based on the error state dynamic equation, and the process noise covariance matrix is ​​determined based on the angular random walk, acceleration random walk, random walk noise of gyroscope zero bias, and random walk noise of accelerometer zero bias.

[0241] In this embodiment, the state transition matrix Used to describe error status from Time's up A linear recurrence relation at each time step. Preferably, the state transition matrix... The error state dynamic equation is obtained by discretizing the continuous-time error state dynamic equation (e.g., using a first-order approximation or exact discretization method). The error state dynamic equation adopts a standard form known in the field of inertial navigation.

[0242] Process noise covariance matrix This is used to describe the uncertainty introduced by the inherent noise of the IMU during the prediction process. In this embodiment, the process noise covariance matrix... The parameters are constructed based on the following four types of IMU noise parameters: angular random walk. and acceleration random walk This describes the noise characteristics of an inertial measurement unit; random walk noise of a gyroscope with zero bias. Random walk noise of accelerometer zero bias : Used to describe the noise driving term that slowly changes over time for gyroscope zero bias and accelerometer zero bias, belonging to the excitation noise of random walk process.

[0243] In this embodiment, the angular random walk, acceleration random walk, gyroscope zero-bias random walk noise, and accelerometer zero-bias random walk noise are all known parameters, which can be obtained by directly consulting the datasheets provided by the manufacturers.

[0244] Using the state transition matrix and the process noise covariance matrix, the error state covariance matrix is ​​predicted.

[0245] ;

[0246] in, Indicates the predicted Error state covariance matrix at time t; for The error state covariance matrix updated after each observation.

[0247] In this embodiment, when a new polarized light absolute heading angle observation value is reached (i.e., the absolute geographic heading angle of the UAV at the current moment is obtained), When the arrival frequency is typically 1Hz-10Hz, the Unscented Kalman Filter (UKF) update process is executed.

[0248] For the current absolute geographic heading angle of the UAV, the unscented Kalman filter update process is performed as follows:

[0249] Extract the predicted heading angle at the current moment from the nominal state prediction value at the current moment. The absolute geographical heading angle of the drone at the current moment. With predicted heading angle The difference is used as the observed new information. .

[0250] In this embodiment, as Figure 4 As shown, the predicted heading angle at the current moment is the heading component directly extracted from the nominal state prediction value at the current moment; this nominal state prediction value is recursively obtained by combining the nominal state at the previous moment with the IMU angular velocity integral (i.e., the forward integration process), without introducing external observation corrections at the current moment. The observation information is linearly related to the error state, that is:

[0251] ;

[0252] in, Denotes the observation matrix, and the observation matrix It is only in the corresponding error state Heading error A row vector where the element at position 1 is 1 and all other elements are 0 is represented as:

[0253] ;

[0254] in, Indicates error status Dimensions.

[0255] This represents observation noise; in this embodiment, we assume observation noise... Zero-mean Gaussian white noise, covariance matrix The accuracy is determined based on the course calculation accuracy. For example, if the standard deviation of the calculation accuracy is... =1.0°, then .

[0256] Based on the mean error state at the current moment and the predicted error state covariance matrix, a symmetric sampling strategy is used to generate... There are Sigma points; among them. Error state The dimension of.

[0257] In this embodiment, since the error state is updated after each unscented Kalman filter update, Reset to a zero vector, indicating The deviation represented has been compensated, so the mean of the error state at the current moment is usually also a zero vector. Assume the error state... The dimension is Generate using a symmetric sampling strategy There are Sigma points, which are a set of samples of possible values ​​for the current error state. Each Sigma point represents a possible error state hypothesis. The specific process is as follows: Determine a scaling factor (such as using a standard recommended value) to control the spread of the sampling points; take the mean of the predicted error state at the current time as the first Sigma point; perform square root decomposition on the predicted error state covariance matrix to obtain its Cholesky decomposition matrix, and multiply this decomposition matrix by the square root of the scaling factor to obtain a set of step vectors. Then, based on the first Sigma point, add and subtract each step vector respectively to generate... A total of symmetrical Sigma points were obtained. Sigma points.

[0258] Propagate each Sigma point through the observation equation to obtain the mean of the predicted observations. Observation covariance and the cross-covariance between error state and observation .

[0259] In this embodiment, after completing the Sigma point sampling, each Sigma point needs to be propagated through the observation equation to calculate three key statistics: the mean of the predicted observations, the observation covariance, and the cross-covariance between the error state and the observations. Specifically, each Sigma point is propagated through the observation equation... To spread, To predict the observations, the observation equation describes the mathematical relationship between the error state and the observations. For each input Sigma point, the observation equation outputs a corresponding predicted observation. After propagation at all Sigma points, a set of predicted observations is obtained, and a weighted average is taken to obtain the mean of the predicted observations. Simultaneously, the observed covariance is calculated. and the cross-covariance between error state and observation Among them, the observation covariance describes the dispersion of a set of predicted observations: if the predicted observations propagated from each Sigma point are close to each other, the observation covariance is small. The cross-covariance between the error state and the observations describes the correlation between the possible values ​​of the error state and the corresponding predicted observations. These three statistics (the mean of the predicted observations, the observation covariance, and the cross-covariance between the error state and the observations) together constitute the core of the unscented Kalman filter update.

[0260] Based on the observed covariance and the cross-covariance between error state and observation Calculate the Kalman gain.

[0261] ;

[0262] in, This is the Kalman gain.

[0263] Calculate observational information Compared with the predicted observation mean The difference is used to update the error state and the predicted error state covariance matrix with the Kalman gain, so as to obtain the error state estimate and error state covariance matrix after observation update at the current time.

[0264] The error state update process is as follows:

[0265] ;

[0266] ;

[0267] in, for Error state estimate updated after each observation; for The error state covariance matrix updated after each observation.

[0268] The error state estimate updated by observations at the current time is injected into the nominal state prediction value at the current time to obtain the nominal state at the current time, and the corrected heading angle is extracted from the nominal state at the current time.

[0269] In this embodiment, the error state estimate updated by observations at the current moment is injected into the nominal state prediction value at the current moment to correct its deviation. The specific injection method is as follows: For attitude, the three-dimensional attitude error vector... Convert to modified quaternion Then multiply by the pose in the nominal state: , This indicates the attitude of the drone after injection; This is an assignment operation. For speed, the speed error is directly added to the speed in the nominal state: , This represents the speed of the drone after injection; for position, the position error is directly added to the position in the nominal state. , This indicates the position of the drone after injection; for gyroscope bias, the gyroscope bias error is directly added to the gyroscope bias in the nominal state: , This indicates the zero bias of the gyroscope after injection; for the zero bias of the accelerometer, the accelerometer zero bias error is directly added to the accelerometer zero bias in the nominal state: , This indicates that the accelerometer has zero bias after injection.

[0270] After error injection, based on the injected UAV attitude, the attitude quaternion is decomposed into three Euler angle components using the standard mathematical transformation relationship between quaternions and Euler angles, corresponding to roll, pitch, and yaw angles, respectively. The yaw angle component is the corrected yaw angle.

[0271] The error state is set to a zero vector to prepare for the correction at the next time step, i.e., to provide the mean of the error state at the current time step.

[0272] In this embodiment, after error injection is completed, since the deviation represented by the error state has been compensated into the nominal state, the error state is reset to a zero vector. It should be noted that the error state covariance matrix does not need to be reset; its current value is retained for the next round of error state covariance matrix prediction.

[0273] The above-described unscented Kalman filter update process is performed periodically (synchronized with the polarization observation frequency).

[0274] The deviation between the corrected heading angle and the UAV's desired heading angle is calculated, and the UAV is controlled to fly along the predetermined route based on this deviation.

[0275] In this embodiment, flight control commands are generated based on a preset flight path (i.e., an air corridor) to drive the UAV to fly stably and accurately along the preset route, ensuring that it always remains within the safe corridor. Specifically, as shown... Figure 5 As shown, for the case of an uncorrected trajectory (divergent), the method of this invention calculates the corrected heading angle and the UAV's desired heading angle (obtained from the safe corridor), thereby obtaining the corrected position / attitude to form the new corrected trajectory of this invention. This trajectory is compared with a preset air corridor and a preset flight path to generate control commands (attitude / throttle), which drive the UAV motors to perform corresponding actions, ultimately obtaining the corrected trajectory of this invention, ensuring that the UAV always flies stably within the safe corridor.

[0276] Experiments have shown that during a 5-minute flight with complete GNSS denial, a UAV using only an IMU may have a horizontal positioning error of over 100 meters and potentially crash into a building. However, the UAV using the method described in this embodiment has its heading error consistently controlled within 3° and its horizontal trajectory error constrained to the center of a 10-meter-wide air corridor, successfully achieving stable passage and precise delivery in complex environments.

[0277] Example 2:

[0278] This embodiment provides a fragmented sky polarization orientation correction system for UAV navigation, such as... Figure 6 As shown, the system includes:

[0279] The image acquisition and preprocessing module is used to acquire raw polarized images containing sky areas in real time during the flight of the UAV and perform distortion correction to obtain distortion-free polarized images.

[0280] The fragmented sky segmentation module is used to identify distortion-free polarization images using a lightweight sky segmentation model, obtain a probability distribution map of the sky region, and extract multiple valid sky fragment regions from the probability distribution map of the sky region.

[0281] The polarization information extraction module is used to extract the polarization information of each valid sky debris region based on the distortion-free polarization image covered by that valid sky debris region.

[0282] The heading angle calculation module is used to calculate the absolute geographic heading angle of the UAV at the current moment based on the polarization information of all valid sky debris regions and using the fragmented polarization heading solution model.

[0283] The filtering and correction module is used to obtain the predicted heading angle at the current moment, and correct the predicted heading angle using an unscented Kalman filter method based on the absolute geographic heading angle of the UAV at the current moment, so as to obtain the corrected heading angle.

[0284] The flight control module is used to calculate the deviation between the corrected heading angle and the UAV's desired heading angle, and to control the UAV to fly along a predetermined route based on this deviation.

[0285] Example 3:

[0286] This embodiment proposes an electronic device, including: one or more processors, and a memory, the memory being used to store instructions, which, when executed by the one or more processors, cause the one or more processors to perform the fragmented sky polarization orientation correction method for UAV navigation.

[0287] The electronic device may be a mobile phone, computer, or tablet computer, etc., and includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the fragmented sky polarization orientation correction method for UAV navigation as described in the embodiments. It is understood that the electronic device may also include input / output (I / O) interfaces and communication components.

[0288] The processor is used to execute all or part of the steps in the fragmented sky polarization orientation correction method for UAV navigation as described in the above embodiments. The memory is used to store various types of data, which may include, for example, instructions for any application or method in the electronic device, as well as application-related data.

[0289] The processor can be implemented as an Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor, or other electronic components, and is used to execute the fragmented sky polarization orientation correction method for UAV navigation described in the above embodiments.

[0290] Example 4:

[0291] This embodiment proposes a computer-readable storage medium that stores executable instructions. When these instructions are executed, if they are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0292] The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the fragmented sky polarization orientation correction method for UAV navigation described in the various embodiments of this application.

[0293] The aforementioned storage media include: flash memory, hard disks, multimedia cards, card-type memory (e.g., SD (Secure Digital Memory Card) or DX (Memory Data Register, MDR) memory), random access memory (RAM), static random-access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, disks, optical discs, servers, APP (Application) app stores, and other media capable of storing program verification codes. These media store computer programs, which, when executed by a processor, can implement the various steps of the fragmented sky polarization orientation correction method for UAV navigation described above.

[0294] Example 5:

[0295] This embodiment proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implements the fragmented sky polarization light orientation correction method for UAV navigation.

[0296] Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a computer program product.

[0297] The various embodiments in this application are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0298] The scope of protection of this application is not limited to the embodiments described above. Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from the scope and spirit of this disclosure. If such modifications and variations fall within the scope of this disclosure and its equivalents, then the intent of this disclosure also includes these modifications and variations.

Claims

1. A method for correcting the orientation of fragmented sky polarized light for UAV navigation, characterized in that, This method includes the following steps: During the flight of the UAV, the original polarization image containing the sky area is acquired in real time and the distortion is corrected to obtain a distortion-free polarization image. A lightweight sky segmentation model is used to identify distortion-free polarization images, obtain a probability distribution map of the sky region, and extract multiple effective sky fragment regions from the probability distribution map of the sky region. For each valid sky debris region, the polarization information of the valid sky debris region is extracted based on the distortion-free polarization image covered by the valid sky debris region. Based on the polarization information of all valid sky debris regions, the absolute geographic heading angle of the UAV at the current moment is calculated using the fragmented polarization heading solution model. The predicted heading angle at the current moment is obtained, and the predicted heading angle is corrected by using an unscented Kalman filter method based on the absolute geographic heading angle of the UAV at the current moment, so as to obtain the corrected heading angle. The deviation between the corrected heading angle and the UAV's desired heading angle is calculated, and the UAV is controlled to fly along the predetermined route based on this deviation.

2. The fragmented sky polarization orientation correction method for UAV navigation according to claim 1, characterized in that, The specific method for acquiring raw polarization images containing sky regions in real time during UAV flight and performing distortion correction to obtain distortion-free polarization images is as follows: During the flight of the drone, the polarization camera on board the drone is used to collect raw polarization images containing the sky area in real time. The original polarization image includes: original polarization component images of at least three different known polarization directions at the same time; Obtain the intrinsic parameter matrix K and distortion coefficient D of the polarization camera; For any original polarization component image in the original polarization image, the distortion of the original polarization component image is corrected using the intrinsic parameter matrix K and the distortion coefficient D to obtain the corrected polarization component image. The original polarization component image is resampled using the corrected polarization component image to obtain a distortion-free polarization component image. Then, all the obtained distortion-free polarization component images are used as the distortion-free polarization image.

3. The fragmented sky polarization orientation correction method for UAV navigation according to claim 2, characterized in that, The lightweight sky segmentation model consists of an encoder and a decoder; The encoder uses MobileNetV3 to extract features from the input image layer by layer, resulting in a multi-scale feature map. The decoder is implemented using a feature pyramid structure, which is used to fuse multi-scale feature maps step by step to obtain the final feature map, and then maps the final feature map to a probability distribution map of the sky region. The decoder performs step-by-step fusion of multi-scale feature maps to obtain the final feature map. The specific process of mapping the final feature map to a probability distribution map of the sky region is as follows: the multi-scale feature maps are arranged in order of increasing resolution, denoted as... ;in Represents multi-scale feature maps; The number of feature maps representing a multi-scale feature map; This represents the first-level feature map; This represents the second-level feature map; Indicates the first Level feature map; The lowest resolution level 1 feature map is obtained through bilinear upsampling. Restore to the previous level feature map After achieving the same resolution, then compare with the feature map. Channel splicing is performed to obtain the first fused feature map. ; From the first fusion feature map To begin, repeat the following process: The first... Fusion Feature Map Restore to feature map After achieving the same resolution, then compare with the feature map. Perform channel splicing to obtain the first... Fusion Feature Map ;in ; Until the first Fusion Feature Map Until then, and will the Fusion Feature Map As the final feature map; After mapping the final feature map to a single-channel feature map using two-dimensional convolution, the probability distribution map of the sky region is obtained by passing the Sigmoid activation function; the value of each pixel in the probability distribution map of the sky region is the probability that the pixel belongs to the sky category, and the value ranges from 0 to 1.

4. The fragmented sky polarization orientation correction method for UAV navigation according to claim 3, characterized in that, The specific method for identifying distortion-free polarization images using a lightweight sky segmentation model, obtaining a probability distribution map of the sky region, and extracting multiple effective sky debris regions from the probability distribution map of the sky region is as follows: Input any distortion-free polarization component image from the distortion-free polarization image into the lightweight sky segmentation model to obtain the probability distribution map of the sky region; The probability distribution map of the sky region is binarized according to a preset threshold to obtain a sky segmentation mask; each pixel in the sky segmentation mask is marked as either sky or non-sky. Connectivity labeling is performed on the sky segmentation mask, marking all adjacent sky pixels as the same connected region, and each connected region as a sky fragment region; For each sky fragment region, calculate the pixel area of ​​the sky fragment region. If the calculated pixel area is less than the preset area threshold, the sky fragment region is regarded as an invalid region and is removed. A sky fragment region whose area per pixel is not less than a preset area threshold is considered a valid sky fragment region.

5. The fragmented sky polarization orientation correction method for UAV navigation according to claim 4, characterized in that, The specific method for extracting the polarization information of each valid sky debris region based on the distortion-free polarization image covered by that valid sky debris region is as follows: The polarization information includes: representative polarization angle, average degree of polarization, and observation direction vector; For any valid sky debris region According to the effective sky debris area Calculate the Stokes parameter for each pixel covered on the undistorted polarization component image in the undistorted polarization image. Calculate the polarization angle of each pixel based on the Stokes parameters. With polarization degree ; Take valid sky debris area Polarization angle of all pixels The median value is used as the effective sky debris region. The representative polarization angle ; Calculate the effective sky debris region polarization degree of all pixels The average value is used as the effective sky debris area. average polarization ; Calculate the effective sky debris region using any distortion-free polarization component image from a distortion-free polarization image. The average coordinates of all pixels within the area are used as the effective sky fragment region. centroid coordinates ;in Indicates the effective sky debris area The horizontal coordinate of the centroid in the pixel coordinate system; Indicates the effective sky debris area The ordinate of the centroid in the pixel coordinate system; The effective sky debris region is determined using the intrinsic parameter matrix K. centroid coordinates By back-projecting onto the camera coordinate system, the effective sky debris region can be obtained. Observation direction vector .

6. The fragmented sky polarization orientation correction method for UAV navigation according to claim 5, characterized in that, The specific method for calculating the absolute geographic heading angle of the UAV at the current moment using the fragmented polarization heading solution model based on the polarization information of all valid sky debris regions is as follows: Obtain the current UTC timestamp and the latitude and longitude of the UAV, and use the Solar Position Algorithm (SPA) to calculate the solar altitude angle at the current moment based on the UTC timestamp and the UAV's latitude and longitude. and solar azimuth This leads to the construction of solar position parameters. ; Based on polarization information and solar position parameters of all valid sky debris regions A nonlinear least squares optimization problem is constructed and used as a solution model for fragmented polarization heading. ； in, Indicates the absolute geographic heading angle of the drone; Indicates to Find the minimum value; The number of effective sky debris areas; This represents the theoretical calculation function for the polarization angle based on the single-scattering Rayleigh model. It is used to calculate the theoretical polarization angle of the effective sky debris region based on the UAV's absolute geographic heading angle, solar position parameters, and the observation direction vector of the effective sky debris region. Based on the fragmented polarization heading calculation model, the Random Sample Consensus (RANSAC) algorithm is used to generate the RANSAC in-point set and the initial heading estimate of the UAV. Using the initial estimate of the UAV's heading as the initial value, a nonlinear optimization algorithm is employed based on the RANSAC interior point set to solve the fragmented polarization heading calculation model, thereby obtaining the UAV's absolute geographic heading angle. .

7. The fragmented sky polarization orientation correction method for UAV navigation according to claim 6, characterized in that, The calculation process of the polarization angle calculation function based on the single-scattering Rayleigh model is as follows: For any valid sky debris region By utilizing the pre-calibrated extrinsic rotation matrix of the polarization camera relative to the aircraft coordinate system, the effective sky debris region is captured. Observation direction vector Transform to body coordinate system; Get the current drone roll angle and drone pitch angle ; Utilizing the current drone roll angle UAV pitch angle and the absolute geographical heading angle of the drone Construct a rotation matrix from the body coordinate system to the geographic coordinate system. ; Using the rotation matrix Effective sky debris area Observation direction vector Transform to the geographic coordinate system to obtain the observation direction vector in the geographic coordinate system. ; Based on the solar position parameters, construct the solar direction vector in the geographic coordinate system. ; Calculate the solar direction vector in the geographic coordinate system. Observation direction vector in geographic coordinate system The vector product yields the effective sky debris region. Theoretical polarization direction vector in geographic coordinate system ; Effective Sky Fragmentation Area Theoretical polarization direction vector in geographic coordinate system Transform to camera coordinate system to obtain effective sky debris region Theoretical polarization direction vector in camera coordinate system ; According to the effective sky debris area Theoretical polarization direction vector in camera coordinate system Calculate the effective sky debris region Theoretical polarization angle .

8. The fragmented sky polarization orientation correction method for UAV navigation according to claim 7, characterized in that, The specific method for generating the RANSAC in-point set and the initial heading estimate of the UAV based on the fragmented polarization heading solution model and using the Random Sample Consensus (RANSAC) algorithm is as follows: Set the maximum number of iterations, start the iteration, and perform the following operations during each iteration: Initialize an empty hypothetical interior point set; The representative polarization angle of each valid sky debris region and observation direction vector As a sample, from The smallest sample set is randomly selected from the samples; the number of samples in the smallest sample set is... ; Based on the minimum sample set, the assumed heading angle value is calculated in reverse using the polarization angle calculation function based on the single-scattering Rayleigh model. ; for For each of the samples, based on the assumed heading angle value Given the solar position parameters and the observed direction vector of the sample, the theoretical polarization angle of the sample is calculated using a polarization angle calculation function based on the Rayleigh model of single scattering. Then, the theoretical polarization angle and the representative polarization angle of the sample are calculated. The residual; All samples with residuals less than a preset threshold are included in the hypothesis interior set of the current iteration. When the maximum number of iterations is reached, the iteration ends. By comparing the number of samples contained in the hypothetical interior point sets of all iteration rounds, the hypothetical interior point set with the largest number of samples is selected as the RANSAC interior point set, and the heading angle hypothesis value corresponding to this hypothetical interior point set is set. As an initial estimate of the drone's heading .

9. The fragmented sky polarization orientation correction method for UAV navigation according to claim 8, characterized in that, The specific method for obtaining the predicted heading angle at the current moment, and correcting the predicted heading angle using an unscented Kalman filter based on the absolute geographic heading angle of the UAV at the current moment, to obtain the corrected heading angle is as follows: Define nominal state ;in, The attitude of the drone; For the speed of the drone; Location of the drone; Zero bias for the gyroscope; To achieve zero bias in the accelerometer; Define error state ;in, Represents the three-dimensional attitude error vector, and , These represent roll error, pitch error, and heading error, respectively. Indicates speed error; Indicates positional error; This indicates the zero bias error of the gyroscope; This indicates the zero bias error of the accelerometer; Obtain the current angular velocity measurement value and acceleration measurement values The attitude, velocity, and position in the nominal state of the previous moment are integrated forward to obtain the predicted nominal state value at the current moment. The state transition matrix is ​​determined based on the error state dynamic equation, and the process noise covariance matrix is ​​determined based on the angular random walk, acceleration random walk, random walk noise of gyroscope zero bias, and random walk noise of accelerometer zero bias. Using the state transition matrix and process noise covariance matrix, the error state covariance matrix is ​​predicted; For the current absolute geographic heading angle of the UAV, the unscented Kalman filter update process is performed as follows: Extract the predicted heading angle at the current moment from the nominal state prediction value at the current moment. The absolute geographical heading angle of the drone at the current moment. With predicted heading angle The difference is used as the observed new information. ; Based on the mean error state at the current moment and the predicted error state covariance matrix, a symmetric sampling strategy is used to generate... There are Sigma points; among them. Error state dimensionality; Propagate each Sigma point through the observation equation to obtain the mean of the predicted observations. Observation covariance and the cross-covariance between error state and observation ; Based on the observed covariance and the cross-covariance between error state and observation Calculate the Kalman gain; Calculate observational information Compared with the predicted observation mean The difference is used to update the error state and the predicted error state covariance matrix with the Kalman gain, so as to obtain the error state estimate and error state covariance matrix updated by observation at the current time. The error state estimate updated by observation at the current time is injected into the nominal state prediction value at the current time to obtain the nominal state at the current time, and the corrected heading angle is extracted from the nominal state at the current time. Set the error state to the zero vector to prepare for correction at the next time step.

10. A fragmented sky polarization orientation correction system for UAV navigation, used to implement the fragmented sky polarization orientation correction method for UAV navigation as described in any one of claims 1-9, characterized in that, The system includes: The image acquisition and preprocessing module is used to acquire raw polarization images containing sky areas in real time during the flight of the UAV and perform distortion correction to obtain distortion-free polarization images. The fragmented sky segmentation module is used to identify distortion-free polarization images using a lightweight sky segmentation model, obtain a probability distribution map of the sky region, and extract multiple effective sky fragment regions from the probability distribution map of the sky region. The polarization information extraction module is used to extract the polarization information of each valid sky debris region based on the distortion-free polarization image covered by the valid sky debris region. The heading angle calculation module is used to calculate the absolute geographic heading angle of the UAV at the current moment based on the polarization information of all valid sky debris regions and the fragmented polarization heading solution model. The filtering and correction module is used to obtain the predicted heading angle at the current moment, and correct the predicted heading angle using an unscented Kalman filter method based on the absolute geographic heading angle of the UAV at the current moment, so as to obtain the corrected heading angle. The flight control module is used to calculate the deviation between the corrected heading angle and the UAV's desired heading angle, and to control the UAV to fly along a predetermined route based on this deviation.

Images