An automatic flight attitude control method and system for unmanned aerial vehicles (UAVs)

Abstract

This invention discloses an automatic flight attitude control method and system for unmanned aerial vehicles (UAVs), relating to the fields of machine vision and UAV flight control technology. The method includes: acquiring two adjacent frames of distorted grayscale images and inertial angular velocity; calculating a dense optical flow field and its divergence field; performing grayscale histogram tail curvature inflection point analysis on the first frame image to generate a continuous highlight weight map, and solving the Poisson equation using the divergence field as the source term to obtain the divergence components; correcting the optical flow field based on the weight map and adaptive suppression coefficients to obtain a corrected optical flow field; extracting the visual rotation rate from the corrected optical flow field and adaptively fusing it with the inertial yaw angular velocity to generate control commands, which are then output as motor speeds via a hybrid control system. This invention can suppress visual measurement interference in high-reflectivity, high-brightness scenarios, improving the stability and reliability of UAV attitude control.

Description

Technical Field

[0001] This invention relates to the fields of machine vision and unmanned aerial vehicle (UAV) flight control technology, and in particular to an automatic flight attitude control method and system for UAVs. Background Technology

[0002] In tasks such as fixed-point hovering, low-altitude close-range flight, and inspection photography of multi-rotor UAVs, they are usually equipped with airborne cameras and inertial measurement units. Visual quantities such as optical flow are calculated by using adjacent image sequences to characterize the relative motion of the aircraft. Combined with inertial angular velocity, attitude stabilization control is achieved. Especially in high-reflection scenarios such as water surfaces, glass curtain walls, photovoltaic panels, and metal roofs, visual inspection is often introduced into the attitude closed loop as a key measurement source to obtain more stable images and higher hovering accuracy. However, under strong light and specular reflection conditions, overexposed highlight areas are prone to appear in the images. At the same time, the rotor downwash will cause small fluctuations in the reflective surface, causing the highlight position to slip. This results in a significant outward diffusion of the optical flow field, which is mixed into the attitude estimation. The control system mistakenly takes the non-real motion as an attitude deviation and generates unnecessary correction output, thus causing problems such as attitude micro-sway, increased energy consumption, and unstable hovering.

[0003] Existing technologies often employ fixed thresholds or empirical rules to remove or downweight bright pixels, or perform general processing on optical flow such as mean filtering, amplitude limiting, and outlier removal. However, these methods lack adaptability to changes in ambient illumination, exposure strategies, and highlight morphology, making it easy to miss or falsely detect bright areas. Consequently, the diffuse apparent motion caused by highlight smoothing still dominates the optical flow statistics. Furthermore, the lack of an interpretable separation mechanism for the component sources of the optical flow field makes it difficult to suppress diffuse apparent motion in a targeted manner. This results in the possibility that the fixed weights may still be used in fusion and control calculations when the reliability of visual measurements is reduced in high-reflectivity, high-brightness scenes. Ultimately, this interferes with the estimation of angular velocity in directions such as yaw and transmits it to the mixing and control execution stage, making it difficult to stably suppress attitude fluctuations caused by highlights. Summary of the Invention

[0004] The purpose of this invention is to solve the problem in the prior art that visual optical flow measurement is easily affected by high-smoothness motion interference in high-reflection and high-brightness scenes, resulting in non-real diffusion motion, which leads to increased attitude estimation error and causes attitude micro-sway and hovering instability. Therefore, this invention proposes an automatic flight attitude control method and system for unmanned aerial vehicles.

[0005] To address the problems existing in the prior art, the present invention adopts the following technical solution: An automatic flight attitude control method for an unmanned aerial vehicle (UAV) includes: S1: Collect two adjacent frames of the UAV's distorted grayscale images, denoted as the first distorted grayscale image and the second distorted grayscale image, and collect the UAV's inertial angular velocity. S2: Determine the dense optical flow field based on the distortion-free grayscale images of two adjacent frames, and obtain the optical flow divergence field by differentiating the dense optical flow field. S3: Perform histogram analysis on the first distortion-reduced grayscale image, and generate a continuous weighted map of highlights based on the curvature inflection point at the tail of the histogram. S4: Calculate the bias index based on the specular continuous weight map and the optical flow divergence field, and obtain the pure divergence component by solving the Poisson equation based on the optical flow divergence field; S5: Determine the suppression coefficient based on the bias index, and correct the dense optical flow field based on the specular continuous weight map, the suppression coefficient, and the pure divergence component to obtain the corrected optical flow field; S6: Extract the visual rotation rate from the corrected optical flow field and generate visual confidence based on the bias index. Based on the visual confidence, fuse the visual rotation rate with the yaw rate in the inertial angular velocity to obtain the fused yaw rate. Generate control commands based on the fused yaw rate.

[0006] Preferably, generating a continuous weighted specular map includes: Construct a grayscale histogram of the first distorted grayscale image and obtain the grayscale value corresponding to the main peak of the histogram; Within the range greater than the gray value of the main peak, calculate the discrete second difference of the logarithmic histogram; The gray level corresponding to the modulus maxima of the discrete second-order difference is taken as the inflection point; Based on the inflection point, a normalized mapping function is constructed, and the weight value of each pixel in the first distortion-free grayscale image is calculated to obtain a continuous weight map of highlights with values ​​ranging from zero to one.

[0007] Preferably, the bias index is calculated based on the specular continuity weighting map and the optical flow divergence field, including: The optical flow divergence field is statistically weighted based on the continuous weighted specular map, and the weighted divergence mean of the specular region and the weighted divergence mean of the background region are calculated respectively. Calculate the mean amplitude of the dense optical flow field; Calculate the difference between the weighted divergence mean of the highlight region and the weighted divergence mean of the background region, and divide the difference by the amplitude mean of the dense optical flow field to obtain the bias index.

[0008] Preferably, the pure divergence components are obtained by solving the Poisson equation based on the optical flow divergence field, including: Construct the Poisson equation, which is as follows: ,in, For the Laplace operator, Let be the potential function. For the optical flow divergence field; The potential function is obtained by solving the Poisson equation using an iterative method, and the gradient of the potential function is calculated. The gradient is then used as the pure divergence component.

[0009] Preferably, the suppression coefficient is determined based on the bias exponent, and the dense optical flow field is corrected based on the specular continuity weight map, the suppression coefficient, and the pure divergence component to obtain the corrected optical flow field, including: Calculate the squared value of the bias index, divide the squared value by the sum of the squared value and one, and obtain the suppression coefficient; The product of the suppression coefficient, the specular continuous weight map, and the pure divergence component is calculated, and the product is subtracted point by point in the dense optical flow field to obtain the corrected optical flow field.

[0010] Preferably, extracting the visual rotation rate in the corrected optical flow field includes: A global affine transformation is performed on the corrected optical flow field to obtain the affine transformation matrix; Calculate the difference between the element in the second row and first column of the affine transformation matrix and the element in the first row and second column, and take half of this difference as the visual rotation rate of the image plane.

[0011] Preferably, a visual credibility is generated based on the bias index, and the yaw rate in the visual rotation rate and inertial angular velocity is fused based on the visual credibility to obtain the fused yaw rate, including: The visual credibility is obtained by dividing one by the square of the bias exponent and summing it with one. Calculate the difference between the visual credibility and the inertial angular velocity, and multiply the difference by the yaw rate in the inertial angular velocity to obtain the first component; The second component is obtained by multiplying the visual credibility by the visual rotation rate; Adding the first component to the second component yields the combined yaw rate.

[0012] Preferably, the control command is generated based on the fused yaw rate, including: Obtain the reference angular velocity corresponding to the target attitude of the UAV; Construct a fused angular velocity, wherein the yaw component of the fused angular velocity is the fused yaw angular velocity, and the roll and pitch components of the fused angular velocity are the roll angular velocity and pitch angular velocity in the inertial angular velocity, respectively. Calculate the error between the reference angular velocity and the fused angular velocity to obtain the angular velocity error; The attitude control torque and total thrust are calculated based on the angular velocity error, and the attitude control torque and total thrust are converted into control commands for the speed of each motor through the UAV's hybrid control matrix and executed.

[0013] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention addresses the problem of diffuse apparent motion introduced by high-smoothness shift in high-reflectivity, high-brightness scenes. First, a dense optical flow field is calculated from two adjacent frames of distorted grayscale images to obtain the optical flow divergence field. Then, histogram tail curvature inflection point analysis is performed on the first distorted grayscale image to generate a continuous weighted map of highlights, enabling the highlight region to obtain a continuously adjustable suppression weight in space. Simultaneously, the Poisson equation is solved using the optical flow divergence field as the source term to obtain the pure divergence component. This component, combined with the continuous weighted map of highlights and adaptive suppression coefficients, is used to directionally correct the dense optical flow field. This reduces the contamination of optical flow measurement by outward diffuse apparent motion caused by high-smoothness shift at the source, lowers the risk of attitude miscorrection and micro-swaying, and improves hovering and close-flight stability in high-reflectivity, high-brightness environments.

[0014] 2. This invention extracts the visual rotation rate through global affine fitting in the corrected optical flow field, and adaptively fuses the visual rotation rate with the yaw rate in the inertial angular velocity based on visual reliability to obtain the fused yaw rate. This fused yaw rate is then constructed together with the inertial roll rate and pitch rate to generate control commands. This allows visual information to fully participate when reliable and automatically yield when unreliable, thereby improving the robustness and consistency of yaw rate estimation. Finally, the attitude control torque and total thrust are calculated through the angular velocity error and the motor speed control command is output through the hybrid control matrix. This makes the control link form a closed loop from visual measurement, motion separation, optical flow correction to fused control and execution, reducing the dependence on fixed thresholds and empirical rules and enhancing the adaptability to different lighting and specular morphological changes. Attached Figure Description

[0015] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings: Figure 1 This is a flowchart illustrating an automatic flight attitude control method for an unmanned aerial vehicle (UAV) according to the present invention. Figure 2 This is a functional block diagram of an automatic flight attitude control system for an unmanned aerial vehicle (UAV) according to the present invention. Figure 3 This is a schematic diagram of the optical flow vector field of the present invention; Figure 4 This is the adaptive response curve of the present invention. Detailed Implementation

[0016] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0017] Example: This example provides an automatic flight attitude control method for an unmanned aerial vehicle (UAV). See [link to example]. Figure 1 Specifically, including: This embodiment applies to hovering or close-flying scenarios of multi-rotor UAVs in highly reflective and high-light environments such as water surfaces, glass curtain walls, and photovoltaic panels. The UAV is equipped with an onboard monocular camera, an inertial measurement unit (IMU), and a flight control module. The onboard camera and IMU achieve synchronized data acquisition through hardware synchronization triggering. The flight control module has built-in preset target attitude reference angular velocity and quadcopter hybrid control matrix. The specific implementation process of each step is as follows: S1: Collect two adjacent frames of the UAV's distorted grayscale images, denoted as the first distorted grayscale image and the second distorted grayscale image, and collect the UAV's inertial angular velocity. The inputs for this step are the original images captured by the airborne camera and the inertial data output by the IMU. The core purpose is to obtain synchronized distortion-free grayscale images and inertial angular velocities, providing basic data for subsequent optical flow calculations and angular velocity fusion.

[0018] Specifically, the drone's onboard camera captures continuous images at a frame rate of 30fps, while the IMU simultaneously collects the drone's inertial data (including angular velocity and acceleration) at a frequency of 100Hz. Hardware synchronization signals ensure that the data acquisition time between the onboard camera and the IMU is synchronized, with the synchronization error controlled within ±1ms. Two consecutively captured frames are selected as adjacent original images, denoted as the first original image and the second original image. Distortion removal preprocessing is performed on both original images. The distortion removal process is based on the intrinsic parameter matrix of the onboard camera (obtained in advance using the Zhang Zhengyou calibration method; the intrinsic parameter matrix includes focal length, principal point coordinates, and distortion coefficients). The undistort function in OpenCV is used to perform distortion removal calculations, resulting in two adjacent distorted color images. Subsequently, the two distorted color images are converted into distorted grayscale images using a weighted grayscale conversion formula: Where I is the grayscale value, and R, G, and B are the pixel values ​​of the red, green, and blue channels of the color image, respectively, with the grayscale value ranging from [0, 255]. The two adjacent frames of the resulting distorted grayscale images are denoted as the first distorted grayscale image. Second distortion-reduced grayscale image Where (x, y) are pixel coordinates, with x ranging from [0, 639] and y ranging from [0, 479] (corresponding to a 640×480 resolution image). Simultaneously, IMU inertial angular velocity data synchronized with the acquisition time of the two frames of distortion-reduced grayscale images are extracted and denoted as inertial angular velocity. ,in For roll rate, For pitch angular velocity, This is the yaw rate.

[0019] To improve the stability of subsequent histogram analysis and optical flow calculation, the first distortion-corrected grayscale image was... Normalization preprocessing is performed; the normalization formula is: ,in This is the first normalized, distortion-free grayscale image. The minimum grayscale value of all pixels in the first distorted grayscale image. The maximum grayscale value. To prevent extremely small positive numbers with a denominator of zero, normalization is performed. The value range is [0,1].

[0020] The output of this step is: the first distorted grayscale image. Second distortion-reduced grayscale image The first distortion-free grayscale image after normalization and inertial angular velocity .

[0021] S2: Determine the dense optical flow field based on the distortion-free grayscale images of two adjacent frames, and obtain the optical flow divergence field by differentiating the dense optical flow field. The input for this step is the first distorted grayscale image. Second distortion-reduced grayscale image The core objective is to calculate the dense optical flow field by the pixel motion relationship between two frames of images, and then obtain the optical flow divergence field by differentiation, providing core data for subsequent pseudo optical flow bias quantization and suppression.

[0022] Determination of dense optical flow field and calculation of optical flow divergence field; Specifically, the dense optical flow field is calculated using the local least squares method, where the physical meaning of the optical flow field is the motion velocity vector of each pixel in two adjacent frames. First, the first distortion-corrected grayscale image... Second distortion-reduced grayscale image Calculate the gray-level gradients in the x-direction, y-direction, and time dimension, respectively, and denote them as follows: , The gradient calculation uses the Sobel operator with a window size of 3×3, and the calculation formula is as follows: in, for Pixel The rate of change of grayscale in direction. for The rate of change of grayscale in direction. The grayscale change rate over time; A local least squares model is constructed based on the optical flow constraint equations. The optical flow constraint equations are as follows: ,in This represents the velocity component of the pixel in the x-direction. The velocity component of the pixel in the y-direction, together with the velocity component in the y-direction, constitutes the optical flow vector of the pixel. For each pixel (x, y), a 5×5 neighborhood window centered on it is selected. The optical flow constraint equations are stacked into a matrix form within the window: ,in This is the gradient matrix within the window (dimension 25×2). This is the temporal gradient vector within the window (dimension 25×1). Let be the optical flow vector to be solved. The optical flow vector is solved using the least squares method, and the solution formula is: ,in Let A be the transpose of A. for The inverse matrix. If If the matrix is ​​singular (i.e., its determinant is zero), then a regularization term is introduced. ( (where I is the regularization coefficient and I is the identity matrix), the solution formula is: To avoid unsolvable exceptions, the above calculations are performed on each pixel in the image one by one to obtain the dense optical flow field. .

[0023] Specifically, the optical flow divergence field is calculated as follows: The physical meaning of the optical flow divergence field is the degree of divergence of the dense optical flow field. Positive divergence indicates that pixel motion tends to spread outward, while negative divergence indicates that it tends to converge inward. The optical flow divergence is obtained by taking the partial derivatives of the x-direction velocity component u and the y-direction velocity component v of the dense optical flow field using the central difference method, and then summing them. The calculation formula is as follows: in The optical flow divergence value at pixel (x,y); Let u be the partial derivative of u in the x-direction, calculated as follows: ; Let v be the partial derivative of v in the y-direction, calculated as follows: For image boundary pixels (x=0, x=639, y=0, y=479), the first-order difference is used instead of the central difference. For example, when x=0, To avoid the problem of boundary pixels being uncalculated, the optical flow divergence field is calculated for each pixel in the dense optical flow field. .

[0024] The output of this step is: a dense optical flow field. and light divergence field .

[0025] S3: Perform histogram analysis on the first distortion-reduced grayscale image, and generate a continuous weighted map of highlights based on the curvature inflection point at the tail of the histogram. The input for this step is the normalized, first distortion-free grayscale image. The core objective is to automatically identify highlight regions through histogram analysis, generate a continuous weight map to quantify the highlight level of each pixel, and provide a weight basis for subsequent targeted suppression of pseudo optical flow in highlight regions.

[0026] Specifically, a gray-level histogram of the first distorted gray-level image is constructed, and the gray-level values ​​corresponding to the main peaks of the histogram are obtained. The normalized gray-level values ​​are used as the abscissa, and the abscissa interval is divided into 256 gray-level buckets (i.e., discrete gray-level k, k=0,1,…,255, corresponding to equally spaced normalized gray-level values ​​[0,1]). The number of pixels in each gray-level bucket is used as the ordinate, and the number of pixels is counted. The distribution of each pixel in different grayscale buckets is used to obtain a grayscale histogram. ,in This represents the number of pixels whose normalized grayscale value falls within the k-th grayscale bucket. (Iterate through...) Find the gray level k0 corresponding to the maximum value among all possible values. k0 is the gray value corresponding to the main peak of the histogram. Its physical meaning is the most densely distributed gray level in the first distortion-free gray image, corresponding to the background main area of ​​the image.

[0027] Specifically, within the range where the gray value is greater than the main peak gray value, the discrete second-order difference of the logarithmic histogram is calculated: since the gray value of the highlight region is usually higher than that of the background main region, only the tail interval of the histogram where k>k0 (i.e., the highlight candidate interval) is analyzed. To suppress the interference of histogram noise on inflection point identification, the histogram within this interval is... Perform a logarithmic transformation, the transformation formula is as follows: ,in ,avoid The anomaly where logarithm cannot be taken when =0. The discrete second-order difference is calculated for the histogram after logarithmic transformation. The physical meaning of the discrete second-order difference is the curvature of the histogram curve, which can effectively characterize the inflection point of the histogram's trend. The calculation formula is: The range of k is k0+1≤k≤254 (to avoid the inability to calculate the second difference for boundary k values).

[0028] Specifically, the gray level corresponding to the maximum magnitude of the discrete second-order difference is taken as the inflection point: the absolute value of the above discrete second-order difference result is calculated. Find the gray level corresponding to the maximum absolute value. , This refers to the curvature inflection point at the tail of the histogram. Physically, this inflection point marks the grayscale boundary between the highlight and non-highlight areas, where the grayscale value is greater than... The area is a highlight area, smaller than The area is a non-highlight area.

[0029] Specifically, based on the inflection point, a normalized mapping function is constructed to calculate the weight value of each pixel in the first distortion-free grayscale image, resulting in a continuous highlight weight map with values ​​ranging from zero to one. The normalized mapping function is then used to convert the normalized grayscale values ​​of each pixel... The weight values ​​mapped to the interval [0,1] are expressed by the following mapping function: ,in The highlight weight value for pixel (x,y) is defined by the `clip` function, which is used to restrict the calculation result to the interval [0,1] (taking 0 when the result is less than 0, taking 1 when it is greater than 1, and keeping the original value when it is between 0 and 1). 1- The length of the interval from the inflection point to the maximum gray level. Used to avoid a denominator of zero. Weight value Physical meaning: =0 indicates that the pixel is a non-highlight pixel and does not need to be suppressed; =1 indicates that the pixel is a strong highlight pixel and needs to be completely suppressed; 0< A value less than 1 indicates that the pixel is a weak highlight pixel, and the suppression intensity increases with the increase of the weight value. The weight value is calculated for each pixel in the first distortion-reduced grayscale image to obtain a continuous highlight weight map. .

[0030] The output of this step is: a continuous weighted specular map. .

[0031] S4: Calculate the bias exponent based on the specular continuous weight map and the optical flow divergence field, and obtain the pure divergence component by solving the Poisson equation based on the optical flow divergence field. The input for this step is a specular continuous weight map. Dense optical flow field and light divergence field The core objective is to quantify the bias degree (bias index) of specular-dominated pseudo-optical flow and separate the pure divergence component (i.e. the main component of pseudo-optical flow) in the optical flow field, so as to provide a quantitative basis and suppression target for subsequent adaptive suppression of pseudo-optical flow.

[0032] Specifically, the weighted divergence mean of the highlight region and the weighted divergence mean of the background region are calculated: the physical meaning of the weighted divergence mean of the highlight region is the weighted average of the optical flow divergence within the highlight region (the weights are the continuous weights of the highlights), and the weighted divergence mean of the background region is the weighted average of the optical flow divergence within the background region (the weights are 1-w(x,y), i.e., the non-highlight weights). The calculation formulas are as follows: , in The weighted mean divergence of the highlight region. Ω is the weighted divergence mean of the background region; Ω is the set of valid pixels in the image (i.e., all 640×480 pixels). Used to avoid a denominator of zero (e.g., when all pixels are non-highlight pixels). ).

[0033] Calculate the mean amplitude of the dense optical flow field: mean amplitude of the dense optical flow field Used to normalize the bias level and avoid the influence of optical flow amplitude on bias quantization, the calculation formula is: ,in Total number of effective pixels, Let be the magnitude of the optical flow vector at pixel (x,y).

[0034] Calculating the bias exponent: The physical meaning of the bias exponent K is the divergence bias intensity of the highlight region relative to the background region. It quantifies the degree of bias of the positive divergence pseudo-optical flow dominated by the highlight. The calculation formula is: ,in The denominator is the difference between the mean divergence of the highlights and the background (a positive difference indicates greater divergence in the highlight region, i.e., a more pronounced pseudo-optical flow bias). Used for normalization, making K a dimensionless parameter; Used to avoid a denominator of zero.

[0035] Constructing the Poisson equation: The Poisson equation is ,in It is the Laplace operator (characterizing the second-order spatial rate of change of the function). Let D be the potential function (used to separate divergence components), and D be the optical flow divergence field obtained in step S2. The physical meaning of this equation is to fit the optical flow divergence field with the Laplace distribution of the potential function, thereby decomposing the optical flow field into divergence and curl components, where the divergence component is the main component of the specular pseudo-optical flow.

[0036] Solving the Poisson equation and calculating its gradient: The Jacobi iterative method is used to solve the Poisson equation. This iterative method is applicable to the grid structure of image pixels, is computationally simple, and easy to implement in engineering. The initial conditions for iteration are... (The initial potential function value of all pixels is 0, and the superscript (0) indicates the iteration number), the iteration formula is: ,in Let be the potential function value of the pixel (x=i, y=j) at the nth iteration. This represents the optical flow divergence value for that pixel. The iteration termination condition is... (Convergence is considered achieved when the maximum difference in the potential function between two adjacent iterations is less than a threshold). The maximum number of iterations is set to 1000 (to avoid computational stagnation caused by non-convergence). The potential function is obtained after iteration convergence. .

[0037] Calculate the gradient of the potential function and treat the gradient as a pure divergence component. The gradient is calculated using the central difference method, and the formula is: ,in , Boundary pixels are calculated using first-order difference (same as the divergence calculation boundary processing method in step S2). Pure divergence components. The physical meaning is the motion component in the optical flow field caused only by divergence, which is the core component of the pseudo optical flow in the specular region.

[0038] The output of this step is: the bias exponent K and the pure divergence component. .

[0039] S5: Determine the suppression coefficient based on the bias index, and correct the dense optical flow field based on the specular continuous weight map, the suppression coefficient, and the pure divergence component to obtain the corrected optical flow field; The inputs for this step are the bias exponent K and the pure divergence component. Highlight Continuous Weight Map Dense optical flow field The core objective is to adaptively adjust the suppression intensity based on the degree of pseudo-optical flow bias, and to suppress only the pure divergence pseudo-optical flow component in the highlight region in a directional manner, thereby obtaining a corrected optical flow field that eliminates pseudo-optical flow interference.

[0040] Calculate the inhibition coefficient: Inhibition coefficient The physical meaning of is the suppression intensity of pseudo-optical flow, and its value ranges from [0,1]. The closer to 1, the stronger the inhibition. The closer the value is to 0, the weaker the inhibition strength. The inhibition coefficient is calculated based on the bias exponent K, using the following formula: The design logic of this formula is: when K=0 (no pseudo-optical flow bias), =0, no suppression; when K increases (pseudo-optical flow bias is enhanced). As K approaches 1, the suppression intensity gradually increases; when K is very large (severe pseudo-optical flow bias), the suppression intensity gradually increases. ≈1, to suppress pseudo-optical flow to the greatest extent and achieve adaptive adjustment of suppression intensity.

[0041] The corrected optical flow field is obtained by correcting the dense optical flow field: a pixel-by-pixel correction method is used to suppress the pure divergence pseudo-optical flow component only in the highlight region. The correction formula is as follows: ,in The corrected optical flow vector at pixel (x,y) is the pixel value of the corrected optical flow field. For the pseudo-optical flow component that needs to be suppressed, α controls the overall suppression intensity. By controlling the degree of suppression in local highlight areas, the two factors are multiplied to achieve precise suppression that is both globally adaptive and locally directional.

[0042] The output of this step is: Corrected optical flow field. The corrected optical flow field eliminates the interference of pseudo-optical flow in the highlight region and is used for visual rotation rate extraction.

[0043] S6: Extract the visual rotation rate from the corrected optical flow field, generate visual confidence based on the bias exponent, fuse the visual rotation rate with the yaw rate in the inertial angular velocity based on the visual confidence, obtain the fused yaw rate, and generate control commands based on the fused yaw rate. The input for this step is the corrected optical flow field. Bias index K, inertial angular velocity The core objective is to extract a reliable visual rotation rate, adaptively fuse it with the inertial angular velocity, generate stable flight control commands, and achieve stable attitude control of the UAV.

[0044] Extracting visual rotation rate: Specifically, a global affine transformation is performed to fit the corrected optical flow field: affine transformations can characterize the overall motion of an image, such as translation, rotation, and scaling. By fitting the corrected optical flow field through a global affine transformation, the overall rotational motion of the image (corresponding to the attitude rotation of the UAV) can be separated. The affine transformation model is as follows: ,in It is a pixel coordinate vector. It is a 2×2 affine transformation matrix (containing rotation and scaling information). This is a translation vector (containing translation information). The least squares method is used to solve it. and The objective function is: This means minimizing the sum of squared errors between the corrected optical flow vector and the affine transformation prediction vector. By differentiating the objective function and setting the derivative to zero, we can obtain... and If anomalies occur during the analytical solution process (such as the objective function not converging), the RANSAC algorithm is used to remove abnormal pixels (such as edge-occluded pixels) and then refit the solution to ensure the reliability of the fitting results.

[0045] Calculate the visual rotation rate: The affine transformation matrix A has the following form: ,in and For rotation-related elements. The visual rotation rate of the image plane. The calculation formula is: The physical meaning of this formula is to separate the overall rotational angular velocity of the image from the affine transformation matrix. This rotational angular velocity corresponds to the attitude rotation motion of the UAV, providing data for attitude measurement at the vision end.

[0046] The fused yaw rate is obtained by fusing the visual rotation rate and yaw rate: Generate visual credibility: Visual credibility The physical meaning of this is to assess the reliability of the visual rotation rate extracted from the optical flow field, and its value ranges from [0,1]. The closer the value is to 1, the more reliable the visual rotation rate. The closer the value is to 0, the less reliable the visual rotation rate. Visual confidence is calculated based on the bias index K, using the following formula: This formula complements the suppression coefficient formula of step S5: when K=0 (no pseudo-optical flow bias), =1, visual rotation rate is completely reliable; as K increases (pseudo-optical flow bias is enhanced). As K approaches 0, the reliability of the visual rotation rate decreases; when K is large (severe pseudo-optical flow bias), the reliability decreases. ≈0, visual rotation rate is basically unreliable, enabling adaptive evaluation of visual credibility.

[0047] The fused yaw rate is obtained by fusing the visual rotation rate and the yaw rate from the inertial angular velocity using a weighted fusion method. in To incorporate yaw rate; As the weight of the inertial angular velocity, The weights are determined by the visual rotation rate. The fusion logic is: when visual credibility is high ( Larger visual reliability depends more on visual rotation rate; when visual reliability is low ( The data from the two sensors (IMU inertial data) is less affected by high light conditions and more stable. By adaptively assigning weights, the advantages of both sensors are complemented, improving the reliability of yaw rate measurement. It should be noted that the visual rotation rate needs to be adjusted before fusion. The unit is converted from rad / frame to rad / s using the following formula: Where f=30Hz is the frame rate of the airborne camera, ensuring consistency with the unit of inertial angular velocity.

[0048] Control commands are generated based on the fused yaw rate: Constructing the fused angular velocity: The yaw component of the fused angular velocity is the fused yaw angular velocity. The roll component and pitch component are the roll angular velocities in the inertial angular velocities, respectively. With pitch angular velocity That is, the fusion angular velocity is The roll and pitch components of the inertial angular velocity were chosen as the corresponding components of the fused angular velocity because the visual measurements in the roll and pitch directions are less affected by the specular environment and do not require fusion correction; while the visual measurements in the yaw direction are more affected by specular pseudo-optical flow and require fusion to improve reliability.

[0049] Calculate angular velocity error: Obtain the reference angular velocity corresponding to the target attitude of the UAV. The reference angular velocity is preset by the flight mission (e.g., for hovering missions, the reference angular velocity is...). The error between the reference angular velocity and the fused angular velocity is calculated to obtain the angular velocity error. The formula is: It represents the degree of deviation between the current attitude angular velocity and the target attitude angular velocity.

[0050] Calculation of attitude control torque and total thrust: The attitude control torque is calculated based on the angular velocity error using a proportional control algorithm. The control formula is as follows: ,in For attitude control torque ( For rolling moment, For pitching moment, (Yaw moment); The moment of inertia matrix of the UAV body (obtained in advance through experimental measurement, is a 3×3 diagonal matrix); =5.0 is the proportional control coefficient (determined through debugging to ensure control stability). Simultaneously, the total thrust T is set according to flight mission requirements (e.g., T equals the drone's weight when hovering).

[0051] Converted into motor speed control commands and executed: The total thrust T and attitude control torque are converted into motor speed control commands through the UAV's hybrid control matrix B. Converted to the square of the speed of each motor, the mixed control relationship is as follows: ,in The control matrix is ​​a 4×4 quadcopter (determined by the structural parameters of the UAV, and is a known matrix). , , , These represent the rotational speeds of the four motors. Inverting the above formula yields the formula for calculating the square of the motor speed: ,in For mixed control matrix The inverse matrix is ​​obtained. The square root of the squared value of the calculated rotational speed is taken to obtain the target rotational speed of each motor. The target rotational speed is sent as a control command to the motor drive module of the UAV, driving the motor to run at the target rotational speed, thereby realizing the attitude stabilization control of the UAV.

[0052] To facilitate understanding of the above embodiments, a specific application scenario of the above embodiments will be used as an example for illustration below: A consumer-grade quadcopter drone (model: XX-400, fuselage weight 1.2kg, maximum takeoff weight 1.5kg) is tasked with hovering at a fixed point in an open outdoor area (hovering accuracy requirement ±0.2m) for low-altitude photography and monitoring.

[0053] In a bright outdoor midday sun environment (solar altitude angle 65°-75°, ambient illuminance 10000-12000 lux), with a light gray cement ground (containing localized areas of high light reflection, with reflected illuminance exceeding 15000 lux), no significant obstructions, and a light breeze (wind speed 2-3 m / s), images captured by the camera are prone to high-saturation areas. Traditional optical flow methods are susceptible to producing false optical flow, leading to instability in the drone's attitude control.

[0054] Airborne camera: mounted in the center of the drone's belly, with the lens facing directly downwards, the optical axis parallel to the vertical direction of the drone body, the minimum hovering height of the lens above the ground is 5m, the lens focal length is 3.6mm, and the sensor size is 1 / 2.3 inches.

[0055] IMU (Inertial Measurement Unit): Integrated into the center of the UAV flight control motherboard, it achieves hardware synchronization with the airborne camera through the SPI interface, with a synchronization error of ≤±1ms.

[0056] Flight control module: Installed at the center of gravity of the UAV fuselage, it receives data from the IMU and camera, executes the control algorithm of this solution, and outputs motor control commands.

[0057] Image data: The airborne camera acquires color images at a frame rate of 30fps with a resolution of 640×480. After acquisition, the images are converted into distortion-free grayscale images in real time by the flight controller's built-in processor. The data format is 8-bit single channel.

[0058] Inertial data: The IMU acquires triaxial angular velocity and triaxial acceleration at a frequency of 100Hz. The angular velocity data is used for attitude fusion in this scheme, and the acceleration data is used to assist in positioning calibration. The angular velocity measurement range is ±2000° / s, and the accuracy is ±0.01° / s.

[0059] Synchronization method: Hardware-triggered synchronization is adopted. The camera outputs a trigger signal when it acquires a frame of image. After receiving the signal, the IMU synchronously records a set of inertial data to ensure that the timestamps of the image and the inertial data are consistent.

[0060] This embodiment provides a set of fixed example values ​​(typical values ​​selected within a range) and demonstrates the calculation process of key intermediate quantities according to steps S1-S6. All data units are consistent with commonly used engineering units. Image resolution: 640×480 (x:0-639, y:0-479); First distortion-corrected grayscale image pixel grayscale values: background pixels (100, 100): 120; highlight pixels (320, 240): 230; IMU inertial angular velocity: =0.02rad / s, =-0.01rad / s, =0.05 rad / s; Camera frame rate: 30Hz; Regularization coefficient: 0.01; Minimal positive number: ; Step S1: Data Acquisition Input: Raw color images captured by the camera (first frame: R=245, G=240, B=235; second frame: R=243, G=238, B=233), IMU synchronous inertial angular velocity data ( =0.02rad / s, =-0.01rad / s, =0.05 rad / s).

[0061] Distortion removal: Based on the camera intrinsic parameter matrix (pre-calibrated: fx=500, fᵧ=500, cx=320, cᵧ=240, distortion coefficients k1=-0.01, k2=0.005), distortion removal is performed using OpenCV's undistort function to obtain a distortion-removed color image.

[0062] Grayscale calculation: using the formula Substitute the example value: The first distorted grayscale image pixel (100, 100): I = 0.299 × 245 + 0.587 × 240 + 0.114 × 235 ≈ 120; The first distortion-corrected grayscale image pixels (320, 240): I = 0.299 × 243 + 0.587 × 238 + 0.114 × 233 ≈ 230; Normalization: Formula ,in =30 (shaded area) =245 (strong highlight area), substituting, we get: Normalized grayscale value of pixel (100, 100): (120 - 30) / (245 - 30 + 10⁻) 6 )≈0.4186; Normalized grayscale value of pixel (320, 240): (230-30) / (245-30+10⁻) 6 )≈0.9302.

[0063] Step S2: Calculation of optical flow field and divergence field: Gray-level gradient calculation: using central difference, formula , , ; Pixels (320, 240): =231-229=2, =232-228=4, =229-230=-1; Dense optical flow calculation: 5×5 neighborhood window, least squares solution, no singular matrix (no regularization required), formula The optical flow vector of the pixel is calculated. pixels / frame; Optical divergence calculation: formula ,in ; Therefore, D(320,240)=0.01+0.01=0.02.

[0064] Step S3: Generation of Continuous Spectrum Weight Map: Input: The first distorted grayscale image output by S1 .

[0065] Gray-scale histogram construction: 256 gray-scale buckets, the main peak of the histogram is k0=110 (corresponding to a normalized gray value of 0.4186, i.e. the background and the main subject). Logarithmic histogram Figure 2 For the second-order difference: in the interval k>110, the inflection point k*=180 corresponding to the maximum value of the discrete second-order difference modulus is calculated (corresponding to a normalized gray value of 0.7814). Weight calculation: Formula : Pixels (320, 240): (0.9302 - 0.7814) / (1 - 0.7814 + 10⁻) 6 )≈0.687, after clipping w=0.687; Pixel (100, 100): (0.4186-0.7814) / (0.2186)≈-1.659, w=0 after clipping.

[0066] Step S4: Calculation of bias exponent and pure divergence components: Inputs: Optical flow field and divergence field output by S2; weight map output by S3.

[0067] Weighted divergence mean: Formula , After analyzing all pixels, we find that μ_H = 0.018 and μ_B = 0.005. Mean optical flow amplitude: Formula Calculated pixels / frame; Bias Index: Formula ; Pure divergence components: Poisson equation After the Jacobi iteration converges, the gradient is calculated for that pixel. Pixels / frame.

[0068] Step S5: Correction of optical flow field calculation: Inputs: S2 Optical flow field, S4 Bias exponent and pure divergence component, S3 Weighted plot.

[0069] Suppression coefficient: Formula ; Corrected optical flow: Formula : Pixels (320, 240): Pixels / frame.

[0070] Step S6: Fusion and Control Command Generation: Input: S5 Correction of optical flow field, S4 Bias index, S1 Inertial angular velocity.

[0071] Visual rotation rate: A obtained from affine fitting 21 =0.002, A 12 =-0.001, formula rad / frame, converted to rad / s: 0.0015 × 30 = 0.045 rad / s; Visual credibility: Formula ; Combined yaw rate: Formula rad / s; Angular velocity error: Reference angular velocity for hovering mission fusion angular velocity ,error rad / s; Control torque and total thrust: J = diag([0.002, 0.002, 0.004]) kg·m², kω = 5.0, torque N·m; Total hovering thrust T=12N (equal to the weight of the drone); Motor speed: The quadcopter control matrix B (known) is inverted to calculate the motor speeds n1=1520r / min, n2=1480r / min, n3=1510r / min, and n4=1490r / min.

[0072] This embodiment adaptively identifies highlight regions through histogram inflection points and suppresses pseudo-optical flow using a continuous weighted graph. The yaw rate measurement deviation is only 0.0043 rad / s (significantly reduced compared to the deviation of over 0.1 rad / s in existing fixed-threshold schemes), providing reliable data support for precise attitude control. Based on a vision-inertial fusion strategy with dynamic fusion weights, the drone can hover stably even in strong outdoor midday light, with hovering accuracy controlled within ±0.15m, exceeding the mission's preset ±0.2m requirement, effectively avoiding drift, oscillation, and other instability caused by highlights. No manual setting of highlight thresholds is required; the adaptive algorithm adapts to a wide range of light intensity variations from 10,000 to 12,000 lux. Simultaneously, differentiated differential calculation is used for boundary pixels, ensuring consistent pixel processing accuracy across the entire image (boundary divergence calculation error ≤0.002), making it stable and applicable to various outdoor high-light open scenes. The core calculation steps all adopt commonly used engineering algorithms (least squares, Jacobi iteration, etc.), and the example data are all typical values ​​that can be collected in practice. Key parameters (regularization coefficient λ=0.01, frame rate 30Hz, etc.) can be implemented through conventional consumer-grade hardware without the need for special high-performance devices, making it easy to apply in batches to similar consumer-grade drone products.

[0073] like Figure 2 The diagram shown is a functional block diagram of an automatic flight attitude control system for an unmanned aerial vehicle (UAV) according to an embodiment of the present invention.

[0074] In this embodiment, the functions of each module / unit are as follows: The data acquisition module is used to acquire two adjacent frames of distorted grayscale images of the UAV and the UAV's inertial angular velocity. The divergence calculation module is used to determine the dense optical flow field based on the distortion-free grayscale images of two adjacent frames, and to differentiate the dense optical flow field to obtain the optical flow divergence field. The weight generation module is used to perform histogram analysis on the first distortion-reduced grayscale image and generate a continuous highlight weight map based on the curvature inflection point at the tail of the histogram. The component separation module is used to calculate the bias index based on the specular continuous weight map and the optical flow divergence field, and to obtain the pure divergence component by solving the Poisson equation based on the optical flow divergence field. The optical flow correction module is used to determine the suppression coefficient based on the bias index, and to correct the dense optical flow field based on the specular continuity weight map, the suppression coefficient and the pure divergence component to obtain the corrected optical flow field. The attitude control module is used to extract the visual rotation rate in the corrected optical flow field, generate visual confidence based on the bias index, fuse the visual rotation rate and the yaw rate in the inertial angular velocity based on the visual confidence to obtain the fused yaw rate, and generate control commands based on the fused yaw rate.

[0075] like Figure 3 As shown, Figure 3 This is a schematic diagram of the optical flow vector field. The red arrows represent the optical flow vectors estimated from two adjacent frames. The overall trend is one of divergence from the center outwards, interspersed with local irregular perturbations, reflecting the superposition of apparent motion caused by high-smoothness shifts and other factors onto the actual motion. The orange arrows represent the vector components of pure divergence obtained by constructing the potential function from the divergence field and taking its gradient. A typical form is a radially diverging field with the center as the source point. The white circle in the middle is usually used to represent invalid / masked regions or the schematic center of the divergence source. The green arrows represent the result after subtracting the pure divergence components from the original optical flow field according to weights and suppression coefficients. The vectors in the diagram tend towards a consistent translation direction, indicating that after the diffuse pseudo-components are suppressed, the remaining optical flow is closer to the actual overall motion and is more suitable for extracting rotation rates and generating attitude control commands.

[0076] like Figure 4 As shown, Figure 4 This is an adaptive response curve. The horizontal axis represents the bias exponent K, ranging from low values ​​on the left (corresponding to no highlights or weak interference) to high values ​​on the right (corresponding to strong interference). The vertical axis represents the amplitude range from zero to one. The blue curve represents the visual confidence ρ, which gradually decreases from close to one to close to zero as K increases, indicating that the weight of visual quantities in fusion and control is gradually reduced when interference increases. The red curve represents the suppression coefficient α, which gradually increases from close to zero to close to one as K increases, indicating that stronger suppression is applied to the diffuse apparent motion component caused by highlights when interference increases. The two curves intersect near a medium K value, reflecting the system's continuous adaptive adjustment mechanism of visual confidence yielding and optical flow suppression enhancing under changes in interference intensity.

[0077] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for automatic flight attitude control of an unmanned aerial vehicle (UAV), characterized in that, include: S1: Collect two adjacent frames of the UAV's distorted grayscale images, denoted as the first distorted grayscale image and the second distorted grayscale image, and collect the UAV's inertial angular velocity. S2: Determine the dense optical flow field based on the distortion-free grayscale images of two adjacent frames, and obtain the optical flow divergence field by differentiating the dense optical flow field. S3: Perform histogram analysis on the first distortion-reduced grayscale image, and generate a continuous weighted map of highlights based on the curvature inflection point at the tail of the histogram. The specific steps for generating a continuous weighted highlight map based on the curvature inflection point at the tail of the histogram are as follows: Construct a grayscale histogram of the first distorted grayscale image and obtain the grayscale value corresponding to the main peak of the histogram; Within the range greater than the gray value of the main peak, calculate the discrete second difference of the logarithmic histogram; The gray level corresponding to the modulus maxima of the discrete second-order difference is taken as the inflection point; Based on the inflection point, a normalized mapping function is constructed, and the weight value of each pixel in the first distortion-free grayscale image is calculated to obtain a continuous weight map of highlights with values ​​ranging from zero to one. S4: Calculate the bias index based on the specular continuous weight map and the optical flow divergence field, and obtain the pure divergence component by solving the Poisson equation based on the optical flow divergence field; The specific steps for calculating the bias exponent based on the specular continuous weight map and the optical flow divergence field, and then solving the Poisson equation based on the optical flow divergence field to obtain the pure divergence components are as follows: The optical flow divergence field is statistically weighted based on the continuous weighted specular map, and the weighted divergence mean of the specular region and the weighted divergence mean of the background region are calculated respectively. Calculate the mean amplitude of the dense optical flow field; Calculate the difference between the weighted divergence mean of the highlight region and the weighted divergence mean of the background region, and divide the difference by the amplitude mean of the dense optical flow field to obtain the bias exponent; Construct the Poisson equation, which is: ,in, For the Laplace operator, Let be the potential function. For the optical flow divergence field; The potential function is obtained by solving the Poisson equation using an iterative method, and the gradient of the potential function is calculated. The gradient is then used as a pure divergence component. S5: Determine the suppression coefficient based on the bias index, and correct the dense optical flow field based on the specular continuous weight map, the suppression coefficient, and the pure divergence component to obtain the corrected optical flow field; The specific steps for determining the inhibition coefficient based on the bias index are as follows: Calculate the squared value of the bias index, divide the squared value by the sum of the squared value and one, and obtain the suppression coefficient; S6: Extract the visual rotation rate from the corrected optical flow field, generate visual confidence based on the bias exponent, fuse the visual rotation rate with the yaw rate in the inertial angular velocity based on the visual confidence, obtain the fused yaw rate, and generate control commands based on the fused yaw rate. The specific steps for generating visual credibility based on the bias index are as follows: The visual credibility is obtained by dividing one by the square of the bias exponent and summing it with one.

2. The automatic flight attitude control method for an unmanned aerial vehicle (UAV) according to claim 1, characterized in that, The dense optical flow field is corrected based on the specular continuity weight map, suppression coefficient, and pure divergence component to obtain the corrected optical flow field, including: The product of the suppression coefficient, the specular continuous weight map, and the pure divergence component is calculated, and the product is subtracted point by point in the dense optical flow field to obtain the corrected optical flow field.

3. The automatic flight attitude control method for an unmanned aerial vehicle (UAV) according to claim 1, characterized in that, Extracting the visual rotation rate from the corrected optical flow field includes: A global affine transformation is performed on the corrected optical flow field to obtain the affine transformation matrix; Calculate the difference between the element in the second row and first column of the affine transformation matrix and the element in the first row and second column, and take half of this difference as the visual rotation rate of the image plane.

4. The automatic flight attitude control method for an unmanned aerial vehicle (UAV) according to claim 1, characterized in that, Based on visual credibility, the yaw rate in the visual rotation rate and inertial angular velocity is fused to obtain the fused yaw rate, which includes: Calculate the difference between the visual credibility and the inertial angular velocity, and multiply the difference by the yaw rate in the inertial angular velocity to obtain the first component; The second component is obtained by multiplying the visual credibility by the visual rotation rate; Adding the first component to the second component yields the combined yaw rate.

5. The automatic flight attitude control method for an unmanned aerial vehicle (UAV) according to claim 1, characterized in that, Control commands are generated based on the fused yaw rate, including: Obtain the reference angular velocity corresponding to the target attitude of the UAV; Construct a fused angular velocity, wherein the yaw component of the fused angular velocity is the fused yaw angular velocity, and the roll and pitch components of the fused angular velocity are the roll angular velocity and pitch angular velocity in the inertial angular velocity, respectively. Calculate the error between the reference angular velocity and the fused angular velocity to obtain the angular velocity error; The attitude control torque and total thrust are calculated based on the angular velocity error, and the attitude control torque and total thrust are converted into control commands for the speed of each motor through the UAV's hybrid control matrix and executed.

6. An automatic flight attitude control system for an unmanned aerial vehicle (UAV), characterized in that, The system is used to execute the automatic flight attitude control method for a UAV according to any one of claims 1-5, the system comprising: The data acquisition module is used to acquire two adjacent frames of distorted grayscale images of the UAV and the UAV's inertial angular velocity. The divergence calculation module is used to determine the dense optical flow field based on the distortion-free grayscale images of two adjacent frames, and to differentiate the dense optical flow field to obtain the optical flow divergence field. The weight generation module is used to perform histogram analysis on the first distortion-reduced grayscale image and generate a continuous highlight weight map based on the curvature inflection point at the tail of the histogram. The component separation module is used to calculate the bias index based on the specular continuous weight map and the optical flow divergence field, and to obtain the pure divergence component by solving the Poisson equation based on the optical flow divergence field. The optical flow correction module is used to determine the suppression coefficient based on the bias index, and to correct the dense optical flow field based on the specular continuity weight map, the suppression coefficient and the pure divergence component to obtain the corrected optical flow field. The attitude control module is used to extract the visual rotation rate in the corrected optical flow field, generate visual confidence based on the bias index, fuse the visual rotation rate and the yaw rate in the inertial angular velocity based on the visual confidence to obtain the fused yaw rate, and generate control commands based on the fused yaw rate.

Images