Aircraft automatic balancing and image stabilizing system based on sensor feedback

Abstract

The invention discloses an aircraft automatic balancing and image stabilizing system based on sensor feedback, and belongs to the technical field of aircraft balance control. The aircraft automatic balancing and image stabilizing system is based on sensor feedback data, measures the angular velocity and acceleration of an aircraft, estimates the attitude of the aircraft in real time, and predicts the future attitude of the aircraft; according to attitude estimation and prediction results, adjusting a control surface of the aircraft in real time, capturing images continuously shot by the aircraft in real time, analyzing whether the continuously shot images shake or not, when shaking occurs, analyzing attitude information and camera settings of the aircraft, and if the shaking occurs, keeping the current camera settings, adjusting the attitude of the aircraft, and if the attitude of the aircraft is a stable attitude, adjusting camera setting; in a complex flight environment, the attitude and image setting of the aircraft can be automatically adjusted, and the stability and image quality of the aircraft in the complex environment are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of aircraft balance control, and specifically relates to an aircraft automatic balance and image stabilization system based on sensor feedback. Background Art

[0002] With the in-depth application of aircraft in fields such as aerial photography, surveying and mapping, and inspection, higher and higher requirements are put forward for the automatic balance and image stabilization performance of aircraft. In the field of aerial photography, in order to obtain clear and stable images and videos, the aircraft needs to maintain a stable flight attitude under various complex environmental conditions. However, during the actual flight process, the aircraft will be disturbed by various factors, such as air currents, wind forces, terrain changes, etc. These factors will cause the instability of the aircraft attitude, resulting in problems such as blurred and jittery images during shooting.

[0003] Existing automatic balance technologies can provide good performance under certain conditions, but in the high-dynamic environment of aircraft (such as large wind speed changes, fast acceleration changes, etc.), these control methods may have problems such as lag, overcompensation, or poor stability; traditional image anti-shake technologies are mostly based on hardware or software methods, and these methods often cannot provide sufficient effects in environments with rapid movement, severe jitter, or high-frequency disturbances. Therefore, it is of great practical significance and application value to develop an efficient and reliable aircraft automatic balance and image stabilization system based on sensor feedback. Summary of the Invention

[0004] Aiming at the deficiencies of the existing technology, the present invention proposes an aircraft automatic balance and image stabilization system based on sensor feedback, which realizes high-precision attitude control and image stabilization of the aircraft through multi-sensor data fusion and intelligent control algorithms.

[0005] To achieve the above object, the present invention provides the following technical solutions: An aircraft automatic balance and image stabilization system based on sensor feedback, comprising: an attitude estimation and measurement module, an attitude balance module, and an image anti-shake module; The attitude estimation and measurement module measures the angular velocity and acceleration of the aircraft based on sensor feedback data, estimates the attitude of the aircraft in real time, and predicts the future attitude of the aircraft; The attitude balance module adjusts the control surfaces of the aircraft in real time according to the attitude estimation and prediction results; The image anti-shake module is used to capture the images continuously shot by the aircraft in real time, analyze whether the continuously shot images are jittery. When jitter occurs, it analyzes the attitude information of the aircraft and the camera settings. If it is caused by the aircraft attitude, it keeps the current camera settings and adjusts the aircraft attitude. If the aircraft attitude is a stable attitude, it adjusts the camera settings.

[0006] Specifically, the attitude estimation and measurement module includes: a data acquisition and fusion unit, an attitude estimation unit, and an attitude prediction unit; The data acquisition and fusion unit uses an IMU, GPS, vision sensor, lidar, barometer, and ultrasonic sensor to collect aircraft data, and performs preprocessing and data fusion; The attitude estimation unit estimates the attitude of the aircraft based on the fused aircraft data; The attitude prediction unit predicts the future attitude of the aircraft based on current and historical attitude data.

[0007] Specifically, estimating the attitude of the aircraft based on the fused aircraft data includes: Setting the state vector of the aircraft, including: attitude q, position p, velocity v, gyroscope bias b g and accelerometer bias b a etc., initializing the attitude, position, velocity, covariance matrix P, process noise Q, and observation noise R of the aircraft; Using the gyroscope measurement value minus the accelerometer bias b a to correct the angular velocity of the aircraft; Converting the accelerometer measurement value to the navigation coordinate system, deducting the influence of gravity, estimating the velocity and position of the aircraft, and updating the covariance matrix; Calculating the residual and Kalman gain between the GPS data and the state vector of the aircraft, and using the Kalman gain to correct the state vector of the aircraft; Extracting the attitude q from the corrected state vector of the aircraft.

[0008] Specifically, adjusting the control surface of the aircraft in real time according to the attitude estimation and prediction results includes: Calculating the error between the current attitude data of the aircraft and the attitude data in the steady state; Setting an adaptive control algorithm and initial coefficients according to the basic parameters of the aircraft and the flight environment, and dynamically adjusting the coefficients in the adaptive control algorithm according to the attitude error and its change rate data of the aircraft; Based on the adjusted coefficients of the adaptive control algorithm and the current attitude error, constructing a mathematical model for power distribution of the front and rear motors of the aircraft, calculating the thrust distribution of each motor, and adjusting the angle of the flight surface.

[0009] Specifically, the image anti-shake module includes: a feature extraction and matching unit, a displacement and rotation unit, and a judgment and compensation unit; The feature extraction and matching unit is used to extract the feature points of the images continuously captured by the aircraft and match the extracted feature points; The displacement and rotation unit calculates the relative displacement and rotation between the continuously captured images of the aircraft according to the feature point matching result. The judgment and compensation unit analyzes whether jitter is found in the continuously captured images of the aircraft and performs compensation.

[0010] Specifically, extracting the feature points of the continuously captured images of the aircraft and matching the extracted feature points includes: Performing feature point mining on the sequence of continuously captured images of the aircraft, constructing an image pyramid, and performing Gaussian filtering and downsampling processing on the real-time flight images of the aircraft in different scale spaces. Taking pixels as the unit, traversing and searching each pyramid image, and determining potential feature points according to the change characteristics of the pixel gray values around the feature points. Calculating the response intensity of each feature point, setting a dynamic threshold, and adjusting it in real time according to the overall contrast and brightness information of the continuously captured images of the aircraft, removing the weak feature points with response intensity lower than the dynamic threshold, and using the non-maximum suppression algorithm to retain the point with the strongest response in the neighborhood of the feature point to obtain a set of feature points. Performing the first and second matches on the set of feature points to obtain the feature point matching result.

[0011] Specifically, performing the first and second matches on the set of feature points to obtain the feature point matching result includes: Performing the first match on the set of feature points, comparing the feature points extracted from the current image with the feature points of the previous frame image one by one to find similar feature point pairs. Performing the second match on the set of feature points, randomly extracting a subset from the first match feature point pairs, fitting the extracted subset into a mathematical model, calculating the inlier rate of the mathematical model, repeating the random extraction, and retaining the mathematical model with the highest inlier rate and the corresponding matching feature point pairs.

[0012] Specifically, calculating the relative displacement and rotation between the continuously captured images of the aircraft according to the feature point matching result includes: According to the matched feature point pairs, using the least squares method to solve the homography matrix. Performing singular value decomposition on the obtained homography matrix, and extracting the relative displacement and rotation angle between the continuously captured images of the aircraft.

[0013] Specifically, analyzing whether jitter is found in the continuously captured images of the aircraft and performing compensation includes: According to the relative displacement and rotation angle between the continuously captured images of the aircraft, and the relative displacement and rotation between the real-time attitude information of the aircraft and the attitude information in the stable state, determining whether jitter occurs in the continuously captured images. If the images captured continuously are jittery, determine whether the displacement and rotation of the continuously captured images are caused by the aircraft attitude. If it is caused by the aircraft attitude, do not process, maintain the current camera settings, and wait for the attitude balance module to adjust the aircraft to a stable state. If the aircraft attitude is stable at this moment, then adjust the camera settings, and perform reverse compensation through the relative displacement and rotation of the continuously captured images to offset the changes; If the images captured continuously are not jittery, then do not process.

[0014] Compared with the prior art, the beneficial effects of the present invention are: 1. The present invention proposes an aircraft automatic balance and image stabilization system based on sensor feedback, which can obtain the attitude information of the aircraft in real time, and dynamically adjust the thrust distribution and control surface angles of the aircraft through an adaptive control algorithm, enabling the aircraft to accurately maintain the set attitude. Even in complex flight environments (such as high wind speeds, rapid turning, etc.), it can automatically adjust, enhancing the anti-interference ability of the aircraft and ensuring stable flight of the aircraft.

[0015] 2. The present invention proposes an aircraft automatic balance and image stabilization system based on sensor feedback, which can dynamically adjust the camera system according to the aircraft during flight, compensating for the image displacement and rotation caused by the aircraft attitude change, making the image stable and clear. Especially during flight, the image will not show obvious blurring or jitter due to attitude changes or external disturbances, improving the image quality.

[0016] 3. The present invention proposes an aircraft automatic balance and image stabilization system based on sensor feedback, which can automatically adjust the attitude and image settings of the aircraft, improving the stability of the aircraft and the image quality in complex environments, thereby enhancing the reliability and efficiency of the aircraft to perform tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is an architecture diagram of an aircraft automatic balance and image stabilization system based on sensor feedback provided by the present invention; Figure 2 It is an architecture diagram of the attitude estimation and measurement module provided by the present invention; Figure 3 It is a flowchart of feature point extraction and matching provided by the present invention; Figure 4 It is an architecture diagram of the image anti-shake module provided by the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0018] The present application will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any form. It should be noted that those of ordinary skill in the art can make several modifications and improvements without departing from the concept of the present application. These all fall within the protection scope of the present application.

[0019] In order to make the purpose, technical solution and advantages of the present application more clear, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0020] It should be noted that if there is no conflict, the various features in the embodiments of the present application can be combined with each other, and all are within the protection scope of the present application. In addition, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from the module division in the device or the flowchart. In addition, the terms "first", "second", "third", etc. used in the present application do not limit the data and execution order, but only distinguish the same items or similar items with basically the same functions and effects.

[0021] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by those skilled in the technical field to which this application belongs. The terms used in this specification in the specification of the present application are only for the purpose of describing specific embodiments and are not used to limit the present application. The term "and / or" used in this specification includes any and all combinations of one or more of the related listed items.

[0022] Embodiment 1

[0023] Please refer to Figures 1 - 4 , an embodiment provided by the present invention: an aircraft automatic balance and image stabilization system based on sensor feedback, including: an attitude measurement module, an attitude balance module, and an image anti-shake module; The attitude estimation and measurement module measures the angular velocity and acceleration of the aircraft based on the sensor feedback data, estimates the attitude of the aircraft in real time, and predicts the future attitude of the aircraft; The attitude estimation and measurement module includes: a data acquisition and fusion unit, an attitude estimation unit, and an attitude prediction unit; The data acquisition and fusion unit uses an IMU, GPS, vision sensor, lidar, barometer, and ultrasonic sensor to collect aircraft data and performs preprocessing and data fusion; In this embodiment, the inertial measurement unit (IMU) includes a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer, which are used to measure the acceleration, angular velocity, and magnetic field strength of the aircraft, providing the basic data for attitude estimation; the global positioning system (GPS) provides the position, speed, and time information of the aircraft, which is used for assisted positioning and navigation, especially in outdoor environments; the vision sensor (camera) is used to capture environmental images, providing visual odometry and optical flow data; the lidar (LiDAR) measures the distance between the aircraft and surrounding objects by emitting laser beams and receiving reflected signals; the barometer measures the atmospheric pressure, which is used to estimate the altitude of the aircraft; the ultrasonic sensor is used for short-distance ranging and is suitable for precise altitude control during low-altitude flight or hovering. The preprocessing includes: data filtering, which preliminarily filters the raw data to remove noise and outliers; data calibration, which calibrates the collected aircraft data to eliminate errors. For example, zero-bias calibration and temperature compensation are performed on the IMU, and lens distortion correction is performed on the vision sensor. For the data fusion, an improved Kalman filter is used to perform hierarchical fusion or adaptive fusion on the preprocessed aircraft data.

[0024] The attitude estimation unit estimates the attitude of the aircraft based on the fused aircraft data. Estimating the attitude of the aircraft based on the fused aircraft data includes: Setting the state vector of the aircraft, including: attitude q (quaternion), position p, velocity v, gyroscope zero bias b g and accelerometer zero bias b a etc., and initializing the attitude, position, velocity, covariance matrix P, process noise Q, and observation noise R of the aircraft. Subtracting the accelerometer zero bias b from the gyroscope measurement value a to correct the angular velocity of the aircraft. Converting the accelerometer measurement value to the navigation coordinate system, subtracting the influence of gravity, estimating the velocity and position of the aircraft, and updating the covariance matrix. Calculating the residual between the GPS data and the state vector of the aircraft and the Kalman gain, and using the Kalman gain to correct the state vector of the aircraft. In this embodiment, the residual between the GPS data and the state vector of the aircraft is the difference between the actual measurement value and the predicted value. The updated angular velocity, velocity, position, etc. data in the above steps are estimated or predicted values, which have a certain difference from the actual data. The Kalman gain determines how to adjust the predicted state according to the measurement data. By weighting the residual with the Kalman gain and updating the state, the correction of the state quantity can be realized, making the predicted state of the aircraft closer to the true state. Through the correction of the difference, real-time estimation can be achieved. Extract the attitude q from the state vector of the corrected aircraft.

[0025] In this embodiment, extracting the attitude q means extracting the quaternion q.

[0026] The attitude prediction unit predicts the future attitude of the aircraft according to the current and historical attitude data.

[0027] The predicting the future attitude of the aircraft according to the current and historical attitude data includes: According to the current and historical attitude data, use the LSTM network to predict the future attitude, and according to the prediction result, adjust the control input in advance to ensure the stable flight of the aircraft.

[0028] The attitude balance module adjusts the control surface of the aircraft in real time according to the attitude estimation and prediction results; The adjusting the control surface of the aircraft in real time according to the attitude estimation and prediction results includes: Calculate the error between the current attitude data of the aircraft and the attitude data in the stable state. According to the basic parameters of the aircraft and the flight environment, set the adaptive control algorithm and the initial coefficient, and dynamically adjust the coefficient in the adaptive control algorithm according to the attitude error of the aircraft and its rate of change data. In this embodiment, an adaptive strategy based on fuzzy logic is adopted. The attitude error and the rate of change of the error are used as fuzzy input variables and divided into multiple fuzzy subsets, such as "negative large", "negative small", "zero", "positive small", "positive large", etc. According to the pre-designed fuzzy rule table, which is constructed based on a large amount of flight experiments and simulation data, determine the adjustment direction and amplitude of the coefficient under different input combinations. For example, when the pitch angle error is "positive large", moderately increase it to quickly correct the large deviation, and at the same time slightly increase it to suppress the too fast attitude change trend; if the error is close to "zero" and the rate of change is stable, gradually reduce the integral term coefficient to avoid integral saturation. Based on the adjusted adaptive control algorithm coefficient and the current attitude error, construct a mathematical model for power distribution for the front and rear motors of the aircraft, calculate the thrust distribution of each motor, and adjust the angle of the flight surface.

[0029] In this embodiment, assume that the aircraft is a quadrotor structure, with two motors in the front and two motors in the rear, and the total power requirement is F total , which is comprehensively determined by the flight control system according to the current flight state (such as hovering, accelerating, climbing, etc.), and combined with the pitch angle error and the coefficient, the power output of the front motor and the power output of the rear motor can be calculated to ensure that the front and rear motors generate an appropriate thrust difference to correct the pitch attitude; For an aircraft equipped with control surfaces (such as a fixed-wing or vertical takeoff and landing fixed-wing aircraft), the control surface angles are calculated based on the yaw and roll attitude errors. Taking the rudder control of yaw as an example, according to the yaw angle error and the adaptive coefficient, the deflection angle of the rudder is calculated through a similar control law. Similarly, for the aileron to control the roll attitude, there is also a corresponding accurate calculation method to obtain the aileron deflection angle, coordinating the power output of the front and rear motors to achieve the fast and stable flight of the UAV in all directions.

[0030] The image anti-shake module is used to capture the images continuously taken by the aircraft in real time, analyze whether the continuously taken images show jitter. When jitter occurs, analyze the attitude information of the aircraft and the camera settings. If it is caused by the aircraft attitude, keep the current camera settings and adjust the aircraft attitude. If the aircraft attitude is a stable attitude, then adjust the camera settings.

[0031] The image anti-shake module includes: a feature extraction and matching unit, a displacement and rotation unit, and a judgment and compensation unit; The feature extraction and matching unit is used to extract the feature points of the images continuously taken by the aircraft and match the extracted feature points; The extraction of the feature points of the images continuously taken by the aircraft and the matching of the extracted feature points include: Use a scale-invariant feature transform algorithm (such as SURF or ORB) to mine the feature points of the image sequence continuously taken by the aircraft, construct an image pyramid, and perform Gaussian filtering and downsampling processing on the real-time flight images of the aircraft in different scale spaces; The purpose of constructing the image pyramid is to: simulate the visual effect of the human eye when observing objects at different distances; Taking pixels as the unit, traverse and search each pyramid image, and determine the potential feature points according to the change characteristics of the pixel gray values around the feature points; Exemplarily, the ORB algorithm defines the feature direction by comparing the gray centroid of the pixel block with the offset direction of the central pixel, which can not only quickly locate the feature points but also endow them with direction attributes, making them have rotational invariance in the subsequent matching process and laying a foundation for accurate anti-shake; Obtain potential feature points through scale neighborhoods, including large-scale neighborhoods and small-scale neighborhoods. Large-scale neighborhood: Select a relatively large neighborhood, such as a 7×7 neighborhood. The large-scale neighborhood can provide more extensive image context information, which can be used to detect the overall contour of the object and the gray value changes in a larger range, making up for the deficiency of the small-scale neighborhood in capturing global information. Small-scale neighborhood: Select a relatively small neighborhood, such as a 3×3 neighborhood. The small-scale neighborhood can focus on the local details of the image, is sensitive to the subtle gray value changes in the image, and can be used to detect local features such as edges and corners in the image; Specific screening method: 1) Set a small-scale threshold according to the gradient magnitude calculated from the small-scale neighborhood. When the gradient magnitude of a certain pixel is greater than this threshold, mark it as a small-scale potential feature point; 2) Set a large-scale threshold according to the standard deviation of the gray value calculated from the large-scale neighborhood. If the standard deviation of a certain pixel is greater than this threshold, mark it as a large-scale potential feature point; 3) Merge the potential feature points obtained by small-scale and large-scale screening.

[0032] The advantages are as follows: The setting of the small-scale threshold can filter out some areas with insignificant gray value changes and preliminarily screen out possible local feature points. The setting of the large-scale threshold can screen out areas with large-scale gray value changes and discover the overall features of the object.

[0033] Calculate the response intensity of each feature point, set a dynamic threshold, and adjust it in real time according to the overall contrast and brightness information of the images continuously captured by the aircraft. Remove the weak feature points with response intensity lower than the dynamic threshold, and use the non-maximum suppression algorithm to retain the point with the strongest response in the neighborhood of the feature point to obtain a set of feature points; Perform the first matching on the set of feature points. Compare the feature points extracted from the current image with the feature points of the previous frame one by one to find similar feature point pairs; In this embodiment, the brute-force matching algorithm based on the Hamming distance is used for the first matching. The Hamming distance measures the difference degree between two binary strings. When the feature point descriptor is binary coded (such as the descriptor generated by the ORB algorithm), it can quickly and efficiently find similar feature point pairs. Set the maximum allowable Hamming distance threshold, which is determined in advance through experiments according to factors such as feature point extraction accuracy and image noise level to ensure the accuracy of the matching. For example, in a low-noise environment, the threshold can be set to 30, and the feature point pairs with a distance higher than this are regarded as unmatched, and the potential matching feature point pairs are preliminarily screened out; Perform the second matching on the set of feature points. Randomly extract a subset from the first-matched feature point pairs, fit the extracted subset into a mathematical model, such as an affine transformation model, calculate the inlier rate of the mathematical model, repeat the random extraction, and retain the mathematical model with the highest inlier rate and the corresponding matching feature point pairs.

[0034] In this embodiment, the inlier rate of the mathematical model is the ratio of the number of point pairs that conform to the mathematical model to the total number of extracted point pairs. Through the second matching, false matching points can be effectively removed, and a high-precision continuous image feature point matching result can be obtained, accurately reflecting the true corresponding relationship between images; The conventional image matching method is to extract feature points, match the feature points, and calculate the similarity. Refer to Figure 3, in this application, a pyramid image is established, potential feature points are extracted through depth traversal, and through a single matching, the initially matched feature point pairs are screened. Through a secondary matching, the unqualified feature point pairs are screened out, and the finally matched feature point pairs are obtained.

[0035] The displacement and rotation unit calculates the relative displacement and rotation between the continuously captured images of the aircraft according to the feature point matching result; The calculating the relative displacement and rotation between the continuously captured images of the aircraft according to the feature point matching result includes: According to the matched feature point pairs, the homography matrix is solved using the least squares method; The homography matrix describes the mapping relationship between two planes, covering transformation information such as translation, rotation, and scaling. Set the coordinates of the feature points on the current image and the coordinates of the corresponding matched points on the previous frame image, construct a system of linear equations to solve the equations simultaneously for a sufficient number (usually more than 4 pairs) of matched points, and obtain 8 independent parameters of the homography matrix; Perform singular value decomposition on the obtained homography matrix, and extract the relative displacement and rotation angle between the continuously captured images of the aircraft.

[0036] Specifically, according to the results of singular value decomposition (SVD) and relevant geometric transformation relationships, the pixel offsets in the horizontal and vertical directions are calculated through a series of mathematical derivations, which reflect the rotation angle of the image around a certain center point. These parameters accurately quantify the motion changes between consecutive images and provide a key basis for subsequent judgment and compensation.

[0037] The judgment and compensation unit is used to analyze whether jitter is found in the continuously captured images of the aircraft and perform compensation.

[0038] The analyzing whether jitter is found in the continuously captured images of the aircraft and performing compensation includes: According to the relative displacement and rotation angle between the continuously captured images of the aircraft, and the relative displacement and rotation between the real-time attitude information of the aircraft and the attitude information in the stable state, it is judged whether jitter occurs in the continuously captured images; If jitter occurs in the continuously captured images, it is judged whether the displacement and rotation of the continuously captured images are caused by the aircraft attitude. If it is caused by the aircraft attitude, no processing is performed, and the current camera settings are maintained, waiting for the attitude balance module to adjust the aircraft to a stable state. If the aircraft attitude is a stable attitude at this moment, the camera settings are adjusted, and reverse compensation is performed through the relative displacement and rotation of the continuously captured images to offset the changes; In this embodiment, for displacement compensation, during the image rendering and output stage, the coordinates of each pixel point are adjusted according to the reverse displacement. For rotation compensation, the rotation matrix transformation is used. If the rotation is around the center point of the image, the reverse cancellation of image jitter is achieved through precise coordinate transformation to ensure the output of stable and clear images. After the compensation is implemented, the feature point matching situation and the change of motion parameters of subsequent consecutive images are monitored in real time. If it is found that there are still significant jitter signs, that is, the feature point matching results show that there is still a large relative displacement or rotation between images, the compensation strategy is readjusted, the reverse compensation parameters are fine-tuned, and even the feature point extraction and matching links are traced back to check whether there are problems. Through continuous iterative optimization, the image anti-shake effect is continuously improved to adapt to the complex and changeable flight shooting environment; If there is no jitter in the continuously captured images, no processing is performed.

[0039] In addition, in the above technical solutions provided in the embodiments of the present application, the parts with the same implementation principles as the corresponding technical solutions in the prior art are not described in detail to avoid excessive elaboration.

[0040] As described in the specific embodiments above, the purpose, technical solutions, and beneficial effects of the present invention have been further described in detail. It should be understood that the above are only specific embodiments of the present invention and are not used to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

Claims

1. An aircraft automatic balancing and image stabilization system based on sensor feedback, characterized in that: include: Posture estimation and measurement module, posture balancing module and image stabilization module; The attitude estimation and measurement module measures the angular velocity and acceleration of the aircraft based on sensor feedback data, estimates the attitude of the aircraft in real time, and predicts the future attitude of the aircraft; The attitude balancing module adjusts the control surface of the aircraft in real time according to the attitude estimation and prediction results; The image stabilization module is used to capture images continuously taken by the aircraft in real time, analyze whether jitter is found in the continuously taken images, and when jitter occurs, analyze the aircraft's attitude information and camera settings. If jitter is caused by the aircraft's attitude, maintain the current camera settings and adjust the aircraft's attitude. If the aircraft's attitude is a stable attitude, adjust the camera settings.

2. The aircraft automatic balancing and image stabilization system based on sensor feedback according to claim 1, characterized in that: The posture estimation and measurement module includes: a data acquisition and fusion unit, a posture estimation unit and a posture prediction unit; The data acquisition and fusion unit uses IMU, GPS, visual sensor, laser radar, barometer, and ultrasonic sensor to collect aircraft data and perform preprocessing and data fusion; The attitude estimation unit estimates the attitude of the aircraft based on the fused aircraft data; The attitude prediction unit predicts the future attitude of the aircraft based on current and historical attitude data.

3. The aircraft automatic balancing and image stabilization system based on sensor feedback according to claim 2, characterized in that: The step of estimating the attitude of the aircraft according to the fused aircraft data includes: Set the state vector of the aircraft, including: attitude q, position p, speed v, gyroscope zero bias b g and accelerometer bias b a , initialize the aircraft's attitude, position, velocity, covariance matrix P, process noise Q and observation noise R; Subtract the accelerometer bias b from the gyroscope measurement a , to correct the angular velocity of the aircraft; Convert the accelerometer measurements to the navigation coordinate system, deduct the effect of gravity, estimate the velocity and position of the aircraft, and update the covariance matrix; Calculate the residual and Kalman gain between the GPS data and the state vector of the aircraft, and use the Kalman gain to correct the state vector of the aircraft; The attitude q is extracted from the corrected state vector of the aircraft.

4. The aircraft automatic balancing and image stabilization system based on sensor feedback according to claim 1, characterized in that: The method of adjusting the control surface of the aircraft in real time according to the attitude estimation and prediction results includes: Calculate the error between the current aircraft's attitude data and the attitude data in a stable state; According to the basic parameters of the aircraft and the flight environment, the adaptive control algorithm and initial coefficients are set, and the coefficients in the adaptive control algorithm are dynamically adjusted according to the attitude error and its rate of change data of the aircraft; Based on the adjusted adaptive control algorithm coefficients and the current attitude error, a mathematical model for power distribution of the front and rear motors of the aircraft is constructed to calculate the thrust distribution of each motor and adjust the heading angle.

5. The aircraft automatic balancing and image stabilization system based on sensor feedback according to claim 1, characterized in that: The image anti-shake module includes: a feature extraction and matching unit, a displacement and rotation unit, and a judgment and compensation unit; The feature extraction and matching unit is used to extract feature points of images continuously taken by the aircraft and match the extracted feature points; The displacement and rotation unit calculates the relative displacement and rotation between the images taken continuously by the aircraft according to the feature point matching result; The judgment and compensation unit is used to analyze whether jitter is found in the images continuously shot by the aircraft and to compensate for it.

6. The aircraft automatic balancing and image stabilization system based on sensor feedback according to claim 5, characterized in that: The step of extracting feature points of images continuously shot by the aircraft and matching the extracted feature points includes: Mining feature points of the image sequences continuously taken by the aircraft, constructing an image pyramid, and performing Gaussian filtering and downsampling on the real-time flight images of the aircraft in different scale spaces; Taking pixels as units, traverse and search each pyramid image, and determine the potential feature points based on the change characteristics of the grayscale values ​​of pixels around the feature points; Calculate the response strength of each feature point, set a dynamic threshold, and adjust in real time according to the overall contrast and brightness information of the images continuously taken by the aircraft. Remove weak feature points whose response strength is lower than the dynamic threshold, and use the non-maximum suppression algorithm to retain the points with the strongest response in the neighborhood of the feature points to obtain a set of feature points. Perform the first and second matching on the feature point set to obtain the feature point matching result.

7. The aircraft automatic balancing and image stabilization system based on sensor feedback according to claim 6, characterized in that: The first and second matching of the feature point set is performed to obtain a feature point matching result, including: Perform the first match on the feature point set, compare the feature points extracted from the current image with the feature points of the previous frame image one by one, and find similar feature point pairs; Perform a second match on the feature point set, randomly extract a subset from the first matching feature point pair, fit the extracted subset into a mathematical model, calculate the inlier rate of the mathematical model, repeat the random extraction, and retain the mathematical model with the highest inlier rate and the corresponding matching feature point pair.

8. The aircraft automatic balancing and image stabilization system based on sensor feedback as claimed in claim 5, characterized in that: The step of calculating the relative displacement and rotation between the images continuously taken by the aircraft according to the feature point matching results includes: According to the matched feature point pairs, the homography matrix is ​​solved using the least squares method; The obtained homography matrix is ​​subjected to singular value decomposition to extract the relative displacement and rotation angle between the images taken continuously by the aircraft.

9. The aircraft automatic balancing and image stabilization system based on sensor feedback as claimed in claim 5, characterized in that: The analyzing whether the images continuously shot by the aircraft are jittered and compensating for the jitters includes: According to the relative displacement and rotation angle between the images taken continuously by the aircraft, and the relative displacement and rotation between the real-time attitude information of the aircraft and the attitude information in a stable state, it is determined whether the images taken continuously are jittery; If the continuously shot images are jittery, determine whether the displacement and rotation of the continuously shot images are caused by the aircraft attitude. If so, do not process it, keep the current camera settings, and wait for the attitude balance module to adjust the aircraft to a stable state. If the aircraft attitude is stable at this moment, adjust the camera settings, and perform reverse compensation through the relative displacement and rotation of the continuously shot images to offset the changes. If there is no shaking in the continuously shot images, no processing will be performed.

Images