Aircraft control method and device, aircraft and computer readable storage medium

Abstract

The invention discloses an aircraft control method and device, an aircraft and a computer readable storage medium, and the method comprises the steps: determining a current first operation state of the aircraft based on current image feature point data of the aircraft and IMU data of an IMU inertial measurement unit; determining whether the aircraft meets a preset zero-speed correction condition or not based on the first operation state; if the aircraft meets the preset zero-speed correction condition, determining an observation equation and a Jacobian matrix corresponding to the aircraft based on the IMU data; and based on the observation equation and the Jacobian matrix, performing correction operation on the current state vector of the aircraft to obtain a target state vector corresponding to the aircraft. Through the current image feature point data IMU data of the aircraft, when it is determined that the aircraft meets the preset zero-speed correction condition, zero-speed correction is performed on the aircraft, the misjudgment condition of the running state of the aircraft can be reduced, and the accuracy of zero-speed correction of the aircraft is improved.

Description

Technical Field

[0001] The present application relates to the field of aircraft technology, and in particular to an aircraft control method, an aircraft, and a computer-readable storage medium. Background Art

[0002] At present, in traditional vehicle navigation, the acceleration and angular velocity information of IMU (Inertial Measurement Unit) is used to detect the stationary state of the carrier. When the carrier is stationary, the speed calculated by the system is used as the speed error term, and Kalman filter estimation is performed to complete the zero speed correction based on the updated state quantity estimate.

[0003] Using IMU information can easily lead to misjudgment of the static state of the carrier. When the acceleration or angular velocity noise of the IMU is too large, or when the body is slightly shaking in the air, it is easy to be misjudged as a moving state. When the body is moving at a constant speed, the acceleration information is in a degraded state and is easily misjudged as a static state.

[0004] Therefore, how to improve the accuracy of zero-speed correction is a problem that needs to be solved urgently. Summary of the invention

[0005] The main purpose of the present application is to provide an aircraft control method, device, aircraft and computer-readable storage medium, aiming to solve the technical problem of how to improve the accuracy of zero-speed correction.

[0006] To achieve the above object, the present application provides an aircraft control method, the aircraft control method comprising the following steps:

[0007] Determine a current first operating state of the aircraft based on current image feature point data of the aircraft and IMU data of an IMU inertial measurement unit;

[0008] determining whether the aircraft satisfies a preset zero-speed correction condition based on the first operating state;

[0009] If the aircraft meets the preset zero-speed correction condition, then determining the observation equation and Jacobian matrix corresponding to the aircraft based on the IMU data;

[0010] Based on the observation equation and the Jacobian matrix, a correction operation is performed on the current state vector of the aircraft to obtain a target state vector corresponding to the aircraft.

[0011] Furthermore, the step of performing a correction operation on the current state vector of the aircraft based on the observation equation and the Jacobian matrix to obtain a target state vector corresponding to the aircraft includes:

[0012] Obtaining a covariance matrix and a sensor noise matrix corresponding to the aircraft;

[0013] Based on the observation equation, the Jacobian matrix, the covariance matrix and the sensor noise matrix, a correction operation is performed on the current state vector of the aircraft to obtain a target state vector corresponding to the aircraft.

[0014] Furthermore, the step of performing a correction operation on the current state vector of the aircraft based on the observation equation, the Jacobian matrix, the covariance matrix and the sensor noise matrix to obtain a target state vector corresponding to the aircraft includes:

[0015] Determining a Kalman gain matrix based on the Jacobian matrix, the covariance matrix, and the sensor noise matrix;

[0016] Based on the observation equation and the Kalman gain matrix, a correction operation is performed on the current state vector of the aircraft to obtain a target state vector corresponding to the aircraft.

[0017] Furthermore, the step of determining the observation equation and Jacobian matrix corresponding to the aircraft based on the IMU data includes:

[0018] Obtaining a velocity estimate corresponding to the aircraft and a rotation parameter from the world coordinate system to the IMU coordinate system;

[0019] Determine an observation equation corresponding to the aircraft based on the IMU data, the velocity estimate, and the rotation parameter;

[0020] A Jacobian matrix corresponding to the aircraft is determined based on the rotation parameters.

[0021] Furthermore, the step of determining the observation equation corresponding to the aircraft based on the IMU data, the velocity estimate and the rotation parameter comprises:

[0022] Based on the IMU data, obtaining a first measurement value and a first deviation value corresponding to an accelerometer of the aircraft, and a second measurement value and a second deviation value corresponding to a gyroscope;

[0023] An observation equation corresponding to the aircraft is determined based on the first measurement value, the first deviation value, the second measurement value, the second deviation value, the velocity estimate value, and the rotation parameter.

[0024] Further, the step of determining whether the aircraft meets a preset zero-speed correction condition based on the first operating state includes:

[0025] Obtain the second operating state and state buffer times corresponding to the aircraft at the previous moment;

[0026] If the first operating state is consistent with the second operating state, resetting the state buffering times;

[0027] Determine whether the first operating state is a stationary state, wherein if the first operating state is a stationary state, determine that the aircraft meets a preset zero-speed correction condition.

[0028] Furthermore, after the step of obtaining the second operating state and the number of state buffering times corresponding to the aircraft at the previous moment, the method further includes:

[0029] If the first running state is inconsistent with the second running state, updating the state buffering times;

[0030] If the updated state buffering times are greater than the preset buffering times, resetting the state buffering times;

[0031] Determine whether the first operating state is a stationary state, wherein if the first operating state is a stationary state, determine that the aircraft meets a preset zero-speed correction condition.

[0032] Furthermore, after the step of updating the state buffer times if the first running state is inconsistent with the second running state, the method further includes:

[0033] If the updated state buffering times are less than or equal to the preset buffering times, the second operating state is used as the first operating state;

[0034] Determine whether the first operating state is a stationary state, wherein if the first operating state is a stationary state, determine that the aircraft meets a preset zero-speed correction condition.

[0035] Furthermore, the step of determining the current first operating state of the aircraft based on the current image feature point data of the aircraft and the IMU data of the IMU inertial measurement unit includes:

[0036] Determine a first variance corresponding to the three-axis output value of the accelerometer and a second variance corresponding to the three-axis output value of the gyroscope based on the IMU data;

[0037] If the first variance is less than a first preset threshold and the second variance is less than a second preset threshold, determining that the first operating state is a stationary state;

[0038] If the first variance is greater than or equal to a first preset threshold or the second variance is greater than or equal to a second preset threshold, a current first operating state of the aircraft is determined based on the image feature point data.

[0039] Further, the step of determining the current first operating state of the aircraft based on the image feature point data includes:

[0040] Based on the image feature point data, determining continuous feature points corresponding to two adjacent frames of images;

[0041] Obtain the disparity corresponding to the continuous feature points, and obtain the average disparity corresponding to each disparity;

[0042] If the average parallax is smaller than a preset parallax, it is determined that the first operating state is a stationary state.

[0043] In addition, to achieve the above-mentioned purpose, the present application also provides an aircraft, the aircraft comprising:

[0044] A first determination module, configured to determine a current first operating state of the aircraft based on current image feature point data of the aircraft and IMU data of an IMU inertial measurement unit;

[0045] A second determination module, configured to determine whether the aircraft satisfies a preset zero speed correction condition based on the first operating state;

[0046] A third determination module is used to determine the observation equation and Jacobian matrix corresponding to the aircraft based on the IMU data if the aircraft meets the preset zero-speed correction condition;

[0047] The correction module is used to perform a correction operation on the current state vector of the aircraft based on the observation equation and the Jacobian matrix to obtain a target state vector corresponding to the aircraft.

[0048] In addition, to achieve the above-mentioned purpose, the present application also provides an aircraft control device, which includes: a memory, a processor, and an aircraft control program stored in the memory and executable on the processor, and the aircraft control program implements the steps of the aforementioned aircraft control method when executed by the processor.

[0049] In addition, to achieve the above-mentioned purpose, the present application also provides a computer-readable storage medium, on which an aircraft control program is stored, and when the aircraft control program is executed by a processor, the steps of the aforementioned aircraft control method are implemented.

[0050] The present application determines the current first operating state of the aircraft based on the current image feature point data of the aircraft and the IMU data of the IMU inertial measurement unit; then determines whether the aircraft meets the preset zero-speed correction condition based on the first operating state; and then if the aircraft meets the preset zero-speed correction condition, determines the observation equation and Jacobian matrix corresponding to the aircraft based on the IMU data; then, based on the observation equation and the Jacobian matrix, performs a correction operation on the current state vector of the aircraft to obtain the target state vector corresponding to the aircraft, and when it is determined that the aircraft meets the preset zero-speed correction condition through the current image feature point data and IMU data of the aircraft, performs zero-speed correction on the aircraft, which can reduce the misjudgment of the aircraft's operating state and improve the accuracy of the aircraft's zero-speed correction. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0052] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0053] Figure 1 A schematic diagram of a flow chart provided for the first embodiment of the aircraft control method of the present application;

[0054] Figure 2 This is a schematic diagram of the module structure of the aircraft according to the embodiment of the present application;

[0055] Figure 3 This is a schematic diagram of the module structure of the aircraft control device according to an embodiment of the present application.

[0056] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0057] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0058] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0059] The main solution of the present application is: based on the current image feature point data of the aircraft and the IMU data of the IMU inertial measurement unit, determine the current first operating state of the aircraft; based on the first operating state, determine whether the aircraft meets the preset zero-speed correction condition; if the aircraft meets the preset zero-speed correction condition, determine the observation equation and Jacobian matrix corresponding to the aircraft based on the IMU data; based on the observation equation and Jacobian matrix, perform correction operations on the current state vector of the aircraft to obtain the target state vector corresponding to the aircraft.

[0060] At present, in traditional vehicle navigation, the acceleration and angular velocity information of IMU is used to detect the stationary state of the carrier. When the carrier is stationary, the speed calculated by the system is used as the speed error term, and Kalman filter estimation is performed to complete the zero speed correction based on the updated state quantity estimate.

[0061] Using IMU information can easily lead to misjudgment of the static state of the carrier. When the acceleration or angular velocity noise of the IMU is too large, or when the body is slightly shaking in the air, it is easy to be misjudged as a moving state. When the body is moving at a constant speed, the acceleration information is in a degraded state and is easily misjudged as a static state.

[0062] In addition, in general SLAM methods, the zero-speed correction mode is immediately entered only through conventional optimization methods or by determining that the carrier is in a stationary state. When the aircraft is hovering in the air, due to the instability of the aircraft, it is easy to switch back and forth between the dynamic mode and the zero-speed correction mode. The switching back and forth between the two modes can easily lead to a jump in the system state.

[0063] Therefore, how to improve the accuracy of zero-speed correction is a problem that needs to be solved urgently.

[0064] The present application uses the current image feature point data IMU data of the aircraft to determine that the aircraft meets the preset zero-speed correction conditions, and then performs zero-speed correction on the aircraft, which can reduce the misjudgment of the aircraft's operating status and improve the accuracy of the aircraft's zero-speed correction.

[0065] It should be noted that the execution subject of this embodiment may be an aircraft, or a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an aircraft control device capable of realizing the above functions, etc., and this embodiment does not specifically limit this. The following takes the aircraft as the execution subject as an example to illustrate this embodiment and the following embodiments.

[0066] Based on this, the present application proposes a first embodiment of an aircraft control method, please refer to Figure 1 , the aircraft control method comprises steps S101 to S104:

[0067] Step S101, determining a current first operating state of the aircraft based on current image feature point data of the aircraft and IMU data of an IMU inertial measurement unit;

[0068] When the aircraft is stationary or hovering in the air, the aircraft obtains current image feature point data and IMU data of the IMU inertial measurement unit, wherein the image feature point data includes feature point data of multiple frames of images, and the aircraft can obtain the image feature point data after processing the captured images through the upstream module, and the IMU data includes accelerometer data and gyroscope data, the accelerometer data includes three-axis output values ​​of the three axes of the accelerometer, a first measurement value corresponding to the accelerometer, and a first deviation value, and the gyroscope data includes three-axis output values ​​of the three axes of the gyroscope, a second measurement value corresponding to the gyroscope, and a second deviation value.

[0069] After acquiring the IMU data and the image feature point data, the current first operating state of the aircraft is determined based on the image feature point data and the IMU data, wherein the first operating state includes a stationary state and a moving state. Specifically, based on the three-axis output values ​​of the three axes of the accelerometer and the three-axis output values ​​of the three axes of the gyroscope in the IMU data, it is determined whether the operating state of the aircraft is a stationary state. If so, it is determined that the first operating state is a stationary state. Otherwise, based on the image feature point data, it is determined whether the operating state of the aircraft is a stationary state. If so, it is determined that the first operating state is a stationary state. Otherwise, it is determined that the first operating state is a moving state.

[0070] Step S102, determining whether the aircraft meets a preset zero speed correction condition based on the first operating state;

[0071] After obtaining the first operating state, the aircraft determines whether the aircraft meets the preset zero-speed correction condition based on the first operating state. Specifically, the second operating state corresponding to the aircraft at the previous moment can be first obtained, and based on the first operating state and the second operating state, it is determined whether the aircraft meets the preset zero-speed correction condition.

[0072] Step S103, if the aircraft meets the preset zero-speed correction condition, determining the observation equation and Jacobian matrix corresponding to the aircraft based on the IMU data;

[0073] If the aircraft meets the preset zero-speed correction conditions, the observation equation and Jacobian matrix corresponding to the aircraft are determined based on the IMU data. Specifically, the aircraft first obtains the velocity estimate corresponding to the aircraft and the rotation parameters from the world coordinate system to the IMU coordinate system, and determines the observation equation and Jacobian matrix corresponding to the aircraft based on the IMU data, the velocity estimate and the rotation parameters.

[0074] Step S104: Based on the observation equation and the Jacobian matrix, a correction operation is performed on the current state vector of the aircraft to obtain a target state vector corresponding to the aircraft.

[0075] After obtaining the observation equation and the Jacobian matrix, the current state vector of the aircraft is corrected based on the observation equation and the Jacobian matrix to obtain the target state vector corresponding to the aircraft. Specifically, the covariance matrix and the sensor noise matrix corresponding to the aircraft can be obtained, and the Kalman gain matrix is ​​determined based on the Jacobian matrix, the covariance matrix and the sensor noise matrix. The current state vector of the aircraft is corrected according to the observation equation and the Kalman gain matrix to obtain the target state vector corresponding to the aircraft.

[0076] The current state vector of the aircraft includes at least the position, angle, speed, accelerometer deviation and gyroscope deviation of the aircraft.

[0077] The first current operating state of the aircraft is determined based on the current image feature point data of the aircraft and the IMU data of the IMU inertial measurement unit; then, based on the first operating state, it is determined whether the aircraft meets the preset zero-speed correction condition; then, if the aircraft meets the preset zero-speed correction condition, the observation equation and Jacobian matrix corresponding to the aircraft are determined based on the IMU data; then, based on the observation equation and the Jacobian matrix, the current state vector of the aircraft is corrected to obtain the target state vector corresponding to the aircraft, and when it is determined that the aircraft meets the preset zero-speed correction condition through the current image feature point data and IMU data of the aircraft, the aircraft is corrected at zero speed, which can reduce the misjudgment of the aircraft's operating state and improve the accuracy of the aircraft's zero-speed correction.

[0078] In a feasible implementation, step S101 may include steps S201 to S203:

[0079] Step S201, determining a first variance corresponding to the three-axis output value of the accelerometer and a second variance corresponding to the three-axis output value of the gyroscope based on the IMU data;

[0080] Step S202: if the first variance is smaller than a first preset threshold and the second variance is smaller than a second preset threshold, determining that the first operating state is a stationary state;

[0081] Step S203: If the first variance is greater than or equal to a first preset threshold or the second variance is greater than or equal to a second preset threshold, determining a current first operating state of the aircraft based on the image feature point data.

[0082] After obtaining the IMU data and the image feature point data, the aircraft obtains the three-axis output values ​​of the three-axis accelerometer and the three-axis output values ​​of the three-axis gyroscope in the IMU data, calculates the first variance based on the three-axis output values ​​of the three-axis accelerometer, and calculates the second variance based on the three-axis output values ​​of the three-axis gyroscope.

[0083] After obtaining the first variance and the second variance, determine whether the first variance is less than a first preset threshold, and determine whether the second variance is less than a second preset value. If the first variance is less than the first preset threshold and the second variance is less than the second preset threshold, determine that the first operating state is a stationary state; wherein the first preset value and the second preset value can be reasonably set according to the aircraft.

[0084] If the first variance is greater than or equal to the first preset threshold, or the second variance is greater than or equal to the second preset threshold, it indicates that the current operating state of the aircraft cannot be accurately determined based on the IMU data, and then the current first operating state of the aircraft is determined based on the image feature point data. Specifically, in a feasible implementation, step S203 may include steps S2031 to S2033:

[0085] Step S2031, determining continuous feature points corresponding to two adjacent frames of images based on the image feature point data;

[0086] Step S2032, obtaining the disparities corresponding to the continuous feature points, and obtaining the average disparity corresponding to each disparity;

[0087] Step S2033: if the average parallax is smaller than a preset parallax, determining that the first operating state is a stationary state.

[0088] Based on the feature point data of each frame image in the image feature point data, continuous feature points corresponding to two adjacent frames of images are determined, and disparity calculation is performed based on the continuous feature points to obtain the disparity corresponding to the continuous feature points, and the average disparity corresponding to each disparity is calculated.

[0089] After obtaining the average parallax, it is determined whether the average parallax is less than a preset parallax. If the average parallax is less than the preset parallax, the first operating state is determined to be a stationary state. If the average parallax is greater than or equal to the preset parallax, the first operating state is determined to be a moving state.

[0090] By determining the first variance corresponding to the three-axis output value of the accelerometer and the second variance corresponding to the three-axis output value of the gyroscope based on the IMU data, and then if the first variance is less than the first preset threshold and the second variance is less than the second preset threshold, it is determined that the first operating state is a stationary state; and then if the first variance is greater than or equal to the first preset threshold or the second variance is greater than or equal to the second preset threshold, the current first operating state of the aircraft is determined based on the image feature point data. The current operating state of the aircraft can be accurately judged through the IMU data and the image feature point data, further improving the accuracy of the zero-speed correction of the aircraft.

[0091] In a feasible implementation, step S102 may include steps S301 to S303:

[0092] Step S301, obtaining the second operating state and state buffering times corresponding to the aircraft at the previous moment;

[0093] Step S302: if the first operating state is consistent with the second operating state, resetting the state buffering times;

[0094] Step S303, determining whether the first operating state is a stationary state, wherein if the first operating state is a stationary state, determining that the aircraft meets a preset zero-speed correction condition.

[0095] After obtaining the first operating state, the aircraft obtains the second operating state and the state buffer count corresponding to the previous moment, and determines whether the first operating state is consistent with the second operating state. If the first operating state is consistent with the second operating state, the state buffer count is reset, for example, the state buffer count is set to 0.

[0096] Next, determine whether the first operating state is a stationary state. If the first operating state is a stationary state, determine that the aircraft meets the preset zero-speed correction condition. Then, whether the aircraft meets the preset zero-speed correction condition can be accurately determined based on the first operating state and the second operating state.

[0097] In a feasible implementation manner, after step S301, the aircraft control method may further include steps S304 to S306:

[0098] Step S304: if the first running state is inconsistent with the second running state, updating the state buffering times;

[0099] Step S305, if the updated state buffering times are greater than the preset buffering times, resetting the state buffering times;

[0100] Step S306, determining whether the first operating state is a stationary state, wherein if the first operating state is a stationary state, determining that the aircraft meets a preset zero-speed correction condition.

[0101] If the first operating state is inconsistent with the second operating state, the state buffer times are updated. Specifically, the state buffer times are increased by 1 to obtain the updated state buffer times, and it is determined whether the updated state buffer times are greater than the preset buffer times, wherein the preset buffer times can be reasonably set.

[0102] If the updated state buffering count is greater than the preset buffering count, the state buffering count is reset, for example, the state buffering count is set to 0.

[0103] Next, determine whether the first operating state is a stationary state. If the first operating state is a stationary state, determine that the aircraft meets the preset zero-speed correction condition, and then accurately judge whether the aircraft meets the preset zero-speed correction condition based on the preset buffer times, the first operating state and the second operating state.

[0104] In a feasible implementation manner, after step S304, the aircraft control method may further include steps S307 to S308:

[0105] Step S307: if the updated state buffering times are less than or equal to the preset buffering times, the second running state is used as the first running state;

[0106] Step S308, determining whether the first operating state is a stationary state, wherein if the first operating state is a stationary state, determining that the aircraft meets a preset zero-speed correction condition.

[0107] If the updated state buffer times are less than or equal to the preset buffer times, the second operating state is used as the first operating state, and it is determined whether the first operating state is a stationary state. If the first operating state is a stationary state, it is determined that the aircraft meets the preset zero-speed correction condition. Furthermore, it is possible to accurately determine whether the aircraft meets the preset zero-speed correction condition based on the preset buffer times, the first operating state, and the second operating state.

[0108] By obtaining the second operating state and the state buffer times corresponding to the previous moment of the aircraft; then if the first operating state is consistent with the second operating state, the state buffer times are reset; and then determining whether the first operating state is a stationary state, wherein if the first operating state is a stationary state, it is determined that the aircraft meets the preset zero-speed correction condition, and whether the aircraft meets the preset zero-speed correction condition can be accurately judged according to the first operating state and the second operating state, thereby further improving the accuracy of the aircraft zero-speed correction.

[0109] In a feasible implementation, step S103 may include steps S401 to S403:

[0110] Step S401, obtaining a velocity estimation value corresponding to the aircraft and a rotation parameter from a world coordinate system to an IMU coordinate system;

[0111] Step S402, determining an observation equation corresponding to the aircraft based on the IMU data, the velocity estimation value and the rotation parameter;

[0112] Step S403: determining the Jacobian matrix corresponding to the aircraft based on the rotation parameters.

[0113] When the aircraft meets the preset zero-speed correction condition, the speed estimation value corresponding to the aircraft and the rotation parameter from the world coordinate system to the IMU coordinate system are obtained, and the observation equation corresponding to the aircraft is determined based on the IMU data, the speed estimation value and the rotation parameter. Specifically, in a feasible implementation manner, step S402 may include steps S4021 to S4022:

[0114] Step S4021, based on the IMU data, obtaining a first measurement value and a first deviation value corresponding to an accelerometer of the aircraft, and a second measurement value and a second deviation value corresponding to a gyroscope;

[0115] Step S4022, determining the observation equation corresponding to the aircraft based on the first measurement value, the first deviation value, the second measurement value, the second deviation value, the speed estimation value and the rotation parameter.

[0116] The velocity estimation value and the rotation parameter are obtained, the first measurement value and the first deviation value corresponding to the accelerometer of the aircraft are obtained in the IMU data, and the second measurement value and the second deviation value corresponding to the gyroscope are obtained.

[0117] Next, based on the first measurement value, the first deviation value, the second measurement value, the second deviation value, the speed estimation value and the rotation parameter, the observation equation corresponding to the aircraft is determined. The formula of the observation equation corresponding to the aircraft is:

[0118]

[0119] in, is the observation equation, a m is the first measurement value corresponding to the accelerometer, ω m is the second measurement value corresponding to the gyroscope, b a is the first deviation value corresponding to the accelerometer, b ω is the second deviation value corresponding to the gyroscope, G is the gravity acceleration, v is the velocity estimation value, is the rotation parameter from the world coordinate system to the IMU coordinate system, and I is the unit matrix.

[0120] After obtaining the observation equation, the Jacobian matrix corresponding to the aircraft is determined based on the rotation parameters, wherein the formula of the Jacobian matrix is:

[0121]

[0122] Among them, H is the Jacobian matrix, G is the gravitational acceleration, is the rotation parameter from the world coordinate system to the IMU coordinate system, and I is the unit matrix.

[0123] By obtaining the velocity estimate corresponding to the aircraft and the rotation parameters from the world coordinate system to the IMU coordinate system; then determining the observation equation corresponding to the aircraft based on the IMU data, the velocity estimate and the rotation parameters; and then determining the Jacobian matrix corresponding to the aircraft based on the rotation parameters, the observation equation and Jacobian matrix corresponding to the aircraft can be obtained through the velocity estimate, IMU data and the rotation parameters, thereby further improving the accuracy of the aircraft's zero-speed correction.

[0124] In a feasible implementation, step S104 may include steps S501 to S502:

[0125] Step S501, obtaining a covariance matrix and a sensor noise matrix corresponding to the aircraft;

[0126] Step S502: Based on the observation equation, the Jacobian matrix, the covariance matrix and the sensor noise matrix, a correction operation is performed on the current state vector of the aircraft to obtain a target state vector corresponding to the aircraft.

[0127] After obtaining the observation equation and Jacobian matrix, the aircraft can obtain the corresponding covariance matrix and sensor noise matrix. The covariance matrix is ​​the system covariance matrix corresponding to the aircraft.

[0128] After obtaining the covariance matrix and the sensor noise matrix, the current state vector of the aircraft is obtained, and the current state vector of the aircraft is corrected according to the observation equation, the Jacobian matrix, the covariance matrix and the sensor noise matrix to obtain the target state vector corresponding to the aircraft. Specifically, in a feasible implementation, step S502 may include steps S5021 to S5022:

[0129] Step S5021, determining a Kalman gain matrix based on the Jacobian matrix, the covariance matrix, and the sensor noise matrix;

[0130] Step S5022: Based on the observation equation and the Kalman gain matrix, a correction operation is performed on the current state vector of the aircraft to obtain a target state vector corresponding to the aircraft.

[0131] After obtaining the covariance matrix and the sensor noise matrix, the Kalman gain matrix is ​​determined based on the Jacobian matrix, the covariance matrix and the sensor noise matrix. The formula of the Kalman gain matrix is:

[0132]

[0133] Where K is the Kalman gain matrix, H is the Jacobian matrix, and R is the sensor noise matrix. is the covariance matrix.

[0134] After obtaining the Kalman gain matrix, the current state vector of the aircraft is obtained, and based on the observation equation and the Kalman gain matrix, the current state vector of the aircraft is corrected to obtain the target state vector corresponding to the aircraft. The formula of the target state vector is:

[0135]

[0136] in, is the target state vector of the aircraft, is the current state vector of the aircraft, is the observation equation of the aircraft, and K is the Kalman gain matrix.

[0137] By obtaining the covariance matrix and sensor noise matrix corresponding to the aircraft; and then based on the observation equation, the Jacobian matrix, the covariance matrix and the sensor noise matrix, performing correction operations on the current state vector of the aircraft to obtain the target state vector corresponding to the aircraft, the current state vector of the aircraft can be accurately corrected to achieve zero-speed correction of the aircraft, thereby further improving the accuracy of zero-speed correction of the aircraft.

[0138] The present application also provides an aircraft, please refer to Figure 2 , the aircraft comprises:

[0139] A first determination module 10 is used to determine a current first operating state of the aircraft based on current image feature point data of the aircraft and IMU data of an IMU inertial measurement unit;

[0140] A second determination module 20, configured to determine whether the aircraft satisfies a preset zero speed correction condition based on the first operating state;

[0141] A third determination module 30 is used to determine the observation equation and Jacobian matrix corresponding to the aircraft based on the IMU data if the aircraft meets the preset zero-speed correction condition;

[0142] The correction module 40 is used to perform a correction operation on the current state vector of the aircraft based on the observation equation and the Jacobian matrix to obtain a target state vector corresponding to the aircraft.

[0143] The aircraft provided in the embodiment of the present application adopts the aircraft control method in the above embodiment, which can solve the technical problem of how to improve the accuracy of zero-speed correction. Compared with the prior art, the beneficial effects of the aircraft provided in the embodiment of the present application are the same as the beneficial effects of the aircraft control method provided in the above embodiment, and other technical features in the aircraft are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0144] The present application provides an aircraft control device, which includes: at least one processor; and a memory that is communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the aircraft control method in the above-mentioned embodiment one.

[0145] Reference below Figure 3 , which shows a schematic diagram of the structure of an aircraft control device suitable for implementing the embodiment of the present application. The aircraft control device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3 The aircraft control device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0146] like Figure 3As shown, the aircraft control device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the aircraft control device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the aircraft control device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an aircraft control device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0147] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0148] The aircraft control device provided by the present application adopts the aircraft control method in the above embodiment, which can solve the technical problem of how to improve the accuracy of zero-speed correction. Compared with the prior art, the beneficial effects of the aircraft control device provided by the present application are the same as the beneficial effects of the aircraft control method provided by the above embodiment, and the other technical features in the aircraft control device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0149] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0150] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0151] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the aircraft control method in the above-mentioned embodiment.

[0152] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0153] The computer-readable storage medium may be included in the aircraft control device; or may exist independently without being assembled into the aircraft control device.

[0154] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the aircraft control device, the aircraft control device: determines the current first operating state of the aircraft based on the current image feature point data of the aircraft and the IMU data of the IMU inertial measurement unit; determines whether the aircraft meets the preset zero-speed correction condition based on the first operating state; if the aircraft meets the preset zero-speed correction condition, determines the observation equation and Jacobian matrix corresponding to the aircraft based on the IMU data; based on the observation equation and the Jacobian matrix, performs a correction operation on the current state vector of the aircraft to obtain the target state vector corresponding to the aircraft.

[0155] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0156] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0157] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0158] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned aircraft control method, and can solve the technical problem of how to improve the accuracy of zero-speed correction. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the aircraft control method provided by the above-mentioned embodiment, and will not be repeated here.

[0159] An embodiment of the present application provides a computer program product, including a computer program, which implements the steps of the above-mentioned aircraft control method when executed by a processor.

[0160] The computer program product provided by the present application can solve the technical problem of how to improve the accuracy of zero-speed correction. Compared with the prior art, the beneficial effects of the computer program product provided by the embodiment of the present application are the same as the beneficial effects of the aircraft control method provided by the above embodiment, which will not be repeated here.

[0161] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for controlling an aircraft, characterized in that: The aircraft control method comprises the following steps: Determine a current first operating state of the aircraft based on current image feature point data of the aircraft and IMU data of an IMU inertial measurement unit; determining whether the aircraft satisfies a preset zero-speed correction condition based on the first operating state; If the aircraft meets the preset zero-speed correction condition, then determining the observation equation and Jacobian matrix corresponding to the aircraft based on the IMU data; Based on the observation equation and the Jacobian matrix, a correction operation is performed on the current state vector of the aircraft to obtain a target state vector corresponding to the aircraft.

2. The aircraft control method according to claim 1, characterized in that: The step of performing a correction operation on the current state vector of the aircraft based on the observation equation and the Jacobian matrix to obtain the target state vector corresponding to the aircraft includes: Obtaining a covariance matrix and a sensor noise matrix corresponding to the aircraft; Based on the observation equation, the Jacobian matrix, the covariance matrix and the sensor noise matrix, a correction operation is performed on the current state vector of the aircraft to obtain a target state vector corresponding to the aircraft.

3. The aircraft control method according to claim 2, characterized in that: The step of performing a correction operation on the current state vector of the aircraft based on the observation equation, the Jacobian matrix, the covariance matrix and the sensor noise matrix to obtain the target state vector corresponding to the aircraft comprises: Determining a Kalman gain matrix based on the Jacobian matrix, the covariance matrix, and the sensor noise matrix; Based on the observation equation and the Kalman gain matrix, a correction operation is performed on the current state vector of the aircraft to obtain a target state vector corresponding to the aircraft.

4. The aircraft control method according to claim 1, characterized in that: The step of determining the observation equation and Jacobian matrix corresponding to the aircraft based on the IMU data comprises: Obtaining a velocity estimate corresponding to the aircraft and a rotation parameter from the world coordinate system to the IMU coordinate system; Determine an observation equation corresponding to the aircraft based on the IMU data, the velocity estimate, and the rotation parameter; A Jacobian matrix corresponding to the aircraft is determined based on the rotation parameters.

5. The aircraft control method according to claim 4, characterized in that: The step of determining the observation equation corresponding to the aircraft based on the IMU data, the velocity estimate and the rotation parameter comprises: Based on the IMU data, obtaining a first measurement value and a first deviation value corresponding to an accelerometer of the aircraft, and a second measurement value and a second deviation value corresponding to a gyroscope; An observation equation corresponding to the aircraft is determined based on the first measurement value, the first deviation value, the second measurement value, the second deviation value, the velocity estimate value, and the rotation parameter.

6. The aircraft control method according to claim 1, characterized in that: The step of determining whether the aircraft meets a preset zero-speed correction condition based on the first operating state comprises: Obtain the second operating state and state buffer times corresponding to the aircraft at the previous moment; If the first operating state is consistent with the second operating state, resetting the state buffering times; Determine whether the first operating state is a stationary state, wherein if the first operating state is a stationary state, determine that the aircraft meets a preset zero-speed correction condition.

7. The aircraft control method according to claim 6, characterized in that: After the step of obtaining the second operating state and the state buffering times corresponding to the aircraft at the previous moment, the method further includes: If the first running state is inconsistent with the second running state, updating the state buffering times; If the updated state buffering times are greater than the preset buffering times, resetting the state buffering times; Determine whether the first operating state is a stationary state, wherein if the first operating state is a stationary state, determine that the aircraft meets a preset zero-speed correction condition.

8. The aircraft control method according to claim 7, characterized in that: After the step of updating the state buffer times if the first running state is inconsistent with the second running state, the method further includes: If the updated state buffering times are less than or equal to the preset buffering times, the second operating state is used as the first operating state; Determine whether the first operating state is a stationary state, wherein if the first operating state is a stationary state, determine that the aircraft meets a preset zero-speed correction condition.

9. The aircraft control method according to any one of claims 1 to 8, characterized in that: The step of determining the current first operating state of the aircraft based on the current image feature point data of the aircraft and the IMU data of the IMU inertial measurement unit comprises: Determine a first variance corresponding to the three-axis output value of the accelerometer and a second variance corresponding to the three-axis output value of the gyroscope based on the IMU data; If the first variance is less than a first preset threshold and the second variance is less than a second preset threshold, determining that the first operating state is a stationary state; If the first variance is greater than or equal to a first preset threshold or the second variance is greater than or equal to a second preset threshold, a current first operating state of the aircraft is determined based on the image feature point data.

10. The aircraft control method according to claim 9, characterized in that: The step of determining the current first operating state of the aircraft based on the image feature point data comprises: Based on the image feature point data, determining continuous feature points corresponding to two adjacent frames of images; Obtain the disparity corresponding to the continuous feature points, and obtain the average disparity corresponding to each disparity; If the average parallax is smaller than a preset parallax, it is determined that the first operating state is a stationary state.

11. An aircraft, characterized in that: The aircraft comprises: A first determination module, configured to determine a current first operating state of the aircraft based on current image feature point data of the aircraft and IMU data of an IMU inertial measurement unit; A second determination module, configured to determine whether the aircraft satisfies a preset zero speed correction condition based on the first operating state; A third determination module is used to determine the observation equation and Jacobian matrix corresponding to the aircraft based on the IMU data if the aircraft meets the preset zero-speed correction condition; The correction module is used to perform a correction operation on the current state vector of the aircraft based on the observation equation and the Jacobian matrix to obtain a target state vector corresponding to the aircraft.

12. An aircraft control device, characterized in that: The aircraft control device comprises: a memory, a processor, and an aircraft control program stored in the memory and executable on the processor. When the aircraft control program is executed by the processor, the steps of the aircraft control method according to any one of claims 1 to 10 are implemented.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an aircraft control program, and when the aircraft control program is executed by a processor, the steps of the aircraft control method according to any one of claims 1 to 10 are implemented.

Images