Visual inertia autonomous positioning method and system for fixed-wing unmanned aerial vehicle integrated with satellite positioning

By integrating satellite positioning, vision sensors and inertial measurement unit IMU data, dynamic initialization and feature point triangulation estimation methods are used to solve the problem of divergence and cumulative error of positioning results during flight of fixed-wing drones, achieving high-precision and reliable autonomous navigation.

CN119845257APending Publication Date: 2025-04-18THE 28TH RES INST OF CHINA ELECTRONICS TECH GROUP CORP
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411969455.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-12-30
Publication Date
2025-04-18

AI Technical Summary

Technical Problem

In the case of large measurement errors in vision sensors and inertial sensors, the positioning results are prone to divergence, which cannot meet the initialization conditions of fixed-wing drones during flight, and there are cumulative errors, which cannot provide reliable long-distance positioning.

Method used

By integrating satellite positioning, vision sensors and inertial measurement unit IMU data, dynamic initialization methods and feature point triangulation estimation are used to update the state to achieve autonomous navigation.

Benefits of technology

It improves the accuracy of autonomous positioning of fixed-wing drones in satellite positioning denial areas, realizes dynamic initialization during flight, reduces cumulative errors, and improves the stability and reliability of the system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119845257A_ABST
    Figure CN119845257A_ABST
Patent Text Reader

Abstract

The invention provides a fixed-wing unmanned aerial vehicle visual inertial autonomous positioning method and system fused with satellite positioning, and the method comprises the steps: 1, receiving the output of an inertial measurement unit IMU, satellite positioning data and visual sensor image data, and initializing an unmanned aerial vehicle state matrix; 2, predicting the state of the inertial measurement unit IMU in the state vector by using the acceleration and angular velocity obtained by the inertial measurement unit IMU; step 3, calculating a camera state and adding the camera state into a state vector, and expanding a state covariance at the same time; 4, three-dimensional feature points are estimated according to historical camera state triangularization; and step 5, using the feature points to restrain and update the state of more than two historical camera states. According to the invention, accurate positioning data in a satellite positioning denial environment can be provided for the fixed-wing unmanned aerial vehicle, and autonomous navigation positioning is realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of data fusion positioning, and in particular relates to a visual inertial autonomous positioning method and system for a fixed-wing unmanned aerial vehicle integrating satellite positioning. Background Art

[0002] Navigation and positioning technology plays an important role in production activities. Satellite positioning signals are easily affected by terrain, weather and complex electromagnetic field environments, resulting in ineffective positioning. Therefore, the demand for high-precision, high-reliability, low-cost autonomous navigation and positioning has seen unprecedented growth. Autonomous navigation and positioning focuses on the reliable positioning of sensor-carrying drones in satellite positioning denial environments. It usually uses environmental sensors such as vision and lasers combined with inertial navigation for relative positioning. Only after obtaining accurate location information can the drone be accurately navigated to the mission location. However, fixed-wing drones usually fly at higher altitudes and faster speeds. The measurement errors of visual sensors and inertial measurement unit IMU (Inertial Measurement Unit) data are higher than those of quadrotors, and mainstream visual navigation and positioning methods are not up to the task.

[0003] At present, the visual inertial navigation and positioning system still has the following problems:

[0004] When the measurement errors of visual sensors and inertial sensors are large, the estimated positioning results of existing methods tend to diverge, resulting in positioning failure. Existing methods mostly use static initialization, and fixed-wing drones cannot meet their initialization conditions during flight. Existing methods have cumulative errors and cannot provide reliable positioning during long-distance flights. Fixed-wing drones have limited space in the cabin, complex electromagnetic environment, and video images and other data are easily interfered by high-power radiation sources such as data links. Summary of the invention

[0005] Purpose of the invention: The technical problem to be solved by the present invention is to provide a visual inertial autonomous positioning method and system for fixed-wing UAVs integrated with satellite positioning in view of the deficiencies of the prior art. By integrating satellite positioning, visual sensors and inertial measurement unit (IMU) data, the accurate position and attitude of the carrier can be obtained to achieve autonomous navigation of the carrier.

[0006] The method comprises the following steps:

[0007] Step 1: Receive the output of the inertial measurement unit (IMU), satellite positioning data, and visual sensor image data to initialize the drone state matrix;

[0008] Step 2, using the acceleration and angular velocity obtained by the inertial measurement unit IMU to predict the state of the inertial measurement unit IMU in the state vector;

[0009] Step 3, calculate the camera state and add it to the state vector, and expand the state covariance;

[0010] Step 4, estimate the 3D feature points based on the historical camera state triangulation;

[0011] Step 5: Update the state using the constraints of the feature points on more than two historical camera states.

[0012] Furthermore, in step 1, the output of the inertial measurement unit IMU is expressed as:

[0013]

[0014] ω m =ω B +b g +n g (2)

[0015] Among them, a m ,ω m are the measurements of the accelerometer and the gyroscope, is the rotation matrix from the world coordinate system to the body coordinate system, a B ,ω B are the real acceleration and angular velocity of the object in the world coordinate system, g E is the gravitational acceleration in the world coordinate system, b a 、b g are the deviation of the accelerometer and the gyroscope, n a 、n g are the noise of the accelerometer and the noise of the gyroscope respectively.

[0016] Furthermore, step 1 also includes: the state vector of the visual inertial navigation system VINS includes the position, velocity, attitude, deviation and inverse depth of the feature point of the body, and the state equation is expressed as:

[0017] x k =f(x k-1 ,u k ,n k ) (3)

[0018] z k =h(x k ,m k ) (4)

[0019] Among them, x k is the state vector at the kth moment, u k is the IMU measurement value at the kth moment, n k is the process noise at the kth moment, z k is the observed value at the kth moment, mk is the observation noise at the kth moment, f is the nonlinear kinematic equation, and h is the nonlinear observation equation.

[0020] Furthermore, step 1 also includes: the nonlinear kinematic equation of the inertial measurement unit IMU is expressed as:

[0021]

[0022] Among them, p E 、v E , They are the position, velocity and posture of the body in the world coordinate system. They represent the first-order differential of the position, velocity, attitude, accelerometer and gyro deviation of the body in the world coordinate system respectively. Yes m The antisymmetric matrix of They are the bias noise of the accelerometer and the bias noise of the gyroscope;

[0023] Get the state recursion model:

[0024]

[0025] Where Δt is the sampling interval of the inertial measurement unit IMU, are the deviation noise of the accelerometer and the gyroscope at time k, respectively; k+1 represents the parameter state at Δt time after time k; e is a natural constant.

[0026] Furthermore, step 1 also includes: converting the feature points from the global coordinate system to the camera coordinate system using the following formula:

[0027] p C =R(p E -t) (15)

[0028] Among them, p E 、p C Respectively represent the position vector of the feature point in the global coordinate system and the position vector in the camera coordinate system, R represents the rotation matrix of the camera, and t represents the translation vector of the camera;

[0029] The position vector of the feature point is normalized and projected to the pixel coordinates using the following formula:

[0030]

[0031] p d =D(p u ) (17)

[0032] p p =Kpd (18)

[0033] Among them, p u represents the normalized coordinates of the feature points, p cz Indicates the depth of the feature point in the camera coordinate system; p d represents the distortion coordinates of the feature points, D represents the distortion model of the camera, and p p represents the pixel coordinates of the feature point, and K represents the intrinsic parameter matrix of the camera.

[0034] Furthermore, step 2 includes: using the following formula for prediction:

[0035]

[0036] in, is the state prediction at time k-1, f p is the state prediction function, u k is the control input.

[0037] Furthermore, in step 3, the camera state is calculated using the following formula:

[0038]

[0039] in C p f Represents the position of the feature point in the camera coordinate system, Represents the rotation matrix from the inertial measurement unit IMU to the camera, Represents the rotation matrix from the global coordinate system to the inertial measurement unit IMU, G p f Represents the position of the feature point in the global coordinate system, G p I Represents the position of the inertial measurement unit IMU in the global coordinate system, which is given by the historical state. G p I Represents the position of the inertial measurement unit IMU in the camera coordinate system.

[0040] Furthermore, in step 3, the camera state is added to the state vector as follows:

[0041]

[0042] in Indicates the state of the relationship measurement unit IMU, Indicates the camera status. represents the state of the feature point, and T represents the matrix transpose.

[0043] Furthermore, in step 3, the state covariance is expanded using the following formula:

[0044]

[0045] in represents the state prediction error covariance matrix, F k-1 is the state prediction Jacobian matrix, Q k-1 is the process noise covariance matrix.

[0046] Furthermore, step 4 includes:

[0047] Step 4-1, feature tracking: track feature points through image data and update the position of feature points observed by the camera in the image;

[0048] Step 4-2, feature triangulation: triangulate the image position of the feature point using the camera pose in the historical camera state to obtain the spatial coordinates of the feature point;

[0049] Step 4-3, null space mapping: map the feature points not included in the state vector to the left null space of the state vector.

[0050] Furthermore, step 5 includes: updating the state using the following formula:

[0051]

[0052] in, is the state estimate at time k, K k is the Kalman gain, z k is the observed data at time k, It is the observation model for state prediction.

[0053] Furthermore, the method of the present invention also includes step 6: when the UAV platform uses an autonomous positioning algorithm to obtain a position, continuously repeating steps 1 to 5 to continuously update the status.

[0054] Furthermore, the present invention also provides a fixed-wing UAV visual inertial autonomous positioning system integrated with satellite positioning, comprising a UAV platform sensor module, an onboard equipment and station communication module, and an onboard computer; the system is used to execute the steps of the method;

[0055] The UAV platform sensor module includes a global shutter infrared camera, a six-axis inertial measurement unit IMU and a magnetic compass, and a Beidou satellite navigation system BDS or a global positioning system GPS satellite positioning antenna;

[0056] The global shutter infrared camera has a resolution of 1280x1024 pixels or more and is used to collect visual sensor image data;

[0057] The six-axis inertial measurement unit IMU and magnetic compass have a publishing frequency of more than 180 Hz and are used to obtain inertial data and orientation data;

[0058] The Beidou satellite navigation system BDS or global positioning system GPS satellite positioning antenna is used to obtain satellite positioning data;

[0059] The onboard equipment and station communication modules include a data link module and its signal transmission antenna, a MIPI-CSI cable, a USB serial port, a TTL serial port and an RJ45 network cable;

[0060] The data link module and its signal transmitting antenna are used for real-time communication between the UAV and the ground control station;

[0061] The MIPI-CSI cable is used for real-time communication connection between the global shutter infrared camera and the onboard computer;

[0062] The USB serial port is used for real-time communication connection between the six-axis inertial measurement unit IMU and the onboard computer;

[0063] The TTL serial port is used for real-time communication connection between the flight controller and the data link module;

[0064] The RJ45 network cable is used for real-time communication connection between the onboard computer and the data link module;

[0065] The processor main frequency of the onboard computer is above 800MHz, and includes one or more MIPI interfaces, one or more USB interfaces and one or more RJ45 network ports;

[0066] The global shutter infrared camera is installed vertically downward under the nose of the aircraft, and the signal transmitting antenna is installed under the horizontal tail to reduce electromagnetic interference.

[0067] Compared with the prior art, the present invention has the following beneficial effects:

[0068] According to the characteristics of the scene, the accuracy of satellite positioning and the autonomy of visual inertial positioning are fully utilized to improve the accuracy of autonomous positioning of fixed-wing UAVs in satellite positioning denied areas.

[0069] The dynamic initialization method is adopted to enable the autonomous positioning algorithm to start running at any time on the fixed-wing platform.

[0070] A sensor module installation configuration mode on a fixed-wing platform is proposed to improve the stability and reliability of system operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.

[0072] Figure 1 It is a flow chart of the method of the present invention.

[0073] Figure 2 It is a system connection diagram of the present invention.

[0074] Figure 3 It is the hardware layout diagram of the system of the present invention. DETAILED DESCRIPTION

[0075] like Figure 1 As shown, in an embodiment of the present invention, a visual inertial autonomous positioning method for a fixed-wing UAV integrating satellite positioning is provided, comprising the following steps:

[0076] Step 1: Receive the output of the inertial measurement unit (IMU), satellite positioning data, and visual sensor image data to initialize the drone state matrix;

[0077] Step 2, using the acceleration and angular velocity obtained by the inertial measurement unit IMU to predict the state of the inertial measurement unit IMU in the state vector;

[0078] Step 3, calculate the camera state and add it to the state vector, and expand the state covariance;

[0079] Step 4, estimate the 3D feature points based on the historical camera state triangulation;

[0080] Step 5: Update the state using the constraints of the feature points on more than two historical camera states.

[0081] In step 1, the inertial measurement unit (IMU) is an inertial sensor that integrates an accelerometer, a gyroscope, and a magnetometer, and can measure the acceleration and angular velocity of an object. The accelerometer can measure the linear acceleration of an object in three axes, and the gyroscope can measure the angular acceleration of an object in three axes. The output of the inertial measurement unit (IMU) is expressed as:

[0082]

[0083] ω m =ω B +b g +n g (2)

[0084] Among them, a m ,ω m are the measurements of the accelerometer and the gyroscope, is the rotation matrix from the world coordinate system to the body coordinate system, aE ,ω B are the real acceleration and angular velocity of the object in the world coordinate system, g E is the gravitational acceleration in the world coordinate system, b a 、b g are the deviation of the accelerometer and the gyroscope, n a 、n g are the noise of the accelerometer and the noise of the gyroscope respectively.

[0085] In VINS, the measurement model of the inertial measurement unit (IMU) takes into account noise and deviation, and its measurement value can be used for state recursion of the inertial navigation system. The state vector of VINS includes information such as the position, velocity, attitude, deviation, and inverse depth of feature points of the body, and the state equation is expressed as:

[0086] x k =f(x k-1 ,u k ,n k ) (3)

[0087] z k =h(x k ,m k ) (4)

[0088] Among them, x k is the state vector at the kth moment, u k is the IMU measurement value at the kth moment, n k is the process noise at the kth moment, z k is the observed value at the kth moment, m k is the observation noise at the kth moment, f is the nonlinear kinematic equation, and h is the nonlinear observation equation.

[0089] The present invention provides a nonlinear kinematic equation and a discrete time state recursive model of an inertial measurement unit (IMU), in which Lie algebra and exponential mapping of Lie group are used. Lie algebra is an algebraic structure that describes the local properties of Lie group, and Lie group is a mathematical object with group operations and differential manifold structure. There is a one-to-one correspondence between Lie algebra and Lie group, which can be converted to each other through exponential mapping and logarithmic mapping. The nonlinear kinematic equation of the inertial measurement unit (IMU) is expressed as:

[0090]

[0091] Among them, p E 、v E , They are the position, velocity and posture of the body in the world coordinate system. Yes m The antisymmetric matrix of are the deviation noise of the accelerometer and gyroscope respectively. After discretizing the above equation, the state recursion model can be obtained:

[0092]

[0093] Where Δt is the sampling interval of the inertial measurement unit IMU, are the deviation noise at the kth moment respectively.

[0094] In summary, the measurement model of the inertial measurement unit (IMU) can be represented as a nonlinear function, the input of which is the state vector of the inertial measurement unit (IMU) and the system noise, and the output is the state vector of the inertial measurement unit (IMU). This model can be used to perform state recursion and optimization of the inertial navigation system.

[0095] Step 1 also includes: First, a transformation function for transforming feature points from the global coordinate system to the camera coordinate system is defined, and the function is given by the following formula:

[0096] p C =R(p E -t) (15)

[0097] Among them, p E 、p C Respectively represent the position vector of the feature point in the global coordinate system and the camera coordinate system, R represents the rotation matrix of the camera, and t represents the translation vector of the camera. This formula reflects the external parameters of the camera, that is, the posture and position of the camera relative to the global coordinate system. In the system of the present invention, the values ​​of R and t are:

[0098]

[0099] t=(0.47355 0.000091144 -0.10555) Secondly, the function of normalizing and projecting the position vector of the feature point to the pixel coordinates is described, which is given by the following formula:

[0100]

[0101] p d =D(p u ) (17)

[0102] p p =Kp d (18)

[0103] Among them, p u represents the normalized coordinates of the feature points, p cz Indicates the depth of the feature point in the camera coordinate system; p drepresents the distortion coordinates of the feature points, D represents the distortion model of the camera, and p p represents the pixel coordinates of the feature points, and K represents the intrinsic parameter matrix of the camera. These formulas reflect the intrinsic parameters of the camera, i.e., the focal length, principal point, distortion coefficient and other parameters of the camera. In the system of the present invention, the values ​​of K and D are:

[0104] K=(1053.364 1055.196 640 512)

[0105] D=(-0.34243 0.11055 -0.00019 -0.00004)

[0106] In summary, the measurement model of feature points can be expressed as a nonlinear function, the input of which is the global coordinates of the feature points, the camera's posture and position, the camera's internal and external parameters and distortion coefficients, and the output is the pixel coordinates of the feature points. This model can be used to estimate and optimize the state of VINS.

[0107] The MSCKF (Multi-State Constraint Kalman Filter) algorithm is an algorithm based on the Extended Kalman Filter (EKF). Its main purpose is to estimate the camera pose, inertial sensor bias and local map by real-time fusion of satellite positioning data, inertial measurement unit (IMU) and image data. The algorithm has high efficiency in real time and accurate attitude estimation, and is suitable for drones, robots and other fields.

[0108] Step 2 includes: using the following formula to make predictions:

[0109]

[0110] in, is the state prediction at time k-1, f p is the state prediction function, u k is the control input.

[0111] In step 3, the camera status is calculated using the following formula:

[0112]

[0113] in C p f Represents the position of the feature point in the camera coordinate system, Represents the rotation matrix from the inertial measurement unit IMU to the camera, Represents the rotation matrix from the global coordinate system to the inertial measurement unit IMU, G p f Represents the position of the feature point in the global coordinate system, G pI Represents the position of the inertial measurement unit IMU in the global coordinate system, which is given by the historical state. C p I Indicates the position of the inertial measurement unit IMU in the camera coordinate system;

[0114] Add the camera state to the state vector as follows:

[0115]

[0116] in Indicates the state of the relationship measurement unit IMU, Indicates the camera status. Represents the state of the feature point, T represents the matrix transpose;

[0117] The state covariance is expanded using the following formula:

[0118]

[0119] in represents the state prediction error covariance matrix, F k-1 is the state prediction Jacobian matrix, Q k-1 is the process noise covariance matrix.

[0120] Step 4 includes:

[0121] Step 4-1, feature tracking: track feature points through image data and update the position of feature points observed by the camera in the image;

[0122] Step 4-2, feature triangulation: triangulate the image position of the feature point using the camera pose in the historical camera state to obtain the spatial coordinates of the feature point;

[0123] Step 4-3, null space mapping: map the feature points not included in the state vector to the left null space of the state vector.

[0124] Step 5 includes: updating the state using the following formula:

[0125]

[0126] in, is the state estimate at time k, K k is the Kalman gain, z k is the observed data at time k, It is the observation model for state prediction.

[0127] The characteristic of MSCKF is that it marginalizes the feature points not included in the state vector through the null space mapping method, thereby improving the computational efficiency. This method effectively solves the dimensionality explosion problem of EKF in the VINS algorithm by reducing the state dimension.

[0128] The present invention also includes step 6: when the UAV platform uses the autonomous positioning algorithm to obtain the position, continuously repeating steps 1 to 5 to continuously update the status.

[0129] In another embodiment of the present invention, Figure 2 As shown, a fixed-wing UAV visual inertial autonomous positioning system integrating satellite positioning is provided, including a UAV platform sensor module, an onboard equipment and station communication module, and an onboard computer. Figure 3 As shown, the position layout of the main components of the UAV platform equipped with this system, in which the visual sensor needs to be installed vertically downward. Figure 2 In the example, IMU is an inertial measurement unit, MIPI-SCI is the connection line protocol between the visual sensor and the onboard computer, TTL and USB are serial communication methods, and RJ45 is a network interface.

[0130] The sensor module of the drone platform includes a global shutter infrared camera with a resolution of 1280x1024 pixels. This system uses a VEYE SC130m camera to collect visual sensor image data; a six-axis inertial measurement unit IMU and a magnetic compass with a release frequency of 180hz to obtain inertial data and orientation data;

[0131] BDS or GPS satellite positioning antenna, used to obtain satellite positioning data.

[0132] The communication modules between onboard equipment and between aircraft and stations include a data link module for real-time communication between the UAV and the ground control station; a MIPI-CSI cable for real-time communication between the global shutter infrared camera and the onboard computer; a USB serial port for real-time communication between the inertial measurement unit IMU and the onboard computer; a TTL serial port for real-time communication between the flight controller and the data link module; and an RJ45 network cable for real-time communication between the onboard computer and the data link module.

[0133] The main frequency of the onboard computer processor is at least 800 MHz. This system uses NVIDIA Jetson Xavier NX as the onboard computer, which includes at least one MIPI interface, one USB interface and one RJ45 network port.

[0134] Among them, the visual sensor is installed vertically downward under the nose of the aircraft, and the data link signal transmitting antenna is installed under the horizontal tail to reduce electromagnetic interference.

[0135] The present invention provides a method and system for visual inertial autonomous positioning of fixed-wing UAVs integrated with satellite positioning. There are many methods and ways to implement the technical solution. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention. All components not specified in this embodiment can be implemented using existing technologies.

Claims

1. A visual inertial autonomous positioning method for fixed-wing UAV integrated with satellite positioning, characterized in that: The following steps are involved: Step 1: Receive the output of the inertial measurement unit (IMU), satellite positioning data, and visual sensor image data to initialize the drone state matrix; Step 2, using the acceleration and angular velocity obtained by the inertial measurement unit IMU to predict the state of the inertial measurement unit IMU in the state vector; Step 3, calculate the camera state and add it to the state vector, and expand the state covariance; Step 4, estimate the 3D feature points based on the historical camera state triangulation; Step 5: Update the state using the constraints of the feature points on more than two historical camera states.

2. The method according to claim 1, characterized in that In step 1, the output of the inertial measurement unit IMU is expressed as: oh m =ω B +b g +n g (2) Among them, a m ,ω m are the measurements of the accelerometer and the gyroscope, is the rotation matrix from the world coordinate system to the body coordinate system, a E ,ω B are the real acceleration and angular velocity of the object in the world coordinate system, g E is the gravitational acceleration in the world coordinate system, b a , b g are the deviation of the accelerometer and the gyroscope, n a 、n g are the noise of the accelerometer and the noise of the gyroscope respectively.

3. The method according to claim 2, characterized in that Step 1 also includes: the state vector of the visual inertial navigation system VINS includes the position, velocity, attitude, deviation and inverse depth of the feature point of the body, and the state equation is expressed as: x k =f(x k-1 ,u k ,n k ) (3) z k =h(x k ,m k ) (4) Among them, x k is the state vector at the kth moment, y k is the IMU measurement value at the kth moment, n k is the process noise at the kth moment, z k is the observed value at the kth moment, m k is the observation noise at the kth moment, f is the nonlinear kinematic equation, and h is the nonlinear observation equation; The nonlinear kinematic equation of the inertial measurement unit IMU is expressed as: Among them, p E 、v E , They are the position, velocity and posture of the body in the world coordinate system. They represent the first-order differential of the position, velocity, attitude, accelerometer and gyro deviation of the body in the world coordinate system respectively. Yes m The antisymmetric matrix of They are the bias noise of the accelerometer and the bias noise of the gyroscope; Get the state recursion model: Where Δt is the sampling interval of the inertial measurement unit IMU, are the deviation noise of the accelerometer and the gyroscope at time k, respectively; k+1 represents the parameter state at Δt time after time k; e is a natural constant.

4. The method according to claim 3, characterized in that Step 1 also includes: converting the feature points from the global coordinate system to the camera coordinate system using the following formula: p C =R(p E -t) (15) Among them, p E 、p C Respectively represent the position vector of the feature point in the global coordinate system and the position vector in the camera coordinate system, R represents the rotation matrix of the camera, and t represents the translation vector of the camera; The position vector of the feature point is normalized and projected to the pixel coordinates using the following formula: p d =D(p u ) (17) p p =Kp d (18) Among them, p u represents the normalized coordinates of the feature points, p cz Indicates the depth of the feature point in the camera coordinate system; p d represents the distortion coordinates of the feature points, D represents the distortion model of the camera, and p p represents the pixel coordinates of the feature point, and K represents the intrinsic parameter matrix of the camera.

5. The method according to claim 4, characterized in that Step 2 includes: using the following formula to make predictions: in, is the state prediction at time k-1, f p is the state prediction function, u k is the control input.

6. The method according to claim 5, characterized in that In step 3, the camera status is calculated using the following formula: in C p f Represents the position of the feature point in the camera coordinate system, Represents the rotation matrix from the inertial measurement unit IMU to the camera, Represents the rotation matrix from the global coordinate system to the inertial measurement unit IMU, G p f represents the position of the feature point in the global coordinate system, G p I Represents the position of the inertial measurement unit IMU in the global coordinate system, which is given by the historical state. C p I Indicates the position of the inertial measurement unit IMU in the camera coordinate system; Add the camera state to the state vector as follows: in Indicates the state of the relationship measurement unit IMU, Indicates the camera status. Represents the state of the feature point, T represents the matrix transpose; The state covariance is expanded using the following formula: in represents the state prediction error covariance matrix, F k-1 is the state prediction Jacobian matrix, Q k-1 is the process noise covariance matrix.

7. The method according to claim 6, characterized in that Step 4 includes: Step 4-1, feature tracking: track feature points through image data and update the position of feature points observed by the camera in the image; Step 4-2, feature triangulation: triangulate the image position of the feature point using the camera pose in the historical camera state to obtain the spatial coordinates of the feature point; Step 4-3, null space mapping: map the feature points not included in the state vector to the left null space of the state vector.

8. The method according to claim 7, characterized in that Step 5 includes: updating the state using the following formula: in, is the state estimate at time k, K k is the Kalman gain, z k is the observed data at time k, It is the observation model for state prediction.

9. The method according to claim 8, characterized in that The method also includes step 6: when the UAV platform uses the autonomous positioning algorithm to obtain the position, continuously repeating steps 1 to 5 to continuously update the status.

10. A fixed-wing UAV visual inertial autonomous positioning system integrating satellite positioning, characterized in that: The system comprises a sensor module for an unmanned aerial vehicle platform, a communication module between onboard equipment and between stations, and an onboard computer; the system is used to execute the steps of the method as claimed in any one of claims 1 to 9; The UAV platform sensor module includes a global shutter infrared camera, a six-axis inertial measurement unit IMU and a magnetic compass, and a Beidou satellite navigation system BDS or a global positioning system GPS satellite positioning antenna; The global shutter infrared camera has a resolution of 1280x1024 pixels or more and is used to collect visual sensor image data; The six-axis inertial measurement unit IMU and magnetic compass have a publishing frequency of more than 180 Hz and are used to obtain inertial data and orientation data; The Beidou satellite navigation system BDS or global positioning system GPS satellite positioning antenna is used to obtain satellite positioning data; The onboard equipment and station communication modules include a data link module and its signal transmission antenna, a MIPI-CSI cable, a USB serial port, a TTL serial port and an RJ45 network cable; The data link module and its signal transmitting antenna are used for real-time communication between the UAV and the ground control station; The MIPI-CSI cable is used for real-time communication connection between the global shutter infrared camera and the onboard computer; The USB serial port is used for real-time communication connection between the six-axis inertial measurement unit IMU and the onboard computer; The TTL serial port is used for real-time communication connection between the flight controller and the data link module; The RJ45 network cable is used for real-time communication connection between the onboard computer and the data link module; The processor main frequency of the onboard computer is above 800MHz, and includes one or more MIPI interfaces, one or more USB interfaces and one or more RJ45 network ports; The global shutter infrared camera is installed vertically downward under the nose of the aircraft, and the signal transmitting antenna is installed under the horizontal tail to reduce electromagnetic interference.

Citation Information

Cited By

  • A method for identifying unmanned aerial vehicle routes

    CN120355748B