Unmanned aerial vehicle flight pose calibration method and device based on multi-sensor fusion

Abstract

The invention discloses an unmanned aerial vehicle flight pose calibration method and device based on multi-sensor fusion, and belongs to the field of dynamic pose measurement. Comprising a plurality of integrated dynamic pose measurement base stations which are arranged around a measurement site and cover all-region signals, and each base station comprises an infrared positioning module, an ultra-wideband reference module and the like; the miniaturized multi-sensor fusion target is installed on a motion carrier of an unmanned aerial vehicle and comprises an inertial navigation measurement unit, an infrared lamp bead, an ultra wide band master control module and the like. The upper computer is used for fusing the infrared positioning data, the ultra-wideband data and the inertial navigation measurement data; and when the positioning confidence of the infrared positioning data is lower than a threshold value, replacing original data with pseudo-infrared positioning data generated by a pseudo-infrared positioning model for fusion. According to the invention, the pose of the unmanned aerial vehicle can be monitored in real time, the problem of large-field-of-view, high-precision and full-angle dynamic pose measurement is solved, various complex dynamic scenes are fully covered, and the accuracy and reliability of dynamic pose measurement are effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of dynamic posture measurement, and in particular to a method and device for calibrating the flight posture of an unmanned aerial vehicle (UAV) based on multi-sensor fusion. Background Art

[0002] Dynamic posture measurement refers to the real-time and continuous tracking of the position and posture changes of the target object. It is crucial for achieving precise control, real-time interaction, state assessment and safety assurance. It is widely used in industrial automation, robotics, AR / VR, motion capture, structural health monitoring and other fields.

[0003] Currently, mature technologies for achieving high-precision dynamic pose measurement primarily include optical motion capture systems, laser trackers, and high-precision inertial navigation systems. Optical motion capture systems rely on triangulation of reflective markers, offering high accuracy and low latency. However, they are costly, complex to deploy, and have limited application scope, requiring the markers to remain visible. Laser trackers typically track only one point or target at a time, have a limited dynamic range, are expensive, and require complex operation. High-precision inertial navigation systems can provide highly accurate pose information, but are also costly and require additional sensors for correction. Advances in machine vision, infrared positioning, ultra-wideband positioning, and inertial measurement technologies have made low-cost dynamic pose measurement feasible. Machine vision and infrared positioning technologies offer high sampling frequency, high accuracy, and low cost, but are susceptible to obstructions and cannot cover the entire operating range. Ultra-wideband positioning technology offers high environmental adaptability, but its accuracy lags significantly behind these technologies. Inertial measurement technology offers strong autonomy and high short-term accuracy, but suffers from cumulative error and divergence over the long term. These low-cost solutions (single sensor or simple fusion) are difficult to meet the growing application requirements in terms of dynamic performance, accuracy, robustness and resistance to cumulative errors. There are still many challenges in building a low-cost, high-dynamic performance, high-robustness and easy-to-deploy multi-sensor fusion dynamic pose measurement method. Summary of the Invention

[0004] In order to solve the above problems, the present invention provides a UAV flight attitude calibration method and device based on multi-sensor fusion. The present invention generates the UAV attitude by fusing infrared positioning data, ultra-wideband data and inertial navigation measurement data, and considers the positioning confidence of the infrared positioning data. The infrared positioning data that does not meet the confidence requirements is replaced by "pseudo data" to improve the accuracy and reliability of dynamic attitude measurement.

[0005] The technical solution adopted in the present invention is as follows:

[0006] In a first aspect, the present invention proposes a UAV flight posture calibration device based on multi-sensor fusion, comprising:

[0007] Multiple integrated dynamic posture measurement base stations are arranged around the measurement site and provide full-area signal coverage. Each base station includes an infrared positioning module for collecting target infrared signals, an ultra-wideband reference module for transmitting and receiving ultra-wideband signals through a first antenna, and a first control module for synchronously processing infrared and ultra-wideband data.

[0008] A miniaturized multi-sensor fusion target, mounted on a UAV motion carrier, includes an inertial navigation measurement unit (INU), infrared lamps, an ultra-wideband (UWB) main control module that communicates with a base station via a second antenna, and a second control module that synchronously collects and transmits inertial navigation measurement data and UWB data. The infrared lamps serve as infrared positioning targets.

[0009] The host computer synchronously receives the infrared positioning data output by the first control module and the ultra-wideband data and inertial navigation measurement data output by the second control module, and fuses them to generate the UAV posture. During the fusion, the positioning confidence of the infrared positioning data is first calculated. When the confidence is not lower than the threshold, the infrared positioning data, ultra-wideband data and inertial navigation measurement data are directly fused. Otherwise, the pseudo-infrared positioning data generated by the pseudo-infrared positioning model is used to replace the original infrared positioning data for fusion.

[0010] Furthermore, the evaluation function used to calculate the positioning reliability of infrared positioning data is the weighted value of the infrared signal strength item and the infrared lamp bead visibility item, wherein the infrared signal strength item is the ratio of the current infrared signal strength to the historical maximum infrared signal strength, and the infrared lamp bead visibility item is the ratio of the current number of visible infrared lamp bead points to the total number of infrared lamp beads.

[0011] Furthermore, the infrared positioning module includes a lens barrel, an infrared fill light located outside the lens barrel, a CCD image sensor located inside the lens barrel, an 800~2500nm infrared filter, a free-form surface lens and a lens combination, and the free-form surface lens is designed based on point-by-point construction theory.

[0012] Furthermore, the miniaturized multi-sensor fusion target includes a transparent shell, an ultra-wideband main control module, an inertial measurement unit, a second control module, infrared lamp beads located on the inner walls around the transparent shell, and a second antenna located on the top of the transparent shell.

[0013] Furthermore, the process of acquiring ultra-wideband data includes:

[0014] The ultra-wideband reference modules in multiple integrated dynamic posture measurement base stations serve as spatial position reference points, with the number being no less than 3; the ultra-wideband main control module of the miniaturized multi-sensor fusion target serves as a mobile signal source, actively transmitting a pulse signal to the first antenna through the second antenna and transmitting it to the first control module, calculating the distance information in the first control module, sending the calculated distance information back to the second antenna, and calculating the ultra-wideband data in the second control module.

[0015] In a second aspect, the present invention proposes a multi-sensor fusion UAV flight posture calibration method, comprising:

[0016] Infrared positioning data, ultra-wideband data, and inertial navigation measurement data are synchronously acquired through multiple integrated dynamic posture measurement base stations and a miniaturized multi-sensor fusion target installed on a UAV motion carrier;

[0017] Calculate the dynamic positioning confidence of infrared positioning data. When the confidence is not lower than the threshold, use adaptive robust Kalman filtering to fuse infrared positioning data, ultra-wideband data and inertial navigation measurement data. Its state vector is defined as , the observation residual equation is defined as ,in represents the state at time k, Indicates the three-dimensional coordinate error between infrared positioning data and inertial navigation measurement data. Indicates the three-dimensional attitude error between infrared positioning data and inertial navigation measurement data. Indicates the constant drift error of the accelerometer in the inertial measurement unit, Indicates the constant drift error of the gyroscope in the inertial measurement unit, 、 、 Respectively represent the three-dimensional coordinates of infrared positioning data, inertial navigation measurement data, and ultra-wideband data positioning; the fused posture is converted into the UAV posture to form the UAV flight trajectory measurement value;

[0018] In addition, a pseudo infrared positioning model is trained based on the LSTM network;

[0019] When the confidence level is lower than the threshold, the pseudo infrared positioning data generated by the pseudo infrared positioning model is used to replace the original infrared positioning data for fusion;

[0020] The UAV flight posture is calibrated based on the UAV flight trajectory measurement value and the given flight trajectory.

[0021] Furthermore, when training the pseudo-infrared positioning model, the three-dimensional coordinates and three-dimensional attitude angles of ultra-wideband data positioning, as well as the three-axis angular velocity and three-axis acceleration in the inertial navigation measurement data are used as input to predict the three-dimensional coordinates and three-dimensional attitude of the infrared positioning data positioning, and the prediction results are used as pseudo-infrared positioning data.

[0022] Furthermore, the infrared positioning data includes the infrared positioning signal strength, the number of visible infrared lamp beads, the three-dimensional coordinates and three-dimensional posture of the infrared positioning target in the global coordinate system; the ultra-wideband data includes the three-dimensional coordinates and three-dimensional posture angle of the ultra-wideband target in the global coordinate system; the inertial navigation measurement data includes three-axis angular velocity and three-axis acceleration, and the three-dimensional coordinates and three-dimensional posture of the inertial navigation measurement unit in the global coordinate system are obtained through inertial navigation solution; the differences in the installation positions of the infrared positioning target, ultra-wideband target and inertial navigation measurement unit in the miniaturized multi-sensor fusion target are negligible.

[0023] Furthermore, when adaptive robust Kalman filtering is used to fuse infrared positioning data, ultra-wideband data and inertial navigation measurement data, the current state vector is used as the initial state, the initial state is iteratively optimized, and the inertial navigation measurement data is corrected according to the final optimized state.

[0024] Furthermore, the corrected inertial navigation measurement data refers to taking the sum of the three-dimensional coordinates obtained by the inertial navigation solution and the three-dimensional coordinate errors in the optimized state as the corrected three-dimensional coordinates, and taking the sum of the three-dimensional posture obtained by the inertial navigation solution and the three-dimensional posture errors in the optimized state as the corrected three-dimensional posture; the corrected result is used as the fused posture.

[0025] The present invention has the following beneficial effects:

[0026] The present invention utilizes infrared positioning sensors, ultra-wideband positioning sensors, and inertial navigation sensors to establish an integrated dynamic posture measurement base station and a miniaturized multi-sensor fusion target. The integrated dynamic posture measurement base station is deployed around the site, and the miniaturized multi-sensor fusion target is installed on the drone carrier. The output data of each sensor is synchronously collected to locate the drone's position. An infrared positioning reliability evaluation function is established. When the infrared positioning reliability exceeds a threshold, the output results of the infrared positioning sensor, ultra-wideband positioning sensor, and inertial navigation sensor are fused. The errors of the ultra-wideband positioning sensor and inertial navigation sensor are learned and modeled based on deep learning technology to generate "pseudo-infrared positioning information." When the infrared positioning reliability falls below the threshold due to factors such as motion occlusion and exceeding the field of view, the "pseudo-infrared positioning information" is fused with the output results of the ultra-wideband positioning sensor and inertial navigation sensor. The present invention can monitor the flight posture of drones in real time, solving the problem of dynamic posture measurement of drones with a large field of view, high precision, and full angle. It comprehensively covers various complex dynamic scenes and effectively improves the accuracy and reliability of dynamic posture measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a schematic diagram of a UAV flight posture calibration device based on multi-sensor fusion in this embodiment;

[0028] Figure 2 It is a structural diagram of the integrated dynamic posture measurement base station;

[0029] Figure 3 It is a partial diagram of the integrated dynamic posture measurement base station;

[0030] Figure 4 It is a schematic diagram of a lens combination;

[0031] Figure 5 It is a schematic diagram of the structure of a miniaturized multi-sensor fusion target;

[0032] Figure 6 is a schematic diagram of the inertial measurement unit;

[0033] In the figure: 1-UAV, 2-integrated dynamic posture measurement base station, 21-first antenna, 22-first control module, 23-lens barrel, 231-first aspheric lens, 232-first spherical lens, 233-second aspheric lens, 234-aperture, 235-third aspheric lens, 236-second spherical lens, 237-fourth aspheric lens, 238-free-form surface lens, 24-mounting stud, 25-infrared fill light, 26-outer cover, 27-housing, 28-ultra-wideband reference module, 3-miniaturized multi-sensor fusion target, 31-inertial navigation measurement unit, 311-gyroscope, 312-accelerometer, 32-infrared lamp bead, 33-second antenna, 34-ultra-wideband main control module, 35-second control module, 36-transparent housing. DETAILED DESCRIPTION

[0034] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0035] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features indicated. Thus, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "plurality" means two or more.

[0036] In the description of the present invention, it should be noted that, unless otherwise specified or limited, the term "mounted" should be understood in a broad sense. For example, it can mean a fixed connection, a detachable connection, or an integral connection; it can mean a mechanical connection or an electrical connection; it can mean a direct connection, an indirect connection through an intermediate medium, or internal communication between two components. Those skilled in the art will be able to understand the specific meanings of the above terms in the present invention in light of specific circumstances.

[0037] The accompanying drawings show various structural schematic diagrams of the embodiments disclosed in the present invention. These drawings are not drawn to scale, and some details are exaggerated and may be omitted for the purpose of clarity.

[0038] like Figure 1 As shown, the present invention proposes a UAV flight attitude calibration device based on multi-sensor fusion. The device primarily comprises multiple integrated dynamic attitude measurement base stations 2 and a miniaturized multi-sensor fusion target 3 mounted on the motion carrier of a UAV 1. The multiple integrated dynamic attitude measurement base stations are arranged around the measurement site to provide full-area signal coverage. Each base station includes an infrared positioning module for collecting target infrared signals, an ultra-wideband reference module for transmitting and receiving ultra-wideband signals via a first antenna, and a first control module for synchronously processing infrared and ultra-wideband data.

[0039] like Figure 2 and Figure 3 As shown, the integrated dynamic posture measurement base station includes a first antenna 21, a first control module 22, an infrared positioning module, an ultra-wideband reference module 28, an outer cover 26 and a housing 27 connected by mounting studs 24. The first antenna 21 is installed above the housing to achieve ultra-wideband signal transmission. The first control module 22 inside the housing is responsible for controlling the ultra-wideband reference module 29 and collecting information output by the infrared positioning module.

[0040] The infrared positioning module includes a lens barrel 23, an infrared fill light 25 located outside the lens barrel 23, and a CCD image sensor 2310, an 800-2500nm infrared filter 239 and a lens assembly located inside the lens barrel. Figure 4As shown, in this embodiment, the lens assembly includes a first aspheric lens 231, a first spherical lens 232, a second aspheric lens 233, an aperture 234, a third aspheric lens 235, a second spherical lens 236, a fourth aspheric lens 237, and a free-form lens 238, arranged along the optical axis from the object side to the image side. The free-form lens is designed based on point-by-point construction theory. The center of the lens barrel is aligned with the center of the CCD image sensor 2310. An 800-2500nm infrared filter is installed on the end of the lens barrel closest to the CCD image sensor 2310, effectively filtering visible light and thus improving infrared positioning performance. The lens barrel houses four aspherical lenses, two spherical lenses, and one free-form lens. The free-form lens is located in front of the infrared filter. Designed based on point-by-point construction theory, it corrects light at the edges, minimizing the inherent edge distortion in imaging and enabling precise control of the light propagation path. In front of the free-form lens are, in order, a fourth aspherical lens 237, a second spherical lens 236, a third aspherical lens 235, an aperture 234, a second aspherical lens 233, a first spherical lens 232, and a first aspherical lens 231. An infrared fill light 25 is mounted in front of the first aspherical lens 231 to enhance infrared imaging. The infrared fill light is fixed to the outer cover 26. The housing of the integrated dynamic posture measurement base station can be mounted on an adjustable stabilizing bracket, allowing for height and angle adjustments.

[0041] The miniaturized multi-sensor fusion target is installed on the UAV motion carrier, such as Figure 5 As shown, the miniaturized multi-sensor fusion target includes a transparent housing 36, an inertial navigation measurement unit 31 installed in the housing, infrared lamp beads 32, an ultra-wideband main control module 34 that communicates with the base station via a second antenna 33, and a second control module 35 that synchronously collects and transmits inertial navigation measurement data and ultra-wideband data. The infrared lamp beads serve as infrared positioning targets, and the second antenna 33 is located outside the housing, above the transparent housing, to achieve ultra-wideband signal transmission. Infrared lamp beads 32 are installed around the transparent housing, and the ultra-wideband main control module 34, the inertial navigation measurement unit 31, etc. are controlled by the second control module 35, which is responsible for synchronously collecting the output information and transmitting the information to the host computer in real time. In this embodiment, information transmission can be achieved by wireless communication.

[0042] Here, the process of acquiring ultra-wideband data includes:

[0043] The ultra-wideband reference modules in multiple integrated dynamic posture measurement base stations serve as spatial position reference points, with the number being no less than 3; the ultra-wideband main control module of the miniaturized multi-sensor fusion target serves as a mobile signal source, actively transmitting a pulse signal to the first antenna through the second antenna and transmitting it to the first control module, calculating the distance information in the first control module, sending the calculated distance information back to the second antenna, and calculating the ultra-wideband data in the second control module.

[0044] like Figure 6 As shown, the inertial navigation measurement unit 31 includes a gyroscope 311 and an accelerometer 312 .

[0045] The host computer synchronously receives the infrared positioning data output by the first control module and the ultra-wideband data and inertial navigation measurement data output by the second control module and fuses them to generate the UAV posture; when fusing, the positioning confidence of the infrared positioning data is first calculated; when the confidence is not lower than a threshold, the infrared positioning data, ultra-wideband data and inertial navigation measurement data are directly fused; otherwise, the pseudo-infrared positioning data generated by the pseudo-infrared positioning model is used to replace the original infrared positioning data for fusion.

[0046] In this embodiment, the evaluation function used to calculate the positioning reliability of infrared positioning data is the weighted value of the infrared signal strength item and the infrared lamp bead visibility item. The infrared signal strength item is the ratio of the current infrared signal strength to the historical maximum infrared signal strength, and the infrared lamp bead visibility item is the ratio of the current number of visible infrared lamp beads to the total number of infrared lamp beads.

[0047] The method for calibrating the UAV flight posture using the above-mentioned UAV flight posture calibration device based on multi-sensor fusion is as follows:

[0048] S1, synchronously acquiring infrared positioning data, ultra-wideband data, and inertial navigation measurement data through multiple integrated dynamic posture measurement base stations and a miniaturized multi-sensor fusion target; the miniaturized multi-sensor fusion target is installed on a UAV motion carrier;

[0049] S2, calculate the dynamic positioning reliability of infrared positioning data;

[0050] The infrared positioning reliability evaluation function is:

[0051]

[0052] in, is the current infrared signal strength, This is the highest infrared signal strength in history. The number of visible points of the current infrared lamp beads, is the total number of infrared lamp beads, 、 is the weight coefficient.

[0053] When the confidence level is not lower than the threshold, the adaptive robust Kalman filter is used to fuse infrared positioning data, ultra-wideband data and inertial navigation measurement data, and its state vector is defined as , the observation residual equation is defined as ,in represents the state at time k, Indicates the three-dimensional coordinate error between infrared positioning data and inertial navigation measurement data. Indicates the three-dimensional attitude error between infrared positioning data and inertial navigation measurement data. Indicates the constant drift error of the accelerometer in the inertial measurement unit, Indicates the constant drift error of the gyroscope in the inertial measurement unit, 、 、 Respectively represent the three-dimensional coordinates of infrared positioning data, inertial navigation measurement data, and ultra-wideband data positioning; the fused posture is converted into the UAV posture to form the UAV flight trajectory measurement value;

[0054] In addition, a pseudo-infrared positioning model is trained based on the LSTM network. When training the pseudo-infrared positioning model, the three-dimensional coordinates and three-dimensional attitude angles of ultra-wideband data positioning, as well as the three-axis angular velocity and three-axis acceleration in the inertial navigation measurement data are used as input to predict the three-dimensional coordinates and three-dimensional attitude of the infrared positioning data, and the prediction results are used as pseudo-infrared positioning data.

[0055] When the confidence level is lower than the threshold, the pseudo infrared positioning data generated by the pseudo infrared positioning model is used to replace the original infrared positioning data for fusion;

[0056] S3, complete the calibration of the UAV flight posture according to the UAV flight trajectory measurement value and the given flight trajectory.

[0057] The infrared positioning module captures images of infrared positioning targets mounted on drones, which are composed of specifically arranged infrared lamp beads, and processes these images using traditional computer vision algorithms: A centroid detection algorithm is used in each frame to accurately detect and locate the two-dimensional pixel coordinates of each infrared lamp bead; the same physical lamp bead observed in different camera views is matched using multi-view geometric constraints; the three-dimensional coordinates of these lamp beads in the global coordinate system are calculated using a triangulation algorithm using successfully matched lamp bead point pairs and their corresponding camera internal and external parameters; based on the known precise geometric layout of the lamp beads on the target and directly using the matched two-dimensional image points, the three-dimensional coordinates of the infrared positioning target of the target in the global coordinate system are calculated using a perspective n-point algorithm. and 3D pose The output infrared positioning data usually includes the infrared positioning signal strength S, the number of visible infrared lamp beads N and the three-dimensional coordinates of the infrared positioning target. and 3D pose ;

[0058] In addition, the UWB positioning sensor output contains three-dimensional coordinates and three-axis attitude information ; The inertial navigation sensor output includes three-axis angular velocity , three-axis acceleration , the three-axis position information is obtained through inertial navigation solution and three-axis attitude information .

[0059] Therefore, the infrared positioning data includes the infrared positioning signal strength, the number of visible infrared lamp beads, the three-dimensional coordinates and three-dimensional posture of the infrared positioning target in the global coordinate system; the ultra-wideband data includes the three-dimensional coordinates and three-dimensional posture angle of the ultra-wideband target in the global coordinate system; the inertial navigation measurement data includes three-axis angular velocity and three-axis acceleration, and the three-dimensional coordinates and three-dimensional posture of the inertial navigation measurement unit in the global coordinate system are obtained through inertial navigation solution; the differences in the installation positions of the infrared positioning target, ultra-wideband target and inertial navigation measurement unit in the miniaturized multi-sensor fusion target are negligible.

[0060] When infrared positioning reliability Above threshold When When the infrared positioning sensor, ultra-wideband positioning sensor and inertial navigation sensor output are fused using adaptive robust Kalman filtering, the errors of ultra-wideband positioning sensor and inertial navigation sensor are modeled based on deep learning technology to generate "pseudo infrared positioning information". In a specific embodiment of the present invention, the process of using adaptive robust Kalman filtering to fuse infrared positioning data, ultra-wideband data and inertial navigation measurement data is as follows:

[0061] Establish the system state equation of the infrared positioning sensor, ultra-wideband positioning sensor and inertial navigation sensor fusion model:

[0062]

[0063]

[0064]

[0065]

[0066] in, is the state vector of the system at time k, is the system state transfer matrix at time k-1, is the system process noise sequence at time k-1, is the noise transfer matrix at time k-1; Represent the position errors of the three axes of the system, The three-axis attitude errors of the system are: They represent the constant drift error of the three-axis accelerometer of the inertial navigation sensor, They represent the constant drift errors of the three-axis gyroscope of the inertial navigation sensor.

[0067] The positioning difference output by the infrared positioning sensor and ultra-wideband positioning sensor and the positioning difference output by the infrared positioning sensor and inertial navigation sensor are used as observation vectors, and the observation equation is expressed as follows:

[0068]

[0069] in, is the observation vector of the system at time k, is the system observation noise sequence at time k; is the system observation transfer matrix at time k.

[0070] The Kalman filter update process is:

[0071] Initialize, let , i=0; let the state vector of the system at time k be As the state vector, the observation vector of the system at time k As the observation vector, perform iterative calculation:

[0072]

[0073]

[0074]

[0075]

[0076] in, for Function in The Jacobian matrix at the point, is the local linearization function, is the Kalman gain value at time k in the i-th iteration, is the one-step prediction error covariance matrix from time k-1 to time k, with the superscript represents transpose, represents the covariance matrix of the white noise sequence, v represents the measurement noise that obeys the Gaussian distribution, It is expressed as the one-step estimated value of the state variable from time k-1 to time k. Expressed as the measured variable value at time k, represents the estimated value of the state variable at time k during the i-th iteration;

[0077] By using the Gauss-Newton method, nonlinear optimization is performed. When it is less than the preset threshold, the iteration is stopped and the last iteration result is output as As the final estimated state at time K .

[0078] State Estimation Including error terms (position error estimation, attitude error estimation, sensor drift estimation). The position information and attitude information output by the calibrated INS are as follows:

[0079]

[0080] in, is the position error estimate, is the attitude error estimate, is the true position estimate, This is the true attitude estimate. Accelerometer drift error and gyroscope offset error are not directly used in the position / attitude output. Instead, they are internally corrected to obtain the final position and attitude estimates.

[0081] In a specific implementation of the present invention, when the infrared positioning reliability Above threshold When the error of ultra-wideband positioning sensor and inertial navigation sensor is learned and modeled based on deep learning technology, "pseudo infrared positioning information" is generated. The input variables of LSTM deep learning network training model are 、 、 、 , the output variable is , train the LSTM deep learning network to determine the hyperparameters, obtain the "pseudo infrared positioning information" model, and output the predicted variables and .

[0082] The LSTM neural network consists of three parts: a forget gate, an input gate, and an output gate. Its calculation principle is common knowledge in the field and will not be repeated here. The LSTM neural network parameters used in this embodiment are: a single-layer LSTM structure, 1 fully connected layer; a single-layer LSTM has 32 LSTM Cell units, an initial learning rate of 0.02, and an optimizer using the adaptive moment estimation optimizer Adam, which can dynamically adjust the learning rate according to the gradient during training; a Dropout layer is added to the fully connected layer, randomly assigning zero weights to neurons in the network to avoid overfitting, and the default value is 0.2. When the infrared positioning reliability Below threshold When When fusion is performed, the pseudo infrared positioning data generated by the pseudo infrared positioning model is used to replace the original infrared positioning data.

[0083] The above examples are merely specific embodiments of the present invention. Obviously, the present invention is not limited to the above examples, and many variations are possible. All variations that can be directly derived or imagined by a person skilled in the art from the disclosure of the present invention should be considered to be within the scope of protection of the present invention.

Claims

1. A UAV flight posture calibration device based on multi-sensor fusion, characterized in that: include: Multiple integrated dynamic posture measurement base stations are arranged around the measurement site and provide full-area signal coverage. Each base station includes an infrared positioning module for collecting target infrared signals, an ultra-wideband reference module for transmitting and receiving ultra-wideband signals through a first antenna, and a first control module for synchronously processing infrared and ultra-wideband data. A miniaturized multi-sensor fusion target, mounted on a UAV motion carrier, includes an inertial navigation measurement unit (INU), infrared lamps, an ultra-wideband (UWB) main control module that communicates with a base station via a second antenna, and a second control module that synchronously collects and transmits inertial navigation measurement data and UWB data. The infrared lamps serve as infrared positioning targets. The host computer synchronously receives the infrared positioning data output by the first control module and the ultra-wideband data and inertial navigation measurement data output by the second control module, and fuses them to generate the UAV posture. During the fusion, the positioning confidence of the infrared positioning data is first calculated. When the confidence is not lower than the threshold, the infrared positioning data, ultra-wideband data and inertial navigation measurement data are directly fused. Otherwise, the pseudo-infrared positioning data generated by the pseudo-infrared positioning model is used to replace the original infrared positioning data for fusion.

2. The UAV flight posture calibration device based on multi-sensor fusion according to claim 1 is characterized in that: The evaluation function used to calculate the positioning reliability of infrared positioning data is the weighted value of the infrared signal strength term and the infrared lamp bead visibility term. The infrared signal strength term is the ratio of the current infrared signal strength to the historical maximum infrared signal strength, and the infrared lamp bead visibility term is the ratio of the current number of visible infrared lamp bead points to the total number of infrared lamp beads.

3. The UAV flight posture calibration device based on multi-sensor fusion according to claim 1 is characterized in that: The infrared positioning module includes a lens barrel, an infrared fill light located outside the lens barrel, a CCD image sensor located inside the lens barrel, an 800-2500nm infrared filter, a free-form surface lens and a lens combination. The free-form surface lens is designed based on point-by-point construction theory.

4. The UAV flight posture calibration device based on multi-sensor fusion according to claim 1 is characterized in that: The miniaturized multi-sensor fusion target includes a transparent shell, an ultra-wideband main control module, an inertial navigation measurement unit, a second control module, infrared lamp beads located on the inner walls around the transparent shell, and a second antenna located on the top of the transparent shell.

5. The UAV flight posture calibration device based on multi-sensor fusion according to claim 3 is characterized in that: The process of acquiring ultra-wideband data includes: The ultra-wideband reference modules in multiple integrated dynamic posture measurement base stations serve as spatial position reference points, with the number being no less than 3; the ultra-wideband main control module of the miniaturized multi-sensor fusion target serves as a mobile signal source, actively transmitting a pulse signal to the first antenna through the second antenna and transmitting it to the first control module, calculating the distance information in the first control module, sending the calculated distance information back to the second antenna, and calculating the ultra-wideband data in the second control module.

6. A multi-sensor fusion UAV flight posture calibration method, characterized in that: include: Infrared positioning data, ultra-wideband data, and inertial navigation measurement data are synchronously acquired through multiple integrated dynamic posture measurement base stations and a miniaturized multi-sensor fusion target installed on a UAV motion carrier; Calculate the dynamic positioning confidence of infrared positioning data. When the confidence is not lower than the threshold, use adaptive robust Kalman filtering to fuse infrared positioning data, ultra-wideband data and inertial navigation measurement data. Its state vector is defined as , the observation residual equation is defined as ,in represents the state at time k, Indicates the three-dimensional coordinate error between infrared positioning data and inertial navigation measurement data. Indicates the three-dimensional attitude error between infrared positioning data and inertial navigation measurement data. Indicates the constant drift error of the accelerometer in the inertial measurement unit, Indicates the constant drift error of the gyroscope in the inertial measurement unit, 、 、 Respectively represent the three-dimensional coordinates of infrared positioning data, inertial navigation measurement data, and ultra-wideband data positioning; the fused posture is converted into the UAV posture to form the UAV flight trajectory measurement value; In addition, a pseudo infrared positioning model is trained based on the LSTM network; When the confidence level is lower than the threshold, the pseudo infrared positioning data generated by the pseudo infrared positioning model is used to replace the original infrared positioning data for fusion; The UAV flight posture is calibrated based on the UAV flight trajectory measurement value and the given flight trajectory.

7. The multi-sensor fusion UAV flight posture calibration method according to claim 6 is characterized in that: When training the pseudo-infrared positioning model, the three-dimensional coordinates and three-dimensional attitude angles of ultra-wideband data positioning, as well as the three-axis angular velocity and three-axis acceleration in the inertial navigation measurement data are used as input to predict the three-dimensional coordinates and three-dimensional attitude of the infrared positioning data, and the prediction results are used as pseudo-infrared positioning data.

8. The multi-sensor fusion UAV flight posture calibration method according to claim 6, characterized in that: The infrared positioning data includes the infrared positioning signal strength, the number of visible infrared lamp beads, the three-dimensional coordinates and three-dimensional attitude of the infrared positioning target in the global coordinate system; the ultra-wideband data includes the three-dimensional coordinates and three-dimensional attitude angle of the ultra-wideband target in the global coordinate system; the inertial navigation measurement data includes three-axis angular velocity and three-axis acceleration, and the three-dimensional coordinates and three-dimensional attitude of the inertial navigation measurement unit in the global coordinate system are obtained through inertial navigation solution; the differences in the installation positions of the infrared positioning target, ultra-wideband target and inertial navigation measurement unit in the miniaturized multi-sensor fusion target are negligible.

9. The multi-sensor fusion UAV flight posture calibration method according to claim 6, characterized in that: When adaptive robust Kalman filtering is used to fuse infrared positioning data, ultra-wideband data and inertial navigation measurement data, the current state vector is used as the initial state, the initial state is iteratively optimized, and the inertial navigation measurement data is corrected according to the final optimized state.

10. The multi-sensor fusion UAV flight posture calibration method according to claim 9, characterized in that: The corrected inertial navigation measurement data refers to taking the sum of the three-dimensional coordinates obtained by the inertial navigation solution and the three-dimensional coordinate errors in the optimized state as the corrected three-dimensional coordinates, and taking the sum of the three-dimensional posture obtained by the inertial navigation solution and the three-dimensional posture errors in the optimized state as the corrected three-dimensional posture; the corrected result is used as the fused posture.

Images