An autonomous flight method, device, and equipment based on multi-sensor fusion positioning

Abstract

This application provides an autonomous flight method, apparatus, and device based on multi-sensor fusion positioning. The autonomous flight method determines the location information and sampling frequency, filters and performs consistency processing on the sampled data to obtain target sampling data, and inputs the flight state observations, obtained input control variables, and predicted flight state values ​​determined from the target sampling data into a flight state estimation model, outputting state estimation values. Then, the location information, state estimation values, and positioning errors of specified sensors are input into a trained positioning estimation prediction model, outputting sensor predicted states and flight prediction parameters. If the sensor predicted states meet the flight requirements, the UAV is controlled to fly to the next location to perform the flight mission according to the flight prediction parameters. Therefore, this application embodiment ensures that the UAV maintains high performance and stability in complex environments without upgrading the UAV's original sensing hardware and chip computing power.

Description

Technical Field

[0001] This application relates to unmanned aerial vehicle (UAV) technology, and in particular to an autonomous flight method, apparatus, and equipment based on multi-sensor fusion positioning. Background Technology

[0002] The development of drone technology, navigation technology, and high-precision surveying technology has significantly reduced the manufacturing cost of drones. As a result, drones are widely used in various fields, with their application in low-altitude environments accelerating rapidly. Although existing navigation technologies are relatively mature, navigation accuracy and reliability still need improvement in low-altitude environments due to various factors. This is especially true in complex urban terrain and densely built-up areas, where building obstruction effects are more pronounced, easily leading to a sharp increase in navigation errors. This can cause instability in navigation signals, even creating signal blind spots in some areas, affecting the aircraft's accurate positioning. Furthermore, the complex electromagnetic environment in cities contains numerous sources of electromagnetic interference, such as high-voltage power lines and radio communication equipment. These interference sources further interfere with navigation signals, reducing navigation accuracy and reliability.

[0003] Currently, expensive sensors and specific algorithms, such as multi-view vision sensors, LiDAR, millimeter-wave radar, and inertial navigation systems, are often used to improve the positioning accuracy and stability of drones. This is combined with the introduction of resource-intensive artificial intelligence technologies such as deep learning to continuously improve and optimize specific algorithms. While existing methods for improving positioning accuracy and stability, such as using advanced sensors and algorithms, and multi-sensor fusion technology, are effective, they often rely on upgrades to sensing hardware and improvements in chip computing power, which may lead to increased costs and power consumption. Summary of the Invention

[0004] This application provides an autonomous flight method, apparatus, and equipment based on multi-sensor fusion positioning, which ensures that the UAV can maintain high performance and stability in complex environments without upgrading the sensing hardware and chip computing power.

[0005] The technical solution provided in this application includes:

[0006] In a first aspect, embodiments of this application provide an autonomous flight method under multi-sensor fusion positioning. The UAV is equipped with multiple sensors that perform different measurement functions required for autonomous flight. Each sensor is electrically connected to the UAV's control system. The autonomous flight method is applied to the control system and includes:

[0007] Based on the received flight mission information and the initial speed of the UAV, determine the location information for the flight mission and the appropriate sampling frequency for each sensor, so that each sensor can collect data according to its corresponding sampling frequency.

[0008] The system receives sampling data collected by each sensor at its respective sampling frequency and sends it in a manner that ensures data integrity. The system then filters the sampling data to obtain filtered sampling data. Finally, the system performs consistency processing on the filtered sampling data to obtain target sampling data with the same timestamp, format, and coordinates.

[0009] The system obtains the predicted flight state of the UAV at the previous moment and the input control quantity for the previous moment, and obtains the observed flight state of the UAV at the current moment from the target sampling data. The observed flight state, the input control quantity and the predicted flight state are input into the updated flight state estimation model, and the estimated flight state of the UAV at the current moment is output.

[0010] The positioning error of the UAV is determined from the positioning data obtained by the designated sensor used to measure the positioning function. The location information, the state estimate and the positioning error are input into the trained positioning estimation prediction model, and the sensor prediction state of the designated sensor and the flight prediction parameters of the UAV to fly to the next location are output.

[0011] If the sensor predicts that the flight conditions meet the requirements, then the UAV is controlled to fly to the next location to perform the flight mission according to the flight prediction parameters.

[0012] In one embodiment of this application, determining the appropriate sampling frequency for each sensor based on the received task information and the initial speed of the UAV includes:

[0013] The received flight mission information is parsed, and the position information, attitude information, sensor data and battery status data of the UAV are obtained from the flight mission information.

[0014] Based on the initial speed of the UAV and the acquired position information, attitude information, sensor data, and battery status data, determine the appropriate sampling frequency for each sensor.

[0015] In one embodiment of this application, the step of inputting the flight state observation value, the input control quantity, and the flight state prediction value into the updated flight state estimation model, and outputting the state estimation value of the UAV's flight state at the current moment, includes:

[0016] The predicted flight state value and the control input are input into the flight state update model to obtain the predicted flight state value of the UAV at the current time relative to the previous time. The flight state update model is a model determined based on the transition relationship quantity used to describe the UAV state from the previous time to the current time, the control input influence coefficient used to describe the influence of the control input quantity at the previous time on the UAV flight state, and the control input quantity at the previous time.

[0017] The predicted flight state value and the observed flight state value are input into the state observation update model, and the state estimate value of the UAV's flight state at the current time is output. The state observation update model is a model determined based on the predicted flight state value of the UAV at the current time relative to the previous time, a function used to describe the relationship between the observed flight state value and the predicted state value, and a function used to determine the weight of the observed flight state value in updating the state estimate.

[0018] In one embodiment of this application, if the sensor predicts that the state does not meet the flight requirements during the execution of a mission by the UAV, the method further includes:

[0019] Real-time detection of the actual sensor status of the specified sensor, and obtaining the actual flight parameters of the UAV from the current location to the next location after completing the flight mission at the current location;

[0020] By comparing the actual flight parameters and flight prediction parameters at the same moment, as well as the difference between the sensor prediction state and the actual sensor state, if the difference meets a set threshold, the model parameters of the positioning estimation prediction model are adjusted, and historical data within a set time period is used as training samples to train the positioning estimation prediction model after adjusting the model parameters, until the model accuracy of the trained positioning estimation prediction model reaches the accuracy threshold, and the trained positioning estimation prediction model is updated as a new positioning estimation prediction model.

[0021] In one embodiment of this application, the flight prediction parameters include at least flight speed, heading angle, and flight altitude.

[0022] Secondly, this application also provides an autonomous flight device based on multi-sensor fusion positioning. The drone is equipped with multiple sensors that perform different measurement functions required for the drone to perform autonomous flight. Each sensor is electrically connected to the drone's control system. The autonomous flight device is applied to the control system. The autonomous flight device includes:

[0023] The data sampling unit is used to determine the location information for the flight mission and the appropriate sampling frequency for each sensor based on the received flight mission information and the initial speed of the UAV, so that each sensor can collect data according to its corresponding sampling frequency.

[0024] The data processing unit is used to receive the sampling data collected by each sensor according to its corresponding sampling frequency and sent in a manner that ensures data integrity, and to filter the sampling data to obtain filtered sampling data. The filtered sampling data is then subjected to consistency processing to obtain target sampling data with the same timestamp, the same format, and the same coordinates.

[0025] The state estimation unit is used to obtain the predicted flight state value of the UAV at the previous moment and the input control quantity for the previous moment, and to obtain the observed flight state value of the UAV at the current moment from the target sampling data. The observed flight state value, the input control quantity and the predicted flight state value are input into the updated flight state estimation model, and the state estimation value of the UAV's flight state at the current moment is output.

[0026] The flight prediction unit is used to determine the positioning error of the UAV from the positioning data obtained from the designated sensors related to the positioning function. It inputs the location information, the state estimation value and the positioning error into the trained positioning estimation prediction model, and outputs the sensor prediction state of the designated sensor and the flight prediction parameters of the UAV to fly to the next location. If the sensor prediction state meets the flight requirements, the flight control unit is triggered.

[0027] The flight control unit is used to control the UAV to fly to the next location to perform a flight mission according to the flight prediction parameters.

[0028] In one embodiment of this application, the data sampling unit is specifically used for:

[0029] The received flight mission information is parsed, and the position information, attitude information, sensor data and battery status data of the UAV are obtained from the flight mission information.

[0030] Based on the initial speed of the UAV and the acquired position information, attitude information, sensor data, and battery status data, determine the appropriate sampling frequency for each sensor.

[0031] In one embodiment of this application, the state estimation unit includes a state estimation value output subunit for inputting the flight state observation value, the input control quantity, and the flight state prediction value into an updated flight state estimation model, and outputting a state estimation value of the UAV's flight state at the current moment. The state estimation value output subunit is used for:

[0032] The predicted flight state value and the control input are input into the flight state update model to obtain the predicted flight state value of the UAV at the current time relative to the previous time. The flight state update model is a model determined based on the transition relationship quantity used to describe the UAV state from the previous time to the current time, the control input influence coefficient used to describe the influence of the control input quantity at the previous time on the UAV flight state, and the control input quantity at the previous time.

[0033] The predicted flight state value and the observed flight state value are input into the state observation update model, and the state estimate value of the UAV's flight state at the current time is output. The state observation update model is a model determined based on the predicted flight state value of the UAV at the current time relative to the previous time, a function used to describe the relationship between the observed flight state value and the predicted state value, and a function used to determine the weight of the observed flight state value in updating the state estimate.

[0034] In one embodiment of this application, if the sensor predicts that the state does not meet the flight requirements during the execution of a mission by the UAV, the autonomous flight device further includes:

[0035] The actual flight parameter acquisition unit is used to detect the actual sensor status of the specified sensor in real time, and to obtain the actual flight parameters of the UAV from the current point to the next point after completing the flight mission at the current point.

[0036] The model update unit is used to compare the actual flight parameters and flight prediction parameters at the same time, as well as the difference between the sensor prediction state and the actual sensor state. If the difference meets a set threshold, the model parameters of the positioning estimation prediction model are adjusted, and historical data within a set time period are used as training samples to train the positioning estimation prediction model after adjusting the model parameters until the model accuracy of the trained positioning estimation prediction model reaches the accuracy threshold. The trained positioning estimation prediction model is then used as the new positioning estimation prediction model for updating.

[0037] As can be seen from the above technical solutions, this application provides an autonomous flight method, apparatus, and device under multi-sensor fusion positioning. This autonomous flight method is applied to the control system of an unmanned aerial vehicle (UAV). Based on flight mission information and the initial speed of the UAV, the location information and suitable sampling frequency for each sensor are determined. The received sampling data is filtered and consistency-processed to obtain target sampling data. The flight state observations determined from the target sampling data, the obtained input control quantities, and the flight state prediction values ​​are input into the flight state estimation model, outputting the state estimation value. Then, the location information, the state estimation value, and the positioning error of the specified sensor are input into a trained positioning estimation prediction model, outputting the sensor prediction state and the flight prediction parameters for the UAV to fly to the next location. If the sensor prediction state meets the flight requirements, the UAV is controlled to fly to the next location according to the flight prediction parameters to perform the flight mission. Therefore, the technical solution provided in this application does not further enhance the original sensing hardware and chip computing power of the UAV. Instead, through the adaptive learning algorithm integrated into the flight state estimation model, the UAV can automatically adjust its positioning parameters and algorithm based on historical flight data and real-time environmental information, thereby improving positioning accuracy and stability. By continuously learning from flight data under different environmental conditions, UAVs can gradually optimize their positioning estimation and prediction models, reducing errors and improving positioning accuracy. Simultaneously, by optimizing the design of the flight state estimation model and fully utilizing computing resources, computational efficiency and real-time performance can be improved, ensuring that UAVs maintain high performance and stability even in complex environments. Attached Figure Description

[0038] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0039] Figure 1 A flowchart illustrating the first autonomous flight method based on multi-sensor fusion positioning provided in this application;

[0040] Figure 2 A flowchart illustrating the second multi-sensor fusion positioning autonomous flight method provided in this application;

[0041] Figure 3 A schematic diagram of the structure of an autonomous flight device under multi-sensor fusion positioning provided in this application;

[0042] Figure 4 This is a schematic diagram of the structure of an electronic device provided in this application. Detailed Implementation

[0043] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0044] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a,” “the,” and “the” used in this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0045] It should be understood that although the terms first, second, third, etc., may be used in this invention to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of this invention, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0046] See Figure 1 , Figure 1 This is a flowchart illustrating an autonomous flight method based on multi-sensor fusion positioning, provided in an embodiment of this application. The UAV is equipped with multiple sensors that perform different measurement functions required for autonomous flight. Each sensor is electrically connected to the control system of the UAV, and the autonomous flight method is applied to the control system.

[0047] In this embodiment, the UAV is equipped with multiple sensors, each measuring different functions. These sensors can be a high-precision GPS module, an IMU (Inertial Measurement Unit), a barometer, and a laser rangefinder. The high-precision GPS module is selected for its high sensitivity and stability to obtain accurate latitude and longitude information. Regular calibration of the GPS module is performed to reduce positioning errors. The IMU includes sensors such as accelerometers and gyroscopes to measure the UAV's attitude, angular velocity, and acceleration information. A high-performance IMU is selected, and precise calibration is performed to ensure data accuracy. The barometer measures the UAV's altitude; a high-precision, low-drift barometer is selected, and regular calibration is performed to improve altitude measurement accuracy. The laser rangefinder measures the distance between the UAV and obstacles; a high-resolution laser rangefinder with high measurement accuracy is selected to ensure accurate distance measurement. Thus, each sensor has a different measurement function and is electrically connected to the UAV's control system, allowing the control system to control each sensor to complete the UAV's flight mission.

[0048] The process of this autonomous flight method is described in detail below, including the following steps:

[0049] Step 101: Based on the received flight mission information and the initial speed of the UAV, determine the location information for the flight mission and the appropriate sampling frequency for each sensor, so that each sensor can collect data according to its corresponding sampling frequency.

[0050] In this embodiment, the flight mission information can be a flight mission issued by an external electronic device to the drone. This electronic device can be a remote monitoring center or a flight mission input by the drone's service provider. This embodiment is not limited to this.

[0051] The reason for determining the sampling frequency for each sensor is that a sampling frequency that is too high will increase the data processing burden, while a sampling frequency that is too low may lead to data loss or decreased accuracy.

[0052] Location information can be understood as the locations on a 3D map where the drone needs to perform its tasks. The appropriate sampling frequency for each sensor can be understood as the sampling frequency matched to each sensor.

[0053] Step 102: Receive sampling data collected by each sensor according to its corresponding sampling frequency and sent in a manner that ensures data integrity, and filter the sampling data to obtain filtered sampling data. Perform consistency processing on the filtered sampling data to obtain target sampling data with the same timestamp, the same format, and the same coordinates.

[0054] After each sensor collects data, the collected data needs to be transmitted to the control system. During the data acquisition process, it is necessary to ensure the integrity of the data. As one example, ensuring data integrity can be achieved by adding check codes or redundant data to detect whether the transmitted data is lost or corrupted, thereby ensuring the integrity of the transmitted data.

[0055] After receiving the data from each sensor, the control system first ensures the completeness of the data. If complete, the data is filtered to remove noise and outliers. Furthermore, since different sensors may have different sampling frequencies and data formats, the filtered data needs to undergo consistency processing. This consistency processing involves data synchronization, including time synchronization, coordinate transformation, and format synchronization. Timestamps or external clock signals are typically used to ensure time synchronization between different sensor data, thereby guaranteeing data consistency and accuracy.

[0056] Step 103: Obtain the predicted flight state value of the UAV at the previous moment and the input control quantity for the previous moment, and obtain the observed flight state value of the UAV at the current moment from the target sampling data. Input the observed flight state value, the input control quantity and the predicted flight state value into the updated flight state estimation model, and output the estimated flight state value of the UAV at the current moment.

[0057] In this embodiment, the predicted flight state of the UAV at the previous moment can be understood as the estimated state of the UAV at the previous moment, that is, the optimal estimate of the UAV's state at the previous moment based on all available observation data and the corresponding control input. The predicted flight state includes at least position, velocity, and orientation.

[0058] Flight status observations can be understood as direct measurements of the UAV's state obtained at the current moment through sensors or other observation methods. These direct measurements include measurements of the UAV's position, velocity, acceleration, and other state parameters. Input control variables are time-dependent, with one input control variable corresponding to each moment. These variables can be stored within the control system or acquired externally; this embodiment does not limit this.

[0059] The flight state estimation model in this step is dynamically changing, and it is the latest flight state estimation model.

[0060] The flight state estimation model in this step achieves information fusion by updating the model through predictions and observations of position, velocity, and direction. This involves employing multi-sensor information fusion technology to combine information from GPS, IMU, and other sources to improve positioning accuracy.

[0061] The state estimate of the UAV's flight state at the current moment is the updated state estimate. It represents the updated estimate of the UAV's state at the current moment based on the state prediction, flight state observations, and the weights of the decision observations in updating the state estimate. The state estimate is closer to the UAV's true state than the state prediction.

[0062] The multi-sensor fusion technology used in this embodiment can fully utilize the advantages of different sensors, reduce the impact of single sensor failure on overall positioning performance, and improve the reliability and stability of positioning.

[0063] Step 104: Determine the positioning error of the UAV from the positioning data obtained from the designated sensor used to measure the positioning function. Input the location information, the state estimation value and the positioning error into the trained positioning estimation prediction model, and output the sensor prediction state of the designated sensor and the flight prediction parameters of the UAV to fly to the next location. If the sensor prediction state meets the flight requirements, then proceed to step 105.

[0064] In this embodiment, the designated sensor can be a high-precision GPS sensor. During flight, positioning errors and sensor status are detected and collected in real time.

[0065] The positioning estimation and prediction model is a model trained through adaptive learning. That is, it uses machine learning algorithms such as support vector machines and random forests to train on historical data and establish a mapping relationship between the location information, the state estimation value, and the positioning error, and the flight prediction parameters and sensor status. In other words, the input of the positioning estimation and prediction model is the location information, the state estimation value, and the positioning error, and the output of the positioning estimation and prediction model is the flight prediction parameters and sensor status.

[0066] The adaptive learning algorithm used in this embodiment can continuously optimize the localization model, reduce errors, and improve localization accuracy.

[0067] Step 105: Control the UAV to fly to the next location and perform the flight mission according to the flight prediction parameters.

[0068] As an example, the flight prediction parameters include at least flight speed, heading angle, and flight altitude.

[0069] This embodiment addresses situations where positioning deviations are large or positioning is impossible by automatically adjusting positioning parameters and algorithms based on historical flight data and real-time environmental information.

[0070] Finish Figure 1 The described process.

[0071] Therefore, the autonomous flight method provided in this application is applied to the control system of an unmanned aerial vehicle (UAV). Based on flight mission information and the initial speed of the UAV, the location information and suitable sampling frequency for each sensor are determined. The received sampling data is filtered and consistency-processed to obtain target sampling data. The flight state observations determined from the target sampling data, the obtained input control quantities, and the predicted flight state values ​​are input into the flight state estimation model, which outputs the estimated state value. Then, the location information, the estimated state value, and the positioning error of the specified sensor are input into a trained positioning estimation prediction model, which outputs the sensor predicted state and the flight prediction parameters for the UAV to fly to the next location. If the sensor predicted state meets the flight requirements, the UAV is controlled to fly to the next location according to the flight prediction parameters to perform the flight mission. It is evident that the technical solution provided in this application does not further enhance the original sensing hardware and chip computing power of the UAV. Instead, through an adaptive learning algorithm integrated into the flight state estimation model, the UAV can automatically adjust its positioning parameters and algorithm based on historical flight data and real-time environmental information, thereby improving positioning accuracy and stability. By continuously learning flight data under different environmental conditions, the UAV can gradually optimize the positioning estimation prediction model, reduce errors, and improve positioning accuracy. Meanwhile, by optimizing the design of the flight state estimation model, making full use of computing resources, and improving computing efficiency and real-time performance, it can be ensured that the UAV can maintain high performance and stability in complex environments.

[0072] In some embodiments, the implementation of step 101, which determines the appropriate sampling frequency for each sensor based on the received task information and the initial speed of the UAV, may include the following steps:

[0073] Step A: Analyze the received flight mission information and obtain the position information, attitude information, sensor data, and battery status data of the UAV from the flight mission information.

[0074] Step B: Based on the initial speed of the UAV and the acquired position information, attitude information, sensor data, and battery status data, determine the appropriate sampling frequency for each sensor.

[0075] In other embodiments, step 104, in which the flight state observation, the input control quantity, and the flight state prediction are input into the updated flight state estimation model to output the estimated flight state of the UAV at the current moment, may include the following steps:

[0076] Step C: Input the predicted flight status value and the control input quantity into the flight status update model to obtain the predicted flight status value of the UAV at the current time relative to the previous time.

[0077] The flight state update model is determined based on the transition relationship quantity describing the UAV state from the previous moment to the current moment, the control input influence coefficient describing the influence of the control input quantity on the UAV flight state at the previous moment, and the control input quantity at the previous moment.

[0078] The transition relationship quantity can be a linear transformation matrix used to describe the transition relationship of the UAV state from the previous moment to the current moment. During UAV flight, the transition relationship quantity typically includes the transition relationships of states such as position, velocity, and acceleration.

[0079] The control input influence coefficient can be a control input matrix. The control input influence coefficient is a linear transformation matrix used to describe how the control input affects the state of the UAV.

[0080] Step D: Input the predicted flight state value and the observed flight state value into the state observation update model, and output the state estimate of the UAV's flight state at the current moment.

[0081] The state observation update model is determined based on the UAV's flight state prediction value at the current time relative to the previous time, a function describing the relationship between the flight state observation value and the state prediction value, and a model for determining the weight of the flight state observation value in updating the state estimate.

[0082] The weights in this step depend on the uncertainty of the predicted state estimate (i.e., the covariance matrix) and the uncertainty of the observation noise (i.e., the observation noise covariance matrix). In UAV fusion localization, the weights in this step are typically used to balance the weights between the predicted and observed values.

[0083] The relationship function in this step is a linear transformation matrix used to describe the relationship between the observed values ​​and the predicted state values. In UAV fusion localization, the relationship function is usually related to the type and layout of the sensors.

[0084] As can be seen, this embodiment improves the accuracy and robustness of positioning by fusing data from multiple sensors (including temperature, humidity, and air pressure). The fusion positioning algorithm used in this embodiment, as its core, can handle nonlinear systems and non-Gaussian noise.

[0085] In some embodiments, see Figure 2 If the sensor predicts that the state does not meet the flight requirements during the operation of this UAV, the following steps will be performed:

[0086] Step 106: Real-time detection of the actual sensor status of the designated sensor, and obtaining the actual flight parameters of the UAV from the current location to the next location after completing the flight mission at the current location.

[0087] Step 107: Compare the actual flight parameters and flight prediction parameters at the same time, as well as the difference between the sensor prediction state and the sensor actual state. If the difference meets the set threshold, then proceed to step 108.

[0088] Step 108: Adjust the model parameters of the positioning estimation prediction model, and use historical data within a set time period as training samples to train the positioning estimation prediction model after adjusting the model parameters until the model accuracy of the trained positioning estimation prediction model reaches the accuracy threshold, and update the trained positioning estimation prediction model as a new positioning estimation prediction model.

[0089] As can be seen, the technical solution provided in this embodiment uses a self-updating mechanism to detect positioning errors, actual flight parameters, and sensor status in real time, dynamically adjusting algorithm parameters to achieve online updating and optimization of the algorithm. It is easy to see that by applying this embodiment through real-time feedback and adjustment, the UAV can continuously adapt to environmental changes and mission requirements, improving the algorithm's adaptability and robustness.

[0090] The technical solutions provided in this application can be applied to the following scenarios and environments.

[0091] (1) Integrated Operation of UAVs and Manned Aircraft in Low-Altitude Airspace: In this scenario, a UAV flight fusion positioning service with self-learning and self-updating capabilities can significantly improve the collaborative operation efficiency of UAVs and manned aircraft in low-altitude airspace. Through real-time perception and avoidance technologies, UAVs can autonomously acquire accurate flight information and avoid collisions with manned aircraft, thereby ensuring the safe operation of the entire airspace. This service can also flexibly select between UAVs and manned aircraft according to mission requirements, reducing overall operating costs and improving the flexibility and efficiency of mission execution.

[0092] (2) Tracking Target UAVs: In fields such as security, military, and rescue, tracking target UAVs requires high-precision, real-time positioning information to ensure the accuracy and effectiveness of tracking. UAV flight fusion positioning services with self-learning and self-updating capabilities can provide reliable positioning support for such UAVs, enabling them to stably track targets in complex environments. For example, in urban security, this service can help UAVs monitor specific areas in real time, detecting and tracking suspicious individuals or vehicles; in the military field, this service can be used for reconnaissance and monitoring enemy movements to obtain battlefield intelligence.

[0093] (3) Remote Sensing and Monitoring: Unmanned aerial vehicles (UAVs) have a wide range of applications in the field of remote sensing, including land use, forest management, and hydrological resource monitoring. A UAV flight fusion positioning service with self-learning and self-updating capabilities can improve the positioning accuracy and stability of UAVs, enabling them to collect and analyze data more accurately. This service can also support the application of UAVs in environmental monitoring and meteorological observation, providing strong support for environmental protection and scientific research.

[0094] Secondly, see Figure 3 , Figure 3 This is a schematic diagram of the structure of an autonomous flight device 300 based on multi-sensor fusion positioning provided in an embodiment of this application. The drone is equipped with multiple sensors with different measurement functions required for the drone to perform autonomous flight. Each sensor is electrically connected to the drone's control system. The autonomous flight device is applied to the control system. The autonomous flight device 300 includes:

[0095] The data sampling unit 301 is used to determine the location information of the flight mission and the appropriate sampling frequency for each sensor based on the received flight mission information and the initial speed of the UAV, so that each sensor can collect data according to its corresponding sampling frequency.

[0096] The data processing unit 302 is used to receive the sampling data collected by each sensor according to its corresponding sampling frequency and sent in a manner that ensures data integrity, and to filter the sampling data to obtain filtered sampling data. The filtered sampling data is then subjected to consistency processing to obtain target sampling data under the same timestamp, the same format, and the same coordinates.

[0097] The state estimation unit 303 is used to obtain the predicted flight state value of the UAV at the previous moment and the input control quantity for the previous moment, and to obtain the observed flight state value of the UAV at the current moment from the target sampling data. The observed flight state value, the input control quantity and the predicted flight state value are input into the updated flight state estimation model, and the state estimation value of the UAV's flight state at the current moment is output.

[0098] The flight prediction unit 304 is used to determine the positioning error of the UAV from the positioning data obtained from the designated sensor related to the positioning function, input the location information, the state estimation value and the positioning error into the trained positioning estimation prediction model, and output the sensor prediction state of the designated sensor and the flight prediction parameters of the UAV to fly to the next location; if the sensor prediction state meets the flight requirements, the flight control unit 305 is triggered.

[0099] The flight control unit 305 is used to control the UAV to fly to the next location to perform a flight mission according to the flight prediction parameters.

[0100] As one embodiment, the data sampling unit is specifically used for:

[0101] The received flight mission information is parsed, and the position information, attitude information, sensor data and battery status data of the UAV are obtained from the flight mission information.

[0102] Based on the initial speed of the UAV and the acquired position information, attitude information, sensor data, and battery status data, determine the appropriate sampling frequency for each sensor.

[0103] As one embodiment, the state estimation unit includes a state estimation output subunit for inputting the flight state observations, the input control variables, and the flight state predictions into an updated flight state estimation model, and outputting a state estimation value of the UAV's flight state at the current moment. The state estimation output subunit is used for:

[0104] The predicted flight state value and the control input are input into the flight state update model to obtain the predicted flight state value of the UAV at the current time relative to the previous time. The flight state update model is a model determined based on the transition relationship quantity used to describe the UAV state from the previous time to the current time, the control input influence coefficient used to describe the influence of the control input quantity at the previous time on the UAV flight state, and the control input quantity at the previous time.

[0105] The predicted flight state value and the observed flight state value are input into the state observation update model, and the state estimate value of the UAV's flight state at the current time is output. The state observation update model is a model determined based on the predicted flight state value of the UAV at the current time relative to the previous time, a function used to describe the relationship between the observed flight state value and the predicted state value, and a function used to determine the weight of the observed flight state value in updating the state estimate.

[0106] As an example, if the sensor predicts that the state does not meet the flight requirements during the execution of a mission by this UAV, the autonomous flight device further includes:

[0107] The actual flight parameter acquisition unit is used to detect the actual sensor status of the specified sensor in real time, and to obtain the actual flight parameters of the UAV from the current point to the next point after completing the flight mission at the current point.

[0108] The model update unit is used to compare the actual flight parameters and flight prediction parameters at the same time, as well as the difference between the sensor prediction state and the actual sensor state. If the difference meets a set threshold, the model parameters of the positioning estimation prediction model are adjusted, and historical data within a set time period are used as training samples to train the positioning estimation prediction model after adjusting the model parameters until the model accuracy of the trained positioning estimation prediction model reaches the accuracy threshold. The trained positioning estimation prediction model is then used as the new positioning estimation prediction model for updating.

[0109] As an example, the flight prediction parameters include at least flight speed, heading angle, and flight altitude.

[0110] Therefore, in the technical solution provided in this embodiment, the autonomous flight device provided in this application embodiment is applied to the control system of a UAV. Based on the flight mission information and the initial speed of the UAV, it determines the location information and a suitable sampling frequency for each sensor. It then filters and performs consistency processing on the received sampling data to obtain target sampling data. The flight state observation values ​​determined from the target sampling data, the obtained input control quantities, and the flight state prediction values ​​are input into the flight state estimation model, outputting the state estimation value. Next, the location information, the state estimation value, and the positioning error of the specified sensor are input into the trained positioning estimation prediction model, outputting the sensor prediction state and the flight prediction parameters for the UAV to fly to the next location. If the sensor prediction state meets the flight requirements, the UAV is controlled to fly to the next location according to the flight prediction parameters to perform the flight mission. It is evident that the technical solution provided in this application embodiment does not further enhance the original sensing hardware and chip computing power of the UAV. Instead, through the adaptive learning algorithm integrated into the flight state estimation model, the UAV can automatically adjust its positioning parameters and algorithm based on historical flight data and real-time environmental information, thereby improving positioning accuracy and stability. By continuously learning from flight data under different environmental conditions, UAVs can gradually optimize their positioning estimation and prediction models, reducing errors and improving positioning accuracy. Simultaneously, by optimizing the design of the flight state estimation model and fully utilizing computing resources, computational efficiency and real-time performance can be improved, ensuring that UAVs maintain high performance and stability even in complex environments.

[0111] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0112] This application also provides an electronic device. From a hardware perspective, a hardware architecture diagram can be found in [reference needed]. Figure 4 As shown, it includes a machine-readable storage medium and a processor, wherein: the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the autonomous flight operation disclosed in the above example.

[0113] The machine-readable storage medium provided in this application embodiment stores machine-executable instructions. When the machine-executable instructions are invoked and executed by a processor, the machine-executable instructions cause the processor to perform the autonomous flight operation disclosed in the above example.

[0114] Here, a machine-readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information, such as executable instructions, data, etc. For example, a machine-readable storage medium can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0115] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0116] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0117] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, embodiments of this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0118] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0119] Furthermore, these computer program instructions can also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in the process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0120] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0121] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0122] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for autonomous flight under multi-sensor fusion positioning, characterized in that, The unmanned aerial vehicle is mounted with a plurality of sensors for measuring different functions required for autonomous flight of the unmanned aerial vehicle, each sensor is electrically connected to a control system of the unmanned aerial vehicle, the autonomous flight method is applied to the control system, and the autonomous flight method comprises the following steps: According to the received flight task information and the initial speed of the unmanned aerial vehicle, the received flight task information is analyzed, the position information, attitude information, sensor data and battery state data of the unmanned aerial vehicle are obtained from the flight task information, and the point position information for executing the flight task and the sampling frequency suitable for each sensor are determined according to the initial speed of the unmanned aerial vehicle and the obtained position information, attitude information, sensor data and battery state data, so that each sensor collects data according to the corresponding sampling frequency. The sampling data collected by each sensor according to the corresponding sampling frequency and transmitted in a manner of adding check code or redundant data to ensure data integrity is received, and the sampling data is filtered to obtain filtered sampling data, and the filtered sampling data is processed for consistency to obtain target sampling data under the same timestamp, the same format and the same coordinate. The flight state prediction value of the unmanned aerial vehicle at the previous time and the input control amount for the previous time are obtained, the flight state observation value of the unmanned aerial vehicle at the current time is obtained from the target sampling data, the flight state prediction value and the control input amount are input into a flight state update model to obtain the flight state prediction value of the unmanned aerial vehicle at the current time relative to the previous time; wherein the flight state update model is a model determined according to a transition relationship quantity for describing the state of the unmanned aerial vehicle from the previous time to the current time, a control input influence coefficient for describing the influence of the control input amount for the previous time on the flight state of the unmanned aerial vehicle and the control input amount of the previous time; the flight state prediction value and the flight state observation value are input into a state observation update model to output the state estimation value of the flight state of the unmanned aerial vehicle at the current time, wherein the state observation update model is a model determined according to the flight state prediction value of the unmanned aerial vehicle at the current time relative to the previous time, a relationship function for describing the relationship between the flight state observation value and the state prediction value and a weight for determining the flight state observation value in updating the state estimation. The positioning error of the unmanned aerial vehicle is determined from positioning data obtained from a specified sensor for measuring a positioning function related to the unmanned aerial vehicle, the point information, the state estimation value, and the positioning error are input into a trained positioning estimation prediction model, and a sensor prediction state of the specified sensor and flight prediction parameters of the unmanned aerial vehicle flying to a next point are output; if the sensor prediction state meets flight requirements, the unmanned aerial vehicle is controlled to fly to the next point according to the flight prediction parameters to perform a flight task; otherwise, a sensor actual state of the specified sensor is detected in real time, and actual flight parameters of the unmanned aerial vehicle flying to the next point after completing the flight task at the current point are obtained; the actual flight parameters and the flight prediction parameters at the same time, and a difference between the sensor prediction state and the sensor actual state are compared, if the difference meets a set threshold, model parameters of the positioning estimation prediction model are adjusted, and historical data in a set time period is used as a training sample to train the positioning estimation prediction model after the model parameters are adjusted, until model precision of the trained positioning estimation prediction model reaches a precision threshold, and the trained positioning estimation prediction model is updated as a new positioning estimation prediction model.

2. The autonomous flight method of claim 1, wherein, The flight prediction parameters at least include flight speed, heading angle, and flight height.

3. An autonomous flying device under multi-sensor fusion positioning, characterized in that, The unmanned aerial vehicle is mounted with a plurality of sensors for measuring different functions required by the unmanned aerial vehicle to perform autonomous flight, each sensor is electrically connected to a control system of the unmanned aerial vehicle, the autonomous flight device is applied to the control system, and the autonomous flight device includes: The data sampling unit is configured to analyze the received flight task information according to the initial speed of the unmanned aerial vehicle, obtain position information, attitude information, sensor data, and battery state data of the unmanned aerial vehicle from the received flight task information, and determine point information for performing the flight task and a sampling frequency suitable for each sensor according to the initial speed of the unmanned aerial vehicle and the obtained position information, attitude information, sensor data, and battery state data, so that each sensor collects data at the respective corresponding sampling frequency. The data processing unit is configured to receive sampling data collected by each sensor at the respective corresponding sampling frequency and transmitted in a manner ensuring data integrity, filter the sampling data to obtain filtered sampling data, and perform consistency processing on the filtered sampling data to obtain target sampling data at the same timestamp, the same format, and the same coordinate. The state estimation unit is configured to obtain a flight state prediction value of the UAV at a previous time and an input control amount for the previous time, and obtain a flight state observation value of the UAV at a current time from the target sample data, input the flight state prediction value and the control input amount into a flight state update model, and obtain a flight state prediction value of the UAV at the current time relative to the previous time; the flight state update model is determined according to a transition relationship quantity for describing a state transition of the UAV from the previous time to the current time, a control input influence coefficient for describing an influence of the control input amount for the previous time on the flight state of the UAV, and the control input amount at the previous time; the flight state prediction value and the flight state observation value are input into a state observation update model, and a state estimation value of the flight state of the UAV at the current time is output, wherein the state observation update model is determined according to the flight state prediction value of the UAV at the current time relative to the previous time, a relationship function between the flight state observation value and the state prediction value, and a weight for determining the flight state observation value in updating the state estimation; The flight prediction unit is configured to determine a positioning error of the UAV from positioning data obtained from a designated sensor for measuring a positioning function, input the point location information, the state estimation value, and the positioning error into a trained positioning estimation prediction model, and output a sensor prediction state of the designated sensor and a flight prediction parameter of the UAV flying to a next point location; if the sensor prediction state meets a flight requirement, the flight control unit is triggered; if the sensor prediction state does not meet the flight requirement, the actual flight parameter obtaining unit is triggered. The flight control unit is configured to control the UAV to fly to the next point location according to the flight prediction parameter to perform a flight task. The actual flight parameter obtaining unit is configured to detect a sensor actual state of the designated sensor in real time, and obtain an actual flight parameter of the UAV flying to the next point location after completing a flight task at a current point location. The model updating unit is configured to compare the actual flight parameter and the flight prediction parameter at the same time, and a difference between the sensor prediction state and the sensor actual state; if the difference meets a set threshold value, the model parameter of the positioning estimation prediction model is adjusted, historical data in a set time period is used as a training sample to train the positioning estimation prediction model after the model parameter is adjusted, until a model precision of the trained positioning estimation prediction model reaches a precision threshold value.

4. The autonomous flying apparatus of claim 3, wherein, The flight prediction parameter at least includes a flight speed, a heading angle, and a flight height.

5. An electronic device, comprising: The processor is configured to execute the machine executable instructions to implement the execution steps of the autonomous flight method in any one of claims 1-2. The processor is configured to execute the machine executable instructions to implement the execution steps of the autonomous flight method in any one of claims 1-2.

Images