Autonomous flight method, device and equipment under multi-sensor fusion positioning

Abstract

The invention provides an autonomous flight method, device and equipment under multi-sensor fusion positioning, and the method comprises the steps: determining point location information and sampling frequency, carrying out the filtering and consistency processing of sampling data, and obtaining target sampling data; inputting a flight state observation value determined from the target sampling data, the obtained input control quantity and the flight state prediction value into a flight state estimation model, and outputting a state estimation value; inputting the point location information, the state estimation value and the positioning error of the specified sensor into a trained positioning estimation prediction model, and outputting a sensor prediction state and a flight prediction parameter; and if the sensor prediction state meets the flight requirement, the unmanned aerial vehicle is controlled to fly to the next point location according to the flight prediction parameters to execute the flight task. Therefore, under the condition that original sensing hardware and chip computing power of the unmanned aerial vehicle are not improved, it can be ensured that the unmanned aerial vehicle can still keep high performance and stability in the complex environment.

Description

Technical Field

[0001] The present application relates to unmanned aerial vehicle technology, and in particular to an autonomous flight method, device and equipment under multi-sensor fusion positioning. Background Art

[0002] The development of UAV technology, navigation technology, and high-precision mapping technology has greatly reduced the production cost of UAVs. Based on this, UAVs are widely used in various fields, among which the application of UAVs in low-altitude areas is also increasing rapidly. Although the existing navigation technology is relatively mature, the navigation accuracy and reliability still need to be improved in low-altitude environments due to the influence of various factors. Especially in urban areas with complex terrain and densely built areas, the occlusion effect of buildings is more significant, and the navigation error is prone to a sharp rise. This will lead to the instability of navigation signals, and even form signal blind spots in some areas, affecting the accurate positioning of aircraft. In addition, the electromagnetic environment in cities is complex, and there are a large number of electromagnetic interference sources, such as high-voltage lines, radio communication equipment, etc. These interference sources will further interfere with the navigation signal, reducing the navigation accuracy and reliability.

[0003] At present, more expensive sensors and specific algorithms, such as multi-eye vision sensors, lidar, millimeter-wave radar, inertial navigation systems, etc., are often used to improve the positioning accuracy and stability of drones. Combined with the introduction of artificial intelligence technologies such as deep learning that consume more resources, specific algorithms are continuously improved and optimized. The existing methods of improving positioning accuracy and stability, such as the use of advanced sensors and algorithms, and multi-sensor fusion technology, although effective, often rely on the upgrade of perception hardware and the improvement of chip computing power, which may lead to increased costs and power consumption. Summary of the invention

[0004] The present application provides an autonomous flight method, device and equipment under multi-sensor fusion positioning to ensure that the drone can maintain high performance and stability in complex environments without upgrading the perception hardware and chip computing power.

[0005] The technical solutions provided in this application include:

[0006] In the first aspect, the embodiment of the present application provides an autonomous flight method under multi-sensor fusion positioning, wherein a plurality of sensors with different measurement functions required for the autonomous flight of the drone are mounted on the drone, and each sensor is electrically connected to the control system of the drone, and the autonomous flight method is applied to the control system, and the autonomous flight method includes:

[0007] According to the received flight mission information and the initial speed of the UAV, determine the point information for executing the flight mission and the sampling frequency suitable for each sensor, so that each sensor collects data according to its corresponding sampling frequency;

[0008] Receive sampling data collected by each sensor at its corresponding sampling frequency and sent in a manner to ensure data integrity, perform filtering on the sampling data to obtain filtered sampling data, perform consistency processing on the filtered sampling data to obtain target sampling data at the same timestamp, format and coordinates;

[0009] Obtaining a flight state prediction value of the UAV at the previous moment and an input control amount for the previous moment, and obtaining a flight state observation value of the UAV at the current moment from the target sampling data, inputting the flight state observation value, the input control amount and the flight state prediction value into an updated flight state estimation model, and outputting a state estimation value of the flight state of the UAV at the current moment;

[0010] Determine the positioning error of the UAV from the positioning data obtained by the designated sensor used to measure the positioning function, input the point information, the state estimation value and the positioning error into the trained positioning estimation prediction model, and output the sensor prediction state of the designated sensor and the flight prediction parameters of the UAV to fly to the next point;

[0011] If the sensor prediction state meets the flight requirements, the drone is controlled to fly to the next point to perform the flight mission according to the flight prediction parameters.

[0012] In one embodiment of the present application, determining a sampling frequency suitable for each sensor according to the received mission information and the initial speed of the drone includes:

[0013] Parse the received flight mission information, and obtain the position information, attitude information, sensor data and battery status data of the UAV from the flight mission information;

[0014] According to the initial speed of the UAV and the acquired position information, attitude information, sensor data and battery status data, the sampling frequency suitable for each sensor is determined.

[0015] In one embodiment of the present application, the flight state observation value, the input control amount and the flight state prediction value are input into the updated flight state estimation model, and the state estimation value of the flight state of the UAV at the current moment is output, including:

[0016] Inputting the flight state prediction value and the control input amount into a flight state update model to obtain the flight state prediction value of the UAV at the current moment relative to the previous moment; wherein the flight state update model is a model determined according to a transfer relationship quantity used to describe the state of the UAV from the previous moment to the current moment, a control input influence coefficient used to describe the influence of the control input amount at the previous moment on the flight state of the UAV, and the control input amount at the previous moment;

[0017] The flight state prediction value and the flight state observation value are input into a state observation update model, and a state estimation value of the UAV's flight state at the current moment is output, wherein the state observation update model is a model determined based on the flight state prediction value of the UAV at the current moment relative to the previous moment, a relationship function for describing the flight state observation value and the state prediction value, and a model for determining the weight of the flight state observation value when updating the state estimation.

[0018] In one embodiment of the present application, during the execution of a mission by the drone, if the sensor prediction state does not meet the flight requirements, the method further includes:

[0019] Real-time detection of the actual state of the designated sensor, and obtaining the actual flight parameters of the UAV to the next point after the current point has completed the flight mission;

[0020] The actual flight parameters and flight prediction parameters at the same time, as well as the difference between the sensor prediction state and the sensor actual state are compared. If the difference meets the set threshold, the model parameters of the positioning estimation prediction model are adjusted, and the historical data within the set time period is used as the training sample to train the positioning estimation prediction model after the model parameters are adjusted, until the model accuracy of the trained positioning estimation prediction model reaches the accuracy threshold, and the trained positioning estimation prediction model is updated as a new positioning estimation prediction model.

[0021] In one embodiment of the present application, the flight prediction parameters include at least flight speed, heading angle and flight altitude.

[0022] In the second aspect, the present application also provides an autonomous flight device under multi-sensor fusion positioning, wherein a plurality of sensors with different measurement functions required for the autonomous flight of the drone are mounted on the drone, each sensor is electrically connected to the control system of the drone, and the autonomous flight device is applied to the control system, and the autonomous flight device includes:

[0023] The data sampling unit is used to determine the point information for executing the flight mission and the sampling frequency suitable for each sensor according to the received flight mission information and the initial speed of the UAV, so that each sensor collects data according to its corresponding sampling frequency;

[0024] A data processing unit is used to receive the sampling data collected by each sensor according to the corresponding sampling frequency and sent in a manner to ensure data integrity, and filter the sampling data to obtain the filtered sampling data, and perform consistency processing on the filtered sampling data to obtain target sampling data under the same timestamp, the same format and the same coordinates;

[0025] A state estimation unit, used to obtain a flight state prediction value of the UAV at the previous moment and an input control amount for the previous moment, and obtain a flight state observation value of the UAV at the current moment from the target sampling data, input the flight state observation value, the input control amount and the flight state prediction value into an updated flight state estimation model, and output a state estimation value of the flight state of the UAV at the current moment;

[0026] A flight prediction unit is used to determine the positioning error of the UAV from the positioning data obtained by the designated sensor used to measure the positioning function, input the point information, the state estimation value and the positioning error into the trained positioning estimation prediction model, and output the sensor prediction state of the designated sensor and the flight prediction parameters of the UAV flying to the next point; if the sensor prediction state meets the flight requirements, the flight control unit is triggered;

[0027] The flight control unit is used to control the UAV to fly to the next point to perform a flight mission according to the flight prediction parameters.

[0028] In one embodiment of the present application, the data sampling unit is specifically used for:

[0029] Parse the received flight mission information, and obtain the position information, attitude information, sensor data and battery status data of the UAV from the flight mission information;

[0030] According to the initial speed of the UAV and the acquired position information, attitude information, sensor data and battery status data, the sampling frequency suitable for each sensor is determined.

[0031] In one embodiment of the present application, the state estimation unit includes a state estimation value output subunit for inputting the flight state observation value, the input control amount and the flight state prediction value into the updated flight state estimation model, and outputting the state estimation value of the flight state of the UAV at the current moment, wherein the state estimation value output subunit is used to:

[0032] Inputting the flight state prediction value and the control input amount into a flight state update model to obtain the flight state prediction value of the UAV at the current moment relative to the previous moment; wherein the flight state update model is a model determined according to a transfer relationship quantity used to describe the state of the UAV from the previous moment to the current moment, a control input influence coefficient used to describe the influence of the control input amount at the previous moment on the flight state of the UAV, and the control input amount at the previous moment;

[0033] The flight state prediction value and the flight state observation value are input into a state observation update model, and a state estimation value of the UAV's flight state at the current moment is output, wherein the state observation update model is a model determined based on the flight state prediction value of the UAV at the current moment relative to the previous moment, a relationship function for describing the flight state observation value and the state prediction value, and a model for determining the weight of the flight state observation value when updating the state estimation.

[0034] In one embodiment of the present application, during the execution of a mission by the drone, if the sensor prediction state does not meet the flight requirements, the autonomous flight device further includes:

[0035] An actual flight parameter obtaining unit, used to detect the actual sensor status of the designated sensor in real time, and to obtain the actual flight parameters of the UAV to the next point after the flight mission has been completed at the current point;

[0036] The model updating unit is used to compare the actual flight parameters and the flight prediction parameters at the same time, and the difference between the sensor prediction state and the sensor actual state. If the difference meets the set threshold, the model parameters of the positioning estimation prediction model are adjusted, and the historical data within the set time period is used as the training sample to train the positioning estimation prediction model after the model parameters are adjusted, until the model accuracy of the trained positioning estimation prediction model reaches the accuracy threshold, and the trained positioning estimation prediction model is used as a new positioning estimation prediction model for updating.

[0037] It can be seen from the above technical solutions that the present application provides an autonomous flight method, device and equipment under multi-sensor fusion positioning. The autonomous flight method is applied to the control system of the unmanned aerial vehicle, and the point information and the sampling frequency suitable for each sensor are determined according to the flight mission information and the initial speed of the unmanned aerial vehicle, and the received sampling data are filtered and processed for consistency to obtain the target sampling data, and the flight state observation value determined from the target sampling data, the input control quantity obtained and the flight state prediction value are input into the flight state estimation model, and the state estimation value is output; the point information, the state estimation value and the positioning error of the specified sensor are input into the trained positioning estimation prediction model, and the sensor prediction state and the flight prediction parameters of the unmanned aerial vehicle flying to the next point are output; if the sensor prediction state meets the flight requirements, the unmanned aerial vehicle is controlled to fly to the next point according to the flight prediction parameters to perform the flight mission. It can be seen that the technical solution provided by the embodiment of the present application will not further improve the original perception hardware and chip computing power of the unmanned aerial vehicle, but through the adaptive learning algorithm integrated in the flight state estimation model, the unmanned aerial vehicle can automatically adjust the positioning parameters and algorithms according to the historical flight data and real-time environmental information, thereby improving the positioning accuracy and stability. By continuously learning flight data under different environmental conditions, the drone can gradually optimize the positioning estimation prediction model, reduce errors, and improve positioning accuracy. At the same time, by optimizing the design of the flight state estimation model, fully utilizing computing resources, improving computing efficiency and real-time performance, it can ensure that the drone can still maintain high performance and stability in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0039] Figure 1 A schematic diagram of the process of the first autonomous flight method under multi-sensor fusion positioning provided in this application;

[0040] Figure 2 A schematic diagram of the flow of the second autonomous flight method under multi-sensor fusion positioning provided in this application;

[0041] Figure 3 A schematic diagram of the structure of an autonomous flying device under multi-sensor fusion positioning provided in this application;

[0042] Figure 4 A schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0043] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0044] The terms used in the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "the" and "the" used in the present invention and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0045] It should be understood that although the terms first, second, third, etc. may be used in the present invention to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0046] See also Figure 1 , Figure 1 A flow chart of an autonomous flight method under multi-sensor fusion positioning provided in an embodiment of the present application, wherein the UAV is equipped with multiple sensors with different measurement functions required for the UAV to perform autonomous flight, each sensor is electrically connected to the control system of the UAV, and the autonomous flight method is applied to the control system.

[0047] In this embodiment, the drone itself is equipped with multiple sensors, each of which measures different functions. These sensors can be high-precision GPS modules, IMU (Inertial Measurement Unit), barometers, and laser rangefinders. Among them, high-precision GPS module: select a GPS module with high sensitivity and stability to obtain high-precision longitude and latitude information. At the same time, the GPS module is calibrated regularly to reduce positioning errors. IMU includes sensors such as accelerometers and gyroscopes, which are used to measure the attitude information, angular velocity information, and acceleration information of the drone. A high-performance IMU can be selected and accurately calibrated to ensure the accuracy of the data. The barometer is used to measure the altitude information of the drone. A high-precision, low-drift barometer is selected and calibrated regularly to improve the accuracy of altitude measurement. The laser rangefinder is used to measure the distance between the drone and the obstacle. A laser rangefinder with high resolution and measurement accuracy is selected to ensure the accuracy of the distance measurement. It can be seen that each sensor has different measurement functions and is electrically connected to the drone's own control system to control each sensor through the control system to complete the flight mission of the drone.

[0048] The process of the autonomous flight method is described in detail below, which specifically includes the following steps:

[0049] Step 101, according to the received flight mission information and the initial speed of the UAV, determine the point information for executing the flight mission and the sampling frequency suitable for each sensor, so that each sensor collects data according to its corresponding sampling frequency.

[0050] In this embodiment, the flight mission information may be a flight mission sent by an external electronic device to the drone. The electronic device may be a remote monitoring center, or may be a flight mission input into the drone by the service provider to which the drone belongs. This embodiment is not limited to this.

[0051] The reason why each sensor determines the sampling frequency is that a sampling frequency that is too high will increase the data processing burden, while a sampling frequency that is too low may result in data loss or reduced accuracy.

[0052] Point information can be understood as the points where the drone needs to perform tasks in the three-dimensional map. The sampling frequency suitable for each sensor can be understood as the sampling frequency matched with each sensor.

[0053] Step 102, receiving the sampling data collected by each sensor according to its corresponding sampling frequency and sent in a manner to ensure data integrity, and filtering the sampling data to obtain the filtered sampling data, and performing consistency processing on the filtered sampling data to obtain the target sampling data under the same timestamp, the same format and the same coordinates.

[0054] After each sensor collects data, it is necessary to transmit the collected data to the control system. During the data collection process, the integrity of the data needs to be ensured. As an embodiment, the implementation method of ensuring data integrity can be to detect whether the transmitted data is lost or damaged by adding a check code or redundant data, thereby ensuring the integrity of the transmitted data.

[0055] After receiving the collected data sent by each sensor, the control system first ensures that the collected data is complete. If complete, the collected data needs to be filtered to remove noise and abnormal values. In addition, since the sampling frequency and data format of different sensors may be different, based on this, the filtered data needs to be processed for consistency. The consistency processing is the data synchronization processing of the collected data. The data synchronization processing includes time synchronization, coordinate conversion and format synchronization. Usually, a timestamp or external clock signal is used to ensure the time synchronization between different sensor data to ensure the consistency and accuracy of the data.

[0056] Step 103, obtain the flight state prediction value of the UAV at the previous moment and the input control amount for the previous moment, and obtain the flight state observation value of the UAV at the current moment from the target sampling data, input the flight state observation value, the input control amount and the flight state prediction value into the updated flight state estimation model, and output the state estimation value of the UAV's flight state at the current moment.

[0057] In this embodiment, the flight state prediction value of the drone at the last moment can be understood as the state estimation value of the drone at the last moment, that is, the optimal estimation of the drone state based on all available observation data and the control input corresponding to the last moment at the last moment. The flight state prediction value includes at least position, speed and direction.

[0058] The flight state observation value can be understood as a direct measurement value of the drone state obtained by sensors or other observation means at the current moment, and the direct measurement value includes the measurement value of the drone's position, speed, acceleration, etc. The input control quantity is related to time, and each moment corresponds to an input control quantity, which can be stored in the control system or obtained from the outside, and this embodiment does not limit this.

[0059] The flight state estimation model in this step changes dynamically, and the flight state estimation model in this step is the latest flight state estimation model.

[0060] The flight state estimation model in this step achieves information fusion by updating the prediction and observation of position, speed and direction. That is, multi-sensor information fusion technology is used to fuse GPS, IMU and other information to improve positioning accuracy.

[0061] The state estimate of the drone's flight state at the current moment is the updated state estimate. It represents the updated estimate of the drone's state at the current moment based on the state prediction value, flight state observation value, and the weight of the decision observation value in updating the state estimate. The state estimate is closer to the drone's true state than the state prediction value.

[0062] The multi-sensor fusion technology adopted in this embodiment can make full use of the advantages of different sensors, reduce the impact of a single sensor failure on the overall positioning performance, and improve the reliability and stability of positioning.

[0063] Step 104, determine the positioning error of the UAV from the positioning data obtained by the designated sensor used to measure the positioning function, input the point information, the state estimation value and the positioning error into the trained positioning estimation prediction model, output the sensor prediction state of the designated sensor and the flight prediction parameters of the UAV to the next point; if the sensor prediction state meets the flight requirements, execute step 105.

[0064] In this embodiment, the designated sensor may be a high-precision GPS sensor. During flight, the positioning error and sensor status are detected and collected in real time.

[0065] The positioning estimation prediction model is a model trained through adaptive learning, that is, historical data is trained using machine learning algorithms such as support vector machines, random forests, etc., to establish a mapping relationship between the point information, the state estimation value and the positioning error, and the flight prediction parameters and sensor states. That is, the input of the positioning estimation prediction model is the point information, the state estimation value and the positioning error, and the output of the positioning estimation prediction model is the flight prediction parameters and sensor states.

[0066] The adaptive learning algorithm used in this embodiment can continuously optimize the positioning model, reduce errors, and improve positioning accuracy.

[0067] Step 105, controlling the UAV to fly to the next point to perform the flight mission according to the flight prediction parameters.

[0068] As an embodiment, the flight prediction parameters include at least flight speed, heading angle and flight altitude.

[0069] In the case where the positioning deviation is large or positioning is impossible, this embodiment can automatically adjust the positioning parameters and algorithms according to historical flight data and real-time environmental information.

[0070] Finish Figure 1 Describe the process.

[0071] It can be seen that the autonomous flight method provided by the embodiment of the present application is applied to the control system of the unmanned aerial vehicle, and the point information and the sampling frequency suitable for each sensor are determined according to the flight mission information and the initial speed of the unmanned aerial vehicle, and the received sampling data are filtered and processed for consistency to obtain the target sampling data, and the flight state observation value determined from the target sampling data, the input control amount obtained and the flight state prediction value are input into the flight state estimation model, and the state estimation value is output; the point information, the state estimation value and the positioning error of the specified sensor are input into the trained positioning estimation prediction model, and the sensor prediction state and the flight prediction parameters of the unmanned aerial vehicle flying to the next point are output; if the sensor prediction state meets the flight requirements, the unmanned aerial vehicle is controlled to fly to the next point according to the flight prediction parameters to perform the flight mission. It can be seen that the technical solution provided by the embodiment of the present application will not further improve the original perception hardware and chip computing power of the unmanned aerial vehicle, but through the adaptive learning algorithm integrated in the flight state estimation model, the unmanned aerial vehicle can automatically adjust the positioning parameters and algorithms according to the historical flight data and real-time environmental information, thereby improving the positioning accuracy and stability. By continuously learning the flight data under different environmental conditions, the unmanned aerial vehicle can gradually optimize the positioning estimation prediction model, reduce errors, and improve positioning accuracy. At the same time, by optimizing the design of the flight state estimation model, we can make full use of computing resources, improve computing efficiency and real-time performance, and ensure that the UAV can maintain high performance and stability in complex environments.

[0072] In some embodiments, the implementation method of determining the sampling frequency suitable for each sensor according to the received mission information and the initial speed of the drone in step 101 may include the following steps:

[0073] Step A: parsing the received flight mission information, and obtaining the position information, attitude information, sensor data and battery status data for the UAV from the flight mission information.

[0074] Step B, determining a sampling frequency suitable for each sensor according to the initial speed of the drone and the acquired position information, attitude information, sensor data and battery status data.

[0075] In other embodiments, in step 104, the flight state observation value, the input control amount and the flight state prediction value are input into the updated flight state estimation model, and the implementation method of outputting the state estimation value of the flight state of the UAV at the current moment may include the following steps:

[0076] Step C: input the flight state prediction value and the control input into a flight state update model to obtain the flight state prediction value of the UAV at the current moment relative to the previous moment.

[0077] Among them, the flight state update model is a model determined based on the transfer relationship quantity used to describe the drone state from the previous moment to the current moment, the control input influence coefficient used to describe the influence of the control input quantity at the previous moment on the drone flight state, and the control input quantity at the previous moment.

[0078] The transfer relationship quantity can be a linear transformation matrix, which is used to describe the transfer relationship of the drone state from the previous moment to the current moment. During the flight of the drone, the transfer relationship quantity usually includes the transfer relationship of the position, speed, acceleration and other states.

[0079] The control input influence coefficient may be a control input matrix. The control input influence coefficient is a linear transformation matrix used to describe how the control input quantity affects the state of the UAV.

[0080] Step D: input the flight state prediction value and the flight state observation value into the state observation update model, and output the state estimation value of the flight state of the UAV at the current moment.

[0081] Among them, the state observation update model is a model determined based on the flight state prediction value of the UAV at the current moment relative to the previous moment, a function used to describe the relationship between the flight state observation value and the state prediction value, and a model used to determine the weight of the flight state observation value when updating the state estimation.

[0082] The weight of this step depends on the uncertainty of the predicted state estimate, i.e., the covariance matrix, and the uncertainty of the observation noise, i.e., the observation noise covariance matrix. In UAV fusion positioning, the weight of this step is usually used to balance the weight between the predicted value and the observed value.

[0083] The relationship function in this step is a linear transformation matrix, which is used to describe the relationship between the observed value and the state prediction value. In UAV fusion positioning, the relationship function is usually related to the type and layout of the sensor.

[0084] It can be seen that this embodiment improves the accuracy and robustness of positioning by fusing data from multiple sensors (including temperature, humidity, air pressure, etc.) The fusion positioning algorithm of this embodiment is used as the core, which can handle nonlinear systems and non-Gaussian noise.

[0085] In some embodiments, see Figure 2 During the mission execution of the UAV, if the sensor prediction state does not meet the flight requirements, execute the following steps:

[0086] Step 106, real-time detection of the actual state of the designated sensor, and obtaining actual flight parameters of the UAV to the next point after the UAV has completed the flight mission at the current point.

[0087] Step 107 , compare the actual flight parameters and the predicted flight parameters at the same time, and the difference between the sensor predicted state and the sensor actual state. If the difference meets the set threshold, execute step 108 .

[0088] Step 108, adjust the model parameters of the positioning estimation prediction model, and use historical data within a set time period as training samples to train the positioning estimation prediction model after the model parameters are adjusted, until the model accuracy of the trained positioning estimation prediction model reaches the accuracy threshold, and update the trained positioning estimation prediction model as a new positioning estimation prediction model.

[0089] It can be seen that the technical solution provided by this embodiment detects positioning errors, actual flight parameters and sensor status in real time through a self-update mechanism, dynamically adjusts algorithm parameters, and realizes online update and optimization of the algorithm. It is not difficult to see that by applying this embodiment through real-time feedback and adjustment, the drone can continuously adapt to environmental changes and mission requirements, and improve the adaptability and robustness of the algorithm.

[0090] The technical solutions provided in the embodiments of the present application can be applied to the following scenarios and environments.

[0091] (1) Fusion operation of drones and manned aircraft in low-altitude airspace: In this scenario, the self-learning and self-updating drone flight fusion positioning service can significantly improve the efficiency of the coordinated operation of drones and manned aircraft in low-altitude airspace. Through real-time perception and avoidance technology, drones can autonomously obtain accurate flight information and avoid collisions with manned aircraft, thereby ensuring the safe operation of the entire airspace. The service can also flexibly choose to use drones or manned aircraft according to mission requirements, reducing overall operating costs and improving the flexibility and efficiency of mission execution.

[0092] (2) Tracking target drones: In the fields of security, military, and rescue, tracking target drones requires high-precision, real-time positioning information to ensure the accuracy and effectiveness of tracking. The self-learning and self-updating drone flight fusion positioning service can provide reliable positioning support for such drones, enabling them to stably track targets in complex environments. For example, in urban security, this service can help drones monitor specific areas in real time and detect and track suspicious people or vehicles; in the military field, this service can be used to scout and monitor enemy movements and obtain battlefield intelligence.

[0093] (3) Remote sensing and monitoring: UAVs are widely used in the field of remote sensing, including land use, forest management, and hydrological resource monitoring. The self-learning and self-updating UAV flight fusion positioning service can improve the positioning accuracy and stability of UAVs, enabling them to collect and analyze data more accurately. This service can also support the application of UAVs in environmental monitoring, meteorological observation, etc., providing strong support for environmental protection and scientific research.

[0094] Second, see Figure 3 , Figure 3 This is a schematic diagram of the structure of an autonomous flight device 300 under multi-sensor fusion positioning provided in an embodiment of the present application. The drone is equipped with multiple sensors with different measurement functions required for the drone to perform autonomous flight. Each sensor is electrically connected to the control system of the drone. The autonomous flight device is applied to the control system. The autonomous flight device 300 includes:

[0095] The data sampling unit 301 is used to determine the point information for executing the flight mission and the sampling frequency suitable for each sensor according to the received flight mission information and the initial speed of the UAV, so that each sensor collects data according to its corresponding sampling frequency;

[0096] The data processing unit 302 is used to receive the sampling data collected by each sensor according to the corresponding sampling frequency and sent in a manner to ensure data integrity, and filter the sampling data to obtain the filtered sampling data, and perform consistency processing on the filtered sampling data to obtain the target sampling data under the same timestamp, the same format and the same coordinates;

[0097] The state estimation unit 303 is used to obtain the flight state prediction value of the UAV at the previous moment and the input control amount for the previous moment, and obtain the flight state observation value of the UAV at the current moment from the target sampling data, input the flight state observation value, the input control amount and the flight state prediction value into the updated flight state estimation model, and output the state estimation value of the flight state of the UAV at the current moment;

[0098] The flight prediction unit 304 is used to determine the positioning error of the UAV from the positioning data obtained by the designated sensor used to measure the positioning function, input the point information, the state estimation value and the positioning error into the trained positioning estimation prediction model, and output the sensor prediction state of the designated sensor and the flight prediction parameters of the UAV flying to the next point; if the sensor prediction state meets the flight requirements, the flight control unit 305 is triggered;

[0099] The flight control unit 305 is used to control the UAV to fly to the next point to perform a flight mission according to the flight prediction parameters.

[0100] As an embodiment, the data sampling unit is specifically used for:

[0101] Parse the received flight mission information, and obtain the position information, attitude information, sensor data and battery status data of the UAV from the flight mission information;

[0102] According to the initial speed of the UAV and the acquired position information, attitude information, sensor data and battery status data, the sampling frequency suitable for each sensor is determined.

[0103] As an embodiment, the state estimation unit includes a state estimation value output subunit for inputting the flight state observation value, the input control amount and the flight state prediction value into the updated flight state estimation model, and outputting the state estimation value of the flight state of the UAV at the current moment, wherein the state estimation value output subunit is used to:

[0104] Inputting the flight state prediction value and the control input amount into a flight state update model to obtain the flight state prediction value of the UAV at the current moment relative to the previous moment; wherein the flight state update model is a model determined according to a transfer relationship quantity used to describe the state of the UAV from the previous moment to the current moment, a control input influence coefficient used to describe the influence of the control input amount at the previous moment on the flight state of the UAV, and the control input amount at the previous moment;

[0105] The flight state prediction value and the flight state observation value are input into a state observation update model, and a state estimation value of the UAV's flight state at the current moment is output, wherein the state observation update model is a model determined based on the flight state prediction value of the UAV at the current moment relative to the previous moment, a relationship function for describing the flight state observation value and the state prediction value, and a model for determining the weight of the flight state observation value when updating the state estimation.

[0106] As an embodiment, during the execution of a mission by the drone, if the sensor prediction state does not meet the flight requirements, the autonomous flight device further includes:

[0107] An actual flight parameter obtaining unit, used to detect the actual sensor status of the designated sensor in real time, and to obtain the actual flight parameters of the UAV to the next point after the flight mission has been completed at the current point;

[0108] The model updating unit is used to compare the actual flight parameters and the flight prediction parameters at the same time, and the difference between the sensor prediction state and the sensor actual state. If the difference meets the set threshold, the model parameters of the positioning estimation prediction model are adjusted, and the historical data within the set time period is used as the training sample to train the positioning estimation prediction model after the model parameters are adjusted, until the model accuracy of the trained positioning estimation prediction model reaches the accuracy threshold, and the trained positioning estimation prediction model is used as a new positioning estimation prediction model for updating.

[0109] As an embodiment, the flight prediction parameters include at least flight speed, heading angle and flight altitude.

[0110] It can be seen that in the technical solution provided in this embodiment, the autonomous flight device provided in the embodiment of the present application is applied to the control system of the unmanned aerial vehicle, and the point information and the sampling frequency suitable for each sensor are determined according to the flight mission information and the initial speed of the unmanned aerial vehicle, and the received sampling data are filtered and processed for consistency to obtain the target sampling data, and the flight state observation value determined from the target sampling data, the input control quantity obtained and the flight state prediction value are input into the flight state estimation model, and the state estimation value is output; the point information, the state estimation value and the positioning error of the specified sensor are input into the trained positioning estimation prediction model, and the sensor prediction state and the flight prediction parameters of the unmanned aerial vehicle flying to the next point are output; if the sensor prediction state meets the flight requirements, the unmanned aerial vehicle is controlled to fly to the next point according to the flight prediction parameters to perform the flight mission. It can be seen that the technical solution provided in the embodiment of the present application will not further improve the original perception hardware and chip computing power of the unmanned aerial vehicle, but through the adaptive learning algorithm integrated in the flight state estimation model, the unmanned aerial vehicle can automatically adjust the positioning parameters and algorithms according to the historical flight data and real-time environmental information, thereby improving the positioning accuracy and stability. By continuously learning flight data under different environmental conditions, the drone can gradually optimize the positioning estimation prediction model, reduce errors, and improve positioning accuracy. At the same time, by optimizing the design of the flight state estimation model, fully utilizing computing resources, improving computing efficiency and real-time performance, it can ensure that the drone can still maintain high performance and stability in complex environments.

[0111] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0112] The present application also provides an electronic device. From the hardware level, the hardware architecture diagram can be seen in Figure 4 As shown. It includes: a machine-readable storage medium and a processor, wherein: the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the autonomous flight operation disclosed in the above example.

[0113] The machine-readable storage medium provided in an embodiment of the present application stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to implement the autonomous flight operation disclosed in the above example.

[0114] Here, the machine-readable storage medium may be any electronic, magnetic, optical or other physical storage device that may contain or store information, such as executable instructions, data, etc. For example, the machine-readable storage medium may be: RAM (RadomAccess Memory), volatile memory, non-volatile memory, flash memory, storage drive (such as hard disk drive), solid state drive, any type of storage disk (such as CD, DVD, etc.), or similar storage medium, or a combination thereof.

[0115] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, a game console, a tablet computer, a wearable device or a combination of any of these devices.

[0116] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0117] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0118] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0119] Moreover, these computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0120] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0121] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial description of the method embodiments. The device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present application. A person of ordinary skill in the art may understand and implement it without creative work.

[0122] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. An autonomous flight method under multi-sensor fusion positioning, characterized in that: The drone is equipped with a plurality of sensors with different measurement functions required for the drone to perform autonomous flight. Each sensor is electrically connected to the control system of the drone. The autonomous flight method is applied to the control system. The autonomous flight method includes: According to the received flight mission information and the initial speed of the UAV, determine the point information for executing the flight mission and the sampling frequency suitable for each sensor, so that each sensor collects data according to its corresponding sampling frequency; Receive sampling data collected by each sensor at its corresponding sampling frequency and sent in a manner to ensure data integrity, perform filtering on the sampling data to obtain filtered sampling data, perform consistency processing on the filtered sampling data to obtain target sampling data at the same timestamp, format and coordinates; Obtaining a flight state prediction value of the UAV at the previous moment and an input control amount for the previous moment, and obtaining a flight state observation value of the UAV at the current moment from the target sampling data, inputting the flight state observation value, the input control amount and the flight state prediction value into an updated flight state estimation model, and outputting a state estimation value of the flight state of the UAV at the current moment; Determine the positioning error of the UAV from the positioning data obtained by the designated sensor used to measure the positioning function, input the point information, the state estimation value and the positioning error into the trained positioning estimation prediction model, and output the sensor prediction state of the designated sensor and the flight prediction parameters of the UAV to fly to the next point; If the sensor prediction state meets the flight requirements, the drone is controlled to fly to the next point to perform the flight mission according to the flight prediction parameters.

2. The autonomous flight method according to claim 1, characterized in that: The method of determining a sampling frequency suitable for each sensor according to the received mission information and the initial speed of the drone includes: Parse the received flight mission information, and obtain the position information, attitude information, sensor data and battery status data of the UAV from the flight mission information; According to the initial speed of the UAV and the acquired position information, attitude information, sensor data and battery status data, the sampling frequency suitable for each sensor is determined.

3. The autonomous flight method according to claim 1, characterized in that: The step of inputting the flight state observation value, the input control amount and the flight state prediction value into the updated flight state estimation model and outputting the state estimation value of the flight state of the UAV at the current moment includes: Inputting the flight state prediction value and the control input amount into a flight state update model to obtain the flight state prediction value of the UAV at the current moment relative to the previous moment; wherein the flight state update model is a model determined according to a transfer relationship quantity used to describe the state of the UAV from the previous moment to the current moment, a control input influence coefficient used to describe the influence of the control input amount at the previous moment on the flight state of the UAV, and the control input amount at the previous moment; The flight state prediction value and the flight state observation value are input into a state observation update model, and a state estimation value of the UAV's flight state at the current moment is output, wherein the state observation update model is a model determined based on the flight state prediction value of the UAV at the current moment relative to the previous moment, a relationship function for describing the flight state observation value and the state prediction value, and a model for determining the weight of the flight state observation value when updating the state estimation.

4. The autonomous flight method according to claim 1, characterized in that: During the mission execution of the UAV, if the sensor prediction state does not meet the flight requirements, it also includes: Real-time detection of the actual state of the designated sensor, and obtaining the actual flight parameters of the UAV to the next point after the current point has completed the flight mission; The actual flight parameters and flight prediction parameters at the same time, as well as the difference between the sensor prediction state and the sensor actual state are compared. If the difference meets the set threshold, the model parameters of the positioning estimation prediction model are adjusted, and the historical data within the set time period is used as the training sample to train the positioning estimation prediction model after the model parameters are adjusted, until the model accuracy of the trained positioning estimation prediction model reaches the accuracy threshold, and the trained positioning estimation prediction model is updated as a new positioning estimation prediction model.

5. The autonomous flight method according to any one of claims 1 to 4, characterized in that: The flight prediction parameters include at least flight speed, heading angle and flight altitude.

6. An autonomous flying device under multi-sensor fusion positioning, characterized in that: The drone is equipped with a plurality of sensors with different measurement functions required for the drone to perform autonomous flight. Each sensor is electrically connected to the control system of the drone. The autonomous flight device is applied to the control system. The autonomous flight device includes: The data sampling unit is used to determine the point information for executing the flight mission and the sampling frequency suitable for each sensor according to the received flight mission information and the initial speed of the UAV, so that each sensor collects data according to its corresponding sampling frequency; A data processing unit is used to receive the sampling data collected by each sensor according to the corresponding sampling frequency and sent in a manner to ensure data integrity, and filter the sampling data to obtain the filtered sampling data, and perform consistency processing on the filtered sampling data to obtain target sampling data under the same timestamp, the same format and the same coordinates; A state estimation unit, used to obtain a flight state prediction value of the UAV at the previous moment and an input control amount for the previous moment, and obtain a flight state observation value of the UAV at the current moment from the target sampling data, input the flight state observation value, the input control amount and the flight state prediction value into an updated flight state estimation model, and output a state estimation value of the flight state of the UAV at the current moment; A flight prediction unit is used to determine the positioning error of the UAV from the positioning data obtained by the designated sensor used to measure the positioning function, input the point information, the state estimation value and the positioning error into the trained positioning estimation prediction model, and output the sensor prediction state of the designated sensor and the flight prediction parameters of the UAV flying to the next point; if the sensor prediction state meets the flight requirements, the flight control unit is triggered; The flight control unit is used to control the UAV to fly to the next point to perform a flight mission according to the flight prediction parameters.

7. The autonomous flying device according to claim 6, characterized in that: The data sampling unit is specifically used for: Parse the received flight mission information, and obtain the position information, attitude information, sensor data and battery status data of the UAV from the flight mission information; According to the initial speed of the UAV and the acquired position information, attitude information, sensor data and battery status data, the sampling frequency suitable for each sensor is determined.

8. The autonomous flying device according to claim 6, characterized in that: The state estimation unit includes a state estimation value output subunit for inputting the flight state observation value, the input control amount and the flight state prediction value into the updated flight state estimation model, and outputting the state estimation value of the flight state of the UAV at the current moment, wherein the state estimation value output subunit is used to: Inputting the flight state prediction value and the control input amount into a flight state update model to obtain the flight state prediction value of the UAV at the current moment relative to the previous moment; wherein the flight state update model is a model determined according to a transfer relationship quantity used to describe the state of the UAV from the previous moment to the current moment, a control input influence coefficient used to describe the influence of the control input amount at the previous moment on the flight state of the UAV, and the control input amount at the previous moment; The flight state prediction value and the flight state observation value are input into a state observation update model, and a state estimation value of the UAV's flight state at the current moment is output, wherein the state observation update model is a model determined based on the flight state prediction value of the UAV at the current moment relative to the previous moment, a relationship function for describing the flight state observation value and the state prediction value, and a model for determining the weight of the flight state observation value when updating the state estimation.

9. The autonomous flying device according to claim 6, characterized in that: During the mission execution of the UAV, if the sensor prediction state does not meet the flight requirements, the autonomous flight device further includes: An actual flight parameter obtaining unit, used to detect the actual sensor status of the designated sensor in real time, and obtain the actual flight parameters of the UAV to the next point after the current point has completed the flight mission; The model updating unit is used to compare the actual flight parameters and the flight prediction parameters at the same time, and the difference between the sensor prediction state and the sensor actual state. If the difference meets the set threshold, the model parameters of the positioning estimation prediction model are adjusted, and the historical data within the set time period is used as the training sample to train the positioning estimation prediction model after the model parameters are adjusted, until the model accuracy of the trained positioning estimation prediction model reaches the accuracy threshold, and the trained positioning estimation prediction model is used as a new positioning estimation prediction model for updating.

10. An electronic device, characterized in that: It includes a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the execution steps of the autonomous flight method described in any one of claims 1 to 5 above.

Images