Airborne emergency landing system based on acceleration sensor and vision fusion

Abstract

The invention provides an airborne emergency landing system based on acceleration sensor and vision fusion, an unmanned aerial vehicle is provided with a 5G module terminal, a parachute is internally provided with an acceleration sensor and is powered by the 5G module terminal, and a pan-tilt camera is connected with a rear-end platform through the 5G module terminal and is used for outputting a pan-tilt video picture. The parachute triggers an automatic parachute opening mechanism through an acceleration sensor and a holder video picture. According to the invention, the acceleration change of the unmanned aerial vehicle is monitored by using the acceleration sensor, and the flight state is accurately monitored and judged in combination with the pan-tilt video data of the pan-tilt camera; an automatic parachute opening mechanism can be triggered by the acceleration sensor and video pictures of the holder, so that the parachute can be quickly and safely started under various emergency conditions; through mounting of the 5G module terminal, a user can realize a 5G network connection function, and meanwhile, the design of the externally expanded parachute ensures that the unmanned aerial vehicle has the emergency landing capability in the flight process.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles, and in particular to an airborne emergency landing system based on acceleration sensor and vision fusion. Background Art

[0002] With the rapid development and widespread application of drone technology, especially in the fields of commerce, agriculture, logistics and public safety, it is particularly important to ensure the safety of drones during flight. The airborne parachute system can provide an effective emergency landing solution when the drone encounters sudden failures, such as power system failure, flight control system failure, etc., to reduce the potential threat of crashes to people and property. In some special scenarios, such as natural disasters and fire monitoring, the airborne parachute can help drones return to the ground quickly and safely when performing missions, ensuring the safety of equipment and data, while providing support for emergency response.

[0003] However, the existing airborne parachute system has many shortcomings. Take DJI drones as an example: due to the occupation of the PSDK interface, DJI drones cannot simultaneously mount other terminal devices through this interface, which directly affects its ability to achieve 5G network connection, video transmission, and data transmission control. This limitation not only reduces the operational flexibility of drones in complex environments, but also limits their application potential in real-time data transmission and remote control. Therefore, it is urgent to explore new solutions to ensure that while ensuring safety, its multi-functional integration capabilities can be improved to meet the growing market demand and technological development trends.

[0004] The existing airborne parachute system uses a method of directly connecting the parachute load through the PSDK interface to obtain the aircraft's flight status information. This method combines the drone's own acceleration sensor to promptly determine and trigger the parachute's opening mechanism in an emergency. However, this design has a significant flaw, that is, the interface occupies the terminal's image and data transmission channels, limiting the drone's versatility. Summary of the invention

[0005] To solve the above problems, the present invention aims to propose an airborne emergency landing system based on acceleration sensor and vision fusion. By using acceleration sensor to monitor the acceleration change of UAV and combining with the pan-tilt video data of pan-tilt camera, accurate monitoring and judgment of flight status can be achieved; both acceleration sensor and pan-tilt video screen can trigger automatic parachute opening mechanism, which can ensure that the parachute can be quickly and safely started in various emergency situations.

[0006] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0007] An airborne emergency landing system based on acceleration sensor and visual fusion, comprising a drone equipped with a parachute and a gimbal camera, the drone being equipped with a 5G module terminal, the parachute having a built-in acceleration sensor and powered by the 5G module terminal, the gimbal camera being connected to a back-end platform via the 5G module terminal for outputting gimbal video images, and the parachute triggering an automatic parachute opening mechanism via the acceleration sensor and the gimbal video images.

[0008] Furthermore, the parachute triggers the automatic opening mechanism through the acceleration sensor as follows: the 5G module terminal supplies power to the parachute. When the parachute is powered on, the built-in acceleration sensor will measure the acceleration value in real time; when the weightlessness condition is triggered, the parachute will automatically open.

[0009] Furthermore, the weightlessness condition is that the weightlessness reaches 0.5 gravitational acceleration and lasts for 1 second.

[0010] Furthermore, the 5G module terminal is connected to the parachute through a serial port. When the 5G module terminal returns to the parachute opening state through the serial port, the 5G module terminal sends a propeller stop command to the drone through the PSDK interface.

[0011] Furthermore, the parachute triggers the automatic parachute opening mechanism through the gimbal video screen as follows: the gimbal video screen captured by the gimbal camera on the drone is transmitted to the back-end platform in real time through the 5G module terminal, and the DepthFM image depth estimation algorithm is deployed on the back-end platform; when it is detected that the nearest obstacle target in the picture is lower than the preset target distance threshold, the platform will send a parachute opening command to the 5G module terminal, and the 5G module terminal will send the "open parachute" command to the parachute through the serial port connection with the parachute, and at the same time send a "stop propeller" command to the drone through the PSDK interface to stop the propellers of the drone from rotating.

[0012] Furthermore, the target distance threshold is 2 meters.

[0013] Furthermore, the DepthFM image depth estimation algorithm performs the following image prediction depth-target distance process: on the basis of obtaining the image and depth pairing, the intrinsic relationship between the conditional signal of the image and the target depth field is combined; x1 represents the depth sample encoded in the latent space, and the starting point x0 corresponds to the encoded representation of the corresponding image; smoothing is performed around the data sample to obtain the corresponding data distribution

[0014]

[0015] Gaussian flow is defined by the following equation:

[0016]

[0017] ut (x|z)=x1-x0,

[0018] φ t (x|z)=tx1+(1-t)x0

[0019] Although x0 and x1 belong to different modalities and data manifolds, the optimal transmission condition between p(x0) and p(x1) is naturally satisfied due to the image-to-depth pairing relationship; this solves the problem of dynamic optimal transmission in the image-to-depth conversion process in the flow matching paradigm, ensuring more stable and faster training; in this setting, the form of the loss function is:

[0020]

[0021] Our depth estimation model is trained on two synthetic datasets, namely the KITTI public dataset and the DBAD drone air-to-ground perspective image dataset; in order to fully evaluate its performance, the true depth is denoted as d and the predicted depth is denoted as ; The evaluation indicators include the mean relative error, which is calculated as follows:

[0022]

[0023] And the δ1 accuracy measure satisfies the condition proportion.

[0024] Beneficial effects: The present invention uses an acceleration sensor to monitor the acceleration changes of the UAV, and combines it with the pan-tilt video data of the pan-tilt camera to achieve accurate monitoring and judgment of the flight status; the acceleration sensor and the pan-tilt video image can both trigger the automatic parachute opening mechanism, which can ensure that the parachute can be quickly and safely activated in various emergency situations. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0026] Figure 1 It is a structural schematic diagram of an airborne emergency landing system based on acceleration sensor and vision fusion according to an embodiment of the present invention;

[0027] Figure 2 It is a schematic diagram of the automatic parachute opening mechanism triggered by the acceleration sensor of the parachute in the airborne emergency landing system based on the fusion of the acceleration sensor and vision according to the embodiment of the present invention;

[0028] Figure 3This is a schematic diagram of the automatic parachute opening mechanism triggered by the gimbal video image in the airborne emergency landing system based on acceleration sensor and vision fusion described in an embodiment of the present invention. DETAILED DESCRIPTION

[0029] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0030] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0031] Example 1

[0032] See also Figure 1-3 : An airborne emergency landing system based on acceleration sensor and visual fusion, including a drone equipped with a parachute and a gimbal camera, the drone is equipped with a 5G module terminal, the parachute has a built-in acceleration sensor and is powered by the 5G module terminal, the gimbal camera is connected to the back-end platform through the 5G module terminal for outputting gimbal video images, and the parachute triggers an automatic parachute opening mechanism through the acceleration sensor and gimbal video images.

[0033] This embodiment uses an acceleration sensor to monitor the acceleration changes of the drone, and combines it with the pan-tilt video data of the pan-tilt camera to achieve accurate monitoring and judgment of the flight status; the acceleration sensor and the pan-tilt video screen can trigger the automatic parachute opening mechanism, which can ensure that the parachute can be quickly and safely launched in various emergency situations; by mounting the 5G module terminal, the user can realize the 5G network connection function, and the design of the external parachute ensures that the drone has the ability to make an emergency landing during the flight; this mechanism not only improves the safety of the drone, but also provides users with higher operational flexibility.

[0034] It should be noted that, when 5G networking is achieved by mounting the 5G module terminal on the drone, this embodiment realizes the function of the 5G module terminal externally extending the parachute; through this design, the user can increase the parachute load without affecting other functions of the drone, ensuring that the parachute can be quickly activated in an emergency to reduce equipment losses; considering the high value of the drone equipment (such as the gimbal is worth 70,000 yuan, and the drone itself is worth more than 60,000 yuan), timely opening of the parachute can effectively protect the safety of the equipment.

[0035] In a specific example, the parachute triggers the automatic opening mechanism through the acceleration sensor as follows: the 5G module terminal supplies power to the parachute. When the parachute is powered on, the built-in acceleration sensor will measure the acceleration value in real time; when the weightlessness condition is triggered, the parachute will automatically open.

[0036] This embodiment achieves accurate parachute opening judgment through the acceleration sensor; the system ensures that the parachute can be activated in time under specific conditions, such as weightlessness such as rapid altitude descent and loss of control.

[0037] In a specific example, the weightlessness condition is that the weightlessness reaches 0.5 gravitational acceleration and lasts for 1 second.

[0038] In a specific example, the 5G module terminal is connected to the parachute through a serial port. When the 5G module terminal returns to the parachute opening state through the serial port, the 5G module terminal sends a propeller stop command to the drone through the PSDK interface.

[0039] In this embodiment, when the 5G module terminal learns that the parachute is opened, it can trigger the automatic propeller stop function and send a propeller stop command through the PSDK interface to control the propeller blades of the drone to stop rotating, thereby avoiding the propeller blades from causing secondary damage to the parachute or the surrounding environment. This not only improves the effectiveness of the parachute, but also enhances the overall flight safety, ensuring the safe landing of the drone in an emergency.

[0040] In a specific example, the parachute triggers the automatic parachute opening mechanism through the gimbal video screen as follows: the gimbal video screen captured by the gimbal camera on the drone is transmitted to the back-end platform in real time through the 5G module terminal, and the DepthFM image depth estimation algorithm is deployed on the back-end platform; when the nearest obstacle target in the picture is detected to be lower than the preset target distance threshold, the platform will send a parachute opening command to the 5G module terminal, and the 5G module terminal will send the "open parachute" command to the parachute through the serial port connection with the parachute, and at the same time send the "stop propeller" command to the drone through the PSDK interface to stop the propellers of the drone from rotating.

[0041] This embodiment achieves accurate parachute opening judgment through the gimbal video screen; the system ensures that under specific conditions, such as when the distance to the target is too close or when it is about to hit an obstacle, the parachute can be activated in time, and the automatic propeller stop function is activated at the same time to prevent the rotating propeller blades from causing secondary damage to the parachute or surrounding objects, which not only improves the effectiveness of the parachute, but also enhances the overall flight safety and ensures the safe landing of the UAV in emergency situations.

[0042] In a specific example, the target distance threshold is 2 meters.

[0043] In a specific example, the DepthFM image depth estimation algorithm performs the following image prediction depth-target distance process: on the basis of obtaining the image and depth pairing, combining the intrinsic relationship between the conditional signal of the image and the target depth field; x1 represents the depth sample encoded in the latent space, and the starting point x0 corresponds to the encoded representation of the corresponding image; smoothing is performed around the data sample to obtain the corresponding data distribution

[0044] Gaussian flow is defined by the following equation:

[0045]

[0046] u t (x|z)=x1-x0,

[0047] φ t (x|z)=tx1+(1-t)x0

[0048] Although x0 and x1 belong to different modalities and data manifolds, the optimal transmission condition between p(x0) and p(x1) is naturally satisfied due to the image-to-depth pairing relationship; this solves the problem of dynamic optimal transmission in the image-to-depth conversion process in the flow matching paradigm, ensuring more stable and faster training; in this setting, the form of the loss function is:

[0049]

[0050] The advantage of this approach is that it fully exploits the intrinsic relationship between image and depth data, allowing the model to learn the mapping from image to depth more effectively. By solving the optimal transfer problem within the flow matching framework, we are able to achieve a more stable and efficient training process, thereby improving the accuracy of depth estimation and the overall performance of the model.

[0051] Our depth estimation model is trained on two synthetic datasets, namely the KITTI public dataset and the DBAD (Drone-based aerial-to-ground view image dataset); in order to fully evaluate its performance, the true depth is denoted as d and the predicted depth is denoted as ; The evaluation indicators include the mean relative error, which is calculated as follows:

[0052]

[0053] And the δ1 accuracy measure satisfies the condition proportion.

[0054] In this embodiment, the δ1 accuracy of the method on the KITTI and DBAD datasets is 93.4% and 91.2% respectively, which meets the expected goal.

[0055] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An airborne emergency landing system based on acceleration sensor and vision fusion, characterized in that: It includes a drone equipped with a parachute and a gimbal camera, wherein a 5G module terminal is installed on the drone, the parachute has a built-in acceleration sensor and is powered by the 5G module terminal, the gimbal camera is connected to a back-end platform via the 5G module terminal for outputting gimbal video images, and the parachute triggers an automatic parachute opening mechanism via the acceleration sensor and the gimbal video images.

2. The airborne emergency landing system based on acceleration sensor and vision fusion according to claim 1 is characterized in that: The parachute triggers the automatic opening mechanism through the acceleration sensor as follows: the 5G module terminal supplies power to the parachute. When the parachute is powered on, the built-in acceleration sensor will measure the acceleration value in real time; when the weightlessness condition is triggered, the parachute will automatically open.

3. The airborne emergency landing system based on acceleration sensor and vision fusion according to claim 2 is characterized in that: The weightlessness condition is that the weightlessness reaches 0.5 gravitational acceleration and lasts for 1 second.

4. The airborne emergency landing system based on acceleration sensor and vision fusion according to claim 2 is characterized in that: The 5G module terminal is connected to the parachute through a serial port. When the 5G module terminal returns to the parachute opening state through the serial port, the 5G module terminal sends a propeller stop command to the drone through the PSDK interface.

5. The airborne emergency landing system based on acceleration sensor and vision fusion according to claim 1, characterized in that: The parachute triggers the automatic parachute opening mechanism through the gimbal video screen as follows: the gimbal video screen captured by the gimbal camera on the drone is transmitted to the back-end platform in real time through the 5G module terminal, and the DepthFM image depth estimation algorithm is deployed on the back-end platform; when the nearest obstacle target in the picture is detected to be lower than the preset target distance threshold, the platform will send a parachute opening command to the 5G module terminal, and the 5G module terminal will send the "parachute opening" command to the parachute through the serial port connection with the parachute, and at the same time send the "stop propeller" command to the drone through the PSDK interface to stop the propellers of the drone from rotating.

6. The airborne emergency landing system based on acceleration sensor and vision fusion according to claim 5 is characterized in that: The target distance threshold is 2 meters.

7. The airborne emergency landing system based on acceleration sensor and vision fusion according to claim 5, characterized in that: The DepthFM image depth estimation algorithm performs image prediction depth-target distance process as follows: on the basis of obtaining image and depth pairing, the intrinsic relationship between the conditional signal of the image and the target depth field is combined; x1 represents the depth sample encoded in the latent space, and the starting point x0 corresponds to the encoded representation of the corresponding image; smoothing is performed around the data sample to obtain the corresponding data distribution Gaussian flow is defined by the following equation: u t (x|z)=x1-x0, φ t (x|z)=tx1+(1-t)x0 Although x0 and x1 belong to different modalities and data manifolds, the optimal transmission condition between p(x0) and p(x1) is naturally satisfied due to the image-to-depth pairing relationship; The problem of dynamic optimal transfer in the image-to-depth conversion process in the flow matching paradigm is solved, ensuring more stable and faster training; in this setting, the loss function takes the form: Our depth estimation model is trained on two synthetic datasets, namely the KITTI public dataset and the DBAD drone air-to-ground perspective image dataset; in order to fully evaluate its performance, the true depth is denoted as d and the predicted depth is denoted as The evaluation indicators include the mean relative error, which is calculated as: And the δ1 accuracy measure satisfies the condition proportion.

Images