A method and device for recovering drones based on sound waves

Abstract

The present invention discloses a method and device for recovering drones based on acoustic waves. By utilizing the fact that inertial sensors are susceptible to the effects of acoustic wave injection, a method for recovering drones based on acoustic waves is implemented. By emitting acoustic waves within multiple resonant frequency ranges and acquiring information about the target drone's flight speed, direction, and position, the method analyzes the target inertial sensor's sampling rate and the phase difference caused by resonance and displacement in real time. Fine-grained acoustic wave modulation is used to produce stable, false data output. Furthermore, through the coordinated injection of multiple sound sources, the target inertial sensor is enabled to produce erroneous outputs in any direction, thereby deceiving the flight control system and causing it to deviate from its intended route. This allows the drone to be recovered promptly in areas with strong radio interference.

Description

Technical Field

[0001] The present invention relates to a method for recovering a drone, and in particular to a method and device for recovering a drone based on sound waves. Background Art

[0002] Currently, drones are widely used in both military and civilian applications. In the military, drones are used for reconnaissance and air strikes; in the civilian sector, they are employed in a variety of fields, including aerial photography, surveying and mapping, and disaster relief. The controller sends control commands to the drone via a remote control or ground tower. Upon receiving the commands, the drone's flight control system uses inertial sensors to determine its flight status (the gyroscope determines flight direction, and the accelerometer determines current flight speed). It then uses GPS to obtain its current location and adjusts its flight status based on the commands, directing the drone to a designated area to perform its mission. Upon completion, the drone is recovered by sending control commands. The transmission of drone control signals primarily relies on radio electromagnetic waves, including technologies such as WiFi, Bluetooth, and 4G networks.

[0003] However, with the continuous emergence and development of new technologies and services in the radio industry, the number of radio stations of varying sizes and nature continues to grow, and radio interference has become more common. Radio interference refers to the interference of electromagnetic energy during radio communications, which can cause problems and affect the quality of normal communication. When a drone enters an area with strong radio interference to perform a mission, radio communications will be blocked, and control commands cannot be smoothly transmitted to the flight control system, making it impossible to recover the drone. Summary of the Invention

[0004] The present invention addresses the problems existing in the prior art and provides a method and device for recovering a drone based on sound waves, namely, a method for controlling the motion trajectory of a drone based on sound waves. By utilizing the characteristic that inertial sensors are easily affected by ultrasonic injection, sound waves are used to control the inertial sensors inside a moving drone to produce stable error outputs, causing the drone's flight control system to misjudge the current flight direction and speed, thereby changing its flight trajectory and achieving the purpose of recovering the drone in an environment with strong radio interference or weak signals.

[0005] Specifically, this method uses sound-emitting devices such as speakers to send modulated sound waves, causing the drone's internal gyroscope or accelerometer to resonate and control its output, causing the drone to misjudge its current flight status, thereby changing the drone's motion trajectory.

[0006] The present invention is achieved through the following technical solutions:

[0007] A method for recovering a drone based on acoustic waves, comprising:

[0008] Obtain at least 2 drone images;

[0009] Calculate the flight direction, flight speed and location information of the drone through visual algorithms and acquired drone images;

[0010] Calculate the control direction and speed required to achieve the recovery purpose based on the flight direction and speed of the drone;

[0011] Continuously emit at least 2 sound waves within the resonant frequency range;

[0012] Obtain at least 2 drone images within the duration of each sound wave;

[0013] Calculate the sampling rate of the drone's internal inertial sensor and the initial phase difference when it is controlled through the drone image;

[0014] Select the inertial sensor type for controlling the target drone, either a gyroscope or an accelerometer, based on the control direction. If you need to change the flight direction of the target drone, control the gyroscope; if you need to control the acceleration or deceleration of the target drone, control the accelerometer.

[0015] Calculate the frequency of the sound waves to be emitted based on the type of inertial sensor and sampling rate selected for control;

[0016] Fine-grained modulation of the sound wave according to the frequency and initial phase difference of the desired emitted sound wave;

[0017] Calculate the gain coefficients of at least three speakers based on the drone's position information and control direction;

[0018] Multiplying the gain coefficient and the modulated sound wave to obtain the sound waves emitted by at least three speakers;

[0019] The control is performed by emitting sound waves, which are emitted through a speaker.

[0020] As a further improvement, the present invention calculates the sampling rate of the inertial sensor inside the drone using the drone image to eliminate the impact caused by the drift of the inertial sensor sampling rate, specifically:

[0021] When the sound wave frequency is within the resonant frequency range of the target inertial sensor, the following relationship exists between the sound wave frequency and the output frequency of the target inertial sensor:

[0022] f p =nFs+f dp , n∈N

[0023] where f pis the frequency of the emitted sound wave, Fs is the sampling rate of the target system, f dp is the frequency of the target inertial sensor output signal, when f dp When it is smaller, this frequency will be reflected as an obvious periodic motion of the UAV. This frequency is obtained by analyzing the captured UAV images and calculating the sampling rate of the inertial sensor in the target UAV by emitting at least two sound waves of different frequencies.

[0024] As a further improvement, the present invention calculates the initial phase difference of the inertial sensor inside the drone when it is controlled through the drone image;

[0025] The initial phase difference is composed of the phase difference introduced by resonance and the phase difference caused by the UAV movement, specifically:

[0026] The formula for the phase difference introduced by the resonance is as follows:

[0027] Where ξ is a constant, is the damping ratio, ω n =2πf n ,ω r =2πf r , f n and f r are the natural frequency of the inertial sensor and the frequency of the emitted sound wave, respectively. Since ξ and ω n Unknown, It cannot be calculated directly. By taking the derivative of the above formula, we get the following formula:

[0028]

[0029] where ω n With ω r Similar, and |ω r -ω n |<<ω n ,therefore can be approximated to 1, so we have is a constant, so the phase difference With ω r Linear correlation, This can be obtained by testing on a drone in advance, and the phase difference introduced by resonance can be calculated using the following formula:

[0030]

[0031] in The phase caused by the resonance when emitting sound waves can be obtained by analyzing the regular movement behavior of the drone through the image captured by the camera;

[0032] The phase difference caused by the UAV motion is calculated using the following formula:

[0033]

[0034] Where ΔL is the change in distance between the drone and the control device, which is calculated from the position information, and v is the speed of sound. The final initial phase difference is:

[0035]

[0036] As a further improvement, the present invention provides a method for fine-grained modulation of the sound wave according to the frequency and initial phase difference of the desired emitted sound wave; the sound wave modulation formula is as follows:

[0037]

[0038] where ω d =2πf d ,ω r =2πf r , f d and f r are the output frequency of the inertial sensor under the action of the emitted sound wave and the frequency of the emitted sound wave, is the initial phase difference.

[0039] As a further improvement, in the sound wave modulation process described in the present invention, in order to ensure that the amplitude of the modulated sound wave does not exceed the upper limit of the speaker's sound output, the following constraints are imposed:

[0040]

[0041] make Within the sound range of the speaker, adjust the initial phase as follows:

[0042]

[0043] As a further improvement, the present invention calculates the gain coefficients of at least three speakers based on the position information and control direction of the drone; specifically:

[0044] The acceleration direction of the accelerometer output is parallel to the direction of the line connecting the sound source and the accelerometer, and the angular velocity direction of the gyroscope output is perpendicular to the direction of the line connecting the sound source and the gyroscope. A new set of spatial basis vectors is formed by connecting at least three or more speakers and the target inertial sensor. The size of the component of the unit vector in the control direction on each basis vector is the gain coefficient of the corresponding speaker.

[0045] The present invention also discloses a UAV recovery device based on sound waves, comprising:

[0046] The first processing module: obtains at least two drone images; calculates the drone's flight direction, flight speed, and position information using a visual algorithm and the obtained drone images; and calculates the control direction and control speed required to achieve the recovery purpose based on the drone's flight direction and flight speed;

[0047] The first transmitting module continuously transmits sound waves within at least two resonant frequency ranges;

[0048] The second acquisition module: obtains at least two drone images within the duration of each sound wave; calculates the sampling rate of the drone's internal inertial sensor and the initial phase difference when it is controlled through the drone images;

[0049] The first selection module: selects the inertial sensor type of the target drone as a gyroscope or an accelerometer according to the control direction. If the flight direction of the target drone needs to be changed, the gyroscope is controlled; if the acceleration or deceleration of the target drone needs to be controlled, the accelerometer is controlled;

[0050] The first calculation module calculates the frequency of the required sound wave according to the selected inertial sensor type and sampling rate;

[0051] The first modulation module: modulates the sound wave in a fine-grained manner according to the frequency and initial phase difference of the required transmitted sound wave;

[0052] The second calculation module calculates the gain coefficients of at least three speakers based on the position information and control direction of the drone; multiplies the gain coefficients by the modulated sound waves to obtain the final sound waves emitted by at least three speakers;

[0053] The second transmitting module transmits sound waves for control, and the sound waves are transmitted through the speaker.

[0054] The beneficial effects of the present invention are as follows:

[0055] This invention exploits the vulnerability of inertial sensors to acoustic wave injection to implement an acoustic wave-based drone recovery method. By emitting acoustic waves within multiple resonant frequency ranges and acquiring information about the target drone's flight speed, direction, and location, the method analyzes the target inertial sensor's sampling rate and the phase difference caused by resonance and displacement in real time. Fine-grained acoustic wave modulation produces stable, spurious data output. Furthermore, through the coordinated injection of multiple sound sources, the target inertial sensor is able to generate erroneous outputs in any direction, thereby deceiving the flight control system and causing it to deviate from its intended route. This allows for the timely recovery of drones in areas with strong radio interference. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a data flow diagram of the present invention;

[0057] Figure 2 is a plot of the sound source location versus the affected axis of the inertial sensor. DETAILED DESCRIPTION

[0058] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments:

[0059] The purpose of the present invention is to address the situation where existing drone communication technology cannot normally transmit control instructions in an environment with strong radio interference. A drone recovery method based on sound waves is proposed. The device for implementing the method of the present invention includes a camera, a speaker, and a controller. Figure 1 It is a data flow diagram of the present invention;

[0060] The specific implementation method of the present invention is as follows:

[0061] Step 1: When the camera captures a drone, it takes at least two images of the drone and transmits them to the controller. In this embodiment, it takes two images.

[0062] Step 2: The controller uses the MVSCRF algorithm based on deep learning to calculate the flight direction, flight speed and position information of the drone through the obtained drone image;

[0063] Step three, the controller calculates the control direction and control speed required to achieve the recovery purpose based on the flight direction and flight speed of the drone. Specifically, it is necessary to control the drone to generate an acceleration opposite to the flight direction until the speed drops to 0, and then control the drone to generate a downward speed until it lands. For example, if the drone is flying to the left, it is necessary to control it to generate an acceleration to the right until the speed drops to 0, and then control it to fly downward until it lands. In order to achieve the target control direction, it is necessary to use sound waves to make the target inertial sensor generate erroneous data that is opposite to the control direction. For example, if the drone is in a state of flying to the left, to control the drone to generate an acceleration to the right, the accelerometer can be made to output an acceleration to the left through sound waves, causing the flight control system to mistakenly judge that it is accelerating to the left. When the accumulated leftward flight speed exceeds the pre-set flight speed, the flight control system will make a decision to accelerate to the right to correct the flight speed, and ultimately the drone generates an acceleration to the right;

[0064] Step 4: Use a loudspeaker to continuously transmit sound waves within a resonant frequency range of at least two or more. In this embodiment, the resonant frequency range is two. The resonant frequency range can be obtained through prior experiments or by referring to the data sheet of the inertial sensor in the drone.

[0065] Step 5: Use the camera to obtain at least two drone images during the duration of each sound wave, three in this embodiment;

[0066] Step 6: The controller uses the drone image to calculate the sampling rate of the drone's internal inertial sensor to eliminate the effects of inertial sensor sampling rate drift. When the sound wave frequency is within the resonant frequency range of the target inertial sensor, the relationship between the sound wave frequency and the output frequency of the target inertial sensor is as follows:

[0067] f p =nFs+f dp , n∈N

[0068] where f p is the frequency of the emitted sound wave, Fs is the sampling rate of the target system, f dp is the frequency of the target inertial sensor output signal, when f dp When it is small, the frequency will be reflected as an obvious periodic motion of the drone. The frequency is obtained by analyzing the position pattern of the drone in the captured drone image. By emitting at least two sound waves of different frequencies, at least two relationship equations are obtained to calculate the sampling rate of the inertial sensor in the target drone. In this embodiment, there are two relationship equations.

[0069] Step 7: The controller uses the drone image to calculate the initial phase difference of the drone's internal inertial sensor. The initial phase difference is composed of the phase difference introduced by resonance and the phase difference caused by the drone's motion. The formula for the phase difference introduced by resonance is as follows:

[0070]

[0071] Where ξ is a constant, is the damping ratio, ω n =2πf n ,ω r =2πf r , f n and f r are the natural frequency of the inertial sensor and the frequency of the emitted sound wave, respectively. Since ξ and ω n Unknown, It cannot be calculated directly. By taking the derivative of the above formula, we get the following formula:

[0072]

[0073] where ω n With ω r Similar, and |ω r -ω n |<<ω n ,therefore can be approximated to 1, so we have is a constant, so the phase difference With ω r Linear correlation, This can be obtained by testing on a drone in advance, and the phase difference introduced by resonance can be calculated using the following formula:

[0074]

[0075] in The phase caused by the resonance when emitting sound waves can be obtained by analyzing the regular movement behavior of the drone through the image captured by the camera;

[0076] The phase difference caused by the UAV motion is calculated using the following formula:

[0077]

[0078] Where ΔL is the change in distance between the drone and the control device, which is calculated from the position information, and v is the speed of sound. The final initial phase difference is:

[0079]

[0080] Step 8: The controller selects the type of inertial sensor to control the target drone, either a gyroscope or an accelerometer, according to the control direction. If the flight direction of the target drone needs to be changed, the gyroscope is controlled; if the acceleration or deceleration of the target drone needs to be controlled, the accelerometer is controlled.

[0081] Step 9. Calculate the frequency f of the sound wave to be emitted based on the selected inertial sensor type and sampling rate. r and the output frequency f of the target inertial sensor d ;

[0082] Step 10: Fine-grainedly modulate the sound wave according to the frequency and initial phase difference of the sound wave to be emitted. The sound wave modulation formula is as follows:

[0083]

[0084] where ω d =2πf d ,ω r =2πf r , f d and f r are the natural frequency of the inertial sensor and the frequency of the emitted sound wave, is the initial phase difference.

[0085] In order to ensure that the amplitude of the modulated sound wave does not exceed the upper limit of the speaker's sound output, the following constraints are imposed on it:

[0086]

[0087] make Within the sound range of the speaker, adjust the initial phase as follows:

[0088]

[0089] Step 11: The controller calculates the gain coefficients of at least three speakers based on the drone's position information and control direction. In this embodiment, there are three speakers. Under the action of sound waves within the resonant frequency range, the acceleration direction output by the accelerometer is parallel to the direction of the line connecting the sound source and the accelerometer, and the angular velocity direction output by the gyroscope is perpendicular to the direction of the line connecting the sound source and the gyroscope. Figure 2 This graph shows the relationship between the sound source position and the affected axis of the inertial sensor. The result of multiple sound waves is their combined direction and magnitude. A new set of spatial basis vectors is formed using the lines connecting the three speakers and the target inertial sensor. The gain coefficient for the corresponding speaker is calculated by calculating the magnitude of the component of the unit vector in the control direction on these three basis vectors.

[0090] Step 12: multiply the modulated sound wave by the gain coefficient to obtain the sound waves emitted by at least three speakers, in this embodiment, three speakers;

[0091] Step 13: Each speaker emits sound waves to control the drone;

[0092] Step 14: Repeat steps 1 to 13, continuously adjusting the emitted sound waves until the target drone is recovered.

[0093] The present invention also discloses a UAV recovery device based on sound waves, comprising:

[0094] The first processing module: obtains at least two drone images; calculates the drone's flight direction, flight speed, and position information using a visual algorithm and the obtained drone images; and calculates the control direction and control speed required to achieve the recovery purpose based on the drone's flight direction and flight speed;

[0095] The first transmitting module continuously transmits sound waves within at least two resonant frequency ranges;

[0096] The second acquisition module: obtains at least two drone images within the duration of each sound wave; calculates the sampling rate of the drone's internal inertial sensor and the initial phase difference when it is controlled through the drone images;

[0097] The first selection module: selects the inertial sensor type of the target drone as a gyroscope or an accelerometer according to the control direction. If the flight direction of the target drone needs to be changed, the gyroscope is controlled; if the acceleration or deceleration of the target drone needs to be controlled, the accelerometer is controlled;

[0098] The first calculation module calculates the frequency of the required sound wave according to the selected inertial sensor type and sampling rate;

[0099] The first modulation module: modulates the sound wave in a fine-grained manner according to the frequency and initial phase difference of the required transmitted sound wave;

[0100] The second calculation module calculates the gain coefficients of at least three speakers based on the position information and control direction of the drone; multiplies the gain coefficients by the modulated sound waves to obtain the final sound waves emitted by at least three speakers;

[0101] The second transmitting module transmits sound waves for control, and the sound waves are transmitted through the speaker.

[0102] The above description is not intended to limit the present invention. It should be noted that a person skilled in the art may make several changes, modifications, additions or substitutions without departing from the essential scope of the present invention. Such improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A drone recovery method based on sound waves, characterized in that: include: Obtain at least 2 drone images; Calculate the flight direction, flight speed and location information of the drone through visual algorithms and acquired drone images; Calculate the control direction and speed required to achieve the recovery purpose based on the flight direction and speed of the drone; Utilize a loudspeaker to continuously emit sound waves within at least two resonant frequency ranges; Obtain at least 2 drone images within each acoustic wave duration; Calculate the sampling rate of the drone's internal inertial sensor and the initial phase difference when it is controlled through the drone image; Select the inertial sensor type for controlling the target drone, either a gyroscope or an accelerometer, based on the control direction. If you need to change the flight direction of the target drone, control the gyroscope; if you need to control the acceleration or deceleration of the target drone, control the accelerometer. Calculate the frequency of the sound waves to be emitted based on the type of inertial sensor and sampling rate selected for control; Fine-grained modulation of the sound wave according to the frequency and initial phase difference of the desired emitted sound wave; Calculate the gain coefficients of at least three speakers based on the drone's position information and control direction; Multiplying the gain coefficient and the modulated sound wave to obtain the sound waves emitted by at least three speakers; transmitting sound waves to control the drone, wherein the sound waves are transmitted through a speaker; Continuously adjust the emitted sound waves until the target drone is recovered; The sound wave is modulated in a fine-grained manner according to the frequency and initial phase difference of the sound wave to be emitted; the sound wave modulation formula is as follows: where ω d =2πf d ,ω r =2πf r ,f d and f r are the output frequency of the inertial sensor under the action of the emitted sound wave and the frequency of the emitted sound wave, is the initial phase difference; During the sound wave modulation process, in order to ensure that the amplitude of the modulated sound wave does not exceed the upper limit of the speaker's sound output, the following constraints are imposed: make Within the sound range of the speaker, adjust the initial phase as follows: The gain coefficients of at least three speakers are calculated based on the position information and control direction of the drone; specifically: The acceleration output by the accelerometer is parallel to the line connecting the sound source and the accelerometer, and the angular velocity output by the gyroscope is perpendicular to the line connecting the sound source and the gyroscope. A new set of spatial basis vectors is formed using the lines connecting at least three speakers and the target inertial sensor. The magnitude of the component of the unit vector in the control direction on each basis vector is the gain coefficient of the corresponding speaker. The sampling rate of the inertial sensor inside the drone is calculated by using the drone image to eliminate the influence caused by the drift of the inertial sensor sampling rate. Specifically, When the sound wave frequency is within the resonant frequency range of the target inertial sensor, the following relationship exists between the sound wave frequency and the output frequency of the target inertial sensor: f p =nFs+f dp ,n∈N where f p is the frequency of the emitted sound wave, Fs is the sampling rate of the target system, f dp is the frequency of the target inertial sensor output signal, when f dp When f is small, dp Will be reflected as the obvious periodic motion of the drone, f dp By analyzing the captured drone image acquisition, the sampling rate of the inertial sensor in the target drone is calculated by emitting at least two sound waves of different frequencies; The initial phase difference when the inertial sensor inside the drone is controlled is calculated by using the drone image; the initial phase difference is composed of the phase difference introduced by the resonance and the phase difference caused by the drone movement, specifically: The formula for the phase difference introduced by the resonance is as follows: Where ξ is a constant, is the damping ratio, ω n =2πf n ,ω r =2πf r ,f n and f r are the natural frequency of the inertial sensor and the frequency of the emitted sound wave, respectively. Since ξ and ω n Unknown, It cannot be calculated directly. By taking the derivative of the above formula, we get the following formula: where ω n With ω r Similar, and |ω r -ω n |<<ω n ,therefore can be approximated to 1, so we have is a constant, so the phase difference With ω r Linear correlation, This can be obtained by testing on a drone in advance, and the phase difference introduced by resonance can be calculated using the following formula: in The phase caused by the resonance when emitting sound waves can be obtained by analyzing the regular movement behavior of the drone through the image captured by the camera; The phase difference caused by the UAV motion is calculated using the following formula: Where ΔL is the change in distance between the drone and the control device, which is calculated from the position information, and v is the speed of sound. The final initial phase difference is:

2. A device for the drone recovery method based on acoustic waves as claimed in claim 1, characterized in that: include: The first processing module is used to obtain at least two drone images; Calculate the flight direction, flight speed, and location of the drone using visual algorithms and the acquired drone images; and calculate the control direction and speed required to achieve the recovery objective based on the drone's flight direction and speed. The first transmitting module is used to continuously transmit sound waves within at least two resonant frequency ranges using a loudspeaker; The second acquisition module is used to obtain at least two drone images within the duration of each sound wave; the sampling rate of the drone's internal inertial sensor and the initial phase difference when it is controlled are calculated based on the drone images; The first selection module is used to select the inertial sensor type for controlling the target drone, either a gyroscope or an accelerometer, according to the control direction. If the flight direction of the target drone needs to be changed, the gyroscope is controlled; if the acceleration or deceleration of the target drone needs to be controlled, the accelerometer is controlled. The first calculation module is used to calculate the frequency of the required transmitted sound waves according to the selected inertial sensor type and sampling rate; The first modulation module is used to modulate the sound wave in a fine-grained manner according to the frequency and initial phase difference of the required transmitted sound wave; The second calculation module is used to calculate the gain coefficient of each of the at least three speakers based on the position information and control direction of the drone; and multiply the gain coefficient by the modulated sound wave to obtain the sound wave emitted by each of the at least three speakers. The second transmitting module is used to transmit sound waves to control the drone, and the sound waves are transmitted through the speaker; The second modulation module is used to continuously adjust the emitted sound waves until the target drone is recovered.

Images