Unmanned aerial vehicle capturing method and device, electronic equipment and storage medium

Abstract

According to the unmanned aerial vehicle capturing method and device, the electronic equipment and the storage medium, the operation sound and image of the unmanned aerial vehicle are collected, the recognition reliability is improved by using double information sources, and whether the target object is the unmanned aerial vehicle or not can be more accurately judged in combination with the sound features and the image features, so that the recognition accuracy is remarkably improved. In addition, the motion state of the unmanned aerial vehicle is analyzed by using a Doppler effect algorithm, and the operation speed of the unmanned aerial vehicle is identified in combination with an optical flow algorithm, thereby realizing comprehensive monitoring of the motion state of the unmanned aerial vehicle. The multi-dimensional monitoring mode not only improves the accuracy of unmanned aerial vehicle position determination, but also provides powerful data support for subsequent unmanned aerial vehicle capture, so that the accuracy of unmanned aerial vehicle identification and the success rate of capture can be significantly improved.

Description

Technical Field

[0001] The present application relates to the field of anti-UAV technology, and in particular to a method, device, electronic device and storage medium for capturing UAVs. Background Art

[0002] With the development of drone technology, more and more drones are used in different fields. However, unauthorized drone flight may cause various safety hazards. For example, drones may enter no-fly zones. Therefore, drone capture is needed to detect and stop these potentially dangerous behaviors in time, thereby maintaining public safety.

[0003] Based on the above situation, the drone capture method used in the prior art has a relatively simple monitoring method for drones, which makes it difficult to ensure the accuracy of drone identification and thus cannot guarantee the success rate of drone capture. Summary of the invention

[0004] In view of this, the purpose of this application is to propose a drone capture method, device, electronic device and storage medium to solve the above-mentioned technical problems.

[0005] Based on the above purpose, the first aspect of the present application provides a drone capture method, which is applied to a controller provided with a capture device, and the method comprises:

[0006] Collecting the running sound of the target monitoring object appearing in the monitoring area and the image of the target monitoring object;

[0007] Determining whether the target monitoring object is a drone based on the operating sound of the target monitoring object and the image of the target monitoring object;

[0008] In response to the target monitoring object being a drone, a Doppler effect algorithm is used to analyze the motion state of the running sound of the target monitoring object to obtain the motion state of the drone, and an optical flow algorithm is used to identify and process the image of the target monitoring object to obtain the running speed of the drone;

[0009] The position of the drone is determined according to the running state of the drone and the running speed of the drone, and the drone is captured by using the capture device based on the position of the drone.

[0010] Optionally, determining whether the target monitoring object is a drone based on the running sound of the target monitoring object and the image of the target monitoring object includes:

[0011] Extracting acoustic feature parameters of the running sound of the target monitoring object to obtain acoustic feature parameters, and extracting features of the image of the target monitoring object to obtain image features;

[0012] Matching the acoustic feature parameters with a pre-established drone sound sample library, determining the sound similarity between the acoustic feature parameters and the sound samples stored in the drone sound sample library, performing bounding box recognition processing according to the image features to generate a bounding box, matching the bounding box with a pre-established drone bounding box sample library, and determining the bounding box similarity between the bounding box and the bounding box samples stored in the drone bounding box sample library;

[0013] In response to the sound similarity being greater than or equal to a preset sound similarity threshold, and the bounding box similarity being greater than or equal to a preset bounding box similarity, determining that the target monitoring object is a drone; or,

[0014] In response to the sound similarity being less than a preset sound similarity threshold, and the bounding box similarity being greater than or equal to a preset bounding box similarity, determining that the target monitoring object is not a drone; or,

[0015] In response to the sound similarity being greater than or equal to a preset sound similarity threshold, and the bounding box similarity being less than a preset bounding box similarity, it is determined that the target monitoring object is not a drone.

[0016] Optionally, in response to the sound similarity being greater than or equal to a preset sound similarity threshold, and the bounding box similarity being greater than or equal to a preset bounding box similarity, after determining that the target monitoring object is a drone, the method further includes:

[0017] Searching for acoustic feature parameters and image features corresponding to the target monitoring object from a pre-built drone feature library, wherein the drone feature library stores a plurality of acoustic feature parameters and image features;

[0018] In response to the absence of acoustic feature parameters and / or image features corresponding to the target monitoring object in the drone feature library, the acoustic feature parameters and / or image features corresponding to the target monitoring object are stored in the drone feature library.

[0019] Optionally, the using a Doppler effect algorithm to perform motion state analysis on the running sound of the target monitoring object to obtain the motion state of the drone includes:

[0020] Performing sound frequency analysis based on the operating sound of the target monitoring object to obtain a sound spectrum;

[0021] Determine the sound frequency change parameters collected by the controller according to the sound spectrum, and determine the running direction of the drone using the sound frequency change parameters;

[0022] Acquiring the sound frequency collected by the controller, the sound frequency emitted by the target monitoring object, and the propagation speed of the sound in the medium;

[0023] Based on the sound frequency collected by the controller, the sound frequency emitted by the target monitoring object and the propagation speed of the sound in the medium, the velocity component of the speed of the drone relative to the controller in the direction of the line connecting the controller and the drone is determined by the following formula:

[0024] vs = (f′-f) × v / f

[0025] Wherein, vs represents the velocity component of the UAV relative to the controller in the direction of the line connecting the controller and the UAV, f′ represents the sound frequency collected by the controller, f represents the sound frequency emitted by the target monitoring object, and v represents the propagation speed of the sound in the medium;

[0026] Obtain the time required for the sound to propagate from the drone to the controller and the angle between the drone's running direction and the line connecting the controller and the drone;

[0027] Based on the time required for sound to propagate from the drone to the controller, the angle between the drone's running direction and the line connecting the controller and the drone, the speed component of the drone's speed relative to the controller in the direction of the line connecting the controller and the drone, and the propagation speed of sound in the medium, the distance of the drone relative to the controller is determined by the following formula, where the motion state of the drone includes the drone's running direction and the distance of the drone relative to the controller:

[0028] d=vt / (1-(vs / v)×cosθ)

[0029] Wherein, d represents the distance of the UAV relative to the controller, θ represents the angle between the UAV's running direction and the line between the controller and the UAV, vs represents the velocity component of the UAV's velocity relative to the controller in the direction of the line between the controller and the UAV, v represents the propagation speed of sound in the medium, and t represents the time required for the sound to propagate from the UAV to the controller.

[0030] Optionally, the image of the target monitoring object includes frame data at different times when the target monitoring object exists;

[0031] The process of using an optical flow algorithm to identify and process the image of the target monitoring object to obtain the running speed of the drone includes:

[0032] Based on the frame data at different times when the target monitoring object exists, determining the motion vector between the frame data at consecutive different times of the target monitoring object;

[0033] The motion vector is subjected to three-dimensional speed conversion processing to obtain the running speed of the UAV.

[0034] Optionally, the operating state of the drone includes the operating direction of the drone and the distance of the drone relative to the controller;

[0035] Determining the position of the drone according to the running state of the drone and the running speed of the drone includes:

[0036] Obtaining the position of the controller and the flight time of the drone;

[0037] Determine the flight distance of the drone using the flight time of the drone and the running speed of the drone;

[0038] Determining a current flight altitude of the drone based on a flight distance of the drone and a distance of the drone relative to the controller;

[0039] The position of the drone is determined based on the position of the controller and the current flight altitude of the drone.

[0040] Optionally, capturing the drone using the capture device based on the position of the drone includes:

[0041] adjusting a capture direction of the capture device based on the position of the drone;

[0042] The gas tank valve of the capture device is controlled to be in an open state along the capture direction, so that a capture net can be launched from the gas tank valve of the capture device to the position of the UAV, and the UAV can be captured by the capture net.

[0043] Based on the same inventive concept, the second aspect of the present application provides a drone capture device, the device is provided on a controller provided with a capture device, and the device comprises:

[0044] A collection module, configured to collect the running sound of the target monitoring object appearing in the monitoring area and the image of the target monitoring object;

[0045] a determination module configured to determine whether the target monitoring object is a drone based on the operating sound of the target monitoring object and the image of the target monitoring object;

[0046] an analysis and recognition module, configured to, in response to the target monitoring object being a drone, use a Doppler effect algorithm to perform motion state analysis on the running sound of the target monitoring object to obtain the motion state of the drone, and use an optical flow algorithm to perform recognition processing on the image of the target monitoring object to obtain the running speed of the drone;

[0047] The capture module is configured to determine the position of the drone according to the running state of the drone and the running speed of the drone, and capture the drone using the capture device based on the position of the drone.

[0048] Based on the same inventive concept, the third aspect of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the method described in the first aspect above when executing the computer program.

[0049] Based on the same inventive concept, the fourth aspect of the present application provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute the method described in the first aspect above.

[0050] From the above, it can be seen that the drone capture method, device, electronic device and storage medium provided by the present application can more accurately determine whether the target object is a drone by collecting the running sound and image of the drone, using dual information sources to improve the reliability of recognition, and combining sound features and image features. The accuracy of recognition is significantly improved. In addition, the Doppler effect algorithm is used to analyze the motion state of the drone, and the optical flow algorithm is combined to identify its running speed, thereby realizing comprehensive monitoring of the drone's motion state. This multi-dimensional monitoring method not only improves the accuracy of drone position determination, but also provides strong data support for subsequent drone capture, thereby significantly improving the accuracy of drone recognition and the success rate of capture. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0052] Figure 1 This is a flow chart of the drone capture method according to an embodiment of the present application;

[0053] Figure 2 A schematic diagram of the framework of the drone detection and capture process of an embodiment of the present application;

[0054] Figure 3 A schematic diagram of a drone identification and capture component according to an embodiment of the present application;

[0055] Figure 4 A schematic diagram of the drone detection and capture process of an embodiment of the present application;

[0056] Figure 5 This is a structural block diagram of a drone capture device according to an embodiment of the present application;

[0057] Figure 6 A schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0059] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0060] It is understandable that before using the technical solutions of each embodiment of the present application, the type, scope of use, usage scenarios, etc. of the personal information involved will be informed to the user in an appropriate manner, and the user's authorization will be obtained.

[0061] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly remind the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can independently choose whether to provide personal information to the electronic device, application, server, storage medium or other software or hardware that performs the operation of the technical solution of the present application according to the prompt message.

[0062] As an optional but non-limiting implementation, in response to receiving the user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0063] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation method of this application. Other methods that meet relevant laws and regulations may also be applied to the implementation method of this application.

[0064] At present, the main methods of capturing drones are: first, through signal interference, affecting the drone’s communication frequency band or Global Positioning System (GPS) to cause the drone to operate abnormally; second, through physical capture by netting or drone interception; third, through laser strikes.

[0065] Currently, there are radar monitoring, photoelectric monitoring, sound monitoring, signal monitoring, etc. for monitoring drones. However, the above single drone monitoring methods cannot automatically identify drones or automatically update the flight characteristics of drones, making it difficult to meet the requirements of automatically optimizing monitoring information as drone technology develops. In terms of drone capture, it is impossible to meet the requirements of automatic identification, flight status judgment, and terminal capture. Although the net capture method can achieve physical capture, since it is launched through gunpowder points, high requirements are placed on the material of the capture net.

[0066] The drone capture method of the present application can solve the problem of inaccurate identification of drones by a single monitoring method, and improve the accuracy of drone identification through a multimodal method of sound and image;

[0067] It can also solve the problem of automatic identification of drones. Through built-in intelligent computing components, it can analyze and determine the physical and flight sound spectrum characteristics of unknown drones in real time, and update the drone feature library to automatically identify drones.

[0068] In addition, it can also solve the problem of automatic detection, identification and capture of drones. Through the integrated design of drone identification and capture, the success rate of terminal capture of drones can be improved.

[0069] The embodiment of the present application provides a method for capturing a drone, which collects the running sound and image of the drone, uses dual information sources to improve the reliability of recognition, and combines the sound features and image features to more accurately determine whether the target object is a drone, thereby significantly improving the accuracy of recognition. In addition, the Doppler effect algorithm is used to analyze the motion state of the drone, and the optical flow algorithm is combined to identify its running speed, thereby achieving comprehensive monitoring of the drone's motion state. This multi-dimensional monitoring method not only improves the accuracy of drone position determination, but also provides strong data support for subsequent drone capture, thereby significantly improving the accuracy of drone recognition and the success rate of capture.

[0070] like Figure 1As shown, the method of this embodiment is applied to a controller provided with a capture device, and the method includes:

[0071] Step 101 : collecting the running sound of the target monitoring object appearing in the monitoring area and the image of the target monitoring object.

[0072] In this step, a sound collection device (such as a microphone, a sound sensor, etc.) is used to capture or record the sound generated when the target monitoring object is running in the monitoring area.

[0073] The operating sound may include the mechanical operation sound of the target object, the friction sound generated when moving, or its unique working sound.

[0074] By collecting these sounds, the state, activity or behavior of the target object can be analyzed and judged.

[0075] Use image acquisition devices (such as cameras, video cameras, etc.) to capture or record visual images of target monitoring objects within the monitoring area.

[0076] Image acquisition can provide visual information about the appearance, position, motion trajectory, etc. of the target object.

[0077] Compared with sound acquisition, image acquisition can usually provide more intuitive and detailed information about the target object, which helps to conduct more accurate monitoring and analysis.

[0078] It combines the two information sources of sound and image to comprehensively and accurately capture and record the activities of the target monitoring object in the monitoring area.

[0079] Step 102: Determine whether the target monitoring object is a drone based on the operating sound of the target monitoring object and the image of the target monitoring object.

[0080] In this step, for example, an acoustic sensor such as a microphone array is used to collect sound signals in the environment. The microphone array can capture sound signals from different directions, and its form can be a linear four-array, a spherical array, etc.

[0081] The collected sound signals are preprocessed by denoising, enhancing and other operations to improve the quality of the sound signals. This step is crucial for subsequent feature extraction and recognition.

[0082] Parameters that can reflect the acoustic characteristics of the drone are extracted from the preprocessed sound signal, including the sound spectrum, power spectrum, Mel cepstral coefficients, etc. During the flight of the drone, the operation of its motor, rotor vibration, and airflow disturbance will generate a certain degree of noise, which has unique acoustic characteristics, such as frequency range (0.3kHz~20kHz), time domain and frequency domain characteristics, etc.

[0083] The extracted sound features are matched with the pre-established drone sound sample library. The sample library contains sound samples of drones of different types and models in the states of takeoff, flight, hovering, landing, etc. The sound is judged to be the sound of a drone by comparing the similarity of the sound features. If the match reaches the preset threshold, it is judged to be the sound of a drone.

[0084] Each drone has a unique "audio fingerprint", which is the sound of the propeller spinning. The microphone detects suspicious areas in the sky, records the audio noise at several locations, and matches it with the drone audio in the database to identify whether it is a drone sound.

[0085] Furthermore, the image to be detected is input into the object detection algorithm.

[0086] Deep learning models such as Convolutional Neural Networks (CNN) are used to extract features from images. These features can help the model understand the structure and content of the image.

[0087] By generating potential candidate regions (i.e., regions that may contain targets), the scope that needs to be analyzed is narrowed.

[0088] The candidate regions are classified to determine the category of the object, and regression analysis is performed to accurately locate the bounding box of the target.

[0089] The detected target category and the corresponding bounding box are output, usually displayed in a visual form on the original image, which can identify whether it is an image of a drone.

[0090] In drone detection, visible light / infrared detection technology can be used, which uses visible light or thermal infrared reflection of targets to detect drones. It uses a combination of over-the-horizon, high-zoom, high-definition, fog-penetrating visible light camera and infrared thermal imager sensor, integrating advanced diffractive optical elements (DOE) optical infrared thermal image point target tracking and detection technology, high-definition laser scanning surface target image recognition algorithm technology, and ten thousand micro-pulse high-precision servo-driven photoelectric turntable technology, so that it can detect, classify and track small drones flying at low altitude and low speed while monitoring in conventional mode, and conduct all-weather and full-space video detection and monitoring of areas that need to be monitored.

[0091] The judgment results based on the running sound and image are combined to improve the accuracy and reliability of recognition. If the judgment results of both methods indicate that the target monitoring object is a drone, then it can be more confident that the object is a drone.

[0092] This method combines both sound and image information, making full use of the acoustic and visual features of drones, thereby improving the accuracy and reliability of recognition. Using this method to monitor and warn drones in real time can ensure safety.

[0093] Step 103, in response to the target monitoring object being a drone, a Doppler effect algorithm is used to analyze the motion state of the running sound of the target monitoring object to obtain the motion state of the drone, and an optical flow algorithm is used to identify and process the image of the target monitoring object to obtain the running speed of the drone.

[0094] In this step, the Doppler effect is a physical phenomenon where the frequency of the sound heard by the observer (i.e., the controller) changes when the sound source (i.e., the drone) moves relative to the medium. Here, the Doppler effect algorithm is used to analyze the sound generated by the drone when it is flying.

[0095] By analyzing the changes in sound frequency, the movement state of the drone can be inferred, such as whether it is approaching or moving away from the monitoring point, as well as possible changes in flight speed, etc. This method does not require direct observation of the drone itself, but only requires analysis of its sound signals, so it may be more covert and effective in some cases.

[0096] Based on the analysis results of the Doppler effect algorithm, the motion state of the drone can be obtained, including key information such as its flight direction and speed changes.

[0097] Optical flow algorithm is an image processing technique used to analyze the motion information of pixels in an image sequence. It estimates the motion of an object by analyzing the temporal changes in the brightness pattern in the image.

[0098] Here, the optical flow algorithm is used to process the drone images to identify and analyze the drone's movement. By analyzing the changes in the drone's position in the image, its running speed can be calculated.

[0099] Based on the analysis of the image sequence by the optical flow algorithm, the speed of the drone can be calculated. This information is of great significance for monitoring the flight behavior of drones and assessing their potential risks.

[0100] Combining the Doppler effect algorithm and the optical flow algorithm to monitor the motion status and running speed of the drone can obtain the drone's motion information more comprehensively and provide strong support for related applications.

[0101] Step 104: determine the position of the drone according to the operating state of the drone and the operating speed of the drone, and capture the drone using the capture device based on the position of the drone.

[0102] In this step, after obtaining the operating status and speed of the drone, these parameters can be used to infer or determine the current position of the drone. This usually involves certain algorithms and calculations, such as estimating the moving distance and direction of the drone through the relationship between speed and time, and then determining its position.

[0103] Once the drone is located, it can be captured using specialized capture equipment, which may include signal jammers (used to disrupt the drone's control or navigation signals), nets (used to physically capture the drone), laser jammers (used to disrupt the drone's sensors or cameras), etc.

[0104] The specific method of capture depends on many factors, including the type of drone, flight altitude, speed, and performance of the capture equipment. In actual operation, it is necessary to choose the appropriate capture strategy and equipment according to the specific situation.

[0105] The multi-dimensional monitoring method not only improves the accuracy of drone location determination, but also provides strong data support for drone capture, thereby significantly improving the accuracy of drone identification and the success rate of capture.

[0106] Through the above scheme, by collecting the sound and image of the drone, using dual information sources to improve the reliability of recognition, and combining the sound features and image features, it can more accurately determine whether the target object is a drone, significantly improving the accuracy of recognition. In addition, the Doppler effect algorithm is used to analyze the motion state of the drone, and the optical flow algorithm is combined to identify its running speed, thereby realizing comprehensive monitoring of the drone's motion state. This multi-dimensional monitoring method not only improves the accuracy of drone position determination, but also provides strong data support for subsequent drone capture, thereby significantly improving the accuracy of drone recognition and the success rate of capture.

[0107] In some embodiments, step 102 includes:

[0108] Step A1, extracting acoustic feature parameters of the running sound of the target monitoring object to obtain acoustic feature parameters, and extracting features of the image of the target monitoring object to obtain image features.

[0109] Step A2, matching the acoustic feature parameters with a pre-established drone sound sample library, determining the sound similarity between the acoustic feature parameters and the sound samples stored in the drone sound sample library, performing bounding box recognition processing according to the image features, generating a bounding box, matching the bounding box with a pre-established drone bounding box sample library, and determining the bounding box similarity between the bounding box and the bounding box samples stored in the drone bounding box sample library.

[0110] Step A3: In response to the sound similarity being greater than or equal to a preset sound similarity threshold, and the bounding box similarity being greater than or equal to a preset bounding box similarity, determining that the target monitoring object is a drone. Or,

[0111] Step A4: In response to the sound similarity being less than a preset sound similarity threshold and the bounding box similarity being greater than or equal to a preset bounding box similarity, determining that the target monitoring object is not a drone. Or,

[0112] Step A5: In response to the sound similarity being greater than or equal to a preset sound similarity threshold, and the bounding box similarity being less than a preset bounding box similarity, determining that the target monitoring object is not a drone.

[0113] In the above scheme, the running sound of the target monitoring object is collected and acoustic characteristic parameters are extracted from it. These parameters can reflect the specific properties of the sound, such as frequency, amplitude, timbre, etc., which are crucial for identifying the sounds emitted by different objects.

[0114] At the same time, feature extraction is performed on the image of the target monitoring object to obtain image features. These features may include shape, color, texture, etc., which help to identify the appearance of the object.

[0115] Next, the extracted acoustic feature parameters are matched with the pre-established drone sound sample library. This sample library contains the sound features of various drones. By calculating the similarity between the acoustic feature parameters and the sound samples in the sample library, it can be preliminarily determined whether the target monitoring object may be a drone.

[0116] At the same time, the image of the target monitored object is processed for bounding box recognition to generate a rectangular box (bounding box) surrounding the target object. Then, this bounding box is matched with the pre-established drone bounding box sample library to calculate the bounding box similarity. This step further verifies whether the target object is a drone based on image features.

[0117] If the sound similarity is greater than or equal to a preset sound similarity threshold, and the bounding box similarity is also greater than or equal to a preset bounding box similarity, then it is determined that the target monitoring object is a drone.

[0118] If the sound similarity is less than the preset sound similarity threshold, the target monitoring object is determined not to be a drone even if the bounding box similarity meets the conditions. This may be because although the appearance is similar, the sound characteristics do not match.

[0119] Similarly, if the sound similarity meets the conditions but the bounding box similarity is less than the preset value, it is also determined that the target monitoring object is not a drone. This may be because the sound features are similar but the appearance features are not consistent.

[0120] This process combines two information sources, sound and image, to improve the accuracy and reliability of drone recognition. By setting the thresholds for sound similarity and bounding box similarity, the sensitivity and accuracy of the recognition system can be flexibly adjusted to suit different application scenarios and needs.

[0121] In some embodiments, after step A3, the method further comprises:

[0122] Step B1, respectively searching for acoustic feature parameters and image features corresponding to the target monitoring object from a pre-built drone feature library, wherein the drone feature library stores a plurality of acoustic feature parameters and image features.

[0123] Step B2: in response to the absence of acoustic feature parameters and / or image features corresponding to the target monitoring object in the drone feature library, the acoustic feature parameters and / or image features corresponding to the target monitoring object are stored in the drone feature library.

[0124] In the above scheme, the acoustic feature parameters and image features corresponding to the target monitoring object are searched in the pre-built UAV feature library.

[0125] This drone feature library is a database that stores the acoustic feature parameters (such as sound frequency, timbre, etc.) and image features (such as shape, color, texture, etc.) of multiple known drones.

[0126] If no acoustic signature parameters and / or image features that exactly match the target monitoring object are found in the drone signature library, this means that the system has encountered a new or unrecorded drone model or variant.

[0127] In this case, the system will store the acoustic characteristic parameters and / or image features of the target monitoring object into the drone feature library so that such drones can be more accurately identified and classified in the future.

[0128] The purpose of this process is to improve the system's ability to identify and classify drones by continuously learning and updating the drone feature library. As the feature library expands and updates, the system can more accurately identify drones of different models and variants, which is of great significance in areas such as security monitoring and air traffic management.

[0129] In some embodiments, in step 103, the use of the Doppler effect algorithm to analyze the motion state of the running sound of the target monitoring object to obtain the motion state of the drone includes:

[0130] Step C1, performing sound frequency analysis based on the operating sound of the target monitoring object to obtain a sound spectrum.

[0131] Step C2, determining the sound frequency change parameters collected by the controller according to the sound spectrum, and determining the running direction of the drone using the sound frequency change parameters.

[0132] Step C3, obtaining the sound frequency collected by the controller, the sound frequency emitted by the target monitoring object, and the propagation speed of the sound in the medium.

[0133] Step C4, based on the sound frequency collected by the controller, the sound frequency emitted by the target monitoring object and the propagation speed of the sound in the medium, the speed component of the speed of the drone relative to the controller in the direction of the connection between the controller and the drone is determined by the following formula:

[0134] vs = (f′-f) × v / f

[0135] Wherein, vs represents the velocity component of the UAV relative to the controller in the direction of the line connecting the controller and the UAV, f′ represents the sound frequency collected by the controller, f represents the sound frequency emitted by the target monitoring object, and v represents the propagation speed of sound in the medium.

[0136] Step C5, obtaining the time required for the sound to propagate from the drone to the controller and the angle between the running direction of the drone and the line connecting the controller and the drone.

[0137] Step C6, based on the time required for the sound to propagate from the drone to the controller, the angle between the running direction of the drone and the line connecting the controller and the drone, the speed component of the speed of the drone relative to the controller in the direction of the line connecting the controller and the drone, and the propagation speed of the sound in the medium, the distance of the drone relative to the controller is determined by the following formula, wherein the motion state of the drone includes the running direction of the drone and the distance of the drone relative to the controller:

[0138] d=vt / (1-(vs / v)×cosθ)

[0139] Wherein, d represents the distance of the UAV relative to the controller, θ represents the angle between the UAV's running direction and the line between the controller and the UAV, vs represents the velocity component of the UAV's velocity relative to the controller in the direction of the line between the controller and the UAV, v represents the propagation speed of sound in the medium, and t represents the time required for the sound to propagate from the UAV to the controller.

[0140] In the above scheme, the sound of the drone is monitored and the sound frequency is analyzed to obtain the sound spectrum. The sound spectrum is a distribution diagram of different frequency components in the sound, reflecting the strength of each frequency in the sound.

[0141] Based on the sound spectrum, the sound frequency change parameters collected by the controller can be determined. These parameters (such as the frequency change trend, change rate, etc.) can be used to infer the direction of the drone. For example, if the sound frequency gradually increases, it may mean that the drone is approaching the controller; if the frequency gradually decreases, it may mean that the drone is moving away.

[0142] Next, the sound frequency f′ collected by the controller, the sound frequency f emitted by the target monitoring object (UAV), and the propagation speed v of the sound in the medium are obtained.

[0143] Using the principle of the Doppler effect, these parameters can be used to calculate the velocity component of the drone's speed relative to the controller in the direction of the line connecting the controller and the drone vs. The Doppler effect refers to the change in the frequency of the sound heard by the observer when the sound source (or receiver) moves relative to the medium.

[0144] In addition, it is necessary to obtain the time t required for the sound to propagate from the UAV to the controller and the angle θ between the UAV’s running direction and the line connecting the controller and the UAV.

[0145] This information can be estimated or measured by additional sensors or algorithms.

[0146] Finally, based on the sound propagation time, angle, velocity component and sound propagation speed, the distance d of the drone relative to the controller is calculated by the formula.

[0147] This formula combines information on speed, time, and angle, using the Doppler effect and geometric relationships to solve for distance.

[0148] In summary, by monitoring and analyzing the sound frequency of the drone when it is running, combined with the Doppler effect and geometric relationship, the direction of the drone and its distance relative to the controller can be inferred. This method is an effective means of positioning and monitoring the motion status of drones under certain conditions (such as the sound of the drone is loud and clear enough and the ambient noise is low).

[0149] In some embodiments, the image of the target monitoring object includes frame data at different times when the target monitoring object exists.

[0150] In step 103, the image of the target monitoring object is recognized and processed by using an optical flow algorithm to obtain the running speed of the drone, including:

[0151] Step D1, based on the frame data at different moments where the target monitoring object exists, determine the motion vector between the frame data at different consecutive moments of the target monitoring object.

[0152] Step D2, performing three-dimensional speed conversion processing on the motion vector to obtain the running speed of the drone.

[0153] In the above scheme, the image of the target monitoring object refers to a series of image frames containing the target monitoring object, and these frame data are captured at different times.

[0154] Optical flow is an image processing technique used to detect the motion pattern of pixels in an image. In this scenario, it is used to analyze the position changes of the target monitoring object between frames of data at different times. By comparing the position of the target monitoring object in consecutive frames, the optical flow algorithm can calculate the motion vector of the object on the image plane.

[0155] The motion vector refers to the direction and distance that the target monitoring object moves from one point to another between consecutive frames. In a two-dimensional image, this is usually expressed as pixel displacement in the horizontal and vertical directions. Through the optical flow algorithm, these displacements can be calculated to obtain the motion trajectory of the target monitoring object in the image.

[0156] Since images are two-dimensional, but the motion of the drone and the target monitoring object occur in three-dimensional space, it is necessary to convert the two-dimensional motion vectors into three-dimensional velocities. This usually involves an understanding of the relative position, distance, and time changes between the drone and the target monitoring object. By considering these additional dimensions (such as depth or height), as well as possible other sensor data, such as the drone's Global Positioning System (GPS) position, accelerometer data, etc., the drone's operating speed (i.e., the drone's flight speed) can be calculated more accurately.

[0157] Finally, through the above three-dimensional speed conversion process, the running speed of the UAV can be quickly and accurately obtained. This speed may be the absolute speed relative to the ground or the relative speed relative to the target monitoring object, depending on the purpose of the analysis and the data used.

[0158] This process combines optical flow algorithms and 3D spatial analysis to quickly and accurately infer the drone’s operating speed from images of the target monitoring object.

[0159] In some embodiments, the operating state of the drone includes the operating direction of the drone and the distance of the drone relative to the controller.

[0160] In step 104, determining the position of the drone according to the running state of the drone and the running speed of the drone includes:

[0161] Step E1, obtaining the position of the controller and the flight time of the drone.

[0162] Step E2: determining the flight distance of the drone using the flight time of the drone and the running speed of the drone.

[0163] Step E3, determining the current flight altitude of the drone based on the flight distance of the drone and the distance of the drone relative to the controller.

[0164] Step E4, determining the position of the drone according to the position of the controller and the current flight altitude of the drone.

[0165] In the above schemes, the position of the controller is usually a known, fixed point, which can be a geographical coordinate (such as longitude and latitude) or a position relative to a reference point.

[0166] The flight time of a drone is the time from the drone taking off to the current moment.

[0167] By multiplying the flight time (in seconds or hours) by the operating speed (in meters per second or kilometers per hour), the total distance covered by the drone during its flight can be calculated.

[0168] The distance of the drone relative to the controller can be obtained by a variety of methods, such as using radio signal ranging, Global Positioning System (GPS) positioning, etc.

[0169] The flight distance and the distance relative to the controller form the two right-angled sides of a right triangle, in which the flight distance is the hypotenuse (assuming that the flight is carried out on a horizontal plane and the curvature of the earth is not considered).

[0170] By using the Pythagorean theorem (or similar geometric principles), the vertical height of the drone relative to the ground (i.e., flight altitude) can be calculated.

[0171] Now that we know the location of the controller and the altitude of the drone, we can determine the drone's position in three-dimensional space through simple geometric calculations.

[0172] This usually involves using the controller's position as a reference point, then determining the drone's specific position based on its altitude and orientation relative to the controller (which may require additional sensor data like a compass or gyroscope).

[0173] In some embodiments, in step 104, capturing the drone using the capture device based on the position of the drone includes:

[0174] Step F1, adjusting the capture direction of the capture device based on the position of the drone.

[0175] Step F2, controlling the gas tank valve of the capture device to be in an open state along the capture direction, so as to launch a capture net from the gas tank valve of the capture device to the position of the drone, and use the capture net to capture the drone.

[0176] In the above scheme, the capture device is a device capable of launching a net or other capture device to intercept and capture the drone.

[0177] Based on the drone’s location information, the direction of the capture equipment will be adjusted to ensure that the net or other capture device can accurately face the drone’s location.

[0178] The capture device may contain one or more tanks that store compressed gas or propellant to provide the energy needed to launch the capture net.

[0179] In this step, the valve of the gas tank is controlled to open along the previously determined capture direction (i.e., towards the drone).

[0180] When the valve is opened, compressed gas or propellant is rapidly released, pushing the net or other capture device forward at high speed in the capture direction.

[0181] As the tank valve opens and the compressed gas or propellant is released, the net or other capture device is launched from the capture device.

[0182] Nets are usually designed to quickly capture and immobilize drones once deployed, preventing them from escaping or continuing to fly.

[0183] The launching process requires precise control to ensure that the net can accurately hit the drone while minimizing disturbance and damage to the surrounding environment.

[0184] Once the net successfully hits the drone and captures it, the capture process is complete.

[0185] Nets are usually designed to keep a drone stable after it has been captured, preventing it from falling off or continuing to fly.

[0186] In some embodiments, the framework diagram of the drone detection and capture process of the present application is as follows: Figure 2 As shown. It uses multimodal sound and image drone detection and capture, with the goal of identifying, locating, determining the operating status, capturing drones through sound and image, and iteratively training the drone feature recognition model. The device combines sound and image drone recognition technology, operating status calculation technology, high-pressure gas net launch technology, and artificial intelligence training technology to achieve the identification, capture, and feature update of unauthorized drones, ensuring airspace safety.

[0187] The sound spectrum characteristics of drones are mainly Doppler characteristics caused by the movement of the drone's internal parts, such as the rotor and engine. Factors that affect the sound spectrum characteristics of drones include: drone type, drone motion state, and the relative position of the drone and the detection device. Therefore, in the terminal interception process, combining sound and image multimodal methods to identify and track drones can not only effectively locate the target drone, but also obtain more information related to the drone, and use it to update the drone feature library in real time, thereby increasing the success rate of detection and interception.

[0188] In addition, the schematic diagram of the drone identification and capture component (i.e., controller) of the present application is as follows: Figure 3 As shown, the drone identification and capture component: through multimodal analysis of the materials obtained by the sound sensor and video monitor, the object is identified and the monitored object is determined to be a drone. According to the Doppler effect content and image analysis technology, the running speed and running direction of the drone are calculated, the running trend is predicted, and the drone status is identified. For the abnormal drone that has been identified, the steering gear of the high-pressure launch device (i.e., the capture device) is controlled to adjust the direction, the high-pressure gas tank valve is controlled, and the capture net is launched to complete the drone capture operation.

[0189] The schematic diagram of the drone detection and capture process of this application is as follows Figure 4 shown.

[0190] Step 401, object acquisition;

[0191] The air situation is monitored by sound sensors and video monitors to determine whether the target monitoring object is a drone. If so, step 402 is executed to run the calculation. If not, the process returns to step 401 to obtain the object.

[0192] Step 402, running calculation;

[0193] The monitoring information is sent to the intelligent computing component in real time for running calculations.

[0194] Step 403, emission control:

[0195] According to the UAV position and operation trend information output by the operation calculation, an angle adjustment signal is sent to the transmitting gimbal.

[0196] Step 404, launch control:

[0197] The transmitting gimbal is controlled to adjust the angle based on the angle adjustment signal.

[0198] Step 405, high pressure gas tank valve:

[0199] When the UAV reaches the range of the catching net, the valve of the high-pressure gas tank is controlled to launch the catching net toward the UAV at the angle adjusted by the launching gimbal, thereby completing the launching of the catching net.

[0200] Intelligent computing component: Completes the filtering of monitoring materials, drone identification, feature extraction, drone operation status calculation, and update of unknown drone feature library. By filtering the materials obtained by the sound sensor and video monitor, the current object is identified as a drone, and then the sound and image features of the drone are extracted to form a sound spectrum and a graph to complete the extraction of drone features. Through the update function, the drone features are compared with the information in the feature library to determine whether it is existing information, and the data update or addition operation of the feature library is completed.

[0201] (1) Doppler effect speed measurement

[0202] The Doppler effect states that the received frequency of a wave becomes higher when the wave source moves toward the observer, and becomes lower when the wave source moves away from the observer. Therefore, according to the Doppler effect, when a drone is running, the running direction, speed, and distance can be determined by calculation based on the frequency changes of its inherent sound spectrum.

[0203] (2) Image recognition technology

[0204] Extract a single feature to express the target. Common and effective recognition methods mainly include features based on local grayscale differences in images (Haar features), features of image data directly obtained from sensors or files without any processing (Raw features), features used to describe the local gradient direction of images (Hog features), and directional gradient histogram features (Histogram features). They are suitable for the feature expression of different targets in different scenarios. Although the algorithm for extracting a single feature is simple, once the feature is not suitable, it will be limited by this feature and the accuracy of the target feature description will be significantly reduced, especially in real scenes. The environment is often very complex and unpredictable, including target deformation, lighting changes, target occlusion, camera shake and other factors. In order to achieve the complementary advantages and disadvantages of various target feature expressions, improve the algorithm robustness and target tracking stability, the image recognition algorithm will extract multiple features of the target at the same time and fuse them according to a certain proportion of weights when performing feature extraction. Through image recognition, the type, running direction and speed of the drone object are determined.

[0205] (3) Multimodal fusion technology

[0206] Multimodality refers to the collaborative reasoning of multiple heterogeneous modal data. In this application, the operating status of the drone is obtained by calculating the sound spectrum of the drone, and combined with the operating speed and other information calculated based on image recognition, the accurate data of the drone's operating status and operating trend data are improved.

[0207] Drone feature library: The drone feature library has built-in spectrum features and image features of common drones, and updates the drone features updated by sound sensors and video monitors in real time to ensure the real-time nature of the information.

[0208] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0209] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0210] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a drone capture device.

[0211] refer to Figure 5 , the drone capture device, the device is equipped on a controller provided with a capture device, the device comprises:

[0212] The collection module 501 is configured to collect the running sound of the target monitoring object appearing in the monitoring area and the image of the target monitoring object;

[0213] A determination module 502 is configured to determine whether the target monitoring object is a drone based on the operation sound of the target monitoring object and the image of the target monitoring object;

[0214] The analysis and identification module 503 is configured to, in response to the target monitoring object being a drone, use a Doppler effect algorithm to perform motion state analysis on the running sound of the target monitoring object to obtain the motion state of the drone, and use an optical flow algorithm to perform recognition processing on the image of the target monitoring object to obtain the running speed of the drone;

[0215] The capture module 504 is configured to determine the position of the drone according to the operating state of the drone and the operating speed of the drone, and capture the drone using the capture device based on the position of the drone.

[0216] In some embodiments, the determining module 502 is specifically configured to:

[0217] Extracting acoustic feature parameters of the running sound of the target monitoring object to obtain acoustic feature parameters, and extracting features of the image of the target monitoring object to obtain image features;

[0218] Matching the acoustic feature parameters with a pre-established drone sound sample library, determining the sound similarity between the acoustic feature parameters and the sound samples stored in the drone sound sample library, performing bounding box recognition processing according to the image features to generate a bounding box, matching the bounding box with a pre-established drone bounding box sample library, and determining the bounding box similarity between the bounding box and the bounding box samples stored in the drone bounding box sample library;

[0219] In response to the sound similarity being greater than or equal to a preset sound similarity threshold, and the bounding box similarity being greater than or equal to a preset bounding box similarity, determining that the target monitoring object is a drone; or,

[0220] In response to the sound similarity being less than a preset sound similarity threshold, and the bounding box similarity being greater than or equal to a preset bounding box similarity, determining that the target monitoring object is not a drone; or,

[0221] In response to the sound similarity being greater than or equal to a preset sound similarity threshold, and the bounding box similarity being less than a preset bounding box similarity, it is determined that the target monitoring object is not a drone.

[0222] In some embodiments, the drone capture device further includes a feature library storage module. In response to the sound similarity being greater than or equal to a preset sound similarity threshold, and the bounding box similarity being greater than or equal to a preset bounding box similarity, after determining that the target monitoring object is a drone, the feature library storage module is specifically configured as follows:

[0223] Searching for acoustic feature parameters and image features corresponding to the target monitoring object from a pre-built drone feature library, wherein the drone feature library stores a plurality of acoustic feature parameters and image features;

[0224] In response to the absence of acoustic feature parameters and / or image features corresponding to the target monitoring object in the drone feature library, the acoustic feature parameters and / or image features corresponding to the target monitoring object are stored in the drone feature library.

[0225] In some embodiments, the analysis and identification module 503 is specifically configured to:

[0226] Performing sound frequency analysis based on the operating sound of the target monitoring object to obtain a sound spectrum;

[0227] Determine the sound frequency change parameters collected by the controller according to the sound spectrum, and determine the running direction of the drone using the sound frequency change parameters;

[0228] Acquiring the sound frequency collected by the controller, the sound frequency emitted by the target monitoring object, and the propagation speed of the sound in the medium;

[0229] Based on the sound frequency collected by the controller, the sound frequency emitted by the target monitoring object and the propagation speed of the sound in the medium, the velocity component of the speed of the drone relative to the controller in the direction of the line connecting the controller and the drone is determined by the following formula:

[0230] vs = (f′-f) × v / f

[0231] Wherein, vs represents the velocity component of the UAV relative to the controller in the direction of the line connecting the controller and the UAV, f′ represents the sound frequency collected by the controller, f represents the sound frequency emitted by the target monitoring object, and v represents the propagation speed of the sound in the medium;

[0232] Obtain the time required for the sound to propagate from the drone to the controller and the angle between the drone's running direction and the line connecting the controller and the drone;

[0233] Based on the time required for sound to propagate from the drone to the controller, the angle between the drone's running direction and the line connecting the controller and the drone, the speed component of the drone's speed relative to the controller in the direction of the line connecting the controller and the drone, and the propagation speed of sound in the medium, the distance of the drone relative to the controller is determined by the following formula, where the motion state of the drone includes the drone's running direction and the distance of the drone relative to the controller:

[0234] d=vt / (1-(vs / v)×cosθ)

[0235] Wherein, d represents the distance of the UAV relative to the controller, θ represents the angle between the UAV's running direction and the line between the controller and the UAV, vs represents the velocity component of the UAV's velocity relative to the controller in the direction of the line between the controller and the UAV, v represents the propagation speed of sound in the medium, and t represents the time required for the sound to propagate from the UAV to the controller.

[0236] In some embodiments, the image of the target monitoring object includes frame data at different times when the target monitoring object exists;

[0237] The analysis and identification module 503 is specifically configured to:

[0238] Based on the frame data at different times when the target monitoring object exists, determining the motion vector between the frame data at consecutive different times of the target monitoring object;

[0239] The motion vector is subjected to three-dimensional speed conversion processing to obtain the running speed of the UAV.

[0240] In some embodiments, the operating state of the drone includes the operating direction of the drone and the distance of the drone relative to the controller;

[0241] The capture module 504 is specifically configured to:

[0242] Obtaining the position of the controller and the flight time of the drone;

[0243] Determine the flight distance of the drone using the flight time of the drone and the running speed of the drone;

[0244] Determining a current flight altitude of the drone based on a flight distance of the drone and a distance of the drone relative to the controller;

[0245] The position of the drone is determined based on the position of the controller and the current flight altitude of the drone.

[0246] In some embodiments, the capture module 504 is specifically configured to:

[0247] adjusting a capture direction of the capture device based on the position of the drone;

[0248] The gas tank valve of the capture device is controlled to be in an open state along the capture direction, so that a capture net can be launched from the gas tank valve of the capture device to the position of the UAV, and the UAV can be captured by the capture net.

[0249] For the convenience of description, the above device is described in terms of functions divided into various modules. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0250] The device of the above embodiment is used to implement the corresponding drone capture method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0251] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the drone capture method described in any of the above embodiments is implemented.

[0252] Figure 6 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 601, a memory 602, an input / output interface 603, a communication interface 604, and a bus 605. The processor 601, the memory 602, the input / output interface 603, and the communication interface 604 are connected to each other in communication within the device through the bus 605.

[0253] The processor 601 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0254] The memory 602 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 602 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 602 and called and executed by the processor 601.

[0255] The input / output interface 603 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device can include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device can include a display, a speaker, a vibrator, an indicator light, etc.

[0256] The communication interface 604 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0257] The bus 605 includes a path for transmitting information between various components of the device (eg, the processor 601 , the memory 602 , the input / output interface 603 , and the communication interface 604 ).

[0258] It should be noted that, although the above device only shows the processor 601, the memory 602, the input / output interface 603, the communication interface 604 and the bus 605, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0259] The electronic device of the above embodiment is used to implement the corresponding drone capture method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0260] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the drone capture method described in any of the above embodiments.

[0261] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0262] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the drone capture method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0263] A person skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0264] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0265] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0266] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the present application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the protection scope of the present application.

Claims

1. A drone capture method, characterized in that: Applied to a controller provided with a capture device, the method comprises: Collecting the running sound of the target monitoring object appearing in the monitoring area and the image of the target monitoring object; Determining whether the target monitoring object is a drone based on the operating sound of the target monitoring object and the image of the target monitoring object; In response to the target monitoring object being a drone, a Doppler effect algorithm is used to analyze the motion state of the running sound of the target monitoring object to obtain the motion state of the drone, and an optical flow algorithm is used to identify and process the image of the target monitoring object to obtain the running speed of the drone; The position of the drone is determined according to the running state of the drone and the running speed of the drone, and the drone is captured by using the capture device based on the position of the drone.

2. The method according to claim 1, characterized in that The determining whether the target monitoring object is a drone based on the running sound of the target monitoring object and the image of the target monitoring object includes: Extracting acoustic feature parameters of the running sound of the target monitoring object to obtain acoustic feature parameters, and extracting features of the image of the target monitoring object to obtain image features; Matching the acoustic feature parameters with a pre-established drone sound sample library, determining the sound similarity between the acoustic feature parameters and the sound samples stored in the drone sound sample library, performing bounding box recognition processing according to the image features to generate a bounding box, matching the bounding box with a pre-established drone bounding box sample library, and determining the bounding box similarity between the bounding box and the bounding box samples stored in the drone bounding box sample library; In response to the sound similarity being greater than or equal to a preset sound similarity threshold, and the bounding box similarity being greater than or equal to a preset bounding box similarity, determining that the target monitoring object is a drone; or, In response to the sound similarity being less than a preset sound similarity threshold, and the bounding box similarity being greater than or equal to a preset bounding box similarity, determining that the target monitoring object is not a drone; or, In response to the sound similarity being greater than or equal to a preset sound similarity threshold, and the bounding box similarity being less than a preset bounding box similarity, it is determined that the target monitoring object is not a drone.

3. The method according to claim 2, characterized in that In response to the sound similarity being greater than or equal to a preset sound similarity threshold, and the bounding box similarity being greater than or equal to a preset bounding box similarity, after determining that the target monitoring object is a drone, the method further includes: Searching for acoustic feature parameters and image features corresponding to the target monitoring object from a pre-built drone feature library, wherein the drone feature library stores a plurality of acoustic feature parameters and image features; In response to the absence of acoustic feature parameters and / or image features corresponding to the target monitoring object in the drone feature library, the acoustic feature parameters and / or image features corresponding to the target monitoring object are stored in the drone feature library.

4. The method according to claim 1, characterized in that: The method of using the Doppler effect algorithm to analyze the motion state of the running sound of the target monitoring object to obtain the motion state of the drone includes: Performing sound frequency analysis based on the operating sound of the target monitoring object to obtain a sound spectrum; Determine the sound frequency change parameters collected by the controller according to the sound spectrum, and determine the running direction of the drone using the sound frequency change parameters; Acquiring the sound frequency collected by the controller, the sound frequency emitted by the target monitoring object, and the propagation speed of the sound in the medium; Based on the sound frequency collected by the controller, the sound frequency emitted by the target monitoring object and the propagation speed of the sound in the medium, the velocity component of the speed of the drone relative to the controller in the direction of the line connecting the controller and the drone is determined by the following formula: vs = (f′-f) × v / f Wherein, vs represents the velocity component of the UAV relative to the controller in the direction of the line connecting the controller and the UAV, f′ represents the sound frequency collected by the controller, f represents the sound frequency emitted by the target monitoring object, and v represents the propagation speed of the sound in the medium; Obtain the time required for the sound to propagate from the drone to the controller and the angle between the drone's running direction and the line connecting the controller and the drone; Based on the time required for sound to propagate from the drone to the controller, the angle between the drone's running direction and the line connecting the controller and the drone, the speed component of the drone's speed relative to the controller in the direction of the line connecting the controller and the drone, and the propagation speed of sound in the medium, the distance of the drone relative to the controller is determined by the following formula, where the motion state of the drone includes the drone's running direction and the distance of the drone relative to the controller: d=vt / (1-(vs / v)×cosθ) Wherein, d represents the distance of the UAV relative to the controller, θ represents the angle between the UAV's running direction and the line between the controller and the UAV, vs represents the velocity component of the UAV's velocity relative to the controller in the direction of the line between the controller and the UAV, v represents the propagation speed of sound in the medium, and t represents the time required for the sound to propagate from the UAV to the controller.

5. The method according to claim 1, characterized in that The image of the target monitoring object includes frame data at different times when the target monitoring object exists; The process of using an optical flow algorithm to identify and process the image of the target monitoring object to obtain the running speed of the drone includes: Based on the frame data at different times when the target monitoring object exists, determining the motion vector between the frame data at consecutive different times of the target monitoring object; The motion vector is subjected to three-dimensional speed conversion processing to obtain the running speed of the UAV.

6. The method according to claim 1, characterized in that The operating state of the drone includes the operating direction of the drone and the distance of the drone relative to the controller; Determining the position of the drone according to the running state of the drone and the running speed of the drone includes: Obtaining the position of the controller and the flight time of the drone; Determine the flight distance of the drone using the flight time of the drone and the running speed of the drone; Determining a current flight altitude of the drone based on a flight distance of the drone and a distance of the drone relative to the controller; The position of the drone is determined based on the position of the controller and the current flight altitude of the drone.

7. The method according to claim 1, characterized in that The capturing of the drone by using the capturing device based on the position of the drone includes: adjusting a capture direction of the capture device based on the position of the drone; The gas tank valve of the capture device is controlled to be in an open state along the capture direction, so that a capture net can be launched from the gas tank valve of the capture device to the position of the UAV, and the UAV can be captured by the capture net.

8. A drone capture device, characterized in that: The device is provided on a controller provided with a capture device, and the device comprises: A collection module, configured to collect the running sound of the target monitoring object appearing in the monitoring area and the image of the target monitoring object; a determination module configured to determine whether the target monitoring object is a drone based on the operating sound of the target monitoring object and the image of the target monitoring object; an analysis and recognition module, configured to, in response to the target monitoring object being a drone, use a Doppler effect algorithm to perform motion state analysis on the running sound of the target monitoring object to obtain the motion state of the drone, and use an optical flow algorithm to perform recognition processing on the image of the target monitoring object to obtain the running speed of the drone; The capture module is configured to determine the position of the drone according to the running state of the drone and the running speed of the drone, and capture the drone using the capture device based on the position of the drone.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 7.

Images