A method and device for intelligent target detection of unmanned aerial vehicles using a laser-assisted event camera

Abstract

This application discloses a method and apparatus for intelligent detection of drone targets using a laser-assisted event camera, relating to the field of drone target detection. The method includes emitting laser light from a laser emitter array in a dark environment to illuminate the drone propellers, and using an event camera to collect drone event sequences at different distances; detecting the current state of the drone based on the drone event sequences; the current state of the drone includes hovering and moving states; and determining the drone target detection result based on the current state of the drone. This application improves the detection effect of high-altitude drones in dark environments.

Description

Technical Field

[0001] This application relates to the field of unmanned aerial vehicle (UAV) target detection, and in particular to a method and apparatus for intelligent UAV target detection using a laser-assisted event camera. Background Technology

[0002] With the widespread application of drone technology, it is playing an increasingly important role in civilian, commercial, and military fields. However, in some special scenarios, such as dark environments like military-sensitive areas at night or around important urban infrastructure, effective monitoring of drones faces many challenges.

[0003] Traditional detection methods, such as visible light-based cameras, cannot acquire clear images in dark environments, making accurate detection of drones difficult. While thermal imaging cameras can overcome the light problem to some extent, their detection performance is still unsatisfactory for drones without obvious thermal radiation characteristics or thermal camouflage. Furthermore, if attackers block all light sources emitted by the drone, including both visible and invisible light devices, detecting high-altitude drones in dark environments becomes even more challenging. Therefore, developing an efficient drone target detection method suitable for dark environments is of significant practical importance. Summary of the Invention

[0004] The purpose of this application is to provide a method and device for intelligent detection of drone targets using a laser-assisted event camera, in order to solve the problem of poor detection performance of high-altitude drones in dark environments.

[0005] To achieve the above objectives, this application provides the following solution:

[0006] In a first aspect, this application provides a method for intelligent detection of unmanned aerial vehicle targets using a laser-assisted event camera, comprising:

[0007] In a dark environment, a laser emitter array is used to emit lasers to illuminate the drone propellers, and an event camera is used to collect drone event sequences at different distances;

[0008] Based on the drone event sequence, detect the current state of the drone; the current state of the drone includes hovering state and moving state.

[0009] Based on the current state of the drone, determine the drone target detection result.

[0010] Secondly, this application provides a laser-assisted event camera-based intelligent target detection device for unmanned aerial vehicles, comprising:

[0011] The event sequence acquisition module is used to emit lasers from a laser emitter array to illuminate the drone propellers in dark environments, and to acquire drone event sequences at different distances using an event camera.

[0012] The drone current state detection module is used to detect the current state of the drone based on the drone event sequence; the current state of the drone includes hovering state and moving state;

[0013] The UAV target detection result module is used to determine the UAV target detection result based on the current state of the UAV.

[0014] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the laser-assisted event camera intelligent target detection method for unmanned aerial vehicles as described in any of the above-mentioned methods.

[0015] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the laser-assisted event camera intelligent target detection method for unmanned aerial vehicles as described above.

[0016] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the intelligent detection method for unmanned aerial vehicle targets using a laser-assisted event camera as described above.

[0017] According to the specific embodiments provided in this application, the following technical effects are disclosed:

[0018] This application utilizes a laser emitter array and an event camera to jointly detect high-altitude drones in dark environments. The laser emitter array provides an active light source, emitting lasers to illuminate the drone even in darkness. Based on the event camera's sensitivity to moving objects, it can accurately capture the drone event sequence generated by the interaction between the laser and the drone. The current state of the drone is then detected based on this event sequence, ultimately outputting the drone target detection result. This significantly improves the accuracy and reliability of drone target detection, solves the problem of difficult high-altitude drone detection in dark environments, and enhances the detection effect of high-altitude drones in dark environments. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1This is an application environment diagram of a laser-assisted event camera-based intelligent target detection method for unmanned aerial vehicles according to one embodiment of this application;

[0021] Figure 2 A flowchart illustrating a laser-assisted event camera-based intelligent target detection method for unmanned aerial vehicles (UAVs) according to an embodiment of this application;

[0022] Figure 3 A flowchart illustrating a laser-assisted event camera-based intelligent target detection method for unmanned aerial vehicles (UAVs) according to another embodiment of this application;

[0023] Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0026] This application provides a laser-assisted event camera-based intelligent target detection method for unmanned aerial vehicles (UAVs). This method is executed by a computer device, such as... Figure 1 As shown, it can be executed by a computer device such as a terminal or server alone, or by both a terminal and a server. In the embodiments of this application, as... Figure 2 As shown, the method includes the following steps.

[0027] S1: In a dark environment, a laser emitter array is used to emit lasers to illuminate the drone propellers, and an event camera is used to collect drone event sequences at different distances.

[0028] S2: Detect the current state of the drone based on the drone event sequence; the current state of the drone includes hovering state and moving state.

[0029] S3: Determine the target detection result of the UAV based on the current state of the UAV.

[0030] In an exemplary embodiment, the event camera and laser emitter work together to detect high-altitude drones in extremely dark environments. Before step S1, the following steps are also included:

[0031] S4: Multiple laser emitters are evenly distributed to form a laser emitter array; the laser emitter array is arranged in a regular matrix to cover the shooting area of ​​the event camera.

[0032] S5: Position the event camera at the center of the laser emitter array so that the shooting angle of the event camera covers the laser emission area of ​​the laser emitter array.

[0033] In practical applications, the specific steps for sensor deployment are as follows:

[0034] 1) Select a suitable laser emitter, such as a green laser emitter with a wavelength of 532nm, which has high brightness and strong penetration. Determine the number and arrangement of laser emitters based on the size and shape of the detection area. For example, in a circular detection area with a radius of 100 meters, nine laser emitters can be arranged in a 3×3 matrix, with adjacent laser emitters spaced 50 meters apart.

[0035] 2) Select a high-sensitivity, high-dynamic-range event camera, such as the Prophesee Metavision EVK4HD, with a pixel size of 4.86um × 4.86um and a resolution of 1280 × 720. Mount the event camera 2 meters above the center of the laser emitter array, and adjust its shooting angle so that its field of view completely covers the laser emission area, and the overlap rate with the laser emitter's field of view reaches more than 80%.

[0036] In an exemplary embodiment, S1 can be replaced by the following steps.

[0037] S11: A high-intensity laser beam is emitted into the sky using the laser emitter array. When a drone enters the laser irradiation range, it illuminates the drone's propellers. The high-intensity laser beam has a wavelength of 532nm and a power between 1-12W.

[0038] S12: Based on the blocking and reflection of laser by the drone propeller, use an event camera to collect drone event sequences generated by the drone propeller cutting the laser at different distances.

[0039] In practical applications, the laser emitter array is turned on to emit a high-intensity laser beam into the sky. When a drone enters the laser irradiation range, the laser shines on the drone's propellers. The rotation of the propellers will block or reflect the laser.

[0040] The event camera outputs event sequence data with microsecond-level temporal resolution, recording the event sequence generated by the drone propeller cutting the laser.

[0041] For drones at different distances, the focal length and shooting parameters of the event camera are adjusted to ensure that the corresponding event sequence can be clearly captured. The distance range for different distances can be set from 50 meters to 500 meters, with each 50-meter interval serving as a sampling distance node, i.e., {50, 100, 150, 200…450, 500}.

[0042] Adjusting the event camera's focal length and shooting parameters specifically includes keeping the internal parameters (contrast threshold, filtering parameters, etc.) at their default settings and not making any adjustments. Therefore, the only adjustable parameters for the event camera are the lens's focal length and aperture. Since the scene is at night, we keep the aperture at its maximum and adjust the lens's focal length to focus on the drone.

[0043] The relationship between focal length f and distance d is as follows: d represents the object distance, which is the distance from the drone to the camera, in meters (m).

[0044] The specific steps for data collection are as follows:

[0045] 1) In extremely dark environments, turn on the laser emitter array, set the emission power of each laser emitter to 2W, and ensure that the laser beam can effectively illuminate the target area. The event camera continuously collects event stream data.

[0046] 2) Adjust the focal length of the event camera for drones at different distances. For example, when the detection distance is 50 meters, adjust the focal length to 10mm; when the detection distance is 100 meters, adjust the focal length to 20mm, and so on, to ensure that a clear event sequence is captured.

[0047] In an exemplary embodiment, S2 can be replaced by the following steps.

[0048] S21: Based on the drone event sequence, divide the shooting area of ​​the event camera into multiple sub-regions, and use the spectrum analysis method to perform spectrum analysis on each sub-region to determine whether the drone in the current sub-region is hovering. If yes, proceed to S22; if no, proceed to S23.

[0049] S22: Output the hovering status of the drone in the current sub-region.

[0050] S23: Use a laser emitter array to emit laser light to illuminate a moving drone, and use optical flow method to detect the moving drone and determine its movement status.

[0051] In practical applications, when a drone hovers, the rotating propellers cut the laser, creating periodic changes in light intensity. The frequency of these changes is directly related to the propeller speed and the number of blades.

[0052] Taking a common quadcopter drone as an example (each motor has 2 propellers, with a rotation speed of 2000-6000 rpm), the optical modulation frequency f' is calculated as follows: f' = rotation speed (RPM) / 60 × number of propellers, corresponding to a frequency range of approximately 80-200Hz.

[0053] Perform FFT on the event sequence of each sub-region, extract the frequency domain signal, and search for the peak in the 80Hz-200Hz frequency band. The peak frequency is required to match the typical modulation frequency of the UAV (allowing ±5% error to accommodate rotation speed fluctuations).

[0054] In an exemplary embodiment, S21 can be replaced by the following steps.

[0055] S211: Based on the UAV event sequence, the shooting area of ​​the event camera is divided into multiple equal sub-regions according to a regular grid.

[0056] S212: The Fast Fourier Transform method is used to perform spectral analysis on the UAV event sequence in each sub-region, converting the time-series signal into a frequency domain signal.

[0057] S213: When the spectrum signal has a peak value in the set frequency range and the peak value amplitude exceeds the set amplitude threshold, it is determined that there is a drone hovering in the sub-region; the set frequency range is the frequency range generated by laser reflection caused by the rotation of the drone propeller.

[0058] In practical applications, 1) the image area captured by the event camera is divided into N equal-sized sub-regions according to a regular grid, and the event sequence within each sub-region is subjected to spectral analysis. Spectral analysis algorithms such as Fast Fourier Transform (FFT) are used to convert the time-domain event sequence into a frequency-domain signal, i.e., the sub-region spectrum.

[0059] 2) If a specific frequency feature appears in the spectrum of a certain sub-region, and the frequency feature matches the laser reflection frequency feature generated by the propeller rotation when the UAV is hovering, it can be determined that there is a UAV hovering in that region.

[0060] For example, when a drone hovers, the frequency of the laser reflection caused by the rotating propeller cutting the laser is in the range of 80Hz-200Hz. When a significant peak appears in this frequency range in the sub-region spectrum, it can be determined that the drone is hovering.

[0061] In practical applications, the specific steps for detecting hovering drones are as follows:

[0062] 1) Divide the image area captured by the event camera into 100 equal-sized sub-regions (10×10). Use the NumPy and SciPy libraries in Python to perform a Fast Fourier Transform on the event sequence within each sub-region.

[0063] 2) Set a frequency matching range, such as the frequency of laser reflection caused by the propeller rotation when the drone hovers within the range of 80Hz-200Hz. When the frequency domain signal of a certain sub-region shows a peak in this frequency range, and the peak amplitude exceeds the set threshold (e.g., the peak amplitude is more than twice the average amplitude), it is determined that there is a drone hovering in that sub-region.

[0064] In an exemplary embodiment, S23 can be replaced by the following steps.

[0065] S231: Analyze multiple consecutive images in a UAV event sequence, use optical flow to detect the motion of pixels in the image, and determine the motion speed.

[0066] S232: When the detected target’s speed exceeds the set speed threshold and the trajectory matches the flight characteristics of the UAV, the detected target is determined to be a moving UAV, and the movement status is determined; the flight characteristics include smooth curves, no sudden changes, etc.

[0067] In practical applications, when a laser beam illuminates a moving drone, it creates the effect of the drone moving on an event camera. By detecting whether there is a rapidly moving object, it is possible to detect whether a drone is moving at high speed.

[0068] In practical applications, the specific steps for detecting mobile drones are as follows:

[0069] 1) Use the optical flow algorithm from the OpenCV library to process multiple consecutive frames of images captured by the event camera. Calculate the motion vector for each pixel in the image, and determine the object's speed and trajectory by analyzing the magnitude and direction of the motion vector.

[0070] 2) Set the drone flight speed threshold to 3m / s. When the detected object’s speed is greater than this threshold and the trajectory is a smooth curve without obvious abrupt changes, it is determined that a moving drone has been detected.

[0071] Through the above specific implementation methods, it is possible to accurately and efficiently detect UAV targets in extremely dark environments.

[0072] In practical applications, such as Figure 3 As shown, this application can also simultaneously use spectral analysis and optical flow methods to detect hovering and moving UAVs, respectively, to achieve hovering detection and movement detection and tracking functions. The specific steps are as follows:

[0073] Step 1: Design a sensor cooperation method between an event camera and a laser emitter, in which the event camera and laser emitter jointly detect high-altitude drones in extremely dark environments.

[0074] Step 2: In extremely dark environments, use a laser emitter to emit a laser to illuminate the drone propellers, and use an event camera to collect drone event sequences at different distances.

[0075] Step 3: Process and analyze the collected event sequences to determine if any drones are hovering or moving. The specific steps are as follows:

[0076] Step 3.1: Divide the area captured by the event camera into N sub-regions, and then perform spectrum analysis on the N sub-regions to detect whether a drone is hovering in a certain area.

[0077] Step 3.2: For a moving drone, the laser beam illuminating the drone on the event camera will create the effect of event movement. The presence of a drone moving at high speed can be detected by detecting whether there is a rapidly moving object.

[0078] This application designs targeted detection algorithms for drones in different states, which improves the comprehensiveness and effectiveness of detection and can adapt to complex extremely dark environment scenarios.

[0079] Based on the same inventive concept, this application also provides a laser-assisted event camera drone target intelligent detection device for implementing the laser-assisted event camera drone target intelligent detection method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more laser-assisted event camera drone target intelligent detection device embodiments provided below can be found in the limitations of the laser-assisted event camera drone target intelligent detection method described above, and will not be repeated here.

[0080] In one exemplary embodiment, a laser-assisted event camera-based intelligent target detection device for unmanned aerial vehicles includes:

[0081] The event sequence acquisition module is used to emit lasers from a laser emitter array to illuminate the drone propellers in dark environments, and to acquire drone event sequences at different distances using an event camera.

[0082] The drone current state detection module is used to detect the current state of the drone based on the drone event sequence; the current state of the drone includes hovering state and moving state.

[0083] The UAV target detection result module is used to determine the UAV target detection result based on the current state of the UAV.

[0084] In one exemplary embodiment, a computer device is provided, such as Figure 4 As shown, the computer device can be a server or a terminal. The computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores high-altitude UAV target detection data in dark environments. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a laser-assisted event camera-based intelligent UAV target detection method.

[0085] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described above.

[0086] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the methods described above.

[0087] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the methods described above.

[0088] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0089] In this application, all actions to acquire signals, information, or data are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with the authorization granted by the owner of the relevant device.

[0090] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0091] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0092] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for intelligent target detection in unmanned aerial vehicles using a laser-assisted event camera, characterized in that, include: In a dark environment, a laser emitter array is used to emit laser light to illuminate the drone's propellers, and an event camera is used to capture drone event sequences at different distances, specifically including: The laser emitter array is used to emit a high-intensity laser beam into the sky. When a drone enters the laser irradiation range, the drone's propeller is irradiated. Based on the blocking and reflection of laser by the drone propeller, an event camera was used to collect drone event sequences caused by the drone propeller cutting the laser at different distances. Based on the drone event sequence, the current state of the drone is detected, specifically including: Based on the drone event sequence, the shooting area of ​​the event camera is divided into multiple sub-regions, and the spectrum analysis method is used to perform spectrum analysis on each sub-region to determine whether the drone in the current sub-region is hovering. If so, output the hovering status of the drone in the current sub-region; If not, a laser emitter array is used to emit laser light to illuminate the moving drone, and optical flow is used to detect the moving drone and determine its movement status; the current state of the drone includes hovering state and moving state. Based on the current state of the drone, determine the drone target detection result.

2. The intelligent target detection method for unmanned aerial vehicles using a laser-assisted event camera according to claim 1, characterized in that, In a dark environment, a laser emitter array is used to emit laser light to illuminate the drone's propellers, and an event camera is used to capture drone event sequences at different distances. This also includes: Multiple laser emitters are evenly distributed to form a laser emitter array; the laser emitter array is arranged in a regular matrix to cover the shooting area of ​​the event camera; The event camera is positioned at the center of the laser emitter array so that the shooting angle of the event camera covers the laser emission area of ​​the laser emitter array.

3. The intelligent target detection method for unmanned aerial vehicles using a laser-assisted event camera according to claim 1, characterized in that, Based on the drone event sequence, the shooting area of ​​the event camera is divided into multiple sub-regions, and a spectrum analysis method is used to perform spectrum analysis on each sub-region to determine whether the drone in the current sub-region is hovering. Specifically, this includes: Based on the drone event sequence, the shooting area of ​​the event camera is divided into multiple equal sub-regions according to a regular grid. The Fast Fourier Transform (FFT) method is used to perform spectral analysis on the UAV event sequences in each sub-region, converting the time-series signals into frequency-domain signals. When the frequency domain signal shows a peak value within a set frequency range and the peak value amplitude exceeds a set amplitude threshold, it is determined that a drone is hovering in that sub-region; the set frequency range is the frequency range generated by laser reflection caused by the rotation of the drone propeller.

4. The intelligent target detection method for unmanned aerial vehicles using a laser-assisted event camera according to claim 1, characterized in that, A laser emitter array is used to emit laser light to illuminate a moving drone, and optical flow is employed to detect the drone and determine its movement status. Specifically, this includes: Analyze multiple consecutive images in a UAV event sequence, use optical flow to detect the motion of pixels in the images, and determine the motion speed; When the detected target's speed exceeds a set speed threshold and its trajectory matches the characteristics of a drone's flight path, the detected target is determined to be a moving drone, and its movement status is determined.

5. A laser-assisted event camera-based intelligent target detection device for unmanned aerial vehicles, characterized in that, The laser-assisted event camera's intelligent drone target detection device performs the intelligent drone target detection method of any one of claims 1-4, wherein the laser-assisted event camera's intelligent drone target detection device comprises: The event sequence acquisition module is used to emit lasers from a laser emitter array to illuminate the drone propellers in dark environments, and to acquire drone event sequences at different distances using an event camera. The drone current state detection module is used to detect the current state of the drone based on the drone event sequence; the current state of the drone includes hovering state and moving state; The UAV target detection result module is used to determine the UAV target detection result based on the current state of the UAV.

6. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the intelligent target detection method for a laser-assisted event camera according to any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the intelligent detection method for unmanned aerial vehicle targets using a laser-assisted event camera as described in any one of claims 1-4.

8. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the intelligent detection method for unmanned aerial vehicle targets using a laser-assisted event camera as described in any one of claims 1-4.

Images