An event camera based moving target imaging detection method and system

Abstract

The application belongs to the technical field of photoelectric imaging detection, and discloses a motion target imaging detection method and system based on an event camera, which comprises the following steps: receiving original event stream data, performing event accumulation on the original event stream data, obtaining a cumulative image and the number of events of a pixel point, discriminating a motion target on the cumulative image according to the distribution of the number of events, when it is determined that the motion target is multiple, predicting the motion speed difference between the motion targets in a scene according to the number of events of the pixel points corresponding to the motion targets, and starting high-frequency pulsed laser imaging detection when the motion speed difference is greater than or equal to a first preset threshold. The application discriminates the motion target according to the number and distribution of event accumulation and predicts the speed difference of the target, uses the high-frequency characteristics of laser to generate dense event stream, enhances the target features, solves the problem of time window selection, and realizes the imaging detection of dynamic targets with large motion speed difference.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of photoelectric imaging detection, and more particularly to a moving target imaging detection method and system based on an event camera. BACKGROUND

[0002] In recent years, dynamic target detection technology, such as aerial moving target detection technology, has attracted widespread attention, and the monitoring of unmanned aerial vehicles is a typical application. Traditional image sensors use integral circuits for imaging, have a low dynamic range, and are easily affected by light to cause overexposure. For the monitoring of unmanned aerial vehicles, the scene is subject to strong light interference, such as direct sunlight, at a specific time, which results in an undesirable imaging effect. The background of a static platform monitoring scene is usually static, and the target is highly mobile. An image sensor images all components in the scene, resulting in a large amount of redundant data and interference from a complex background, making it difficult to detect unmanned aerial vehicle targets.

[0003] An event camera is a dynamic vision sensor that generates event data based on a differential circuit and has a high dynamic range and high temporal resolution. However, target detection algorithms based on event stream data usually require the selection of a time accumulation parameter to construct an event frame. When the speed difference between highly mobile targets in the scene is too large, it is difficult to select a time window. A small time window results in the lack of features of low-speed targets, and a large time window results in the smearing of high-speed moving targets and a decrease in temporal resolution. On the other hand, when there is strong light interference in the environment or the reflectivity of the target is low, it is difficult to trigger events.

[0004] In the prior art, an event camera and a traditional image sensor are combined for detection, event data and image data are fused, and detection of different scale unmanned aerial vehicle targets in different environments is achieved. However, traditional image sensors face problems such as image overexposure and motion blur in strong light interference and high-speed motion scenes, and fusion with event data cannot achieve complementary effects, which may even result in a decrease in detection effect. Moreover, the problem of time window selection caused by a large speed difference between targets in the scene is not solved. Another solution provides a detection algorithm suitable for various light environments, calculates feature map matching degrees by combining event camera, color camera, and infrared camera data, and improves the robustness to false targets. However, multiple cameras are used for joint detection, which has high equipment complexity. Therefore, there is an urgent need for an event camera imaging detection method for high-mobility dynamic targets in strong light interference. SUMMARY

[0005] In view of the above defects or improvement needs of the prior art, the application provides an event camera-based moving target imaging detection method and system, which is used to solve the problem that the prior event camera imaging detection technology has difficulty in selecting a time window when the target speed difference is large, thereby making it difficult to image and detect the target.

[0006] To achieve the above object, according to one aspect of the application, an event camera-based moving target imaging detection method is provided, comprising:

[0007] S1, receiving scene original event stream data acquired by an event camera, accumulating events of the original event stream data, acquiring a cumulative image and the number of events of any pixel point on the cumulative image;

[0008] S2, discriminating moving targets on the cumulative image according to the number of events;

[0009] S3, when it is determined that there are multiple moving targets, predicting the moving speed difference between the moving targets in the scene according to the number of events of the pixel points corresponding to the moving targets;

[0010] S4, when the moving speed difference between the moving targets is greater than or equal to a first preset threshold, starting high-frequency pulsed laser imaging detection; the high-frequency pulsed laser imaging detection is specifically as follows:

[0011] Emitting high-frequency pulsed laser to produce light intensity change in the scene, using the high-frequency pulsed laser to trigger event generation to construct a high-time-resolution event frame, and performing scene imaging detection based on the high-time-resolution event frame.

[0012] According to the event camera-based moving target imaging detection method provided by the application, S1 specifically comprises: selecting multiple time windows of different sizes to accumulate events of the original event stream data, and acquiring multiple cumulative images;

[0013] Correspondingly, in S2, the moving targets on any of the cumulative images are discriminated; and in S3, for any of the cumulative images, when it is determined that there are multiple moving targets, the moving speed difference between the moving targets in the scene is predicted according to the number of events of the pixel points corresponding to the moving targets.

[0014] In S4, for any of the cumulative images, when the moving speed difference between the moving targets is greater than or equal to a first preset threshold, high-frequency pulsed laser imaging detection is started.

[0015] According to the event camera-based moving target imaging detection method provided by the application, the calculation formula of the number of events of any pixel point on the cumulative image is:

[0016] ;

[0017] ;

[0018] in, for Total number of events within the time period; The first accumulated event One time window; Indicates the first The polarity of an event; for Within the time period The total number of positive and negative events at a pixel; ) indicates the first The position coordinates in each event.

[0019] According to the moving target imaging detection method based on an event camera provided by the present invention, S2 specifically involves: obtaining multiple event clusters by clustering algorithm based on the event quantity distribution of each pixel on the accumulated image, wherein each event cluster represents a moving target, thereby realizing the discrimination of the moving target.

[0020] According to the moving target imaging and detection method based on an event camera provided by the present invention, S2 specifically includes:

[0021] Based on the number of events at any pixel in the accumulated image, a set of activation points is established by taking pixels with an event count greater than 0 as activation points.

[0022] For any activation point, traverse all pixels within the preset neighborhood window of the activation point. When the number of events of all pixels is greater than or equal to the corresponding preset threshold, and the total number of events of all pixels is greater than or equal to the total preset threshold, the activation point is determined to be a core point, and the set of activation points is traversed to establish a core point set.

[0023] When the number of events for other pixels within the preset neighborhood window of a core point is greater than 0, the core point is considered to be directly density-reachable to the other pixels. A chain is established by traversing the core point set, starting from any core point. Each chain consists of multiple pixels arranged sequentially, where any two adjacent pixels are directly density-reachable, and the core point at the start of each chain is density-reachable to the other pixels. When the starting points of two chains are the same core point, the other pixels in the two chains are considered density-connected. When two core points are density-reachable from each other, the two core points are considered density-connected.

[0024] The set of the largest density-connected pixels is formed into an event cluster. Any two pixels in an event cluster are density-connected, thus performing clustering of event clusters. Each event cluster corresponds to a moving target.

[0025] According to the event camera-based moving target imaging detection method provided in the application, the motion speed difference between the moving targets in the scene is predicted according to the event number of the pixel points corresponding to the moving targets in S3, and specifically includes:

[0026] The density of the moving targets is obtained according to the event number of the pixel points corresponding to the moving targets, and the motion speed difference between the moving targets is predicted through the density comparison of the moving targets.

[0027] The density of the moving targets is the ratio between the total event number of the pixel points corresponding to the moving targets and the spatial scale of the moving targets, and the spatial scale is the maximum envelope rectangular area of the pixel points corresponding to the moving targets.

[0028] According to the event camera-based moving target imaging detection method provided in the application, the calculation formula of the motion speed difference between the moving targets in S3 is as follows:

[0029] ;

[0030] Wherein, is the motion speed difference between the moving targets m and the moving targets n . is the density corresponding to the moving targets m . is the density corresponding to the moving targets n .

[0031] According to the event camera-based moving target imaging detection method provided in the application, the method further includes:

[0032] The total event number of the cumulative image is obtained according to the event number of any pixel point on the cumulative image.

[0033] When the total event number of the cumulative image is greater than or equal to the second preset threshold and less than or equal to the third preset threshold, it is determined that the ambient light intensity causes interference to the imaging detection, and at this time, the high-frequency pulsed laser imaging detection is also started.

[0034] According to the event camera-based moving target imaging detection method provided in the application, the high-frequency pulsed laser imaging detection further includes:

[0035] A narrow-band optical filter corresponding to the spectral segment of the high-frequency pulsed laser is arranged in front of the event camera to filter the ambient light.

[0036] And, the event stream data triggered by the high-frequency pulsed laser is received, and the event frame is accumulated and constructed along the time axis and is subjected to frame extraction processing.

[0037] According to the event frame after the frame extraction processing, the motion speed of the moving target is acquired through scene imaging detection, and the frame extraction rate in the frame extraction processing is adjusted according to the motion speed feedback, so that the greater the motion speed is, the greater the extraction rate is.

[0038] According to another aspect of the present application, an event camera-based moving target imaging detection system is provided, comprising a pulsed laser active illumination module, a specific spectral band laser receiving module and a data post-processing module, the pulsed laser active illumination module is built by using a signal generator, a laser emitter and a beam expander, the specific spectral band laser receiving module is built by using a narrow-band optical filter and an event camera, and the data post-processing module is built by using a controller.

[0039] The controller is connected with the pulsed laser active illumination module and the specific spectral band laser receiving module respectively, and is used for realizing the event camera-based moving target imaging detection method according to any one of the above.

[0040] Overall, compared with the prior art, the event camera-based moving target imaging detection method and system provided by the present application has the following advantages:

[0041] 1. The number and distribution of events are used to distinguish moving targets and predict the speed difference of moving targets in a scene. When the speed difference between moving targets is large, high-frequency light is used to trigger the event camera by changing the light intensity, so that the high-frequency characteristics of laser are used to generate dense event flow, which can enhance the target features, solve the problem of time window selection, and realize high-time-resolution event frame imaging detection of dynamic targets with large speed difference;

[0042] 2. A plurality of time windows of different sizes are selected for event accumulation, and the motion target is distinguished and the speed difference is judged through a plurality of event images, which can better adapt to the scene of a plurality of moving targets with different speeds and improve the accuracy of judgment;

[0043] 3. When clustering and distinguishing moving targets, the judgment of the core point not only needs to consider the total number of surrounding events (greater than or equal to the total threshold), but also needs to meet certain requirements for the distribution of events (each position in the neighborhood is greater than or equal to the corresponding threshold), which can make the selection of core points more accurate and have stronger anti-noise interference ability;

[0044] 4. By filtering the remaining spectral components with a narrow-band filter, combined with the high dynamic characteristics of the event camera, stable event output can be achieved, and imaging detection with anti-ultrastrong light interference can be realized; for different target environments, the laser spectral interval can be selected to improve the robustness of the system to scene changes; the application can solve the problem of event frame time accumulation parameter selection when the motion speed of dynamic targets such as high-mobility unmanned aerial vehicles is greatly different, and can also realize target imaging detection under ultrastrong light interference; at the same time, according to the detection result, the frame extraction parameter is fed back and adjusted, the redundant event frames are removed, and the calculation pressure of the model is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 is a flow chart of the motion target imaging detection method based on the event camera provided by the application.

[0046] Figure 2 is a complete schematic diagram of the motion target imaging detection method based on the event camera provided by the application.

[0047] Figure 3 is a schematic diagram of the clustering algorithm for motion target discrimination provided by the application.

[0048] Figure 4 is a schematic diagram of the corresponding preset number threshold of each pixel point in the preset neighborhood window provided by the application.

[0049] Figure 5 is a schematic diagram of the motion target imaging detection system based on the event camera provided by the application.

[0050] In all the drawings, the same reference signs are used to represent the same elements or structures, wherein:

[0051] 10-pulse laser active illumination module; 100-signal generator; 101-laser emitter; 102-beam expander; 20-specific spectral band laser receiving module; 200-narrow-band filter; 201-tripod; 202-event camera; 30-data post-processing module; 300-power supply module; 301-external data storage; 302-industrial computer. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical scheme and advantages of the application clearer, the application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the application and do not limit the application. In addition, the technical features involved in each embodiment of the application described below can be combined with each other as long as there is no conflict.

[0053] Please refer to Figure 1The embodiment provides a motion target imaging detection method based on an event camera, and the motion target imaging detection method comprises the following steps:

[0054] S1, receiving scene original event stream data acquired by an event camera, performing event accumulation on the original event stream data, acquiring a cumulative image and an event number of any pixel point on the cumulative image;

[0055] S2, discriminating a motion target on the cumulative image according to an event number distribution;

[0056] S3, when the motion target is determined to be multiple, predicting a motion speed difference between the motion targets in a scene according to the event numbers of the pixel points corresponding to the motion targets;

[0057] S4, when the motion speed difference between the motion targets is greater than or equal to a first preset threshold, starting high-frequency pulsed laser imaging detection; the high-frequency pulsed laser imaging detection is specifically as follows:

[0058] Emitting high-frequency pulsed laser to generate light intensity change in the scene, using the high-frequency pulsed laser to trigger event generation to construct a high-time-resolution event frame, and performing scene imaging detection based on the high-time-resolution event frame.

[0059] Further, S1 specifically comprises: selecting multiple time windows with different sizes to perform event accumulation on the original event stream data, and acquiring multiple cumulative images;

[0060] Correspondingly, in S2, the motion target on any cumulative image is discriminated; and in S3, for any cumulative image, when the motion target is determined to be multiple, the motion speed difference between the motion targets in the scene is predicted according to the event numbers of the pixel points corresponding to the motion targets.

[0061] In S4, for any cumulative image, when the motion speed difference between the motion targets is greater than or equal to the first preset threshold, the high-frequency pulsed laser imaging detection is started.

[0062] Further, S2 is specifically as follows: according to the event number distribution of each pixel point on the cumulative image, a plurality of event clusters are obtained through a clustering algorithm, each event cluster represents a motion target, and then the discrimination of the motion target is realized;

[0063] Correspondingly, S2 specifically comprises:

[0064] According to the event number of any pixel point on the cumulative image, a pixel point with an event number greater than 0 is regarded as an active point to establish an active point set;

[0065] For any activation point, all pixel points in the preset neighborhood window of the any activation point are traversed, when the event number of all pixel points is greater than or equal to the corresponding preset number threshold, and the sum of the event number of all pixel points is greater than or equal to the sum of the preset number threshold, it is determined that the any activation point is a core point, and a core point set is established by traversing the activation point set;

[0066] When the event number of other pixel points in the preset neighborhood window of the core point is greater than 0, it is considered that the core point is directly density reachable to the other pixel points, based on this, a chain is established by taking any core point in the core point set as a starting point, each chain is composed of a plurality of pixel points arranged in sequence, wherein any two adjacent pixel points are directly density reachable, and the core point at the starting point of each chain is density reachable to other pixel points; when the starting points of two chains are the same core point, it is considered that the other pixel points of the two chains are density connected; when two core points are mutually density reachable, it is considered that the two core points are density connected;

[0067] The maximum pixel point set satisfying density connection is constructed into an event cluster, any two pixel points in the event cluster are density connected, so that the clustering and division of the event cluster are performed, and each event cluster corresponds to a moving target.

[0068] In some specific embodiments, with reference to Figure 2 , S1 is specifically as follows: receiving original event stream data , wherein (x i, y i) represents the position coordinates in the i th event; p i represents the polarity of the i th event; k t i represents the timestamp of the i th event; the optimal time window parameter corresponding to different moving speed targets in the scene is different, so that different size time accumulation windows are selected k , satisfying , respectively, to obtain event images by event accumulation. k n The height and width of the two-dimensional accumulation image obtained by event accumulation are , the event number of each position on the pixel plane, that is, the accumulation image plane, is calculated for different time parameters, and the event number of any pixel point on the accumulation image is obtained, which is specifically as follows:

[0069]

[0070] ;

[0071] ;

[0072] , wherein is the total number of events in the time period; ​​​​​a time window for accumulating events; representing the polarity of the k total number of positive and negative events at a pixel point within a

[0073] S2 discriminates the moving target as follows: set the preset neighborhood window size as , set the minimum event number threshold, i.e. the preset number threshold, as , wherein , the minimum event total number threshold in the preset neighborhood window, i.e. the preset number threshold total, is:

[0074] .

[0075] For the of any pixel point in the accumulated image, if the condition:

[0076] is met, it is recorded as an active point, and the active point set For

[0077] , if the condition: is met, it is recorded as a core point, and the core point set

[0078] , wherein , and ; then

[0079] is the core point, which is recorded as ; wherein , and , when , let , when , let , and when , let . .

[0080] Neighborhood pixels of the core point:

[0081] ,

[0082] If: , and ;

[0083] and:

[0084] ;

[0085] and / or: ​​​​​

[0086] ;

[0087] then is a boundary point, denoted as , the set D of the rest of the elements in which are not core points and not boundary points are noise points, denoted as

[0088] Further, the neighborhood pixels of a core point , for , represent the pixel coordinates, if:

[0089] ;

[0090] then to are directly density connected.

[0091] When there is a chain:

[0092] ;

[0093] where , there are to directly density connected, then to are density connected. The starting point of the directly density connected is a core point, and the ending point is a core point or a boundary point. When the ending point is a core point, the directly density connected point of the ending core point can be established to extend the chain. When the ending point is a boundary point, the chain ends. Directly density connected is density connected, but density connected is not necessarily directly density connected.

[0094] Take a core point as the starting point to establish a chain. If to are density connected, and to are density connected, or and are both core points and are density connected to each other, then and are density connected. Similarly, represent the pixel coordinates.

[0095] Referring to Figure 3 , the largest set satisfying density connection constitutes a cluster. Any two points in the cluster are density connected, and the total number of clusters on the pixel plane, i.e., the cumulative image, is Each cluster represents a moving object, and the moving object set corresponding to the first time window is as follows :

[0096] .

[0097] In this embodiment, when determining the core point, it is required to meet that the total sum of the number of events of all pixel points in the preset neighborhood window is greater than or equal to the preset number threshold sum, and it is also required to meet that each pixel point in the preset neighborhood window is greater than or equal to the corresponding preset number threshold. This setting is equivalent to that when judging the core point, not only the total number of surrounding events (greater than or equal to the total threshold) needs to be considered, but also the distribution of events needs to meet certain requirements (each position in the neighborhood needs to be greater than or equal to the corresponding threshold), which can make the selection of the core point more accurate and the anti-noise interference ability stronger.

[0098] For example, referring to Figure 3 , due to the influence of the device itself and complex light, many interference events will be generated, such as Figure 3 , if only a total threshold is used, it is easy to misjudge it as a core point, such as Figure 3 , when two clusters are connected relatively close, this misjudgment may connect the two clusters, as shown in Figure 3 , by setting a threshold for each position in the neighborhood, the distribution of events is constrained, and more accurate category division is obtained.

[0099] For the preset number threshold corresponding to each pixel point in the preset neighborhood window, a neighborhood template can be set, that is, the threshold setting of each pixel point position is performed on the template of the preset neighborhood window size, that is, each pixel point position on this neighborhood template has a preset number threshold, and then when actually judging the pixel type, the template is used for comparison, for example, whether the pixel x , y is a core point or a boundary point, whether the number of events of each pixel position in the neighborhood with x , y as the center and the size of h × w exceeds the threshold of the corresponding position in the template is judged; at the same time, whether the total sum of the number of all events in the neighborhood is greater than or equal to the total threshold is also judged.

[0100] ​The setting of the preset number threshold of the pixel points in the preset neighborhood window can be based on the distribution of the target event stream, affected by different illuminations, noise interference and event density, and different target types (shapes) and motion conditions result in different event distributions, which can make the distribution of the threshold in the preset neighborhood window consistent with the distribution of the target event stream. The specific threshold is set according to experience (the specific threshold can be adjusted through multiple clustering experiments). Reference Figure 4 The setting of the preset number threshold of the pixel points in the preset neighborhood window can also gradually decrease from inside to outside, that is, the closer to the middle of the preset neighborhood window, the larger the preset number threshold, which can remove the interference events to a certain extent and realize the suppression of the edge interference.

[0101] Further, S3 comprises predicting the motion speed difference between the motion targets in the scene according to the event number of the pixel points corresponding to the motion targets, specifically comprising:

[0102] According to the event number of the pixel points corresponding to the motion targets, the density of the motion targets is obtained, and the motion speed difference between the motion targets is predicted by comparing the densities of the motion targets.

[0103] The density of the motion targets is the ratio between the total number of events of the pixel points corresponding to the motion targets and the spatial scale of the motion targets, and the spatial scale is the maximum envelope rectangular area of the pixel points corresponding to the motion targets.

[0104] S3 specifically comprises the following steps:

[0105] calculating the number of pixel points in each cluster of the cumulative image corresponding to the i-th time window to form a pixel point number set :

[0106] ;

[0107] The pixel value, i.e. the event number, of the midpoint of the i-th cluster is recorded as , and the total event number corresponding to the i-th cluster is:

[0108] ;

[0109] wherein represents the j-th pixel point in the i-th cluster; represents the number of pixel points in the i-th cluster; j

[0110] ​​​​​​​The sum of the number of events in all clusters constitutes a set:

[0111] .

[0112] The maximum and minimum horizontal and vertical coordinates of the pixel points in each cluster are calculated as follows:

[0113] ;

[0114] ;

[0115] ;

[0116] ;

[0117] The spatial scale of the mth cluster is:

[0118] ; Similarly, the set of spatial scales of all clusters is:

[0119]

[0120] ; The density of the mth cluster is:

[0121] ;

[0122] Similarly, the set of densities of all clusters is:

[0123] .

[0124] For any two clusters m and n in the time period T, the difference between the two clusters, i.e., the difference in the motion speed between the moving targets in S3, is specifically as follows:

[0125] ; wherein,

[0126] is the difference in the motion speed between the moving target m and the moving target n; is the density corresponding to the moving target m; is the density corresponding to the moving target n.

[0127] m The sum of the number of events of all pixel points in the time period T is: n m n

[0128]

[0129] ​​​​​​​​ ;

[0130] If , then the high-frequency pulsed laser is turned on for imaging detection. The high-frequency characteristics can be used to select a smaller time accumulation parameter to construct a high time resolution event frame. Active light changes are used to enhance target features. is the first preset threshold value, which can be set according to experience values; is the second preset threshold value; is the third preset threshold value. The embodiment takes into account that event triggering can be difficult when there is strong light interference, so that the total number of events is relatively small, which is less than the threshold value The threshold value

[0131] is related to different target motion conditions, and is generally not more than the total number of event triggers without strong light interference. In addition, a small number of noise events can be caused by device or scene illumination, so the total number of events needs to be greater than or equal to the threshold value . The threshold value can also be set according to experience, and is generally not less than the number of noise events.

[0132] The motion target imaging detection method based on the event camera further includes:

[0133] According to the number of events of any pixel point on the accumulated image, the total number of events of the accumulated image is obtained.

[0134] When the total number of events of the accumulated image is greater than or equal to the second preset threshold value and less than or equal to the third preset threshold value, it is determined that the environmental light intensity causes interference to the imaging detection, and at this time the high-frequency pulsed laser imaging detection is also turned on.

[0135] Further, the high-frequency pulsed laser imaging detection further includes:

[0136] A narrowband optical filter corresponding to the spectral band of the high-frequency pulsed laser is arranged in front of the event camera to filter the environmental light. The high dynamic characteristics of the event camera are used to achieve interference with the environmental strong light to a certain extent. However, when there is strong light interference in the scene, the event triggering threshold value is greatly improved and the event triggering stability is reduced due to the logarithmic relationship between the output voltage of the event camera and the input light. The present application uses the active pulsed laser and the narrowband optical filter to cooperate with the event camera to generate stable event stream data.

[0137] In addition, event stream data triggered by changes in the reflection of the high-frequency pulsed laser are received, and event frames are constructed along the time axis and are subjected to frame extraction processing. The frame extraction parameter and​ Perform frame extraction ( The specific extraction rates are as follows:

[0138] ;

[0139] That is, each Before extracting from frame data Frame data is inspected, and the remaining data is discarded.

[0140] The motion velocity of moving targets is obtained by scene imaging detection based on the event frames after frame extraction. The frame extraction rate is adjusted according to the motion velocity feedback, so that the extraction rate increases with the motion velocity. Specifically, the target position can be calculated and the target motion velocity predicted using target detection methods such as convolutional neural networks, and the frame extraction parameters are adjusted according to the target velocity feedback. F 1 and F 2. Reduce the amount of data and decrease the processing time.

[0141] Furthermore, this embodiment also provides a moving target imaging and detection system based on an event camera, including a pulsed laser active illumination module, a specific spectral band laser receiving module, and a data post-processing module. The pulsed laser active illumination module is constructed using a signal generator, a laser emitter, and a beam expander; the specific spectral band laser receiving module is constructed using a narrowband filter and an event camera; and the data post-processing module is constructed using a controller. The controller is connected to the pulsed laser active illumination module and the specific spectral band laser receiving module respectively, and is used to implement the moving target imaging and detection method based on an event camera described above.

[0142] In some specific embodiments, a highly maneuverable unmanned aerial vehicle (UAV) imaging detection system and method based on an event camera is provided, including the following steps:

[0143] The system is built in three parts: a pulsed laser active illumination module using a signal generator, laser emitter, and beam expander; a laser receiving module for a specific spectral band using a narrowband filter, tripod, and event camera; and a data post-processing module using a power supply module and external data storage.

[0144] After system initialization, it receives raw event stream data, selects time windows of different sizes to accumulate events, and obtains multiple event clusters through clustering, with each event cluster representing a moving target.

[0145] The density of each target is calculated for different time windows. By comparing the density of different targets within the same time period, the differences in the movement speed of targets in the scene are predicted.

[0146] For the scene with single motion speed, the event stream is directly projected along the time axis to construct an event frame, which is input into the target detection module to calculate the target position.

[0147] For the scene with large motion speed difference between targets, high-frequency pulsed laser is emitted to generate light intensity change, high-time-resolution event frame is constructed, target position and motion speed are obtained through target detection algorithm, and the frame extraction parameter is adjusted according to the target motion speed to reduce the redundancy of event data.

[0148] The embodiment provides an event camera-based high-maneuverable unmanned aerial vehicle imaging detection system and method, the core idea of which is to cluster targets in different time windows, compare the density of the targets, predict the motion speed difference between the targets in the scene, use the light intensity change of high-frequency pulsed laser to enhance the target features and construct high-time-resolution event frame to suppress motion tailing for the case that the event frame accumulated in a single time window is missing target features, motion tailing and insufficient time resolution due to too large speed difference; for the scene with strong light interference, pulsed light illumination of a specific spectrum and narrow-band optical filter are used in combination with the high dynamic characteristics of the event camera to realize stable and high-quality pulsed event data triggering.

[0149] Further, the system is divided into three parts, a pulsed laser active illumination module 10 is built by using a signal generator 100, a laser emitter 101 and a beam expander 102, a specific spectrum laser receiving module 20 is built by using a narrow-band optical filter 200, a tripod 201 and an event camera 202, a controller is connected to an industrial computer 302, a data post-processing module 30 is built by using a power supply module 300 in combination with an external data storage 301.

[0150] The overall schematic diagram is shown in Figure 5 The pulsed laser active illumination module 10 is built by using the signal generator 100, the laser emitter 101 and the beam expander 102 to generate an active pulsed illumination signal of a specific spectrum, and the specific spectrum is as follows:

[0151] The pulsed signal generator 100 is adjusted to generate a high-frequency pulsed modulation signal of a specific frequency, the higher the pulsed frequency, the higher the density of the event response triggered by the reflected light intensity, thereby improving the maximum time resolution of the event frame construction; at the same time, the active pulsed laser of the specific frequency can increase the range of the target reflected light intensity change and improve the anti-interference ability of the system to the environment with strong light, the signal generator 100 can generate a pulsed modulation signal with a frequency adjustable from 1 to 15000 Hz, and the specific frequency can be 10000-15000 Hz.

[0152] The laser emitter 101 receives the modulated signal output by the signal generator 100. Different spectral ranges of laser can be selected according to the absorption rate of different target materials and the atmospheric attenuation degree of laser propagation in different scenes, and the laser intensity can be adjusted to improve the robustness of the system in different environments.

[0153] Because different target materials have different absorption rates for specific spectral range laser, and different atmospheric environments have different attenuation rates for specific spectral range, a spectral range with higher response, better penetration and higher reflectivity should be selected according to the target and atmospheric environment to be detected. For example, the laser emitter 101 with a spectral range of (532±0.5) nm to (532±10) nm receives the pulse modulated signal output by the signal generator 100, adjusts the laser intensity, and generates laser with corresponding frequency and spectral range. The pulse frequency is the same as the frequency of the modulated signal output by the signal generator.

[0154] The beam expander 102 is adjusted to change the diameter of the laser beam, reduce the laser divergence angle, and change the coverage. The pulsed laser active illumination is used to generate light intensity changes, and the event camera realizes dense event response to the target in the scene.

[0155] For the scene with strong light interference, a narrow-band optical filter 200 corresponding to the spectral range of the pulsed laser signal output by the laser emitter 101 is selected to filter out the remaining spectral components and increase the proportion of active pulsed laser in the reflected light intensity, thereby improving the signal-to-noise ratio. Combined with the high dynamic characteristics of the event camera 202, the suppression of environmental strong light interference is realized, as follows:

[0156] The narrow-band optical filter 200 is placed in front of the lens of the event camera 202 to build a specific frequency laser receiving module. By adjusting the lens and aperture, the target imaging effect of the system can be improved. The spectral range of the narrow-band optical filter 200 is (532±10) nm, which corresponds to the spectral range of the laser generated by the pulsed laser active illumination module 10, so that other spectral components in the ambient light are filtered out.

[0157] Those skilled in the art will readily understand that the above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An event camera based moving target imaging detection method, characterized in that, The method comprises the following steps: S1, receiving scene raw event stream data acquired by an event camera, accumulating events of the raw event stream data, acquiring a cumulative image and the number of events of any pixel point on the cumulative image; S2, discriminating a moving target on the cumulative image according to the distribution of the number of events; S3, when it is determined that there are multiple moving targets, predicting the speed difference between the moving targets in the scene according to the number of events of the pixel points corresponding to the moving targets; S4, when the speed difference between the moving targets is greater than or equal to a first preset threshold, starting high-frequency pulsed laser imaging detection; the high-frequency pulsed laser imaging detection specifically comprises the following steps: emitting high-frequency pulsed laser to the scene to generate light intensity change, triggering event generation by the high-frequency pulsed laser to construct a high-time-resolution event frame, and performing scene imaging detection based on the high-time-resolution event frame; S1 specifically comprises the following steps: selecting multiple time windows of different sizes to accumulate events of the raw event stream data and acquire multiple cumulative images; correspondingly, in S2, the moving target on any of the cumulative images is discriminated; in S3, for any of the cumulative images, when it is determined that there are multiple moving targets, the speed difference between the moving targets in the scene is predicted according to the number of events of the pixel points corresponding to the moving targets; in S4, for any of the cumulative images, when the speed difference between the moving targets is greater than or equal to a first preset threshold, high-frequency pulsed laser imaging detection is started; in S3, the speed difference between the moving targets in the scene is predicted according to the number of events of the pixel points corresponding to the moving targets, specifically comprising the following steps: according to the number of events of the pixel points corresponding to the moving targets, the density of the moving targets is acquired, and the speed difference between the moving targets is predicted by comparing the densities of the moving targets; wherein the density of the moving targets is the ratio between the total number of events of the pixel points corresponding to the moving targets and the spatial scale of the moving targets, and the spatial scale is the area of the maximum envelope rectangle of the pixel points corresponding to the moving targets; the high-frequency pulsed laser imaging detection further comprises the following steps: a narrow-band optical filter corresponding to the spectral segment of the high-frequency pulsed laser is arranged in front of the event camera to filter ambient light.

2. The event camera based moving object imaging detection method of claim 1, wherein, The calculation formula of the number of events of any pixel point on the cumulative image is as follows: ； ； in, for Total number of events within the time period; The first accumulated event One time window; Indicates the first The polarity of an event; for Within the time period The total number of positive and negative events at a pixel; ) indicates the first The position coordinates in each event.

3. The event camera based moving object imaging detection method according to claim 1 or 2, characterized in that, S2 specifically comprises the following steps: according to the distribution of the number of events of each pixel point on the cumulative image, a plurality of event clusters are obtained by a clustering algorithm, each event cluster represents a moving target, and the discrimination of the moving target is realized.

4. The event camera based moving object imaging detection method of claim 3, wherein, S2 specifically comprises the following steps: according to the number of events of any pixel point on the cumulative image, a pixel point with an event number greater than 0 is taken as an active point to establish an active point set; for any active point, all pixel points in a preset neighborhood window of the active point are traversed, when the number of events of all the pixel points is greater than or equal to a corresponding preset number threshold and the total number of events of all the pixel points is greater than or equal to a preset number threshold sum, it is determined that the active point is a core point, and a core point set is established by traversing the active point set; When the number of events for other pixels within the preset neighborhood window of a core point is greater than 0, the core point is considered to be directly density-reachable to the other pixels. A chain is established by traversing the core point set, starting from any core point. Each chain consists of multiple pixels arranged sequentially, where any two adjacent pixels are directly density-reachable, and the core point at the start of each chain is density-reachable to the other pixels. When the starting points of two chains are the same core point, the other pixels in the two chains are considered density-connected. When two core points are density-reachable from each other, the two core points are considered density-connected. The set of the largest density-connected pixels is formed into an event cluster. Any two pixels in an event cluster are density-connected, thus performing clustering of event clusters. Each event cluster corresponds to a moving target.

5. The event camera based moving object imaging detection method of claim 1, wherein, The difference in motion speed between the moving targets described in S3 is obtained using the following formula: ； wherein, is a moving object m and a moving object n a difference in moving speed between the two objects; is a moving object m a corresponding density; is a moving object n a corresponding density.

6. The event camera based moving object imaging detection method of claim 1 or 2, wherein, Also includes: Based on the number of events at any pixel in the accumulated image, obtain the total number of events in the accumulated image; When the total number of events in the accumulated image is greater than or equal to the second preset threshold and less than or equal to the third preset threshold, it is determined that the ambient light intensity interferes with the imaging detection, and high-frequency pulsed laser imaging detection is also activated at this time.

7. The event camera based moving object imaging detection method of claim 1 or 2, wherein, The high-frequency pulsed laser imaging detection also includes: Receive event stream data triggered by high-frequency pulsed laser, accumulate and construct event frames along the time axis, and perform frame extraction processing; The motion speed of moving targets is obtained by scene imaging detection based on the event frames after frame extraction, and the frame extraction rate in the frame extraction process is adjusted according to the motion speed feedback so that the extraction rate is greater when the motion speed is greater.

8. An event camera based moving object imaging detection system, characterized in that, It includes a pulsed laser active illumination module, a specific spectral band laser receiving module, and a data post-processing module. The pulsed laser active illumination module is built using a signal generator, a laser transmitter, and a beam expander. The specific spectral band laser receiving module is built using a narrowband filter and an event camera. The data post-processing module is built using a controller. The controller is connected to the pulsed laser active illumination module and the specific spectral band laser receiving module respectively, and is used to implement the moving target imaging and detection method based on the event camera as described in any one of claims 1-7.

Images