A multi-target tracking method, device and computer equipment based on event camera

Abstract

The present application proposes a multi-target tracking method, device and computer equipment based on an event camera, the method includes obtaining the number of trigger events in each sampling period of multiple consecutive sampling periods in the measured scene; if the number of trigger events in multiple consecutive sampling periods exceeds the preset environmental noise threshold, the trigger events in the current latest sampling period are processed by the DBSCAN clustering algorithm to determine the number of moving objects in the measured scene and the initial position triggered by each of the moving objects; according to the data stream of the trigger event, an event image frame containing all moving objects in the measured scene is determined, and the moving objects are tracked according to the event image frame. The present application quickly and accurately tracks and locates multiple moving objects in the measured scene, while ensuring low latency and extremely low computational complexity, reducing power consumption and being applied to mobile platforms.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a multi-target tracking method, device and computer equipment based on an event camera. Background Art

[0002] There are two main methods for detecting and tracking target objects: (1) by adopting traditional image processing methods, image preprocessing is required, such as image smoothing, feature extraction, etc., and then moving object detection is performed by using traditional image processing algorithms such as frame difference method, or color matching, boundary recognition and other methods are performed to detect objects of specific color or shape; (2) by using neural networks, deep learning and other technologies, specific target objects are detected and tracked on traditional images. However, in order to ensure the accuracy of detecting and tracking target objects, the above two methods need to process the entire image, but such processing will generate a lot of calculations and require powerful hardware support; in addition, for high-speed moving objects, due to the imaging principle of traditional cameras, the final image obtained will produce dynamic blur, which will cause certain difficulties in detection and tracking, and the delay is high, and fast response cannot be guaranteed in high-speed scenes. Summary of the invention

[0003] The present application aims to solve one of the technical problems in the related art at least to some extent.

[0004] To this end, the first purpose of this application is to propose a multi-target tracking method based on an event camera, which solves the various drawbacks of the existing methods, quickly and accurately tracks and locates multiple moving objects in the measured scene, while ensuring low latency and extremely low computational complexity, reducing power consumption and being applicable to mobile platforms.

[0005] The second objective of the present application is to propose a multi-target tracking device based on an event camera.

[0006] The third objective of the present application is to provide a computer device.

[0007] To achieve the above-mentioned purpose, a multi-target tracking method based on an event camera is proposed in a first aspect of the present application, comprising:

[0008] Obtain the number of trigger events in each sampling period of multiple consecutive sampling periods in the measured scene;

[0009] If the number of trigger events in multiple consecutive sampling periods exceeds the preset environmental noise threshold, the trigger events in the current latest sampling period are processed by the DBSCAN clustering algorithm to determine the number of moving objects in the measured scene and the initial position triggered by each moving object;

[0010] An event image frame including all moving objects in the detected scene is determined according to the data stream of the triggering event, and the moving objects are tracked according to the event image frame.

[0011] The multi-target tracking method based on an event camera in the embodiment of the present application obtains the number of trigger events in each sampling period of multiple consecutive sampling periods in the measured scene; if the number of trigger events in multiple consecutive sampling periods exceeds the preset environmental noise threshold, the trigger events in the current latest sampling period are processed by the DBSCAN clustering algorithm to determine the number of moving objects in the measured scene and the initial position triggered by each of the moving objects; the event image frame containing all the moving objects in the measured scene is determined according to the data stream of the trigger event, and the moving objects are tracked according to the event image frame. The present application quickly and accurately tracks and locates multiple moving objects in the measured scene, while ensuring low latency and extremely low computational complexity, reducing power consumption and being applicable to mobile platforms.

[0012] Optionally, in an embodiment of the present application, before obtaining the number of triggering events in each sampling period of multiple sampling periods in the measured scene, the method further includes:

[0013] Collecting the number of triggering events in each sampling period of a plurality of consecutive sampling periods in the measured scene, wherein the measured scene is a static scene;

[0014] The variance of the number of triggering events in a plurality of consecutive sampling periods is calculated, and if the variance is less than a preset threshold, the preset environmental noise threshold is determined according to the mean value of the number of triggering events in a plurality of consecutive sampling periods.

[0015] Optionally, in one embodiment of the present application, before the triggering events in the multiple sampling periods are processed by the DBSCAN clustering algorithm, the method includes:

[0016] Determine a radius of a neighboring area that matches the trigger event and a preset number of trigger events within the radius of the neighboring area.

[0017] Optionally, in one embodiment of the present application, the triggering events within the multiple sampling periods are processed by the DBSCAN clustering algorithm to determine the number of moving objects in the measured scene and the initial position triggered by each of the moving objects, specifically including:

[0018] Determine the actual number of trigger events contained in the radius of the neighboring area by any trigger event among the trigger events through the DBSCAN clustering algorithm, wherein if the actual number is not less than the preset number, determine the category of any trigger event as a core point;

[0019] If the actual number is less than the preset number, but the trigger event is within the radius of the neighboring area of ​​any core point, then the category of any trigger event is determined to be an edge point;

[0020] If the actual number is less than the preset number, and the trigger event is not within the radius of the neighboring area of ​​any core point, then the category of any trigger event is determined to be a noise point;

[0021] The number of moving objects in the detected scene and the initial position triggered by each moving object are determined according to the noise points, the core points and the edge points.

[0022] Optionally, in one embodiment of the present application, the category of any trigger event is determined by the following formula:

[0023]

[0024] Among them, p is the category of any trigger event, core is the core point, border is the edge point, noise is the noise point, ∈ is the radius of the neighboring area matching the trigger event, COUNT(N(p,∈)) is the actual number of trigger events contained in the area with ∈ as the radius of point p, and minPts is the preset number of trigger events contained in the area with ∈ as the radius.

[0025] Optionally, in one embodiment of the present application, the method further includes:

[0026] The DBSCAN clustering results of the initial positions triggered by each moving object are weighted:

[0027] If any of the triggering events is a core point of the initial position, the weight corresponding to the point is the first weight.

[0028] If any of the triggering events is an edge point of the initial position, the weight corresponding to the point is a second weight, wherein the first weight is greater than the first weight.

[0029] Optionally, in an embodiment of the present application, tracking all moving objects according to the event image frame includes:

[0030] Determine the moving direction and distance of the moving object in the event image frame by using an optical flow error algorithm, wherein the event image frame includes a first event image frame and a second event image frame;

[0031] The step of using an optical flow error algorithm to determine the moving direction and distance of the moving object corresponding to the event image frame specifically includes:

[0032] Determining a first coordinate position of a center of mass of the moving object in the first event image frame;

[0033] After one or more of the preset sampling periods, determining a second coordinate position of the centroid point in the second event image frame;

[0034] The motion component of the centroid in the direction of the coordinate axis is determined according to the first coordinate position and the second coordinate position, and the motion trajectory of the moving object is determined according to the motion component of the centroid.

[0035] Optionally, in one embodiment of the present application, determining the motion trajectory of the moving object according to the motion component of the centroid includes: determining the motion trajectory of the moving object by obtaining the motion component of the centroid in the direction of the coordinate axis when the optical flow error is minimum through the following formula:

[0036]

[0037] Wherein, e(d) is the optical flow error, C is the coordinate set of the trigger event composed of the DBSCAN clustering results of the initial position triggered by any moving object, (x, y) is the coordinate of any trigger event in C on the first event image frame, dx and dy are the motion components of the center of mass of the moving object in the direction of the coordinate axis after one or more of the preset sampling periods, (x+dx, y+dy) is the position of any trigger event in C on the second event image frame, I(x, y) is the pixel brightness value of any trigger event in C in the first event image frame, J(x+dx, y+dy) is the pixel brightness value of any trigger event in C in the second event image frame, and weight(x, y) is the different weight coefficients corresponding to the core points and edge points.

[0038] To achieve the above-mentioned purpose, the second embodiment of the present application proposes a multi-target tracking device based on an event camera, comprising:

[0039] An acquisition module is used to acquire the number of trigger events in each sampling period of a plurality of consecutive sampling periods in the measured scene;

[0040] A determination module, configured to process the triggering events in the current latest sampling period by using a DBSCAN clustering algorithm if the number of triggering events in a plurality of consecutive sampling periods exceeds a preset environmental noise threshold, so as to determine the number of moving objects in the measured scene and the initial position triggered by each of the moving objects;

[0041] The tracking module is used to determine the positions of all moving objects in the detected scene in the event image frame according to the data stream of the triggering event, and to track the moving objects according to the event image frame.

[0042] The multi-target tracking device based on an event camera in the embodiment of the present application obtains the number of triggering events in each sampling period of multiple consecutive sampling periods in the measured scene; if the number of triggering events in multiple consecutive sampling periods exceeds the preset environmental noise threshold, the triggering events in the current latest sampling period are processed by the DBSCAN clustering algorithm to determine the number of moving objects in the measured scene and the initial position triggered by each of the moving objects; the event image frame containing all the moving objects in the measured scene is determined according to the data stream of the triggering event, and the moving objects are tracked according to the event image frame. The present application quickly and accurately tracks and locates multiple moving objects in the measured scene, while ensuring low latency and extremely low computational complexity, reducing power consumption and being applicable to mobile platforms.

[0043] To achieve the above-mentioned purpose, the third aspect embodiment of the present application proposes a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the method of the first aspect embodiment of the present application.

[0044] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0046] Figure 1 A schematic diagram comparing the event camera data provided by an embodiment of the present application with the output of a traditional camera;

[0047] Figure 2 A flowchart of a multi-target tracking method based on an event camera provided in Embodiment 1 of the present application;

[0048] Figure 3 A schematic diagram of a trigger event generated by a moving object provided in an embodiment of the present application;

[0049] Figure 4 A schematic diagram of the event camera output data stream and its DBSCAN clustering result provided in an embodiment of the present application;

[0050] Figure 5 A schematic diagram of moving object tracking provided in an embodiment of the present application;

[0051] Figure 6 A schematic diagram of the structure of a multi-target tracking device based on an event camera provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be construed as limiting the present application. On the contrary, the embodiments of the present application include all changes, modifications and equivalents that fall within the spirit and connotation of the appended claims.

[0053] Based on the above description, the problem in the field of computer vision technology is that for high-speed moving objects, the images finally acquired by traditional cameras will produce dynamic blur, which makes detection and tracking difficult, and has high latency. It is impossible to guarantee a fast response in high-speed scenes, and the entire image needs to be processed, which generates a lot of calculations. In response to such problems, this application proposes a multi-target tracking method based on an event camera, which can quickly and accurately track and locate multiple moving objects in the scene under test, while ensuring extremely low computational complexity, reducing power consumption and being applied to mobile platforms.

[0054] Compared with traditional cameras, event cameras can track moving objects in dim light at night or in backlight overexposure, with a wider detection range and suitable for more occasions. Compared with detection methods such as frame difference method used by traditional cameras, event cameras can sense the appearance of moving objects more quickly in the form of data streams, and can adjust the detection frequency, providing a perception efficiency of several milliseconds to more than ten milliseconds, which greatly improves efficiency. The output comparison between event cameras and traditional cameras is as follows: Figure 1 shown.

[0055] The following describes the event camera-based multi-target tracking method, device, and computer equipment according to embodiments of the present application with reference to the accompanying drawings.

[0056] Figure 2 This is a flowchart of a multi-target tracking method based on an event camera provided in Example 1 of the present application.

[0057] like Figure 2 As shown, the multi-target tracking method based on event camera proposed in the embodiment of the present application includes the following steps:

[0058] Step 110, obtaining the number of triggering events in each sampling period of a plurality of consecutive sampling periods in the measured scene;

[0059] Step 120, if the number of triggering events in a plurality of consecutive sampling periods exceeds a preset environmental noise threshold, the triggering events in the current latest sampling period are processed by a DBSCAN clustering algorithm to determine the number of moving objects in the measured scene and the initial position triggered by each of the moving objects;

[0060] Step 130, based on the data stream of the triggering event, restore the event data stream data into an event image frame, determine the center of mass points of all moving objects in the current event image frame, and track the moving objects detected therein based on the event image frame.

[0061] Specifically, an event camera is a sensor that responds to changes in pixel brightness. It can respond to individual pixels asynchronously. When the brightness change value of a pixel exceeds a given threshold, the pixel is triggered and outputs a trigger event. The brightness change value is determined by the following formula:

[0062]

[0063] Where d is the brightness difference between the pixel at time t and time t0, (x, y) is the location of the trigger event, Lt(x, y) is the brightness value of the pixel at time t, Lt0(x, y) is the brightness value of the pixel at time t0, t is the timestamp of the current trigger event, and t0 is the timestamp of the previous trigger event.

[0064] Specifically, the data output by the event camera is in the form of a streaming output data stream, encoded in chronological order, and each trigger event is represented by a four-tuple (x, y, t, m), where (x, y) is the position of the trigger event, t is the timestamp of the current trigger event, and m is the polarity of the brightness change.

[0065] In addition, in the above embodiment, when the absolute value of d is less than the set threshold, it indicates that the event is not triggered, and m=0 is set;

[0066] When the absolute value of d is not less than the set threshold and d is greater than zero, it indicates a positive event is triggered, and m is set to +1;

[0067] When the absolute value of d is not less than the set threshold and d is less than zero, it indicates that a negative event is triggered, and m=-1 is set.

[0068] Furthermore, in one embodiment of the present application, before step 110, the following is further included:

[0069] Step 101, collecting the number of triggering events in each sampling period of a plurality of consecutive sampling periods in the measured scene, wherein the measured scene is a static scene;

[0070] Step 102: Calculate the variance of the number of triggering events in multiple sampling periods. If the variance is less than a preset threshold, determine the preset environmental noise threshold according to the mean value of the number of triggering events in multiple sampling periods.

[0071] It should be noted that the amount of noise of the event camera in the above embodiment is significantly greater than that of a traditional camera, including the inherent noise of the event camera and the noise caused by the lighting in the environment. In order to minimize the impact of environmental noise and reduce the amount of computation, the scene under test needs to be dynamically sampled at a fixed sampling period before the program starts running.

[0072] Specifically, dynamic sampling can be performed by the following formula:

[0073]

[0074] Where E is the number of all trigger events in the measured scene within the sampling period, Δt is the sampling period, (x, y) is the position of the trigger event, and t is the timestamp of the current trigger event;

[0075] Furthermore, in order to ensure that moving objects can be detected in a timely manner, a shorter sampling period is usually adopted, including but not limited to 1ms, 3ms, 5ms, and 10ms.

[0076] Specifically, after sampling is completed, in order to ensure the accuracy of environmental noise, the variance of the number of trigger events in the continuous sampling period is determined by the following formula:

[0077]

[0078] Among them, S 2 is the variance of the number of trigger events in consecutive sampling periods, n is the number of consecutive sampling periods, Ei represents the number of all trigger events in the measured scene in the i-th sampling period, is the mean value of the number of trigger events in consecutive sampling periods;

[0079] Furthermore, when the variance S of the number of trigger events in consecutive sampling periods 2 When it is less than the set threshold, the environmental noise threshold is determined by the following formula:

[0080]

[0081] Where T is the environmental noise threshold, n is the number of consecutive sampling cycles, Ei represents the number of all trigger events in the measured scene within the i-th sampling cycle, and k is the proportional coefficient;

[0082] Furthermore, in order to further reduce the impact of environmental noise, the proportional coefficient k in the embodiment of the present application can be set between 1.5-2.5. For example, k can be 1.5, 1.6, 1.65, 1.7, 1.75, 1.8, 1.85, 1.9, 2.0, 2.1, 2.2, 2.3, 2.4, 2, 5, etc., that is, the proportional coefficient k can be set between 1.5-2.5.

[0083] In the embodiment of the present application, before the triggering events in the plurality of sampling periods are processed by the DBSCAN clustering algorithm, the following steps are included:

[0084] Determine a radius of a neighboring area that matches the trigger event and a preset number of trigger events within the radius of the neighboring area.

[0085] In an embodiment of the present application, the triggering events in the plurality of consecutive sampling periods are processed by the DBSCAN clustering algorithm to determine the number of moving objects in the measured scene and the initial position triggered by each moving object, specifically including:

[0086] Determine the actual number of trigger events contained in the radius of the neighboring area by any trigger event among the trigger events through the DBSCAN clustering algorithm, wherein if the actual number is not less than the preset number, determine the category of any trigger event as a core point;

[0087] If the actual number is less than the preset number, but the trigger event is within the radius of the neighboring area of ​​any core point, then the category of any trigger event is determined to be an edge point;

[0088] If the actual number is less than the preset number, and the trigger event is not within the radius of the neighboring area of ​​any core point, then the category of any trigger event is determined to be a noise point;

[0089] The number of moving objects in the detected scene and the initial position triggered by each moving object are determined according to the noise points, the core points and the edge points.

[0090] Specifically, due to the characteristics of event cameras, for data streams with moving objects, most trigger events are generated at the edge. Figure 3 As shown, using DBSCAN clustering can better divide different objects. The output data stream of the event camera and its DBSCAN clustering result are shown in Figure 4 As shown, Figure 4 (a) is a schematic diagram of the event camera output data flow. Figure 4 (b) Schematic diagram of DBSCAN clustering results of event camera output data stream.

[0091] Furthermore, since the noise generated by the event camera is independent and the generation of a single noise point is unrelated to the surrounding triggering events, the DBSCAN clustering algorithm can better filter the noise points.

[0092] In the embodiment of the present application, the category of any trigger event is determined by the following formula:

[0093]

[0094] Among them, p is the category of any triggering event, core is the core point, border is the edge point, noise is the noise point, ∈ is the radius of the neighboring area matching the triggering event, COUNT(N(p, ∈)) is the actual number of triggering events included in the area with radius ∈ centered at point p, and minPts is the preset number of triggering events included in the area with radius ∈.

[0095] Specifically, the category of any triggering event is determined through the following procedure:

[0096] For the sample set D = {x1, x2,..., xn}, mark all the points in it as unprocessed.

[0097] for each object p in the data set D

[0098] if p has been classified then continue

[0099] Calculate N(p, ∈)

[0100] if N(p, ∈) < minPts then mark this point as a noise point

[0101] else mark p as a core point and create a new cluster C, and put p into cluster C

[0102] S = {set of all points q in the neighborhood of p}

[0103] for all elements t in S

[0104] if t has been marked as a noise point then modify it to an edge point

[0105] if t has been marked then continue

[0106] Classify t into cluster C

[0107] if N(t, ∈) ≥ minPts then

[0108] Include the points in the neighborhood of point t in the set S, and mark t as a core point In the embodiments of the present application, tracking the moving object according to the event image frame includes:

[0109] Use the optical flow error algorithm to determine the moving direction and distance of the moving object in the event image frame, where the event image frame includes a first event image frame and a second event image frame;

[0110] The use of the optical flow error algorithm to determine the moving direction and distance of the moving object corresponding to the event image frame specifically includes:

[0111] Determining a first coordinate position of a center of mass of the moving object in the first event image frame;

[0112] After one or more of the preset sampling periods, determining a second coordinate position of the centroid point in the second event image frame;

[0113] The motion component of the centroid in the direction of the coordinate axis is determined according to the first coordinate position and the second coordinate position, and the motion trajectory of the moving object is determined according to the motion component of the centroid.

[0114] It should be noted that the traditional optical flow operation only calculates the grayscale matching error of a single key point. This application regards all events triggered by the moving object as a whole, calculates the overall optical flow error, and gives a higher weight to the core points obtained by the DBSCAN algorithm, thereby determining the movement direction and distance of the moving object.

[0115] In an embodiment of the present application, determining the motion trajectory of the moving object according to the motion component of the centroid includes: determining the motion trajectory of the moving object by obtaining the motion component of the centroid in the direction of the coordinate axis when the optical flow error is minimum through the following formula:

[0116] e(d)=e(d x ,d y )=∑ (x,y)∈C weight (x,y) (I(x,y)-J(x+d x ,y+d y )) 2

[0117] Wherein, e(d) is the optical flow error, C is the coordinate set of the trigger event composed of the DBSCAN clustering results of the initial position triggered by any moving object, (x, y) is the coordinate of any trigger event in C on the first event image frame, dx and dy are the motion components of any trigger event in C in the coordinate axis direction after one or more of the preset sampling cycles, (x+dx, y+dy) is the position of any trigger event in C on the second event image frame, I(x, y) is the pixel brightness value of any trigger event in C in the first event image frame, J(x+dx, y+dy) is the pixel brightness value of any trigger event in C in the second event image frame, and weight(x, y) is the different weight coefficients corresponding to the core points and edge points.

[0118] It should be noted that, for a detected moving object, the centroid of the moving object is determined to be the center position of the moving object, and the center position is tracked. Figure 5 As shown, Figure 5(a) is the tracking diagram of the second frame. Figure 5 (b) is a tracking diagram of the third frame, where the line segments are tracking trajectories.

[0119] Specifically, the event image frame is a binary image. When the motion trajectory of an object is calculated based on the optical flow, the calculated optical flow error can better reflect the deviation of the motion. Due to the use of a shorter sampling period, it is assumed that within the sampling time, all triggering events triggered by the moving object have the same motion trajectory, that is, they have the same dx and dy.

[0120] The multi-target tracking method based on an event camera in the embodiment of the present application obtains the number of trigger events in each sampling period of multiple consecutive sampling periods in the measured scene; if the number of trigger events in multiple consecutive sampling periods exceeds the preset environmental noise threshold, the trigger events in the current latest sampling period are processed by the DBSCAN clustering algorithm to determine the number of moving objects in the measured scene and the initial position triggered by each of the moving objects; the event image frame containing all the moving objects in the measured scene is determined according to the data stream of the trigger event, and the moving objects are tracked according to the event image frame. The present application quickly and accurately tracks and locates multiple moving objects in the measured scene, while ensuring low latency and extremely low computational complexity, reducing power consumption and being applicable to mobile platforms.

[0121] In order to implement the above embodiments, the present application also proposes a multi-target tracking device based on an event camera.

[0122] Figure 6 A schematic diagram of the structure of a multi-target tracking device based on an event camera provided in an embodiment of the present application.

[0123] like Figure 6 As shown, the multi-target tracking device based on event camera includes: an acquisition module 610 , a determination module 620 and a tracking module 630 .

[0124] An acquisition module 610 is used to acquire the number of triggering events in each sampling period of a plurality of consecutive sampling periods in the measured scene;

[0125] A determination module 620 is configured to process the triggering events in the current latest sampling period by using a DBSCAN clustering algorithm to determine the number of moving objects in the measured scene and the initial position triggered by each moving object if the number of triggering events in multiple consecutive sampling periods exceeds a preset environmental noise threshold;

[0126] The tracking module 630 is used to determine the positions of all moving objects in the detected scene in the event image frame according to the data stream of the triggering event, and track the moving objects according to the event image frame.

[0127] The multi-target tracking device based on an event camera in the embodiment of the present application obtains the number of triggering events in each sampling period of multiple consecutive sampling periods in the measured scene; if the number of triggering events in multiple consecutive sampling periods exceeds the preset environmental noise threshold, the triggering events in the current latest sampling period are processed by the DBSCAN clustering algorithm to determine the number of moving objects in the measured scene and the initial position triggered by each of the moving objects; the event image frame containing all the moving objects in the measured scene is determined according to the data stream of the triggering event, and the moving objects are tracked according to the event image frame. The present application quickly and accurately tracks and locates multiple moving objects in the measured scene, while ensuring low latency and extremely low computational complexity, reducing power consumption and being applicable to mobile platforms.

[0128] It should be noted that the aforementioned explanation of the embodiment of the multi-target tracking method based on an event camera is also applicable to the multi-target tracking device based on an event camera of this embodiment, and will not be repeated here.

[0129] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application.

[0130] In order to implement the above embodiments, the embodiments of the present application also propose a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method described in Embodiment 1 of the present application is implemented.

[0131] It should be noted that, in the description of this application, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, the meaning of "plurality" is two or more.

[0132] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0133] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0134] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0135] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0136] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0137] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0138] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A multi-target tracking method based on event camera, characterized in that: The method comprises: Obtain the number of trigger events in each sampling period of multiple consecutive sampling periods in the measured scene; If the number of trigger events in multiple consecutive sampling periods exceeds the preset environmental noise threshold, the radius of the neighboring area matching the trigger event and the preset number of trigger events within the radius of the neighboring area are determined, and the trigger events in the current latest sampling period are processed by the DBSCAN clustering algorithm to determine the number of moving objects in the measured scene and the initial position triggered by each of the moving objects; wherein, the actual number of trigger events contained in the radius of the neighboring area of ​​any trigger event among the trigger events is determined by the DBSCAN clustering algorithm, and if the actual number is not less than the preset number, the category of any trigger event is determined to be a core point; if the actual number is less than the preset number, but the trigger event is within the radius of the neighboring area of ​​any core point, the category of any trigger event is determined to be an edge point; if the actual number is less than the preset number, and the trigger event is not within the radius of the neighboring area of ​​any core point, the category of any trigger event is determined to be a noise point; the number of moving objects in the measured scene and the initial position triggered by each of the moving objects are determined according to the noise points, the core points and the edge points; An event image frame including all moving objects in the detected scene is determined according to the data stream of the triggering event, and the moving objects are tracked according to the event image frame.

2. The method according to claim 1, characterized in that Before obtaining the number of triggering events in each sampling period of a plurality of sampling periods in the measured scene, the method further includes: Collecting the number of triggering events in each sampling period of a plurality of consecutive sampling periods in the measured scene, wherein the measured scene is a static scene; The variance of the number of triggering events in a plurality of consecutive sampling periods is calculated, and if the variance is less than a preset threshold, the preset environmental noise threshold is determined according to the mean value of the number of triggering events in a plurality of consecutive sampling periods.

3. The method according to claim 1, characterized in that The category of any of the trigger events is determined by the following formula: Among them, p is the category of any trigger event, core is the core point, border is the edge point, noise is the noise point, ∈ is the radius of the neighboring area matching the trigger event, COUNT(N(p,∈)) is the actual number of trigger events contained in the area with ∈ as the radius of point p, and minPts is the preset number of trigger events contained in the area with ∈ as the radius.

4. The method according to claim 1, characterized in that The method further comprises: The DBSCAN clustering results of the initial positions triggered by each moving object are weighted: If any of the trigger events is a core point of the initial position, the weight corresponding to the point is the first weight. If any of the triggering events is an edge point of the initial position, the weight corresponding to the point is a second weight, wherein the first weight is greater than the first weight.

5. The method according to claim 4, characterized in that The tracking of the moving object according to the event image frame includes: Determine the moving direction and distance of the moving object in the event image frame by using an optical flow error algorithm, wherein the event image frame includes a first event image frame and a second event image frame; The step of using an optical flow error algorithm to determine the moving direction and distance of the moving object corresponding to the event image frame specifically includes: Determining a first coordinate position of a center of mass of the moving object in the first event image frame; After one or more of the preset sampling periods, determining a second coordinate position of the centroid point in the second event image frame; The motion component of the centroid in the direction of the coordinate axis is determined according to the first coordinate position and the second coordinate position, and the motion trajectory of the moving object is determined according to the motion component of the centroid.

6. The method according to claim 5, characterized in that Determining the motion trajectory of the moving object according to the motion component of the centroid includes: determining the motion trajectory of the moving object by obtaining the motion component of the centroid in the direction of the coordinate axis when the optical flow error is minimum through the following formula: e(d)=e(d x ,d y )=∑ (x,y)∈C weight (x,y) (I(x,y)-J(x+d x ,y+d y )) 2 Wherein, e(d) is the optical flow error, C is the coordinate set of the trigger event composed of the DBSCAN clustering results of the initial position triggered by any moving object, (x, y) is the coordinate of any trigger event in C on the first event image frame, dx and dy are the motion components of the center of mass of the moving object in the direction of the coordinate axis after one or more of the preset sampling periods, (x+dx, y+dy) is the position of any trigger event in C on the second event image frame, I(x, y) is the pixel brightness value of any trigger event in C in the first event image frame, J(x+dx, y+dy) is the pixel brightness value of any trigger event in C in the second event image frame, and weight(x, y) is the different weight coefficients corresponding to the core points and edge points.

7. A multi-target tracking device based on an event camera, characterized in that: The device comprises: An acquisition module is used to acquire the number of trigger events in each sampling period of a plurality of consecutive sampling periods in the measured scene; A determination module, for determining, if the number of trigger events in a plurality of consecutive sampling periods exceeds a preset environmental noise threshold, a neighborhood radius matching the trigger event and a preset number of trigger events within the neighborhood radius, and processing the trigger events in the current latest sampling period by a DBSCAN clustering algorithm to determine the number of moving objects in the measured scene and the initial position triggered by each of the moving objects; wherein, the actual number of trigger events contained in the neighborhood radius of any trigger event among the trigger events is determined by the DBSCAN clustering algorithm, and if the actual number is not less than the preset number, the category of any trigger event is determined to be a core point; if the actual number is less than the preset number, but the trigger event is within the neighborhood radius of any core point, the category of any trigger event is determined to be an edge point; if the actual number is less than the preset number, and the trigger event is not within the neighborhood radius of any core point, the category of any trigger event is determined to be a noise point; the number of moving objects in the measured scene and the initial position triggered by each of the moving objects are determined according to the noise points, the core points and the edge points; The tracking module is used to determine the positions of all moving objects in the detected scene in the event image frame according to the data stream of the triggering event, and to track the moving objects according to the event image frame.

8. A computer device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

Images