Object detection method and system based on adaptive time window and time threshold

Abstract

The application discloses an object detection method and system based on adaptive time window and time threshold, and the method comprises the following steps: acquiring event camera data, acquiring an event set in an optimal time window through an adaptive time window method; eliminating a part of events generated by self-motion of the event camera from the event set in the optimal time window through an adaptive compensation algorithm, and acquiring a compensated event set; constructing an event frame image through an adaptive threshold method according to the acquired optimal time window and the compensated event set; and performing iterative image matrix fitting on the constructed event frame image to acquire a final detection result. The application provides an object detection method based on adaptive time window and time threshold, realizes detection of fast-moving objects through an adaptive time window, self-motion compensation of a camera, a threshold of a time image and an iterative image matrix fitting method, thereby realizing the requirement of obstacle avoidance of fast-moving obstacles in the environment and improving the obstacle avoidance performance of the unmanned aerial vehicle.

Description

Technical Field

[0001] This invention relates to the field of obstacle avoidance technology for unmanned aerial vehicles (UAVs), specifically to an object detection method and system based on adaptive time windows and time thresholds. Background Technology

[0002] Because drones pose a certain risk of collision during flight and operation, obstacle avoidance systems are integrated into drones to minimize the incidence of damage and injuries to people and buildings caused by operational errors or accidents. This makes the drone system more autonomous, safe, and reliable. A key step in achieving obstacle avoidance is the ability to detect obstacles.

[0003] In the field of drone obstacle avoidance, vision-based obstacle avoidance methods dominate. These include obstacle avoidance algorithms based on monocular cameras, binocular cameras, or depth cameras. These algorithms rely on frame-based vision sensors mounted on the drone to obtain visual information about the current scene, then use corresponding perception algorithms to detect the presence of obstacles in the environment, and finally execute obstacle avoidance actions.

[0004] In recent years, with the continuous improvement of camera performance and the enhancement of computer hardware computing power, this solution has been widely used in drones. For example, DJI's Mavic 3 uses a binocular vision system with multiple cameras and an infrared sensor on the bottom of the fuselage to achieve omnidirectional obstacle avoidance.

[0005] However, this type of method still has two problems:

[0006] (1) Traditional visual obstacle avoidance methods assume that obstacles in the environment are static or quasi-static (low-speed moving). Most obstacle avoidance methods using standard cameras currently assume that there are no fast-moving objects in the current scene, so existing methods are not applicable to obstacles that are moving fast in the environment.

[0007] (2) Standard cameras suffer from motion blur due to their hardware limitations, which increases the difficulty of obstacle detection. Since standard cameras require a certain exposure time, the motion blur they produce can cause detection algorithms to fail. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of the above-mentioned background technology and provide an object detection method and system based on adaptive time windows and time thresholds.

[0009] In a first aspect, this application provides an object detection method based on adaptive time windows and time thresholds, comprising the following steps:

[0010] Acquire event camera data and obtain the set of events within the optimal time window using an adaptive time window method;

[0011] An adaptive compensation algorithm is used to eliminate a portion of the events generated by camera self-motion from the event set within the optimal time window, thereby obtaining the compensated event set.

[0012] Based on the obtained optimal time window and the compensated event set, an event frame image is constructed using an adaptive thresholding method;

[0013] The constructed event frame image is subjected to iterative image moment fitting to obtain the final detection result.

[0014] According to the first aspect, in a first possible implementation of the first aspect, the step of acquiring event camera data and obtaining the event set within the optimal time window using an adaptive time window method specifically includes the following steps:

[0015] The time window is initially estimated by taking the timestamp of the first event as the start time of the current time window;

[0016] Obtain the detection status of moving objects within the previous time window;

[0017] Based on the obtained detection conditions, different optimal time window selection strategies are implemented to obtain the event set within the optimal time range.

[0018] According to the first possible implementation of the first aspect, in the second possible implementation of the first aspect, the step of executing different optimal time window selection strategies based on the acquired detection conditions to obtain the event set within the optimal time range specifically includes the following steps:

[0019] When a moving object is detected in the previous time window, the optimal time window is selected based on the size of the previous time window, and the event set within the optimal time window is obtained.

[0020] If no moving object is detected in the previous time window, the optimal time window is selected based on the size of the smallest time window, and the set of events within the optimal time window is obtained.

[0021] According to the first aspect, in the third possible implementation of the first aspect, the step of eliminating a portion of the events generated by camera self-motion from the event set within the optimal time window using an adaptive compensation algorithm to obtain the compensated event set specifically includes the following steps:

[0022] Perform rotational motion compensation on the event set within the optimal time window;

[0023] Perform translational motion compensation on the event set after rotational motion compensation to obtain the compensated event set.

[0024] According to the first aspect, in the fourth possible implementation of the first aspect, the step of constructing the event frame image using an adaptive thresholding method based on the obtained optimal time window and the compensated event set specifically includes the following steps:

[0025] Construct a time graph within the current time window using the compensated set of events;

[0026] The constructed time image is filtered for background events to obtain the filtered time image.

[0027] A threshold method is applied to process the filtered time image to obtain the event frame image.

[0028] According to the first aspect, in the fifth possible implementation of the first aspect, the step of iteratively fitting image moments to the constructed event frame image to obtain the final detection result specifically includes the following steps:

[0029] Apply mean filtering to the event frame image to remove salt-and-pepper noise and obtain the filtered event frame image;

[0030] The midpoint of the detection box in the time frame image after initialization and filtering is the position of the maximum timestamp of the event frame image;

[0031] The final detection result is obtained by iterating from the position with the largest timestamp of the event frame, updating the points in the detection box and the size of the detection box.

[0032] Secondly, this application provides a moving object detection system based on adaptive time windows and time thresholds, comprising:

[0033] The event camera data acquisition module is used to acquire the set of events within the optimal time window using an adaptive time window method.

[0034] The event set acquisition module is communicatively connected to the event camera data acquisition module. It is used to eliminate a portion of the events generated by the self-motion of the event camera from the event set within the optimal time window through an adaptive compensation algorithm, and obtain the compensated event set.

[0035] The event frame image acquisition module is communicatively connected to the event set acquisition module and is used to construct an event frame image using an adaptive threshold method based on the acquired optimal time window and the compensated event set.

[0036] The detection result acquisition module is communicatively connected to the event frame image acquisition module and is used to perform iterative image moment fitting on the constructed event frame image to obtain the final detection result.

[0037] According to the second aspect, in a first possible implementation of the second aspect, the event camera data acquisition module includes:

[0038] The time window preliminary estimation unit is used to make a preliminary estimation of the time window, taking the timestamp of the first event as the start time of the current time window.

[0039] The detection condition acquisition unit is used to acquire the detection condition of moving objects within the previous time window;

[0040] The event set acquisition unit is communicatively connected to the current time window acquisition unit and the detection condition acquisition unit. It is used to execute different optimal time window selection strategies based on the acquired detection conditions to acquire the event set within the optimal time range.

[0041] According to the second possible implementation of the first aspect, in the second possible implementation of the first aspect, the event set acquisition unit includes:

[0042] The first event set acquisition sub-unit is used to select the optimal time window based on the size of the previous time window when a moving object is detected in the previous time window, and to acquire the event set within the optimal time window.

[0043] The second event set acquisition sub-unit is used to select the optimal time window based on the size of the smallest time window when no moving object is detected in the previous time window, and to acquire the event set within the optimal time window.

[0044] According to the second aspect, in a second possible implementation of the second aspect, the event frame image acquisition module includes:

[0045] The rotational motion compensation subunit is used to perform rotational motion compensation on the event set within the optimal time window.

[0046] The translational motion compensation subunit is communicatively connected to the rotational motion compensation subunit and is used to perform translational motion compensation on the event set after rotational motion compensation to obtain the compensated event set.

[0047] Compared with the prior art, the advantages of the present invention are as follows:

[0048] This application provides an object detection method based on adaptive time windows and time thresholds. The method uses adaptive time windows, camera self-motion compensation, time image thresholding, and iterative image moment fitting to detect fast-moving objects, thereby meeting the obstacle avoidance requirements of fast-moving obstacles in the environment and improving the obstacle avoidance performance of UAVs. Attached Figure Description

[0049] Figure 1This is a flowchart of the object detection method based on adaptive time window and time threshold according to an embodiment of the present invention;

[0050] Figure 2(a) is a result of the object detection method based on adaptive time window and time threshold in an embodiment of the present invention for detecting moving objects within the initial time window Δt0;

[0051] Figure 2(b) is a diagram showing the detection results of the moving object detection method based on adaptive time window and time threshold in an embodiment of the present invention within the first time window Δt1.

[0052] Figure 2(c) is a diagram showing the detection results of the moving object detection method based on adaptive time window and time threshold in an embodiment of the present invention within the second time window Δt2.

[0053] Figure 2(d) is a diagram showing the detection results of the object detection method based on adaptive time window and time threshold in this embodiment of the invention for detecting moving objects within the third time window Δt3;

[0054] Figure 2(e) is a diagram showing the detection results of the object detection method based on adaptive time window and time threshold in this embodiment of the invention for detecting moving objects within the fourth time window Δt4;

[0055] Figure 2(f) is a diagram showing the detection results of the object detection method based on adaptive time window and time threshold in this embodiment of the invention for detecting moving objects within the fifth time window Δt5;

[0056] Figure 3 This is a functional block diagram of an object detection system based on adaptive time windows and time thresholds according to an embodiment of the present invention. Detailed Implementation

[0057] Referring now to specific embodiments of the invention, examples of which are illustrated in the accompanying drawings. Although the invention will be described in conjunction with specific embodiments, it will be understood that it is not intended to limit the invention to the described embodiments. Rather, it is intended to cover variations, modifications, and equivalents included within the spirit and scope of the invention as defined by the appended claims. It should be noted that the method steps described herein can be implemented by any functional block or functional arrangement, and any functional block or functional arrangement can be implemented as a physical entity or a logical entity, or a combination of both.

[0058] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0059] Note: The examples described below are merely specific examples and are not intended to limit the embodiments of the present invention to the specific steps, values, conditions, data, order, etc. Those skilled in the art can utilize the concept of the present invention to construct more embodiments not mentioned herein by reading this specification.

[0060] Drones face certain collision risks during flight and operation, so it is necessary to integrate obstacle avoidance systems on drones to make them more autonomous, safe, and reliable. However, traditional visual obstacle avoidance methods assume that obstacles in the environment are static or quasi-static, which is not suitable for obstacles that move quickly in the environment, and their obstacle avoidance performance is not perfect.

[0061] In view of this, this application provides an object detection method based on adaptive time windows and time thresholds to solve the technical problem that existing UAVs cannot achieve obstacle avoidance requirements for fast-moving obstacles in the environment.

[0062] Firstly, please refer to Figure 1 This application provides an object detection method based on adaptive time windows and time thresholds for obstacle avoidance detection of moving objects by a drone camera, including the following steps:

[0063] Step S1: Acquire event camera data and obtain the event set within the optimal time window using the adaptive time window method to maximize the contrast between the moving and static areas;

[0064] Step S2: Eliminate a portion of the events generated by camera self-motion from the event set within the optimal time window using an adaptive compensation algorithm to obtain the compensated event set;

[0065] Step S3: Based on the obtained optimal time window and the compensated event set, construct the event frame image using an adaptive thresholding method;

[0066] Step S4: Perform iterative image moment fitting on the constructed event frame image to obtain the final detection result.

[0067] This application provides an object detection method based on adaptive time windows and time thresholds. The method reduces the amount of event processing required for moving object detection by using adaptive time windows, improves detection accuracy by eliminating events generated by camera self-motion through camera self-motion compensation, filters background in time images by applying thresholds to time images, and achieves the detection of fast-moving objects by using iterative image moment fitting methods. This enables the detection of obstacles that move rapidly in the environment and improves the obstacle avoidance performance of UAVs.

[0068] The final detection result includes whether a moving object was detected or not.

[0069] In one embodiment, step S1, acquiring event camera data and obtaining the event set within the optimal time window using an adaptive time window method, specifically includes the following steps:

[0070] Step S11: Assume the timestamp of the first event in the current time window is t0. Using the timestamp of the first event as the start time of the current time window, initially estimate the time window size as Δt0, and the maximum number of events to be processed is N. Since each pixel in the event camera's internal circuitry triggers events based on relative brightness changes, the number of events transmitted will be large when there are fast-moving objects in the scene. Based on this principle, first calculate the number of events within 5ms. Since events are output according to their timestamp size, binary search can be used to accelerate this search process. To initially estimate the time window, sample the number of events within the range [t0, t0+5ms].

[0071] Index=Lower_Bound(E,t0+5ms)

[0072] The input event stream data is: Where e i = {x, y, p, t}, Lower_Bound is a binary search function, that is, to find the first position in the event stream data E with a timestamp greater than or equal to t0+5ms, and to find the position of the event with the sampling time by applying the binary search method, so as to obtain the number of events generated within the sampling time;

[0073] Step S12: Obtain the detection status of moving objects within the previous time window;

[0074] Step S13: Based on the obtained detection conditions, execute different optimal time window selection strategies to obtain the event set within the optimal time range.

[0075] In one embodiment, step S13, which involves executing different optimal time window selection strategies based on the acquired detection conditions to obtain the event set within the optimal time range, specifically includes the following steps:

[0076] Step S131: When a moving object is detected in the previous time window, select the optimal time window according to the size of the previous time window and obtain the event set within the optimal time window;

[0077] Step S132: When no moving object is detected in the previous time window, select the optimal time window according to the size of the minimum time window and obtain the event set within the optimal time window.

[0078] In one embodiment, step S13, based on the acquired detection conditions, specifically implements different optimal time window selection strategies as follows: When a moving object is detected in the previous time window, this moving object may still exist in the current scene. In this case, the current time window should be calculated by multiplying the size of the previous time window by a scaling factor, i.e., calculated based on the value of the previous time window and the number of events within the sampling time window. Otherwise, when no moving object is detected in the previous time window, this moving object still does not exist in the current scene. The current time window is calculated by multiplying the minimum time window by a scaling factor, where the scaling factor α is calculated by dividing the maximum number of events processed at one time by the total number of events within these 5ms. Finally, an optimal time window is designed to prevent the calculated time window from being too large, resulting in too many events being processed at one time, as shown in the following formula:

[0079]

[0080] Where, Δt i-1 Δt represents the size of the previous time window. i Let n be the optimal time window size at time i; i This represents the number of events within the time window [t0, t0+5ms]; N represents the maximum number of events processed at one time, t min For the minimum time window, t max For the maximum time window, the object is detected in Δt. i-1 The `else` option means that a moving object was detected in the previous time window, while `else` means that no moving object was detected in the previous time window.

[0081] In one embodiment, step S2, which involves eliminating a portion of the events generated by camera self-motion from the event set within the optimal time window using an adaptive compensation algorithm to eliminate motion blur caused by the camera's hardware limitations and motion blur during exposure time, thereby improving the accuracy of the detection algorithm and obtaining the compensated event set, specifically includes the following steps:

[0082] Step S21: Perform rotational motion compensation on the event set within the optimal time window. Specifically, apply the Rodriguez rotation formula to the event set within the optimal time window at time i. Rotational motion compensation is performed on the event points in the data.

[0083] v rot =v+(1-cosθ)K 2 v+sinθ·Kv

[0084] Where v is the homogeneous coordinate of the event point, θ is the rotation angle obtained by multiplying the three-axis angular velocity obtained by the IMU by time, and K is an antisymmetric matrix. rot It is the new coordinate vector of v after rotational motion compensation;

[0085] Step S22: Perform translational motion compensation on the event set after rotational motion compensation to obtain the compensated event set:

[0086]

[0087] Among them, X e Represents a collection of events The position of the event point in the camera coordinate system, T WE It is the transformation matrix from coordinates in the world coordinate system to coordinates in the event camera coordinate system. It is the transformation matrix from the event camera coordinate system to the world coordinate system. It is the translational motion compensation matrix of the event camera during the time interval t-t0.

[0088] In one embodiment, to filter the background in the time image, step S3, the step of constructing the event frame image using an adaptive thresholding method based on the obtained optimal time window and the compensated event set, specifically includes the following steps:

[0089] Since traditional computer vision methods operate on images, to apply these algorithms to event data, the timestamps of events within this time window, after motion compensation, are used to construct a time image. To ensure that the value of a moving object's position in the image is larger than that of a static object, the minimum timestamp of all event points at that position is used as the value of that position in the time image. Specifically, the time image I within the current time window is constructed using the compensated event set; it is a W×H two-dimensional matrix, where I... i，j The calculation is as follows:

[0090]

[0091] Among them, I i，j e represents the value at position (i, j) in the time-mapping image. k (t) is the timestamp of the k-th event, e k (x) is the x-coordinate of the k-th event, e k (y) is the y-coordinate of the k-th event.

[0092] The constructed time image is filtered for background events to obtain the filtered time image, retaining only the regions of moving objects.

[0093] After obtaining the time image, the values ​​in the background region are relatively small. To filter out background events and retain only the regions of moving objects, this invention uses an adaptive threshold formula. The filtered time image is processed using a threshold θ(ω, v) method to obtain event frame images. The threshold is calculated using the following adaptive threshold formula:

[0094] θ(ω,v)=mean(I)+a||ω||+b||v||+c

[0095] Where mean represents the average value function; a, b, and c are hyperparameters; and ω and v are the event camera parameters at [t0, t0+Δt]. i The adaptive threshold formula combines the average timestamp of the event frame, the camera's angular velocity, and the linear velocity. The average timestamp is used because events generated by moving objects have relatively large timestamps, while background events have relatively small timestamps; therefore, they are typically larger than the average timestamp of the event frame. The camera's angular velocity and linear velocity are used because the faster the speed, the more frequently events occur, and therefore the higher the threshold should be.

[0096] Applying a thresholding method to time image I yields the corresponding event frame image.

[0097] in, For the event frame image of the j-th event at time i, I i，j This represents the value at position (i, j) in the time-lapse image.

[0098] In one embodiment, step S4, the step of iteratively fitting image moments to the constructed event frame image to obtain the final detection result, specifically includes the following steps:

[0099] Apply mean filtering to the event frame image to remove salt-and-pepper noise, and obtain the filtered event frame image. Specifically, for the time frame image... The mean filter is applied to remove a certain amount of salt-and-pepper noise, as shown below:

[0100]

[0101] in, The filtered event frame image is the image of the point with coordinates (x, y).

[0102] Midpoint of the detection box in the time frame image after initialization filtering For event frame images The maximum timestamp position, the initial size of the detection box is

[0103] The final detection result is obtained by iterating from the position with the largest timestamp of the event frame, updating the points in the detection box and the size of the detection box.

[0104] In one embodiment, the step of iterating from the position with the largest timestamp of the event frame, updating the points in the detection box and the size of the detection box, and obtaining the final detection result specifically includes the following steps:

[0105] Update the coordinates of the points in the detection box using the following formula. and detection box size

[0106]

[0107]

[0108] Where, μ 00 It is the zeroth central moment of the previous region, μ 20 and μ 02 It is the second central moment of the previous region. Let be the coordinates of the midpoint of the detection box at the k-th iteration. Let L be the coordinates of the midpoint of the detection box during the (k-1)th iteration. k This represents the size of the detection box during the k-th iteration.

[0109] Repeat the above update of the coordinates of the points in the detection box. and detection box size L 0 The process continues until a moving object is detected or no moving object is detected, providing a basis for the drone to avoid obstacles.

[0110] This application uses event frame images. Mean filtering is applied to eliminate salt-and-pepper noise generated by the event camera, and then image moment fitting is applied to obtain the final detection result, so as to meet the obstacle avoidance detection requirements of moving objects.

[0111] To verify the feasibility of the method of this invention, a scenario was designed as follows: First, in an office setting, a drone equipped with an event camera hovers at a certain height; then, a basketball is thrown from the right side of the camera to the left. In Figures 2(a)-(f), the brighter the whiter the area, the larger the value at that location. It can be seen that the proposed method has a good detection effect on the edge parts of moving objects and can effectively filter out static areas in the current scene from the temporal image. The results show that the method proposed in this invention can effectively detect moving objects using data generated by the event camera.

[0112] Secondly, such as Figure 3As shown, based on the same inventive concept, this application provides a moving object detection system based on adaptive time windows and time thresholds, including an event camera data acquisition module 100, an event set acquisition module 200, an event frame image acquisition module 300, and a detection result acquisition module 400. The event camera data acquisition module 100 is used to acquire the event set within the optimal time window using an adaptive time window method. The event set acquisition module 200 is communicatively connected to the event camera data acquisition module 100 and is used to eliminate a portion of the events generated by the event camera's self-motion from the event set within the optimal time window using an adaptive compensation algorithm to acquire a compensated event set. The event frame image acquisition module 300 is communicatively connected to the event set acquisition module 200 and is used to construct an event frame image using an adaptive threshold method based on the acquired optimal time window and the compensated event set. The detection result acquisition module 400 is communicatively connected to the event frame image acquisition module 300 and is used to perform iterative image moment fitting on the constructed event frame image to obtain the final detection result.

[0113] In one embodiment, the event camera data acquisition module includes a time window preliminary estimation unit, a detection condition acquisition unit, and an event set acquisition unit. The time window preliminary estimation unit is used to initially estimate the time window using the timestamp of the first event as the start time of the current time window. The detection condition acquisition unit is used to acquire the detection condition of the moving object within the previous time window. The event set acquisition unit is communicatively connected to the current time window acquisition unit and the detection condition acquisition unit, and is used to execute different optimal time window selection strategies based on the acquired detection condition to acquire the event set within the optimal time range.

[0114] In one embodiment, the event set acquisition unit includes a first event set acquisition subunit and a second event set acquisition subunit. The first event set acquisition subunit is used to select an optimal time window based on the size of the previous time window when a moving object is detected in the previous time window, and acquire the event set within the optimal time window. The second event set acquisition subunit is used to select an optimal time window based on the size of the minimum time window when no moving object is detected in the previous time window, and acquire the event set within the optimal time window.

[0115] In one embodiment, the event frame image acquisition module includes a rotational motion compensation subunit and a translational motion compensation subunit, used to perform rotational motion compensation on the event set within the optimal time window; and is communicatively connected to the rotational motion compensation subunit, used to perform translational motion compensation on the rotationally compensated event set, and to acquire the compensated event set.

[0116] Based on the same inventive concept, embodiments of this application also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements all or part of the method steps of the above method.

[0117] The present invention can implement all or part of the processes in the above methods, or it can be accomplished by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0118] Based on the same inventive concept, embodiments of this application also provide an electronic device, including a memory and a processor. The memory stores a computer program that runs on the processor. When the processor executes the computer program, it implements all or part of the method steps described above.

[0119] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the computer device, connecting all parts of the computer device through various interfaces and lines.

[0120] Memory can be used to store computer programs and / or modules. The processor performs various functions of the computer device by running or executing the computer programs and / or modules stored in the memory, and by accessing data stored in the memory. Memory can primarily include a program storage area and a data storage area. The program storage area can store the operating system and at least one application program required for a function (e.g., sound playback, image playback, etc.); the data storage area can store data created based on the use of the mobile phone (e.g., audio data, video data, etc.). Furthermore, memory can include high-speed random access memory, and can also include non-volatile memory, such as hard disks, RAM, plug-in hard disks, SmartMedia Cards (SMC), Secure Digital (SD) cards, Flash Cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0121] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, servers, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0122] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), servers, and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0123] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0124] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0125] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. An object detection method based on adaptive time window and time threshold, characterized in that, Includes the following steps: Acquire event camera data and obtain the set of events within the optimal time window using an adaptive time window method; An adaptive compensation algorithm is used to eliminate a portion of the events generated by camera self-motion from the event set within the optimal time window, thereby obtaining the compensated event set. Based on the obtained optimal time window and the compensated event set, an event frame image is constructed using an adaptive thresholding method; The constructed event frame image is iteratively fitted with image moments to obtain the final detection result; The step of acquiring event camera data and obtaining the event set within the optimal time window using an adaptive time window method specifically includes the following steps: The time window is initially estimated by taking the timestamp of the first event as the start time of the current time window; Obtain the detection status of moving objects within the previous time window; Based on the obtained detection conditions, different optimal time window selection strategies are implemented to obtain the event set within the optimal time range; The step of constructing the event frame image using an adaptive thresholding method based on the obtained optimal time window and the compensated event set specifically includes the following steps: Construct a time graph within the current time window using the compensated set of events; The constructed time image is filtered for background events to obtain the filtered time image. The filtered temporal image is processed using a threshold method to obtain the event frame image. The step of iteratively fitting image moments to the constructed event frame image to obtain the final detection result specifically includes the following steps: Apply mean filtering to the event frame image to remove salt-and-pepper noise and obtain the filtered event frame image; The midpoint of the detection box in the time frame image after initialization and filtering is the position of the maximum timestamp of the event frame image; The final detection result is obtained by iterating from the position with the largest timestamp of the event frame, updating the points in the detection box and the size of the detection box; The step of executing different optimal time window selection strategies based on the acquired detection conditions to obtain the event set within the optimal time range specifically includes the following steps: When a moving object is detected in the previous time window, the optimal time window is selected based on the size of the previous time window, and the event set within the optimal time window is obtained. If no moving object is detected in the previous time window, the optimal time window is selected based on the size of the smallest time window, and the set of events within the optimal time window is obtained.

2. The object detection method based on adaptive time window and time threshold as described in claim 1, characterized in that, The step of eliminating a portion of the events generated by camera self-motion from the event set within the optimal time window using an adaptive compensation algorithm to obtain the compensated event set specifically includes the following steps: Perform rotational motion compensation on the event set within the optimal time window; Perform translational motion compensation on the event set after rotational motion compensation to obtain the compensated event set.

3. A moving object detection system based on adaptive time window and time threshold, characterized in that, include: The event camera data acquisition module is used to acquire the set of events within the optimal time window using an adaptive time window method. The event set acquisition module is communicatively connected to the event camera data acquisition module. It is used to eliminate a portion of the events generated by the self-motion of the event camera from the event set within the optimal time window through an adaptive compensation algorithm, and obtain the compensated event set. The event frame image acquisition module is communicatively connected to the event set acquisition module and is used to construct an event frame image using an adaptive threshold method based on the acquired optimal time window and the compensated event set. The detection result acquisition module is communicatively connected to the event frame image acquisition module and is used to perform iterative image moment fitting on the constructed event frame image to obtain the final detection result. The event camera data acquisition module includes: The time window preliminary estimation unit is used to make a preliminary estimation of the time window, taking the timestamp of the first event as the start time of the current time window. The detection condition acquisition unit is used to acquire the detection condition of moving objects within the previous time window; The event set acquisition unit is communicatively connected to the current time window acquisition unit and the detection condition acquisition unit, and is used to execute different optimal time window selection strategies based on the acquired detection conditions to acquire the event set within the optimal time range. The step of constructing the event frame image using an adaptive thresholding method based on the obtained optimal time window and the compensated event set specifically includes the following steps: Construct a time graph within the current time window using the compensated set of events; The constructed time image is filtered for background events to obtain the filtered time image. The filtered temporal image is processed using a threshold method to obtain the event frame image. The step of iteratively fitting image moments to the constructed event frame image to obtain the final detection result specifically includes the following steps: Apply mean filtering to the event frame image to remove salt-and-pepper noise and obtain the filtered event frame image; The midpoint of the detection box in the time frame image after initialization and filtering is the position of the maximum timestamp of the event frame image; The final detection result is obtained by iterating from the position with the largest timestamp of the event frame, updating the points in the detection box and the size of the detection box; The event set acquisition unit includes: The first event set acquisition sub-unit is used to select the optimal time window based on the size of the previous time window when a moving object is detected in the previous time window, and to acquire the event set within the optimal time window. The second event set acquisition sub-unit is used to select the optimal time window based on the size of the smallest time window when no moving object is detected in the previous time window, and to acquire the event set within the optimal time window.

4. The moving object detection system with adaptive time window and time threshold as described in claim 3, characterized in that, The event frame image acquisition module includes: The rotational motion compensation subunit is used to perform rotational motion compensation on the event set within the optimal time window. The translational motion compensation subunit is communicatively connected to the rotational motion compensation subunit and is used to perform translational motion compensation on the event set after rotational motion compensation to obtain the compensated event set.

Images