Depth information detection method and device of moving target, equipment and medium

Abstract

The invention provides a depth information detection method and device of a moving target, equipment and a medium, and belongs to the technical field of computer vision. The method comprises the following steps: acquiring an event feature matrix containing a target object in an initial region from different visual angles; performing evaluation to obtain a density number value of each discrete spatial position in each event feature matrix, and performing preliminary screening on the initial region according to the density number value to obtain a candidate region; performing evaluation to obtain a density area value representing the size of the target object in each candidate region, and performing secondary screening on the candidate regions according to the density area values to obtain a target region; performing matching processing on the event information in the corresponding target area under different viewing angles, and determining parallax of the event feature matrix of the target object under different viewing angles; and determining target depth information between the target object and the binocular event camera based on the parallax. According to the invention, the depth information detection accuracy of the moving target can be improved under the condition of limited computing resources.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a method, apparatus, device, and medium for detecting depth information of a moving target. Background Art

[0002] Depth information detection of moving targets refers to the process of determining the distance of moving objects in a scene relative to an observer (such as a camera) in computer vision technology. This process aims to calculate the three-dimensional position information of each moving target in a dynamic environment, especially the position along the viewing direction (usually called the Z axis), which is the so-called "depth".

[0003] In related technologies, event images captured by event cameras are converted into traditional images, and the depth of moving objects is then determined based on these traditional images. However, this method introduces unnecessary redundant computations, resulting in poor depth detection accuracy for moving objects when computing resources are limited. Summary of the Invention

[0004] The main purpose of the embodiments of the present application is to propose a method, device, equipment and medium for detecting depth information of a moving target, so as to improve the accuracy of depth information detection of a moving target under limited computing resources.

[0005] To achieve the above objectives, a first aspect of an embodiment of the present application provides a method for detecting depth information of a moving target, comprising:

[0006] Using a pre-calibrated binocular event camera, the event feature matrix containing the target object in the initial area is obtained from different perspectives;

[0007] For each event feature matrix, evaluate and obtain the density value of each discrete spatial location representing the spatial aggregation of event information, and preliminarily screen the initial region based on the density value to obtain the candidate region;

[0008] Evaluate and obtain the density area value representing the size of the target object in each candidate area, and perform a secondary screening of the candidate areas based on the density area value to obtain the target area under the current viewing angle;

[0009] Matching the event information in the corresponding target area under different viewing angles to determine the disparity of the event feature matrix of the target object under different viewing angles;

[0010] The target depth information between the target object and the binocular event camera is determined based on the disparity.

[0011] In some embodiments, a pre-calibrated binocular event camera is used to obtain event feature matrices containing target objects in the initial area from different perspectives, including:

[0012] Use the pre-calibrated binocular event camera to obtain the initial event feature matrix;

[0013] For each perspective, the number of event information corresponding to each discrete spatial position in the initial area is counted under the initial time window to obtain an event feature queue;

[0014] A mask matrix of the event feature queue is generated using a preset incremental array. If the number of non-zero eigenvalues in the mask matrix is within a preset first threshold range, the initial event feature matrix is used as the event feature matrix under the corresponding perspective.

[0015] In some embodiments, after generating the mask matrix of the event feature queue using the preset incremental array, the method further includes:

[0016] If the number of non-zero eigenvalues in the mask matrix is lower than a preset first threshold range, the initial time window is expanded to obtain an updated time window;

[0017] Re-acquire the event feature matrix within the initial region under the current perspective based on the updated time window;

[0018] If the number of non-zero eigenvalues in the mask matrix exceeds a preset first threshold range, adjusting the incremental array to obtain an updated incremental array;

[0019] The updated incremental array is used as a new preset incremental array, and the preset incremental array is used to continue processing the event feature queue until the event feature matrix in the initial area under the current viewing angle is obtained.

[0020] In some embodiments, for each event feature matrix, a density value of each discrete spatial location representing the spatial aggregation of event information is evaluated, and initial regions are preliminarily screened based on the density value to obtain candidate regions, including:

[0021] Based on the preset step size, each event feature matrix is downsampled to obtain a downsampled feature matrix;

[0022] For the downsampled feature matrix, the density value of the spatial aggregation of event information represented by each discrete spatial position is evaluated;

[0023] Based on the density value, multiple candidate coordinates are selected from the downsampled feature matrix, and each candidate coordinate is fused to obtain a fusion area;

[0024] The minimum circumscribed matrix of the fusion region is selected as the initial candidate region, and the initial candidate region is upsampled to obtain the candidate region after preliminary screening.

[0025] In some embodiments, the density area value representing the size of the target object in each candidate area is evaluated, and the candidate areas are secondary screened based on the density area value to obtain the target area under the current viewing angle, including:

[0026] Obtaining an expected size value of the target object, and determining a second threshold value based on the expected size value, a preset step size, and a preset conservative value;

[0027] The density area value representing the size of the target object in each fusion area is evaluated and obtained. If the density area value is greater than a second threshold, the corresponding fusion area is determined to be the target area under the current viewing angle.

[0028] In some embodiments, evaluating and obtaining density area values representing the size of the target object in each candidate region, and performing secondary screening on the candidate regions based on the density area values to obtain the target region under the current viewing angle further includes:

[0029] Calculate the size of the pixel area represented by the non-zero eigenvalue in the candidate area, and calculate the area ratio between the pixel area size and the candidate area to obtain the density area value represented by the size of the target object;

[0030] If the density area value is greater than a preset third threshold, the corresponding candidate area is determined to be the target area under the current viewing angle.

[0031] In some embodiments, matching processing is performed on event information in corresponding target areas at different viewing angles to determine the disparity of event feature matrices of the target object at different viewing angles, including:

[0032] Performing coordinate transformation and clustering processing on the event information in the event feature matrix under different viewing angles to obtain multiple center points determined under different viewing angles;

[0033] Calculate the similarity between the center points under different viewing angles, and determine the two most similar center points under different viewing angles as a pair of matching points representing the same target object;

[0034] The distance difference between the matching points at different viewing angles is calculated, and the disparity of the event feature matrix of the target object at different viewing angles is determined based on the distance difference.

[0035] In some embodiments, determining target depth information between a target object and a binocular event camera based on disparity includes:

[0036] Get the average focal length and baseline length of the binocular event camera after calibration;

[0037] The target depth information between the target object and the binocular event camera is determined according to the ratio of the average focal length, the baseline length and the parallax.

[0038] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a device for detecting depth information of a moving object, comprising:

[0039] The acquisition module is used to obtain the event feature matrix containing the target object in the initial area from different perspectives using a pre-calibrated binocular event camera;

[0040] The first screening module is used to evaluate each event feature matrix to obtain the density value of the spatial aggregation of event information at each discrete spatial location, and preliminarily screen the initial area based on the density value to obtain the candidate area;

[0041] The second screening module is used to evaluate and obtain the density area value representing the size of the target object in each candidate area, and perform a secondary screening of the candidate areas based on the density area value to obtain the target area under the current viewing angle;

[0042] A disparity determination module is used to match event information in corresponding target areas under different viewing angles and determine the disparity of the event feature matrix of the target object under different viewing angles;

[0043] The target depth information determination module is used to determine the target depth information between the target object and the binocular event camera based on parallax.

[0044] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it implements the depth information detection method of the moving target of the first aspect mentioned above.

[0045] To achieve the above-mentioned purpose, the fourth aspect of the embodiments of the present application proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the depth information detection method of the moving target in the first aspect mentioned above.

[0046] The present application proposes a method, apparatus, device, and medium for detecting the depth information of a moving target. The method utilizes a pre-calibrated binocular event camera to obtain an event feature matrix containing a target object in an initial region from different perspectives. For each event feature matrix, the method evaluates the density of each discrete spatial location representing the spatial aggregation of event information, and preliminarily screens the initial region based on the density value to obtain candidate regions. The method evaluates the density of each candidate region representing the size of the target object, and performs a secondary screening of the candidate regions based on the density value to obtain the target region at the current perspective. The method matches the event information in the corresponding target region at different perspectives to determine the disparity of the event feature matrix of the target object at different perspectives. The method then determines the target depth information between the target object and the binocular event camera based on the disparity. By utilizing a multi-level screening mechanism, the present application improves the accuracy of target object depth information detection while reducing the waste of computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is an optional flowchart of the depth information detection method for a moving target provided in an embodiment of the present application;

[0048] Figure 2 yes Figure 1 An optional implementation flowchart of step 101;

[0049] Figure 3 yes Figure 2 An optional implementation flowchart after step (101.3);

[0050] Figure 4 yes Figure 1 An optional implementation flowchart of step 102;

[0051] Figure 5 yes Figure 1 An optional implementation flowchart of step 103;

[0052] Figure 6 yes Figure 1 Another optional implementation flow chart of step 103;

[0053] Figure 7 yes Figure 1 An optional implementation flowchart of step 104;

[0054] Figure 8 This is an optional mapping processing diagram of the depth information detection method for a moving target provided in an embodiment of the present application;

[0055] Figure 9 yes Figure 1 An optional implementation flowchart of step 105;

[0056] Figure 10 This is an optional data processing diagram of the depth information detection method for a moving target provided in an embodiment of the present application;

[0057] Figure 11 This is a schematic diagram of an optional module of the depth information detection device for a moving target provided in an embodiment of the present application;

[0058] Figure 12 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0060] It should be noted that although the device schematics illustrate functional module divisions and the flowcharts illustrate logical sequences, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the device or the sequence in the flowcharts. The terms "first," "second," and so on, in the specification, claims, and drawings, are used to distinguish similar items and are not necessarily used to describe a specific sequence or precedence.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0062] Depth information detection of moving targets refers to the process of determining the distance of moving objects in a scene relative to an observer (such as a camera) in computer vision technology. This process aims to calculate the three-dimensional position information of each moving target in a dynamic environment, especially the position along the viewing direction (usually called the Z axis), which is the so-called "depth".

[0063] In related technologies, event images captured by event cameras are converted into traditional images, and the depth of moving objects is then determined based on these traditional images. However, this method introduces unnecessary redundant computations, resulting in poor depth detection accuracy for moving objects when computing resources are limited.

[0064] Based on this, the embodiments of the present application provide a method, apparatus, device and medium for detecting depth information of a moving target, aiming to improve the accuracy of depth information detection of a moving target under limited computing resources.

[0065] In an optional application scenario, the depth information detection device for a moving target provided in an embodiment of the present application (for ease of description, it may also be referred to as a "detection device") can be set in a server, wherein a binocular event camera is also provided in the server, and the detection device performs the following steps: using a pre-calibrated binocular event camera, an event feature matrix containing a target object in an initial area is obtained from different perspectives; for each event feature matrix, the density quantity value representing the spatial aggregation of event information at each discrete spatial position is evaluated, and the initial area is preliminarily screened according to the density quantity value to obtain a candidate area; the density area value representing the size of the target object in each candidate area is evaluated, and the candidate area is secondary screened according to the density area value to obtain the target area under the current perspective; the event information in the corresponding target area under different perspectives is matched and processed to determine the disparity of the event feature matrix of the target object under different perspectives; and the target depth information between the target object and the binocular event camera is determined based on the disparity.

[0066] It can be understood that the detection device using the depth information detection method for moving targets proposed in the embodiments of the present application improves the accuracy of depth information detection of target objects by using a multi-level screening mechanism, while also reducing the waste of computing resources, thereby achieving high-precision depth estimation between the moving target and the binocular event camera, and thus providing strong technical support for applications such as autonomous driving, robot navigation and augmented reality in actual use.

[0067] It should be noted that, in the embodiments of the present application, when it comes to information related to user characteristics such as user basic information or user identity, the user's permission and / or consent will be obtained first, and the collection, use and processing of these data will comply with relevant laws, regulations and standards. In addition, when the embodiments of the present application need to obtain the user's sensitive personal information, the user's separate permission or separate consent will be obtained first. After clearly obtaining the user's separate permission or separate consent, the necessary data for enabling the normal operation of the embodiments of the present application will be obtained. For example, before the embodiment of the present application utilizes a binocular event camera to obtain the event feature matrix of the target object, the consent of the corresponding management personnel will be obtained first, otherwise the event feature matrix of the target object that cannot be applied to the embodiment of the present application will be obtained. In addition, the other relevant data obtained by the detection device of the embodiment of the present application are all authorized data after obtaining the consent of the relevant personnel, which will not be repeated here.

[0068] In the embodiment of the present application, the description will be made from the perspective of a detection device, which can be integrated into a computer device, such as a server. Figure 1 As shown, Figure 1 This is an optional flowchart of the depth information detection method for a moving target provided in an embodiment of the present application. Figure 1 The method may include but is not limited to the following steps 101 to 105. When the depth information detection device of a moving target executes the depth information detection method of a moving target, the specific process is as follows. It should be noted that the present embodiment Figure 1 The order of step 101 to step 105 is not specifically limited, and the order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0069] Step 101: Using a pre-calibrated binocular event camera, obtain an event feature matrix containing a target object in an initial area from different viewing angles.

[0070] Step 101 is described in detail below.

[0071] A binocular event camera is a visual sensor system consisting of two synchronized event cameras, similar to the human eye's binocular vision. Unlike traditional cameras, which capture a complete image frame, each event camera independently responds to changes in light intensity in the scene (e.g., object motion, light flickering) and asynchronously outputs a stream of pixel-level events. These events contain the time, location, and polarity (brightening or dimming) of the change. It should be noted that binocular event cameras can be either left-right or top-down, and the specific configuration can be tailored to the specific situation.

[0072] The perspective refers to the position and orientation of the binocular event camera relative to the moving target being photographed. Obtaining event feature matrices containing the target object from different perspectives provides the information necessary for depth perception and target positioning. Similar to human binocular vision, the event feature matrices from two different perspectives reflect the difference (parallax) in the target's appearance on the imaging planes of the left and right cameras. By analyzing and comparing the relative position and event distribution of the target in these two matrices, the parallax of the moving target can be calculated, its depth information can be estimated, and the target's three-dimensional positioning can be achieved.

[0073] Among them, the initial area refers to the specific area containing the target object in the scene captured by the binocular event camera after certain screening and processing. The target object (moving target) refers to any object moving in the initial area, and the number of target objects is at least one, that is, when the initial area includes multiple target objects, the detection device proposed in the embodiment of the present application can detect and obtain the depth information corresponding to each target object respectively. In addition, the target object can be determined according to the specific task applied by the detection device. For example, in the application scenario of automatic driving of a car, the target object can be a person, a vehicle, an animal, etc.

[0074] Furthermore, "pre-calibrated" means that the binocular event camera has completed the calibration process. After calibration, the binocular event camera can ensure that the output event information accurately reflects the brightness changes of the target object in the actual physical space. Calibration generally involves determining the internal parameters of the camera (such as focal length, principal point position, and distortion coefficient, etc.) and, in some cases, external parameters (such as the position and orientation of the camera relative to a known coordinate system). The specific calibration parameters can be adaptively adjusted according to actual conditions, and the embodiments of the present application do not impose any restrictions on this.

[0075] Furthermore, this application takes the binocular event camera, which is a top-bottom event camera, as an example to further explain:

[0076] Using a set of synchronization signals to connect two event cameras, they can synchronize their acquisition. Unlike traditional cameras, the output of an event camera is continuous event information {t, x, y, p}, where t is the timestamp of the event, (x, y) is the horizontal and vertical coordinates of the event information (event), and p∈{0,1} is the polarity of the event.

[0077] By performing binocular positioning on two synchronized event cameras, the pixel coordinate systems of the two event cameras are aligned, and the coordinate mapping tables of the two event cameras before and after alignment are constructed, as shown in formulas (1) and (2). L and Source R is the pixel coordinate system where the binocular event camera is located, and Target is the pixel coordinate system after the binocular event camera is aligned. and The preset coordinate mapping tables are used to map the horizontal and vertical coordinates of the two event cameras to the Target coordinate system.

[0078]

[0079] Furthermore, upon completing the binocular event camera calibration, the average focal length f and the baseline length b between the event cameras are obtained. After calibration, the binocular event camera directly uses floating-point coordinates to represent event data, avoiding the traditional image processing step of re-interpolation to form spatially aligned pixels. This improves data processing efficiency, preserves the high temporal and spatial resolution of events, and reduces errors introduced by interpolation, resulting in more accurate depth recognition of target objects.

[0080] In some embodiments, as Figure 2 As shown, Figure 2 yes Figure 1 Step 101 in the flowchart is an optional implementation, using a pre-calibrated binocular event camera to obtain an event feature matrix containing a target object in the initial area from different perspectives, including the following steps:

[0081] (101.1) Using a pre-calibrated binocular event camera, obtain an initial event feature matrix;

[0082] (101.2) For each perspective, the number of event information corresponding to each discrete spatial position in the initial area is counted under the initial time window to obtain an event feature queue;

[0083] (101.3) Generate a mask matrix of the event feature queue using a preset incremental array. If the number of non-zero eigenvalues in the mask matrix is within a preset first threshold range, use the initial event feature matrix as the event feature matrix under the corresponding perspective.

[0084] Steps (101.1) to (101.3) are described in detail below.

[0085] In some embodiments, the event feature matrix F corresponding to the current moment is generated by the following method: i :

[0086] Input: Event feature queue List = {α i-L+1 ,α i-L+2 ,…α i}、Preset increment array th_array=[th min ,…,th max ], the first threshold range (including the minimum lower bound mask min and the maximum upper bound mask max );

[0087] Output: event feature matrix F i ;

[0088] Initial event feature matrix F i =0;

[0089]

[0090] Specifically, for a binocular event camera with a resolution of H×W, within the Δt time window, the number of events received at each discrete spatial position (pixel position) is counted (without distinguishing the polarity of the event), and an H×W event number statistical matrix α is generated. i ; Then, each event is stored in the event feature queue List of length L in turn = {α i-L+1 ,α i-L+2 ,…α i}; Then, the mask matrix mask of the event feature queue is generated using the preset increment array. If the number of non-zero eigenvalues in the mask matrix mask is within the preset first threshold range [mask min , mask max], the initial event feature matrix is used as the event feature matrix under the corresponding perspective. Otherwise, the event feature matrix is retrieved by the following steps:

[0091] In some embodiments, as Figure 3 As shown, Figure 3 yes Figure 2 An optional implementation flowchart after step (101.3) in the embodiment further includes the following steps after generating the mask matrix of the event feature queue using the preset incrementing array:

[0092] (A.1) If the number of non-zero eigenvalues in the mask matrix is lower than a preset first threshold range, expanding the initial time window to obtain an updated time window;

[0093] (A.2) Re-acquire the event feature matrix within the initial region under the current viewing angle based on the updated time window;

[0094] (A.3) if the number of non-zero eigenvalues in the mask matrix exceeds a preset first threshold range, adjusting the incremental array to obtain an updated incremental array;

[0095] (A.4) The updated incremental array is used as a new preset incremental array, and the preset incremental array is used to continue processing the event feature queue until the event feature matrix in the initial area under the current viewing angle is obtained.

[0096] Steps (A.1) to (A.4) are described in detail below.

[0097] In some embodiments, such as the embodiments given in steps (101.1) to (101.3), when the number of non-zero eigenvalues in the mask matrix mask.sum() is lower than (or does not exceed) the minimum lower bound mask min When , the initial time window is expanded to obtain the updated time window; and based on the updated time window, the event feature matrix in the initial area under the current perspective is re-obtained:

[0098] else if mask.sum()≤mask min :

[0099] break

[0100] Exemplarily, in the mask matrix generated by the initial time window (1 millisecond), the number of non-zero eigenvalues is lower than the preset first threshold range (500 to 2000 pixels), which means that the number of events is too small and the time window needs to be expanded; therefore, step (A.1) is executed to expand the initial time window from 1 millisecond to 2 milliseconds to obtain an updated time window; then step (A.2) is executed to re-count the number of events in the initial area under the current viewing angle based on the updated time window of 2 milliseconds, thereby obtaining a new event feature matrix.

[0101] Furthermore, in the embodiment given in steps (101.1) to (101.3), when the number of non-zero eigenvalues in the mask matrix mask.sum() exceeds (or is not less than) the maximum upper bound mask max , adjust the incrementing array to obtain the updated incrementing array; and use the updated incrementing array as the new preset incrementing array, and continue to process the event feature queue using the preset incrementing array until the event feature matrix in the initial area under the current viewing angle is obtained:

[0102] else if mask.sum()≥mask max :

[0103] continue

[0104] Exemplarily, in the initial time window, the number of non-zero eigenvalues in the mask matrix exceeds the preset maximum upper limit, for example, exceeds 2000 pixels, which means that the number of events is too large, and the threshold needs to be adjusted and step (A.3) is executed to adjust the preset incremental threshold array [5, 10, 15, 20] to [10, 15, 20, 25] to obtain an updated incremental array; then return to step (101.3) and use the updated incremental array to continue processing the event feature queue until the number of non-zero eigenvalues in the generated mask matrix falls within the range of 500 to 2000. The event feature matrix obtained at this time is the event feature matrix in the initial area under the current perspective.

[0105] Step 102 : For each event feature matrix, evaluate and obtain the density value of each discrete spatial position representing the spatial aggregation of event information, and preliminarily screen the initial region based on the density value to obtain the candidate region.

[0106] Step 102 is described in detail below.

[0107] In some embodiments, when the time window Δt is small enough, events are often sparse in space. In this case, the event feature matrix F iIt is also relatively sparse, which is not conducive to solving depth information. In addition, the event feature matrix covers most or all of the field of view of the binocular event camera, so it is large in size and contains other unnecessary events besides the target object. Compared with the traditional method of directly processing the event feature matrix obtained in step 101, the embodiment of the present application evaluates the "density quantity value" of each discrete spatial position, whose value represents the degree of event aggregation at the position, and quickly reduces the event feature matrix obtained in the initial area to a limited number of activity-intensive areas based on the density quantity value, thereby greatly reducing the data and spatial range that need to be processed in subsequent steps, significantly reducing the overall computational burden of the algorithm, and being able to achieve high-precision depth information detection of real-time target objects under limited computing resources.

[0108] In some embodiments, as Figure 4 As shown, Figure 4 yes Figure 1 Step 102 in the flowchart is an optional implementation flow chart, which evaluates each event feature matrix to obtain the density value of each discrete spatial location representing the spatial aggregation of event information, and preliminarily screens the initial region based on the density value to obtain the candidate region, including the following steps:

[0109] (102.1) Based on a preset step size, downsampling is performed on each event feature matrix to obtain a downsampled feature matrix;

[0110] (102.2) For the downsampled feature matrix, evaluate and obtain the density value of the spatial aggregation of event information at each discrete spatial location;

[0111] (102.3) Based on the density value, multiple candidate coordinates are selected from the downsampled feature matrix, and each candidate coordinate is fused to obtain a fused region;

[0112] (102.4) The minimum circumscribed matrix of the fusion region is selected as the initial candidate region, and the initial candidate region is upsampled to obtain the candidate region after preliminary screening.

[0113] Steps (102.1) to (102.4) are described in detail below.

[0114] In some embodiments, F i Perform downsampling processing to S H ×S W The eigenvalues in the initial region are summed with the preset step size, as shown in formula (3), to generate a smaller down-sampled feature matrix F′ i , while downsampling can also achieve the effect of smoothing features, which facilitates the subsequent solution of the depth information of the target object:

[0115]

[0116] Furthermore, the downsampled feature matrix F′ i Gaussian kernel density estimation is used, and the cumulative value of events in the 3×3 neighborhood of each discrete spatial position is taken as the density quantity value; assuming the preset density threshold is T=15, when the density quantity value of a discrete spatial position exceeds T (such as the area where the target object is located can reach 25-30), the coordinates involved are marked as valid candidate coordinates.

[0117] It should be noted that the specific method for evaluating the downsampled feature matrix to obtain the density quantity value is not limited to the Gaussian kernel density estimation method, and can be adaptively adjusted according to actual conditions. The embodiments of the present application do not impose any restrictions on this.

[0118] Furthermore, the K candidate coordinates with the largest eigenvalues are selected, and for each candidate coordinate, the connected domain labeling method is used to fuse the coordinates of the adjacent (four-neighborhood or eight-neighborhood) candidate coordinates to obtain K′ (K′≤K) fusion regions {C1, C2, … C K′}; Then, take the minimum bounding rectangle of each fusion area and get F i ′’s K′ initial candidate regions {S1′,S2′,…S′ K′}(C k′ ∈S′ k′ ); Then, the initial candidate area is up-sampled or simply mapped to the coordinates to obtain the moving target in F i The candidate region obtained after the preliminary screening is denoted as

[0119] Step 103 : Evaluate and obtain the density area value representing the size of the target object in each candidate area, and perform a secondary screening on the candidate areas according to the density area value to obtain the target area under the current viewing angle.

[0120] Step 103 is described in detail below.

[0121] In some embodiments, after completing the preliminary screening, a secondary screening process is performed by using the density area values of the evaluated candidate areas to effectively remove noise or areas whose sizes do not meet the expected targets, further reducing resource waste, improving the detection accuracy of target objects and the reliability of subsequent depth information estimation.

[0122] In some embodiments, as Figure 5 As shown, Figure 5 yes Figure 1Step 103 in the flowchart is an optional implementation, which evaluates and obtains the density area value representing the size of the target object in each candidate area, and performs a secondary screening of the candidate areas based on the density area value to obtain the target area under the current viewing angle, including the following steps:

[0123] (103.1) Obtaining an expected size of the target object, and determining a second threshold based on the expected size, a preset step size, and a preset conservative value;

[0124] (103.2) Evaluate and obtain a density area value representing the size of the target object in each fusion area. If the density area value is greater than a second threshold, determine that the corresponding fusion area is the target area under the current viewing angle.

[0125] The following describes steps (103.1) to (103.2) in detail.

[0126] In some embodiments, the candidate regions include not only the region where the moving target is located, but also other noisy regions (such as black background regions and low-light regions). Based on this, the embodiment of the present application further filters out the region where the moving target is located through the following steps:

[0127] Specifically, each fusion region C is evaluated k′ The number of coordinates n included k′ And the number of coordinates is used as the density area value to pass n k′ The size of the initial small noise area is excluded. For example, assuming that the minimum moving target to be detected is i The size of the area on m ×W m (expected size value), theoretically the target corresponds to F′ i The size of the Considering the F i Downsample to F′ i The process of n may cause the actual calculated area to become smaller, so determining n k′ The corresponding second threshold is Among them, S H 、S W is the preset step size, and 1 is the preset conservative value.

[0128] when When the fusion area C k′ The corresponding area S′ k′ will be judged as a noise area; on the contrary, when When the fusion area C k′ The corresponding area S′ k′ , the fused area will be determined as the target area under the current viewing angle, thus realizing the secondary screening process of the candidate area.

[0129] In some embodiments, as Figure 6 As shown, Figure 6 yes Figure 1 Another optional implementation flowchart of step 103 in the embodiment of the present invention includes evaluating the density area value representing the size of the target object in each candidate area, and performing a secondary screening of the candidate areas based on the density area value to obtain the target area under the current viewing angle, further comprising the following steps:

[0130] (B.1) Calculate the size of the pixel region represented by non-zero eigenvalues within the candidate region, and calculate the area ratio between the pixel region size and the candidate region to obtain a density area value representing the size of the target object;

[0131] (B.2) If the density area value is greater than a preset third threshold, the corresponding candidate area is determined to be the target area under the current viewing angle.

[0132] Steps (B.1) to (B.2) are described in detail below.

[0133] In some embodiments, in addition to performing secondary screening to obtain the target area according to steps (103.1) to (103.2), the target area can also be obtained by calculating F i middle The number of non-zero elements in the region and The ratio of the areas r k′ As the density area value, through r k′ Excluding the noise area, r k′ The determination process of is shown in formula (4). Specifically, when the moving target is in a certain area, the event coordinates in the area are dense and concentrated, while the event coordinates in the noise area are sparse and uniform, which leads to the r k′ The value is large, and the r in the noise area k′ is relatively small. k′ By setting the third threshold, the area where the moving target is located can be filtered out.

[0134]

[0135] In this way, the detection device can obtain the moving target in F i The approximate position on the ground is obtained by setting a smaller Δt (unit: microsecond) to achieve the moving target. Time resolution detection.

[0136] It should be noted that the specific numerical value of the third threshold can be set according to actual conditions, and the embodiments of the present application do not limit this.

[0137] Step 104 : performing matching processing on the event information in the corresponding target area at different viewing angles to determine the disparity of the event feature matrix of the target object at different viewing angles.

[0138] Step 104 is described in detail below.

[0139] In some embodiments, after completing two screening processes, the event information in the corresponding target area under different perspectives is matched to calculate the disparity of the target object under different perspectives, so as to accurately estimate the depth information of the moving target based on the disparity.

[0140] In some embodiments, as Figure 7 As shown, Figure 7 yes Figure 1 Step 104 in the flowchart is an optional implementation, which matches the event information in the corresponding target area under different viewing angles to determine the disparity of the event feature matrix of the target object under different viewing angles, including the following steps:

[0141] (104.1) performing coordinate transformation and clustering processing on event information in the event feature matrix at different viewing angles to obtain multiple center points determined at different viewing angles;

[0142] (104.2) Calculate the similarity between the center points under different viewing angles, and determine the two most similar center points under different viewing angles as a pair of matching points representing the same target object;

[0143] (104.3) Calculate the distance difference between the matching points at different viewing angles, and determine the disparity of the event feature matrix of the target object at different viewing angles based on the distance difference.

[0144] Steps (104.1) to (104.3) are described in detail below.

[0145] The embodiment of this application is in F i The above calculation results in K′ regions, corresponding to K′ moving targets; the statistics are in F i The location information of the non-zero eigenvalues in the K′ region is expressed as D={A 1 ,A 2 ,…A K′}, where A k ={X k ,Y k}, X k Represents the set of horizontal coordinates, Y k In the binocular event camera, the position information of K′ moving targets are represented as and

[0146] Furthermore, through the mapping table in step 101 and The location information A of each moving target area k′ Mapped to the same pixel coordinate system, the new position information is represented as A′ k′ , the process is as follows:

[0147]

[0148] As shown in Figure 8, Figure 8 This is an optional mapping processing diagram of the depth information detection method of the moving target provided by the embodiment of the present application. The dotted box on the left is the area D where the moving target is located, and the dots in the area D represent all the uncorrected event coordinates of the moving target (such as A 1 ), by looking up the mapping table, these event coordinates find the corresponding corrected coordinates (such as A' 1 ),like Figure 8 As shown in the right area D'.

[0149] Furthermore, based on the fact that the coordinate distribution of the same moving target outputted by the binocular event camera in the same time interval is very similar, a pairwise matching process is performed on multiple moving targets of the two event cameras. Specifically, for the moving targets detected by the two cameras, and First, and Convert to the same coordinate origin through formula (5) to generate new coordinates and Where mean(·) represents the averaging operation:

[0150]

[0151] Further, and As two two-dimensional discrete distributions and The two discrete distributions are clustered. The clustering method can be the mean shift method (MeanShift method) or other clustering methods. The specific method can be set according to the actual situation. The embodiment of the present application does not limit this. Then, u center points are obtained respectively. and v centers and It is the feature point with the highest event density for the same moving target in different event cameras. In theory, the two have similar spatial distributions.

[0152] Among them, due to and There are many events in the data, and the similarity calculation between the two is relatively cumbersome. In order to reduce the amount of calculation, we use and Calculation of Wasserstein distance between two distributions and The similarity of is shown in formulas (6) and (7), where Π is and The set of all possible joint distributions, x and y represent and Coordinates in:

[0153]

[0154] Using formula (7), calculate D′ one by one L and D′ R The similarity between the two moving targets in the CNN is calculated, and the two targets with the largest values are selected as the same moving target, so that the matching between K′ moving targets can be achieved.

[0155] Furthermore, for the matched moving target and That is, the u center points corresponding to them and v centers Since both cameras have already been calibrated and corrected for distortion, the parallax of the two targets can be calculated by simply calculating the coordinate difference between them. When the binocular cameras are placed horizontally, the parallax of the two moving targets is the difference in their horizontal coordinates. When placed vertically, the parallax of the two moving targets is the difference in their vertical coordinates.

[0156] Since the corrected event coordinates are floating-point numbers that are sparsely arranged in space, the stereo matching algorithm based on traditional images is no longer applicable. Based on this, the embodiment of the present application also designs a sub-pixel disparity calculation method based on a clustering method, which is designed as follows:

[0157] <1> When the binocular event camera is placed horizontally, calculate and The difference between the vertical coordinates of any two center points is less than the specified threshold T error A pair of center points are defined as "matching points"; similarly, when the binocular event camera is placed vertically, the difference between the horizontal coordinates of any two center points is calculated in sequence, and the difference is taken as the value less than the specified threshold T error A pair of center points of are defined as “matching points”.

[0158] <2> For each pair of “matching points” of the same moving target, the coherent point drift algorithm (CPD) is used to calculate the translation distance of all event coordinates within the area where the matching points are located.

[0159] ①For each pair of matching points, take the coordinates of all events within the coordinate neighborhood r to form two point sets M and N;

[0160] ② Since M and N have already undergone preliminary distortion correction and epipolar correction, ideally, the transformation between them can be considered a translation transformation. However, since event camera calibration is difficult to achieve, there must be a certain rotation transformation between the two. Therefore, in reality, the transformation between the two can be considered a rigid transformation. Assuming that the transformation parameters are the rotation matrix R and the translation matrix T, the CPD algorithm is used to calculate the translation matrix between M and N.

[0161] ③ Calculate the translation matrix between all matching points of the moving target in sequence. Take the average of all translation matrices and get It is used to represent the relative disparity of the moving target in two cameras, where d x is the horizontal average relative disparity, d y is the vertical average relative disparity. The disparity calculated at this time is the relative disparity obtained after the transformation of formula (6), so the absolute disparity between the two is:

[0162]

[0163] <3> When the binocular event cameras are placed horizontally, the horizontal average disparity is used as the disparity between the two cameras; when the binocular event cameras are placed vertically, the vertical average disparity is used as the disparity between the two cameras. This achieves a sub-pixel displacement calculation method based on a clustering method.

[0164] Step 105 : Determine target depth information between the target object and the binocular event camera based on the disparity.

[0165] Step 105 is described in detail below.

[0166] In some embodiments, through step-by-step screening (based on density quantity values and density area values) and cross-perspective matching, the moving target is accurately located and the disparity is calculated, ultimately achieving efficient and robust acquisition of moving target depth information, which is particularly suitable for high-speed moving target depth information detection scenarios with limited computing resources.

[0167] In some embodiments, as Figure 9 As shown, Figure 9 yes Figure 1 An optional implementation flowchart of step 105 in FIG. 1 is for determining target depth information between a target object and a binocular event camera based on disparity, including the following steps:

[0168] (105.1) Obtain the average focal length and baseline length of the binocular event camera after calibration;

[0169] (105.2) Determine target depth information between the target object and the binocular event camera based on the ratio of the average focal length, the baseline length, and the parallax.

[0170] The following describes steps (105.1) to (105.2) in detail.

[0171] In some embodiments, after obtaining the disparities of all moving objects, the disparities of K′ moving objects are recorded as {d1, d1, …d K′}, then for the i-th disparity d i The moving target, its depth distance i It can be calculated by formula (8), where f and b are calibrated in step 101:

[0172]

[0173] like Figure 10 As shown, Figure 10 This is an optional data processing diagram of the depth information detection method of the moving target provided by the embodiment of the present application. In order to better understand the beneficial effects brought by the embodiment of the present application, the following will be combined with Figure 10 The complete example is shown below. The following steps can be found in the manual for more detailed instructions:

[0174] Step (a): binocular event camera information collection:

[0175] Use pre-calibrated upper and lower binocular event cameras (in actual application, it can also be a binocular event camera, this is just an example) to capture the motion scene simultaneously; Figure 10 The figure shows that two event cameras ("upper eye" and "lower eye") respectively record events (red / blue dots) of brightness changes in their fields of view, forming their own corresponding event feature queues (List). Among them, the green grid represents the pixel space of each binocular event camera.

[0176] Step (b), calculation of the area where the moving target is located:

[0177] Perform preliminary screening on the acquired event feature queue to obtain the candidate area where the moving target is located, such as Figure 10 The event density distribution corresponding to each discrete spatial location is represented by a three-dimensional column. The obvious peaks include "object 1 (obj1)" and "object 2 (obj2)" in the upper column, and "object 3 (obj3)" and "object 4 (obj4)" in the lower column, which represent the areas with the most concentrated event activities, that is, potential "moving target areas".

[0178] Step (c): High-speed detection of moving targets:

[0179] The candidate areas are screened twice to further narrow the range of the moving target and obtain the target area, which facilitates the subsequent determination of the depth information of the moving target based on the target area with accurate range and key coordinate information.

[0180] Step (d), event coordinate distortion correction and moving target matching:

[0181] Apply the pre-calibrated distortion and epipolar correction transformation to the event coordinates extracted in step (c) to obtain undistorted and aligned floating-point coordinates. Perform feature point extraction on the target area and match the target objects in the target area under different viewing angles based on the extracted feature points. Figure 10 In the above example, obj1 and obj3 are represented as the same target object, and obj2 and obj4 are represented as the same target object.

[0182] Step (e), moving target disparity calculation and depth estimation:

[0183] For a set of matched target objects (such as obj1 and obj3), multiple corresponding local regions or feature points are found inside them (such as patch a corresponds to patch a1, patch b corresponds to patch b1, etc.). Then, the CPD algorithm is used to calculate the precise relative translation vector T between each pair of matching patches to determine the disparity of the event feature matrix of the target object at different viewing angles. Next, the geometric relationship between the binocular event camera is used to calculate the distance between the target object and the camera to obtain depth information. Figure 10 The depth calculation formula is given in: d_obj1 = (T1 + T2 + T3 + T4 + T5) / 5, depth_obj1 = focal * baseline / d_obj1, where focal represents the focal length, baseline represents the baseline distance, and d_obj1 represents the parallax.

[0184] like Figure 11 As shown, Figure 11 201 is an optional module diagram of a depth information detection device for a moving target provided in an embodiment of the present application. The depth information detection device for a moving target includes the following modules 201 to 205:

[0185] An acquisition module 201 is configured to acquire an event feature matrix containing a target object in an initial area from different viewing angles using a pre-calibrated binocular event camera;

[0186] The first screening module 202 is used to evaluate, for each event feature matrix, the density value of each discrete spatial location representing the spatial aggregation of event information, and preliminarily screen the initial region based on the density value to obtain the candidate region;

[0187] The second screening module 203 is used to evaluate and obtain the density area value representing the size of the target object in each candidate area, and perform a secondary screening of the candidate areas based on the density area value to obtain the target area under the current viewing angle;

[0188] The disparity determination module 204 is used to match the event information in the corresponding target area under different viewing angles to determine the disparity of the event feature matrix of the target object under different viewing angles;

[0189] The target depth information determining module 205 is configured to determine target depth information between the target object and the binocular event camera based on parallax.

[0190] The present application proposes a method, apparatus, device, and medium for detecting the depth information of a moving target. The method utilizes a pre-calibrated binocular event camera to obtain an event feature matrix containing a target object in an initial region from different perspectives. For each event feature matrix, the method evaluates the density of each discrete spatial location representing the spatial aggregation of event information, and preliminarily screens the initial region based on the density value to obtain candidate regions. The method evaluates the density of each candidate region representing the size of the target object, and performs a secondary screening of the candidate regions based on the density value to obtain the target region at the current perspective. The method matches the event information in the corresponding target region at different perspectives to determine the disparity of the event feature matrix of the target object at different perspectives. The method then determines the target depth information between the target object and the binocular event camera based on the disparity. By utilizing a multi-level screening mechanism, the present application improves the accuracy of target object depth information detection while reducing the waste of computing resources.

[0191] The specific implementation of the moving target depth information detection device is substantially the same as the specific embodiment of the above-mentioned moving target depth information detection method, and will not be described in detail here.

[0192] In addition, embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results.

[0193] Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0194] The depth information detection method of a moving target provided in the embodiment of the present application relates to the field of artificial intelligence technology. The depth information detection method of a moving target provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or can be configured as a server cluster or a distributed system composed of multiple physical servers, or can be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application of the depth information detection method for a moving target, etc., but is not limited to the above forms.

[0195] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0196] The present application also provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-mentioned method for detecting depth information of a moving target. The electronic device can be any smart terminal, such as a tablet computer or an in-vehicle computer.

[0197] like Figure 12 As shown, Figure 12 : is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application, the electronic device includes:

[0198] The processor 301 may be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0199] The memory 302 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 302 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 302 and is called by the processor 301 to execute the depth information detection method for a moving target in the embodiments of this application.

[0200] Input / output interface 303, used to implement information input and output;

[0201] Communication interface 304, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0202] bus 305 , which transmits information between the various components of the device (e.g., processor 301 , memory 302 , input / output interface 303 , and communication interface 304 );

[0203] The processor 301 , the memory 302 , the input / output interface 303 and the communication interface 304 are connected to each other in communication within the device via the bus 305 .

[0204] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the method for detecting depth information of a moving target is implemented.

[0205] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0206] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0207] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0208] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0209] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0210] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0211] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0212] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0213] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0214] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0215] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0216] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A method for detecting depth information of a moving target, characterized in that: include: Using a pre-calibrated binocular event camera, the event feature matrix containing the target object in the initial area is obtained from different perspectives; For each of the event feature matrices, evaluating and obtaining density values of spatial aggregation of event information at each discrete spatial position, and preliminarily screening the initial regions based on the density values to obtain candidate regions; Evaluate and obtain a density area value representing the size of the target object in each candidate area, and perform a secondary screening of the candidate areas based on the density area value to obtain the target area under the current viewing angle; performing matching processing on the event information corresponding to the target area at different viewing angles to determine the disparity of the event feature matrix of the target object at different viewing angles; Target depth information between the target object and the binocular event camera is determined based on the disparity.

2. The method for detecting depth information of a moving object according to claim 1, wherein: The method uses a pre-calibrated binocular event camera to obtain an event feature matrix containing a target object in an initial area from different perspectives, including: Use the pre-calibrated binocular event camera to obtain the initial event feature matrix; For each viewing angle, the number of event information corresponding to each discrete spatial position in the initial area is counted in the initial time window to obtain an event feature queue; A mask matrix of the event feature queue is generated using a preset incremental array. If the number of non-zero eigenvalues in the mask matrix is within a preset first threshold range, the initial event feature matrix is used as the event feature matrix under the corresponding perspective.

3. The method for detecting depth information of a moving object according to claim 2, wherein: After generating the mask matrix of the event feature queue by using the preset incremental array, the method further includes: If the number of non-zero eigenvalues in the mask matrix is lower than a preset first threshold range, expanding the initial time window to obtain an updated time window; Reacquiring the event feature matrix within the initial area under the current viewing angle based on the update time window; If the number of non-zero eigenvalues in the mask matrix exceeds a preset first threshold range, adjusting the incremental array to obtain an updated incremental array; The updated incremental array is used as a new preset incremental array, and the event feature queue is continuously processed using the preset incremental array until an event feature matrix within the initial area under the current viewing angle is obtained.

4. The method for detecting depth information of a moving object according to claim 1, wherein: For each event feature matrix, the density value of each discrete spatial position representing the spatial aggregation of event information is evaluated, and the initial region is preliminarily screened according to the density value to obtain candidate regions, including: Based on a preset step size, downsampling processing is performed on each of the event feature matrices to obtain a downsampled feature matrix; For the downsampled feature matrix, evaluating and obtaining a density value of spatial aggregation of event information represented by each discrete spatial position; Based on the density quantity value, a plurality of candidate coordinates are screened from the down-sampled feature matrix, and a fusion process is performed on each of the candidate coordinates to obtain a fusion region; The minimum circumscribed matrix of the fused region is selected as an initial candidate region, and upsampling is performed on the initial candidate region to obtain the candidate region after preliminary screening.

5. The method for detecting depth information of a moving object according to claim 4, wherein: The evaluation obtains a density area value representing the size of the target object in each candidate area, and performs a secondary screening on the candidate area according to the density area value to obtain the target area under the current viewing angle, including: Obtaining an expected size value of the target object, and determining a second threshold based on the expected size value, the preset step size, and a preset conservative value; The density area value representing the size of the target object in each fusion area is evaluated and obtained. If the density area value is greater than the second threshold, the corresponding fusion area is determined to be the target area under the current viewing angle.

6. The method for detecting depth information of a moving object according to claim 1, wherein: The evaluation obtains a density area value representing the size of the target object in each candidate area, and performs a secondary screening on the candidate areas according to the density area value to obtain the target area under the current viewing angle, further comprising: Calculating the size of a pixel region represented by a non-zero eigenvalue within the candidate region, and calculating the area ratio between the pixel region size and the candidate region to obtain a density area value represented by the size of the target object; If the density area value is greater than a preset third threshold, the corresponding candidate area is determined to be the target area under the current viewing angle.

7. The method for detecting depth information of a moving object according to claim 1, wherein: The matching process is performed on the event information corresponding to the target area at different viewing angles to determine the disparity of the event feature matrix of the target object at different viewing angles, including: Performing coordinate transformation and clustering processing on the event information in the event feature matrix at different viewing angles to obtain a plurality of center points determined at different viewing angles; Calculating the similarity between the center points under different viewing angles, and determining the two most similar center points under different viewing angles as a pair of matching points representing the same target object; The distance difference between the matching points at different viewing angles is calculated, and the disparity of the event feature matrix of the target object at different viewing angles is determined according to the distance difference.

8. The method for detecting depth information of a moving object according to claim 1, wherein: The determining target depth information between the target object and the binocular event camera based on the disparity includes: Obtaining the average focal length and baseline length of the binocular event camera after calibration is completed; The target depth information between the target object and the binocular event camera is determined according to the ratio of the average focal length, the baseline length and the parallax.

9. A device for detecting depth information of a moving target, characterized in that: include: The acquisition module is used to obtain the event feature matrix containing the target object in the initial area from different perspectives using a pre-calibrated binocular event camera; A first screening module is configured to evaluate, for each event feature matrix, a density value of spatial aggregation of event information at each discrete spatial location, and preliminarily screen the initial region based on the density value to obtain a candidate region; A second screening module is configured to evaluate and obtain a density area value representing the size of the target object in each candidate area, and perform a secondary screening of the candidate areas based on the density area value to obtain the target area under the current viewing angle; a disparity determination module, configured to perform matching processing on the event information corresponding to the target area at different viewing angles, and determine the disparity of the event feature matrix of the target object at different viewing angles; A target depth information determination module is configured to determine target depth information between the target object and the binocular event camera based on the disparity.

10. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the depth information detection method of a moving target according to any one of claims 1 to 8 when executing the computer program.

11. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for detecting depth information of a moving target according to any one of claims 1 to 8 is implemented.

Images