A method, system and device for realizing microsecond-level 3D detection based on binocular DVS

Abstract

The application discloses a kind of based on binocular DVS implementation microsecond 3D detection method, system and device, for realizing microsecond 3D detection under high-speed motion scene, improve the real-time of detection result.The method of the application comprises: calibrating binocular DVS camera, obtains the internal and external parameters of the binocular DVS camera;Two groups of DVS data are obtained by using the calibrated binocular DVS camera for shooting, and the DVS data include the polarity of pixel;The gradient features of pixels in the two groups of DVS data are extracted respectively;Matching of feature point pairs is carried out based on the polarity and the gradient feature;Disparity information between the feature point pairs is obtained, and depth information is calculated according to the internal and external parameters of the binocular DVS camera and the disparity information.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, and in particular to a method, system and device for realizing microsecond-level 3D detection based on binocular DVS. BACKGROUND

[0002] Conventional 3D detection schemes include monocular 3D, binocular, multi-camera 3D, etc. Among them, monocular 3D target detection cannot provide sufficient 3D information, and it is difficult to accurately predict the position information of the target. The binocular 3D detection scheme records different perspectives of the same scene using two cameras, and performs image processing and comparison, which can measure the distance of the target object relative to the camera. In addition to the camera-based detection scheme, common 3D detection schemes also include using laser radar for triangulation, using time flight method and structure light 3D measurement technology for measurement. Different 3D detection schemes differ in application scenarios, cost, accuracy, etc., and selecting a suitable scheme for specific needs requires comprehensive consideration of various factors.

[0003] In the prior art, due to the limitations of the hardware performance of the camera and the resolution of the collected images, etc., the frame rate is basically within 60fps. For 3D cameras with VGA and above resolution, the frame rate is generally within 30fps. In high-speed moving object detection, the camera needs to capture the position and state of the object in a very short time. If the frame rate of the camera is low, only a small number of image frames can be captured per second. When a high-speed moving object passes through the camera, there may be obvious image blur or ghosting, etc., which makes it difficult for the prior art to detect 3D of high-speed moving objects. SUMMARY

[0004] The present application provides a method, system and device for realizing microsecond-level 3D detection based on binocular DVS, which is used to realize microsecond-level 3D detection in high-speed motion scenarios and improve the real-time performance of the detection results.

[0005] The first aspect of the present application provides a method for realizing microsecond-level 3D detection based on binocular DVS, comprising:

[0006] Calibrating a binocular DVS camera to obtain the internal and external parameters of the binocular DVS camera;

[0007] Using the calibrated binocular DVS camera to take pictures to obtain two groups of DVS data, wherein the DVS data includes the polarity of the pixels;

[0008] Respectively extracting the gradient features of the pixels in the two groups of DVS data;

[0009] Matching the feature points based on the polarity and the gradient features;

[0010] Obtaining disparity information between the feature point pairs, and calculating depth information according to internal and external parameters of the binocular DVS camera and the disparity information.

[0011] Optionally, the extracting gradient features of pixels in the two groups of DVS data respectively comprises:

[0012] Calculating gradient features of target pixels in the two groups of DVS data respectively, the target pixels being polar pixels, and the gradient features including gradient modulus and gradient direction;

[0013] Calculating gradient features of neighbor pixels around the target pixel based on a preset scale.

[0014] Optionally, the gradient modulus of the target pixel is:

[0015]

[0016] The gradient direction of the target pixel is:

[0017]

[0018] wherein (x, y) is the coordinate of the target pixel, m(x, y) is the gradient modulus of the target pixel, θ(x, y) is the gradient direction of the target pixel, L(x, y) is the polarity value of the target pixel, L(x+1, y), L(x-1, y), L(x, y+1) and L(x, y-1) represent polarity values of adjacent positions of the target pixel respectively.

[0019] Optionally, before the matching of the feature point pairs based on the polarity and the gradient features, the method further comprises:

[0020] Transforming the gradient direction of the target pixel to a target direction and recording a transformation angle;

[0021] Transforming the gradient directions of the neighbor pixels according to the transformation angle to obtain transformed gradient features;

[0022] The matching of the feature point pairs based on the polarity and the gradient features comprises:

[0023] Matching the feature point pairs based on the polarity and the transformed gradient features.

[0024] Optionally, the matching of the feature point pairs based on the polarity and the gradient features comprises:

[0025] Screening out to-be-matched feature points according to the polarity;

[0026] Calculating vector differences between the to-be-matched feature points according to the gradient features;

[0027] If a vector difference between two to-be-matched feature points is less than a preset vector difference threshold, it is determined that the two to-be-matched feature points are a matched feature point pair.

[0028] Optionally, the filtering out of the to-be-matched feature point according to the polarity information of the target pixel and the neighborhood pixels comprises:

[0029] subtracting the polarities of the target pixel and the neighborhood pixels to obtain a difference matrix;

[0030] summing up absolute values of each element in the difference matrix to obtain a difference degree value;

[0031] If the difference degree value is less than a preset difference degree threshold, it is determined that the target pixel is a to-be-matched feature point.

[0032] Optionally, the preset scale is 3*3, 4*4, 5*5 or 6*6.

[0033] The second aspect of the present application provides a system for realizing microsecond-level 3D detection based on a binocular DVS, comprising:

[0034] a calibration unit configured to calibrate a binocular DVS camera to obtain internal and external parameters of the binocular DVS camera;

[0035] a shooting unit configured to use the calibrated binocular DVS camera to shoot to obtain two groups of DVS data, wherein the DVS data comprises the polarity of pixels;

[0036] an extraction unit configured to extract gradient features of the pixels in the two groups of DVS data, respectively;

[0037] a matching unit configured to match feature point pairs based on the polarities and the gradient features;

[0038] a calculation unit configured to obtain disparity information between the feature point pairs and calculate depth information according to the internal and external parameters of the binocular DVS camera and the disparity information.

[0039] Optionally, the extraction unit is specifically configured to:

[0040] calculate gradient features of target pixels in the two groups of DVS data, respectively, wherein the target pixels are pixels with polarity, and the gradient features comprise gradient modulus and gradient direction;

[0041] calculate gradient features of neighborhood pixels around the target pixels based on a preset scale.

[0042] Optionally, the gradient modulus of the target pixel is:

[0043]

[0044] a gradient direction of the target pixel is:

[0045]

[0046] wherein (x, y) is a coordinate of the target pixel, m(x, y) is a gradient modulus of the target pixel, θ(x, y) is a gradient direction of the target pixel, L(x, y) is a polarity value of the target pixel, L(x+1, y), L(x-1, y), L(x, y+1), L(x, y-1) respectively represent polarity values of adjacent positions of the target pixel.

[0047] Optionally, the system further comprises a transformation unit;

[0048] the transformation unit is configured to:

[0049] transform the gradient direction of the target pixel to a target direction, and record a transformation angle;

[0050] transform the gradient direction of the neighborhood pixel according to the transformation angle, to obtain a transformed gradient feature;

[0051] the matching unit is specifically configured to:

[0052] match the feature point pairs based on the polarity and the transformed gradient feature.

[0053] Optionally, the matching unit is specifically configured to:

[0054] screen out to-be-matched feature points according to the polarity;

[0055] calculate vector differences between the to-be-matched feature points according to the gradient feature;

[0056] if the vector difference between two to-be-matched feature points is less than a preset vector difference threshold, determine that the two to-be-matched feature points are matched feature point pairs.

[0057] Optionally, the matching unit is specifically configured to:

[0058] subtract the polarity of the target pixel from the polarity of the neighborhood pixel, to obtain a difference matrix;

[0059] sum absolute values of each element in the difference matrix, to obtain a difference degree value;

[0060] if the difference degree value is less than a preset difference degree threshold, determine that the target pixel is a to-be-matched feature point.

[0061] A third aspect of the present application provides a device for realizing microsecond-level 3D detection based on a binocular DVS, the device comprising:

[0062] a processor, a memory, an input / output unit and a bus;

[0063] the processor is connected with the memory, the input / output unit and the bus;

[0064] the memory stores a program, and the processor invokes the program to execute the method for realizing microsecond-level 3D detection based on binocular DVS in the first aspect and any optional one of the first aspect.

[0065] The fourth aspect of the present application provides a computer readable storage medium, which stores a program, and the program executes the method for realizing microsecond-level 3D detection based on binocular DVS in the first aspect and any optional one of the first aspect when executed on a computer.

[0066] From the above technical solutions, the present application has the following advantages:

[0067] The DVS camera is realized based on dynamic vision sensor technology, can capture the motion and change of an object at very low delay, and realizes object perception in a high-speed motion scene. Two groups of DVS data output by the binocular DVS camera are acquired, and the pixel polarity and gradient features in the DVS data are used to match the feature point pairs, and then the depth information is obtained through depth calculation according to the disparity information of the feature point pairs. Through the combination of high-speed perception of DVS and depth calculation of binocular vision, microsecond-level depth perception of a fast motion scene is realized, thereby realizing 3D detection at a microsecond time scale, and the real-time performance of the detection result is significantly improved. BRIEF DESCRIPTION OF DRAWINGS

[0068] In order to more clearly illustrate the technical solutions in the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0069] Figure 1 An embodiment flowchart of the method for realizing microsecond-level 3D detection based on binocular DVS provided by the present application is shown in the figure.

[0070] Figure 2 A schematic diagram of the depth information calculation principle in the method for realizing microsecond-level 3D detection based on binocular DVS provided by the present application is shown in the figure.

[0071] Figure 3 Another embodiment flowchart of the method for realizing microsecond-level 3D detection based on binocular DVS provided by the present application is shown in the figure.

[0072] Figure 4 A schematic diagram of gradient features in the method for providing a microsecond-level 3D detection based on a binocular DVS for an application;

[0073] Figure 5 A transformation schematic diagram of gradient features in the method for providing a microsecond-level 3D detection based on a binocular DVS for an application;

[0074] Figure 6 An embodiment structure schematic diagram of the system for providing a microsecond-level 3D detection based on a binocular DVS for the application;

[0075] Figure 7 An embodiment structure schematic diagram of the device for providing a microsecond-level 3D detection based on a binocular DVS for the application. DETAILED DESCRIPTION

[0076] The application provides a method, system and device for providing a microsecond-level 3D detection based on a binocular DVS, which is used to realize a microsecond-level 3D detection in a high-speed motion scene and improve the real-time performance of the detection result.

[0077] It should be noted that the method for providing a microsecond-level 3D detection based on a binocular DVS is especially suitable for a fast motion scene, such as robot navigation, unmanned vehicle, etc. The method can be applied to a terminal, such as a vehicle, an unmanned aerial vehicle or a mobile robot, and can also be applied to a server. For convenience of description, the terminal is taken as an execution subject for example in the application.

[0078] Please refer to Figure 1 , Figure 1 An embodiment of the method for providing a microsecond-level 3D detection based on a binocular DVS for the application, the method comprises:

[0079] 101. Calibrating a binocular DVS camera to obtain internal and external parameters of the binocular DVS camera;

[0080] The DVS camera is a camera based on dynamic vision sensor technology. Compared with a traditional frame sensor, the DVS camera adopts an event-driven manner and only generates events when a significant change in a scene is perceived, thereby effectively reducing data redundancy and having extremely low latency and high dynamic range. The working principle of the DVS camera is to perceive the change in brightness at the pixel level to generate event stream data. When the camera perceives a change in brightness, the corresponding pixel will generate an event, which contains the timestamp of the event and the position information of the pixel. In the embodiment, a binocular DVS camera is used to realize 3D detection, that is, two DVS camera modules are constructed to obtain binocular disparity information, so as to calculate the depth of an object in a scene.

[0081] Before using a stereo DVS camera, it needs to be calibrated to obtain its intrinsic and extrinsic parameters, enabling accurate mapping of pixel coordinates to 3D space. It's important to note that because DVS cameras acquire event stream data rather than traditional image frames, the calibration process requires specialized feature point extraction methods, such as edge detection, optical flow estimation, or convolutional neural networks. The terminal then uses the extracted feature points and their corresponding world coordinates to calibrate the camera's intrinsic and extrinsic parameters using the Zhang Zhengyou calibration method. These parameters include intrinsic parameters (parameters within the camera, such as focal length and principal point coordinates) and extrinsic parameters (parameters outside the camera, such as rotation matrices and translation vectors). These parameters describe the camera's geometric characteristics and positional relationships.

[0082] 102. Two sets of DVS data were obtained by shooting with a calibrated binocular DVS camera. The DVS data includes the polarity of the pixels.

[0083] The terminal uses a calibrated binocular DVS camera to capture images of the target scene, generating two sets of DVS data, corresponding to the left and right cameras respectively. The data acquired by the binocular DVS camera consists of an event stream, where each event contains information about changes in light intensity, as well as the pixel position and polarity. The data output by the binocular DVS camera is in the format (x, y, polarity, timestamp), corresponding to the pixel coordinates [x, y], the polarity of the detected light change (polarity), and the time of the detected event (timestamp).

[0084] The terminal needs to obtain the polarity information of pixels in the DVS data. Polarity refers to the direction of light intensity change for each event in the event stream, indicating whether the event is caused by an increase in brightness (positive polarity) or a decrease in brightness (negative polarity). Polarity information is an important component of DVS data, providing information about the direction and intensity of light intensity changes in the scene. Therefore, pixel polarity is helpful for subsequent feature point extraction and matching.

[0085] 103. Extract the gradient features of pixels from the two sets of DVS data respectively;

[0086] The terminal also needs to calculate the gradient at each pixel location to extract the gradient features of pixels in the two sets of DVS data. The gradient typically consists of two components: the horizontal gradient (x-direction) and the vertical gradient (y-direction). The terminal can use gradient calculation algorithms such as Sobel or Prewitt; specific algorithms are not limited here. The gradient features of pixels in DVS data reflect the intensity and direction of polarity changes at each pixel location. By extracting the gradient features of pixels in the two sets of DVS data, edges, textures, and other structures can be identified, enhancing the perception of light intensity changes and aiding in subsequent feature point pair matching.

[0087] 104、based on the polarity and gradient features, match the feature point pairs;

[0088] The terminal matches the feature points in the two groups of DVS data by obtaining the polarity and gradient features of the pixels in the two groups of DVS data. The purpose of matching is to find similar or identical feature point pairs. Specifically, the terminal compares the polarity and gradient features of the pixels in the two groups of DVS data, calculates the similarity measure between them, and then filters out feature point pairs with high similarity according to the similarity measure, and establishes the mapping relationship between them. It should be noted that the terminal can use various measurement methods, such as Euclidean distance, cosine similarity, etc., which are not limited here.

[0089] 105、obtain the disparity information between the feature point pairs, and calculate the depth information according to the internal and external parameters of the binocular DVS camera and the disparity information.

[0090] The terminal can obtain the feature point pairs of the binocular DVS camera through step 104. For each feature point pair, the terminal can calculate its disparity information, that is, the difference in pixel position in the left and right cameras. Then, using the binocular geometry and the known internal and external parameters of the binocular DVS camera, the distance between the feature point pairs can be calculated through the triangular relationship, and then the depth information in the scene can be calculated. Please refer to Figure 2 Figure 2 is a schematic diagram of the depth information calculation principle of the binocular DVS camera. Through the disparity information xl-xr between the feature point pairs, the depth information z can be calculated using the triangular relationship. Where f is the focal length of the DVS camera, b is the baseline distance between the two DVS cameras, that is, the distance between the optical centers of the two DVS cameras, and z is the depth.

[0091] In this embodiment, the DVS camera is realized based on dynamic vision sensor technology, which can capture the motion and changes of objects at very low delay, and realize object perception in high-speed motion scenes. By obtaining two groups of DVS data output by the binocular DVS camera, and using the polarity and gradient features of the pixels in the DVS data to match the feature point pairs, and then calculating the depth information according to the disparity information of the feature point pairs. By combining the high-speed perception of DVS and the depth calculation of binocular vision, the microsecond-level depth perception of the fast motion scene is realized, thereby realizing the microsecond-level 3D detection, and significantly improving the real-time performance of the detection result.

[0092] The method for realizing microsecond-level 3D detection based on binocular DVS provided by the present application will be described in detail below. Please refer to Figure 3 Figure 3 Another embodiment of the method for realizing microsecond-level 3D detection based on binocular DVS provided by the present application is provided, which comprises:

[0093] ​​301. calibrate the binocular DVS camera to obtain the internal and external parameters of the binocular DVS camera;

[0094] 302. use the calibrated binocular DVS camera to take pictures to obtain two sets of DVS data, the DVS data including the polarity of the pixels;

[0095] In this embodiment, steps 301 to 302 are similar to steps 101 to 102 of the foregoing embodiment, which will not be described here.

[0096] 303. respectively calculate the gradient features of the target pixels in the two sets of DVS data, the target pixels being the pixels with polarity, and the gradient features including the gradient modulus and the gradient direction;

[0097] The terminal calculates the gradient features of the target pixels in the two sets of DVS data, the target pixels being the pixels with polarity, i.e. the region pixels with polarity change. For each event in the DVS data, the terminal extracts the pixel position information thereof, filters out the target pixel (x, y) and calculates the gradient features thereof, which specifically include the gradient modulus and the gradient direction. The gradient modulus represents the intensity or degree of polarity change at the position of the target pixel. The greater the gradient modulus, the more intense the polarity change at the position. The gradient direction represents the direction of polarity change at the position of the target pixel.

[0098] Specifically, the gradient modulus of the target pixel is:

[0099]

[0100] The gradient direction of the target pixel is:

[0101]

[0102] where (x, y) is the coordinate of the target pixel, m(x, y) is the gradient modulus of the target pixel, θ(x, y) is the gradient direction of the target pixel, L(x, y) is the polarity value of the target pixel, L(x+1, y), L(x-1, y), L(x, y+1) and L(x, y-1) represent the polarity values of the adjacent positions of the target pixel, respectively.

[0103] 304. calculate the gradient features of the neighborhood pixels around the target pixel based on a preset scale;

[0104] The terminal extracts the neighborhood pixels of a corresponding size centered on the target pixel according to the preset scale, and calculates the gradient modulus and the gradient direction of each neighborhood pixel. Under different scales, the features of the same object or structure can have different manifestations, so when calculating the gradient features of the neighborhood pixels around the target pixel, using different preset scales can obtain more feature information of different details to capture the information in the scene more comprehensively.

[0105] In some specific embodiments, the preset scale is specifically set as 3*3, 4*4, 5*5 and 6*6. Please refer to Figure 4 , Figure 4 The gradient feature of the calculated target pixel and the neighborhood pixels is shown in the figure, where the arrow direction corresponds to the gradient direction at the pixel position, and the arrow length corresponds to the gradient modulus at the pixel position.

[0106] 305, transform the gradient direction of the target pixel to a target direction and record the transformation angle, transform the gradient direction of the neighborhood pixels according to the transformation angle to obtain the transformed gradient feature;

[0107] The terminal transforms the gradient direction at the target pixel position and rotates it to the target direction, and records the corresponding transformation angle. The target direction can be specifically set as the direction of horizontal right. For the neighborhood pixels around the target pixel, the terminal also transforms the gradient direction according to the recorded transformation angle, thereby obtaining new gradient features. By transforming the gradient direction, the calculation of the disordered gradient features when matching can be more concise, which helps to improve the robustness and accuracy of matching. And it can reduce the algorithmic consumption of the matching algorithm in practical application, and improve the matching speed of the feature point pair.

[0108] Specifically, the terminal transforms the gradient direction by the following formula:

[0109]

[0110] Where v' is the new vector, i.e. the transformed gradient feature, v is the original vector, cos(θ) is the transformation angle, and n is the normal vector.

[0111] Please refer to Figure 5 , Figure 5 The gradient direction transformation schematic diagram is shown in the figure, i.e. the terminal rotates the gradient direction at the target pixel position to the direction of horizontal right, and the rest of the neighborhood pixels are rotated.

[0112] 306, screen out the to-be-matched feature points according to the polarity;

[0113] The polarity information of the pixel itself represents the event stream polarity of each pixel, which is usually +1 (positive polarity), -1 (negative polarity) or 0 (no polarity). The polarity information can capture the dynamic changes in the scene, such as light changes or object movements. Therefore, according to the polarity information of the target pixel and the neighborhood pixels, the pixel points whose polarity changes meet certain conditions can be preliminarily screened out and determined as to-be-matched feature points, and the rest of the pixel points do not participate in the subsequent matching.

[0114] In some specific embodiments, the terminal uses the following screening method:

[0115] S1, subtract the polarity of the target pixel from the polarity of the neighborhood pixels to obtain a difference matrix;

[0116] For each target pixel, the terminal subtracts the polarity of its neighborhood pixels, i.e., the 3x3, 4x4, 5x5 or 6x6 pixels around the target pixel, to obtain a difference matrix. The difference matrix represents the difference information of the neighborhood around the target pixel.

[0117] S2, sum the absolute values of each element in the difference matrix to obtain a difference value;

[0118] For each element in the difference matrix, the terminal takes the absolute value of each element, making all difference values non-negative, and then sums each element to obtain a total difference value.

[0119] S3, if the difference value is less than a preset difference threshold, determine the target pixel as a matching feature point.

[0120] The terminal compares the total difference value with a preset difference threshold. If the total difference value is less than the preset difference threshold, it means that the polarity change around the target pixel meets the condition, and the target pixel is determined as a matching pixel. It should be noted that the difference threshold can be set to 10% of the number of elements in the difference matrix, which can be adjusted flexibly according to actual requirements.

[0121] The large polarity change in the local area may be caused by object boundaries or textures, etc. These areas are prone to cause mismatching. Therefore, by this method, some local areas with large polarity changes can be excluded, and only the pixels with small polarity changes in the local area are determined as matching feature points, so that the subsequent matching process is more robust, which helps to prevent the influence of abnormal values or noise on the matching result and reduces the matching calculation amount.

[0122] 307, calculate the vector difference between the matching feature points according to the gradient feature, and if the vector difference between two matching feature points is less than a preset vector difference threshold, determine the two matching feature points as a matching feature point pair;

[0123] The terminal calculates the vector difference between the matching feature points in the two groups of DVS data according to the gradient feature, i.e., calculates the difference between the gradient direction and the gradient size of the matching feature points, and then compares the vector difference with a preset vector difference threshold. If the vector difference between two matching feature points is less than the preset vector difference threshold, the two matching feature points are determined as a matching feature point pair, so as to select the feature point pair with similar gradient direction and size.

[0124] It should be noted that the feature point pair extraction and matching method in steps 303 to 305 can also be used when calibrating a binocular DVS camera.

[0125] 308、acquire disparity information between the feature point pairs, and calculate depth information according to the internal and external parameters of the binocular DVS camera and the disparity information.

[0126] In the embodiment, step 308 is similar to step 105 of the foregoing embodiment, and thus is not described herein again.

[0127] In the embodiment, the DVS camera is implemented based on a dynamic vision sensor technology, can capture the motion and change of an object at a very low delay, and realizes object perception in a high-speed motion scene. Two groups of DVS data output by the binocular DVS camera are acquired, and the feature point pairs are matched by using the pixel polarity and gradient feature in the DVS data. Then, the depth information is calculated according to the disparity information of the feature point pairs. By combining the high-speed perception of the DVS and the depth calculation of the binocular vision, the microsecond-level depth perception of the fast motion scene is realized, so that the microsecond-level 3D detection is realized. Moreover, the gradient feature extracted needs to be directionally transformed before matching, so that the matching is more consistent, and thus the matching process is simplified, the matching speed and accuracy are improved, and the real-time performance of the detection result is further improved.

[0128] Please refer to Figure 6 , Figure 6 An embodiment of a system for realizing microsecond-level 3D detection based on a binocular DVS provided in the application includes:

[0129] A calibration unit 601 is configured to calibrate the binocular DVS camera to obtain the internal and external parameters of the binocular DVS camera.

[0130] A shooting unit 602 is configured to use the calibrated binocular DVS camera to shoot to obtain two groups of DVS data, and the DVS data includes the polarity of the pixels.

[0131] An extraction unit 603 is configured to extract the gradient feature of the pixels in the two groups of DVS data, respectively.

[0132] A matching unit 604 is configured to match the feature point pairs based on the polarity and the gradient feature.

[0133] A calculation unit 605 is configured to acquire the disparity information between the feature point pairs, and calculate the depth information according to the internal and external parameters of the binocular DVS camera and the disparity information.

[0134] Optionally, the extraction unit 603 is specifically configured to:

[0135] The gradient feature of the target pixels in the two groups of DVS data is calculated, respectively, the target pixels are the pixels with polarity, and the gradient feature includes the gradient modulus and the gradient direction.

[0136] The gradient feature of the neighborhood pixel around the target pixel is calculated based on a preset scale.

[0137] Optionally, the gradient module of the target pixel is:

[0138]

[0139] The gradient direction of the target pixel is:

[0140]

[0141] wherein (x, y) is the coordinate of the target pixel, m(x, y) is the gradient module of the target pixel, θ(x, y) is the gradient direction of the target pixel, L(x, y) is the polarity value of the target pixel, L(x+1, y), L(x-1, y), L(x, y+1), and L(x, y-1) represent the polarity values of the neighboring positions of the target pixel respectively.

[0142] Optionally, the system further comprises a transformation unit 606.

[0143] The transformation unit 606 is configured to:

[0144] transform the gradient direction of the target pixel to a target direction and record a transformation angle;

[0145] transform the gradient direction of the neighborhood pixel according to the transformation angle to obtain a transformed gradient feature;

[0146] The matching unit 604 is specifically configured to:

[0147] match the feature point pairs based on the polarity and the transformed gradient feature.

[0148] Optionally, the matching unit 604 is specifically configured to:

[0149] select the to-be-matched feature points according to the polarity;

[0150] calculate the vector difference between the to-be-matched feature points according to the gradient feature;

[0151] if the vector difference between the two to-be-matched feature points is less than a preset vector difference threshold, determine that the two to-be-matched feature points are the matched feature point pair.

[0152] Optionally, the matching unit 604 is specifically configured to:

[0153] subtract the polarity of the target pixel from the polarity of the neighborhood pixel to obtain a difference matrix;

[0154] sum the absolute values of each element in the difference matrix to obtain a difference degree value;

[0155] If the difference value is less than the preset difference threshold, it is determined that the target pixel is a feature point to be matched.

[0156] In the system of the embodiment, the functions of the units are the same as those of the foregoing Figure 1 or Figure 3 The steps in the method embodiment are not described herein again.

[0157] The application further provides a device for realizing microsecond-level 3D detection based on binocular DVS, please refer to Figure 7 , Figure 7 An embodiment of the device for realizing microsecond-level 3D detection based on binocular DVS provided by the application comprises:

[0158] The processor 701, the memory 702, the input and output unit 703, and the bus 704 are connected;

[0159] The processor 701 is connected with the memory 702, the input and output unit 703, and the bus 704;

[0160] The memory 702 stores a program, and the processor 701 invokes the program to execute any of the above methods for realizing microsecond-level 3D detection based on binocular DVS.

[0161] The application further relates to a computer readable storage medium, and the computer readable storage medium stores a program, and the program is characterized in that, when the program runs on a computer, the computer executes any of the above methods for realizing microsecond-level 3D detection based on binocular DVS.

[0162] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, and will not be described herein again.

[0163] In the several embodiments provided by the application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic. The division of the units is only a logical function division. There can be another division manner for actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0164] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0165] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0166] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical scheme of the present application essentially or the part that contributes to the prior art or the whole or part of the technical scheme can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, read-only memory), a random access memory (RAM, random access memory), a magnetic disk or an optical disk, and various program code storage media.

Claims

1. A method for realizing microsecond-level 3D detection based on binocular DVS, characterized in that, The method comprises: calibrating a binocular DVS camera to obtain internal and external parameters of the binocular DVS camera; photographing using the calibrated binocular DVS camera to obtain two groups of DVS data, the DVS data comprising polarities of pixels; extracting gradient features of the pixels in the two groups of DVS data respectively; matching feature point pairs based on the polarities and the gradient features; obtaining disparity information between the feature point pairs and calculating depth information according to the internal and external parameters of the binocular DVS camera and the disparity information; the extracting of the gradient features of the pixels in the two groups of DVS data respectively comprises: calculating gradient features of target pixels in the two groups of DVS data respectively, the target pixels being pixels with polarities, the gradient features comprising gradient modulus and gradient direction; calculating gradient features of neighborhood pixels around the target pixels based on a preset scale; the matching of the feature point pairs based on the polarities and the gradient features comprises: screening out to-be-matched feature points according to the polarities; calculating vector differences between the to-be-matched feature points according to the gradient features; if the vector difference between two to-be-matched feature points is less than a preset vector difference threshold, determining that the two to-be-matched feature points are a matched feature point pair; the screening out of the to-be-matched feature points according to the polarities comprises: subtracting the polarities of the target pixels and the neighborhood pixels to obtain a difference matrix; summing absolute values of each element in the difference matrix to obtain a difference degree value; if the difference degree value is less than a preset difference degree threshold, determining that the target pixel is a to-be-matched feature point.

2. The method of claim 1, wherein, the gradient modulus of the target pixel is: ； the gradient direction of the target pixel is: ； wherein (x, y) is the coordinate of the target pixel, m(x, y) is the gradient modulus of the target pixel, θ(x, y) is the gradient direction of the target pixel, L(x, y) is the polarity value of the target pixel, L(x+1, y), L(x-1, y), L(x, y+1) and L(x, y-1) represent polarity values of positions adjacent to the target pixel respectively.

3. The method of claim 1, wherein, Before the matching of the feature point pairs based on the polarities and the gradient features, the method further comprises: transforming the gradient direction of the target pixel to a target direction and recording a transformation angle; transforming the gradient directions of the neighborhood pixels according to the transformation angle to obtain transformed gradient features; the matching of the feature point pairs based on the polarities and the gradient features comprises: matching the feature point pairs based on the polarities and the transformed gradient features.

4. The method according to any one of claims 1 to 3, characterized in that, The preset scale is 3*3, 4*4, 5*5 or 6*6.

5. A system for implementing microsecond-level 3D detection based on binocular DVS, characterized in that, The system comprises: a calibration unit configured to calibrate a binocular DVS camera to obtain internal and external parameters of the binocular DVS camera; a photographing unit configured to photograph using the calibrated binocular DVS camera to obtain two groups of DVS data, the DVS data comprising polarities of pixels; an extraction unit configured to extract gradient features of the pixels in the two groups of DVS data respectively; a matching unit configured to match feature point pairs based on the polarities and the gradient features. The computing unit is configured to obtain disparity information between the feature point pairs, and calculate depth information according to internal and external parameters of the binocular DVS camera and the disparity information. The extraction unit is specifically configured to: respectively calculate gradient features of target pixels in the two groups of DVS data, the target pixels being polar pixels, and the gradient features including gradient modulus and gradient direction; calculate gradient features of neighborhood pixels around the target pixels based on a preset scale; The matching unit is specifically configured to: screen out to-be-matched feature points according to the polarities; calculate vector differences between the to-be-matched feature points according to the gradient features; if the vector difference between two to-be-matched feature points is less than a preset vector difference threshold, determine that the two to-be-matched feature points are a matched feature point pair. The matching unit is specifically further configured to: subtract polarities of the target pixels and the neighborhood pixels to obtain a difference matrix; sum absolute values of each element in the difference matrix to obtain a difference degree value; if the difference degree value is less than a preset difference degree threshold, determine that the target pixel is a to-be-matched feature point.

6. An apparatus for achieving microsecond-level 3D detection based on binocular DVS, characterized in that, The device comprises: a processor, a memory, an input / output unit and a bus; the processor is connected with the memory, the input / output unit and the bus; the memory stores a program, and the processor invokes the program to execute the method in any one of claims 1 to 4. 7.A computer readable storage medium, which stores a program, and the program performs the method in any one of claims 1 to 4 when executed on a computer.

Images