A high-speed surface 3D reconstruction method based on dynamic vision sensor focus change method

Abstract

The present invention discloses a surface three-dimensional reconstruction method based on a dynamic vision sensor. The present invention utilizes a dynamic vision sensor to replace a traditional frame image CCD / CMOS image sensor to obtain raw data in a very short time. The present invention utilizes the intrinsic relationship between the positive and negative event rate sequence generated by the dynamic sensor and the quasi-focus position to replace the contrast analysis method based on the traditional frame image focus change method, thereby achieving a processing speed higher than the traditional focus change method.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision surface three-dimensional reconstruction, and in particular to a surface three-dimensional reconstruction method based on a dynamic vision sensor focus change method. Background Art

[0002] In recent years, optical 3D surface reconstruction technology has been widely used in a variety of fields, including industrial parts inspection, reverse engineering, cultural relic protection, and medical aesthetics, due to its non-destructive and high-precision characteristics. Generally speaking, the principles underlying optical 3D surface reconstruction technology include: 1. Stripe structured light 3D reconstruction method, 2. Binocular vision 3D reconstruction method, 3. Confocal microscopy measurement method, 4. Interference-based 3D reconstruction method, and 5. Focus-variation-based 3D reconstruction method. The focus-variation-based 3D reconstruction method works as follows: a microscope objective with a small depth of field is used to image the surface to be measured onto a high-resolution CCD / CMOS sensor surface, and the image obtained by the sensor is acquired by a computer. A motor is used to move the surface to be measured along the optical axis, and each minute displacement of the motor generates an image on the image sensor. Images captured at different object distances are then analyzed for accuracy at each position in the image to determine the object distance at which the image is clearest, thereby obtaining depth information for the entire surface.

[0003] It is worth noting that the current three-dimensional reconstruction methods of the focus change method all use area array image sensors such as CCD or CMOS to acquire images. However, the dynamic range of CCD or CMOS sensors is limited (between 60 and 70 dB). When the reflectivity of the surface to be measured varies greatly, it is easy to cause the high-reflectivity area to be overexposed and the low-reflectivity area to be too dark.

[0004] Furthermore, the focus shift method places very high demands on the sensor's radiometric resolution. Radiometric resolution refers to the smallest difference in luminosity that a sensor can distinguish. For example, the most common eight-bit CMOS sensor can only distinguish 256 grayscale levels within its dynamic range, meaning it cannot distinguish texture differences within the same grayscale level. The focus shift method, on the other hand, acquires depth information by determining texture clarity. Area array image sensors with higher radiometric resolution and a higher number of bits are very expensive, and higher radiometric resolution also translates to lower image resolution or worse noise performance.

[0005] Furthermore, the focus variation method based on traditional image sensors requires multiple exposures of images at different object distances. When the motor reaches a specific object distance, it must first stop and stabilize so that the CCD / CMOS image sensor can capture an image free of motion blur. To achieve sufficient vertical resolution, the entire object distance range must be sampled in sufficient detail. Consequently, acquiring images at all sampling points takes a considerable amount of time, making this inspection process inefficient and inefficient in industrial production.

[0006] Therefore, the application space of three-dimensional reconstruction systems based on traditional image sensor focus change methods is often very limited. Summary of the Invention

[0007] In order to overcome the defects of low radiation resolution, small dynamic range and slow detection speed in the traditional image sensor focus change method, the present invention provides a high-speed surface three-dimensional reconstruction method based on the dynamic vision sensor focus change method, which can achieve high-precision, high dynamic range and high-speed three-dimensional surface reconstruction.

[0008] The present invention provides a surface three-dimensional reconstruction method based on a dynamic vision sensor, which comprises the following steps:

[0009] Step 1: Building a 3D reconstruction system based on the focus change method; in the 3D reconstruction system, a dynamic vision sensor is used as an imaging sensor of the surface to be measured;

[0010] Step 2: Move the surface to be measured along the optical axis. The dynamic vision sensor detects changes in the light intensity incident on the sensor surface during the movement of the surface to be measured. These changes in light intensity are marked as positive events and negative events according to the brightness and darkness, respectively. After the movement is completed, the computer connected to the dynamic vision sensor stores the original event point cloud data generated by this process. These event raw data are arranged in order according to the pixel coordinates (x, y) of the event and the time point t at which the event occurred. The X and Y directions are defined as the directions in which the pixel coordinates x and y increase, respectively, and the T direction is the direction in which the time point t at which the event occurs increases, forming an XYT three-dimensional event point cloud cube.

[0011] Step 3: Divide the XYT 3D event point cloud cube obtained in step 2 into blocks in the T direction according to the time interval of Δt. For each pixel coordinate (x, y), count the number of positive events and the number of negative events in each time interval of Δt. Finally, a positive event rate cube and a negative event rate cube are obtained. The elements in the cube are integer variables, representing the total number of events generated by each pixel in each time interval.

[0012] Step 4: Perform wavelet filtering on the positive event rate cube and the negative event rate cube obtained in step 3 along the T direction to obtain the filtered positive and negative event rate cubes;

[0013] Step 5: Split the filtered positive and negative event rate cubes obtained in Step 4 in the X and Y directions to obtain multiple one-dimensional sequences equal to the number of pixels in the dynamic vision sensor, each sequence representing a one-dimensional sequence of positive and negative event rates for that pixel; for each pixel, construct a focus error evaluation function based on the symmetric distribution of the positive and negative event rate sequences about the focus time point, and obtain the focus time point. The specific meaning of the focus time point is: the time point when the image is clearest during motion along the optical axis;

[0014] Step 6: Based on the correspondence between the time point and the displacement along the optical axis, the relative depth z is obtained, thereby obtaining the three-dimensional coordinates (x, y, z) of the pixel point; after all pixel points complete the in-focus position judgment, the three-dimensional morphological characteristics of the surface to be measured are obtained.

[0015] As a preferred embodiment of the present invention, the three-dimensional topography surface obtained according to step six is ​​not necessarily smooth due to the presence of noise, and the method further comprises step seven of optimizing the continuity of the three-dimensional topography;

[0016] Step 7: Utilize the event rate information of the pixels surrounding each pixel to jointly optimize the depth z of the pixel so that the final three-dimensional shape is continuous. The joint optimization process includes: searching for pixels whose depth coordinate z and the depth mean of the eight pixels in their neighborhood in the three-dimensional shape surface obtained in step 6 are greater than a given threshold; for all such pixels, sum the unfiltered positive and negative event rate sequence of the eight pixels in its neighborhood with its own unfiltered positive and negative event rate sequence as the new positive and negative event rate sequence of the pixel; perform wavelet filtering on the new positive and negative event rate sequence, and repeat steps 5 and 6 to obtain the new three-dimensional coordinates of the pixel.

[0017] As a preferred embodiment of the present invention, the step three is specifically as follows:

[0018] 3.1) Divide the original event point cloud data obtained in step 2 into blocks in the time direction according to a certain time interval Δt, and count the total number of positive events and negative events that occur in each time interval Δt for each pixel (x, y);

[0019] 3.2) Assuming the number of pixels in the x-direction is M and the number of pixels in the y-direction is N, and at a given time interval Δt, the entire acquisition period is divided into T equal segments, resulting in M*N positive event rate sequences and negative event rate sequences of length T. These M*N positive event rate sequences and negative event rate sequences of length T are stacked according to the pixel arrangement to form positive event rate cubes and negative event rate cubes of size M*N*T.

[0020] As a preferred embodiment of the present invention, the step 4 is specifically as follows:

[0021] For the selected pixel point (x p ,y p ), the positive event rate sequence and negative event rate sequence are obtained in step three. The positive and negative event rate sequences obtained in step three are subjected to wavelet filtering to remove high-frequency noise and retain low-frequency information to obtain filtered positive and negative event rate sequences; after filtering the positive and negative event rate sequences corresponding to all pixels, these sequences are stacked in the order of pixel arrangement to form the filtered positive and negative event rate cubes.

[0022] As a preferred embodiment of the present invention, the step 5 is specifically as follows:

[0023] 5.1) The process of the surface to be measured moving along the optical axis with the motor is equivalent to the process of focusing by continuously changing the object distance at each point; for the pixel (x p ,y p ), the brightness change from defocus to quasi-focus should be consistent with the brightness change from quasi-focus to defocus, so the positive and negative event rate curves are symmetrically distributed, and the symmetrical center of the positive and negative event rate curves is the time point t when the focus is clearest. * ;

[0024] 5.2) Transform the subscripts of the positive and negative event rate sequences P[t] and N[t] to their initial values ​​of 0.

[0025] P′[t]←P[t+t1];

[0026] N′[t]←N[t+t1];

[0027] P[t] and N[t] are the positive and negative event rate sequences after noise removal, and their domain is [t1, t2]; t1 represents the time point when the motion along the optical axis begins, and t2 represents the time point when the motion along the optical axis ends; after subscript transformation, the domain of P′[t] and N′[t] is [0, t2-t1];

[0028] 5.3) Define the focus error evaluation function as

[0029]

[0030] Where mse(a) is the focus error evaluation function defined in the present invention. For different independent variables a, the smaller the value of mse(a), the better the The closer to the time point t when the focus is clearest * a is the optimization parameter for the in-focus error; t is the time series independent variable, representing the subscript of the positive and negative event rate time series; k is the parameter for the preset in-focus time point inspection range, and the value of k can be a real number from 0 to 1. A larger k value means a smaller in-focus time point inspection range.

[0031] 5.4) The symmetric center of the positive and negative event rates, i.e. the time point t when the focus is clearest * It is given by the following formula:

[0032]

[0033] Compared to existing technologies, the method proposed in this paper can obtain high-resolution three-dimensional topography information of the surface being measured through a single, rapid scan with a single sensor. By replacing traditional frame-image CCD / CMOS image sensors with dynamic vision sensors, raw data can be acquired in a very short time. The inherent relationship between the positive and negative event rate sequences generated by the dynamic sensor and the in-focus position is utilized to replace the contrast analysis method based on traditional frame-image focus change methods, achieving processing speeds higher than those of traditional focus change methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a schematic diagram of a 3D reconstruction system based on the focus change method;

[0035] Figure 2 is a schematic diagram of the surface to be measured used in Example 1 of the present invention;

[0036] Figure 3 is the original event point cloud image collected in Example 1 of the present invention;

[0037] Figure 4 is an event polarity rate curve diagram of a pixel examined in Example 1 of the present invention;

[0038] Figure 5 is the focus error evaluation function of the pixel examined in Example 1 of the present invention;

[0039] Figure 6 is a comparison diagram of the surface to be measured and the reconstructed surface in Example 1 of the present invention;

[0040] Figure 7 1 is a real picture of the surface to be measured at different object distances used in Example 2 of the present invention;

[0041] Figure 8 is the original event point cloud image collected by Example 2 of the present invention;

[0042] Figure 9 is an event polarity rate curve diagram of a pixel examined in Example 2 of the present invention;

[0043] Figure 10 is the focus error evaluation function of the pixel examined in Example 2 of the present invention;

[0044] Figure 11 Schematic diagram of the reconstructed surface of Example 2 of the present invention.

[0045] In the figure, 1. Dynamic vision sensor, 2. Microscope eyepiece, 3. Illumination light source, 4. Semi-transparent and semi-reflective mirror, 5. Microscope objective lens with small depth of field, 6. Surface to be measured, 7. Vertical movement of the surface to be measured driven by a drive motor, 8. Images of the same area taken at different object distances, 9. Light source, 10. Analyzer, 11. Polarizer, 12. Ring illumination light source. DETAILED DESCRIPTION

[0046] The present invention will be further described and illustrated below in conjunction with specific embodiments. The embodiments are merely illustrative of the present disclosure and do not limit its scope. The technical features of the various embodiments of the present invention may be combined accordingly, provided that there is no conflict between them.

[0047] See Figure 1 , wherein device 1 is a dynamic visual sensor, used to obtain information on changes in the brightness of light incident on each pixel of the sensor during the movement of the measured plane along the optical axis; device 2 microscope eyepiece, device 4 semi-transparent and semi-reflective mirror, and device 5 small depth of field microscope objective lens together constitute the optical imaging part of the system described in this patent; device 3 illumination light source and the light 9 emitted by it and device 12 ring illumination light source together constitute the optical illumination part of the system; device 10 polarizer and device 11 polarizer are optional parts of the system, used to reduce the interference of mirror reflected light in samples with obvious polarization characteristics; the remaining marked 6 in the figure represents the surface to be measured, 7 refers to the movement of the surface to be measured along the optical axis driven by the drive motor, and 8 represents the process of the image plane changing from blurred to clear and then blurred during the movement of the sample along the optical axis.

[0048] The principle of three-dimensional reconstruction using the system described above is: use a microscope objective with a small depth of field to image the surface to be measured onto the sensor surface of a dynamic vision sensor 1, and use a computer to obtain the brightness change information of each pixel obtained by the sensor. This information is called an event, darkening is called a negative event, and brightening is called a positive event; since the sample moves along the optical axis, the image surface changes from blurry to clear and then to blurry again, and for each pixel, its brightness change is symmetrical, where the center of symmetry is the position where the image is clearest; the distribution of positive and negative events of each pixel on the time axis reflects the change in brightness, so the time series of positive events and the time series of negative events are also symmetrical about the time point when the image is clearest; in this way, each pixel point can be subjected to quasi-focus analysis to obtain the object distance when the image is clearest at each position in the image, thereby obtaining the depth information of the entire surface.

[0049] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:

[0050] Example 1:

[0051] The embodiment of the present invention shows the process of using the method of the present invention to reconstruct a conical three-dimensional surface with black and white stripes. Figure 2 , where the left side is the projection image of the three-dimensional surface in the direction perpendicular to the optical axis, and the right side is the stereoscopic pattern of the three-dimensional surface.

[0052] See also Figure 1 The present invention is based on a three-dimensional reconstruction system using the focus change method. The surface to be measured 6 moves vertically along the optical axis, following a motor 7. The surface is imaged by the optical system consisting of components 2-5 and 9-12 onto the surface of a dynamic vision sensor 1. The brightness changes of each pixel are detected and recorded by a computer system.

[0053] The present invention provides a method for three-dimensional surface reconstruction based on a dynamic vision sensor focus change method. The method mainly includes the following six steps:

[0054] 1) Construction Figure 1 The three-dimensional reconstruction system based on the focus change method is shown.

[0055] 2) Acquisition of original event point cloud data.

[0056] 3) Event rate cubed acquisition.

[0057] 4) Event rate cubic sub-pixel filtering.

[0058] 5) Determine the in-focus time point of each pixel.

[0059] 6) Conversion between the time point of focus and the surface depth.

[0060] 7) Optimization of surface three-dimensional morphology continuity.

[0061] In this embodiment, the specific content of step 2) is:

[0062] 2.1) Figure 2 The surface shown is placed Figure 1 The surface to be measured 6 is moved at a constant speed along the optical axis by a motor. It is worth noting that, unlike the traditional 3D surface reconstruction method based on focus variation, the method proposed in the present invention does not require pausing exposure and shooting during the data acquisition process.

[0063] 2.2) According to the working principle of dynamic vision sensors, when the brightness of any pixel changes beyond a set threshold, the sensor can record this change with microsecond accuracy. This change information is called an event. Brightening is defined as a positive polarity event, and darkening is defined as a negative polarity event:

[0064] in Represents the pixel (x, y) at time point ti The event that occurred; L(x,y,t i ) represents the pixel (x, y) at time point t i The brightness, L(x,y,t i-1 ) represents the pixel (x, y) at the time point t when the last event occurred at the pixel i-1 C represents the brightness threshold.

[0065] 2.3) As the surface follows the motor along the optical axis, points at different locations experience a transition from blur to sharpness and back to blur again. The resulting pixel brightness changes are captured by the dynamic vision sensor and recorded by the computer system to generate raw event data.

[0066] 2.4) These discrete event points are grouped according to pixel coordinates and arranged in chronological order of the events to obtain the grouped and ordered original event point cloud cube data. Figure 3 The star-shaped annotations in the image represent negative events, and the hollow circle annotations represent positive events.

[0067] In this embodiment, the specific content of step 3) is:

[0068] 3.1) The grouped and ordered original event point cloud data obtained in step 2.4) is divided into blocks in the time direction according to a certain time interval Δt. For each pixel (x, y), the total number of positive events and the total number of negative events occurring in each time interval Δt are counted.

[0069] 3.2) Assume that the number of pixels in the x-direction is M and the number of pixels in the y-direction is N. At a given time interval Δt, the entire acquisition period is divided into T equal segments. This yields M*N sequences of positive and negative event rates, each of length T. These M*N sequences of positive event rate are stacked according to the pixel arrangement to form a positive event rate cube of size M*N*T. The negative event rate cube is similarly generated.

[0070] 3.3) According to the requirements of 3D surface reconstruction, it is necessary to calculate the depth information of each pixel. Therefore, without loss of generality, in the following steps 4) to 6), we arbitrarily select a pixel (x p ,y p ) as an example to examine its depth.

[0071] In this embodiment, the specific content of step 4) is:

[0072] 4.1) For the selected pixel (x p ,y p), we obtain its positive event rate sequence and negative event rate sequence in step 5). Since there is a certain amount of noise in the process of generating events by dynamic visual sensors, the obtained sequence needs to be subjected to noise reduction filtering.

[0073] 4.2) The positive and negative event rate sequence obtained in step 3.2) is filtered using wavelet to remove high-frequency noise and retain low-frequency information. The filtered positive and negative event rate sequence is obtained by Figure 4 , where the dotted curve represents the negative event rate curve N[t], and the realization curve represents the positive event rate curve P[t], t∈[t1,t2]. It can be seen that the positive and negative event rate sequences are roughly symmetrically distributed about a certain time t.

[0074] In this embodiment, the specific content of step 5) is:

[0075] 5.1) The process of the surface to be measured moving along the optical axis with the motor is equivalent to the process of focusing by continuously changing the object distance at each point. p ,y p ), the brightness change from defocus to quasi-focus should be consistent with the brightness change from quasi-focus to defocus, which leads to Figure 4 The symmetrical distribution of the positive and negative event rate curves. And we know that the symmetrical center of the positive and negative event rate curves is the time point t when the point is most clearly focused. * .

[0076] 5.2) Initialize the subscripts of the positive and negative event rate sequences P[t] and N[t] obtained in 4.2) to 0.

[0077] P′[t]←P[t+t1];

[0078] N′[t]←N[t+t1];

[0079] P[t] and N[t] are the positive and negative event rate sequences after noise removal, and their domain is [t1, t2]; t1 represents the time point when the motion along the optical axis begins, and t2 represents the time point when the motion along the optical axis ends; after subscript transformation, the domain of P′[t] and N′[t] is [0, t2-t1];

[0080] 5.3) Define the focus error evaluation function as

[0081]

[0082] In this embodiment, k is set to 0.5, mse(a) is the focus error evaluation function defined in the present invention, and for different independent variables a, the smaller the function mse(a) value is, the better the The closer to the time point t when the focus is clearest *a is the optimization parameter for the in-focus error; t is the time series independent variable, representing the subscript of the positive and negative event rate time series; k is the parameter for the preset in-focus time point inspection range, and the value of k can be a real number from 0 to 1. A larger k value means a smaller in-focus time point inspection range.

[0083] 5.4) The symmetric center of the positive and negative event rates, i.e. the time point t when the focus is clearest * It is given by the following formula:

[0084]

[0085] See Figure 5 The horizontal coordinate a corresponding to the lowest point of the focus error evaluation function mse(a) is 132. In this embodiment, the event rate starting time point t1 is 0. Therefore,

[0086] In this embodiment, the specific content of step 6) is:

[0087] 6.1) Since the subscript t in the event rate sequence corresponds to the displacement of the motor, the time point t at which the focus is clearest is obtained. * It can also be compared with the corresponding object distance u * Correspondingly, we get the observation point (x p ,y p )'s three-dimensional coordinate information (x p ,y p ,u * ).

[0088] 6.2) Repeat steps 4) to 6) for all remaining pixels. We will obtain the 3D coordinate information of each pixel and achieve 3D reconstruction of the surface.

[0089] In this embodiment, the specific content of step 7) is:

[0090] 7.1) The depth of the three-dimensional coordinates of each point obtained in step 6.2) comes only from the change in the event polarity rate of the point. When the texture feature of the point is weak, the number of events generated is small, and it is easily disturbed by the noise of the dynamic visual sensor. Therefore, the surface morphology obtained will have discrete peaks and valleys. In order to ensure the continuity of surface reconstruction and improve the accuracy of three-dimensional surface reconstruction by the focus change method, it is necessary to jointly optimize the depth of each point with the event rate information of its surrounding neighborhood. The method for judging peaks and valleys is: if the absolute value of the difference between the depth of a pixel and the mean depth of the 8 pixels in its neighborhood is greater than the set threshold, it is a discrete peak or valley.

[0091] 7.2) Analyze the reconstructed 3D surface obtained in step 6.2) and re-estimate the depth of discrete peaks and valleys in conjunction with their surrounding neighborhood. After finding these pixels using the method for determining peaks and valleys in step 7.1), for each pixel, add the positive and negative event rate sequences of the pixel itself and the 8 neighboring pixels (not denoised in step 3) to form the new event rate sequence for that pixel.

[0092] 7.3) Repeat steps 5) to 6) for the positive and negative event rate sequences obtained in 7.2) to obtain the three-dimensional coordinates (x q ,y q ,u *′ ). See Figure 6 The left side shows the actual three-dimensional topography of the surface to be measured, and the right side shows the three-dimensional topography of the surface to be measured reconstructed using the method of the present invention. It can be seen that the method of the present invention can very accurately reconstruct the three-dimensional topography of the surface to be measured.

[0093] Example 2:

[0094] The embodiment of the present invention demonstrates the experimental process of three-dimensional reconstruction of a surface with a complex texture. The method mentioned in this patent has been described in detail in Example 1. There are no redundant steps in Example 2. Therefore, in Example 2, only the experimental results are given, and the text involving detailed experimental steps is omitted here.

[0095] See Figure 7 , from left to right, show images of the surface under test captured using a traditional CCD / CMOS sensor at different object distances. The surface under test exhibits a slope with a low left edge and a high right edge, along with a complex texture. The experiment in this example was continued by replacing the traditional CCD / CMOS image sensor with a dynamic vision sensor.

[0096] This method mainly includes the following seven steps:

[0097] 1) Construction Figure 1 The three-dimensional reconstruction system based on the focus change method is shown.

[0098] 2) Acquisition of original event point cloud data.

[0099] The original event point cloud data obtained can be found in Figure 8 .

[0100] 3) Event rate cubed acquisition.

[0101] In Example 2, in steps 4) to 6), we also take one pixel as an example to illustrate, and consider the pixel coordinates as (151, 151).

[0102] 4) Event rate cubic pixel filtering

[0103] See Figure 9 , is the positive and negative event rate sequence after pixel filtering, where the solid line is the positive event sequence and the dotted line is the negative event sequence.

[0104] 5) Determine the in-focus time point of each pixel.

[0105] See Figure 10 The filtered positive and negative event rate sequence is calculated using the focus error evaluation function given in the present invention to obtain the focus error sequence. As can be seen from the figure, in this embodiment, when the focus error is minimum, the corresponding horizontal axis coordinate is 193. In this embodiment, the event rate starting time point t1 = 0. The obtained t * =193 / 2+0=96.5ms.

[0106] 6) Conversion between the time point of focus and the surface depth.

[0107] According to the relationship between the displacement and time recorded by the displacement stage, we can get t * The corresponding relative displacement is 78 μm.

[0108] 7) Optimization of surface three-dimensional morphology continuity.

[0109] After performing steps 4) to 6) for each pixel, the depth discontinuity position is optimized for continuity to obtain the reconstructed 3D surface. Figure 11 . It can be seen that Figure 11 The reconstructed three-dimensional surface shows an overall slope shape of "low on the left and high on the right". Figure 7 In contrast, the method of the present invention can reveal subtle bumps and depressions that cannot be displayed by traditional CCD / CMOS sensors, demonstrating the practical application effect of the present invention.

[0110] The above-described embodiments merely illustrate several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. Persons skilled in the art will readily appreciate that variations and modifications may be made without departing from the scope of the present invention, all of which fall within the scope of protection of the present invention.

Claims

1. A surface three-dimensional reconstruction method based on a dynamic visual sensor, characterized in that The following steps are involved: Step 1: Building a 3D reconstruction system based on the focus change method; in the 3D reconstruction system, a dynamic vision sensor is used as an imaging sensor of the surface to be measured; Step 2: Move the surface to be measured along the optical axis. The dynamic vision sensor detects changes in the light intensity incident on the sensor surface during the movement of the surface to be measured. These changes in light intensity are marked as positive events and negative events according to the brightness and darkness, respectively. After the movement is completed, the computer connected to the dynamic vision sensor stores the original event point cloud data generated by this process. These event raw data are arranged in order according to the pixel coordinates (x, y) of the event and the time point t at which the event occurred. The X and Y directions are defined as the directions in which the pixel coordinates x and y increase, respectively, and the T direction is the direction in which the time point t at which the event occurs increases, forming an XYT three-dimensional event point cloud cube. Step 3: Divide the XYT 3D event point cloud cube obtained in step 2 into blocks in the T direction according to the time interval of Δt. For each pixel coordinate (x, y), count the number of positive events and the number of negative events in each time interval of Δt. Finally, a positive event rate cube and a negative event rate cube are obtained. The elements in the cube are integer variables, representing the total number of events generated by each pixel in each time interval. Step 4: Perform wavelet filtering on the positive event rate cube and the negative event rate cube obtained in step 3 along the T direction to obtain the filtered positive and negative event rate cubes; Step 5: Split the filtered positive and negative event rate cubes obtained in Step 4 in the X and Y directions to obtain multiple one-dimensional sequences equal to the number of pixels in the dynamic vision sensor, each sequence representing a one-dimensional sequence of positive and negative event rates for that pixel; for each pixel, construct a focus error evaluation function based on the symmetric distribution of the positive and negative event rate sequences about the focus time point, and obtain the focus time point. The specific meaning of the focus time point is: the time point when the image is clearest during motion along the optical axis; Described step 5 is specifically: 5.1) The process of the surface to be measured moving along the optical axis with the motor is equivalent to the process of focusing by continuously changing the object distance at each point; for the pixel (x p ,y p ), the brightness change from defocus to quasi-focus should be consistent with the brightness change from quasi-focus to defocus, so the positive and negative event rate curves are symmetrically distributed, and the symmetry center of the positive and negative event rate curves is (x p ,y p ) The time point when the focus is clearest t * ; 5.2) Transform the subscripts of the positive and negative event rate sequences P[t] and N[t] to their initial values ​​of 0. P ′ [t]←P[t+t1]; N ′ [t]←N[t+t1]; P[t] and N[t] are the positive and negative event rate sequences after noise removal, and their domain is [t1, t2]; t1 represents the time point when the motion along the optical axis begins, and t2 represents the time point when the motion along the optical axis ends. After subscript transformation, P ′ [t] and N ′ The domain of [t] is [0, t2-t1]; 5.3) Define the focus error evaluation function as Where mse(a) is the defined focus error evaluation function. For the focus error optimization parameter a, the smaller the mse(a) value is, the better the The closer to the time point t when the focus is clearest * ; a is the focus error optimization parameter; t is the time series independent variable, representing the subscript of the positive and negative event rate time series; k is the preset focus time point inspection range parameter, and the value of k can be a real number from 0 to 1. The larger the k value, the smaller the focus time point inspection range; 5.4) The symmetric center of the positive and negative event rates, i.e. the time point t when the focus is clearest * It is given by the following formula: Step 6: Based on the correspondence between the time point and the displacement along the optical axis, the relative depth z is obtained, thereby obtaining the three-dimensional coordinates (x, y, z) of the pixel point; After all pixel points have completed the in-focus position judgment, the three-dimensional topography characteristics of the surface to be measured are obtained.

2. The method for three-dimensional surface reconstruction based on a dynamic visual sensor according to claim 1, characterized in that: The three-dimensional topography surface obtained according to step six is ​​not necessarily smooth due to the presence of noise, and the method further includes step seven of optimizing the continuity of the three-dimensional topography; Step 7: Utilize the event rate information of the pixels surrounding each pixel to jointly optimize the depth z of the pixel so that the final 3D shape is continuous. The joint optimization process includes: searching for pixels in the three-dimensional topography surface obtained in step 6 whose depth coordinate z and the depth mean of the eight pixels in its neighborhood are greater than a given threshold; For all such pixels, the unfiltered positive and negative event rate sequences of the eight pixels in its neighborhood are summed with its own unfiltered positive and negative event rate sequence to obtain the new positive and negative event rate sequence of the pixel; the new positive and negative event rate sequence is processed by wavelet filtering, and steps five and six are repeated to obtain the new three-dimensional coordinates of the pixel.

3. The method for three-dimensional surface reconstruction based on a dynamic visual sensor according to claim 1, characterized in that: Described step 3 is specifically: 3.1) Divide the original event point cloud data obtained in step 2 into blocks in the time direction according to a certain time interval Δt, and count the total number of positive events and negative events that occur in each time interval Δt for each pixel (x, y); 3.2) Assuming the number of pixels in the x-direction is M and the number of pixels in the y-direction is N, and at a given time interval Δt, the entire acquisition period is divided into T equal segments, resulting in M*N positive event rate sequences and negative event rate sequences of length T. These M*N positive event rate sequences and negative event rate sequences of length T are stacked according to the pixel arrangement to form positive event rate cubes and negative event rate cubes of size M*N*T.

4. The method for three-dimensional surface reconstruction based on a dynamic visual sensor according to claim 1, characterized in that: Described step 4 is specifically: For the selected pixel point (x p ,y p ), the positive event rate sequence and negative event rate sequence are obtained in step three. The positive and negative event rate sequences obtained in step three are subjected to wavelet filtering to remove high-frequency noise and retain low-frequency information to obtain filtered positive and negative event rate sequences; after filtering the positive and negative event rate sequences corresponding to all pixels, these sequences are stacked in the order of pixel arrangement to form the filtered positive and negative event rate cubes.

Images