Depth data measurement head and method

Abstract

A depth data measurement head and a measurement method are disclosed. The measurement head comprises a projection device configured to project a line-shaped light to an imaging area; an image sensor configured to image the imaging area, the image sensor comprising a plurality of pixels, each pixel sending a valid signal with a time stamp when the light quantity received by the pixel in a unit time changes over a threshold; and a processor configured to generate a two-dimensional image of a fringe pattern based on the time stamp corresponding to the fringe pattern brightness and the pixel position reporting the valid signal, and to obtain depth information of an object in the imaging area according to the two-dimensional image. The scheme uses a line-shaped light projected by scanning, combines an image sensor capable of independently generating a valid signal when the brightness changes, and uses a time stamp for valid signal and pixel screening, so that imaging can be performed only for the changed valid signal, thereby avoiding the waste of measurement head computing power caused by redundant information processing.

Description

Technical Field

[0001] This disclosure relates to the field of three-dimensional data measurement, and more particularly to a depth data measurement head and method. Background Technology

[0002] A depth data camera is a device that acquires depth information of a target object. These cameras are widely used in fields such as 3D scanning and 3D modeling. For example, an increasing number of smartphones are now equipped with depth cameras for facial recognition. In existing technologies, stripe light coding can be used to achieve high-precision imaging. However, with the ever-increasing frame rate requirements for capturing dynamic depth information of target objects, the characteristic of stripe light coding—requiring the capture of multiple images of different stripes to synthesize a single depth image—places extremely high demands on the computing power and bandwidth of depth cameras, especially their image processors.

[0003] Therefore, a solution is needed that can achieve higher frame rates for depth images with less computational cost. Summary of the Invention

[0004] One technical problem this disclosure aims to solve is to provide a depth data measurement scheme that uses scanned linear light, combined with an image sensor that can independently generate valid signals when the brightness of each pixel changes, and uses timestamps to filter valid signals and pixels. This allows imaging only of valid signals that have changed, thereby avoiding the waste of computing power of the measurement head due to redundant information processing. As a result, the frame rate of depth image acquisition can be further improved based on the existing computing power of the measurement head, or the computing power requirement of the image sensor can be reduced while maintaining the frame rate of acquisition.

[0005] According to a first aspect of this disclosure, a depth data measurement head is provided, comprising: a projection device for projecting linear light moving along a first direction onto an imaging region, wherein the length direction of the linear light is a second direction perpendicular to the first direction; an image sensor for imaging the imaging region, the image sensor comprising a plurality of pixels, each pixel sending a valid signal with a timestamp when the change in light received by the pixel in a unit time exceeds a threshold; and a processor for generating a two-dimensional image of the stripe pattern based on the brightness of the stripe pattern corresponding to the timestamp and the pixel position reporting the valid signal, and obtaining depth information of a subject within the imaging region based on the two-dimensional image.

[0006] Optionally, when the pixel row of the image sensor forms an angle with the second direction, and multiple pixels in the same pixel row send valid signals with the same timestamp, the processor selects pixels from the multiple pixels for generating the two-dimensional image based on the light intensity distribution of the linear light in the first direction.

[0007] Optionally, when multiple pixels in the same pixel row send valid signals with the same timestamp, the processor deletes pixels in that pixel row that are far from the current scanning range for the generation of the two-dimensional image.

[0008] Optionally, when a single pixel sends multiple valid signals with successive timestamps, the processor selects one of the multiple successive timestamps as the timestamp of the valid signal corresponding to that pixel.

[0009] Optionally, the projection device completes one pattern scan within a scanning period T. Within the scanning period T, the projection device adjusts the brightness of the linear light in the moving projection according to the single stripe pattern to be projected. Furthermore, the projection device adjusts the brightness of the linear light according to different stripe patterns in each of the N successive scanning periods T. The processor generates N two-dimensional images corresponding to the N stripe patterns based on the N scanning periods T, and synthesizes a single depth image of the object in the imaging area based on the N two-dimensional images.

[0010] Optionally, the projection device completes one pattern scan within a scanning period T. Within the scanning period T, the projection device alternately adjusts the brightness of the linear light in the moving projection according to the N stripe patterns to be projected. The processor generates N two-dimensional images corresponding to the N stripe patterns based on the scanning period T, and synthesizes a single depth image of the object in the imaging area based on the N two-dimensional images.

[0011] Optionally, within the scanning period T, the projection device adjusts the brightness of the linear light according to N stripe patterns, including: the scanning period T includes multiple projection cycle sub-cycles; within each projection cycle sub-cycle, the projection device alternately projects linear light with brightness corresponding to the positions of the N stripe patterns; and the processor, based on timestamps, correlates the pixel positions of the valid signals with the projection brightness of the linear light and divides them into N categories, each corresponding to a stripe pattern, thereby generating N two-dimensional images corresponding to the N stripe patterns, and synthesizing a single depth image of the object within the imaging area based on the N two-dimensional images.

[0012] Optionally, the N stripe patterns are four sinusoidal four-step phase-shift patterns in which there are no pixels of the same brightness at the same position between adjacent patterns.

[0013] Optionally, the scanning projection range of the projection device is aligned with the imaging area of ​​the image sensor.

[0014] Optionally, the projection device completes one scan within a scanning period T, and the processor assigns brightness corresponding to a predetermined stripe pattern to the pixel position corresponding to the timestamp.

[0015] Optionally, the processor assigns N brightness values ​​corresponding to N predetermined stripe patterns to the pixel positions of the corresponding timestamps, thereby generating N two-dimensional images corresponding to the N stripe patterns, and synthesizing a single depth image of the subject within the imaging area based on the N two-dimensional images.

[0016] According to a second aspect of this disclosure, a depth data measurement method is provided, comprising: projecting a linear light moving along a first direction onto an imaging region, wherein the length direction of the linear light is a second direction perpendicular to the first direction; imaging the imaging region using an image sensor, the image sensor comprising a plurality of pixels, each pixel sending a valid signal with a timestamp when the change in light received by the pixel in a unit time exceeds a threshold; and generating a two-dimensional image of the stripe pattern based on the brightness of the stripe pattern corresponding to the timestamp and the pixel position of the valid signal, and obtaining depth information of a subject within the imaging region based on the two-dimensional image.

[0017] Optionally, the method further includes aligning the scanning projection range of the projection device with the imaging area of ​​the image sensor.

[0018] Optionally, projecting linear light moving along a first direction onto the imaging area includes: projecting the linear light with brightness corresponding to different stripe patterns in N successive scan cycles T; generating a two-dimensional image of the stripe pattern based on the linear light projection brightness corresponding to the timestamp and the pixel position of the reported valid signal; and obtaining the depth information of the subject within the imaging area based on the two-dimensional image includes: generating N two-dimensional images corresponding to N stripe patterns based on the N scan cycles T; and synthesizing a single depth image of the subject within the imaging area based on the N two-dimensional images.

[0019] Optionally, projecting linear light moving along a first direction onto the imaging area includes: alternately projecting linear light with brightness corresponding to the positions of N stripe patterns to complete the projection of N stripe patterns within one scan cycle T; generating a two-dimensional image of the stripe patterns based on the linear light projection brightness corresponding to the timestamp and the pixel position of the reported valid signal; and obtaining the depth information of the subject within the imaging area based on the two-dimensional image includes: generating N two-dimensional images corresponding to the N stripe patterns based on the one scan cycle T; and synthesizing a single depth image of the subject within the imaging area based on the N two-dimensional images.

[0020] Optionally, projecting linear light moving along a first direction onto the imaging area includes: completing one scan within a scan cycle T; generating a two-dimensional image of the stripe pattern based on the linear light projection brightness corresponding to the timestamp and the pixel position of the reported valid signal; and obtaining the depth information of the subject within the imaging area based on the two-dimensional image includes: assigning N brightness values ​​corresponding to N predetermined stripe patterns to the pixel positions corresponding to the timestamps, thereby generating N two-dimensional images corresponding to the N stripe patterns, and synthesizing a single depth image of the subject within the imaging area based on the N two-dimensional images.

[0021] Therefore, the depth data measurement scheme of the present invention, by scanning the projected linear light, combined with an image sensor that can independently generate effective signals when the brightness of each pixel changes, and using timestamps to filter effective signals and pixels, can image only the effective signals that have changed, thereby avoiding the waste of computing power of the measurement head due to redundant information processing, and can obtain multiple stripe patterns required for depth data calculation based on a single scan by corresponding positions. Attached Figure Description

[0022] The above and other objects, features and advantages of this disclosure will become more apparent from the more detailed description of exemplary embodiments thereof taken in conjunction with the accompanying drawings, wherein like reference numerals generally denote like parts.

[0023] Figure 1 An example of time-coded striped structured light is shown.

[0024] Figure 2 A schematic diagram illustrating the coding based on five patterns is shown.

[0025] Figure 3 An example of a binocular imaging depth data measurement head is shown.

[0026] Figure 4 A schematic diagram of the composition of a depth data measurement head according to an embodiment of the present invention is shown.

[0027] Figure 5 An example of a four-step phase-shifting sinusoidal fringe pattern is shown.

[0028] Figure 6 Examples of linear light projection and imaging are shown, both when the subject is absent and when it is present in the imaging area.

[0029] Figure 7 This illustrates a cyclic sub-period in a four-step phase-shift fringe pattern. The brightness distribution of each pattern within the image.

[0030] Figure 8A -B shows Figure 4 An example of magnified operation of the projection device shown.

[0031] Figure 9 A simplified perspective diagram of the projection device used in this invention is shown.

[0032] Figure 10 A schematic flowchart of a depth data measurement method according to an embodiment of the present invention is shown. Detailed Implementation

[0033] Preferred embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0034] Structured light technology actively projects a known pattern onto an imaging area. The deformation of the structured light pattern on the surface of the subject in the imaging area can be used to calculate the surface depth information of the subject in a vision system. Structured light needs to be encoded, and one widely used type of structured light is striped coded structured light.

[0035] As the name suggests, stripe-coded structured light is structured light that encodes based on the distribution of stripes. It includes time-coded stripe structured light, Gray code stripe structured light, and four-step phase-shifting stripe structured light. The principles of various types of coded stripe structured light will be explained below with reference to the accompanying diagrams.

[0036] Figure 1 An example of time-coded striped structured light is shown. The figure briefly illustrates the encoding principle of striped structured light using a two-grayscale five-bit binary time-coding method. As shown, the projection device can sequentially project five patterns onto the object being measured in the imaging area. These five patterns divide the projection space into 32 regions using two grayscale levels, one dark and one dark. The first projected pattern includes only two regions, one dark and one bright; the second pattern includes four regions from left to right: dark-bright-dark-bright; and so on, until the fifth pattern includes 32 regions formed by alternating dark and bright areas.

[0037] In each pattern, each region corresponds to its own projection angle. It can be assumed that bright regions correspond to code "1" and dark regions correspond to code "0". The code values ​​of a point on the object in the projection space in the five code patterns are combined in the projection order to obtain the region code value of that point. This determines the region where the point is located and then decodes it to obtain the scanning angle of that point. Figure 2A schematic diagram illustrating encoding based on five patterns is shown. As illustrated, each pixel acquires an encoding of either "1" or "0" in the five images, thus each pixel receives a five-bit binary encoded value. In a monocular imaging system (i.e., using only one image sensor to image these five patterns), the decoding process may include comparison and calculation with the encoded value of a reference image. In a binocular imaging system, the decoding process can be simplified by directly matching the encoded values ​​of each point in the first and second image sensors.

[0038] It should be understood that to improve matching accuracy, the number of projected patterns in the time-coded image can be increased, for example, to 6, 7, or even more. For instance, in binocular imaging applications, 10 time-coded stripe patterns mean that each pixel in each left and right image frame contains 10 region-coded values ​​that are either 0 or 1, thereby enabling higher precision (e.g., pixel-level) for left-right image matching. With the projection rate of the projection device remaining constant, compared to... Figure 1 The example of five coded patterns and ten patterns is equivalent to achieving higher-precision image matching at a higher time domain cost (twice as much).

[0039] Figure 3 An example of a binocular imaging depth data measurement head is shown. For example... Figure 3 As shown, the depth data measurement head 300 includes a projection device 310 and two image sensors 320_1 and 320_2.

[0040] The projection device 310 is used to scan and project structured light with stripe coding onto the imaging area. For example, in five consecutive projection cycles, the projection device 310 can project such... Figure 1 The five patterns shown can be used to generate depth data. Images 320_1 and 320_2, which can be referred to as first and second image sensors respectively, have a predetermined relative positional relationship and are used to capture images of the imaging area to obtain first and second two-dimensional image frames under structured light illumination, respectively. For example, when projected onto the projection device 310... Figure 1 In the case of the five patterns shown, the first and second image sensors 320_1 and 320_2 can respectively image the areas projected with these five patterns (e.g., within five synchronized image frame imaging cycles) within five image frame imaging cycles. Figure 3 Imaging is performed on the imaging plane and the area within a certain range before and after it.

[0041] like Figure 3 As shown, in order to project stripe patterns using different codes, the projection device 310 can... z Projected in the direction (i.e., toward the imaging area) yLinear light extending in a certain direction. In different embodiments, the projection of the aforementioned linear light is preferably pre-shaped (i.e., the emitted light itself is linear light), or it can be... y A light spot moving in a certain direction (i.e., a linear beam obtained from scanning). The projected linear beam can... x It moves continuously in the direction to cover the entire imaging area. Figure 3 The lower perspective view of the imaging area provides a more easily understood illustration of the scanning of linear light.

[0042] For ease of explanation, the direction of the light emitted from the measuring head can be defined as follows: z Direction, the vertical direction of the shooting plane is y Direction, horizontal direction is x Direction. In the subsequent description of this invention, the direction in which the linear light scan proceeds can be described, i.e. Figure 3 The horizontal direction shown in the figure ( x The direction is considered the first direction, the direction in which the linear light extends, i.e. Figure 3 The vertical direction shown in the figure ( y The direction is considered as the second direction. Therefore, the fringe structured light projected by the projection device can be in... y Linear light extending in the direction (second direction) x The result of moving in the direction (first direction) (accompanied by a change in the brightness of the linear light itself). Although in other embodiments, it may also be for the horizontal x Linear light extending in the direction y The striped structured light obtained by moving in a certain direction is synchronized and imaged, but vertical striped light is still preferred to be used in the present invention.

[0043] Furthermore, the measuring head 300 also includes a synchronization device 330. The synchronization device 330 is connected to the projection device 310 and the first and second image sensors 320_1 and 320_2, respectively, to achieve synchronization among the three.

[0044] In one embodiment, the first and second image sensors 320_1 and 320_2 are global image sensors. In order to achieve imaging of the complete projected stripe pattern, the synchronization device 330 needs to enable the first and second image sensors 320_1 and 320_2 to start exposure when each image begins scanning and end exposure after the image scan is completed.

[0045] In another embodiment, the first and second image sensors 320_1 and 320_2 are single-row exposure-controllable image sensors. In this case, as... Figure 3As shown, the synchronization device 330 enables the area scanned by the projection device 310 to correspond to the exposure activation of the imaging column areas of the first and second image sensors 320_1 and 320_2. Due to the presence of the object within the imaging area, the projected stripes will undergo horizontal displacement within a certain range, therefore the imaging column activated each time needs to have a certain width. Thus, the range of pixel columns imaging at each moment can be controlled by utilizing the one-dimensional characteristics of the stripe image, thereby reducing the adverse effects of ambient light on the measurement results. To further reduce the influence of ambient light, the projection device is particularly suitable for projecting light that is not easily confused with ambient light, such as infrared light.

[0046] Whether it is global exposure or column exposure, the first and second image sensors 320_1 and 320_2 simultaneously capture five groups of a total of 10 images to achieve time encoding of each pixel and left-right matching, thereby obtaining a single depth image of the subject in the imaging space.

[0047] To further enhance the ability to capture the movement of the subject, if we utilize... Figure 3 The depth imaging scheme shown requires further improvement in the frame rate of the image sensor. For example, when generating 30 frames of depth images per second, the image sensor used for five-stripe pattern synthesis needs a frame rate of 150 frames per second; and when generating 60 frames of depth images per second, the frame rate needs to be as high as 300 frames per second. Such frame rates place extremely high demands on the bandwidth and computing power of the image sensor, thus limiting the development of depth data measurement equipment.

[0048] To address this, the present invention proposes a depth data measurement scheme. This scheme uses linear light projected by scanning, combined with an image sensor that can independently generate effective signals when the brightness of each pixel changes, and uses timestamps to filter effective signals and pixels. It can image only the effective signals that have changed, thereby avoiding the waste of computing power of the measurement head due to redundant information processing. Thus, it can further improve the frame rate of depth image acquisition (referring to the frame rate of the synthesized depth image) on the basis of the existing computing power of the measurement head, or reduce the demand for computing power while maintaining the frame rate of acquisition.

[0049] Figure 4 A schematic diagram of the composition of a depth data measurement head according to an embodiment of the present invention is shown. As shown, the measurement head 400 includes a projection device 410, an image sensor 420, and a processor 430.

[0050] Here, with Figure 3 Similarly, the projection device 410 is used to project along a first direction (as shown in the figure) onto the imaging area. xA linear light beam moving in a direction, wherein the length direction of the linear light beam is a second direction perpendicular to the first direction (illustrated as follows). y (Direction). As shown in the figure, the linear light generated by the projection device 410 can be emitted in the z-direction (and its scanning range, for example, within ±10° of the z-direction) and projected onto the imaging area of ​​the image sensor. In a preferred embodiment, before obtaining depth data of the subject within the imaging area, an alignment operation can be performed first, for example, aligning the scanning projection range of the projection device 410 with the imaging area of ​​the image sensor 420. The alignment operation can be performed using a calibration plane, for example, without placing a subject, using a relatively flat plane within the imaging area (e.g., a wall perpendicular to the z-direction) as a calibration plane to perform stripe image scanning and image imaging using the image sensor. The scanning angle of the projection device 410 and / or the imaging angle of the image sensor 420 can be adjusted so that the image sensor can image a complete projected image that substantially covers the entire imaging range of the image sensor.

[0051] In this invention, the image sensor 420 and Figure 3 The image sensor shown is different; it is not a conventional global sensor or a rolling shutter sensor. Although used to image an imaging area, each pixel of the image sensor 420 can send a timestamped valid signal when the amount of light received exceeds a threshold per unit time.

[0052] Here, the unit time can be a very small time interval, such as 1 μs, and preferably, each pixel in the same image sensor can be calculated using the same start and end times. The threshold can be set according to the application scenario, but it needs to be within the upper and lower limits achievable by the image sensor.

[0053] When the image sensor 420 has, for example, a 960x720 pixel array, each of these 691,200 pixels can independently generate and transmit a valid signal with a timestamp. For ease of explanation, the following will use a four-step phase-shift fringe coding example. Figure 5 An example of a four-step phase-shift sinusoidal fringe pattern is shown. As illustrated, the four-step phase-shift sinusoidal encoding comprises four patterns that differ from each other by π / 2, and each pattern includes the same number of cyclic sub-cycles within one scan period T. (will be combined) Figure 7 (details), such as Figure 5 The 16 sub-cycles are shown in the diagram. Here, the projection is used. Figure 5 Taking pattern 1 as an example. tWhen 0=0, the linear light projected by the projection device 410 begins to illuminate the leftmost side of the imaging area. Since there is no subject obstructing the leftmost side of the imaging area, at this time, after a unit time, that is, t At a time of 1 = 1 μs, the first pixel column of the image sensor 420 is... t 0 to t The amount of light received per unit time changes from 0 to 1. a 1, a 1 is greater than the threshold a If 0, then each of the 720 pixels in the first pixel column of the image sensor 420 generates a valid signal with a timestamp (e.g., represented as on@ starting at a unit time). t 0, or represented as on@ starting from a unit of time. t 1. For ease of expression, the following will uniformly use the end time of the unit time as the representation method, that is, the valid signal meter is on@ at this time. t 1). Since the linear light is in motion (e.g., moving at a constant speed in the first direction), if the scanning speed of the linear light is set appropriately, the linear light can be made to... t 0 to t 1 scans the imaging range of the first column of pixels exactly within a unit of time, and in t At time 1 = 1 μs, scanning of the imaging area corresponding to the second column of pixels (assuming no subject occlusion) begins. Similarly, after reaching the unit time, i.e., t At time 2 = 2μs, the second pixel column of the image sensor 420 is... t 1 to t The amount of light received per unit time changes from 0 to 2. a 2, a 2 is also greater than the threshold. a If 0, then each of the 720 pixels in the second pixel column of the image sensor 420 generates a valid signal with a timestamp (e.g., represented as on@ starting at a unit time). t 2) Since the linear light is in motion, properly setting the scanning speed of the linear light can make the linear light... t 1 to t 2 scans the imaging range of the second column of pixels exactly within a unit of time, and in t The scanning of the imaging area corresponding to the third column of pixels (assuming no subject occlusion) begins at time 2 = 2 μs. This process can be repeated, and eventually... t 959 At time 959μs, scanning begins on the imaging area corresponding to the last column (i.e., the 960th column) of pixels (assuming no subject occlusion), and after reaching the unit time, i.e. t 960The scanning of the imaging area corresponding to the 1000th column of pixels is completed at 960μs = 0.96ms. The 960th pixel column of the image sensor 420 is in... t 959 arrive t 960 The amount of light received per unit time changes from 0 to a 960 , a 960 Also greater than the threshold a If 0, then each of the 720 pixels in the 960th pixel column of the image sensor 420 generates a valid signal with a timestamp (e.g., represented as on@ starting at a unit time). t 960 ).

[0054] Therefore, the processor 430 can generate a two-dimensional image of the stripe pattern based on the brightness of the stripe pattern corresponding to the timestamp and the pixel position of the reported valid signal, and obtain the depth information of the object in the imaging area based on the two-dimensional image.

[0055] Similarly, in the example above, t 0=0 to t 960 During a 0.96 ms time interval of 1000 μs, the projection device 410 completes one scan of the linear light, for example, a scan from left to right, which can correspond to... Figure 4 A single scan is a sequence of scans from the start position to the end position.

[0056] When the scanning speed and scanning position of the projection device correspond well with the imaging area of ​​each pixel column of the image sensor 420 as described in the example above, if there is no imaging object in the imaging area, but only a wall perpendicular to the projection direction, such as... Figure 4 The calibration plane shown indicates that the processor 430 can... t 1 = 1 μs to t 960 In 960 time points of 960 μs, at each time point, a valid signal of one pixel column from the image sensor 420 is received. For example, in t At time 1, a valid signal for the first pixel column is received. t At time 2, a valid signal for the second pixel column is received, and so on, until finally at... t 960 The processor receives a valid signal from the last pixel column at any given time. Based on this, it can then reconstruct the data according to the corresponding timestamps. Figure 5 The pattern shown in Pattern 1. As shown in the example above, the valid signal timestamp can be... t 1 pixel assigned tThe brightness of the stripe pattern at time 1 is used to determine the effective signal timestamp. t 2 pixels are given t The brightness of the stripe pattern at time 2 is calculated, and so on, until a two-dimensional image of the stripe pattern is obtained.

[0057] When the imaging area includes the subject, the linear light projected will change due to the reflection of the subject. Therefore, the number of pixels reporting valid signals at the same timestamp will not be limited to a single pixel column.

[0058] like Figure 4 As shown, when the imaging area includes a spherical subject, the projected linear light will be deformed. These deformations will be reflected in the change of the pixel position that reports the valid signal, thereby allowing the depth information of the subject to be determined.

[0059] For ease of understanding, Figure 6 Examples of linear light projection and imaging are shown, both when the subject is absent and when it is present in the imaging area. Figure 6 The upper part shows the pattern to be projected by the projection device 410 (or, in different embodiments, the pattern may be restored by the processor 430). Figure 6 The lower part shows the pixel position at a certain moment during the projection process when a linear light sweeps across the area, causing a valid signal to be reported.

[0060] Specifically, Figure 6 The left side shows a case where there is no subject in the imaging area (e.g., only a calibration wall perpendicular to the z-direction), while the right side shows a case where the imaging area includes one subject (e.g., Figure 4 The case of the sphere shown.

[0061] In one embodiment of the invention, the projection device 410, for example, from... Figure 4 During a single scan from the start to the end position, as shown, the intensity of the linear light can be continuously varied, thereby achieving the following within one scan cycle T: Figure 6 The projection of the striped structured light pattern shown in the upper left corner. In the projection device 410 as described above... t 0=0 to t 960 In the example where a linear light scan is completed within a 0.96ms time interval of 960μs (for ease of explanation, it can be assumed that the projection device needs 40μs to prepare for the next scan after completing one scan, so a scan period T = 960μs + 40μs = 1000μs = 1ms can be set), t 480 At any moment, the projection device 410, as Figure 6The lower left image shows the scanning reaching the center of the imaging area, where the linear light is at its brightest. At this point, if there is no subject in the imaging area, the 500th pixel corresponding to the image sensor 420 experiences a change in light intensity per unit time and reports it on@. t 480 If the imaging region includes, for example, Figure 6 When photographing a spherical subject as shown on the right, then... t 480 At any given moment, not all pixels in the 480th column of the image sensor 420 experience a change in light intensity within that unit of time; only the upper and lower pixels in the 480th column that are not obscured by the subject undergo a change in light intensity. t 480 Report; Reporting in the middle of the pixel array on@ t 480 The pixels are located before the 480th column of pixels. Therefore, the depth data measurement head of the present invention can align the projected brightness with the pixels reporting valid signals based on timestamps, thereby enabling the processor 430 to reconstruct images similar to those in the imaging area when a spherical subject is included. Figure 6 The image shown in the upper right corner is a two-dimensional striped image containing depth information of the subject.

[0062] As previously described, the processor 430 can be used to generate a two-dimensional image of the stripe pattern based on the stripe pattern brightness corresponding to the timestamp and the pixel position reporting a valid signal, and to obtain the depth information of the object within the imaging area based on the two-dimensional image. In some embodiments, the stripe pattern brightness corresponding to the timestamp can be the brightness of the stripe image actually projected by the projection device 410 at the corresponding timestamp. In other embodiments, since each pixel of the image sensor 420 can only report a valid signal when the change in luminous flux per unit time is greater than a threshold (in other words, the valid signal itself cannot characterize the light intensity of the projected linear light itself), the stripe pattern brightness corresponding to the timestamp may not be the brightness of the stripe image actually projected by the projection device 410 at the corresponding timestamp (in this case, the projection device 410 only needs to ensure that the linear light sweeping across will cause the corresponding pixel to report a valid signal), but only the brightness corresponding to the predetermined stripe pattern.

[0063] First, an embodiment in which the brightness of the stripe pattern corresponding to the timestamp is more easily understood is described, which is the brightness of the stripe image actually projected by the projection device 410 at the corresponding timestamp.

[0064] As mentioned earlier, striped structured light coding typically requires projecting different stripe patterns multiple times and extracting depth information of the subject based on the pixel encoding of multiple patterns. For example, in Figure 1 and Figure 2In the example of time-coded fringe structured light shown, five different fringe patterns need to be projected sequentially to generate a depth image. However, Figure 5 In the example of the four-step phase-shifting sinusoidal fringe structured light shown, four different fringe patterns need to be projected successively to generate a depth image.

[0065] Therefore, in one embodiment of the present invention, the projection device 410 can be used to complete one pattern scan within one scanning cycle T. Within the scanning cycle T, the projection device adjusts the brightness of the linear light in the moving projection according to the single stripe pattern to be projected. Furthermore, the projection device adjusts the brightness of the linear light according to different stripe patterns in each of the N successive scanning cycles T. The processor generates N two-dimensional images corresponding to the N stripe patterns based on the N scanning cycles T, and synthesizes a single depth image of the object in the imaging area based on the N two-dimensional images.

[0066] Similarly Figure 5 Taking the pattern shown as an example, the projection device 410, for example, from... Figure 4 During the first scan from the start to the end position, pattern 1 can be projected; during the second scan, pattern 2 can be projected; during the third scan, pattern 3 can be projected; and during the fourth scan, pattern 4 can be projected. Each projection corresponds to a projection period T. At T=1ms, projecting four patterns takes 4ms. During each projection, the linear light can change its brightness according to the corresponding fringe image, combined with an appropriate projection speed, and assign brightness values ​​to the corresponding pixel positions based on the effective time timestamp. Thus, after four projections, the processor synthesizes four two-dimensional images, and then synthesizes a single depth image of the object within the imaging area based on these four two-dimensional images.

[0067] In another embodiment of the present invention, multiple patterns can be projected in a single scan. In this case, the projection device completes one pattern scan within a scan period T. Within the scan period T, the projection device alternately adjusts the brightness of the linear light in the moving projection according to the N stripe patterns to be projected. The processor generates N two-dimensional images corresponding to the N stripe patterns based on the scan period T, and synthesizes a single depth image of the object in the imaging area based on the N two-dimensional images.

[0068] The following will combine Figure 7 The projection and imaging schemes for scanning one pattern at a time and scanning multiple patterns at a time will be explained separately. Specifically, Figure 7This illustrates a cyclic sub-period in a four-step phase-shift fringe pattern. The brightness distribution of each pattern within the figure. As shown in the figure, patterns 1-4 follow the following brightness distribution: P1=Q / 2 sin θ +Q / 2 P2 = -Q / 2 cos θ +Q / 2 P3 = -Q / 2 sin θ +Q / 2 P4 = Q / 2 cos θ +Q / 2 Here, Q can be considered as the brightness of the brightest part of the pattern. θ That is, the four-step phase shift angle value corresponding to time t.

[0069] exist Figure 5 In the example shown, each scan pattern consists of 16 cyclic sub-cycles. When the imaging area perfectly corresponds to the projection area, it means that the 960 columns of pixels in the image sensor correspond to 16 cyclic sub-cycles. Each cycle sub-cycle This corresponds to 80 columns of pixels. Based on the example above, this means the linear light needs 80μs to scan. Figure 7 The diagram shows a sub-cycle. One of patterns 1-4. When Figure 7 The first cycle of the projection is shown. When there are stripes, the stripes begin to correspond to... t 0, the end of the stripe corresponds to t 80 .

[0070] In the example where the projection device 410 is used to complete one pattern scan within one scanning cycle T, the projection device 410 can first project... Figure 5 Pattern 1 shown, i.e., repeated projection Figure 7 A single cyclic sub-cycle of pattern 1 shown The stripes are repeated 16 times, taking 1 ms. Accordingly, the image sensor 420 records a timestamped valid signal during the first scan cycle, allowing the processor 430 to assign brightness information to each pixel position using the timestamps and thereby generate the first two-dimensional image obtained under pattern 1 scanning. Subsequently, the projection device 410 can project... Figure 5 Pattern 2 shown, i.e., repeated projection Figure 7 A single cyclic sub-period of pattern 2 shown The stripes are repeated 16 times, taking 1 ms. Accordingly, the image sensor 420 records a timestamped signal during the second scan cycle, allowing the processor 430 to assign brightness information to each pixel position using the timestamps and generate the second two-dimensional image obtained under pattern 2 scanning. Subsequently, the projection device 410 can project... Figure 5 Pattern 3 shown, i.e., repeated projection Figure 7 A single cyclic sub-cycle of pattern 3 shown The stripes are repeated 16 times, taking 1 ms. Correspondingly, the image sensor 420 records a timestamped valid signal during the third scan cycle, allowing the processor 430 to assign brightness information to each pixel position using the timestamps and generate the third two-dimensional image obtained under pattern 3 scanning. Finally, the projection device 410 can project... Figure 5 Pattern 4 shown, i.e., repeated projection Figure 7 A single cyclic sub-period of pattern 4 shown The stripes are repeated 16 times, taking 1 ms. Correspondingly, the image sensor 420 records a valid signal with a timestamp during the fourth scan cycle, allowing the processor 430 to assign brightness information to each pixel position using the timestamps and generate the fourth two-dimensional image obtained under pattern 4 scanning. In other words, during t0~t... 960 At each moment, each pixel in the image sensor completes its corresponding valid signal report, in the subsequent three scan cycles from t0 to t1. 960 At any given moment, each pixel in the image sensor completes its corresponding valid signal report. The valid signal reports between different scan cycles are used to generate two-dimensional images for their respective cycles. Thus, after four scans taking 4ms, four two-dimensional images are obtained. The processor 430 can then process these four two-dimensional images to obtain depth data information of the object within the imaging area.

[0071] In the example where the projection device 410 is used to complete multiple pattern scans within one scan cycle T, it involves projection with varying brightness at the same location. For example... Figure 7 As shown, in t When 0=0, P1=Q / 2; P2=0; P3=Q / 2; P4=Q. Therefore, when the image sensor reports a valid signal unit time of 1μs as described above, four intensity changes are required at the initial position to complete the transformation projection of the four patterns. Therefore, it is possible to... t 0_1 When =0, the intensity of the linear light is Q / 2; t 0_2 At 1μs, the intensity of the linear light is 0; t 0_3 At 2μs, the intensity of the linear light is Q / 2;t 0_4 At 3μs, the intensity of the linear light is Q. t When 1 = 4 μs (corresponding to) θ =2π / 80), P1= 0.54 Q; P2 = 0.02 Q; P3 = 0.46 Q; P4 = 0.998 Q. Therefore, it is possible to t 1_1 At 4μs, the intensity of the linear light is 0.54. Q; In t 0_2 At 5μs, the intensity of the linear light is 0.02. Q; In t 0_3 At 6μs, the intensity of the linear light is 0.46. Q; In t 0_4 At 7μs, the intensity of the linear light is 0.998. Q. Here, a sub-cycle of pattern intensity transformation every 4 μs can be called a projection cycle sub-cycle. Within one projection cycle sub-cycle, the projection device switches between N intensities corresponding to the same position in N patterns. Thus, in t 0 ~ t 960 Each moment actually includes 4 sub-moments, and the projection of each of the 4 patterns is completed 960 times within 3840μs (=960μs x 4). In this implementation, the projection device 410 alternately projects linear light with brightness corresponding to the positions of the N stripe patterns. The processor, based on timestamps, corresponds the pixel positions of the effective signals with the projection brightness of the linear light and divides them into N categories corresponding to a stripe pattern (for example, timestamps divisible by 4 out of the 3840 timestamps are considered to correspond to the first pattern, timestamps with a remainder of 1 when divided by 4 are considered to correspond to the second pattern, timestamps with a remainder of 2 when divided by 4 are considered to correspond to the third pattern, and timestamps with a remainder of 3 when divided by 4 are considered to correspond to the fourth pattern). This generates N two-dimensional images corresponding to the N stripe patterns, and synthesizes a single depth image of the subject within the imaging area based on the N two-dimensional images.

[0072] In one embodiment, the linear projection position of the projection device can remain constant within each projection cycle sub-cycle. For example, the projection device can remain stationary for 4 μs at the corresponding position for each pixel column to complete the switching of brightness corresponding to the four patterns, then move to the position corresponding to the next pixel column and remain stationary for another 4 μs, and so on. In another embodiment, within one projection cycle sub-cycle, the linear projection distance of the imaging area of ​​the projection device does not exceed the corresponding width of a single column of pixels. In other embodiments, different pixel columns can be selected to correspond to different patterns (in which case precise calibration of the scanning position and imaging area is required).

[0073] In the previous scheme of scanning multiple patterns, the N stripe patterns were patterns in which adjacent patterns did not have pixels of the same brightness at the same position, for example... Figure 5 and Figure 7 The four sine wave four-step phase shift patterns shown are illustrated.

[0074] As mentioned earlier, since the effective signals reported by each pixel of the image sensor 420 cannot characterize the light intensity of the currently projected linear light, the brightness of the stripe pattern corresponding to the timestamp may not be the brightness of the stripe image actually projected by the projection device 410 at the corresponding timestamp. In this case, the projection device 410 can complete one scan within one scan cycle T, and the processor 430 can assign brightness corresponding to a predetermined stripe pattern to the pixel position of the corresponding timestamp. Specifically, the processor can assign N brightness values ​​corresponding to N predetermined stripe patterns to the pixel position of the corresponding timestamp, thereby generating N two-dimensional images corresponding to N stripe patterns, and synthesizing a single depth image of the object in the imaging area based on the N two-dimensional images.

[0075] Here, also based on Figure 5 and Figure 7 Taking the acquisition of depth data of a photographed object using four sinusoidal four-step phase-shift patterns as an example, assuming that the projection device 410 completes one scan from the start position to the end position within 960 μs, the projection device 410 can perform linear light scanning at any brightness that allows the scanned pixels of the image sensor 420 to generate valid signals. For example, the projection device 410 projects linear light at a constant brightness above a threshold brightness, and each pixel of the image sensor 420 can report a valid signal with a timestamp when it is scanned by the linear light. Subsequently, the processor 430 can perform depth data processing based on the stripe coding algorithm (e.g., based on...). Figure 1 The time code shown Figure 5The corresponding two-dimensional image is constructed using the pattern corresponding to the sinusoidal wave four-step phase shift code (shown, or the rectangular wave four-step phase shift code or Gray code, not shown). When using the 960x720 pixel image sensor 420, the processor 430 can obtain the timestamp from... t 0 to t 960 The effective signals from each pixel. Therefore, the processor 430 will use a preset pattern (e.g., Figure 5 The processor 430 divides pattern 1) of the four-step phase-shifting pattern into 960 equal parts, obtains 960 brightness distribution values ​​in the first direction, and assigns these brightness values ​​to the pixel positions corresponding to the timestamps, thereby constructing the first two-dimensional image. Subsequently, the processor 430 divides pattern 2 of the four-step phase-shifting pattern into 960 equal parts, obtains 960 brightness distribution values ​​in the first direction, and assigns these brightness values ​​to the pixel positions corresponding to the timestamps, thereby constructing the second two-dimensional image; divides pattern 3 of the four-step phase-shifting pattern into 960 equal parts, obtains 960 brightness distribution values ​​in the first direction, and assigns these brightness values ​​to the pixel positions corresponding to the timestamps, thereby constructing the third two-dimensional image; divides pattern 4 of the four-step phase-shifting pattern into 960 equal parts, obtains 960 brightness distribution values ​​in the first direction, and assigns these brightness values ​​to the pixel positions corresponding to the timestamps, thereby constructing the fourth two-dimensional image. Therefore, by using the timestamped valid signals reported by each pixel obtained through a single scan by the projection device, the processor 430 can construct an arbitrary two-dimensional image (typically a set of stripe images following a certain encoding algorithm, for example) based on the brightness at the corresponding position in the predetermined image. Figure 5 The four sinusoidal four-step phase-shift images shown can be used to calculate the depth data of the subject, thereby greatly improving the calculation efficiency.

[0076] Furthermore, it should be understood that, for ease of understanding, the example above assumes an ideal linear light (i.e., an ideal "line segment" without width) that sweeps exactly across the width of a column of pixels within a unit interval. However, in actual operation, linear light extending along the second direction often has a certain width (in the first direction) and may also have a non-uniform brightness distribution in the width direction. Since each pixel of the image sensor reports a valid signal when the amount of light passing through it exceeds a threshold per unit time (either an increase or decrease in light passing through it), changes in light intensity along the trailing edge of the linear light or even within the linear light itself may cause a pixel to report a valid signal. Therefore, when the pixel row of the image sensor forms an angle with the second direction, preferably perpendicular, i.e., parallel to the first direction, and multiple pixels in the same pixel row send valid signals with the same timestamp, the processor 430 selects pixels from the multiple pixels for the generation of the two-dimensional image based on the light intensity distribution of the linear light in the first direction. For example, the processor 430 can directly select the valid signal reported by the leading edge of the linear light and ignore the subsequent valid signals (for example, when the linear light causes multiple adjacent pixels on the same pixel row to report valid signals with the same timestamp, only the valid signal reported by the first pixel is retained).

[0077] Furthermore, when multiple pixels in the same pixel row send valid signals with the same timestamp, the processor deletes pixels in that row that are far from the current scan range for use in generating the two-dimensional image. For example, when linear light just enters one side of the image sensor's imaging area, valid signals reported by pixels from the other side can be ignored, thereby avoiding the influence of noise signals.

[0078] Furthermore, when a single pixel sends multiple valid signals with successive timestamps, the processor selects one of the successive timestamps as the timestamp of the corresponding valid signal for that pixel. For example, depending on the specific algorithm, either the first timestamp or the last timestamp can be selected. This is because in actual operation, linear light is not an ideal line; for example, it has width and brightness variations.

[0079] exist Figure 4 In the example shown, the depth data measurement head of the present invention may include only a single image sensor 420. In other embodiments, the depth data measurement head of the present invention may include two image sensors, each of which includes multiple pixels, and each pixel sends a valid signal with a timestamp when the change in the amount of light received per unit time exceeds a threshold, thereby realizing binocular setup.

[0080] In addition, the projection device of the present invention can achieve linear light scanning projection by rotating the rotating mechanism. Figure 8A -B shows Figure 4 An enlarged operation example of the projection device shown. Specifically, in the projection device 410, the laser generator (such as...) Figure 8A The laser emitted by the laser generator 411 (shown in detail in section B) is reflected by a reflection mechanism (such as...) Figure 8A The reflective mechanism 412, as detailed in section -B, scans the projection onto the calibration plane ( Figure 4 The gray area in the image is used to focus on the subject in the imaging area (e.g., ...). Figure 4 Active structured light projection is performed on the sphere (in the image). In practical applications, the laser generator 411 is used to generate linear infrared laser. In different embodiments, the laser generator 411 can perform high-speed switching to scan the projected structured light with alternating bright and dark areas corresponding to the stripe coding, wherein high-speed switching may include high-speed switching of the laser generator and high-speed coding switching; the laser generator 411 may also maintain a constant brightness for scanning without brightness switching (in which case the processor 430 assigns brightness values ​​according to predetermined positions as described above).

[0081] In one embodiment, the laser generator 411 may be a linear laser generator, generating lasers in... y Linear light extending in direction ( Figure 8A -B is perpendicular to the plane of the paper). This linear light is then projected onto the imaging plane by a reflective mechanism 412 that can oscillate along an axis in the x-direction. The oscillation of the reflective mechanism 412 is shown in the attached figure. Figure 8B As shown, this enables reciprocating linear optical scanning within the AB range (from A to B).

[0082] In one embodiment, the aforementioned reflection mechanism 412 may be a micromirror device (also known as a digital micromirror device, DMD) and may be implemented as a MEMS (microelectromechanical system). Figure 9 A simplified perspective diagram of the projection device used in this invention is shown. Figure 9 As shown, the point laser generated by the laser can be transformed into linear light (corresponding to the linear laser generator 411 in Figure 8) through a lens. The linear light is then reflected by a micromirror device in the form of MEMS, and the reflected linear light is then projected into the external space through a light window.

[0083] The present invention can also be implemented as a depth data measurement method. Figure 10 A schematic flowchart of a depth data measurement method according to an embodiment of the present invention is shown. This method can be performed by the depth data measurement head of the present invention.

[0084] In step S1010, a linear light moving along a first direction is projected onto the imaging area, wherein the length direction of the linear light is a second direction perpendicular to the first direction.

[0085] In step S1020, an image sensor is used to image the imaging area. The image sensor includes multiple pixels. When the change in the amount of light received by each pixel exceeds a threshold within a unit of time, it sends an effective signal with a timestamp.

[0086] In step S1030, a two-dimensional image of the stripe pattern is generated based on the brightness of the stripe pattern corresponding to the timestamp and the pixel position of the effective signal, and the depth information of the object in the imaging area is obtained based on the two-dimensional image.

[0087] Alignment and calibration can be performed before measurement. Therefore, the measurement method of the present invention may further include aligning the scanning projection range of the projection device with the imaging area of ​​the image sensor.

[0088] In different embodiments, depth data can be acquired in different ways to obtain the required multiple stripe patterns.

[0089] In an embodiment of one-scan-one-image, projecting linear light moving along a first direction onto the imaging area includes projecting the linear light with brightness corresponding to different fringe patterns in N successive scan cycles T. Generating a two-dimensional image of the fringe pattern based on the linear light projection brightness corresponding to a timestamp and the pixel position of the reported valid signal, and obtaining the depth information of the subject within the imaging area from the two-dimensional image, may include: generating N two-dimensional images corresponding to N fringe patterns based on the N scan cycles T, and synthesizing a single depth image of the subject within the imaging area from the N two-dimensional images.

[0090] In an embodiment of multiple imaging in a single scan, projecting linear light moving along a first direction onto the imaging area includes: alternately projecting linear light with brightness corresponding to the positions of N stripe patterns to complete the projection of N stripe patterns within one scan cycle T. Generating a two-dimensional image of the stripe patterns based on the linear light projection brightness corresponding to the timestamp and the pixel position of the reported valid signal, and obtaining the depth information of the object within the imaging area based on the two-dimensional image includes: generating N two-dimensional images corresponding to the N stripe patterns based on the one scan cycle T, and synthesizing a single depth image of the object within the imaging area based on the N two-dimensional images.

[0091] In an embodiment where a single scan is performed and the images are synthesized based on predetermined positions, projecting linear light moving along a first direction onto the imaging area includes completing one scan within a scan cycle T. Generating a two-dimensional image of the stripe pattern based on the linear light projection brightness corresponding to a timestamp and the pixel position of the reported valid signal, and determining the depth information of the object within the imaging area based on the two-dimensional image, includes assigning N brightness values ​​corresponding to N predetermined stripe patterns to the pixel positions corresponding to the timestamps, thereby generating N two-dimensional images corresponding to the N stripe patterns, and synthesizing a single depth image of the object within the imaging area based on the N two-dimensional images.

[0092] The depth data measuring head and measuring method according to the present invention have been described in detail above with reference to the accompanying drawings.

[0093] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A depth data measurement head, comprising: A projection device for projecting linear light moving along a first direction onto an imaging area, wherein the longitudinal direction of the linear light is a second direction perpendicular to the first direction; An image sensor is used to image the imaging area. The image sensor includes multiple pixels. When the change in the amount of light received by each pixel exceeds a threshold within a unit of time, it sends an effective signal with a timestamp. as well as The processor is configured to generate a two-dimensional image of the stripe pattern based on the brightness of the stripe pattern corresponding to the timestamp and the pixel position of the reported valid signal, and to obtain the depth information of the object within the imaging area based on the two-dimensional image. When a single pixel sends multiple valid signals with successive timestamps, the processor selects one of the multiple successive timestamps as the timestamp of the valid signal corresponding to that pixel.

2. The measuring head as described in claim 1, wherein, The pixel row of the image sensor forms an angle with the second direction, and when multiple pixels in the same pixel row send valid signals with the same timestamp, the processor selects pixels from the multiple pixels for generating the two-dimensional image based on the light intensity distribution of the linear light in the first direction.

3. The measuring head as described in claim 1, wherein, When multiple pixels in the same pixel row send valid signals with the same timestamp, the processor deletes pixels in that pixel row that are far from the current scan range for the generation of the two-dimensional image.

4. The measuring head as described in claim 1, wherein, The projection device completes one pattern scan within a scanning period T. Within the scanning period T, the projection device adjusts the brightness of the linear light during projection according to the single stripe pattern to be projected. Furthermore, the projection device adjusts the brightness of the linear light according to different stripe patterns in each of the N successive scanning cycles T. The processor generates N two-dimensional images corresponding to the N stripe patterns based on the N scanning cycles T, and synthesizes a single depth image of the object within the imaging area based on the N two-dimensional images.

5. The measuring head as described in claim 1, wherein, The projection device completes multiple pattern scans within a scanning period T. Within this scanning period T, the projection device alternately adjusts the brightness of the linear light during the moving projection according to the N stripe patterns to be projected. The processor generates N two-dimensional images corresponding to N stripe patterns based on the one scanning cycle T, and synthesizes a single depth image of the object in the imaging area based on the N two-dimensional images.

6. The measuring head as described in claim 5, wherein, Within the scanning period T, the projection device adjusts the brightness of the linear light according to N stripe patterns, including: The scanning period T includes multiple projection cycle sub-cycles. Within each projection cycle sub-cycle, the projection device alternately projects linear light with brightness corresponding to the positions of the N stripe patterns. Furthermore, the processor correlates the pixel positions of the valid signals with the brightness of the linear light projection based on the timestamp, and divides them into N categories, each corresponding to a stripe pattern, thereby generating N two-dimensional images corresponding to the N stripe patterns, and synthesizing a single depth image of the subject within the imaging area based on the N two-dimensional images.

7. The measuring head as described in claim 6, wherein, The N stripe patterns are four sinusoidal four-step phase-shift patterns in which there are no pixels of the same brightness at the same position between adjacent patterns.

8. The measuring head as claimed in claim 1, wherein, Align the scanning projection range of the projection device with the imaging area of ​​the image sensor.

9. The measuring head as claimed in claim 1, wherein, The projection device completes one scan within a scanning period T, and the processor assigns brightness corresponding to a predetermined stripe pattern to the pixel position corresponding to the timestamp.

10. The measuring head as claimed in claim 9, wherein, The processor assigns N brightness values ​​corresponding to N predetermined stripe patterns to the pixel positions of the corresponding timestamps, thereby generating N two-dimensional images corresponding to the N stripe patterns, and synthesizes a single depth image of the subject within the imaging area based on the N two-dimensional images.

11. A method for measuring depth data, comprising: A linear light beam moving along a first direction is projected onto the imaging area, wherein the length direction of the linear light beam is a second direction perpendicular to the first direction; The imaging area is imaged using an image sensor, which includes multiple pixels. Each pixel sends a valid signal with a timestamp when the change in the amount of light received exceeds a threshold within a unit of time. as well as Based on the brightness of the stripe pattern corresponding to the timestamp and the pixel position of the effective signal, a two-dimensional image of the stripe pattern is generated, and the depth information of the object within the imaging area is obtained from the two-dimensional image. Specifically, when a single pixel sends multiple valid signals with successive timestamps, one of the multiple successive timestamps is selected as the timestamp of the valid signal corresponding to that pixel.

12. The method of claim 11, further comprising: Align the scanning projection range of the projection device with the imaging area of ​​the image sensor.

13. The method of claim 11, wherein, Projecting linear light that moves along a first direction onto the imaging region includes: The linear light is projected by varying the brightness of different stripe patterns within N successive scan cycles T, and Based on the linear light projection brightness corresponding to the timestamp and the pixel position of the reported valid signal, a two-dimensional image of the stripe pattern is generated, and the depth information of the subject within the imaging area is obtained from the two-dimensional image, including: Based on the N scanning cycles T, N two-dimensional images corresponding to the stripe pattern are generated, and a single depth image of the object within the imaging area is synthesized based on the N two-dimensional images.

14. The method of claim 11, wherein, Projecting linear light that moves along a first direction onto the imaging region includes: Alternating projection of linear light with brightness corresponding to the positions of N stripe patterns to complete the projection of N stripe patterns within one scan cycle T, and Based on the linear light projection brightness corresponding to the timestamp and the pixel position of the reported valid signal, a two-dimensional image of the stripe pattern is generated, and the depth information of the subject within the imaging area is obtained from the two-dimensional image, including: Based on the scanning cycle T, N two-dimensional images corresponding to the stripe pattern are generated, and a single depth image of the object within the imaging area is synthesized based on the N two-dimensional images.

15. The method of claim 11, wherein, Projecting linear light that moves along a first direction onto the imaging region includes: One scan is completed within one scan cycle T. Based on the linear light projection brightness corresponding to the timestamp and the pixel position of the reported valid signal, a two-dimensional image of the stripe pattern is generated, and the depth information of the subject within the imaging area is obtained from the two-dimensional image, including: N brightness values ​​corresponding to N predetermined stripe patterns are assigned to the pixel positions corresponding to the timestamps, thereby generating N two-dimensional images corresponding to the N stripe patterns, and a single depth image of the subject within the imaging area is synthesized based on the N two-dimensional images.

Images