Depth data measurement head, measurement device and measurement method

Abstract

A depth data measurement head, measurement device, and measurement method are disclosed. The measurement head includes: a projection device for projecting linear light onto an imaging area; and an image sensor comprising N groups of pixels uniformly distributed on the imaging surface, each group of pixels having an exposure switching period t with a phase interval of 2π / N between them. e Exposure is performed. The projection device completes one pattern scan within a scanning cycle, which includes multiple cyclic sub-cycles. In each cyclic sub-cycle, the projection period t is... p The image sensor comprises N waveform projection areas, each 2π / N wide, with light intensity encoded based on an imaging pattern. Thus, during a single pattern scan within a scanning cycle, each of the N groups of pixels in the image sensor images a different stripe pattern, and there is a 2π / N phase shift between the N stripe patterns. This invention enables the acquisition of multiple images from a single linear light scan, significantly improving the speed of depth map synthesis and making it suitable for capturing moving targets.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional imaging, and more specifically, to a depth data measurement head, a measurement device, and a measurement method. Background Technology

[0002] A depth camera is a device that acquires depth information of a target object. These cameras are widely used in fields such as 3D scanning and 3D modeling. For example, an increasing number of smartphones are now equipped with depth cameras for facial recognition. In existing technologies, stripe light coding can be used to achieve high-precision imaging. However, stripe light coding requires capturing multiple images of different stripes and then synthesizing a single depth image. Therefore, the resulting depth image has a low frame rate and cannot meet the requirements of real-time, high-precision dynamic imaging.

[0003] Therefore, an improved depth data measurement scheme is needed. Summary of the Invention

[0004] One technical problem this disclosure aims to solve is to provide an improved depth data measurement scheme that utilizes an image sensor with different groups of pixels equipped with phase-shift exposure to image linear light projected in different sub-cycles with successive phase shifts. This allows different groups of pixels in the image sensor to acquire different phase-shift fringe images during a single scan of the linear light, thereby achieving the acquisition of multiple fringe images from a single linear light scan. This significantly improves the speed of depth map synthesis and is suitable for photographing moving targets.

[0005] According to a first aspect of this disclosure, a depth imaging measurement head is provided, comprising: a projection device for projecting linear light moving along a first direction onto an imaging region, wherein the longitudinal direction of the linear light is a second direction perpendicular to the first direction; and an image sensor comprising N groups of pixels uniformly distributed on an imaging surface, each group of pixels having an exposure switching period t spaced apart from each other by a phase of 2π / N. e Exposure is performed, where N is an integer greater than 1, and the projection device is in the scanning cycle. One pattern scan is completed within a certain period of time, and the scanning cycle is... It includes multiple cyclic sub-cycles. In each cycle sub-cycle In this process, the linear light has a projection period t p The projection period t performs brightness changes. p The duration is related to the exposure switching period t e The durations are the same, and the projection period t is the same. p It includes N waveform projection regions with a width of 2π / N, and the projected light intensity of each waveform projection region is encoded so that during the scanning period... When a pattern scan is completed, each of the N groups of pixels of the image sensor images a different stripe pattern, and the N stripe patterns constitute a set of N-step phase shift patterns with a 2π / N phase shift between each other.

[0006] Optionally, the image sensor includes a plurality of pixel units, each pixel unit including one pixel belonging to a group of N pixels.

[0007] Optionally, the projection period t of the linear light p The exposure switching period t of the first group of pixels e synchronous.

[0008] Optionally, in each sub-cycle T i In the process, each projection period t of the linear light... p The internally projected waveforms are identical and are rectangular waves with a 2π / N phase bright area and a 6π / N phase dark area, wherein the N stripe patterns are stripe patterns that repeat in the bright and dark areas.

[0009] Optionally, the linear optical scan corresponds to the dwell time t on each column of pixels. c Not less than the scan period Divided by the column number C, the dwell time t c The projection period t is p More than 10 times.

[0010] Optionally, in each sub-cycle T i In this process, the linear light has a projection period t p Projected m times, and each sub-period T i The duration is greater than the dwell time t c .

[0011] Optionally, each pixel in the image sensor includes a corresponding charge storage unit during the scanning cycle. When a pattern scan is completed, a set of N-step phase shift patterns is obtained from the N sets of charge storage units corresponding to each of the N sets of pixels. The set of N-step phase shift patterns is used to generate a depth map of the imaging area.

[0012] Optionally, the projection device in the first scanning cycle The image sensor completes one pattern scan within a certain period, so that each of the N groups of pixels images a different stripe pattern, and the N stripe patterns form a Gray code pattern; the projection device completes one pattern scan in the second scanning cycle. A pattern scan is completed within the time frame so that each of the N groups of pixels of the image sensor images a different stripe pattern, and the N stripe patterns constitute the set of N-step phase shift patterns. A depth map of the imaging region is generated from the set of N-step phase shift patterns based on the set of Gray code patterns.

[0013] Optionally, the projection device includes: a light-emitting device for generating linear light; and a reflective device for reflecting the linear light to project linear light moving in a direction perpendicular to the stripe direction onto the shooting area at a predetermined frequency, wherein the length direction of the linear light is the length direction of the projected stripe, and the reflective device includes one of the following: a mechanical galvanometer that reciprocates at the predetermined frequency; a micromirror device that reciprocates at the predetermined frequency; and a mechanical rotating mirror that rotates unidirectionally at the predetermined frequency.

[0014] Optionally, the image sensor includes a first image sensor and a second image sensor with fixed relative positions, wherein the first image sensor and the second image sensor each include the N groups of pixels and are exposed synchronously with each other.

[0015] Optionally, the projection device performs α scanning cycles. The pattern is scanned α times within each scan cycle. It includes multiple cyclic sub-cycles. In each cycle sub-cycle It includes N sub-cycles T1-T N In each sub-cycle T i In this process, the linear light has a projection period t p The projection period t is used to perform brightness and darkness changes. p The duration is related to the exposure switching period t e The durations are the same, and the projection period t is the same. p Including the bright area, among which, in the sub-cycle T1-T N During the projection period t, the bright area is... p The position of the [something] changes with a phase interval of 2π / αN, such that in each scan cycle... During one pattern scan, the N groups of pixels of the image sensor each image a different stripe pattern, and this occurs over α scan cycles. When α pattern scans are completed, αN stripe patterns form a set of αN-step phase shift patterns with a phase shift of 2π / αN between them, where α is an integer greater than or equal to 2.

[0016] Optionally, the projection device during the scanning cycle One pattern scan is completed within a certain period of time, and the scanning cycle is... It includes multiple cyclic sub-cycles. In each cycle sub-cycle It includes N sub-cycles T1-T N In each sub-cycle T i In this process, the linear light has a projection period t p The projection period t performs brightness changes. p The duration is related to the exposure switching period t e The durations are the same, and the projection period t is the same. p Including the bright area, among which, in the sub-cycle T1-T N During the projection period t, the bright area is... p The position of the [something] changes with a phase interval of 2π / N, such that during the scan cycle... When a pattern scan is completed, each of the N groups of pixels in the image sensor images a different stripe pattern, and the N stripe patterns constitute a set of N-step phase-shift patterns with a phase shift of 2π / N between them. Optionally, N = 2 n n is an integer greater than or equal to 1.

[0017] According to a second aspect of this disclosure, a depth data measurement apparatus is provided, comprising: a depth data measurement head according to the first aspect, and a processor connected to the depth data measurement head for use in the scanning cycle. When a pattern scan is completed, a depth map of the imaging region is obtained from the acquired N stripe patterns.

[0018] According to a third aspect of this disclosure, a depth data measurement method is provided, comprising: projecting a linear light moving along a first direction onto an imaging region, wherein the length direction of the linear light is a second direction perpendicular to the first direction, and the projected linear light is in a scanning cycle. One pattern scan is completed within a certain period of time, and the scanning cycle is... It includes multiple cyclic sub-cycles. In each cycle sub-cycle In this process, the linear light has a projection period t p The projection period t performs brightness changes. p The duration is related to the exposure switching period t e The durations are the same, and the projection period t is the same. p The system includes N waveform projection areas with a width of 2π / N, and the projected light intensity of each waveform projection area is encoded, where N is an integer greater than 1. An image sensor comprising N groups of pixels uniformly distributed on the imaging surface is used to capture images of the imaging area to obtain N image frames under the linear light scan projection, wherein each group of pixels has an exposure switching period t with a phase interval of 2π / N between them. eExposure is performed; and depth data of the object under test within the imaging area is obtained based on the image frame, wherein the projected light intensity of each waveform projection area is encoded such that during the scanning cycle... When a pattern scan is completed, each of the N groups of pixels of the image sensor images a different stripe pattern, and the N stripe patterns constitute a set of N-step phase shift patterns with a 2π / N phase shift between each other.

[0019] Therefore, the depth imaging measurement head of the present invention can simultaneously acquire N-step phase shift maps under a single linear light scan, thereby improving the imaging speed. Attached Figure Description

[0020] The above and other objects, features and advantages of this disclosure will become more apparent from the more detailed description of exemplary embodiments thereof taken in conjunction with the accompanying drawings, wherein like reference numerals generally denote like parts.

[0021] Figure 1 This demonstrates the principle of depth imaging using stripe-coded structured light.

[0022] Figure 2 This shows another example of projected stripe-coded structured light.

[0023] Figure 3 A schematic diagram of the composition of a depth data measurement head according to an embodiment of the present invention is shown.

[0024] Figure 4A -B shows Figure 3 Example of magnification operation of the projection device shown.

[0025] Figure 5 A simplified perspective diagram of the projection device used in this invention is shown.

[0026] Figure 6 An example of the pixel composition of the image sensor used in this invention is shown.

[0027] Figure 7 An example is shown showing the relative exposure periods between different groups of pixels on the same image sensor.

[0028] Figure 8 A cyclic sub-cycle is shown. A schematic diagram of the imaging of different groups of pixels.

[0029] Figure 9 An example is shown of the relative relationship between the projected light waveform and the exposure cycles of pixel group 1-4 when performing sinusoidal four-step phase-shift pattern imaging.

[0030] Figure 10This shows the completion of one scan cycle. An example of patterns 1-4 obtained from pixel groups 1-4 respectively.

[0031] Figure 11 An example of depth map imaging using Gray code combined with four-step phase shifting is shown.

[0032] Figure 12 A cyclic sub-cycle is shown. A schematic diagram of the imaging of different groups of pixels.

[0033] Figure 13A -D shows the relative relationship between the projected light waveforms of sub-cycles T1-T4 and the exposure cycles of pixel groups 1-4.

[0034] Figure 14 This shows the completion of one scan cycle. An example of patterns 1-4 obtained from pixel groups 1-4 respectively.

[0035] Figure 15 A schematic diagram of a depth data measurement device according to an embodiment of the present invention is shown.

[0036] Figure 16 A schematic flowchart of a depth data measurement method according to an embodiment of the present invention is shown. Detailed Implementation

[0037] Preferred embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0038] According to the principle of structured light measurement, the key to the entire measurement system is whether the scanning angle α can be accurately determined. The scanning angle of point and line structured light can be calculated and determined by mechanical devices such as rotating mirrors. The significance of image encoding and decoding lies in determining the scanning angle of the encoded structured light, i.e., the surface structured light system. Figure 1This diagram illustrates the principle of depth imaging using striped coded structured light. For ease of understanding, the diagram briefly explains the coding principle of striped structured light using a two-grayscale, three-bit binary time-coding method. The projection device sequentially projects three patterns, as shown in the diagram, onto the object being measured in the imaging area. Each pattern divides the projection space into eight regions using two grayscale levels (bright and dark). Each region corresponds to its own projection angle; it can be assumed that bright regions correspond to code "1" and dark regions correspond to code "0". By combining the coding values ​​of a point on the object in the three coded patterns according to the projection order, the region coding value of that point is obtained. This determines the region where the point is located and, consequently, the scanning angle of that point is decoded.

[0039] To improve matching accuracy, the number of projected patterns in time coding can be increased. Figure 2 Another example of projected stripe-coded structured light is shown. Specifically, the figure illustrates a five-bit binary time-coded light with two gray levels. In applications such as binocular imaging, this means that each pixel in each frame of the left and right images contains five region-coded values ​​of either 0 or 1, thereby enabling left-right image matching with higher precision (e.g., pixel-level). With the projection rate of the projection device remaining constant, compared to... Figure 1 The three coded patterns Figure 2 The example is equivalent to achieving higher-precision image matching at a higher temporal cost.

[0040] Figure 3 A schematic diagram illustrates the principle of using linear light to project fringe images to obtain depth data. (Example) Figure 3 As shown, the depth data measurement head 300 includes a projection device 310 and two image sensors 320_1 and 320_2. In a monocular implementation, the depth data measurement head 300 can also use a single image sensor for image capture.

[0041] The projection device 310 is used to scan and project structured light with stripe coding onto the imaging area. For example, during three consecutive image frame projection cycles, the projection device 310 can project such... Figure 1 The three patterns shown can be used to generate depth data. Images 320_1 and 320_2, which can be referred to as first and second image sensors respectively, have a predetermined relative positional relationship and are used to capture images of the imaging area to obtain first and second two-dimensional image frames under structured light illumination, respectively. For example, when projected onto the projection device 310... Figure 1 In the case of the three patterns shown, the first and second image sensors 320_1 and 320_2 can respectively image the areas projected with the three patterns (e.g., ...) within three synchronized image frame imaging cycles. Figure 3 Imaging is performed on the imaging plane and the area within a certain range before and after it.

[0042] like Figure 3 As shown, the projection device 310 can project a linear light extending in the x direction in the z direction (i.e., toward the imaging area). The projected linear light can move continuously in the y direction to cover the entire imaging area. Figure 3 The lower part provides a more easily understood illustration of the scanning of linear light using a perspective view of the shooting area.

[0043] In this disclosure, the direction of the light emanating from the measuring head is defined as the z-direction, the vertical direction of the imaging plane is defined as the x-direction, and the horizontal direction is defined as the y-direction. Therefore, the fringe structured light projected by the projection device can be the result of a linear light extending in the x-direction moving in the y-direction. Although in other embodiments, fringe structured light obtained by moving a linear light extending in the horizontal y-direction in the x-direction can also be synchronized and imaged, this disclosure preferably uses vertical fringe light for illustration.

[0044] Figure 4A -B shows Figure 3 An example of magnification operation of the projection device shown. Specifically, as... Figure 3 As shown, in the projection device 310, the laser generator (such as...) Figure 4A The laser emitted by the laser generator 411 (shown in detail in section B) is projected through a projection mechanism (such as...) Figure 4A The projection mechanism 412, as detailed in section -B, scans and projects onto the shooting area ( Figure 3 The gray area in the image is used to measure the object to be tested in the shooting area (e.g., Figure 3 Active structured light projection is performed on the area being photographed (in the image). A pair of image sensors 320_1 and 320_2 image the area being photographed, thereby acquiring the image frames required for depth data calculation. Figure 3 As shown, the dashed lines emitted by the projection device 310 represent its projection range, while the dashed lines emitted by the image sensors 320_1 and 320_2 represent their respective imaging ranges. The imaging area is typically located in the overlapping region of the projection and imaging ranges of these three devices.

[0045] In practical applications, the laser generator is used to generate linear and / or infrared laser light, and the laser generator performs high-speed switching to scan the projected structured light, which corresponds to alternating bright and dark stripes, with the stripe coding. High-speed switching may include high-speed switching of the laser generator and high-speed coding switching.

[0046] In one embodiment, the laser generator can emit laser light of uniform intensity, and the projected stripe pattern is achieved by turning the laser generator on and off. In this case, since the laser generator projects light of only one intensity with different periodic duty cycles, each pixel of the image sensor integrates the projected light to determine the "presence" or "absence" of the illumination light; therefore, the equipped image sensor can be a monochrome image sensor.

[0047] In another embodiment, the laser generator itself can emit laser light with varying intensity, for example, a laser whose emitted light intensity varies sinusoidally over a large period depending on the applied power. This sinusoidally varying laser light can be combined with stripe projection, thereby scanning and projecting a pattern of alternating bright and dark areas with varying brightness between the bright stripes. In this case, the image sensor needs to be capable of distinguishing and imaging different light intensities, and therefore can be a multi-level grayscale image sensor. Clearly, grayscale projection and imaging can provide more accurate pixel-to-pixel matching than black-and-white projection and imaging, thereby improving the accuracy of depth data measurement.

[0048] In one embodiment, the laser generator 411 may be a linear laser generator, generating linear light extending in the x-direction. Figure 4A -B (perpendicular to the plane of the paper). This linear light is then projected onto the imaging plane by a reflective mechanism 412 that can oscillate along an axis in the x-direction. The oscillation of the reflective mechanism 412 is shown in the attached figure. Figure 4B As shown, the projection mechanism 412 (e.g., a reflector) is capable of scanning within the range of angle α, thereby achieving reciprocating linear light scanning within the range AB of the imaging plane.

[0049] It should be understood that in order to achieve the projection of a striped pattern, the linear light itself needs to undergo changes in brightness (or, in a simpler implementation, changes in brightness and darkness) as it continuously moves in the y-direction. For example, in situations requiring scanning... Figure 1 When scanning the first pattern, the laser generator 411 remains off as the projection mechanism 412 scans through the front α / 2 angle. When scanning to the rear α / 2 angle, the laser generator 411 turns on, thus creating a pattern with a dark left side and a bright right side. And when scanning... Figure 1 When scanning the second pattern, the laser generator 411 remains off as the projection mechanism 412 scans through an angle of 0 to α / 4. When scanning to an angle of α / 2 to α / 2, the laser generator 411 turns on. When scanning through an angle of α / 2 to 3α / 4, the laser generator 411 turns off again. When scanning to an angle of 3α / 4 to α, the laser generator 411 turns on again. This achieves a dark-light-dark-light pattern. Similarly, this can be achieved with more frequent changes based on the rotation angle. Figure 1 The third pattern and Figure 2The pattern shown is a finer stripe.

[0050] In one embodiment, the aforementioned reflection mechanism 412 may be a micromirror device (also known as a digital micromirror device, DMD) and may be implemented as a MEMS (microelectromechanical system). Figure 5 A simplified perspective diagram of the projection device used in this invention is shown. Figure 5 As shown, the point laser generated by the laser can be transformed into linear light (corresponding to the linear laser generator 411 in Figure 4) through a lens. This linear light is then reflected by a micromirror device in the form of a MEMS, and the reflected linear light is then projected into the external space through an optical window. Micromirror devices have extremely high performance; for example, commercially available DMDs can reciprocate at a highly stable frequency of 2 kHz, thus laying the foundation for high-performance depth imaging.

[0051] In order to obtain a high-precision depth map, Figure 3 The depth data measurement head shown needs to project multiple different fringe patterns sequentially. In other words, existing methods for synthesizing depth maps using captured fringe patterns sacrifice accuracy for time-domain performance. Furthermore, since different fringe patterns captured in N consecutive imaging cycles are used to synthesize a single depth map, existing depth data measurement methods are only applicable when the subject remains stationary during these N imaging cycles. This significantly limits the application scope of techniques for obtaining depth data using actively projected fringe images.

[0052] In view of this, the present invention proposes a novel depth data measurement scheme. This scheme utilizes an image sensor equipped with different groups of pixels capable of phase-shift exposure. By cleverly configuring the changes in brightness and darkness of the projected linear light, different groups of pixels on the image sensor can each acquire different phase-shifted fringe images during a single scan of the linear light, thereby achieving the acquisition of multiple fringe images from a single linear light scan. This significantly improves the speed of depth map synthesis and is suitable for photographing moving targets.

[0053] In one embodiment, the present invention can be implemented as a depth imaging measurement head, comprising: a projection device and an image sensor. The projection device can be used to project an image onto the imaging area along a first direction (e.g., ...). Figure 3 A linear light moving in the y-direction, wherein the length direction of the linear light is a second direction perpendicular to the first direction (e.g., ...). Figure 3 (x-direction in the image). In one embodiment, the projection device may have... Figure 5 The implementation structure shown includes a linear light generating device and a projection mechanism for reflecting and projecting linear light, and capable of changing the projection direction within a certain angle.

[0054] An image sensor comprising multiple groups of pixels uniformly distributed on an imaging surface, each group of pixels having an exposure switching period t with a phase interval of 2π / N between them. e Exposure is performed, where N is an integer greater than 1.

[0055] For ease of understanding, we will use N=4 as an example to illustrate the structure and exposure of the image sensor. Figure 6 An example of the pixel composition of the image sensor used in this invention is shown. Also, for ease of explanation, Figure 6 The example shown is 16x24 pixels. It should be understood that real-world image sensors can have more pixels, such as 600x800 pixels. Figure 6 The image sensor shown includes 4 (N=4) groups of pixels uniformly distributed across the entire imaging surface, represented by symbols 1, 2, 3, and 4 in the illustrated blocks. Here, "uniformly distributed" across the entire imaging surface means that when linear light is projected in a scanning direction along the y-axis, each type of pixel has the same (or approximately the same) number of pixels illuminated within the currently illuminated area. In a preferred embodiment, these 4 groups of pixels are as follows: Figure 6 As shown, the pixels are spaced apart at intervals of one pixel. That is, it can be considered that... Figure 6 The image sensor shown includes multiple “pixel units” (as shown by the bold black box in the figure; in the example of Figure 4, it may include 8x12 identical pixel units), each pixel unit including one pixel belonging to one of four groups of pixels.

[0056] In other embodiments, each group of pixels may also be distributed in units of two pixels (e.g., two pixels arranged adjacently in the x-direction) spaced apart from each other.

[0057] Figure 7 An example illustrating the relative exposure periods between different groups of pixels on the same image sensor is shown. Figure 7 In the example, the four groups of pixels have the same exposure switching period t. e All pixels on the image sensor switch with a 50% duty cycle; in other words, all pixels have the same exposure switching waveform. The difference lies in the fact that successive groups of pixel waveforms have the same phase difference of π / 2. In one implementation, the exposure switching period t... e For example, a typical value of 20ns is taken. This means that each pixel in the image sensor operates with an interval of 10ns to receive exposure and 10ns to turn off, but the second group of pixels turns on 5ns later than the first group, the third group of pixels turns on 5ns later than the second group, and the fourth group of pixels turns on 5ns later than the third group (which can also be regarded as 5ns earlier than the first group of pixels).

[0058] The image sensor used can be as Figure 6 and Figure 7 As shown, when performing grouped phase-shift exposure, the linear light projection of the projection device can be cleverly set to achieve the acquisition of multiple images in a single scan.

[0059] Specifically, the projection device can be in one scanning cycle (e.g., denoted as the scanning cycle). A pattern scan is completed within ) days. Within the scanning cycle Inside, it is assumed that linear light sweeps across the imaging region at a uniform speed, and in multiple cyclic sub-periods. This process is repeated. The following will describe each cycle sub-cycle. A light projection embodiment is described in detail below.

[0060] In each cycle sub-cycle In this process, the linear light has a projection period t p The projection period t performs brightness changes. p The duration is related to the exposure switching period t e The duration is the same. Projection period t p It includes N waveform projection regions with a width of 2π / N, and the projected light intensity of each waveform projection region is encoded so that during the scanning period... When a pattern scan is completed, each of the N groups of pixels of the image sensor images a different stripe pattern, and the N stripe patterns constitute a set of N-step phase shift patterns with a 2π / N phase shift between each other.

[0061] In one embodiment, each waveform projection region corresponds to a projected rectangular wave of width 2π / N or 0 (0 can also be regarded as a rectangular wave with light intensity of 0), and a set of N-step phase shift patterns corresponds to one cyclic sub-cycle. The light intensity distribution is used to determine the light intensity of each waveform projection area.

[0062] In one embodiment, a set of N-step phase-shift patterns is a sinusoidal four-step phase-shift pattern, and each projection period t is calculated based on the exposure of the N sets of pixels corresponding to the waveform projection area. p The light intensity value of each waveform projection area is not less than zero.

[0063] For ease of explanation, we assume N = 4, and that the line laser scanning projection period is t. p The laser intensities in each 1 / 4 cycle within that time period are Q1 / Q2 / Q3 / Q4, respectively. Therefore, in one cyclic sub-cycle... Within, the integral brightness of the four pixels is as follows:

[0064] P1 = Σ(Q1 + Q2)

[0065] P2 = Σ(Q2 + Q3)

[0066] P3 = Σ(Q3 + Q4)

[0067] P4=Σ(Q1+Q4) (1)

[0068] Therefore, the values ​​of Q1 to Q4 can be determined according to the pattern type corresponding to the required 4-step phase-shift imaging.

[0069] Figure 8 A cyclic sub-cycle is shown. A schematic diagram of the imaging of different groups of pixels. Figure 8 In the example shown, the desired phase-shift pattern is a sinusoidal light waveform with a phase difference of π / 2. In this case, the values ​​of the four pixel groups P1 to P4 can be:

[0070] P1 = Q / 2 * sint + Q / 2

[0071] P2 = -Q / 2 * cost + Q / 2

[0072] P3 = -Q / 2 * sint + Q / 2

[0073] P4 = Q / 2 * cost + Q / 2 (2)

[0074] Here, Q can be seen as Figure 8 The integrated brightness at the brightest point of the phase-shifted fringe shown can be represented by the value of t, which corresponds to a single cycle. At different angular positions, the value of t when linear light scans at a constant speed can correspond to pixels at different positions in the image sensor.

[0075] The brightness values ​​of Q1 to Q4 with respect to t can be obtained by reversing equation (2). Since there are N = 4 unknowns Q1 to Q4 in equation (2), and the rank of equation (2) is N-1 = 3, Q1 to Q4 can actually have infinitely many solutions. One solution value is given below:

[0076] Q1 = A / 2 * sint + A / 2

[0077] Q2 = 0

[0078] Q3 = -A / 2 * cost + A / 2

[0079] Q4 = A / 2(cost - sint) (3)

[0080] Where O*A=Q, O is, for example, the number of exposure cycles received by each pixel when the linear light sweeps (for example, in the following example, each pixel column can complete 100 exposures in 2us of linear light sweep, which can be regarded as O=100).

[0081] However, since light intensity cannot be negative, the values ​​of Q1 to Q4 must remain non-negative. Equation (3) holds true in the range of t = 0 to π / 4, but when the linear light scans to the range of t = π / 4 to π / 2, another solution value can be given based on equation (2):

[0082] Q1 = A / 2 * cost + A / 2

[0083] Q2 = A / 2(sint - cost)

[0084] Q3 = -A / 2 * sint + A / 2

[0085] Q4=0 (4)

[0086] Figure 9 An example is shown showing the relative relationship between the waveform of the projected light and the exposure cycles of pixel group 1-4 when performing sinusoidal four-step phase-shift pattern imaging.

[0087] As shown in the figure and combined with Figure 8 According to equation (1), at t = 0, the values ​​of P1 to P4 can correspond to Q / 2, 0, Q / 2 and Q respectively.

[0088] Assuming scan cycle The latency is 3.84 ms, and the image sensor includes 1920 columns with a linear light projection period t. p The value is 20ns, which is the time it takes for each pixel to be scanned by the linear light, i.e., the dwell time t. c It can be equal to the scan period. Divide by the column number C, i.e., 3.84ms / 1920 = 2us, or, considering that the linear light has a certain width, the dwell time t c The exposure time is no less than 2µs, so each pixel column can complete 100 exposures within 2µs of linear light sweep. At this time, O = 100.

[0089] In one embodiment, linear light can be made to... Initially, maintain the first 100 projection cycles t p It has waveforms corresponding to Q1 = A / 2, Q2 = Q3 = 0, and Q4 = A / 2 at t = 0 (where A = Q / 0 = Q / 100). In a preferred embodiment, the waveform can be adjusted based on a small change in t during each projection period t. p The values ​​of Q1 to Q4 are fine-tuned, for example, by solving the value of equation (2).

[0090] As mentioned earlier, repeating the sub-cycle Reaching a predetermined number of scans, for example, M scans, and thus completing one scan cycle. Figure 10 This shows the completion of one scan cycle. Patterns 1-4 are obtained from pixel groups 1-4 respectively. For example, in Figure 10 In this example, M can be equal to 16, that is, the sub-cycle is repeated 16 times. This yields a four-step phase shift diagram with a phase difference of π / 2. Clearly, Figure 10 An example of a 4-step phase-shift pattern for a sine wave is shown. This was acquired using an image sensor with 1920 columns, as in the example above. Figure 10 The four-step phase-shift pattern shown has 32 repeating sinusoidal fringes, where each sinusoidal fringe covers 60 pixel columns (1920 / 32 = 60), or 30 pixel unit columns. In other words, a linear optical scan covers one cyclic sub-cycle. 60 * 100 = 6000 projections are required. In each projection cycle t... p In the embodiment where the values ​​of Q1 to Q4 are fine-tuned for example, as shown in equation (3), each projection period t p Compared to the previous projection period t p For each t, ​​there exists a small increment of Δt = 2π / 6000 = π / 3000, which leads to different values.

[0091] Back Figure 9 As the linear light scans from the position corresponding to t=0 to the position corresponding to t=π / 6, the values ​​of Q1 to Q4 can be solved based on, for example, equation (3). At t=π / 6, it can be assumed that the linear light has scanned to the 5th pixel column in the current cycle of 60 pixel columns, and based on equation (3), the values ​​corresponding to t=π / 6 are Q1=3A / 4, Q2=0, Q3=0.183A, and Q4=0.067A.

[0092] As the linear light scans from the position corresponding to t = π / 6 to the position corresponding to t = π / 3, the values ​​of Q1 to Q4 can be continuously solved. Since cost = sint at t = π / 4, and cost < sint in the subsequent π / 4 to 3π / 4 process, equation (3) showing Q4 < 0 is no longer applicable. At this time, based on equation (4), we can solve for Q1 = 3A / 4, Q2 = 0.183A, Q3 = 0.067A, and Q4 = 0 at t = π / 3. Furthermore, at t = π / 3, it can be considered that the linear light has scanned to the 10th pixel column in the current cycle of 60 pixel columns.

[0093] As the linear light scans from the position corresponding to t = π / 3 to the position corresponding to t = π / 2, the values ​​of Q1 to Q4 can be solved based on equation (4). At t = π / 2, it can be assumed that the linear light has scanned to the 15th pixel column in the current cycle of 60 pixel columns, and based on equation (4), the values ​​of Q1 = A / 2, Q2 = A / 2, Q3 = 0, and Q4 = 0 corresponding to t = π / 2 can be solved. It should be noted that since pixels 1-4 undergo 4-step phase-shift imaging, the values ​​of Q1 to Q4 in each projection cycle t = π / 2 can be solved. p All of them have Q1+Q2+Q3+Q4=A, and since pixels 1-4 are spaced 2π / N apart, their phases also have a projection period t. p Exposure is performed and the on-time accounts for 10%, therefore, in each projection cycle t p A total of 2A*t can be obtained from pixels 1-4. p / 4 of the integral brightness.

[0094] As described above Figure 9 Examples of Q1 to Q4 values ​​for the first π / 2 of a 2π cycle in a sinusoidal N-step phase shift image are described, with imaging examples for pixels 1-4 respectively. Those skilled in the art can apply equation (2) and... Figure 9 For example, we continue to find the values ​​of Q1 to Q4 in the last 3π / 2 of a 2π cycle.

[0095] In addition, it should be understood that although Figure 8 and Figure 9 To facilitate the explanation of the changes in the stripe image and waveform, sub-periods T1-T4 are shown (each sub-period T1-T4 corresponds to a 2π cyclic sub-period). The 0 to π / 2 part, π / 2 to π part, π to 3π / 2 part and 3π / 2 to 2π part in the above, but when realizing the above 4-step phase shift pattern of sine wave, it is only necessary to obtain one cyclic sub-period according to the above formula (2). The change in the brightness value of the internally projected laser does not require additional sub-period T1-T4 division.

[0096] In one embodiment, a more accurate depth image can be obtained by combining other fringe lights with N-step phase-shift imaging. Figure 11 This illustrates an example of depth map imaging using Gray code combined with four-step phase shifting. In practical applications, this can be achieved by including more cyclic sub-periods within each four-step phase shifting pattern. This improves the imaging accuracy of the depth map. However, due to the periodic repetition of the 4-step phase-shift pattern, there is a problem of not being able to identify depth jumps across cycles. In this case, Gray code can be used to perform preliminary imaging of the object within the imaging space, followed by 4-step phase-shift imaging. Specifically, the projection device can... The projection device completes one pattern scan within a certain period (e.g., by solving the corresponding waveform of each linear light projection cycle using the illustrated brightness and darkness diagram, so that each of the N groups of pixels of the image sensor images a different stripe pattern, and the N stripe patterns constitute a Gray code pattern). The projection device completes one pattern scan within the second scan cycle. A pattern scan is completed within the time frame so that each of the N groups of pixels of the image sensor images a different stripe pattern, and the N stripe patterns constitute the set of N-step phase shift patterns. A depth map of the imaging region is generated from the set of N-step phase shift patterns based on the set of Gray code patterns.

[0097] In addition, multiple scan cycles were introduced. At this time, not only can other stripe patterns plus N-step phase shift patterns be implemented, but also αN-step phase shift patterns can be implemented. Here, α is an integer greater than or equal to 2. Specifically, the projection device performs α scan cycles. The pattern is scanned α times within each scan cycle. It includes multiple cyclic sub-cycles. In each cycle sub-cycle It includes N sub-cycles T1-T N In each sub-cycle T i In this process, the linear light has a projection period t p The projection period t is used to perform brightness and darkness changes. p The duration is related to the exposure switching period t e The durations are the same, and the projection period t is the same. p Including the bright area, among which, in the sub-cycle T1-T N During the projection period t, the bright area is... p The position of the [something] changes with a phase interval of 2π / αN, such that in each scan cycle... During one pattern scan, the N groups of pixels of the image sensor each image a different stripe pattern, and this occurs over α scan cycles. When α pattern scans are completed, αN stripe patterns form a set of αN-step phase shift patterns with a phase shift of 2π / αN between them, where α is an integer greater than or equal to 2.

[0098] For ease of understanding, let's take α = 2 and N = 4 as an example. That is, using exposure switching periods t, which include 4 sets of phases spaced 2π / N apart,... e An image sensor that exposes pixels and performs two scans is used to acquire an eight-step phase-shift pattern.

[0099] To acquire an eight-step phase-shift pattern using four sets of pixel image sensors, the projection device needs to perform two scans. In the first scan, i.e., during the first scan cycle... The middle includes multiple cyclic sub-cycles. In each cycle sub-cycle The projection includes four sub-periods T1-T4, and the bright area is in the projection period t. p The position of the [something] changes at π / 4 phase intervals, such that in the first scan cycle... During a single pattern scan, the first four images of the eight-step phase-shift pattern are acquired. In the second scan, i.e., during the second scan cycle... The middle also includes multiple cyclic sub-cycles. In each cycle sub-cycle Similarly, it includes N sub-periods T1-T4, and the bright area is in the projection period t. p The position of the bright area changes at π / 4 phase intervals (but the phase of the bright area is different from that in the first scan), so that in the second scan cycle... During a single pattern scan, the last four images of the eight-step phase-shift pattern are acquired. This allows for the acquisition of the eight-step phase-shift pattern in just two scans, thus achieving higher depth imaging accuracy.

[0100] Furthermore, although the example of N=4 was given above for ease of explanation, N can take other values ​​in other embodiments. Specifically, N=2 n n is an integer greater than or equal to 1. Therefore, it is possible to achieve, for example, higher precision phase shifting using an 8-step phase shifting sensor with 8 pixel groups, or 16-step phase shifting using a 16-step phase shifting sensor with 16 pixel groups.

[0101] Each pixel in the image sensor may include a corresponding charge storage unit during the scan cycle. When a pattern scan is completed, a set of N-step phase shift patterns is obtained from the N sets of charge storage units corresponding to each of the N sets of pixels. The set of N-step phase shift patterns is used to generate a depth map of the imaging area.

[0102] Figure 12 A cyclic sub-cycle is shown. A schematic diagram of the imaging of different groups of pixels. Figure 12 In the example shown, the desired phase-shift pattern is a bright-dark stripe waveform with a phase difference of π / 2. In this case, the values ​​of the four pixel groups P1 to P4 can be:

[0103] P1=Q, when t=0~π; =0, when t=π~2π

[0104] P2=0, when t=0~π / 2; =Q, when t=π / 2~3π / 2; =0, when t=3π / 2~2π

[0105] P3=0, when t=0~π; =Q, when t=π~2π

[0106] P4=Q, when t=0~π / 2; =0, when t=π / 2~3π / 2; =Q, when t=3π / 2~2π (5)

[0107] Therefore, based on equations (1) and (5), we solve for Q1 to Q4 and obtain the following optimized solution:

[0108] Q1=A, when t=0~π / 2; =0, when t=π / 2~2π

[0109] Q2=0, when t=0~π / 2; =A, when t=π / 2~π; =0, when t=π~2π

[0110] Q3=0, when t=0~π; =A, when t=π~3π / 2; =0, when t=3π / 2~2π

[0111] Q4=0, when t=0~3π / 2; =A, when t=3π / 2~2π (6)

[0112] Where O*A=Q, O is, for example, the number of exposure cycles received by each pixel when the linear light sweeps across.

[0113] In the optimization solution based on equation (6), each cyclic sub-cycle can be... The timeline is divided into four sub-cycles, T1-T4, each corresponding to a 2π cyclic sub-cycle. The bright area is defined in the 0 to π / 2 portion, the π / 2 to π portion, the π to 3π / 2 portion, and the 3π / 2 to 2π portion, and the bright area is defined in the projection period t. p The position of the element changes at 2π / 4 phase intervals.

[0114] In the example of N=4, each cycle sub-cycle It includes four sub-cycles, T1-T4. Figure 13A -D shows the relative relationship between the projected light waveforms of sub-cycles T1-T4 and the exposure cycles of pixel groups 1-4.

[0115] like Figure 13A As shown in -D, the projected laser is synchronized with the exposure switching period t. e The same projection period t p Laser projection is performed, and each laser projection cycle t can be considered as... p They always maintain the same exposure switching period t as the first group of pixels. e Synchronization (although the laser projection period t) pThe laser is switched on at different phases in different sub-cycles and on and off with a duty cycle of 25% (i.e., the bright area is 2π / N), meaning the waveform is a rectangular wave with a 25% duty cycle. In the four sub-cycles T1-T4, the projected laser is... p The projection activation time within each pixel corresponds to the exposure activation time of the first to fourth pixel groups.

[0116] Specifically, firstly, as Figure 13A As shown, in sub-period T1, the projected laser is in each projection period t p It remains on during the first π / 2 phase. Since the first and fourth groups of pixels are also on, it can remain on during each projection period t. p The reflected projected light is exposed within the first π / 2 phase, allowing charge accumulation to occur within the corresponding pixel, as shown by the gray rectangle in the figure. After completing the predetermined m1 projection cycles, the sub-cycle T1 ends, and the projected laser enters each projection cycle t. p The sub-period T2 remains active within the π / 2 to π phase.

[0117] like Figure 13B As shown, in sub-period T2, the projected laser is in each projection period t p It remains on during the π / 2 to π phase. During this time, since the first group of pixels and the second group of pixels are also on, it can be turned on in each projection period t. p The reflected projected light is exposed within the π / 2 to π phase range, allowing charge accumulation to occur within the corresponding pixels, as shown by the gray rectangles in the figure. After completing a predetermined m² projection cycles, the sub-cycle T2 ends, and the projected laser enters each projection cycle t. p The sub-period T3 remains active within the π to 3π / 2 phase.

[0118] like Figure 13C As shown, in sub-period T3, the projected laser is in each projection period t p It remains on during the π to 3π / 2 phase. During this time, since the second and third groups of pixels are also on, it can remain on in each projection period t. p The reflected projected light is exposed within a phase of π to 3π / 2, allowing charge accumulation to occur within the corresponding pixels, as shown by the gray rectangles in the figure. After completing a predetermined m³ projection cycles, the sub-cycle T3 ends, and the projected laser enters each projection cycle t. p The sub-period T4 remains active within the 3π / 2 to 2π phase.

[0119] like Figure 13D As shown, in sub-period T4, the projected laser is in each projection period t pIt remains on during the 3π / 2 to 2π phase. During this time, since the third and fourth groups of pixels are also on, it can remain on in each projection period t. p The reflected projected light is exposed within the 3π / 2 to 2π phase range, allowing charge accumulation to occur within the corresponding pixels, as shown by the gray rectangles in the figure. After completing the predetermined m4 projection cycles, sub-cycle T4 ends. At this point, one cycle sub-cycle is completed. The projection, and then proceed to the next cycle sub-cycle. The sub-cycle T1.

[0120] In a simple implementation, the number of projection cycles for each sub-cycle can be made the same, i.e., m1 = m2 = m3 = m4, meaning the duration of T1-T4 is the same. In this case, if the linear light sweeps across the imaging plane at a uniform speed, then the following can be obtained: Figure 12 The phase shift diagram is shown. Combined with, as shown... Figure 13A As shown in -D, in sub-cycle T1, the illuminated area of ​​the projected laser falls within the exposure range of the first and fourth pixel groups, therefore the first and fourth pixel groups correspond to bright fringes; in sub-cycle T2, the illuminated area of ​​the projected laser falls within the exposure range of the first and second pixel groups, therefore the first and second pixel groups correspond to bright fringes; in sub-cycle T3, the illuminated area of ​​the projected laser falls within the exposure range of the second and third pixel groups, therefore the second and third pixel groups correspond to bright fringes; in sub-cycle T4, the illuminated area of ​​the projected laser falls within the exposure range of the third and fourth pixel groups, therefore the third and fourth pixel groups correspond to bright fringes. The above sub-cycles are repeated. Reaching a predetermined number of scans, for example, M scans, and thus completing one scan cycle. Figure 14 This shows the completion of one scan cycle. Patterns 1-4 are obtained from pixel groups 1-4 respectively. For example, in Figure 14 In this example, M can be equal to 16, that is, the sub-cycle is repeated 16 times. This yields a four-step phase shift diagram of bright and dark fringes with a phase difference of π / 2.

[0121] Therefore, when obtaining the N-step phase shift map of bright and dark fringes, in each cycle sub-cycle... It includes N sub-cycles T1-T N In each sub-cycle T i In this process, the linear light has a projection period t p The projection period t performs brightness changes. p The duration is related to the exposure switching period t e The durations are the same, and the projection period t is the same. p Including the bright area, among which, in the sub-cycle T1-T NDuring the projection period t, the bright area is... p The position of the element changes with a phase interval of 2π / N, thereby allowing the scanning period to... When a pattern scan is completed, each of the N groups of pixels in the image sensor images a different stripe pattern, and there is a phase shift of 2π / N between the N stripe patterns. Thus, a single scan of linear light, for example from left to right (corresponding to one scan cycle),... It can directly obtain a set of N-step phase shift patterns from the N sets of pixels of the image sensor.

[0122] Linear optical scanning corresponds to the dwell time t on each column of pixels. c Not less than the scan period Divide by the column number C. To ensure sufficient exposure, the dwell time t c The projection period t is p More than 10 times, preferably, the dwell time t c The projection period t is p More than 50 times. For example, assuming a scan cycle The frame rate is 3.84ms (i.e., the frame rate is 1000 / 3.84≈260 frames / s), and the image sensor includes 1920 columns (in a system with...). Figure 6 In the case of the pixel unit shown (with 960 columns of pixels), the time it takes for each pixel to be swept by the linear light, i.e., the dwell time t, is... c It can be equal to the scan period. Divide by the column number C, i.e., 3.84ms / 1920 = 2us, or, considering that the linear light has a certain width, the dwell time t c Not less than 2µs. Online light projection period t p With a wavelength of 20ns and a duty cycle of 25%, each pixel column can complete 100 exposures within a 2µs linear light scan, and this is consistent with the current projection period t. p The two groups of pixels corresponding to the bright areas can achieve an exposure time of 5ns x 100 = 0.5us. In addition, since the time it takes for the linear light to scan through a column of pixels is long enough compared to the exposure period of the pixels (e.g., 100 times in the example above), for the currently illuminated column of pixels, the linear light itself can be approximated as a light source that does not move in position during these 2us.

[0123] The dwell time t in each pixel column c When it is not less than 2µs, if the pixel unit has, for example Figure 6 The 2x2 arrangement of 4 pixels shown means that each pixel unit is scanned by linear light for at least 4µs. To achieve a four-step phase shift, each sub-period T requires...i The duration of the light sweep is not less than the time it takes for the linear light to sweep across one column of pixel units. This is achieved using an image sensor with 1920 columns as shown in the example above. Figure 10 The four-step phase-shift map shown has 32 fringes (16 bright fringes and 16 dark fringes), and each fringe covers 60 pixel columns (1920 / 32 = 60), or 30 pixel unit columns. For example... Figure 9 As shown, due to each sub-cycle T i Corresponding to half a stripe, this covers 30 pixel columns, or 15 pixel unit columns, with a duration of 2µs x 30 = 60µs. Therefore, in this example, m1 = m2 = m3 = m4 = m = 60µs / 20ns = 3000. Since the linear light projection period t... p Exposure switching period t of each pixel group e Since the duration is the same, when the linear light scan passes through half the distance of the stripe, the corresponding groups of pixels have also switched on and off 3000 times.

[0124] As described above Figures 12-14 The linear light being projected is given by the projection period t. p An example of a rectangular wave with a bright region of 2π / N phase. In this example, the projected linear light is always the same as the projection period t. p The duty cycle is 100 / N% (in the example above where N=4, the duty cycle is 25%, corresponding to the bright area of ​​the 2π / N phase) and each different projection period t p The projected light intensity in the inner bright region preferably has the same rectangular wave. In each sub-cycle T... i In the process, each projection period t of the linear light... p The internally projected waveforms are identical and are rectangular waves with a 2π / N phase bright area and a 6π / N phase dark area (e.g., each projection period t). p In the middle, the laser is in t p It is activated within a time interval of / N, and (N-1)*t p (Turn off within / N time intervals). Between successive sub-cycles, the projection period of the linear light and the ratio of bright to dark areas remain unchanged; only the position of the bright area changes. And this constitutes one scan cycle. Multiple cyclic sub-cycles Internal repetition. Therefore, the resulting set of N-step phase-shift patterns is: Figure 14 The striped pattern shown has a clear distinction between bright and dark areas and repeats multiple times.

[0125] The above is based on a rectangular wave with a 100 / N% duty cycle and constant brightness. Figure 14 The illustrated example of bright and dark stripes can be considered as an embodiment of the present invention based on the linear light projection period t. p Within this, each 2π / N phase is individually adjustable, thus combining a phase difference of 2π / N and an exposure period equal to t.p An image sensor is a special case of an imaging scheme that generates a set of N-step phase-shift patterns.

[0126] The projection and imaging scheme of the present invention can be used in a monocular scheme (i.e., a scheme equipped with one image sensor) or a binocular scheme (i.e., a scheme equipped with two image sensors with fixed relative positions for synchronous imaging). When the image sensors include a first image sensor and a second image sensor with fixed relative positions, the first image sensor and the second image sensor can each include the N groups of pixels and are exposed synchronously with each other. In other words, in the binocular scheme, during the scanning cycle... When completing one pattern scan, the first image sensor can acquire N stripe images, the second image sensor can acquire N stripe images, and these 2N stripe images can be used to obtain depth data.

[0127] Furthermore, to achieve scanning projection, the projection device of the present invention includes: a light-emitting device for generating linear light; and a reflective device for reflecting the linear light, which projects linear light moving in a direction perpendicular to the stripe direction onto the imaging area at a predetermined frequency, wherein the length direction of the linear light is the length direction of the projected stripes. The reflective device includes one of the following: a mechanical galvanometer that reciprocates at the predetermined frequency; a micromirror device that reciprocates at the predetermined frequency; and a mechanical rotating mirror that rotates unidirectionally at the predetermined frequency. Here, the projected linear light can be linear light with a high-order Gaussian or flat-top Gaussian distribution, thereby providing a highly uniform brightness distribution in the width direction of the linear light.

[0128] The present invention also discloses a measuring device using the aforementioned measuring head. Specifically, a depth data measuring device may include the depth data measuring head described above, and a processor connected to the depth data measuring head, for use in the scanning cycle. When a pattern scan is completed, a depth map of the imaging region is obtained from the acquired N stripe patterns. In a binocular scheme, the processor can determine the depth data of the object in the imaging region based on the predetermined relative positions of the first and second image sensors and the N first two-dimensional image frames and N second two-dimensional image frames obtained from the structured light imaging. In different embodiments, the measuring head can have a relatively independent package or can be packaged together with the processor in the measuring device.

[0129] Figure 15 A schematic diagram of a depth data measurement apparatus according to an embodiment of the present invention is shown. As shown, the measurement apparatus 1500 may include a measurement head and a processor 1530 as described above. The measurement head includes a projection device 1510 and two image sensors 1520.

[0130] The processor 1530 is connected to the measuring head, for example, to the projection device 1510 and each of the two image sensors 1520, for determining the depth data of the object in the shooting area based on the predetermined relative positions of the first and second image sensors 1520_1 and 1520_2 and the N first two-dimensional image frames and N second two-dimensional image frames obtained by the structured light imaging.

[0131] Figure 16 A schematic flowchart of a depth data measurement method according to an embodiment of the present invention is shown. This method can be implemented using the depth data measurement head and measuring device of the present invention.

[0132] In step S1610, a linear light beam moving along a first direction is projected onto the imaging area, wherein the length direction of the linear light beam is a second direction perpendicular to the first direction, and the projected linear light beam is within the scanning cycle. One pattern scan is completed within a certain period of time, and the scanning cycle is... It includes multiple cyclic sub-cycles. In each cycle sub-cycle In this process, the linear light has a projection period t p The projection period t performs brightness changes. p The duration is related to the exposure switching period t e The durations are the same, and the projection period t is the same. p It includes N waveform projection areas with a width of 2π / N, and the projected light intensity of each waveform projection area is encoded, where N is an integer greater than 1.

[0133] In step S1620, an image sensor comprising N groups of pixels uniformly distributed on the imaging surface is used to capture images of the imaging area to obtain N image frames under the linear light scan projection, wherein each group of pixels has an exposure switching period t with a phase interval of 2π / N between them. e To expose it.

[0134] In step S1630, the depth data of the object under test within the imaging area is obtained based on the image frame.

[0135] The intensity of the projected light in each waveform projection region is encoded so that during the scanning cycle... When a pattern scan is completed, each of the N groups of pixels of the image sensor images a different stripe pattern, and the N stripe patterns constitute a set of N-step phase shift patterns with a 2π / N phase shift between each other.

[0136] The depth data measurement head, measuring device, and measuring method according to the present invention have been described in detail above with reference to the accompanying drawings. The depth data measurement scheme of the present invention utilizes an image sensor equipped with different groups of pixels capable of phase-shift exposure to image projected phase-shifted linear light. This allows different groups of pixels in the image sensor to acquire different phase-shifted fringe images during a single scan of the linear light, thereby achieving the acquisition of multiple fringe images in a single linear light scan. This significantly improves the speed of depth map synthesis and is suitable for photographing moving targets.

[0137] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems and methods according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0138] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A depth imaging measurement head, comprising: A projection device for projecting linear light moving along a first direction onto an imaging area, wherein the longitudinal direction of the linear light is a second direction perpendicular to the first direction; An image sensor comprising N groups of pixels uniformly distributed on an imaging surface, each group of pixels having an exposure switching period t with a phase interval of 2π / N between them. e Exposure is performed, where N is an integer greater than 1. The projection device during the scanning cycle One pattern scan is completed within a certain period of time, and the scanning cycle is... It includes multiple cyclic sub-cycles. In each cycle sub-cycle In this process, the linear light has a projection period t p The projection period t performs brightness changes. p The duration is related to the exposure switching period t e The durations are the same, and the projection period t is the same. p It includes N waveform projection regions with a width of 2π / N, and the projected light intensity of each waveform projection region is encoded so that during the scanning period... When a pattern scan is completed, each of the N groups of pixels of the image sensor images a different stripe pattern, and the N stripe patterns constitute a set of N-step phase shift patterns with a 2π / N phase shift between each other.

2. The depth imaging measurement head as described in claim 1, wherein, The image sensor includes multiple pixel units, each pixel unit including one pixel belonging to N groups of pixels.

3. The depth imaging measurement head as described in claim 1, wherein, The projection period t of the linear light p The exposure switching period t of the first group of pixels in the N groups of pixels e synchronous.

4. The depth imaging measurement head as described in claim 1, wherein, N=2 n , n It is an integer greater than or equal to 1.

5. The depth imaging measurement head as described in claim 1, wherein, Each waveform projection region corresponds to a rectangular wave of width 2π / N or 0, and a set of N-step phase shift patterns corresponds to one cyclic sub-cycle. The light intensity distribution is used to determine the light intensity of each waveform projection area.

6. The depth imaging measurement head as described in claim 5, wherein, The set of N-step phase shift patterns is a sinusoidal four-step phase shift pattern, and the projection period t is calculated based on the exposure of the N sets of pixels corresponding to the waveform projection area. p The light intensity value of each waveform projection area is not less than zero.

7. The depth imaging measurement head as described in claim 1, wherein, Linear optical scanning corresponds to the dwell time t on each column of pixels. c Not less than the scan period Divided by the column number C, the dwell time t c The projection period t is p More than 10 times.

8. The depth imaging measurement head as described in claim 7, wherein, In each sub-cycle T i In this process, the linear light has a projection period t p Projected m times, and each sub-period T i The duration is greater than the dwell time t c .

9. The depth imaging measurement head as described in claim 1, wherein, Each pixel in the image sensor includes a corresponding charge storage unit, during the scanning cycle. When a pattern scan is completed, the set of N-step phase shift patterns is obtained from the N sets of charge storage units corresponding to each of the N sets of pixels. The set of N-step phase shift patterns is used to generate a depth map of the imaging area.

10. The measuring head as claimed in claim 1, wherein, The projection device in the first scanning cycle The image sensor completes one pattern scan so that each of the N groups of pixels images a different stripe pattern, and the N stripe patterns form a Gray code pattern. The projection device in the second scanning cycle One pattern scan is completed within the time frame, so that each of the N groups of pixels of the image sensor images a different stripe pattern, and the N stripe patterns constitute the set of N-step phase-shift patterns. Specifically, a depth map of the imaging region is generated from the set of Gray code patterns based on the set of N-step phase shift patterns.

11. The measuring head as claimed in claim 1, wherein, The projection device includes: Light-emitting device for generating linear light; and A reflecting device for reflecting linear light, which is projected onto a shooting area at a predetermined frequency and moves in a direction perpendicular to the stripe direction, wherein the longitudinal direction of the linear light is the longitudinal direction of the projected stripes, and the reflecting device includes one of the following: A mechanical galvanometer that reciprocates at the predetermined frequency; Micromirror devices that reciprocate at a predetermined frequency; and A mechanical rotating mirror that rotates unidirectionally at a predetermined frequency.

12. The measuring head as claimed in claim 1, wherein, The image sensor includes a first image sensor and a second image sensor with fixed relative positions, wherein the first image sensor and the second image sensor each include the N groups of pixels and are exposed synchronously with each other.

13. The measuring head as claimed in claim 1, wherein, The projection device performs α scanning cycles The pattern is scanned α times within each scan cycle. It includes multiple cyclic sub-cycles. In each cycle sub-cycle It includes N sub-cycles T1-T N In each sub-cycle T i In this process, the linear light has a projection period t p The projection period t is used to perform brightness and darkness changes. p The duration is related to the exposure switching period t e The durations are the same, and the projection period t is the same. p Including the bright area, among which, in the sub-cycle T1-T N During the projection period t, the bright area is... p The position of the [something] changes with a phase interval of 2π / αN, such that in each scan cycle... During one pattern scan, the N groups of pixels of the image sensor each image a different stripe pattern, and this occurs over α scan cycles. When α pattern scans are completed, αN stripe patterns form a set of αN-step phase shift patterns with a phase shift of 2π / αN between them, where α is an integer greater than or equal to 2.

14. The measuring head as claimed in claim 1, wherein, In each cycle sub-cycle It includes N sub-cycles T1-T N In each sub-cycle T i In this process, the linear light has a projection period t p The projection period t is used to perform brightness and darkness changes. p The duration is related to the exposure switching period t e The durations are the same, and the projection period t is the same. p Including the bright area, among which, in the sub-cycle T1-T N During the projection period t, the bright area is... p The position of the [something] changes with a phase interval of 2π / N, such that during the scan cycle... When a pattern scan is completed, each of the N groups of pixels of the image sensor images a different stripe pattern, and the N stripe patterns constitute an N-step phase shift pattern of bright and dark stripes with a phase shift of 2π / N between them.

15. A depth data measurement device, comprising: The depth data measurement head as described in any one of claims 1-14, and A processor connected to the depth data measurement head is used in the scanning cycle. When a pattern scan is completed, a depth map of the imaging region is obtained from the acquired N stripe patterns.

16. A depth data measurement method, comprising: A linear light beam moving along a first direction is projected onto the imaging area, wherein the length direction of the linear light beam is a second direction perpendicular to the first direction, and the projected linear light beam is within the scanning cycle. One pattern scan is completed within a certain period of time, and the scanning cycle is... It includes multiple cyclic sub-cycles. In each cycle sub-cycle In this process, the linear light has a projection period t p The projection period t performs brightness changes. p Duration and exposure switching cycle t e The durations are the same, and the projection period t is the same. p It includes N waveform projection areas with a width of 2π / N, and the projected light intensity of each waveform projection area is encoded, where N is an integer greater than 1; The imaging area is captured using an image sensor comprising N groups of pixels uniformly distributed on the imaging surface to obtain N image frames under the linear light scan projection, wherein each group of pixels is exposed at an exposure switching period t with a phase interval of 2π / N. e Expose; and The depth data of the object under test within the imaging area is obtained based on the image frame. The intensity of the projected light in each waveform projection region is encoded such that during the scanning cycle... When a pattern scan is completed, each of the N groups of pixels of the image sensor images a different stripe pattern, and the N stripe patterns constitute a set of N-step phase shift patterns with a 2π / N phase shift between each other.

Images