Distance image acquisition device and distance image acquisition method

Abstract

To prevent reduction in resolution of a distance image picked up by using a dot light source.SOLUTION: A distance image pickup device comprises: a light source unit that irradiates a subject with an optical pulse; a light receiving unit having pixel circuits that are arranged in plurality in a two-dimensional matrix, each provided with a photoelectric conversion element generating electric charges according to incident light and a plurality of charge accumulation units accumulating the electric charges, a pixel driving circuit that distributes the electric charges to each of the charge accumulation units for accumulation at accumulation timing synchronized with the irradiation with the optical pulse, and electric charge discharge means that discharges the electric charges in a period that is not the accumulation timing; and a distance operation unit that calculates a distance to the subject on the basis of an amount of electric charges accumulated in each of the charge accumulation units. The optical pulse is structured light formed by a plurality of rays of dot light. At least one ray of first dot light of the plurality of rays of dot light has an elliptical shape in which a ratio of a major axis length to a minor axis length is equal to or more than a threshold.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a distance image capturing device and a distance image capturing method.

Background Art

[0002] A time-of-flight (TOF) type distance image capturing device that measures the distance between a measuring device and an object based on the flight time of light in space (measurement space) by utilizing the fact that the speed of light is known has been realized (see, for example, Patent Document 1). In such a distance image capturing device, the delay time from the time when a light pulse is irradiated until the reflected light reflected by the subject returns is obtained by causing the reflected light to enter an imaging element and distributing and accumulating charges corresponding to the amount of the reflected light to a plurality of charge accumulation parts, and the distance to the subject is calculated using the delay time and the speed of light.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] One of the factors that determines the longest distance measurable in a system that captures images by the TOF method is the power of the measurement light source (irradiation intensity per unit area). Although it is possible to increase the light source output by increasing the current flowing through the light emitting element, increasing the current flowing through the light emitting element leads to an increase in heat generation and power consumption of the imaging system, and there is a problem that the laser safety with respect to the human body decreases.

[0005] As a countermeasure to the above problem, one method is to use a dot light source instead of a uniform diffuse light source, which is normally used as a distance measuring light source. By using a diffuser plate that converts the light emitted from the light source element into dot light arranged in a periodic pattern, the power of the light can be concentrated into the dots. Even with the same current flowing through the light-emitting element, using a dot light source makes it possible to measure longer distances than when using diffuse light that uniformly illuminates the illumination surface.

[0006] However, when measuring distance using a dot light source, only the area illuminated by the dot light can be measured, and the area between the dots, i.e., the area where the subject is not illuminated, cannot be measured. As a result, there was a problem in that distance images captured using a dot light source had lower resolution than distance images measured using uniform diffuse light.

[0007] The present invention has been made in response to the above-mentioned problems, and aims to provide a distance image acquisition device and a distance image acquisition method that can suppress the decrease in resolution of distance images captured using a dot light source. [Means for solving the problem]

[0008] The distance image capturing apparatus of the present invention comprises: a light source unit that irradiates a subject with light pulses; a plurality of pixel circuits arranged in a two-dimensional matrix, each having a photoelectric conversion element that generates an electric charge corresponding to the incident light and a plurality of charge storage units that store the electric charge; a pixel driving circuit that distributes and stores the electric charge in each of the charge storage units at an accumulation timing synchronized with the irradiation of the light pulses; a light receiving unit having a charge discharge means for discharging the electric charge during periods when there is no accumulation timing; and a distance calculation unit that calculates the distance to the subject based on the amount of electric charge stored in each of the charge storage units, wherein the light pulse is structured light composed of a plurality of dot lights, and at least one first dot light among the plurality of dot lights has an elliptical shape in which the ratio of the major axis length to the minor axis length is greater than or equal to a threshold. Furthermore, at least a portion of the first dot light overlaps with at least a portion of another dot light adjacent to the first dot light in the longitudinal direction. do. Furthermore, the distance image capturing device of the present invention comprises a light source unit that irradiates a subject with light pulses, a plurality of pixel circuits arranged in a two-dimensional matrix, each having a photoelectric conversion element that generates an electric charge corresponding to the incident light and a plurality of charge storage units that store the electric charge, a pixel driving circuit that distributes and stores the electric charge in each of the charge storage units at an accumulation timing synchronized with the irradiation of the light pulses, a light receiving unit having a charge discharge means for discharging the electric charge during periods when there is no accumulation timing, and a distance calculation unit that calculates the distance to the subject based on the amount of electric charge stored in each of the charge storage units, wherein the light pulses are structured light composed of a plurality of dot lights, at least one first dot light among the plurality of dot lights has an elliptical shape in which the ratio of the major axis length to the minor axis length is greater than or equal to a threshold, the light source unit has a plurality of light source elements that can each independently irradiate the light pulses, and the major axis directions of the elliptical dot light emitted from at least two of the plurality of light source elements are different from each other.

[0011] In the distance image acquisition device of the present invention, the major axis directions of the elliptical dot light emitted from at least two of the plurality of light source elements are different from each other.

[0012] In the distance image acquisition device of the present invention, the major axis directions of the elliptical dot light emitted from at least two of the plurality of light source elements are orthogonal to each other.

[0013] In the distance image capturing device of the present invention, the first interval along the minor axis direction of dot light with elliptical major axes in the same direction emitted from a first group of light sources consisting of at least two light sources from the plurality of light sources, and the second interval along the minor axis direction of dot light with elliptical major axes in the same direction emitted from a second group of light sources consisting of at least two light sources different from the first group of light sources, are different from each other.

[0014] In the distance image acquisition device of the present invention, the major axis direction of the elliptical dot light emitted from at least two of the plurality of light source elements is oblique to the mounting surface of the imaging device, and is not perpendicular or horizontal.

[0015] In the distance image acquisition device of the present invention, the angle in the direction of the major axis of the elliptical dot light emitted from two of the plurality of light source elements is 45 degrees or 135 degrees with respect to the mounting surface of the imaging device.

[0016] In the distance image acquisition device of the present invention, at least one of the plurality of light source elements is a diffuse light source.

[0017] The distance image capturing method of the present invention is a distance image capturing method performed by a distance image capturing apparatus including a light source unit that irradiates a subject with an optical pulse, a pixel circuit in a two-dimensional matrix form including a photoelectric conversion element that generates charges according to the incident light and a plurality of charge storage units that store the charges, a pixel driving circuit that distributes and stores the charges in each of the charge storage units at an accumulation timing synchronized with the irradiation of the optical pulse, a light receiving unit having a charge discharging means that discharges the charges during a period other than the accumulation timing, and a distance calculation unit that calculates the distance to the subject based on the amount of charge stored in each of the charge storage units. The optical pulse is structured light composed of a plurality of dot lights, and at least one first dot light among the plurality of dot lights has an elliptical shape in which the ratio of the major axis length to the minor axis length is equal to or greater than a threshold value. Furthermore, at least a portion of the first dot light overlaps with at least a portion of another dot light adjacent to the first dot light in the longitudinal direction. to do. Furthermore, the distance image acquisition method of the present invention is a distance image acquisition method performed by a distance image acquisition device comprising: a light source unit that irradiates a subject with light pulses; a plurality of pixel circuits arranged in a two-dimensional matrix, each having a photoelectric conversion element that generates an electric charge corresponding to the incident light and a plurality of charge storage units that store the electric charge; a pixel driving circuit that distributes and stores the electric charge to each of the charge storage units at an accumulation timing synchronized with the irradiation of the light pulses; a light receiving unit having a charge discharge means for discharging the electric charge during periods when there is no accumulation timing; and a distance calculation unit that calculates the distance to the subject based on the amount of electric charge stored in each of the charge storage units, wherein the light pulses are structured light composed of a plurality of dot lights, at least one first dot light among the plurality of dot lights has an elliptical shape in which the ratio of the major axis length to the minor axis length is greater than or equal to a threshold, the light source unit has a plurality of light source elements each capable of independently irradiating the light pulses, and the major axis directions of the elliptical dot light emitted from at least two of the plurality of light source elements are different from each other.

Effect of the Invention

[0018] According to the present invention, it is possible to suppress a decrease in the resolution of a distance image captured using a dot light source.

Brief Description of the Drawings

[0019] [Figure 1] It is a block diagram showing a schematic configuration of a distance image capturing apparatus 1 of an embodiment. [Figure 2] It is a block diagram showing a schematic configuration of a distance image sensor 32 of an embodiment. [Figure 3] It is a circuit diagram showing an example of the configuration of a pixel 321 of an embodiment. [Figure 4] It is a diagram showing an example in which dot light Dt of an embodiment is irradiated on a subject OB. [Figure 5] It is a diagram showing an example in which dot light Dt of an embodiment is irradiated on a subject OB. [Figure 6] It is a diagram showing an example in which dot light Dt of an embodiment is irradiated on a subject OB. [Figure 7] It is a diagram showing an example in which dot light Dt of an embodiment is irradiated on a subject OB. [Figure 8] This is a diagram showing an example in which the dot light Dt of the embodiment irradiates the subject OB.

Embodiments for Carrying Out the Invention

[0020] Hereinafter, the distance image capturing device of the embodiment will be described with reference to the drawings.

[0021] FIG. 1 is a block diagram showing a schematic configuration of the distance image capturing device of the embodiment. The distance image capturing device 1 includes, for example, a light source unit 2, a light receiving unit 3, and a distance image processing unit 4. FIG. 1 also shows a subject OB, which is an object for measuring distance in the distance image capturing device 1.

[0022] The light source unit 2 irradiates an optical pulse PO onto a measurement target space where the subject OB, which is the target for measuring distance in the distance image capturing device 1, exists, in accordance with the control from the distance image processing unit 4. The light source unit 2 is, for example, a surface-emitting semiconductor laser module such as a vertical cavity surface emitting laser (VCSEL). The light source unit 2 includes a light source device 21 and a diffusion plate 22.

[0023] The light source device 21 is a light source that emits laser light in a near-infrared wavelength band (for example, a wavelength band with a wavelength of 850 nm to 940 nm) that becomes the optical pulse PO for irradiating the subject OB. The light source device 21 is, for example, a semiconductor laser light emitting element. The light source device 21 emits pulsed laser light in accordance with the control from the timing control unit 41.

[0024] The diffusion plate 22 is an optical component that diffuses the laser light in the near-infrared wavelength band emitted by the light source device 21 to the area of the surface for irradiating the subject OB. The pulsed laser light diffused by the diffusion plate 22 is emitted as the optical pulse PO and irradiates the subject OB.

[0025] The light-receiving unit 3 receives the reflected light RL of the light pulse PO reflected by the subject OB, which is the object whose distance is to be measured in the distance image capturing device 1, and outputs a pixel signal corresponding to the received reflected light RL. The light-receiving unit 3 comprises a lens 31 and a distance image sensor 32.

[0026] Lens 31 is an optical lens that guides the incident reflected light RL to the distance image sensor 32. Lens 31 emits the incident reflected light RL towards the distance image sensor 32, causing it to be received (incident) by the pixels in the light-receiving area of ​​the distance image sensor 32.

[0027] The distance image sensor 32 is an image sensor used in the distance image acquisition device 1. The distance image sensor 32 has multiple pixels in a two-dimensional light-receiving area. Within each pixel of the distance image sensor 32, there is one photoelectric conversion element, multiple charge storage units corresponding to this one photoelectric conversion element, and a component that distributes charge to each charge storage unit. In other words, the pixel is an image sensor with a distribution configuration that distributes and stores charge in multiple charge storage units.

[0028] The distance image sensor 32 distributes the charge generated by the photoelectric conversion element to its respective charge storage units in accordance with the control from the timing control unit 41. The distance image sensor 32 also outputs a pixel signal corresponding to the amount of charge distributed to the charge storage units. The distance image sensor 32 has multiple pixels arranged in a two-dimensional matrix, and outputs a pixel signal for one frame corresponding to each pixel.

[0029] The distance image processing unit 4 controls the distance image acquisition device 1 and calculates the distance to the subject OB. The distance image processing unit 4 comprises a timing control unit 41, a distance calculation unit 42, and a measurement control unit 43.

[0030] The timing control unit 41 controls the timing of outputting various control signals required for measurement, in accordance with the control of the measurement control unit 43. These various control signals include, for example, a signal to control the irradiation of the light pulse PO, a signal to distribute and store the reflected light RL in multiple charge storage units, and a signal to control the number of storage cycles per frame. The number of storage cycles is the number of times the process of distributing and storing charge in the charge storage unit CS (see Figure 3) is repeated. The exposure time is the product of this number of storage cycles and the time width (storage time width) for storing charge in each charge storage unit per charge distribution cycle.

[0031] The distance calculation unit 42 outputs distance information calculated based on the pixel signals output from the distance image sensor 32, determining the distance to the subject OB. The distance calculation unit 42 calculates the delay time from the irradiation of the light pulse PO to the reception of the reflected light RL based on the amount of charge accumulated in the multiple charge storage units. The distance calculation unit 42 calculates the distance to the subject OB according to the calculated delay time.

[0032] The measurement control unit 43 controls the timing control unit 41. For example, the measurement control unit 43 sets the number of times to accumulate per frame and the accumulation time width, and controls the timing control unit 41 so that imaging is performed according to the set settings.

[0033] In this configuration, the distance image acquisition device 1 receives reflected light RL from the light pulse PO in the near-infrared wavelength band that the light source unit 2 irradiates onto the subject OB, and the light receiving unit 3 receives the reflected light RL from the subject OB. The distance image processing unit 4 then outputs distance information, which is the distance to the subject OB measured.

[0034] In Figure 1, the distance image processing unit 4 is shown as being located inside the distance image imaging device 1. However, the distance image processing unit 4 may be a component located outside the distance image imaging device 1.

[0035] Here, the configuration of the distance image sensor 32 used as an image sensor in the distance image acquisition device 1 will be explained using Figure 2. Figure 2 is a block diagram showing the schematic configuration of the image sensor (distance image sensor 32) used in the distance image acquisition device 1 of this embodiment.

[0036] As shown in Figure 2, the distance image sensor 32 includes, for example, a light-receiving area 320 on which multiple pixels 321 are arranged, a control circuit 322, a vertical scanning circuit 323 having a distribution operation, a horizontal scanning circuit 324, and a pixel signal processing circuit 325.

[0037] The light-receiving region 320 is a region in which multiple pixels 321 are arranged, and Figure 2 shows an example in which they are arranged in a two-dimensional matrix of 8 rows and 8 columns. Each pixel 321 accumulates a charge corresponding to the amount of light it receives. The control circuit 322 comprehensively controls the distance image sensor 32. For example, the control circuit 322 controls the operation of the components of the distance image sensor 32 in response to instructions from the timing control unit 41 of the distance image processing unit 4. Note that the control of the components of the distance image sensor 32 may be directly performed by the timing control unit 41, in which case the control circuit 322 can be omitted.

[0038] The vertical scanning circuit 323 controls the pixels 321 arranged in the light-receiving area 320 row by row in response to control from the control circuit 322. The vertical scanning circuit 323 causes the pixel signal processing circuit 325 to output a voltage signal corresponding to the amount of charge stored in each of the charge storage units CS of the pixels 321. In this case, the vertical scanning circuit 323 distributes and stores the charge converted by the photoelectric conversion element in each of the charge storage units of the pixels 321. In other words, the vertical scanning circuit 323 is an example of a "pixel driving circuit".

[0039] The pixel signal processing circuit 325 is a circuit that performs predetermined signal processing (for example, noise suppression processing or A / D conversion processing) on ​​the voltage signal output from the corresponding vertical signal line from the pixel 321 of each column, in response to control from the control circuit 322.

[0040] The horizontal scanning circuit 324 is a circuit that sequentially outputs signals from the pixel signal processing circuit 325 to the horizontal signal line in response to control from the control circuit 322. As a result, pixel signals corresponding to the amount of charge accumulated for one frame are sequentially output to the distance image processing unit 4 via the horizontal signal line.

[0041] In the following explanation, we will assume that the pixel signal processing circuit 325 performs A / D conversion processing and that the pixel signal is a digital signal.

[0042] Here, the configuration of the pixels 321 arranged within the light-receiving area 320 of the distance image sensor 32 will be described using Figure 3. Figure 3 is a circuit diagram showing an example of the configuration of pixels 321 arranged within the light-receiving area 320 of the distance image sensor 32 of the embodiment. Figure 3 shows an example of the configuration of one pixel 321 among a plurality of pixels 321 arranged within the light-receiving area 320. The pixel 321 is an example of a configuration that includes three pixel signal readout units.

[0043] Pixel 321 comprises one photoelectric conversion element PD, a drain gate transistor GD, and three pixel signal readout units RU that output a voltage signal from the corresponding output terminal O. Each pixel signal readout unit RU comprises a readout gate transistor G, a floating diffusion FD, a charge storage capacitor C, a reset gate transistor RT, a source follower gate transistor SF, and a selection gate transistor SL. In each pixel signal readout unit RU, a charge storage unit CS is formed by the floating diffusion FD and the charge storage capacitor C.

[0044] In Figure 3, the three pixel signal readout units RU are distinguished by adding the numbers "1", "2", or "3" after the code "RU" of each unit. Similarly, each component of the three pixel signal readout units RU is represented by indicating the number representing the respective pixel signal readout unit RU after its code, thus distinguishing the pixel signal readout unit RU that each component corresponds to.

[0045] In the pixel 321 shown in Figure 3, the pixel signal readout unit RU1, which outputs a voltage signal from the output terminal O1, comprises a readout gate transistor G1, a floating diffusion FD1, a charge storage capacitor C1, a reset gate transistor RT1, a source follower gate transistor SF1, and a selection gate transistor SL1. In the pixel signal readout unit RU1, the floating diffusion FD1 and the charge storage capacitor C1 constitute a charge storage unit CS1. Pixel signal readout units RU2 to RU3 have a similar configuration.

[0046] Furthermore, the configuration of pixels arranged in the distance image sensor 32 is not limited to the configuration with three pixel signal readout units RU as shown in Figure 3, but any pixel with multiple pixel signal readout units RU is acceptable. In other words, the number of pixel signal readout units RU (charge storage units CS) provided in the pixels arranged in the distance image sensor 32 may be two, or four or more.

[0047] Furthermore, in the pixel 321 configuration shown in Figure 3, an example is shown in which the charge storage unit CS is composed of a floating diffusion FD and a charge storage capacitance C. However, the charge storage unit CS only needs to be composed of a floating diffusion FD, and the pixel 321 may be configured without a charge storage capacitance C.

[0048] Furthermore, while Figure 3 shows an example of a configuration in which pixel 321 includes a drain gate transistor GD, a configuration without a drain gate transistor GD is also acceptable if there is no need to discard the charge accumulated (remaining) in the photoelectric conversion element PD.

[0049] The photoelectric conversion element PD is an embedded photodiode that converts incident light into electricity to generate an electric charge and stores the generated charge. The structure of the photoelectric conversion element PD can be arbitrary. For example, the photoelectric conversion element PD may be a PN photodiode with a structure in which a P-type semiconductor and an N-type semiconductor are joined, or a PIN photodiode with a structure in which an I-type semiconductor is sandwiched between a P-type semiconductor and an N-type semiconductor. Furthermore, the photoelectric conversion element PD is not limited to a photodiode, but may be, for example, a photogate type photoelectric conversion element.

[0050] In pixel 321, the photoelectric conversion element PD converts incident light at an accumulation timing synchronized with the timing of irradiation with the light pulse PO into electric charge, and distributes and stores the converted charge in each of the three charge storage units CS. For light incident on pixel 321 at timings other than the accumulation timing, the charge converted by the photoelectric conversion element PD is discharged from the drain gate transistor GD, preventing it from accumulating in the charge storage unit CS. This process of charge accumulation at the accumulation timing and charge discarding at timings other than the accumulation timing is repeated over one frame, after which a readout period is provided. During the readout period, the horizontal scanning circuit 324 outputs an electrical signal corresponding to the amount of charge accumulated in each of the charge storage units CS for one frame to the distance calculation unit 42.

[0051] The distance calculation unit 42 calculates the delay time Td by the following equation (1), utilizing the fact that the amount of charge corresponding to the reflected light RL component is distributed and accumulated in the two charge storage units CS at a ratio corresponding to the delay time Td until the reflected light RL is incident on the distance image capturing device 1. The distance calculation unit 42 calculates the round-trip distance to the subject S by multiplying the delay time Td obtained by equation (1) by the speed of light (velocity). Then, the distance calculation unit 42 determines the distance to the subject S by dividing the round-trip distance calculated above by 1 / 2. Note that equation (1) assumes that the amount of charge corresponding to the ambient light component is accumulated in charge storage unit CS1, and the amount of charge corresponding to the reflected light RL component is distributed and accumulated in charge storage units CS2 and CS3.

[0052] Td=To×(Q3-Q1) / (Q2+Q3-2×Q1) …(1) However, To is the period during which the optical pulse PO was irradiated. Q1 is the amount of charge stored in the charge storage unit CS1. Q2 is the amount of charge stored in the charge storage unit CS2. Q3 is the amount of charge stored in the charge storage unit CS3.

[0053] In this embodiment, dot light is used as the light source emitted by the light source unit 2. The dot light source is, for example, structured light composed of multiple dots arranged periodically.

[0054] By using a dot light source, a non-uniform, localized light pulse PO is irradiated onto the subject OB. Using dot light allows for an increase in the power of the irradiated light (irradiation intensity per unit area) without increasing the light source output, and thus increases the reach of the irradiated light and the measurable distance. On the other hand, when using a dot light source, distance cannot be measured in areas not irradiated by the dot light, resulting in a problem of low resolution.

[0055] As a countermeasure, in this embodiment, dot light having an elliptical shape is used as the light pulse PO. For example, the light source unit 2 has a plurality of light source elements, each capable of independently irradiating a light pulse PO (dot light having an elliptical shape).

[0056] Figure 4 shows an example of dot light in an embodiment. Figure 4 schematically shows how the distance image acquisition device 1 irradiates the subject OB with dot light Dt as an optical pulse PO. As shown in Figure 4, the dot light Dt has an elliptical shape in which the ratio of the major axis length LA to the minor axis length SA is greater than or equal to a threshold. The threshold here can be set arbitrarily. For example, the shape of the dot light source is an elliptical shape in which the ratio of the major axis length LA to the minor axis length SA is 2 or more. In this embodiment, by using dot light Dt having an elliptical shape, the area of ​​the subject OB that is not illuminated by light can be reduced compared to when circular dot light Dt is used. Therefore, it is possible to suppress a decrease in resolution.

[0057] The variations of the dot light Dt will be explained below using Figures 5 to 8. Figures 5 to 8 show other examples of the dot light in the first embodiment.

[0058] Figure 5 shows a configuration in which a portion of the dot light Dt overlaps with a portion of another dot light Dt. In this embodiment, at least a portion of the dot light Dt1 may overlap with at least a portion of another dot light Dt2 adjacent to the long axis of the dot light Dt1. This makes it possible to further reduce the area of ​​the subject OB that is not illuminated by light.

[0059] In Figure 5, the overlapping along the long axis is illustrated as an example, but the overlapping may also occur along the short axis. Furthermore, the overlapping may be configured along any axis set diagonally to the long and short axes. By overlapping at least a portion of the dot light Dt with other dot light Dt, the area that is not illuminated by light can be reduced, and the same effects as described above can be achieved.

[0060] As shown in Figure 5, by irradiating multiple dot beams Dt so that they overlap along the long axis, the light pulse PO can be made into a linear beam. In the following description, the light obtained by irradiating multiple dot beams Dt so that they overlap along the long axis will be referred to as "line beam L". Note that a line light source may be used instead of, or in conjunction with, the dot light source in this embodiment.

[0061] Figure 6 shows how multiple line beams L intersect. In this embodiment, multiple line beams L may be configured to point in different directions from one another. That is, the major axes of the elliptical dot beams Dt may be configured to be different from each other. This allows the subject OB to be illuminated in a mesh-like pattern, and the areas of the subject OB that are not illuminated can be reduced.

[0062] Furthermore, if all the line beams are directed parallel to each other, there is a possibility that the area illuminated around the subject OB may narrow if the beams overlap for some reason. However, by directing the line beams L in different directions, it is possible to prevent the beams from overlapping except at intersections. Therefore, the area illuminated around the subject OB can be prevented from narrowing.

[0063] Figure 6 shows an example where the directions of the line rays in the group consisting of line rays L1 to L6 and the group consisting of line rays L7 to L12 are orthogonal to each other, i.e., 90 degrees apart. In this way, the major axes of the elliptical dot rays Dt emitted from at least two of the multiple light source elements may be configured to be orthogonal to each other.

[0064] Furthermore, the configuration is not limited to one where the angle between the two line beams L is 90 degrees; it is sufficient that at least the directions of the two line beams L are different. In other words, the angle between the two line beams L is greater than 0 degrees and less than or equal to 90 degrees.

[0065] Figure 7 shows how the distances between parallel line beams L differ. For example, the distance K1 from line beam L101 to line beam L102 and the distance K2 from line beam L103 to line beam L104 are of different magnitudes.

[0066] Thus, in this embodiment, the spacing of the line light L may be configured to be different in the vertical and horizontal directions. That is, the first spacing along the minor axis direction of the elliptical dot light Dt emitted from a first group of light sources consisting of at least two light sources among a plurality of light sources, where the major axes are in the same direction, and the second spacing along the minor axis direction of the elliptical dot light Dt emitted from a second group of light sources consisting of at least two light sources different from the first group, where the major axes are in the same direction, may be configured to be different from each other. This makes it possible to perform measurements in the subject OB by dividing it into areas where distance is measured finely and areas where distance is measured coarsely.

[0067] For example, when measuring a subject OB that moves horizontally, it is possible to increase the resolution of distance measurement in the horizontal direction and decrease the resolution in the vertical direction. In such cases, the light is shone in such a way that the spacing of the line light L along the vertical direction is narrowed and the spacing of the line light L along the horizontal direction is widened. This makes it possible to measure the area where distance measurement is desired at fine intervals with high resolution.

[0068] Furthermore, the spacing between the vertically aligned line light groups L, or the spacing between the vertically aligned line light groups L, may be configured to be different, i.e., not equally spaced. For example, this can be applied to cases where, such as when measuring the distance of a short subject, the resolution at the top is lower and the resolution from the center to the bottom is higher. In this case, the horizontally aligned line light groups L are illuminated in such a way that the spacing between them is wider at the top and narrower from the center to the bottom. This makes it possible to measure the area where the subject OB exists with high resolution.

[0069] Figure 8 shows how line light L is projected onto the mounting surface (ground) of the depth image acquisition device 1 in a direction different from the vertical and horizontal directions. For example, the angle between line light L201 and line segment Ax along the x-axis (horizontal direction) is 135 degrees. Also, the angle between line light L202 and line segment Ax is 45 degrees.

[0070] Thus, in this embodiment, the line light L may be configured to point in a direction oblique to the ground. That is, the angle in the direction of the major axis of the elliptical dot light emitted from two of the multiple light source elements is 45 degrees or 135 degrees with respect to the mounting surface of the imaging device.

[0071] Thus, in this embodiment, the line light L may be configured to point in an oblique direction with respect to the ground. That is, the major axis direction of the elliptical dot light Dt emitted from two of the multiple light source elements may be configured to be oblique to the mounting surface of the distance image acquisition device 1, rather than perpendicular or horizontal.

[0072] When attempting to create both a line of light L perpendicular to the ground and a line of light L horizontally by combining a light source with a diffuser plate, separate diffusers are required for the vertical and horizontal directions if the FOI (Field of Illumination) is not the same for the horizontal and vertical directions. In contrast, when aligning two diagonal line of light L perpendicularly, two diffusers of the same design that illuminate the diagonal line of light L can be prepared and installed with their tops and bottoms reversed. This allows the two diagonal line of light L to be perpendicular. Therefore, by aligning the diagonal direction perpendicularly, it is possible to reduce the equipment cost related to the light source unit 2 compared to aligning the horizontal and vertical directions perpendicularly.

[0073] Furthermore, in this embodiment, a configuration may be used in combination with a uniform diffusion light source element in addition to the dot light Dt. A uniform diffusion light source element is a light source element that irradiates the irradiation surface with uniform light. By using a uniform diffusion light source element, it becomes possible to irradiate the subject OB with uniform light.

[0074] This allows the distance image acquisition device 1 of this embodiment to use different light sources depending on the distance to the subject OB. For example, when measuring the distance to a subject OB that is close, a uniform diffuse light source is used. This means that if the subject OB is nearby and sufficient light can be irradiated onto the subject OB without using dot light Dt, the distance can be measured with high resolution using the uniform diffuse light source element without unnecessarily reducing the resolution by using dot light Dt. On the other hand, when measuring the distance to a subject OB that is far away, a dot light source is used. This means that even if the subject OB is far away, the amount of reflected light RL can be increased by using dot light Dt, and the shape of the dot light Dt can be made elliptical to minimize the area of ​​the subject OB that is not illuminated by light, thereby suppressing a decrease in resolution when measuring the distance. Therefore, the dot light and diffuse light source can be used depending on the measurement conditions according to the distance to the subject OB, and a decrease in resolution can be suppressed in both the near and far distance cases.

[0075] As described above, in the distance image acquisition device 1 of this embodiment, the light pulse PO is structured light composed of multiple dot lights Dt. At least one of the multiple dot lights Dt (first dot light) has an elliptical shape in which the ratio of the major axis length to the minor axis length is greater than or equal to a threshold. This makes it possible in the distance image acquisition device 1 of this embodiment to locally increase the power of the light irradiated using the dot lights Dt while reducing the area in the subject OB that is not illuminated by light. Therefore, it is possible to suppress a decrease in the resolution of the distance image captured using the dot light source.

[0076] Furthermore, in the distance image capturing device 1 of the embodiment, at least a portion of at least one dot light Dt1 (first dot light) among the multiple dot lights Dt overlaps with at least a portion of another dot light Dt2 adjacent to the dot light Dt1 in the longitudinal direction. As a result, the distance image capturing device 1 of the embodiment makes it possible to further reduce the area in the subject OB that is not illuminated by light.

[0077] Furthermore, in the distance image acquisition device 1 of this embodiment, the light source unit 2 has a plurality of light source elements, each capable of independently irradiating with an optical pulse PO. As a result, in the distance image acquisition device 1 of this embodiment, each of the plurality of light source elements can be used to irradiate an elliptical dot of light, increasing the degree of freedom in setting the position and orientation of the irradiated dot of light.

[0078] Furthermore, in the distance image acquisition device 1 of this embodiment, the major axis directions of the elliptical dot light emitted from at least two of the multiple light source elements are different from each other. As a result, in the distance image acquisition device 1 of this embodiment, line light L that intersects with the subject OB can be used to illuminate the subject OB in a mesh-like pattern, making it possible to reduce the areas of the subject OB that are not comprehensively illuminated.

[0079] Furthermore, in the distance image acquisition device 1 of the embodiment, the major axes of the elliptical dot light emitted from at least two of the multiple light source elements are orthogonal to each other. As a result, the distance image acquisition device 1 of the embodiment can illuminate the subject OB in a grid pattern, producing the same effects as described above.

[0080] Furthermore, in the distance image imaging device 1 of the embodiment, distances K1 and K2 are different. Distance K1 is the distance from line light L101 to line light L102, and is an example of a "first interval". Distance K2 is the distance from line light L103 to line light L104, and is an example of a "second interval". As a result, the distance image imaging device 1 of the embodiment can perform measurements with different resolutions in the horizontal or vertical direction. For example, when measuring a subject OB that moves horizontally, it becomes possible to measure the distance that changes according to the horizontal movement with high resolution.

[0081] Furthermore, in the distance image acquisition device 1 of the embodiment, the spacing along the minor axis of the elliptical dot light emitted from at least three of the multiple light source elements, whose major axes are in the same direction, is different from one another. As a result, the distance image acquisition device 1 of the embodiment can perform measurements with a resolution corresponding to the area in which the subject exists. For example, when measuring a short subject OB, it is possible to measure the area where the upper subject OB does not exist with a low resolution, and measure the area where the subject OB exists from the center downwards with a high resolution.

[0082] Furthermore, in the distance image acquisition device 1 of the embodiment, the major axis direction of the elliptical dot light emitted from at least two of the multiple light source elements is oblique to the mounting surface of the imaging device, rather than perpendicular or horizontal. As a result, in the distance image acquisition device 1 of the embodiment, such line light can be realized by mounting two diffusers that irradiate line light L in an oblique direction so that their tops and bottoms are inverted relative to each other, and the device cost can be reduced compared to using separate diffusers for vertical and horizontal illumination.

[0083] Furthermore, in the distance image capturing device 1 of the embodiment, the angle in the direction of the major axis of the elliptical dot light emitted from at least two of the multiple light source elements is 45 degrees or 135 degrees with respect to the mounting surface of the imaging device. As a result, the distance image capturing device 1 of the embodiment can irradiate line light obliquely to the ground and symmetrically in the left, right, up, and down directions, achieving the same effects as described above.

[0084] Furthermore, in the distance image acquisition device 1 of this embodiment, at least one of the multiple light source elements is a diffuse light source. As a result, the distance image acquisition device 1 of this embodiment can use dot light and a diffuse light source depending on the measurement conditions corresponding to the distance to the subject OB, and can suppress a decrease in resolution in both short-range and long-range cases.

[0085] The distance image acquisition device 1 and distance image processing unit 4 in the above-described embodiment may be implemented in whole or in part by a computer. In that case, the program for implementing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be loaded into a computer system and executed. Here, "computer system" includes hardware such as the OS and peripheral devices. Furthermore, "computer-readable recording medium" refers to portable media such as flexible disks, magneto-optical disks, ROMs, CD-ROMs, and storage devices such as hard disks built into a computer system. Moreover, "computer-readable recording medium" may also include those that dynamically hold programs for a short period of time, such as communication lines used when transmitting programs via networks such as the Internet or communication lines such as telephone lines, and those that hold programs for a certain period of time, such as volatile memory inside a computer system that acts as a server or client in such cases. Furthermore, the above-mentioned program may be for implementing a part of the above-mentioned function, or it may be a program that can implement the above-mentioned function in combination with a program already recorded in the computer system, or it may be implemented using a programmable logic device such as an FPGA.

[0086] While embodiments of this invention have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments and includes designs and the like that do not depart from the spirit of this invention. [Explanation of symbols]

[0087] 1... Distance imaging device 2...Light source section 3...Light receiving section 32... Distance image sensor 321... pixels 42...Distance calculation section CS…Charge storage section PO... Light pulse RL…Reflected light Dt... Dot Light L...Line light

Claims

1. A light source unit that irradiates the subject with light pulses, A pixel circuit arranged in a two-dimensional matrix, comprising a photoelectric conversion element that generates an electric charge corresponding to incident light and a plurality of charge storage units that store the electric charge; a pixel driving circuit that distributes and stores the electric charge to each of the charge storage units at an accumulation timing synchronized with the irradiation of the light pulse; and a light receiving unit having a charge discharge means for discharging the electric charge during periods when it is not the accumulation timing. A distance calculation unit that calculates the distance to the subject based on the amount of charge accumulated in each of the charge storage units, Equipped with, The aforementioned light pulse is structured light composed of multiple dots of light, Of the plurality of dot lights, at least one first dot light has an elliptical shape in which the ratio of the major axis length to the minor axis length is greater than or equal to a threshold, and at least a portion of the first dot light overlaps with at least a portion of another dot light adjacent to the first dot light in the direction of the major axis. Distance imaging device.

2. A light source unit that irradiates the subject with light pulses, A pixel circuit arranged in a two-dimensional matrix, comprising a photoelectric conversion element that generates an electric charge corresponding to incident light and a plurality of charge storage units that store the electric charge; a pixel driving circuit that distributes and stores the electric charge to each of the charge storage units at an accumulation timing synchronized with the irradiation of the light pulse; and a light receiving unit having a charge discharge means for discharging the electric charge during periods when it is not the accumulation timing. A distance calculation unit that calculates the distance to the subject based on the amount of charge accumulated in each of the charge storage units, Equipped with, The aforementioned light pulse is structured light composed of multiple dots of light, At least one of the plurality of dot lights, the first dot light, has an elliptical shape in which the ratio of the major axis length to the minor axis length is greater than or equal to a threshold. The light source unit has a plurality of light source elements, each capable of independently irradiating the light pulse, and the major axis directions of the elliptical dot light emitted from at least two of the plurality of light source elements are different from each other. Distance imaging device.

3. In the elliptical dot light emitted from at least two of the aforementioned multiple light source elements, the major axes of the light are orthogonal to each other. The distance image acquisition device according to claim 2.

4. Regarding the first interval along the minor axis direction of dot light in which the major axes of the ellipse shape are in the same direction, emitted from a first group of light sources consisting of at least two light sources from the plurality of light sources, and the second interval along the minor axis direction of dot light in which the major axes of the ellipse shape are in the same direction, emitted from a second group of light sources consisting of at least two light sources different from the first group of light sources, the first interval and the second interval are different from each other. The distance image acquisition device according to claim 3.

5. The major axis direction of the elliptical dot light emitted from at least two of the plurality of light source elements is oblique to the mounting surface of the imaging device, and is neither perpendicular nor horizontal. The distance image acquisition device according to claim 3.

6. The angle in the direction of the major axis of the elliptical dot light emitted from two of the aforementioned multiple light source elements is 45 degrees or 135 degrees with respect to the mounting surface of the imaging device. The distance image acquisition device according to claim 5.

7. At least one of the aforementioned plurality of light source elements is a diffuse light source. The distance image acquisition device according to claim 2.

8. A distance image capturing method performed by a distance image capturing device comprising: a light source unit that irradiates a subject with light pulses; a plurality of pixel circuits arranged in a two-dimensional matrix, each having a photoelectric conversion element that generates an electric charge corresponding to the incident light and a plurality of charge storage units that store the electric charge; a pixel driving circuit that distributes and stores the electric charge in each of the charge storage units at an accumulation timing synchronized with the irradiation of the light pulses; a light receiving unit having a charge discharge means for discharging the electric charge during periods when the accumulation timing is not occurring; and a distance calculation unit that calculates the distance to the subject based on the amount of electric charge stored in each of the charge storage units, wherein The aforementioned light pulse is structured light composed of multiple dots of light, Of the plurality of dot lights, at least one first dot light has an elliptical shape in which the ratio of the major axis length to the minor axis length is greater than or equal to a threshold, and at least a portion of the first dot light overlaps with at least a portion of another dot light adjacent to the first dot light in the direction of the major axis. A method for acquiring distance images.

9. A distance image capturing method performed by a distance image capturing device comprising: a light source unit that irradiates a subject with light pulses; a plurality of pixel circuits arranged in a two-dimensional matrix, each having a photoelectric conversion element that generates an electric charge corresponding to the incident light and a plurality of charge storage units that store the electric charge; a pixel driving circuit that distributes and stores the electric charge in each of the charge storage units at an accumulation timing synchronized with the irradiation of the light pulses; a light receiving unit having a charge discharge means for discharging the electric charge during periods when the accumulation timing is not occurring; and a distance calculation unit that calculates the distance to the subject based on the amount of electric charge stored in each of the charge storage units, wherein The aforementioned light pulse is structured light composed of multiple dots of light, At least one of the plurality of dot lights, the first dot light, has an elliptical shape in which the ratio of the major axis length to the minor axis length is greater than or equal to a threshold. The light source unit has a plurality of light source elements, each capable of independently irradiating the light pulse, and the major axis directions of the elliptical dot light emitted from at least two of the plurality of light source elements are different from each other. A method for acquiring distance images.

Images