Range image capturing device and range image capturing method

Abstract

A range imaging device (1) is provided with: a light source unit (2) that irradiates a subject (OB) with a light pulse (PO); and a light receiving unit (3) having a pixel circuit (321) provided with a photoelectric conversion element (PD) for generating charges corresponding to incident light and a plurality of charge storage units (CS) for storing the charges, a pixel drive circuit (323) for distributing and storing the charges to the charge storage units (CS), and a charge discharge means (GD) for discharging the charges from the charge storage units (CS). The charge discharge unit discharges charges; and a distance calculation unit (42) that calculates the distance to the subject (OB), the light pulse (PO) being structured light comprising a plurality of point lights (Dt), at least one first point light (Dt1) among the plurality of point lights (Dt) having an elliptical shape in which the ratio of the long axis length (LA) to the short axis length (SA) is equal to or greater than a threshold value.

Description

Technical Field

[0001] The present invention relates to a distance image camera device and a distance image camera method. This application claims priority based on Special Application No. 2022-092847 filed in Japan on June 8, 2022, Special Application No. 2022-191493 filed in Japan on November 30, 2022, and Special Application No. 2022-191128, and the contents thereof are hereby incorporated by reference. Background Art

[0002] Time of Flight (hereinafter referred to as "TOF") distance image sensors have been realized, which use the fact that the speed of light is known to measure the distance between a measuring device and an object based on the flight time of light in a space (measurement space) (for example, see Patent Document 1). In such a distance image camera, reflected light is made incident on an imaging element, and charges corresponding to the amount of reflected light are distributed and accumulated in a plurality of charge accumulation parts, thereby obtaining a delay time from the time of irradiation of a light pulse to the time when the reflected light after being reflected by the subject returns, and the distance to the subject is calculated using the delay time and the speed of light.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent No. 4235729 Summary of the invention

[0006] Problems to be solved by the invention

[0007] In a system that uses the TOF method to take images, one of the factors that determine the longest distance that can be measured is the power of the light source used for measurement (irradiation intensity per unit area). Although the light source output can be increased by increasing the current flowing in the light-emitting element, increasing the current flowing in the light-emitting element will cause the imaging system to generate heat, increase power consumption, and reduce the safety of the laser to the human body.

[0008] As a countermeasure to the above problem, there is the following means: use a point light source instead of a uniform diffuse light source that is usually used as a distance measurement light source. By using a diffuser that converts the light emitted from the light source element into periodically arranged point light, the power of the light can be concentrated at a point, and even if the current flowing in the light-emitting element is the same, the use of a point light source can measure a farther distance than the case of using diffuse light that uniformly illuminates the irradiated surface.

[0009] However, when using a point light source to measure distance, the distance can only be measured for the area illuminated by the point light, and the distance cannot be measured for the area between the points, that is, the area of ​​the subject that is not illuminated by the irradiation light. Therefore, there is a problem that the resolution of the distance image captured using a point light source is lower than that of the distance image measured using uniform diffuse light.

[0010] As a countermeasure to the above problem, there is the following means: use linear light radiated in a linear shape instead of using uniform diffuse light which is usually used as distance measuring light. For example, by radiating two line groups consisting of a plurality of parallel line lights orthogonally, it is possible to suppress the necessary light source output and the reduction in resolution compared to the case of measuring the distance image using uniform diffuse light.

[0011] However, in the above-mentioned means using line light, the brightness of the intersection of the orthogonal line lights becomes twice the brightness of the line lights of other parts, so the output of the pixel corresponding to the intersection is saturated, and there is a possibility that the distance measurement cannot be performed. On the other hand, in the case of reducing the output to avoid saturation, in the above-mentioned means using line light, there is a possibility that the resolution of the line part other than the intersection and the measurable distance are reduced.

[0012] The present invention is completed based on the above-mentioned problems, and its purpose is to provide a distance image capture device and a distance image capture method that can suppress the reduction in resolution of a distance image captured using a point light source, and to provide a distance image capture device and a distance image capture method that can alleviate the signal output difference between the intersection of line light and the portion other than the intersection and suppress the reduction in the overall resolution of the distance image and the measurable distance.

[0013] Means for solving problems

[0014] The first mode of the present invention is a distance image camera device, comprising: a light source unit, which irradiates a light pulse to a subject; a light receiving unit, which has a pixel circuit, a pixel driving circuit and a charge discharge unit, wherein the pixel circuit is arranged in a two-dimensional matrix and has a photoelectric conversion element that generates charges corresponding to the incident light and a plurality of charge storage units that accumulate the charges, the pixel driving circuit distributes the charges to the charge storage units and accumulates them at an accumulation timing synchronized with the irradiation of the light pulse, and the charge discharge unit discharges the charges during a period other than the accumulation timing; and a distance calculation unit, which calculates the distance to the subject based on the amount of charge accumulated by the charge storage units, the light pulse is a structured light composed of a plurality of point lights, and at least one first point light among the plurality of point lights has an elliptical shape with a ratio of a major axis length to a minor axis length that is greater than a threshold value.

[0015] A second aspect of the present invention is the distance image capturing device according to the first aspect, wherein at least a portion of the first point light overlaps with at least a portion of another point light adjacent to the first point light in the long axis direction.

[0016] A third aspect of the present invention is the distance image capturing device according to the first aspect, wherein the light source unit includes a plurality of light source elements capable of independently irradiating the light pulses.

[0017] A fourth aspect of the present invention is the distance image capturing device according to the third aspect, wherein major axis directions of the elliptical spot lights emitted from at least two of the plurality of light source elements are different from each other.

[0018] A fifth aspect of the present invention is the distance image capturing device according to the fourth aspect, wherein major axis directions of the elliptical spot lights emitted from at least two of the plurality of light source elements are orthogonal to each other.

[0019] The sixth aspect of the present invention is a distance image camera device according to the fifth aspect, wherein a first interval along the short axis direction of point lights with their major axes in the same direction and emitted from a first light source element group consisting of at least two light source elements among the plurality of light source elements, and a second interval along the short axis direction of point lights with their major axes in the same direction and emitted from a second light source element group consisting of at least two light source elements different from the first light source element group, are different from each other.

[0020] The seventh mode of the present invention is a distance image camera device according to the fifth mode, wherein the major axis direction of the elliptical point light emitted from at least two of the multiple light source elements is an inclined direction other than the vertical direction and the horizontal direction relative to the setting surface of the camera device.

[0021] The eighth aspect of the present invention is a distance image camera device according to the seventh aspect, wherein the angle formed by the major axis direction of the elliptical point light emitted from two of the plurality of light source elements relative to the setting surface of the camera device is 45 degrees or 135 degrees.

[0022] A ninth aspect of the present invention is the distance image capturing device according to the third aspect, wherein at least one of the plurality of light source elements is a diffuse light source.

[0023] The tenth mode of the present invention is a distance image capturing method, which is performed by a distance image capturing device, wherein the distance image capturing device comprises: a light source unit, which irradiates a light pulse to a subject; a light receiving unit, which has a pixel circuit, a pixel driving circuit and a charge discharge unit, wherein the pixel circuit is arranged in a two-dimensional matrix in plurality and comprises a photoelectric conversion element that generates a charge corresponding to the incident light and a plurality of charge storage units that accumulate the charge, wherein the pixel driving circuit distributes the charge to the charge storage units respectively and accumulates the charge at an accumulation timing synchronized with the irradiation of the light pulse, and the charge discharge unit discharges the charge during a period other than the accumulation timing; and a distance calculation unit, which calculates the distance to the subject based on the amount of charge accumulated by the charge storage units respectively, wherein in the distance image capturing method, the light pulse is a structured light composed of a plurality of point lights, and at least one of the plurality of point lights has an elliptical shape in which the ratio of the major axis length to the minor axis length is greater than a threshold value.

[0024] The eleventh aspect of the present invention is a distance image camera device, comprising: a light source unit for irradiating a light pulse to a subject; a light receiving unit having a pixel circuit, a pixel driving circuit, and a charge discharge unit, wherein the pixel circuit is arranged in a two-dimensional matrix and has a plurality of photoelectric conversion elements for generating charges corresponding to incident light and a plurality of charge storage units for storing the charges, the pixel driving circuit distributing the charges to the charge storage units at a storage timing synchronized with the irradiation of the light pulse according to a frame period and causing the charges to be stored, and the charge discharge unit distributing the charges to the charge storage units at a storage timing not synchronized with the irradiation of the light pulse according to a frame period. The charge is discharged during a period of time; and a distance calculation unit calculates the distance to the subject based on the amount of charge accumulated in the charge accumulation unit, the light source unit radiates the light pulse as structured light composed of periodically arranged line lights, and the light pulses are orthogonal to each other in two directions of line lights, the frame period has a plurality of sub-frame periods with different accumulation periods for accumulating the charge in the charge accumulation unit, and the distance calculation unit calculates the distance to the subject using the sub-frame period in which the amount of charge does not exceed a preset threshold value among the plurality of sub-frame periods.

[0025] The 12th mode of the present invention is a distance image camera device according to the 11th mode, wherein the multiple subframe periods are a first subframe period in which the accumulation period is a predetermined period set in advance, and a second subframe period in which the accumulation period is a period greater than twice the predetermined period, and the distance calculation unit calculates the distance to the subject using the subframe period in which the charge amount does not exceed the threshold and the accumulation period is longer among the first subframe period and the second subframe period.

[0026] A thirteenth aspect of the present invention is a distance image camera device, comprising: a light source unit for irradiating a light pulse to a subject; a light receiving unit having a pixel circuit, a pixel driving circuit, and a charge discharge unit, wherein a plurality of the pixel circuits are arranged in a two-dimensional matrix and have a photoelectric conversion element that generates charges corresponding to the incident light and a plurality of charge storage units that store the charges, wherein the pixel driving circuit distributes the charges to the charge storage units at a storage timing synchronized with the irradiation of the light pulse according to a frame cycle and stores the charges, and the charge discharge unit discharges the charges during a period other than the storage timing; and a distance calculation unit for calculating the distance to the subject based on the amount of charge stored in the charge storage units, wherein the light source unit is capable of radiating structured light composed of periodically arranged line lights in two directions orthogonal to each other, as the light pulse, and the frame cycle has a plurality of sub-frame cycles in which the line directions of the line lights are different, and the distance calculation unit calculates the synthesized distance to the subject based on the amount of charge stored in each of the plurality of sub-frame cycles in which the line directions are different.

[0027] The 14th mode of the present invention is a distance image camera device according to the 13th mode, wherein the light source unit irradiates the light pulse by alternately changing the line light with different line directions in each sub-frame period in the frame period, and the distance calculation unit calculates the synthesized distance to the subject based on the amount of charge accumulated in each of the multiple sub-frame periods in which the line directions are alternately changed.

[0028] The 15th mode of the present invention is a distance image camera device according to the 13th mode, wherein the light source unit changes the ratio of the sub-frame periods corresponding to the line lights in the two directions in the plurality of sub-frame periods according to the moving direction of the subject in the frame period, and irradiates the light pulse.

[0029] The 16th mode of the present invention is a distance image camera device according to the 15th mode, wherein the light source unit irradiates the light pulse in the frame period by changing the proportion of the sub-frame period corresponding to the direction in which the movement amount of the subject is larger among the two directions to a proportion greater than the sub-frame period corresponding to the direction in which the movement amount of the subject is smaller.

[0030] The 17th mode of the present invention is a distance image camera device according to any one of the 11th mode to the 16th mode, wherein the light source unit has a plurality of light source elements that can independently irradiate the light pulses respectively, and the line light is formed by radiating a plurality of point lights discretely into a line by the plurality of light source elements, or by radiating the plurality of point lights in an overlapping manner into a line.

[0031] The 18th mode of the present invention is a distance image camera device according to the 17th mode, wherein the shape of the point light is an elliptical shape with a ratio of a major axis length to a minor axis length of 2 or more, and the line light is formed into a linear shape along the direction of the major axis.

[0032] The 19th mode of the present invention is a distance image capturing method, which is performed by a distance image capturing device, wherein the distance image capturing device comprises: a light source unit, which irradiates a light pulse to a subject; a light receiving unit, which has a pixel circuit, a pixel driving circuit and a charge discharge unit, wherein the pixel circuit is arranged in a two-dimensional matrix in plurality and has a photoelectric conversion element that generates a charge corresponding to the incident light and a plurality of charge storage units that store the charge, wherein the pixel driving circuit distributes the charge to the charge storage units respectively and stores the charge at a storage timing synchronized with the irradiation of the light pulse according to a frame period, and the charge discharge unit stores the charge when the charge is not received by the storage unit. The electric charges are discharged during a timing period; and a distance calculation unit calculates the distance to the subject based on the amount of charge respectively accumulated in the charge accumulation unit, wherein in the distance image capture method, the light source unit radiates the light pulse as structured light composed of periodically arranged line lights, and the light pulses are orthogonal to each other in two directions of the line lights, and the frame period has a plurality of sub-frame periods with different accumulation periods for accumulating the electric charges in the charge accumulation unit, and the distance calculation unit calculates the distance to the subject using the sub-frame period in which the amount of charge does not exceed a preset threshold value among the plurality of sub-frame periods.

[0033] The 20th mode of the present invention is a distance image capturing method, which is performed by a distance image capturing device, wherein the distance image capturing device comprises: a light source unit, which irradiates a light pulse to a subject; a light receiving unit, which has a pixel circuit, a pixel driving circuit and a charge discharge unit, wherein the pixel circuit is arranged in a two-dimensional matrix in plurality and has a photoelectric conversion element that generates a charge corresponding to the incident light and a plurality of charge storage units that store the charge, wherein the pixel driving circuit distributes the charge to the charge storage units respectively and stores the charge at a storage timing synchronized with the irradiation of the light pulse according to a frame period, and the charge discharge units are arranged in parallel. The charges are discharged during periods other than the accumulation timing; and a distance calculation unit calculates the distance to the subject based on the charges respectively accumulated in the charge accumulation unit, wherein in the distance image capture method, the light source unit is capable of radiating structured light composed of periodically arranged line lights, and the line lights are in two directions orthogonal to each other, as the light pulses, and the frame period has a plurality of sub-frame periods with different line directions of the line lights, and the distance calculation unit calculates the synthesized distance to the subject based on the charges accumulated in each of the plurality of sub-frame periods with different line directions.

[0034] The 21st mode of the present invention is a distance image camera device, comprising: a light source unit for irradiating a light pulse to a subject; a light receiving unit having a pixel circuit, a pixel driving circuit and a charge discharge unit, wherein the pixel circuit is arranged in a two-dimensional matrix in plurality and has a photoelectric conversion element for generating a charge corresponding to the incident light and a plurality of charge storage units for storing the charge, the pixel driving circuit respectively distributing the charge to the charge storage units and storing the charge at a storage timing synchronized with the irradiation of the light pulse according to a frame period, and the charge discharge unit is configured to discharge the charge at a time when the charge discharge is not in the storage timing. The charges are discharged during the period; and a distance calculation unit calculates the distance to the subject based on the amount of charges respectively accumulated in the charge accumulation unit, the light source unit radiates the light pulse as structured light composed of periodically arranged line lights, and the line lights in two directions are orthogonal to each other, and the first pixel circuit corresponding to the position for receiving reflected light from the subject irradiated orthogonally to each other among the plurality of pixel circuits is configured to have lower sensitivity to reflected light than the second pixel circuits corresponding to other positions.

[0035] A 22nd aspect of the present invention is the distance image capture device according to the 21st aspect, further comprising a filter layer, wherein the filter layer is arranged on the first pixel circuit and reflects or absorbs a part of the reflected light.

[0036] A 23rd aspect of the present invention is the distance image capturing device according to the 21st aspect, wherein a capacity of the charge storage section of the first pixel circuit is configured to be larger than a capacity of the charge storage section of the second pixel circuit.

[0037] A 24th aspect of the present invention is the distance image capture device according to the 21st aspect, wherein the volume of the photoelectric conversion element of the first pixel circuit is configured to be smaller than the volume of the photoelectric conversion element of the second pixel circuit.

[0038] A 25th aspect of the present invention is the distance image capturing device according to the 21st aspect, wherein the number of allocations in the frame period of the first pixel circuit is configured to be smaller than the number of allocations in the frame period of the second pixel circuit.

[0039] The 26th mode of the present invention is a distance image camera device according to any one of the 21st mode to the 25th mode, wherein the light source unit has a plurality of light source elements that can independently irradiate the light pulses respectively, and the line light is formed by radiating a plurality of point lights discretely into a line by the plurality of light source elements, or by radiating the plurality of point lights in an overlapping manner into a line.

[0040] In addition, the 27th mode of the present invention is a distance image camera device according to the 26th mode, wherein the shape of the point light is an elliptical shape with a ratio of the major axis length to the minor axis length being greater than 2, and the line light is formed into a linear shape along the direction of the major axis.

[0041] Effects of the Invention

[0042] According to the present invention, it is possible to suppress a reduction in resolution of a distance image captured using a point light source, and to alleviate a signal output difference between an intersection of line lights and portions other than the intersection, thereby suppressing a reduction in resolution of the entire distance image and a measurable distance. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a block diagram showing a schematic configuration of a distance image capturing device according to the first embodiment.

[0044] Figure 2 This is a block diagram showing a schematic configuration of a distance image sensor according to the first embodiment.

[0045] Figure 3 This is a circuit diagram showing an example of the configuration of a pixel according to the first embodiment.

[0046] Figure 4 This is a diagram showing an example in which a subject is irradiated with spot light according to the first embodiment.

[0047] Figure 5 This is a diagram showing an example in which a subject is irradiated with spot light according to the first embodiment.

[0048] Figure 6 This is a diagram showing an example in which a subject is irradiated with spot light according to the first embodiment.

[0049] Figure 7 This is a diagram showing an example in which a subject is irradiated with spot light according to the first embodiment.

[0050] Figure 8 This is a diagram showing an example in which a subject is irradiated with spot light according to the first embodiment.

[0051] Fig. 9 This is a block diagram showing an example of a distance image capturing device according to the second embodiment.

[0052] Fig.10 This is a block diagram showing an example of a distance image sensor in the second embodiment.

[0053] Fig.11 This is a block diagram showing an example of a pixel circuit in the second embodiment.

[0054] Fig.12FIG. 1 is a first diagram showing an example of line light in the second embodiment.

[0055] Fig.13 FIG. 2 is a second diagram showing an example of line light in the second embodiment.

[0056] Fig.14 FIG. 3 is a third diagram showing an example of line light in the second embodiment.

[0057] Fig.15 This is a diagram showing an example of a frame period in the second embodiment.

[0058] Fig.16 This is a flowchart showing an example of the operation of the distance image capturing device according to the second embodiment.

[0059] Fig.17 This is a block diagram showing an example of a distance image capturing device according to a third embodiment.

[0060] Fig.18 This is a diagram showing an example of a frame period in the third embodiment.

[0061] Fig.19 This is a flowchart showing an example of the operation of the distance image capturing device according to the third embodiment.

[0062] Fig. 20 It is a diagram showing a modified example of the frame period in the third embodiment.

[0063] Fig.21 This is a flowchart showing a modified example of the operation of the distance image capturing device according to the third embodiment.

[0064] Fig. 22 It is a diagram showing an example of a distance image capturing device according to a fourth embodiment. DETAILED DESCRIPTION

[0065] (First embodiment)

[0066] Hereinafter, a distance image capturing device according to a first embodiment will be described with reference to the drawings.

[0067] Figure 1 1 is a block diagram showing a schematic configuration of a distance image capturing device according to the first embodiment. The distance image capturing device 1 includes, for example, a light source unit 2 , a light receiving unit 3 , and a distance image processing unit 4 . Figure 1 4 also shows a subject OB, which is an object whose distance is measured by the distance image capturing device 1 .

[0068] The light source unit 2 irradiates the light pulse PO to the space of the measurement object where the object OB of the object to be measured in the distance image camera 1 exists, according to the control from the distance image processing unit 4. The light source unit 2 is, for example, a surface-emitting semiconductor laser module such as a vertical cavity surface emitting laser (VCSEL). The light source unit 2 includes a light source device 21 and a diffusion plate 22.

[0069] The light source device 21 is a light source that emits laser light in a near-infrared band (e.g., a band with a wavelength of 850 nm to 940 nm) as a light pulse PO irradiated to the subject OB. The light source device 21 is, for example, a semiconductor laser light emitting element. The light source device 21 emits pulsed laser light under control from the timing control unit 41.

[0070] The diffuser plate 22 is an optical member that diffuses the near-infrared laser light emitted from the light source device 21 to an area of ​​the surface irradiated to the object OB. The pulsed laser light diffused by the diffuser plate 22 is emitted as a light pulse PO and irradiates the object OB.

[0071] The light receiving unit 3 receives reflected light RL of the light pulse PO reflected by the object OB for measuring the distance in the distance image capturing device 1 , and outputs a pixel signal corresponding to the received reflected light RL. The light receiving unit 3 includes a lens 31 and a distance image sensor 32 .

[0072] The lens 31 is an optical lens that guides the incident reflected light RL to the distance image sensor 32. The lens 31 emits the incident reflected light RL toward the distance image sensor 32, and causes the pixels provided in the light receiving area of ​​the distance image sensor 32 to receive the light (enter).

[0073] The distance image sensor 32 is an imaging element used in the distance image imaging device 1. The distance image sensor 32 has a plurality of pixels in a two-dimensional light receiving area. Each pixel of the distance image sensor 32 is provided with a photoelectric conversion element, a plurality of charge storage units corresponding to the photoelectric conversion element, and a component for distributing charges to each charge storage unit. That is, the pixel is an imaging element with a distribution structure for distributing charges to a plurality of charge storage units and accumulating them.

[0074] The distance image sensor 32 distributes the charges generated by the photoelectric conversion element to each charge storage unit according to the control from the timing control unit 41. In addition, the distance image sensor 32 outputs a pixel signal corresponding to the amount of charge distributed to the charge storage unit. In the distance image sensor 32, a plurality of pixels are arranged in a two-dimensional matrix, and the pixel signal corresponding to one frame of each pixel is output.

[0075] The distance image processing unit 4 controls the distance image capturing device 1 to calculate the distance to the object OB. The distance image processing unit 4 includes a timing control unit 41 , a distance calculation unit 42 , and a measurement control unit 43 .

[0076] The timing control unit 41 controls the timing of outputting various control signals required for measurement according to the control of the measurement control unit 43. The various control signals here include, for example, a signal for controlling the irradiation of the light pulse PO, a signal for distributing and accumulating the reflected light RL to a plurality of charge accumulation units, and a signal for controlling the number of accumulations per frame. The number of accumulations refers to the number of times the charge accumulation unit CS (see Figure 3 ) is repeated the number of times the charge distribution and accumulation process is repeated. The product of the number of accumulation times and the time period (accumulation time period) for accumulating charges in each charge accumulation section in each charge distribution and accumulation process becomes the exposure time.

[0077] The distance calculation unit 42 outputs distance information obtained by calculating the distance to the object OB based on the pixel signal output from the distance image sensor 32. The distance calculation unit 42 calculates the delay time from the irradiation light pulse PO to the reception of the reflected light RL based on the charge amount accumulated in the plurality of charge accumulation units. The distance calculation unit 42 calculates the distance to the object OB based on the calculated delay time Td.

[0078] The measurement control unit 43 controls the timing control unit 41. For example, the measurement control unit 43 sets the number of accumulation times and the accumulation time period for one frame, and controls the timing control unit 41 to perform imaging according to the set contents.

[0079] According to this configuration, in the distance image camera 1, the light pulse PO in the near-infrared band irradiated by the light source unit 2 to the subject OB is reflected by the subject OB, and the reflected light RL is received by the light receiving unit 3, and the distance image processing unit 4 outputs the distance information obtained by measuring the distance to the subject OB.

[0080] in addition, Figure 1 2 shows the distance image capturing device 1 having a configuration in which the distance image processing unit 4 is provided inside the distance image capturing device 1 , but the distance image processing unit 4 may be a component provided outside the distance image capturing device 1 .

[0081] Here, use Figure 2 The configuration of the distance image sensor 32 used as an imaging element in the distance image imaging device 1 will be described. Figure 2 This is a block diagram showing a schematic configuration of an image pickup element (range image sensor 32 ) used in the range image pickup device 1 according to the first embodiment.

[0082] like Figure 2As shown, the distance image sensor 32 includes, for example, a light receiving area 320 in which a plurality of pixels (pixel circuits) 321 are arranged, a control circuit 322 , a vertical scanning circuit 323 having a distribution operation, a horizontal scanning circuit 324 , and a pixel signal processing circuit 325 .

[0083] The light receiving area 320 is an area where a plurality of pixels 321 are arranged. Figure 2 3 shows an example of being arranged in a two-dimensional matrix of 8 rows and 8 columns. The pixel 321 accumulates a charge corresponding to the amount of light received. The control circuit 322 generally controls the distance image sensor 32. The control circuit 322 controls the operation of the components of the distance image sensor 32, for example, according to the instruction from the timing control unit 41 of the distance image processing unit 4. In addition, it is also possible to have a configuration in which the control of the components of the distance image sensor 32 is directly performed by the timing control unit 41, in which case the control circuit 322 can also be omitted.

[0084] The vertical scanning circuit 323 is a circuit that controls the pixels 321 arranged in the light receiving area 320 for each row according to the control from the control circuit 322. The vertical scanning circuit 323 outputs a voltage signal corresponding to the amount of charge accumulated in the charge accumulation section CS of the pixel 321 to the pixel signal processing circuit 325. In this case, the vertical scanning circuit 323 distributes and accumulates the charge converted by the photoelectric conversion element to the charge accumulation section of the pixel 321. That is, the vertical scanning circuit 323 is an example of a "pixel driving circuit".

[0085] The pixel signal processing circuit 325 is a circuit that performs predetermined signal processing (for example, noise suppression processing, A / D conversion processing, etc.) on the voltage signal output from the pixels 321 of each column to the corresponding vertical signal line under the control of the control circuit 322 .

[0086] The horizontal scanning circuit 324 sequentially outputs the signal output from the pixel signal processing circuit 325 to the horizontal signal line according to the control from the control circuit 322. Thus, the pixel signal corresponding to the charge amount accumulated for one frame is sequentially output to the image processing unit 4 via the horizontal signal line.

[0087] In the following description, it is assumed that the pixel signal processing circuit 325 performs A / D conversion processing and the pixel signal is a digital signal.

[0088] Here, use Figure 3 The configuration of the pixels 321 arranged in the light receiving area 320 included in the distance image sensor 32 will be described. Figure 3 1 is a circuit diagram showing an example of the configuration of pixels 321 arranged in the light receiving region 320 of the distance image sensor 32 according to the first embodiment. Figure 3 An example of the configuration of one pixel 321 among the plurality of pixels 321 arranged in the light receiving region 320 is shown. The pixel 321 is an example of a configuration including three pixel signal readout units.

[0089] The pixel 321 includes one photoelectric conversion element PD, a drain-gate transistor (charge discharge transistor) GD (charge discharge unit, charge discharge section), and three pixel signal readout sections RU that output voltage signals from corresponding output terminals O. The pixel signal readout sections RU each include a readout gate transistor G, a floating diffusion region FD, a charge storage capacity C, a reset gate transistor RT, a source follower gate transistor SF, and a select gate transistor SL. In each pixel signal readout section RU, the floating diffusion region FD and the charge storage capacity C constitute a charge storage section CS.

[0090] In addition, Figure 3 In the figure, the numbers "1", "2" or "3" are added after the reference numerals "RU" of the three pixel signal readout units RU to distinguish each pixel signal readout unit RU. Similarly, for each component of the three pixel signal readout units RU, the numbers representing each pixel signal readout unit RU are shown after the reference numerals to distinguish the pixel signal readout units RU corresponding to each component.

[0091] exist Figure 3 In the pixel 321 shown, the pixel signal readout section RU1 that outputs a voltage signal from the output terminal O1 includes a readout gate transistor G1, a floating diffusion region FD1, a charge storage capacity C1, a reset gate transistor RT1, a source follower gate transistor SF1, and a select gate transistor SL1. In the pixel signal readout section RU1, the floating diffusion region FD1 and the charge storage capacity C1 constitute a charge storage section CS1. The pixel signal readout sections RU2 and RU3 have the same configuration.

[0092] In addition, the configuration of the pixels arranged in the distance image sensor 32 is not limited to Figure 3 The structure with three pixel signal readout sections RU as shown in the figure can be any pixel as long as it has a structure with multiple pixel signal readout sections RU. That is, the number of pixel signal readout sections RU (charge storage sections CS) included in the pixels arranged in the distance image sensor 32 can be two or four or more.

[0093] In addition, Figure 3In the pixel 321 of the structure shown, an example is shown in which the charge storage unit CS is composed of the floating diffusion region FD and the charge storage capacity C. However, the charge storage unit CS only needs to be composed of the floating diffusion region FD, and the pixel 321 may not have the charge storage capacity C.

[0094] In addition, Figure 3 The pixel 321 of the structure shown in the figure shows an example of a structure including a drain-gate transistor GD. However, when it is not necessary to discard the charge accumulated (remaining) in the photoelectric conversion element PD, a structure without the drain-gate transistor GD is also possible.

[0095] The photoelectric conversion element PD is an embedded photodiode that generates electric charge by photoelectric conversion of incident light and accumulates the generated electric charge. The structure of the photoelectric conversion element PD can be arbitrary. The photoelectric conversion element PD can be, for example, a PN photodiode having a structure in which a P-type semiconductor and an N-type semiconductor are joined together, or a PIN photodiode having a structure in which an I-type semiconductor is sandwiched between a P-type semiconductor and an N-type semiconductor. In addition, the photoelectric conversion element PD is not limited to a photodiode, and can be, for example, a grating-type photoelectric conversion element.

[0096] In the pixel 321, the photoelectric conversion element PD converts the light incident at the accumulation timing synchronized with the timing of the irradiation light pulse PO into electric charge, and distributes the converted electric charge to the three charge accumulation units CS respectively and accumulates them. In addition, with respect to the light incident to the pixel 321 at a timing other than the accumulation timing, the electric charge converted by the photoelectric conversion element PD is discharged from the drain gate transistor GD, and is not accumulated in the charge accumulation unit CS. After the accumulation of electric charge at the accumulation timing and the discarding of electric charge at a timing other than the accumulation timing are repeated throughout one frame, a readout period is set. During the readout period, an electric signal corresponding to the amount of electric charge accumulated by each charge accumulation unit CS for one frame is output to the distance calculation unit 42 through the horizontal scanning circuit 324.

[0097] The distance calculation unit 42 calculates the delay time Td by the following formula (1) using the fact that the charge amount corresponding to the reflected light RL component is distributed and accumulated in the two charge accumulation units CS in a ratio corresponding to the delay time Td until the reflected light RL is incident on the distance image capture device 1. The distance calculation unit 42 multiplies the delay time Td calculated by the formula (1) by the speed of light (speed) to calculate the round-trip distance to the subject S. Then, the distance calculation unit 42 makes the round-trip distance calculated above become 1 / 2 to calculate the distance to the subject S. In addition, in the formula (1), it is assumed that the charge amount corresponding to the external light component is accumulated in the charge accumulation unit CS1, and the charge amount corresponding to the reflected light RL component is distributed and accumulated in the charge accumulation units CS2 and CS3.

[0098] Td=To×(Q3-Q1) / (Q2+Q3-2×Q1)…(1)

[0099] Here, To light is the period of the irradiation pulse PO.

[0100] Q1 is the amount of charge accumulated in the charge accumulation unit CS1.

[0101] Q2 is the amount of charge accumulated in the charge storage unit CS2.

[0102] Q3 is the amount of charge accumulated in the charge accumulation unit CS3.

[0103] In the first embodiment, point light is used as the light source emitted by the light source unit 2. The point light source is, for example, structured light composed of a plurality of point lights arranged periodically.

[0104] By using a point light source, the object OB is irradiated with uneven local light pulses PO. By using point light, the power of the irradiated light (irradiation intensity per unit area) can be increased without increasing the light source output, and the reach of the irradiated light can be increased to increase the measurable distance. On the other hand, when a point light source is used, the distance cannot be measured for the area not irradiated with the point light, and there is a problem of reduced resolution.

[0105] As a countermeasure, in the first embodiment, a point light having an elliptical shape is used as the light pulse PO. For example, the light source unit 2 includes a plurality of light source elements that can independently emit the light pulse PO (the point light having an elliptical shape).

[0106] Figure 4 FIG. 2 is a diagram showing an example of point light in the first embodiment. Figure 4 FIG. 2 schematically shows a situation in which the distance image capturing device 1 irradiates the object OB with a point light Dt as a light pulse PO. Figure 4 As shown, the point light Dt has an elliptical shape in which the ratio of the major axis length LA to the minor axis length SA is greater than a threshold value. The threshold value here can also be set arbitrarily. For example, the shape of the point light source is an elliptical shape in which the ratio of the major axis length LA to the minor axis length SA is greater than 2. In this way, in the first embodiment, by using the point light Dt having an elliptical shape, the area of ​​the object OB that is not illuminated by light can be reduced compared to the case of using the circular point light Dt. Therefore, the reduction in resolution can be suppressed.

[0107] Below, use Figures 5 to 8 The change of the spot light Dt will be described. Figures 5 to 8 It is a diagram showing another example of the spot light in the first embodiment.

[0108] Figure 5, a situation where a part of the point light Dt overlaps with a part of other point light Dt is shown. Thus, in the first embodiment, at least a part of the point light Dt1 may overlap with at least a part of other point light Dt2 adjacent to the long axis direction of the point light Dt1. Thus, the area of ​​the object OB that is not illuminated by light can be further reduced.

[0109] In addition, Figure 5 In the above description, the overlapping along the long axis is exemplified, but the overlapping along the short axis may also be performed. In addition, the overlapping along any axial direction set to be inclined relative to the long axis and the short axis may also be configured. By making at least a part of the point light Dt overlap with other point lights Dt, the area not irradiated with light can be reduced, and the same effect as the above effect can be achieved.

[0110] like Figure 5 As shown in FIG. 1 , by irradiating light by overlapping a plurality of point lights Dt in the long axis direction, the light pulse PO can be made into a linear light. In the following description, the light irradiated by overlapping a plurality of point lights Dt in the long axis direction is referred to as “linear light L”. In addition, a linear light source may be used in place of the point light source in the first embodiment or together with the point light source.

[0111] Figure 6 , a situation where a plurality of line lights L are crossed is shown. Thus, in the first embodiment, a plurality of line lights L may be configured to indicate mutually different directions. That is, the major axis directions of the elliptical point lights Dt may be configured to be mutually different. Thus, the object OB can be irradiated with light in a mesh shape, and the area in the object OB that is not irradiated with light can be further reduced.

[0112] Furthermore, when all the line lights are parallel in direction, if the lights overlap for some reason, the area irradiated with light to the object OB may be narrowed, but by making the directions of the line lights L different from each other, the light of the line lights L can be prevented from overlapping except at the intersection. Therefore, the area irradiated with light on the object OB can be prevented from being narrowed.

[0113] exist Figure 6 , the directions of the line lights in the group consisting of line lights L1 to L6 and the group consisting of line lights L7 to L12 are orthogonal to each other, in other words, 90 degrees. In this way, the major axis directions of the elliptical point lights Dt emitted from at least two of the plurality of light source elements may be orthogonal to each other.

[0114] Furthermore, the angle between the two line lights L is not limited to 90 degrees, and at least the directions of the two line lights L are different. In other words, the angle between the two line lights L is larger than 0 degrees and equal to or smaller than 90 degrees.

[0115] Figure 7 2 shows a case where the distances between the parallel line lights L are different. For example, the distance K1 between the line light L101 and the line light L102 and the distance K2 between the line light L103 and the line light L104 are different in size.

[0116] Thus, in the first embodiment, the intervals of the line lights L may be different between the longitudinal direction and the lateral direction. That is, the first interval along the short axis direction of the elliptical point lights Dt emitted from the first light source element group consisting of at least two light source elements among the plurality of light source elements and the second interval along the short axis direction of the elliptical point lights Dt emitted from the second light source element group different from the first light source element group and consisting of at least two light source elements and the long axis direction is in the same direction may be different from each other. Thus, in the subject OB, it is possible to perform measurement that distinguishes the area for fine distance measurement from the part for rough distance measurement.

[0117] For example, when the subject OB moving in the horizontal direction is taken as the measurement object, it is considered to increase the resolution of distance measurement in the horizontal direction and reduce the resolution in the vertical direction. In this case, light is irradiated in a manner that narrows the interval of the line light L in the vertical direction and widens the interval of the line light L in the horizontal direction. As a result, the area where the distance measurement is desired at a fine interval can be measured with a higher resolution.

[0118] In addition, the intervals of the line lights L groups along the longitudinal direction or the intervals of the line lights L groups along the longitudinal direction may be different intervals, in other words, unequal intervals. For example, it can be applied to a case where the resolution of the upper part is lowered and the resolution from the center to the bottom is increased for measurement when measuring the distance of a subject with a relatively low height. In this case, light is irradiated in a manner that the intervals of the line lights L along the lateral direction are widened for the upper part and narrowed for the intervals of the line lights L from the center to the bottom. Thus, the area where the subject OB exists can be measured with a higher resolution.

[0119] Figure 8 2 shows a case where the line light L is irradiated in an inclined direction different from the vertical direction (vertical direction) and the horizontal direction relative to the installation surface (ground) of the range image camera device 1. For example, the angle between the line light L201 and the line segment Ax along the x-axis direction (horizontal direction) is 135 degrees. In addition, the angle between the line light L202 and the line segment Ax is 45 degrees.

[0120] Thus, in the first embodiment, the line light L may be inclined relative to the ground. That is, the major axis direction of the elliptical point light emitted from two of the plurality of light source elements may be at an angle of 45 degrees or 135 degrees relative to the installation surface of the imaging device.

[0121] Thus, in the first embodiment, the line light L may be configured to have an inclined direction relative to the ground. That is, the major axis direction of the elliptical point light Dt emitted from two of the plurality of light source elements may be an inclined direction other than the vertical direction and the horizontal direction relative to the installation surface of the range image camera device 1.

[0122] If you want to combine a diffuser in the light source to realize a line light L perpendicular to the ground and a horizontal line light L, then when the FOI (Field of Illumination) in the horizontal direction and the vertical direction are different, it is necessary to prepare separate diffusers for vertical and horizontal use. On the other hand, in the case of making two line lights L in an inclined direction orthogonal, as long as two diffusers of the same design that irradiate the line light L in the inclined direction are prepared and installed in a manner that the two diffusers are opposite to each other up and down, the two line lights L in the inclined direction can be made orthogonal. Therefore, by making it go straight in the inclined direction, the device cost related to the light source unit 2 can be suppressed compared to going straight in the horizontal direction and the vertical direction.

[0123] In the first embodiment, a uniform diffusion light source element may be used in addition to the point light Dt. The uniform diffusion light source element is a light source element that irradiates uniform light onto the irradiation surface. By using the uniform diffusion light source element, the subject OB can be irradiated with uniform light.

[0124] Thus, in the distance image capturing device 1 of the first embodiment, it is possible to distinguish the light source to be used according to the distance to the subject OB. For example, when measuring a short distance, in other words, when measuring the distance to the subject OB existing at a relatively close position, a uniform diffuse light source is used. Thus, when the subject OB exists at a close distance and a sufficient amount of light can be irradiated to the subject OB even without using the point light Dt, the distance can be measured with a higher resolution using the uniform diffuse light source element without intentionally using the point light Dt to reduce the resolution. On the other hand, when measuring a long distance, in other words, when measuring the distance to the subject OB existing at a relatively far position, a point light source is used. Thus, even when the subject OB exists at a far distance, the light amount of the reflected light RL is increased by using the point light Dt, and the shape of the point light Dt is set to an ellipse to minimize the area in the subject OB that is not irradiated with light, thereby suppressing the reduction in resolution. Therefore, it is possible to distinguish between the use of the point light and the diffuse light source according to the measurement condition corresponding to the distance to the subject OB, and it is possible to suppress the reduction in resolution in both the short distance and the long distance.

[0125] As described above, in the distance image camera device 1 of the first embodiment, the light pulse PO is a structured light composed of a plurality of point lights Dt. At least one point light Dt1 (the first point light) among the plurality of point lights Dt has an elliptical shape in which the ratio of the major axis length to the minor axis length is greater than a threshold value. Thus, in the distance image camera device 1 of the first embodiment, the power of the light irradiated by the point light Dt can be locally increased, and the area in the object OB that is not irradiated with light can be reduced. Therefore, the resolution reduction of the distance image captured by using the point light source can be suppressed.

[0126] In addition, in the distance image camera 1 of the first embodiment, at least a portion of at least one point light Dt1 (first point light) among the plurality of point lights Dt overlaps with at least a portion of another point light Dt2 adjacent to the long axis direction of the point light Dt1. Thus, in the distance image camera 1 of the first embodiment, the area of ​​the object OB that is not illuminated by light can be further reduced.

[0127] In addition, in the distance image camera device 1 of the first embodiment, the light source unit 2 has a plurality of light source elements that can independently irradiate the light pulses PO. Therefore, in the distance image camera device 1 of the first embodiment, each of the plurality of light source elements can be used to irradiate an elliptical point light, which can increase the degree of freedom in setting the position and direction of the irradiated point light.

[0128] In addition, in the distance image capturing device 1 of the first embodiment, the major axis directions of the elliptical point lights emitted from at least two of the plurality of light source elements are different from each other. Thus, in the distance image capturing device 1 of the first embodiment, the subject OB can be illuminated in a mesh shape using the intersecting line lights L, and the area of ​​the subject OB that is not illuminated by the light can be comprehensively reduced.

[0129] In addition, in the distance image capturing device 1 of the first embodiment, the major axis directions of the elliptical point lights emitted from at least two of the plurality of light source elements are orthogonal to each other. Thus, in the distance image capturing device 1 of the first embodiment, it is possible to illuminate the subject OB in a grid pattern, achieving the same effect as described above.

[0130] In addition, in the distance image pickup device 1 of the first embodiment, the distances K1 and K2 are different. The distance K1 is the distance from the line light L101 to the line light L102, and is an example of the "first interval". The distance K2 is the distance from the line light L103 to the line light L104, and is an example of the "second interval". Thus, in the distance image pickup device 1 of the first embodiment, it is possible to perform measurement with different resolutions in the horizontal or vertical directions. For example, when the subject OB moving horizontally is used as the measurement object, the distance that changes with the horizontal movement can be measured with a higher resolution.

[0131] In addition, in the distance image pickup device 1 of the first embodiment, the intervals along the short axis direction of the elliptical point lights radiated from at least three of the plurality of light source elements, whose long axis directions are in the same direction, are different from each other. Thus, in the distance image pickup device 1 of the first embodiment, it is possible to perform measurement with a resolution corresponding to the area where the subject exists. For example, when a low-height subject OB is used as the measurement object, the upper area where the subject OB does not exist can be measured with a lower resolution, and the area from the center to the bottom where the subject OB exists can be measured with a higher resolution.

[0132] In addition, in the distance image camera device 1 of the first embodiment, the major axis direction of the elliptical point light emitted from at least two of the plurality of light source elements is an inclined direction other than the vertical direction and the horizontal direction with respect to the installation surface of the camera device. Therefore, in the distance image camera device 1 of the first embodiment, by installing two diffusers for irradiating the line light L in the inclined direction in a manner opposite to each other up and down, such line light can be realized, and the device cost can be suppressed compared to the case of using separate diffusers for vertical and horizontal use.

[0133] In addition, in the distance image camera device 1 of the first embodiment, the major axis directions of the elliptical point lights emitted from at least two of the plurality of light source elements respectively form an angle of 45 degrees or 135 degrees with respect to the installation surface of the camera device. Thus, in the distance image camera device 1 of the first embodiment, the line light can be irradiated to the object in an inclined direction and leftward, rightward, and upward with respect to the ground, achieving the same effect as the above effect.

[0134] In addition, in the distance image capturing device 1 of the first embodiment, at least one of the plurality of light source elements is a diffuse light source. Thus, in the distance image capturing device 1 of the first embodiment, it is possible to distinguish between the use of point light and diffuse light sources according to the measurement conditions corresponding to the distance to the object OB, and it is possible to suppress the reduction in resolution in both the short distance and the long distance.

[0135] All or part of the distance image camera device 1 and the distance image processing unit 4 in the first embodiment can also be implemented by a computer. In this case, the program for implementing the function can be recorded in a computer-readable recording medium, and the program recorded in the recording medium can be read into the computer system and executed. In addition, the "computer system" mentioned here is set to include hardware such as OS and peripheral devices. In addition, "computer-readable recording medium" refers to removable media such as floppy disks, optical magnetic disks, ROMs, CD-ROMs, and storage devices such as hard disks built into the computer system. In addition, "computer-readable recording medium" also includes a medium that dynamically maintains the program for a short period of time, such as a communication line when the program is sent via a network such as the Internet or a communication line such as a telephone line, and a medium that maintains the program for a certain period of time, such as a volatile memory inside a computer system that becomes a server or client in this case. In addition, the above program can be a program for implementing a part of the above function, a program that realizes the above function by combining with a program already recorded in the computer system, or a program implemented using a programmable logic device such as an FPGA.

[0136] As mentioned above, although the embodiment of this invention is described in detail with reference to drawings, a specific structure is not limited to this embodiment, and also includes the design etc. which do not deviate from the scope of the meaning of this invention.

[0137] (Second embodiment)

[0138] Hereinafter, a range image capturing device and a range image capturing method according to a second embodiment of the present invention will be described with reference to the drawings.

[0139] Fig. 9 1 is a block diagram showing an example of the distance image capturing device 101 according to the second embodiment. Fig. 9As shown in FIG. 1 , the distance image capturing device 101 includes a light source unit 102, a light receiving unit 103, and a distance image processing unit 104. Fig. 9 , the subject OB is also shown as an object whose distance is measured by the distance image capturing device 101. The distance image capturing element is, for example, a distance image sensor 132 (described later) in the light receiving unit 103.

[0140] The light source unit 102 emits a light pulse PO toward the object OB. The light source unit 102 emits a light pulse PO toward the space of the object to be imaged where the object OB, the object of which the distance is measured in the distance image capturing device 101, exists, according to the control from the distance image processing unit 104. The light source unit 102 is, for example, a surface-emitting semiconductor laser module such as a vertical cavity surface emitting laser (VCSEL). The light source unit 102 emits, for example, line light L (see Figure 12 to Figure 14 ) is a structured light, i.e., a light pulse PO, and two lines of light L in two directions are orthogonal to each other. In addition, for details of the line light L, refer to Figure 12 to Figure 14 In addition, the light source unit 102 includes a light source device 121 and a diffusion plate 122 .

[0141] The light source device 121 is a light source that emits laser light in a near-infrared band (e.g., a band with a wavelength of 850 nm to 940 nm) as a light pulse PO irradiated to the subject OB. The light source device 121 is, for example, a semiconductor laser light emitting element. The light source device 121 emits pulsed laser light under the control of the measurement control unit 143.

[0142] The diffuser plate 122 is an optical component that diffuses the near-infrared laser light emitted from the light source device 121 to an area of ​​the surface that is irradiated to the object OB. The pulsed laser light diffused by the diffuser plate 122 is emitted as a light pulse PO and irradiates the object OB.

[0143] The light receiving unit 103 receives reflected light RL of the light pulse PO reflected by the object OB whose distance is to be measured in the distance image capturing device 101 , and outputs a pixel signal corresponding to the received reflected light RL. The light receiving unit 103 includes a lens 131 and a distance image sensor 132 .

[0144] The lens 131 is an optical lens that guides the incident reflected light RL to the distance image sensor 132. The lens 131 emits the incident reflected light RL toward the distance image sensor 132 so that the pixel circuit 421 provided in the light receiving area of ​​the distance image sensor 132 receives the light (is incident thereon).

[0145] The distance image sensor 132 is an image pickup element used in the distance image pickup device 101. The distance image sensor 132 includes a plurality of pixel circuits 421 in a two-dimensional light receiving area and a pixel driving circuit 422 for controlling each of the pixel circuits 421.

[0146] The pixel circuit 421 includes one photoelectric conversion element (for example, a photoelectric conversion element PD described later), a plurality of charge storage units corresponding to the one photoelectric conversion element (for example, charge storage units CS (CS1 to CS4) described later), and components for distributing charges to the respective charge storage units.

[0147] The pixel driving circuit 422 turns on the transfer transistors G (described later) and the charge storage units CS (CS1 to CS4) at a predetermined storage timing synchronized with the irradiation of the light pulse PO, and distributes and stores the charge to the charge storage unit CS. Fig.10 To be described later.

[0148] The distance image sensor 132 distributes the charge generated by the photoelectric conversion element to each charge storage unit according to the control from the measurement control unit 143. In addition, the distance image sensor 132 outputs a pixel signal corresponding to the amount of charge distributed to the charge storage unit. In the distance image sensor 132, a plurality of pixel circuits are arranged in a two-dimensional matrix, and the pixel signal corresponding to one frame of each pixel circuit is output.

[0149] Here, refer to Fig.10 , the detailed structure of the distance image sensor 132 is described. Fig.10 : is a block diagram showing an example of the distance image sensor in this embodiment.

[0150] like Fig.10 As shown, the distance image sensor 132 includes, for example, a light receiving area 420 provided with a plurality of pixel circuits 421, and a pixel driving circuit 422. The pixel driving circuit 422 includes a vertical scanning circuit 423 having a distribution operation, a horizontal scanning circuit 424, a pixel signal processing circuit 425, and a control circuit 426.

[0151] The light receiving area 420 is an area where a plurality of pixel circuits 421 are arranged. Fig.10 4 shows an example of a two-dimensional matrix configuration of 8 rows and 8 columns. A plurality of pixel circuits 421 are configured in a two-dimensional matrix configuration, and charge corresponding to the amount of light received is accumulated. In addition, the detailed configuration of the pixel circuit 421 will be referred to in Fig.11 To be described later.

[0152] The control circuit 426 generally controls the distance image sensor 132. For example, the control circuit 426 controls the operation of the components of the distance image sensor 132 according to the instruction of the measurement control unit 143 from the distance image processing unit 104. In addition, the control of the components of the distance image sensor 132 may be directly performed by the measurement control unit 143, and in this case, the control circuit 426 can also be omitted.

[0153] The vertical scanning circuit 423 is a circuit that controls the pixel circuits 421 arranged in the light receiving area 420 for each row according to the control from the control circuit 426. The vertical scanning circuit 423 outputs a voltage signal corresponding to the amount of charge accumulated in the charge accumulation section CS of the pixel circuit 421 to the pixel signal processing circuit 425. In this case, the vertical scanning circuit 423 distributes the charge converted by the photoelectric conversion element to the charge accumulation section of the pixel circuit 421 and accumulates it.

[0154] The pixel signal processing circuit 425 performs predetermined signal processing (for example, noise suppression processing, A / D conversion processing, etc.) on the voltage signal output from the pixel circuit 421 of each column according to the control from the control circuit 426 .

[0155] The horizontal scanning circuit 424 is a circuit that sequentially outputs the signal output from the pixel signal processing circuit 425 in a time series according to the control from the control circuit 426. Thus, pixel signals corresponding to the amount of charge accumulated for one frame are sequentially output to the range image processing unit 104. In the following description, it is assumed that the pixel signal processing circuit 425 performs A / D conversion processing and the pixel signal is a digital signal.

[0156] Next, refer to Fig.11 , the structure of the pixel circuit 421 arranged in the light receiving area 420 included in the distance image sensor 132 is described. Fig.11 4 is a block diagram showing an example of the pixel circuit 421 in this embodiment. Fig.11 The pixel circuit 421 shown is a configuration example including four pixel signal readout sections RU (RU1 to RU4).

[0157] like Fig.11As shown, the pixel circuit 421 includes a photoelectric conversion element PD, a charge discharge transistor (drain gate transistor) GD (charge discharge unit, charge discharge section), and four pixel signal readout sections RU (RU1 to RU4) that output voltage signals from corresponding output terminals O (O1 to O4). The pixel signal readout sections RU each include a transfer transistor G, a floating diffusion region FD, a charge storage capacity C, a reset transistor RT, a source follower transistor SF, and a selection transistor SL. The floating diffusion region FD and the charge storage capacity C constitute a charge storage section CS.

[0158] exist Fig.11 In the pixel circuit 421 shown, a pixel signal readout section RU1 that outputs a voltage signal from an output terminal O1 includes a transfer transistor G1, a floating diffusion region FD1, a charge storage capacity C1, a reset transistor RT1, a source follower transistor SF1, and a selection transistor SL1. In the pixel signal readout section RU1, a charge storage section CS1 is formed by the floating diffusion region FD1 and the charge storage capacity C1. The pixel signal readout sections RU2 to RU4 are also formed in the same manner.

[0159] The photoelectric conversion element PD is an embedded photodiode that performs photoelectric conversion on incident light to generate charges corresponding to the incident light (incident light) and accumulates the generated charges. In the present embodiment, the incident light is incident from a space of a measurement target.

[0160] In the pixel circuit 421 , charges generated by the photoelectric conversion element PD converting incident light are distributed to four charge storage units CS ( CS1 to CS4 ), and voltage signals corresponding to the amount of the distributed charges are output to the pixel signal processing circuit 425 .

[0161] In addition, the configuration of the pixel circuit disposed in the distance image sensor 132 is not limited to Fig.11 The configuration including four pixel signal readout units RU (RU1 to RU4) as shown may be a pixel circuit including a plurality of pixel signal readout units RU including one or more pixel signal readout units RU.

[0162] In addition, in the driving of the pixel circuit 421, the light pulse PO is irradiated at the irradiation time To, and the reflected light RL is received by the distance image sensor 132 after being delayed by the delay time Td. The pixel driving circuit 422 is controlled by the measurement control unit 143, and according to the frame cycle, the accumulation driving signals TX1 to TX4 are supplied and distributed to the transfer transistors G (G1, G2, G3, G4) according to their respective timings in synchronization with the irradiation of the light pulse PO, and the charges generated by the photoelectric conversion element PD are sequentially accumulated in the charge accumulation units CS1, CS2, CS3, CS4.

[0163] Then, the pixel driving circuit 422 controls the reset transistor RT and the selection transistor SL respectively according to the driving signals RST and SEL, converts the charge accumulated in the charge storage unit CS into an electrical signal through the source follower transistor SF, and outputs the generated electrical signal to the distance calculation unit 142 via the output terminal O.

[0164] Furthermore, the pixel driving circuit 422 discharges (eliminates) the charges generated in the photoelectric conversion element PD by flowing them to the power supply VDD according to the driving signal RSTD under the control of the measurement control unit 143 .

[0165] return Fig. 9 As described above, the distance image processing unit 104 controls the distance image camera 101 to calculate the distance to the object OB. The distance image processing unit 104 measures the distance to the object OB existing in the measurement space as the measurement distance based on the amount of charge accumulated by the charge accumulation unit CS. In addition, the distance image processing unit 104 includes a timing control unit 141, a distance calculation unit 142, and a measurement control unit 143.

[0166] The timing control unit 141 controls the timing of outputting various control signals required for measurement according to the control of the measurement control unit 143. The various control signals here are, for example, a signal for controlling the irradiation of the light pulse PO, a signal for distributing and accumulating the reflected light RL to a plurality of charge accumulation units, a signal for controlling the number of accumulations per frame, etc. The number of accumulations refers to the number of times the process of distributing and accumulating charges to the charge accumulation unit CS is repeated, and is the number of distributions preset in the frame cycle. The product of the number of accumulations and the time period (accumulation time period) for accumulating charges in each charge accumulation unit in each process of distributing and accumulating charges becomes the exposure time.

[0167] The distance calculation unit 142 outputs distance information obtained by calculating the distance to the object OB based on the pixel signal output from the distance image sensor 132. The distance calculation unit 142 calculates the delay time from the irradiation light pulse PO to the reception of the reflected light RL based on the amount of charge accumulated in the plurality of charge accumulation units CS. The distance calculation unit 142 calculates the distance to the object OB based on the calculated delay time.

[0168] In addition, in the present embodiment, the frame period has a plurality of sub-frame periods in which the charge is accumulated in the charge accumulation unit CS for different periods of time. The distance calculation unit 142 calculates the distance to the object OB using a sub-frame period in which the charge amount does not exceed a preset threshold value (the charge amount is not saturated) among the plurality of sub-frame periods. In addition, the details of the frame period and the sub-frame period will be described later.

[0169] The distance calculation unit 142 uses the fact that the amount of charge corresponding to the reflected light RL component is distributed and accumulated to the two charge accumulation units CS in a ratio corresponding to the delay time Td until the reflected light RL is incident on the distance image camera device 101, and calculates the delay time Td by the following formula (2). The distance calculation unit 142 multiplies the delay time Td calculated by the formula (2) by the speed of light (speed), thereby calculating the round-trip distance to the object OB. Then, the distance calculation unit 142 makes the round-trip distance calculated above become 1 / 2, thereby calculating the distance to the object OB. In addition, in formula (2), it is assumed that the amount of charge corresponding to the external light component is accumulated in the charge accumulation unit CS1, and the amount of charge corresponding to the reflected light RL component is distributed and accumulated to the charge accumulation units CS2 and CS3.

[0170] Td=To×(Q3-Q1) / (Q2+Q3-2×Q1)…(2)

[0171] Here, To is the period of irradiation of the light pulse PO.

[0172] Q1 is the amount of charge accumulated in the charge accumulation unit CS1.

[0173] Q2 is the amount of charge accumulated in the charge storage unit CS2.

[0174] Q3 is the amount of charge accumulated in the charge accumulation unit CS3.

[0175] The measurement control unit 143 controls the timing control unit 141. For example, the measurement control unit 143 sets the number of accumulation times and the accumulation time period for one frame, and controls the timing control unit 141 so as to perform imaging with the set contents. That is, the measurement control unit 143 sets a frame period including a plurality of subframe periods, and controls the timing control unit 141 so as to perform imaging with the set contents.

[0176] With this configuration, in the distance image camera device 101, the light receiving unit 103 receives the reflected light RL that is reflected by the subject OB from the light pulse PO in the near-infrared band irradiated by the light source unit 102 onto the subject OB, and the distance image processing unit 104 outputs the distance information (distance image) that measures the distance to the subject OB.

[0177] Next, the operation of the distance image capturing device 101 according to the present embodiment will be described with reference to the accompanying drawings. Figure 12 to Figure 14 , the linear light L outputted by the light source unit 102 in this embodiment is described. Figure 12 to Figure 14 In the description, the X-axis direction is set as the horizontal axis direction (lateral direction) and the Y-axis direction is set as the vertical axis direction (vertical direction).

[0178] In the present embodiment, point light is used as the light source emitted by the light source unit 102. The point light is, for example, structured light composed of a plurality of point lights arranged periodically.

[0179] By using point light, the object OB is irradiated with uneven local light pulses PO. By using point light, the power of the irradiated light (irradiation intensity per unit area) can be increased without increasing the light source output, and the reach of the irradiated light can be increased to increase the measurable distance. On the other hand, when point light is used, the distance cannot be measured for the area not irradiated with the point light, and there is a problem of reduced resolution.

[0180] As a countermeasure, in the present embodiment, an elliptical point light is used as the light pulse PO. For example, the light source unit 102 includes a plurality of light source elements that can independently emit the light pulse PO (using line light of the elliptical point light).

[0181] Figure 12 to Figure 14 FIG. 1 is a diagram showing an example of line light L in this embodiment. Figure 12 to Figure 14 , a state where the range image capturing device 101 irradiates the object OB with line light L as a light pulse PO is schematically shown.

[0182] exist Fig.12 In FIG. 1 , an example of discretely radiating a plurality of point lights Dt to form a line light L is shown. Here, an example of irradiating a horizontal line light L is shown. Fig.12 As shown, the line light L is formed by discretely radiating a plurality of point lights Dt into a line shape. Here, the point light Dt has an elliptical shape in which the ratio of the major axis length LA to the minor axis length SA is greater than a threshold value. For example, the shape of the point light source is an elliptical shape in which the ratio of the major axis length LA to the minor axis length SA is greater than 2.

[0183] As described above, in the present embodiment, by using the elliptical spot light Dt, the area of ​​the object OB that is not irradiated with light can be reduced compared to the case of using the circular spot light Dt. Therefore, a decrease in resolution can be suppressed.

[0184] In addition, Fig.13 In FIG. 1 , an example of radiating a plurality of point lights Dt in an overlapping manner to form a line light L is shown. Here, an example of irradiating a horizontal line light L is shown. Fig.13 As shown, the line light L is formed by overlapping a part of the point light Dt with a part of other point light Dt. In this way, in the present embodiment, the point light Dt1 is irradiated in a manner that at least a part of the point light Dt1 overlaps at least a part of other point light Dt2 adjacent to the long axis direction of the point light Dt1, thereby forming a continuous line light L. In addition, a line light source may be used instead of or together with the point light source in the present embodiment.

[0185] In addition, Fig.14 , a case where a plurality of line lights L are crossed is shown in FIG. 1 . Thus, in the present embodiment, the light source unit 102 irradiates the object OB with light pulses in such a manner that the plurality of line lights L show mutually different directions. In this case, the major axis directions of the point lights Dt in the elliptical shape may be mutually different. Thus, the object OB can be irradiated with light in a mesh shape, and the area in the object OB that is not irradiated with light can be further reduced.

[0186] In addition, Fig.14 , the direction of the line light in the group consisting of line light L1 to L6 (longitudinal line light) and the direction of the line light in the group consisting of line light L7 to L12 (horizontal line light) are shown to be orthogonal to each other, in other words, at an angle of 90 degrees. In this way, the major axis directions of the elliptical point light Dt emitted from at least two of the plurality of light source elements are orthogonal to each other.

[0187] Furthermore, the angle between the two line lights L is not limited to 90 degrees, and at least the directions of the two line lights L are different. In other words, the angle between the two line lights L is larger than 0 degrees and equal to or smaller than 90 degrees.

[0188] As described above, in the present embodiment, the light source unit 102 radiates the light pulses PO which are structured light composed of periodically arranged line lights L, and the line lights L in two directions are orthogonal to each other.

[0189] Next, refer to Figures 9 to 11 , the basic operation of distance measurement by the distance image capturing device 101 is described.

[0190] In the driving of the pixel circuit 421 of the distance image pickup device 101, the light pulse PO is irradiated at the irradiation time To, and the reflected light RL is received by the distance image sensor 132 after being delayed by the delay time Td. The vertical scanning circuit 423 distributes and accumulates the charges generated by the photoelectric conversion element PD in the order of the charge storage units CS1, CS2, CS3, and CS4 in synchronization with the irradiation of the light pulse PO.

[0191] At this time, the vertical scanning circuit 423 turns on the transfer transistor G1 provided on the transfer path for transferring charges from the photoelectric conversion element PD to the charge accumulation section CS1. As a result, the charges obtained by the photoelectric conversion element PD are stored in the charge accumulation section CS1 via the transfer transistor G1. After that, the vertical scanning circuit 423 turns off the transfer transistor G1 (non-conducting state). As a result, the transfer of charges to the charge accumulation section CS1 stops. In this way, the vertical scanning circuit 423 causes the charge accumulation section CS1 to accumulate charges. The same is true for the charge accumulation sections CS2 to CS4.

[0192] At this time, during the charge accumulation period in which the charges are distributed to the charge accumulation section CS, an accumulation cycle in which the accumulation drive signals TX1 to TX4 are supplied to the transfer transistors G1 to G4 , respectively, is repeated.

[0193] Then, the charge corresponding to the incident light is transferred from the photoelectric conversion element PD to the charge storage units CS1 to CS4 via the transfer transistors G1 to G4. A plurality of accumulation cycles are repeated during the charge accumulation period. Thus, in each accumulation cycle of the charge storage units CS1 to CS4 during the charge accumulation period, charge is accumulated in each of the charge storage units CS1 to CS4.

[0194] Furthermore, when the vertical scanning circuit 423 repeats the accumulation cycles of the charge accumulation units CS1 to CS4, after the transfer (distribution) of the charge to the charge accumulation unit CS4 is completed, the charge discharge transistor GD provided on the discharge path for discharging the charge from the photoelectric conversion element PD is turned on.

[0195] Thus, the charge discharge transistor GD discards the charge generated in the photoelectric conversion element PD after the accumulation cycle of the charge accumulation unit CS4 immediately before the accumulation cycle for the charge accumulation unit CS1 (ie, resets the photoelectric conversion element PD).

[0196] Then, the vertical scanning circuit 423 sequentially outputs voltage signals from all the pixel circuits 421 arranged in the light receiving area 420 to the pixel signal processing circuit 425 in units of rows (horizontal arrangement) of the pixel circuits 421 .

[0197] Then, the pixel signal processing circuit 425 performs signal processing such as A / D conversion on the input voltage signals and outputs the signals to the horizontal scanning circuit 424. The horizontal scanning circuit 424 outputs the processed voltage signals to the distance calculation unit 142 in the order of the columns of the light receiving regions 420.

[0198] The above-mentioned accumulation of charges in the charge storage unit CS and the discarding of charges obtained by photoelectric conversion by the photoelectric conversion element PD by the vertical scanning circuit 423 are repeated throughout one frame. Thus, charges corresponding to the amount of light received by the distance image pickup device 101 in a predetermined time interval are accumulated in each of the charge storage units CS. The horizontal scanning circuit 424 outputs an electrical signal corresponding to the amount of charge accumulated in each charge storage unit CS for one frame to the distance calculation unit 142.

[0199] In addition, in this embodiment, the frame period FT has a plurality of sub-frame periods SFT. Fig.15, the frame period FT and sub-frame period SFT in this embodiment are explained.

[0200] Fig.15 It is a diagram showing an example of the frame period FT in this embodiment.

[0201] like Fig.15 As shown, the frame period FT in this embodiment includes two sub-frame periods SFT1 and SFT2. In both the sub-frame period SFT1 and SFT2, the object OB is illuminated simultaneously. Fig.14 Orthogonal vertical and horizontal light lines as shown.

[0202] The subframe period SFT1 (first subframe period) includes an accumulation period Tc1 (charge accumulation period) for accumulating charges in the charge accumulation section CS and a readout period Trd for reading out the charges accumulated in the charge accumulation section CS. The accumulation period Tc1 is, for example, a predetermined period in which the charges of the charge accumulation section CS are not saturated (do not exceed a predetermined threshold) at the intersection of the above-mentioned line light L. The accumulation period Tc1 is a period that is less than 1 / 2 of the accumulation period Tc2 of the subframe period SFT2. In addition, the accumulation period Tc1 (charge accumulation period) is determined by the number of accumulations.

[0203] In addition, the subframe period SFT2 (second subframe period) has an accumulation period Tc2 (charge accumulation period) for accumulating charges in the charge accumulation section CS and a readout period Trd for reading out the charges accumulated in the charge accumulation section CS. The subframe period SFT2 is a subframe period SFT having an accumulation period different from that of the subframe period SFT1. The accumulation period Tc2 is, for example, a period pre-set to enable appropriate measurement of the distance image at a portion other than the intersection of the above-mentioned line light L. The accumulation period Tc2 is a period that is more than twice the accumulation period Tc1 of the subframe period SFT1. In addition, the accumulation period Tc2 is the same as the accumulation period Tc1 and is determined by the number of accumulations.

[0204] The distance calculation unit 142 calculates the distance to the object OB using the sub-frame period SFT in which the charge amount does not exceed a preset threshold value, out of the sub-frame period SFT1 and the sub-frame period SFT2 described above.

[0205] For example, when the amount of charge in the subframe period SFT2 does not exceed a preset threshold, the distance calculation unit 142 calculates the distance to the object OB using the subframe period SFT2. Alternatively, when the amount of charge exceeds a preset threshold, the distance calculation unit 142 calculates the distance to the object OB using the subframe period SFT1 in which the accumulation period Tc is shorter than the subframe period SFT2.

[0206] That is, the distance calculation unit 142 calculates the distance to the object OB using, for example, the subframe period SFT1 and the subframe period SFT2 in which the charge amount does not exceed the threshold and the accumulation period is longer. In addition, when calculating the distance to the object OB, the distance calculation unit 142 uses the above-mentioned formula (2) to calculate the distance to the object OB.

[0207] Next, refer to Fig.16 Calculation processing of the distance to the object OB by the distance image capturing device 101 according to the present embodiment will be described in detail. Fig.16 1 is a flowchart showing an example of the operation of the distance image capturing device 101 according to the present embodiment. Here, an example of the calculation process of the distance to the object OB will be described.

[0208] like Fig.16 As shown, the distance image camera device 101 first performs charge accumulation and acquisition of charge amount according to the frame period FT including the first sub-frame period (sub-frame period SFT1) and the second sub-frame period (sub-frame period SFT2) having different accumulation periods Tc (step S101). The measurement control unit 143 of the distance image camera device 101 instructs the timing control unit 141 to accumulate charge and acquire charge amount according to the frame period FT, and the timing control unit 141 performs charge accumulation and acquisition of charge amount according to the frame period FT including the sub-frame period SFT1 and the sub-frame period SFT2 in response to the control of the measurement control unit 143.

[0209] Next, the measurement control unit 143 of the distance image pickup device 101 determines whether the charge amount obtained in the second sub-frame period (sub-frame period SFT2) exceeds the threshold value (step S102). When the charge amount obtained in the sub-frame period SFT2 exceeds the threshold value (step S102: Yes), the measurement control unit 143 causes the process to proceed to step S104. On the other hand, when the charge amount obtained in the sub-frame period SFT2 does not exceed the threshold value (step S102: No), the measurement control unit 143 causes the process to proceed to step S103.

[0210] In step S103, the measurement control unit 143 instructs the distance calculation unit 142 to perform distance calculation processing, and the distance calculation unit 142 calculates the distance to the object OB based on the charge amount obtained in the second subframe period (subframe period SFT2). The distance calculation unit 142 calculates the distance to the object OB using, for example, equation (2) and the charge amount obtained in the subframe period SFT2. Here, the distance calculation unit 142 calculates the distance of the portion other than the intersection of the line light L using the charge amount obtained in the subframe period SFT2. After the processing of step S103, the distance calculation unit 142 advances the processing to step S107.

[0211] In step S104, the measurement control unit 143 determines whether the charge amount obtained in the first sub-frame period (sub-frame period SFT1) exceeds the threshold value. If the charge amount obtained in the sub-frame period SFT1 exceeds the threshold value (step S104: Yes), the measurement control unit 143 advances the process to step S106. On the other hand, if the charge amount obtained in the sub-frame period SFT1 does not exceed the threshold value (step S104: No), the measurement control unit 143 advances the process to step S105.

[0212] In step S105, the measurement control unit 143 instructs the distance calculation unit 142 to perform distance calculation processing, and the distance calculation unit 142 calculates the distance to the object OB based on the charge amount obtained in the first subframe period (subframe period SFT1). The distance calculation unit 142 calculates the distance to the object OB using, for example, equation (2) and the charge amount obtained in the subframe period SFT1. Here, the distance calculation unit 142 calculates the distance of the intersection of the line light L using the charge amount obtained in the subframe period SFT1. After the processing of step S105, the distance calculation unit 142 advances the processing to step S107.

[0213] In step S106, the measurement control unit 143 executes error processing. This corresponds to a case where the charge amount acquired in the first subframe period exceeds the threshold value (saturation), so the measurement control unit 143 executes error processing such as re-measurement processing and warning output.

[0214] Next, in step S107, the measurement control unit 143 determines whether the execution of all pixel circuits 421 has been completed. That is, the measurement control unit 143 determines whether the calculation process of the distance of one frame period FT has been completed. When the execution of all pixel circuits 421 has been completed (step S107: Yes), the measurement control unit 143 ends the process. On the other hand, when the execution of all pixel circuits 421 has not been completed (step S107: No), the measurement control unit 143 returns the process to step S102 and executes the calculation process of the distance of the next pixel circuit 421.

[0215] Thus, in the distance image capturing device 101 of the present embodiment, the distance calculation unit 142 calculates the distance to the object OB using the subframe period SFT1 and the subframe period SFT2 in which the charge amount does not exceed the threshold and the accumulation period is longer.

[0216] As described above, the distance image capturing device 101 of the present embodiment includes a light source unit 102, a light receiving unit 103, and a distance calculation unit 142. The light source unit 102 irradiates a light pulse PO to the object OB. The light receiving unit 103 includes a pixel circuit 421, a pixel driving circuit 422, and a charge discharge transistor GD (charge discharge unit). The pixel circuit 421 is arranged in a two-dimensional matrix, and includes a plurality of photoelectric conversion elements PD that generate charges corresponding to incident light and a plurality of charge storage units CS that store charges. The pixel driving circuit 422 distributes charges to the charge storage units CS at the storage timing synchronized with the irradiation of the light pulse PO according to the frame period FT and stores them. The charge discharge transistor GD (charge discharge unit) discharges charges during a period other than the storage timing. The distance calculation unit 142 calculates the distance to the object OB based on the amount of charge stored in each charge storage unit CS. The light source unit 102 emits structured light composed of periodically arranged line lights L, i.e., light pulses PO, and the line lights L in two directions are orthogonal to each other. The frame period FT has a plurality of sub-frame periods SFT with different accumulation periods for accumulating charges in the charge accumulation unit CS. The distance calculation unit 142 calculates the distance to the object OB using a sub-frame period SFT in which the charge amount does not exceed a preset threshold value among the plurality of sub-frame periods SFT.

[0217] Thus, the distance image camera device 101 of the present embodiment uses a sub-frame period SFT in which the charge amount does not exceed a preset threshold value among a plurality of sub-frame periods SFT with different accumulation periods to calculate the distance to the object OB. Therefore, the distance image camera device 101 of the present embodiment can calculate the distance to the object OB using, for example, a sub-frame period SFT in which the charge accumulation portion CS is not saturated at the intersection of the line light L. Thus, the distance image camera device 101 of the present embodiment can mitigate the signal output difference between the intersection of the line light L and the portion other than the intersection, and suppress the reduction in the overall resolution of the distance image and the measurable distance. In addition, by using the line light L, the distance image camera device 101 of the present embodiment can suppress the reduction in resolution while suppressing the necessary light source output.

[0218] In addition, in the present embodiment, the plurality of subframe periods SFT are a subframe period SFT1 (first subframe period) in which the accumulation period is a predetermined period (e.g., accumulation period Tc1) set in advance, and a subframe period SFT2 (second subframe period) in which the accumulation period is a period that is twice or more of the predetermined period (e.g., accumulation period Tc2). The distance calculation unit 142 calculates the distance to the object OB using the subframe period SFT1 and the subframe period SFT2 in which the charge amount does not exceed the threshold and the accumulation period is longer.

[0219] Thus, in the distance image capturing device 101 of the present embodiment, the distance to the object OB is calculated using the subframe period SFT1 (first subframe period) at the intersection of the line light L, and the distance to the object OB is calculated using the subframe period SFT2 (second subframe period) at the portion other than the intersection. Thus, in the distance image capturing device 101 of the present embodiment, the saturation of the charge at the intersection of the line light L can be suppressed while maintaining the resolution of the portion other than the intersection of the line light L, and the distance to the object OB can be measured at the intersection of the line light L with high accuracy.

[0220] In addition, in this embodiment, the light source unit 102 has a plurality of light source elements that can independently emit light pulses PO. The line light L is formed by radiating a plurality of point lights Dt in a discrete linear form from a plurality of light source elements, or by radiating a plurality of point lights Dt in a linear form in an overlapping manner (for example, see Fig.12 ) Thus, the distance image capturing device 101 of the present embodiment can suppress a reduction in resolution by using the line light L.

[0221] In addition, in the present embodiment, the shape of the spot light Dt is an ellipse whose ratio of the major axis length to the minor axis length is greater than 2, and the line light L is formed in a linear shape along the major axis. Thus, the range image capturing device 101 of the present embodiment can reduce the area of ​​the object OB that is not illuminated by light by using the spot light Dt having an ellipse shape, compared with the case of using the spot light Dt having a circular shape, and thus can further suppress the reduction in resolution.

[0222] In addition, the distance image capturing method of the present embodiment is a distance image capturing method of the distance image capturing device 101 having the above-mentioned light source unit 102, light receiving unit 103, and distance calculation unit 142. In the distance image capturing method, the light source unit 102 emits structured light composed of periodically arranged line lights L, that is, light pulses PO, and the line lights L in two directions are mutually orthogonal. The frame period FT has a plurality of sub-frame periods SFT with different accumulation periods for accumulating charges in the charge accumulation unit CS. Moreover, the distance calculation unit 142 calculates the distance to the object OB using the sub-frame period SFT in which the charge amount does not exceed a preset threshold value among the plurality of sub-frame periods SFT.

[0223] Therefore, the distance image capturing method of this embodiment has the same effect as the above-mentioned distance image capturing device 101, which can alleviate the signal output difference between the intersection of the line light L and the part other than the intersection, and suppress the reduction of the overall resolution of the distance image and the measurable distance.

[0224] (Third embodiment)

[0225] Next, a distance image capturing device 101a according to a third embodiment will be described with reference to the drawings. In the third embodiment, a modified example is described in which the distance to the object OB is measured and calculated alternately using the light pulses PO of the longitudinal line light L and the light pulses PO of the lateral line light L in a plurality of subframe periods.

[0226] Fig.17 1 is a block diagram showing an example of a distance image capturing device 101a according to the third embodiment. Fig.17 As shown in FIG. 1 , the distance image capturing device 101a includes a light source unit 102, a light receiving unit 103, and a distance image processing unit 104a. Fig.17 101a also shows the object OB, which is the object whose distance is measured by the distance image capturing device 101a. The distance image capturing element is, for example, the distance image sensor 132 in the light receiving unit 103.

[0227] In addition, Fig.17 In Fig. 9 The same components are denoted by the same reference numerals, and their description is omitted. Fig.10 as well as Fig.11 The second embodiment shown is the same, so its description is omitted here.

[0228] The distance image processing unit 104a controls the distance image camera 101a to calculate the distance to the object OB. The distance image processing unit 104a measures the distance to the object OB existing in the measurement space as the measurement distance based on the amount of charge accumulated in each charge accumulation unit CS. In addition, the distance image processing unit 104 includes a timing control unit 141, a distance calculation unit 142a, and a measurement control unit 143a.

[0229] In this embodiment, the process of the distance calculation unit 142a and the measurement control unit 143a is the same as the second embodiment except that the process is different. The measurement control unit 143a uses the timing control unit 141 to control the distance measurement to the object OB according to the frame period FT having a plurality of subframe periods SFT. Fig.18 , the frame period FT in this embodiment is explained.

[0230] Fig.18 FIG. 1 is a diagram showing an example of a frame period FT in this embodiment. Fig.18 As shown, the frame period FT includes a plurality of sub-frame periods SFT (a sub-frame period SFT1 and a sub-frame period SFT2) in which the line directions of the line light L are different.

[0231] Subframe period SFT1 (first subframe period) is a subframe period SFT for irradiating light pulses PO of linear light L parallel to the longitudinal direction. Subframe period SFT2 (second subframe period) is a subframe period SFT for irradiating light pulses PO of linear light L parallel to the transverse direction.

[0232] Thus, in the present embodiment, the light source unit 102 can emit structured light composed of periodically arranged line lights L, and the line lights L in two directions orthogonal to each other, as the light pulses PO. In addition, the measurement control unit 143a uses the timing control unit 141 to perform charge accumulation and acquisition of the charge amount according to the frame period FT of a plurality of sub-frame periods SFT having different line directions of the line lights L. In addition, in the present embodiment, the sub-frame period SFT1 and the sub-frame period SFT2 have the same accumulation period and readout period.

[0233] return Fig.17 As described above, the distance calculation unit 142a calculates the synthesized distance to the object OB based on the charge amounts respectively accumulated in a plurality of subframe periods SFT having different line directions. For example, the distance calculation unit 142a calculates the distance to the object OB based on the charge amount synthesized by the charge amount obtained by the subframe period SFT1 and the charge amount obtained by the subframe period SFT2.

[0234] In addition, in this embodiment, the light source unit 102 is Fig.18 As shown, in the frame period FT, the line light L having different line directions is alternately changed every sub-frame period SFT to emit the light pulse PO. The distance calculation unit 142a calculates the synthesized distance to the object OB based on the charge amounts respectively accumulated in the plurality of sub-frame periods SFT in which the line directions are alternately changed.

[0235] Next, the operation of the distance image capturing device 101 a according to the present embodiment will be described with reference to the drawings. Fig.19 1 is a flowchart showing an example of the operation of the distance image capturing device 101a according to the present embodiment. Here, an example of the calculation process of the distance to the object OB according to the present embodiment will be described.

[0236] like Fig.19 As shown, the distance image camera 101a first performs charge accumulation and charge amount acquisition according to the frame period including the first subframe period (subframe period SFT1) of the longitudinal light and the second subframe period (subframe period SFT2) of the transverse light (step S201). The measurement control unit 143a of the distance image camera 101a instructs the timing control unit 141 to perform a charge accumulation and charge amount acquisition according to the frame period including the first subframe period (subframe period SFT1) of the longitudinal light and the second subframe period (subframe period SFT2) of the transverse light, for example. Fig.18Charge accumulation and charge amount acquisition are performed in the frame period FT shown. The timing control unit 141 executes charge accumulation and charge amount acquisition according to the frame period FT including the sub-frame period SFT1 and the sub-frame period SFT2 in response to the control of the measurement control unit 143a.

[0237] Next, the measurement control unit 143a instructs the distance calculation unit 142a to perform distance calculation processing, and the distance calculation unit 142a combines the charge amount obtained in the first subframe period (subframe period SFT1) with the charge amount obtained in the second subframe period (subframe period SFT2) to calculate the distance to the object OB (step S202). The distance calculation unit 142a, for example, adds the charge amount obtained in the subframe period SFT1 and the charge amount obtained in the subframe period SFT2, and calculates the distance to the object OB using the above-mentioned formula (2).

[0238] Next, the measurement control unit 143a determines whether the execution of all pixel circuits 421 has been completed (step S203). That is, the measurement control unit 143a determines whether the calculation process of the distance of one frame period FT has been completed. When the execution of all pixel circuits 421 has been completed (step S203: Yes), the measurement control unit 143a ends the process. On the other hand, when the execution of all pixel circuits 421 has not been completed (step S203: No), the measurement control unit 143a returns the process to step S202 and executes the calculation process of the distance of the next pixel circuit 421.

[0239] Next, refer to Fig. 20 as well as Fig.21 , a modification example of the distance image capturing device 101a of the present embodiment will be described. Fig. 20 FIG. 4 is a diagram showing a modified example of the frame period FT in this embodiment. Fig. 20 In the illustrated variation, the frame period FT has three sub-frame periods SFT.

[0240] In this modification, the light source unit 102 irradiates the light pulse PO by changing the proportion of the sub-frame periods SFT corresponding to the line light L in two directions, among a plurality of sub-frame periods SFT, according to the moving direction of the object OB in the frame period FT. Here, the light source unit 102 irradiates the light pulse PO by changing the proportion of the sub-frame period SFT corresponding to the direction in which the moving amount of the object OB is larger in the two directions in the frame period FT to be larger than the proportion of the sub-frame period SFT corresponding to the direction in which the moving amount of the object OB is smaller.

[0241] For example, Fig. 20 The example shown in (a) shows an example in which the subject OB moves in the lateral direction. Fig. 20In (a), the frame period FT includes a sub-frame period SFT1-1 of the vertical line light, a sub-frame period SFT1-2 of the vertical line light, and a sub-frame period SFT2 of the horizontal line light.

[0242] In this case, since the object OB moves in the lateral direction, by increasing the ratio of the sub-frame period SFT1 of the vertical line light, it is easy to detect the object OB and the measurement accuracy can be improved.

[0243] in addition, Fig. 20 The example shown in (b) shows an example in which the subject OB moves in the vertical direction. Fig. 20 In (b), the frame period FT includes a sub-frame period SFT1 of the vertical line light, a sub-frame period SFT2-1 of the horizontal line light, and a sub-frame period SFT2-2 of the horizontal line light.

[0244] In this case, the object OB moves in the longitudinal direction, so by increasing the ratio of the subframe period SFT2 of the horizontal light, the object OB is easily detected, which can improve the measurement accuracy. In addition, in this embodiment, the moving direction of the object OB can be specified by the user, or the moving direction can be determined based on the measured image information.

[0245] Next, refer to Fig.21 , a process of changing the ratio of the sub-frame period SFT of the distance image capturing device 101a according to the present embodiment will be described. Fig.21 1 is a flowchart showing an example of the operation of the distance image capturing device 101a according to the present embodiment. Here, a process of changing the ratio of the sub-frame period SFT according to the moving direction of the object OB will be described.

[0246] like Fig.21 As shown in FIG. 1 , the measurement control unit 143a of the distance image pickup device 101a determines whether the moving direction of the object OB is the horizontal axis direction (step S301). When the moving direction of the object OB is the horizontal axis direction (step S301: Yes), the measurement control unit 143a causes the process to proceed to step S302. When the moving direction of the object OB is not the horizontal axis direction (vertical direction) (step S301: No), the measurement control unit 143a causes the process to proceed to step S303.

[0247] In step S302, the measurement control unit 143a increases the ratio of the subframe period SFT1 in the vertical axis direction. The measurement control unit 143a controls the timing control unit 141, for example, Fig. 20 As shown in (a) of FIG. 1 , the ratio of the subframe period SFT1 in the longitudinal axis direction (vertical line light) is increased. After the process of step S302 , the measurement control unit 143 a ends the process.

[0248] In step S303, the measurement control unit 143a increases the ratio of the subframe period SFT2 in the horizontal axis direction. The measurement control unit 143a controls the timing control unit 141, for example, Fig. 20 As shown in (b) of FIG. 1 , the ratio of the subframe period SFT2 in the horizontal axis direction (horizontal light) is increased. After the process of step S303 , the measurement control unit 143 a ends the process.

[0249] As described above, the distance image capturing device 101a of the present embodiment includes a light source unit 102, a light receiving unit 103, and a distance calculation unit 142a. The light source unit 102 irradiates the light pulse PO to the object OB. The light receiving unit 103 includes a pixel circuit 421, a pixel driving circuit 422, and a charge discharge transistor GD (charge discharge unit). The pixel circuit 421 is arranged in a two-dimensional matrix, and includes a plurality of photoelectric conversion elements PD that generate charges corresponding to the incident light and a plurality of charge storage units CS that store charges. The pixel driving circuit 422 distributes charges to the charge storage units CS at the storage timing synchronized with the irradiation of the light pulse PO according to the frame period FT and stores them. The charge discharge transistor GD (charge discharge unit) discharges the charges during the period other than the storage timing. The distance calculation unit 142a calculates the distance to the object OB based on the amount of charge stored in each charge storage unit CS. The light source unit 102 can emit structured light composed of periodically arranged line lights L, and the line lights L are in two directions orthogonal to each other, as light pulses PO. The frame period FT has a plurality of sub-frame periods SFT with different line directions of the line lights L (for example, a sub-frame period SFT1 of the vertical line lights and a sub-frame period SFT2 of the horizontal line lights). The distance calculation unit 142a calculates the synthesized distance to the object OB based on the amount of charge accumulated in each of the plurality of sub-frame periods SFT with different line directions.

[0250] Thus, the distance image camera 101a of the present embodiment uses the amount of charge accumulated in each of the plurality of sub-frame periods SFT with different line directions (e.g., the sub-frame period SFT1 of the longitudinal line light and the sub-frame period SFT2 of the transverse line light). In the distance image camera 101a of the present embodiment, since the sub-frame period SFT in one direction is irradiated, the amount of charge (amount of light) is not saturated at the intersection of the line light L as in the case where the line light L in two directions is irradiated simultaneously. Thus, the distance image camera 101a of the present embodiment can alleviate the signal output difference between the intersection of the line light L and the portion other than the intersection, and suppress the reduction in the resolution of the overall distance image and the measurable distance.

[0251] In addition, in the present embodiment, the light source unit 102 alternately changes the line light L with different line directions in each sub-frame period SFT in the frame period FT and irradiates the light pulse PO. For example, the light source unit 102 alternately irradiates the sub-frame period SFT1 of the vertical line light and the sub-frame period SFT2 of the horizontal line light. The distance calculation unit 142a calculates the synthesized distance to the object OB based on the amount of charge accumulated in each of the plurality of sub-frame periods SFT in which the line directions are alternately changed.

[0252] Therefore, the distance image camera device 101a of the present embodiment alternately changes the line light L with different line directions according to each sub-frame period SFT and irradiates the light pulse PO, and calculates the synthesized distance to the subject OB based on the amount of charge accumulated in each of them. Therefore, the same effect as the case of simultaneously irradiating the line light L with different line directions (such as the case of the second embodiment) can be obtained, and the reduction in resolution can be suppressed while suppressing the necessary light source output.

[0253] In addition, in the present embodiment, the light source unit 102 changes the ratio of the sub-frame periods SFT corresponding to the line light L in two directions in a plurality of sub-frame periods SFT according to the moving direction of the object OB in the frame period FT, and irradiates the light pulse PO. For example, the light source unit 102 changes the ratio of the sub-frame period SFT corresponding to the direction in which the moving amount of the object OB is larger in the two directions in the frame period FT to be greater than the ratio of the sub-frame period SFT corresponding to the direction in which the moving amount of the object OB is smaller, and irradiates the light pulse PO. As a result, the distance image capturing device 101a of the present embodiment can easily detect the moving object OB, and can improve the measurement accuracy.

[0254] In addition, the distance image capturing method of the present embodiment is a distance image capturing method of the distance image capturing device 101a having the above-mentioned light source unit 102, light receiving unit 103, and distance calculation unit 142a. In the distance image capturing method, the light source unit 102 can emit structured light composed of periodically arranged line lights L as light pulses PO, and the line lights L are in two directions orthogonal to each other. The frame period FT has a plurality of sub-frame periods SFT with different line directions of the line lights L. Moreover, the distance calculation unit 142a calculates the synthesized distance to the object OB based on the amount of charge accumulated in each of the plurality of sub-frame periods SFT with different line directions. As a result, the distance image capturing method of the present embodiment has the same effect as the above-mentioned distance image capturing device 101a, and can suppress the reduction in resolution while suppressing the necessary light source output.

[0255] (Fourth embodiment)

[0256] Next, a distance image capturing device 101b according to a fourth embodiment will be described with reference to the drawings. In the fourth embodiment, a method of replacing the pixel circuit 421 corresponding to the intersection of the line light L with a pixel circuit for preventing saturation will be described.

[0257] Fig. 22 FIG. 1 is a diagram showing an example of a distance image capturing device 101b according to a fourth embodiment. Fig. 22 An example of the configuration of the distance image sensor 132a in this embodiment is shown in . In addition, the configuration other than the distance image sensor 132a in this embodiment is the same as that in the second embodiment, and therefore the description thereof is omitted here.

[0258] exist Fig. 22 , the reflected light LH represents the reflected light of the horizontal line light L, and the reflected light LV represents the reflected light of the vertical line light L. In addition, the pixel circuit 421a (first pixel circuit) represents the pixel circuit corresponding to the position (intersection of the reflected light) where the reflected light from the object OB irradiated orthogonally to each other by the line light L is received, and the pixel circuit 421a (second pixel circuit) represents the pixel circuit corresponding to the other position.

[0259] The pixel circuit 421a (first pixel circuit) is provided with a sensitivity suppressing unit, and is configured to have a lower sensitivity to reflected light than the pixel circuit 421 (second pixel circuit) corresponding to other positions. For example, the pixel circuit 421a includes a filter layer disposed on the pixel circuit 421a as a sensitivity suppressing unit, which reflects or absorbs a portion of the reflected light. The filter layer is configured, for example, so that the sensitivity of the pixel circuit 421a becomes less than 1 / 2 of the sensitivity of other positions. Here, the filter layer is, for example, an organic thin film that absorbs a portion of the reflected light by a method such as photolithography.

[0260] In addition, as the sensitivity suppressing unit, the capacity of the charge storage unit CS of the pixel circuit 421a (first pixel circuit) may be larger than the capacity of the charge storage unit CS of the pixel circuit 421 (second pixel circuit). Fig.11 The charge storage capacity C1 to the charge storage capacity C4 of the pixel circuit 421 a shown are, for example, twice or more the capacity of the pixel circuit 421 .

[0261] In addition, the sensitivity suppressor may be configured such that the volume of the photoelectric conversion element PD of the pixel circuit 421a is smaller than that of the pixel circuit 421. That is, the volume of the photoelectric conversion element PD of the pixel circuit 421a may be 1 / 2 or less of that of the pixel circuit 421.

[0262] In addition, as the sensitivity suppressing unit, the number of allocations (cumulative number of times) of the pixel circuit 421a within the frame period FT may be smaller than the number of allocations (cumulative number of times) within the frame period FT of the pixel circuit 421. That is, the number of allocations (cumulative number of times) of the pixel circuit 421a may be less than 1 / 2 of the number of allocations (cumulative number of times) of the pixel circuit 421.

[0263] In the operation of the distance image capturing device 101b of this embodiment, the frame period FT does not include the subframe period SFT, but becomes a single frame. Fig.14 Line lights L in two orthogonal directions as shown are simultaneously irradiated as light pulses PO, and the image distance is calculated based on the acquired charge amount.

[0264] As described above, the distance image capturing device 101b of the present embodiment includes a light source unit 102, a light receiving unit 103, and a distance calculation unit 142. The light source unit 102 irradiates the light pulse PO to the object OB. The light receiving unit 103 includes a pixel circuit 421, a pixel driving circuit 422, and a charge discharge transistor GD (charge discharge unit). The pixel circuit 421 is arranged in a two-dimensional matrix, and includes a plurality of photoelectric conversion elements PD that generate charges corresponding to the incident light and a plurality of charge storage units CS that store charges. The pixel driving circuit 422 distributes charges to the charge storage units CS at the storage timing synchronized with the irradiation of the light pulse PO according to the frame period FT and stores them. The charge discharge transistor GD (charge discharge unit) discharges charges during a period other than the storage timing. The distance calculation unit 142 calculates the distance to the object OB based on the amount of charge stored in each charge storage unit CS. The light source unit 102 emits structured light consisting of periodically arranged line lights L, that is, light pulses PO, and the line lights L in two directions are orthogonal to each other. Furthermore, the pixel circuit 421a (first pixel circuit) corresponding to the position receiving reflected light from the object OB irradiated orthogonally to each other by the line lights L among the plurality of pixel circuits 421 is configured to have a lower sensitivity to reflected light than the pixel circuits 421 (second pixel circuit) corresponding to other positions.

[0265] Thus, the distance image camera device 101b of the present embodiment is configured such that the pixel circuit 421a (first pixel circuit) is less sensitive to reflected light than the pixel circuit 421 (second pixel circuit) corresponding to other positions, so that the charge amount (light amount) saturation can be suppressed at the intersection of the line light L. Thus, the distance image camera device 101b of the present embodiment can alleviate the signal output difference between the intersection of the line light L and the portion other than the intersection, and suppress the reduction in the resolution of the entire distance image and the measurable distance. In addition, the distance image camera device 101b of the present embodiment can suppress the reduction in resolution while suppressing the necessary light source output by using the line light L.

[0266] In addition, the distance image camera device 101b of this embodiment includes a filter layer disposed on the pixel circuit 421a (first pixel circuit) to reflect or absorb a portion of the reflected light. Thus, the distance image camera device 101b of this embodiment uses the filter layer to make the pixel circuit 421a (first pixel circuit) appropriately less sensitive to reflected light than the pixel circuit 421 (second pixel circuit) corresponding to another position.

[0267] In addition, in the present embodiment, the capacity of the charge storage unit CS of the pixel circuit 421a may be larger than the capacity of the charge storage unit CS of the pixel circuit 421 (the second pixel circuit). Thus, the capacity of the charge storage unit CS of the pixel circuit 421a of the distance image pickup device 101b of the present embodiment is larger than the capacity of the charge storage unit CS of the pixel circuit 421 (the second pixel circuit), so that the sensitivity of the pixel circuit 421a to reflected light can be appropriately lower than that of the pixel circuit 421 (the second pixel circuit) corresponding to other positions.

[0268] In addition, in the present embodiment, the volume of the photoelectric conversion element PD of the pixel circuit 421a may be smaller than the volume of the photoelectric conversion element PD of the pixel circuit 421a. Thus, the distance image pickup device 101b of the present embodiment has a low photoelectric conversion capability because the volume of the photoelectric conversion element PD of the pixel circuit 421a is smaller than the volume of the photoelectric conversion element PD of the pixel circuit 421a, and thus the sensitivity of the pixel circuit 421a to reflected light can be appropriately lower than that of the pixel circuit 421 (second pixel circuit) corresponding to other positions.

[0269] In the present embodiment, the number of allocations (accumulated numbers) of the pixel circuit 421 a within the frame period FT may be smaller than the number of allocations (accumulated numbers) of the pixel circuit 421 within the frame period FT.

[0270] Therefore, the pixel circuit 421a of the distance image camera device 101b of this embodiment, whose allocation count (accumulated count) is less than the allocation count (accumulated count) of the pixel circuit 421, has a suitably lower sensitivity to reflected light than the pixel circuit 421 (the second pixel circuit) corresponding to other positions.

[0271] In addition, the present invention is not limited to the above-mentioned embodiments, and can be changed within the scope of the main purpose of the present invention. For example, in the above-mentioned embodiments, the pixel circuit 421 (421a) is described as having four charge storage units CS (CS1, CS2, CS3, CS4), but it is not limited to this, and it can also have more than 3 N charge storage units CS.

[0272] In addition, in the above-mentioned embodiments, an example is described in which the distance image processing unit 104 (104a) is provided inside the distance image camera device 101 (101a, 101b), but it is not limited to this. The distance image processing unit 104 (104a) can also be configured to be provided outside the distance image camera device 101 (101a, 101b).

[0273] In addition, in the above-mentioned embodiments, the photoelectric conversion element PD is described as an example of an embedded photodiode that generates electric charges by photoelectric conversion of incident light and accumulates the generated electric charges, but it is not limited to this, and the structure of the photoelectric conversion element PD can be arbitrary. The photoelectric conversion element PD can also be, for example, a PN photodiode having a structure in which a P-type semiconductor and an N-type semiconductor are joined, or a PIN photodiode having a structure in which an I-type semiconductor is sandwiched between a P-type semiconductor and an N-type semiconductor. In addition, the photoelectric conversion element PD is not limited to a photodiode, and can also be, for example, a photoelectric conversion element of a photogate type.

[0274] Furthermore, in the second embodiment described above, an example in which the frame period FT includes two sub-frame periods SFT having different accumulation periods has been described, but the present invention is not limited thereto and the frame period FT may include three or more sub-frame periods SFT.

[0275] In the third embodiment described above, an example was described in which one sub-frame period SFT1 - 2 (or sub-frame period SFT2 - 2 ) was added when the ratio of the sub-frame period SFT was increased, but two or more sub-frame periods may be added.

[0276] In addition, each component of the distance image camera 101 (101a, 101b) has a computer system inside. Then, the program for realizing the functions of each component of the distance image camera 101 (101a, 101b) can be recorded on a computer-readable recording medium, and the program recorded on the recording medium can be read into the computer system and executed, thereby processing each component of the distance image camera 101 (101a, 101b). Here, "reading the program recorded on the recording medium into the computer system and executing it" includes installing the program in the computer system. The "computer system" mentioned here includes hardware such as OS and peripheral devices.

[0277] In addition, the "computer system" may also include a plurality of computer devices connected via a network including a communication line such as the Internet, WAN, LAN, or a dedicated line. In addition, the "computer-readable recording medium" refers to a removable medium such as a floppy disk, a magneto-optical disk, a ROM, a CD-ROM, or a storage device such as a hard disk built into the computer system. In this way, the recording medium storing the program may also be a non-temporary recording medium such as a CD-ROM.

[0278] In addition, the recording medium also includes an internal or external recording medium that can be accessed from a distribution server in order to distribute the program. In addition, the program may be divided into multiple parts, and after being downloaded at different time intervals, the parts are combined in each component of the distance image camera 101 (101a, 101b), or the divided programs may be distributed using different distribution servers. Furthermore, the "computer-readable recording medium" may also be a recording medium that keeps the program for a certain period of time, such as a volatile memory (RAM) inside a computer system that becomes a server or client when the program is sent via a network. In addition, the above-mentioned program may also be a program for realizing a part of the above-mentioned functions. Moreover, it may also be a so-called differential file (differential program) that can realize the above-mentioned functions by combining with a program already recorded in a computer system.

[0279] In addition, part or all of the above functions may be implemented as an integrated circuit such as LSI (Large Scale Integration). Each of the above functions may be independently processed by a processor, or part or all of them may be integrated and processed. In addition, the method of integrated circuitization is not limited to LSI, and may also be implemented by a dedicated circuit or a general-purpose processor. In addition, when a technology for integrated circuitization that replaces LSI emerges due to the advancement of semiconductor technology, an integrated circuit based on the technology may also be used.

[0280] As mentioned above, although the embodiment of this invention is described in detail with reference to drawings, a specific structure is not limited to this embodiment, and also includes the design etc. which do not deviate from the scope of the summary of this invention.

[0281] Industrial Applicability

[0282] According to the present invention, it is possible to suppress a reduction in resolution of a distance image captured using a point light source, and to alleviate a signal output difference between an intersection of line lights and portions other than the intersection, thereby suppressing a reduction in resolution of the entire distance image and a measurable distance.

[0283] Description of Reference Numerals

[0284] 1 Distance image camera device

[0285] 2 Light source

[0286] 3 Light receiving part

[0287] 32 Distance Image Sensor

[0288] 321 pixels

[0289] 42 Distance calculation unit

[0290] CS Charge storage unit

[0291] PO Light Pulse

[0292] RL Reflected Light

[0293] Dt Point Light

[0294] L line light

[0295] 101, 101a, 101b distance image camera device

[0296] 102 Light source

[0297] 103 Light receiving part

[0298] 104, 104a distance image processing unit

[0299] 121 Light source device

[0300] 122 Diffuser Plate

[0301] 131 Lens

[0302] 132, 132a distance image sensor

[0303] 141 Timing Control Unit

[0304] 142, 142a distance calculation unit

[0305] 143, 143a measurement control unit

[0306] 420 light receiving area

[0307] 421, 421a pixel circuit

[0308] 422 pixel driving circuit

[0309] 423 vertical scanning circuit

[0310] 424 horizontal scanning circuit

[0311] 425 pixel signal processing circuit

[0312] 426 Control Circuit

[0313] CS, CS1, CS2, CS3, CS4 charge storage unit

[0314] FD, FD1, FD2, FD3, FD4 floating diffusion areas

[0315] FT frame period

[0316] G, G1, G2, G3, G4 transfer transistors

[0317] GD Charge Drain Transistor

[0318] OB Subject

[0319] PD Photoelectric conversion element

[0320] PO Light Pulse

[0321] RL Reflected Light

[0322] RT1, RT2, RT3, RT4 reset transistors

[0323] SF, SF1, SF2, SF3, SF4 source follower transistors

[0324] SFT1, SFT2, SFT3 subframe period

[0325] SL, SL1, SL2, SL3, SL4 selection transistor

[0326] Dt, Dt1, Dt2 point light

[0327] L, L1~L12 line light

Claims

1. A distance image camera device, comprising: A light source unit irradiates light pulses to a subject; a light receiving unit having a pixel circuit, a pixel driving circuit, and a charge discharge unit, wherein the pixel circuit is arranged in a two-dimensional matrix and includes a photoelectric conversion element that generates charges corresponding to incident light and a plurality of charge storage units that store the charges, the pixel driving circuit distributes the charges to the charge storage units at a storage timing synchronized with the irradiation of the light pulse and stores the charges, and the charge discharge unit discharges the charges during a period other than the storage timing; and a distance calculation unit that calculates the distance to the subject based on the amount of charge accumulated by the charge accumulation unit; The light pulse is a structured light composed of multiple point lights. At least one first point light among the plurality of point lights has an elliptical shape in which the ratio of the major axis length to the minor axis length is equal to or greater than a threshold value.

2. The distance image camera device according to claim 1, wherein: At least a portion of the first point light overlaps with at least a portion of another point light adjacent to the first point light in the long-axis direction.

3. The distance image camera device according to claim 1, wherein: The light source unit includes a plurality of light source elements capable of independently irradiating the light pulses.

4. The distance image camera device according to claim 3, wherein: The major axis directions of the elliptical spot lights emitted from at least two of the plurality of light source elements are different from each other.

5. The distance image camera device according to claim 4, wherein: The major axis directions of the elliptical spot lights emitted from at least two of the plurality of light source elements are orthogonal to each other.

6. The distance image camera device according to claim 5, wherein: Regarding a first interval along a short axis direction among point lights with their major axes in the same direction and emitted from a first light source element group composed of at least two light source elements among the plurality of light source elements, and a second interval along a short axis direction among point lights with their major axes in the same direction and emitted from a second light source element group composed of at least two light source elements different from the first light source element group, the first interval and the second interval are different from each other.

7. The distance image camera device according to claim 5, wherein: The major axis direction of the elliptical spot light emitted from at least two of the plurality of light source elements is an inclined direction other than a vertical direction and a horizontal direction with respect to the installation surface of the imaging device.

8. The distance image camera device according to claim 7, wherein: The angle formed by the major axis direction of the elliptical spot light emitted from two of the plurality of light source elements with respect to the installation surface of the imaging device is 45 degrees or 135 degrees.

9. The distance image camera device according to claim 3, wherein: At least one of the plurality of light source elements is a diffuse light source.

10. A distance image capturing method, performed by a distance image capturing device, The distance image camera device comprises: A light source unit irradiates light pulses to a subject; a light receiving unit having a pixel circuit, a pixel driving circuit, and a charge discharge unit, wherein the pixel circuit is arranged in a two-dimensional matrix and includes a photoelectric conversion element that generates charges corresponding to incident light and a plurality of charge storage units that store the charges, the pixel driving circuit distributes the charges to the charge storage units at a storage timing synchronized with the irradiation of the light pulse and stores the charges, and the charge discharge unit discharges the charges during a period other than the storage timing; and a distance calculation unit that calculates the distance to the subject based on the amount of charge accumulated by the charge accumulation unit; In the range image capturing method, The light pulse is a structured light composed of multiple point lights. At least one first point light among the plurality of point lights has an elliptical shape in which the ratio of the major axis length to the minor axis length is equal to or greater than a threshold value.

11. A distance image camera device, comprising: A light source unit irradiates light pulses to the subject; a light receiving unit having a pixel circuit, a pixel driving circuit, and a charge discharge unit, wherein the pixel circuit is arranged in a two-dimensional matrix in plurality and includes a photoelectric conversion element that generates charges corresponding to incident light and a plurality of charge storage units that store the charges, the pixel driving circuit distributes the charges to the charge storage units at storage timings synchronized with irradiation of the light pulses according to a frame cycle and stores the charges, and the charge discharge unit discharges the charges during a period other than the storage timing; and a distance calculation unit that calculates the distance to the subject based on the amount of charge accumulated by the charge accumulation unit; The light source unit radiates the light pulses as structured light composed of periodically arranged line lights, and the line lights in two directions are orthogonal to each other. The frame period includes a plurality of sub-frame periods in which the charge accumulation section accumulates the charge for different periods of time. The distance calculation unit calculates the distance to the subject using the sub-frame period in which the charge amount does not exceed a preset threshold value among the plurality of sub-frame periods.

12. The distance image camera device according to claim 11, wherein: The plurality of subframe periods are a first subframe period in which the accumulation period is a predetermined period set in advance, and a second subframe period in which the accumulation period is a period that is twice or more of the predetermined period. The distance calculation unit calculates the distance to the subject using the subframe period of the first subframe period and the second subframe period in which the charge amount does not exceed the threshold value and the accumulation period is longer.

13. A distance image camera device, comprising: A light source unit irradiates light pulses to the subject; a light receiving unit having a pixel circuit, a pixel driving circuit, and a charge discharge unit, wherein the pixel circuit is arranged in a two-dimensional matrix in plurality and includes a photoelectric conversion element that generates charges corresponding to incident light and a plurality of charge storage units that store the charges, the pixel driving circuit distributes the charges to the charge storage units at storage timings synchronized with irradiation of the light pulses according to a frame cycle and stores the charges, and the charge discharge unit discharges the charges during a period other than the storage timing; and a distance calculation unit that calculates the distance to the subject based on the amount of charge accumulated by the charge accumulation unit; The light source unit can emit structured light composed of periodically arranged line lights, and the line lights are in two directions orthogonal to each other, as the light pulses. The frame period has a plurality of sub-frame periods in which the line directions of the line light are different. The distance calculation unit calculates a combined distance to the subject based on the amount of charge accumulated in each of the plurality of sub-frame periods having different line directions.

14. The distance image camera device according to claim 13, wherein: The light source unit irradiates the light pulses by alternately changing the line lights having different line directions every sub-frame period in the frame period. The distance calculation unit calculates a combined distance to the subject based on the amount of charge accumulated in each of the plurality of sub-frame periods in which the line direction is alternately changed.

15. The distance image camera device according to claim 13, wherein: The light source unit irradiates the light pulse by changing the ratio of the sub-frame periods corresponding to the line lights in the two directions, among the plurality of sub-frame periods, in accordance with the moving direction of the subject in the frame period.

16. The distance image camera device according to claim 15, wherein: The light source unit irradiates the light pulse by changing the ratio of the sub-frame period corresponding to the direction in which the subject moves more greatly among the two directions to be greater than the ratio of the sub-frame period corresponding to the direction in which the subject moves less greatly in the frame period.

17. The distance image camera device according to any one of claims 11 to 16, wherein: The light source unit includes a plurality of light source elements capable of independently irradiating the light pulses. The line light is formed by the plurality of light source elements discretely radiating a plurality of point lights into a line shape, or by radiating the plurality of point lights so as to overlap each other into a line shape.

18. The distance image camera device according to claim 17, wherein: The shape of the point light is an ellipse whose ratio of the major axis length to the minor axis length is greater than 2. The line light is formed into a line shape along the direction of the long axis.

19. A distance image capturing method, performed by a distance image capturing device, The distance image camera device comprises: A light source unit irradiates light pulses to a subject; a light receiving unit having a pixel circuit, a pixel driving circuit, and a charge discharge unit, wherein the pixel circuit is arranged in a two-dimensional matrix in plurality and includes a photoelectric conversion element that generates charges corresponding to incident light and a plurality of charge storage units that store the charges, the pixel driving circuit distributes the charges to the charge storage units at storage timings synchronized with irradiation of the light pulses according to a frame cycle and stores the charges, and the charge discharge unit discharges the charges during a period other than the storage timing; and a distance calculation unit that calculates the distance to the subject based on the amount of charge accumulated by the charge accumulation unit; In the distance image capturing method, The light source unit radiates the light pulses as structured light composed of periodically arranged line lights, and the line lights in two directions are orthogonal to each other. The frame period includes a plurality of sub-frame periods in which the charge accumulation section accumulates the charge for different periods of time. The distance calculation unit calculates the distance to the subject using the sub-frame period in which the charge amount does not exceed a preset threshold value among the plurality of sub-frame periods.

20. A distance image capturing method, performed by a distance image capturing device, The distance image camera device comprises: A light source unit irradiates light pulses to the subject; a light receiving unit having a pixel circuit, a pixel driving circuit, and a charge discharge unit, wherein the pixel circuit is arranged in a two-dimensional matrix in plurality and includes a photoelectric conversion element that generates charges corresponding to incident light and a plurality of charge storage units that store the charges, the pixel driving circuit distributes the charges to the charge storage units at storage timings synchronized with irradiation of the light pulses according to a frame cycle and stores the charges, and the charge discharge unit discharges the charges during a period other than the storage timing; and a distance calculation unit that calculates the distance to the subject based on the amount of charge accumulated by the charge accumulation unit; In the range image capturing method, The light source unit can emit structured light composed of periodically arranged line lights, and the line lights are in two directions orthogonal to each other, as the light pulses. The frame period has a plurality of sub-frame periods in which the line directions of the line light are different. The distance calculation unit calculates a combined distance to the subject based on the amount of charge accumulated in each of the plurality of sub-frame periods having different line directions.

21. A distance image camera device, comprising: A light source unit irradiates light pulses to the subject; a light receiving unit having a pixel circuit, a pixel driving circuit, and a charge discharge unit, wherein the pixel circuit is arranged in a two-dimensional matrix in plurality and includes a photoelectric conversion element that generates charges corresponding to incident light and a plurality of charge storage units that store the charges, the pixel driving circuit distributes the charges to the charge storage units at storage timings synchronized with irradiation of the light pulses according to a frame cycle and stores the charges, and the charge discharge unit discharges the charges during a period other than the storage timing; and a distance calculation unit that calculates the distance to the subject based on the amount of charge accumulated by the charge accumulation unit; The light source unit radiates the light pulses as structured light composed of periodically arranged line lights, and the line lights in two directions are orthogonal to each other. Among the plurality of pixel circuits, a first pixel circuit corresponding to a position receiving reflected light from the subject irradiated orthogonally by the line light is configured to have lower sensitivity to reflected light than a second pixel circuit corresponding to another position.

22. The distance image camera device according to claim 21, wherein: A filter layer is provided, and the filter layer is arranged on the first pixel circuit and reflects or absorbs a part of the reflected light.

23. The distance image camera device according to claim 21, wherein: The charge storage section of the first pixel circuit is configured to have a capacity greater than a capacity of the charge storage section of the second pixel circuit.

24. The distance image camera device according to claim 21, wherein: The volume of the photoelectric conversion element of the first pixel circuit is configured to be smaller than the volume of the photoelectric conversion element of the second pixel circuit.

25. The distance image camera device according to claim 21, wherein: The number of allocations to the first pixel circuit in the frame period is configured to be smaller than the number of allocations to the second pixel circuit in the frame period.

26. The distance image camera device according to any one of claims 21 to 25, wherein: The light source unit includes a plurality of light source elements capable of independently irradiating the light pulses. The line light is formed by the plurality of light source elements discretely radiating a plurality of point lights into a line shape, or by radiating the plurality of point lights so as to overlap each other into a line shape.

27. The distance image camera device according to claim 26, wherein: The shape of the point light is an ellipse whose ratio of the major axis length to the minor axis length is greater than 2. The line light is formed into a line shape along the direction of the long axis.

Images