Range image shooting device and range image shooting method

Abstract

The range image capturing device of the present invention includes a light source unit, a range image sensor, and a range image processing unit. The light source unit is used to irradiate a measurement space serving as a space to be measured with irradiation light. The range image sensor is used to receive light including reflected light from an object in the measurement space as incident light, accumulate charge generated by the incident light in each pixel, and generate a range image consisting of the charge amount accumulated in each pixel. The range image processing unit is used to store, from the charge amount in the range image, a signal value based on non-controlled charge, independent of control for charge accumulation in the range image sensor, and use the stored signal value to correct and obtain the distance to the object in the space.

Description

Technical Field

[0001] The present invention relates to a distance image shooting device and a distance image shooting method.

[0002] This application claims priority from PCT / JP2020 / 20221, filed on May 22, 2020, the contents of which are incorporated herein by reference. Background Art

[0003] Traditionally, time-of-flight (TOF) distance image sensors have been developed, leveraging the known speed of light to measure the distance between a measuring instrument and an object based on the time of flight of light in a space (the measurement space). In a TOF distance image sensor, a pulse of light (e.g., near-infrared light) is irradiated onto the object being measured. The distance between the measuring instrument and the object is measured based on the difference between the time of the irradiated light pulse and the time it takes for the light pulse (reflected light) to return from the object in the measurement space. In other words, the time of flight of light between the measuring instrument and the object is used to measure the distance (e.g., see Patent Document 1).

[0004] In a TOF distance image sensor, a photoelectric conversion element converts the amount of incident light into electric charge, accumulates the converted electric charge in a charge accumulation unit, and converts an analog voltage corresponding to the amount of the accumulated electric charge into a digital value via an AD converter.

[0005] Furthermore, a TOF distance image sensor calculates the distance between the measuring instrument and the object based on information on the flight time of light between the measuring instrument and the object, which is included in an analog voltage or digital value corresponding to the charge amount.

[0006] At this time, when the reflected light used for distance measurement is incident, the background light in the measurement space environment is included. In order to accurately determine the distance, it is necessary to remove the background light from the incident light and obtain information only on the reflected light.

[0007] Therefore, in order to remove (eliminate) the influence of background light in the measurement space environment during distance measurement, a period in which the irradiation light is not always irradiated is set, and the amount of light received by only the background light is accumulated. When calculating the distance to the object, the charge caused by the background light accumulated during the period in which the irradiation light is not irradiated is subtracted from the charge caused by the incident light input during the period in which the reflected light from the object (including information on the distance to the object) is received, thereby obtaining the charge caused only by the reflected light.

[0008] Prior art literature

[0009] Patent Literature

[0010] Patent Document 1: Japanese Patent Application Laid-Open No. 2004-294420 Summary of the Invention

[0011] (Problems to be solved by the invention)

[0012] In Patent Document 1, in the case of a TOF sensor, irradiation light is performed multiple times, charges generated by light received at each irradiation are accumulated, and an analog voltage corresponding to the accumulated charges is used for distance measurement.

[0013] At this time, every time the incident light is irradiated, reflected light from the object is incident. The distance is calculated based on the ratio of the charges Q2 and Q3 generated by the incident light. At this time, the light received by the TOF sensor includes not only the reflected light from the object but also the background light in the environment.

[0014] Thus, each of the charges Q2 and Q3 generated by the incident light includes the charge Q1 generated by the background light in addition to the charge generated by the reflected light.

[0015] The charge Q1 caused by background light will cause a decrease in accuracy when calculating the distance from the TOF sensor to the object.

[0016] Therefore, before irradiating the illumination light, only the background light is made incident on the TOF sensor, the charge Q1 is obtained, and the distance L is calculated by the following equation (1).

[0017] L=((Q3-Q1) / (Q2+Q3-2Q1))×(cTw) / 2 (1)

[0018] In equation (1), c is the speed of light, and Tw is the pulse width of the irradiated light. Using equation (1), the distance L from the TOF sensor to the object can be calculated. Here, (cTw) / 2 represents the maximum distance (maximum measurable distance) that can be measured using the light pulse PO.

[0019] However, if Figure 11 As shown, in a distance image capturing device using a TOF sensor, a lens 31 is provided to focus light on a pixel area of a distance image sensor 32 (sensor chip) having pixel elements formed therein.

[0020] exist Figure 11 In (A), the lens 31 forms an image of incident light having a predetermined viewing angle α for each pixel in the pixel area of the range image sensor 32 .

[0021] Therefore, if Figure 11 As shown in FIG. 5B , for each pixel of the distance image sensor 32 , incident light having various incident angles from the entire lens surface of the lens 31 , for example, which has a spherical shape, forms an image at each pixel in the distance image sensor 32 .

[0022] Here, in the distance image sensor 32 , the incident angle θ2 of the incident light on the pixel of the imaging point 502 at the end corresponding to the angle of view α2 is larger than the incident angle θ1 of the incident light on the pixel of the imaging point 501 corresponding to the angle of view α1 on the axis f of the lens 31 .

[0023] The charge Q1 generated by the background light varies according to the change in the incident angle θ of the light incident on each pixel. Furthermore, the charge accumulated in each pixel includes a different non-control charge QB1.

[0024] Furthermore, the charges Q2 and Q3 accumulated in each pixel also include different non-control charges QB2 and QB3. These non-control charges are charges other than the control charges that are collected by the photoelectric conversion element (photodiode PD, described later) and distributed and accumulated in the charge accumulation unit FD by controlling the on / off control of the readout gate transistor G.

[0025] Figure 12 1 and 2 are diagrams explaining the generation of non-control charges QB1 , QB2 , and QB3 included in the charges Q1 , Q2 , and Q3 , respectively, depending on the angle of incidence of incident light on a pixel. Figure 12 (A) shows a planar structure of an example pixel in a TOF sensor. Charge Q1 is generated by the photodiode PD, the pixel's photoelectric conversion element, and distributed to the charge storage unit FD1 (corresponding to the charge storage unit CS1 described later). More strictly speaking, charge generated by photoelectric conversion in the silicon substrate forming the pixel including the photodiode PD is collected in the photodiode PD and distributed to the charge storage unit FD1 by turning on the readout gate transistor G1.

[0026] Similarly, the charge collected by the photodiode PD and distributed to the charge storage unit FD2 (corresponding to the charge storage unit CS2 described later) becomes charge Q2, and the charge distributed to the charge storage unit FD3 (corresponding to the charge storage unit CS2 described later) becomes charge Q3.

[0027] When read gate transistor G1 is turned on, the charge collected in photodiode PD is distributed to charge storage unit FD1. When read gate transistor G2 is turned on, the charge collected in photodiode PD is distributed to charge storage unit FD2. Furthermore, when read gate transistor G3 is turned on, the charge collected in photodiode PD is distributed to charge storage unit FD3.

[0028] Figure 12 (B) shows Figure 12The structure of the cross section along line segment AA' in (A) shows a case where the incident light on the sensor chip surface of the range image sensor 32 is not 0°, but is at an incident angle of -20° relative to a plane parallel to line segment AA' and perpendicular to the sensor chip plane.

[0029] Here, each of the charge storage units FD1, FD2, and FD3 includes control charges QA1, QA2, and QA3, which are charges collected by the photodiode PD and distributed via the readout gate transistors G1, G2, and G3, respectively, and non-control charges QB1, QB2, and QB3, which flow in without passing through the readout gate transistors G1, G2, and G3. Here, each of the non-control charges QB1, QB2, and QB3 represents charges that flow into the charge storage units FD1, FD2, and FD3 without readout control being performed in the pixel circuit.

[0030] The non-control charges QB1 , QB2 , and QB3 are different from the background light, and vary in the charge storage units FD1 , FD2 , and FD3 according to the incident angle of the incident light.

[0031] exist Figure 12 In case (B), more incident light enters the charge storage unit FD2 than the charge storage unit FD3 , so more non-control charge QB2 flows into the charge storage unit FD2 than non-control charge QB3 flows into the charge storage unit FD3 .

[0032] Figure 13 This diagram shows that the charge accumulated in the charge accumulation unit changes depending on the angle of incidence of incident light with respect to the sensor chip. Figure 13 (A) shows the experimental concept of changing the angle of inclination of the surface formed by the pixels of the distance image sensor 32 relative to the radiation (irradiation) direction of the collimated light by pseudo-simplifying the change of the incident angle of the incident light relative to the pixels of the distance image sensor 32 using the lens 31.

[0033] Figure 13 In (B), the horizontal axis represents the angle of incidence, and the vertical axis represents the digital value (LSB) of each of the charge amounts Q1 (solid line), Q2 (single-dot chain line), and Q3 (double-dot chain line) in each of the charge storage units FD1, FD2, and FD3. The amount of incident light decreases by the amount that the projected area is reduced due to the incident angle of the incident light. Specifically, if the angle of incidence is θ, it changes to cos (cosine) θ times. Therefore, ideally, the amount of charge collected in the photodiode PD also changes to cos (cosine) θ times when the angle of incidence is θ. Here, the control charges QA1, QA2, and QA3 collected in the photodiode PD and distributed to the charge storage units FD1, FD2, and FD3 respectively through each of the readout gate transistors G1, G2, and G3 are the same.

[0034] Therefore, for example, when the incident angle θ is negative, the charge Q2 has a larger charge amount than the other charges Q1 and Q3 . This is because the non-control charges QB1 , QB2 , and QB3 differ depending on the incident angle of the incident light.

[0035] Figure 14 1 is a conceptual diagram showing the relationship between the non-control charges QB1 , QB2 , and QB3 in each of the charges Q1 , Q2 , and Q3 . Figure 14 (A) shows the charges Q1, Q2, and Q3 when the non-control charges QB1, QB2, and QB3 are the same, for example, when the incident angle is small (smaller than 20° described later) such as 0°. Figure 14 In case (A), as shown in equation (1), by removing charge Q1 from each of charges Q2 and Q3, all charges generated by background light are removed, and the distance can be calculated with high accuracy based only on the charges generated by reflected light.

[0036] on the other hand, Figure 14 (B) shows the charges Q1, Q2, and Q3 when the non-control charges QB1, QB2, and QB3 are different, for example, when the incident angle is a large angle of incidence such as 20°. Figure 14 In case (B), as shown in formula (1), even if charge Q1 is subtracted from each of charges Q2 and Q3, the amount of charge generated by the background light in each of charges Q2 and Q3 is different from that of charge Q1, and the difference caused by the difference between the non-controlled charges QB1, QB2, and QB3 will remain in the subtraction result. It is impossible to completely remove the charge generated by the background light, so the distance cannot be calculated with high precision.

[0037] In the past, the amount of non-controlled charges QB1, QB2, and QB3 flowing from the photodiode PD into the charge storage units FD1, FD2, and FD3, respectively, was a negligible ratio (ratio within the error range) compared to the amount of charge generated by reflected light and collected by the photodiode PD and distributed to the charge storage units FD1, FD2, and FD3, respectively, by the conduction of the readout gate transistors G1, G2, and G3. The accuracy of the distance calculated by the above formula (1) was also within the error range.

[0038] However, in order to improve the resolution of the range image or further reduce the chip size of the range image sensor 32 , it is necessary to reduce the area where pixels are formed.

[0039] Therefore, the amount of charge generated by the reflected light decreases according to the area ratio, and the difference in the charge amounts of the non-control charges QB1, QB2, and QB3 becomes a non-negligible ratio compared to the charge amount generated by the reflected light. The distance accuracy calculated by formula (1) will decrease as the pixel area decreases.

[0040] In view of the above-mentioned problems, an object is to provide a distance image capturing device and a distance image capturing method that, even when the pixel area in the distance image sensor is reduced, is not affected by each non-control charge included in each charge accumulated in the charge storage unit, the charge amount of which varies depending on the incident angle of the incident light, and can determine the distance between the object and itself with the same accuracy as when the pixel area is not reduced.

[0041] (Solutions to solve problems)

[0042] The range image capturing device of the present invention includes: a light source unit for irradiating a measurement space, which is a space of a measurement object; a range image sensor for receiving light including reflected light from an object in the measurement space as incident light, accumulating charge generated by the incident light in each pixel, and generating a range image consisting of the charge amount of the charge accumulated in each pixel; and a range image processing unit for storing, from the charge amount in the range image, a signal value based on non-controlled charge included in the charge amount, independently of control for accumulating the charge in the range image sensor, and correcting the distance to the object in the space using the stored signal value.

[0043] In the distance image capturing device of the present invention, the distance image sensor includes a pixel circuit in each pixel that performs control for accumulating charges in a charge accumulation unit, the pixel circuit comprising: a photoelectric conversion element for collecting the charges generated in response to the incident light; and the charge accumulation unit for accumulating the charges during a frame period; the distance image processing unit subtracts an adjustment voltage corresponding to the amount of the non-controlled charges from an input voltage corresponding to the amount of charges accumulated in the charge accumulation unit, and measures the distance between the distance image sensor and the measurement object, wherein the non-controlled charges are charges that flow into the charge accumulation unit independently of the control of the pixel circuit.

[0044] In the range image capturing device of the present invention, the charge accumulation unit includes: at least one first charge accumulation unit for accumulating background light charges generated by receiving background light of the space; and two or more second charge accumulation units for accumulating reflected light charges generated by receiving reflected light of the object from the irradiated light.

[0045] The distance image capturing device of the present invention includes a storage unit, which stores: a reference background light voltage obtained from the first charge accumulation unit and the second charge accumulation unit, respectively, which is pre-measured in a state where the irradiation light is not irradiated under specific ambient light and causes each of the first charge accumulation unit and the second charge accumulation unit to accumulate the charge controlled by the pixel circuit; and a reference base voltage obtained from the first charge accumulation unit and the second charge accumulation unit, respectively, without causing each of the first charge accumulation unit and the second charge accumulation unit to accumulate the charge controlled by the pixel circuit.

[0046] In the range image capturing device of the present invention, the range image sensor irradiates the measurement space with irradiation light, receives light including reflected light from an object in the measurement space as incident light, and accumulates charge generated by the incident light in each of the first charge accumulation section and the second charge accumulation section. The range image processing section divides the background light voltage generated by the charge accumulated in the first charge accumulation section by the reference background light voltage measured in advance and stored in the storage section to obtain an adjustment ratio, and multiplies each of the reference reference voltages by the adjustment ratio to calculate the adjustment voltages respectively relative to the input voltages.

[0047] In the range image capturing device of the present invention, the frame period includes each of a first frame period and a second frame period. During the first frame period, the range image sensor irradiates the measurement space with irradiation light, receives light including reflected light from an object in the measurement space as incident light, and causes the charge controlled by the pixel circuit to be accumulated in each of the first charge accumulation section and the second charge accumulation section. The range image processing section obtains the input voltage generated by the accumulated charge. During the second frame period, the range image sensor irradiates the measurement space with irradiation light, receives light including reflected light from an object in the measurement space as incident light, and does not cause the charge controlled by the pixel circuit to be accumulated in each of the first charge accumulation section and the second charge accumulation section. The range image processing section obtains an adjustment voltage corresponding to the non-controlled charge in each of the first charge accumulation section and the second charge accumulation section.

[0048] The range image capturing device of the present invention includes a storage unit, which stores: a reference reflected light voltage obtained from the first charge accumulation unit and the second charge accumulation unit, respectively, which is pre-measured in a state where the irradiation light is irradiated in a light-shielded environment, so that each of the first charge accumulation unit and the second charge accumulation unit accumulates the charge controlled by the pixel circuit; and a reference base voltage obtained from the first charge accumulation unit and the second charge accumulation unit, respectively, without causing each of the first charge accumulation unit and the second charge accumulation unit to accumulate the charge controlled by the pixel circuit.

[0049] In the distance image capturing device of the present invention, the distance image processing unit calculates an adjustment ratio by dividing the result of adding the input voltages generated by the charges accumulated in each of the first charge accumulation unit and the second charge accumulation unit by the result of adding each of the reference reflected light voltages, and multiplies the reference base voltage by the adjustment ratio to calculate the adjustment voltage relative to each of the input voltages.

[0050] In the range image capturing device of the present invention, the frame period includes each of a first frame period and a second frame period. During the first frame period, the range image sensor irradiates the measurement space with irradiation light in a light-shielded environment, receives light including reflected light from an object in the measurement space as incident light, and causes the charge controlled by the pixel circuit to be accumulated in each of the first charge accumulation section and the second charge accumulation section. The range image processing section obtains the input voltage generated by the charge accumulated in each of the first charge accumulation section and the second charge accumulation section. During the second frame period, the range image sensor irradiates the measurement space with irradiation light in a light-shielded environment, receives light including reflected light from an object in the measurement space as incident light, and does not cause the charge controlled by the pixel circuit to be accumulated in each of the first charge accumulation section and the second charge accumulation section. The range image processing section obtains an adjustment voltage corresponding to the non-controlled charge from the charge in each of the first charge accumulation section and the second charge accumulation section.

[0051] The range image capturing device of the present invention includes a storage unit that stores: a reference background light voltage, which is pre-measured in a state under specific ambient light without irradiation of the irradiation light and is obtained from the first charge accumulator and the second charge accumulator, respectively, so that each of the first charge accumulator and the second charge accumulator accumulates the charge controlled by the pixel circuit; and a first reference reference voltage, which is obtained from the first charge accumulator and the second charge accumulator, respectively, as a reference reference voltage without accumulating the charge controlled by the pixel circuit; further, a reference reflected light voltage, which is pre-measured in a state under a light-shielded environment with irradiation of the irradiation light and is obtained from the first charge accumulator and the second charge accumulator, respectively, so that each of the first charge accumulator and the second charge accumulator accumulates the charge controlled by the pixel circuit; and a second reference reference voltage, which is obtained from the first charge accumulator and the second charge accumulator, respectively, as a reference reference voltage without accumulating the charge controlled by the pixel circuit.

[0052] In the distance image capturing device of the present invention, the distance image sensor irradiates the measurement space with irradiation light, receives light including reflected light from an object in the measurement space as incident light, and accumulates charge generated by the incident light in each of the pixels. The distance image processing unit divides the input voltage generated by the accumulated charge by the reference background light voltage measured in advance and stored in the storage unit to obtain a first adjustment ratio as an adjustment ratio, and multiplies each of the first reference base voltages by the first adjustment ratio to calculate a first adjustment voltage respectively relative to the input voltage. Furthermore, the unit divides each sum of the input voltages by each sum of the reference reflected light voltages to obtain a second adjustment ratio as an adjustment ratio, and multiplies the second reference base voltage by the second adjustment ratio to calculate a second adjustment voltage respectively relative to each of the input voltages. The adjustment voltage is calculated by adding the first adjustment voltage and the second adjustment voltage.

[0053] The distance image capturing device of the present invention includes a storage unit, which respectively obtains: a reference background light voltage obtained from each of the N (N is an integer greater than or equal to 3) charge storage units in advance in a state where the irradiation light is not irradiated under specific ambient light, so that each of the N (N is an integer greater than or equal to 3) charge storage units accumulates the charge controlled by the pixel circuit; a first reference reference voltage obtained from each of the charge storage units as a reference reference voltage without each of the charge storage units accumulating the charge controlled by the pixel circuit; a reference reflected light voltage obtained from each of the charge storage units in advance in a state where the irradiation light is irradiated in a light-shielded environment, so that each of the N charge storage units accumulates the charge controlled by the pixel circuit; and a reference reference voltage obtained from each of the charge storage units as a reference reference voltage without each of the charge storage units accumulating the charge controlled by the pixel circuit. The second reference voltage is obtained; the storage unit respectively stores: a reference background light control voltage obtained from each of the reference background light voltage and the first reference voltage, corresponding to the charge amount of the control charge generated by the background light and collected by the photoelectric conversion element and respectively distributed to the N charge storage units for storage; a reference background light non-control voltage corresponding to the charge amount of the non-control charge generated by the background light and not distributed to the N charge storage units and flowing in; a reference reflected light control voltage corresponding to the charge amount of the control charge generated by the reflected light and collected by the photoelectric conversion element and respectively distributed to the N charge storage units and accumulated; and a reference reflected light non-control voltage corresponding to the charge amount of the non-control charge generated by the reflected light and not distributed and flowing into the N charge storage units and accumulated.

[0054] The distance image capturing device of the present invention includes a distance calculation unit, which calculates and obtains a first ratio as a ratio of the amount of background light in the pre-measured environment obtained from a part or all of the reference background light voltage, the first reference base voltage, the reference background light control voltage, and the reference background light non-control voltage stored in the storage unit to the amount of background light in the capturing state, and calculates and obtains a second ratio as a ratio of the amount of reflected light in the pre-measured environment obtained from a part or all of the reference reflected light voltage, the second reference base voltage, the reference reflected light control voltage, and the reference reflected light non-control voltage stored in the storage unit to the amount of reflected light in the capturing state, solves simultaneous equations obtained using each of the first and second ratios, calculates a reflected light control voltage corresponding to the amount of control charge generated by the reflected light in the capturing state and accumulated in the charge accumulation unit, and obtains the distance to the object.

[0055] The range image capturing device of the present invention includes a storage unit, which respectively obtains: a reference reflected light voltage corresponding to the charge amount of the control charge accumulated by the distribution and the non-control charge that flows in and is accumulated independently of the distribution, which is pre-measured in a state where the irradiation light is irradiated, and is controlled to be accumulated in each of the N charge accumulation units; and a second reference reference voltage obtained as a reference reference voltage corresponding to the charge amount of the non-control charge that flows in and is accumulated without performing accumulation control on each of the charge accumulation units; the storage unit respectively stores: a reference reflected light control voltage corresponding to the charge amount of the control charge generated by the reflected light, collected by the photoelectric conversion element, and respectively distributed to the N charge accumulation units for accumulation, which is obtained from each of the reference reflected light voltage and the second reference reference voltage; and a reference reflected light non-control voltage corresponding to the charge amount of the non-control charge generated by the reflected light and flowing into and accumulated in the N charge accumulation units without being distributed.

[0056] The distance image capturing device of the present invention includes a distance calculation unit. In an acquisition state for acquiring the distance, the frame period includes each of a first frame period and a second frame period. In one of the first frame period and the second frame period, the distance image sensor obtains a first reference voltage as a reference voltage corresponding to each charge amount of the non-controlled charge accumulated in each of the N charge accumulation units without irradiating the measurement space with the irradiation light. In the other of the two frame periods, the distance image sensor irradiates the measurement space with the irradiation light and distributes the irradiation light to each of the N charge accumulation units. Control of the charge, obtaining a voltage for calculating the distance from each of the charge storage units, wherein the voltage corresponds to the charge amount including the control charge accumulated by distribution and the non-control charge flowing in independently of the distribution, calculating and obtaining the ratio of the amount of reflected light under the pre-measured environment obtained from part or all of the reference reflected light voltage, the second reference base voltage, the first reference base voltage, and the voltage for calculating the distance stored in the storage unit, solving the simultaneous equations, calculating the reflected light control voltage corresponding to the charge amount of the control charge, and obtaining the distance to the object.

[0057] The distance image capturing device of the present invention includes a distance calculation unit. When acquiring the distance in a light-shielded environment or a dark environment where ambient light is negligible, the frame period includes each of a first frame period and a second frame period. In one of the first frame period and the second frame period, the distance image sensor irradiates the irradiation light and receives incident light from a measurement space. Non-controlled charges flowing into each of the N charge accumulation units are accumulated without accumulating charges controlled by the pixel circuit. A second reference voltage is obtained from each of the charge accumulation units. In the two frame periods, In another embodiment, a reference reflected light voltage is obtained, in which the distance image sensor irradiates the irradiation light, receives the incident light from the measurement space, and distributes and accumulates the charge from the photoelectric conversion element in each of the charge accumulation parts through the control of the pixel circuit, and obtains the voltage of the distance corresponding to the charge amount contained in the non-controlled charge and the control charge controlled and accumulated by the pixel circuit, solves the simultaneous equations using the second reference base voltage and the reference reflected light voltage, calculates only the reflected light control voltage corresponding to the charge amount of the control charge, and finds the distance to the object.

[0058] The distance image capturing device of the present invention further includes a lens, the lens being used to receive incident light from the space, and the distance image sensor receives the incident light via the lens.

[0059] The distance image capturing device of the present invention further includes a lens for receiving incident light from the space. Since the incident light is incident on the pixels respectively through the lens, the pixels whose adjustment ratio is within a predetermined differential range are divided into groups corresponding to the characteristics of the lens, and the middle value of the reference reference voltage in the group is used as the reference reference voltage for all pixels in the group.

[0060] In the range image capturing device of the present invention, since the incident light is incident on the pixels respectively via the lenses, a plurality of characteristics of the lenses are stored in the storage unit in correspondence with the characteristics of the lenses.

[0061] In the range image capturing device of the present invention, since the incident light is incident on the pixels respectively via the lens, an adjustment function is stored in the storage unit corresponding to the characteristics of the lens, and the adjustment function outputs the adjustment ratio corresponding to the position of each pixel.

[0062] The distance image capturing method of the present invention includes: a distance image generating step, in which a distance image sensor irradiates a measurement space serving as a measurement object space from a light source portion, receives light including reflected light from an object in the measurement space as incident light, accumulates charge generated by the incident light in each pixel, and generates a distance image having a charge amount of the charge accumulated in each pixel; and a distance image processing step, in which the distance to the object in the space is obtained by correcting the charge amount, wherein the corrected charge amount removes non-control charge included in the charge amount that is independent of control for accumulating the charge in the distance image sensor, from the charge amount in the distance image.

[0063] (Effects of the Invention)

[0064] The present invention can provide a distance image capturing device and a distance image capturing method, which can calculate the distance between an object and itself with the same accuracy as when the pixel area is not reduced, without being affected by each non-control charge included in each charge accumulated in the charge storage unit, the charge amount of which varies depending on the incident angle of the incident light. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 This is a block diagram showing a schematic configuration of a range image capturing device according to a first embodiment of the present invention.

[0066] Figure 2 This is a block diagram showing a schematic configuration of an imaging element (range image sensor 32 ) used in the range image capturing device 1 according to the first embodiment of the present invention.

[0067] Figure 3 This is a circuit diagram showing an example of the configuration of a pixel 321 arranged in the light-receiving pixel section 320 of the imaging element (range image sensor 32 ) used in the range image capturing device 1 according to the first embodiment of the present invention.

[0068] Figure 4 This is a timing chart showing the driving timing of the pixels 321 arranged in the light-receiving pixel section 320 of the imaging element (range image sensor 32 ) used in the range image capturing device 1 according to the first embodiment of the present invention.

[0069] Figure 5 This is a timing chart illustrating the process of acquiring each of the reference background light voltage and the reference standard voltage in the first embodiment of the present invention.

[0070] Figure 6 It is a timing chart explaining the period of non-controlled charge accumulation.

[0071] Figure 73 is a diagram showing a change tendency of the incident angle of incident light entering each pixel in the characteristics of the lens 31 .

[0072] Figure 8 This is a timing chart illustrating the process of acquiring each of the reference background light voltage and the reference standard voltage in the second embodiment of the present invention.

[0073] Figure 9 It is a timing chart illustrating the process of acquiring each of the reference background light voltage and the reference standard voltage in the third embodiment of the present invention.

[0074] Figure 10 This is a timing chart illustrating the process of acquiring each of the reference background light voltage and the reference standard voltage in the fourth embodiment of the present invention.

[0075] Figure 11 A conceptual diagram illustrating the incident angle of light relative to each pixel in a range image sensor.

[0076] Figure 12 1 and 2 are diagrams explaining the generation of non-control charges QB1 , QB2 , and QB3 included in the charges Q1 , Q2 , and Q3 , respectively, depending on the angle of incidence of incident light on a pixel.

[0077] Figure 13 This diagram shows that the charge accumulated in the charge accumulation unit changes depending on the angle of incidence of incident light with respect to the sensor chip.

[0078] Figure 14 1 is a conceptual diagram showing the relationship between the non-control charges QB1 , QB2 , and QB3 in each of the charges Q1 , Q2 , and Q3 .

[0079] Figure 15 It is a conceptual diagram showing the relationship between the charge amounts QCB1, QCB2, QCB3, the charge amounts QFB1, QFB2, QFB3, the charge amounts QCL2, QCL3, and the charge amounts QFL1, QFL2, QFL3 in each of the charge amounts Q1, Q2, Q3 stored in the charge storage units CS1, CS2, and CS3.

[0080] Figure 16 It is a conceptual diagram showing the relationship between the charge amounts QCB1, QCB2, QCB3, QCB4 and the charge amounts QFB1, QFB2, QFB3, QFB4 and the charge amounts QCL2, QCL3 and the charge amounts QFL1, QFL2, QFL3, QFL4 in each of the charge storage units CS1, CS2, CS3 and CS4. DETAILED DESCRIPTION

[0081] <First embodiment>

[0082] Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. Figure 1 1 is a block diagram showing a schematic configuration of a distance image capturing device according to a first embodiment of the present invention. Figure 1 1 and 2 also show the subject S whose distance is measured by the distance image capturing device 1 .

[0083] Figure 1 The distance image capturing device 1 having the structure shown includes a light source unit 2 , a light receiving unit 3 , and a distance image processing unit 4 .

[0084] Under the control of the distance image processing unit 4, the light source unit 2 irradiates the space (measurement space P) of the distance image capturing device 1 containing the object S whose distance is to be measured with intermittent light pulses PO at a predetermined period. The light source unit 2 is, for example, a surface-emitting semiconductor laser module such as a vertical cavity surface emitting laser (VCSEL).

[0085] The light source device 21 is, for example, a light source that emits laser light in the near-infrared wavelength range (e.g., a wavelength range of 850 nm to 940 nm) as light pulses PO directed toward the subject S. The light source device 21 is, for example, a semiconductor laser light emitting element. The light source device 21 emits light pulses as pulsed laser light in response to control by the timing control unit 41.

[0086] The diffuser plate 22 is an optical component that diffuses the near-infrared laser light emitted by the light source device 21 to a predetermined cross-sectional area and irradiates the light toward the measurement space P where the subject S is located. The pulsed laser light diffused by the diffuser plate 22 is emitted from the light source unit 2 as light pulses PO and irradiates the subject S in the measurement space P.

[0087] The light receiving unit 3 receives reflected light RL of the light pulse PO reflected by the subject S whose distance is to be measured in the distance image capturing device 1 , and outputs a pixel signal corresponding to the received reflected light RL.

[0088] The lens 31 is an optical lens that guides the incident reflected light RL toward the distance image sensor 32. The lens 31 emits the incident reflected light RL toward the distance image sensor 32 so that the light is received (entered) by pixels included in a light receiving area of the distance image sensor 32.

[0089] The distance image sensor 32 is an imaging element used in the distance image capturing device 1. It is a distributed-structure imaging element with multiple pixels in a two-dimensional light-receiving area. Each pixel comprises a photoelectric conversion element, multiple charge storage units corresponding to the photoelectric conversion element, and a structure for distributing charge to each charge storage unit. In response to control by the timing control unit 41, the distance image sensor 32 distributes the charge generated by the photoelectric conversion elements constituting the pixel to each charge storage unit and outputs a pixel signal corresponding to the amount of charge distributed to each charge storage unit.

[0090] In addition, in the distance image sensor 32 , a plurality of pixels are arranged in a two-dimensional grid (matrix) form, and a pixel signal corresponding to one frame of each pixel is output.

[0091] The distance image processing unit 4 is a control unit that controls the entire distance image capturing apparatus 1 and is also a calculation unit that calculates the distance to the subject S to be measured in the distance image capturing apparatus 1. The distance image processing unit 4 includes a timing control unit 41, a distance calculation unit 42, and a correction parameter storage unit 43.

[0092] The timing control unit 41 controls the timing at which the light source unit 2 irradiates the subject S with the light pulse PO, the timing at which the distance image sensor 32 included in the light receiving unit 3 receives and accumulates the reflected light RL, and the like.

[0093] The distance calculation unit 42 outputs distance information obtained by calculating the distance between the distance image capturing device 1 and the subject S based on the pixel signal output from the distance image sensor 32. Furthermore, when calculating the distance information between the distance image capturing device 1 and the subject S, the distance calculation unit 42 uses a voltage component corresponding to the non-control charge included in the input voltage for distance calculation, obtained from the charge amount in the pixel signal, as an adjustment voltage, and subtracts it from the input voltage to obtain a corrected input voltage, and then performs distance calculation according to equation (1) (described in detail later).

[0094] The correction parameter storage unit 43 stores a correction parameter (described later in detail) for generating an adjustment voltage to be subtracted from the input voltage when the distance calculation unit 42 subtracts the voltage corresponding to the non-control charge from the input voltage.

[0095] According to such a structure, in the distance image shooting device 1, the light receiving unit 3 receives the reflected light RL after the light pulse PO is reflected by the subject S, wherein the light pulse PO is the light in the near-infrared band irradiated by the light source unit 2 to the subject S, and the distance image processing unit 4 calculates and outputs the distance information of the distance to the subject S by removing the voltage component corresponding to the non-control charge corresponding to the incident light.

[0096] In addition, although Figure 1, the distance image capturing device 1 is shown as having a configuration including the distance image processing unit 4 therein. However, the distance image processing unit 4 may be a component provided outside the distance image capturing device 1 .

[0097] Next, the configuration of the distance image sensor 32 used as an imaging element in the distance image capturing device 1 will be described. Figure 2 1 is a block diagram showing a schematic configuration of an imaging element (range image sensor 32) used in the range image capturing device 1 according to the first embodiment of the present invention. Figure 2 In FIG, the distance image sensor 32 includes: a light receiving pixel unit 320 having a plurality of pixels 321; a control circuit 322; a vertical scanning circuit 323; a horizontal scanning circuit 324; a pixel signal processing circuit 325; and a pixel driving circuit 326. Figure 2 The illustrated distance image sensor 32 shows an example of a light-receiving pixel section 320 in which a plurality of pixels 321 are arranged in a two-dimensional grid pattern of 8 rows and 8 columns.

[0098] The control circuit 322 controls the components of the distance image sensor 32, such as the vertical scanning circuit 323, the horizontal scanning circuit 324, the pixel signal processing circuit 325, and the pixel driving circuit 326. The control circuit 322 controls the operation of the components of the distance image sensor 32 in response to control from, for example, the distance image processing unit 4 (more specifically, the timing control unit 41) of the distance image capturing device 1. Alternatively, the control of the components of the distance image sensor 32 by the control circuit 322 may be directly performed by, for example, the distance image processing unit 4 (more specifically, the timing control unit 41). In this case, the distance image sensor 32 may not include the control circuit 322.

[0099] The pixel driving circuit 326 outputs an accumulation driving signal (accumulation driving signals TX1, TX2, TX3 described later), a reset signal (reset signals RST1, RST2, RST3 described later) and a reset driving signal (reset driving signal RSTD described later) according to the column units of the pixels 321 arranged in a grid pattern in the light-receiving pixel section 320, wherein the accumulation driving signal distributes and accumulates the charges generated by the photoelectric conversion elements (also referred to as photoelectric conversion elements PD or photodiodes PD described later) of the pixels 321 arranged in a grid pattern into a plurality of charge storage sections (charge storage sections CS1, CS2, CS3 described later) of the pixels 321.

[0100] The vertical scanning circuit 323 is a driving circuit that, in response to control by the control circuit 322, controls each pixel 321 arranged within the light-receiving pixel section 320 and outputs (reads) a signal (hereinafter referred to as a "voltage signal") corresponding to the amount of charge obtained by photoelectric conversion of incident light from each pixel 321 to the corresponding vertical signal line 327. The vertical scanning circuit 323 outputs a control signal (hereinafter referred to as a selection drive signal SEL1, SEL2, and SEL3) for driving (controlling) the pixels 321 and reading the control signal, for each row of pixels 321 arranged in a grid pattern within the light-receiving pixel section 320.

[0101] Thus, the voltage signal in the pixel 321 corresponding to the amount of charge allocated to each charge storage unit (charge storage units CS1, CS2, CS3 described later) is read out by each corresponding vertical signal line 327 according to each column of the light-receiving pixel unit 320 and output to the pixel signal processing circuit 325.

[0102] In the light-receiving pixel section 320, the pixel 321 receives the reflected light RL after the light pulse PO irradiated by the light source section 2 to the subject S is reflected by the subject S, and generates a charge corresponding to the amount of light of the received reflected light RL (the amount of light received). In each pixel 321, the pixel driving circuit 326 outputs an accumulation driving signal, so that the charge corresponding to the amount of light of the received reflected light RL (the amount of light received) is distributed and accumulated in one of the multiple charge accumulation sections. Then, in the pixel 321, the vertical scanning circuit 323 outputs a selection driving signal as a readout driving signal, thereby outputting a voltage signal of a size corresponding to the amount of charge distributed and accumulated in each charge accumulation section to the corresponding vertical signal line 327. In addition, a detailed description of the structure and driving (control) method of the pixel 321 will be given later.

[0103] The pixel signal processing circuit 325 is a signal processing circuit that, in response to control by the vertical scanning circuit 323, performs predetermined signal processing on the voltage signal output from the pixels 321 in each column to the corresponding vertical signal line 327. Predetermined signal processing includes, for example, noise suppression processing that suppresses noise contained in the voltage signal using correlated double sampling (CDS).

[0104] Alternatively, the pixel signal processing circuit 325 may be a pixel signal processing circuit group composed of a plurality of pixel signal processing circuits corresponding to each column of the light-receiving pixel unit 320. In this case, the pixel signal processing circuit 325 responds to control by the control circuit 322 and outputs a voltage signal after predetermined signal processing to an internal AD conversion circuit. The AD conversion circuit responds to control by the horizontal scanning circuit 324 and outputs a digital value after AD conversion to a horizontal signal line 329 for each row of the light-receiving pixel unit 320.

[0105] The vertical scanning circuit 323 sequentially outputs a readout driving signal for outputting a voltage signal corresponding to the pixels 321 of each column to the pixel signal processing circuit 325 .

[0106] The horizontal scanning circuit 324, in response to control by the control circuit 322, sequentially outputs (reads) the digital values obtained by A / D conversion of the signal-processed voltage signals to the horizontal signal lines 329. Consequently, the signal-processed voltage signals for one frame output by the pixel signal processing circuit 325 are sequentially output as pixel signals for one frame to the outside of the distance image sensor 32 via the horizontal signal lines 329. At this point, the distance image sensor 32 outputs the signal-processed voltage signals as pixel signals from an output circuit (not shown), such as an output amplifier, to the outside of the distance image sensor 32.

[0107] The following description will describe a situation in which the pixel signal processing circuit 325 of the distance image sensor 32 performs noise suppression processing on the voltage signal output from the pixel 321, and then performs A / D conversion processing in the AD conversion circuit and outputs it, that is, a situation in which the voltage signal converted into a digital value is output from the horizontal signal line 329.

[0108] Next, the structure of the pixel 321 arranged in the light-receiving pixel section 320 included in the distance image sensor 32 will be described. Figure 3 1 is a circuit diagram showing an example of the structure of a pixel 321 arranged in the light receiving pixel section 320 of the imaging element (range image sensor 32) used in the range image capturing device 1 according to the embodiment of the present invention. Figure 3 2 shows an example of the structure of one pixel 321 among the plurality of pixels 321 arranged in the light-receiving pixel section 320. As an example of the structure, the pixel 321 includes three pixel signal readout sections.

[0109] Pixel 321 includes a photoelectric conversion element PD; a drain-gate transistor GD; and three pixel signal readout units RU that output voltage signals from corresponding output terminals O. Each pixel signal readout unit RU includes a readout gate transistor G; a floating diffusion FD; a charge storage capacitor C; a reset gate transistor RT; a source-follower gate transistor SF; and a select gate transistor SL. In each pixel signal readout unit RU, a charge storage unit CS is formed by the floating diffusion FD and the charge storage capacitor C. The drain-gate transistor GD, the readout gate transistor G, the reset gate transistor RT, the source-follower gate transistor SF, and the select gate transistor SL are N-channel MOS transistors.

[0110] In addition, Figure 3In the figure, each pixel signal readout unit RU is distinguished by adding a number "1", "2" or "3" after the label "RU" of the three pixel signal readout units RU. In addition, similarly, each structural unit of the three pixel signal readout units RU is distinguished by showing a number representing each pixel signal readout unit RU after the label to represent the pixel signal readout unit RU corresponding to each structural unit. Figure 3 In the illustrated pixel 321, the pixel signal readout unit RU1, which outputs a voltage signal from an output terminal O1, includes a gate transistor G1, a floating diffusion FD1, a charge storage capacitor C1, a reset gate transistor RT1, a source follower gate transistor SF1, and a select gate transistor SL1. Within the pixel signal readout unit RU1, the charge storage unit CS1 is comprised of the floating diffusion FD1 and the charge storage capacitor C1. The pixel signal readout units RU2 and RU3 have similar structures.

[0111] The photoelectric conversion element PD is an embedded photodiode that generates charge by photoelectric conversion of incident light and accumulates the generated charge. Furthermore, in the present invention, there is no particular limitation on the structure of the photoelectric conversion element PD included in the pixel 321. Therefore, the photoelectric conversion element PD may be, for example, a PN photodiode having a structure in which a P-type semiconductor and an N-type semiconductor are bonded, or a PIN photodiode having a structure in which an I-type semiconductor is sandwiched between the P-type semiconductor and the N-type semiconductor. Furthermore, the photoelectric conversion element included in the pixel 321 is not limited to a photodiode; for example, a photogate-type photoelectric conversion element may also be used.

[0112] The drain-gate transistor GD is a transistor for discarding the charge generated and accumulated by the photoelectric conversion element PD in response to the drive signal input from the vertical scanning circuit 323 but not transferred to each pixel signal readout unit RU. In other words, the drain-gate transistor GD is a transistor for resetting the charge generated by the photoelectric conversion element PD but not used for measuring the distance to the subject S.

[0113] The read gate transistor G is a transistor for transferring the charge generated and accumulated in the photoelectric conversion element PD in response to the drive signal input from the vertical scanning circuit 323 to the corresponding charge storage unit CS. The charge transferred by the read gate transistor G is held (accumulated) in the corresponding charge storage unit CS.

[0114] Here, in the pixel signal readout unit RU1, the source of the readout gate transistor G1 is connected to the second terminal of the photoelectric conversion element PD, the gate is connected to the signal line LTX1 that propagates and stores the drive signal TX1, and the drain is connected to the floating diffusion FD1 and the first terminal of the charge storage capacitor C1.

[0115] Similarly, in the pixel signal readout unit RU2, the source of the readout gate transistor G2 is connected to the second terminal of the photoelectric conversion element PD, the gate is connected to the signal line LTX2 that propagates and stores the drive signal TX2, and the drain is connected to the floating diffusion FD2 and the first terminal of the charge storage capacitor C2.

[0116] In addition, similarly, in the pixel signal readout unit RU3, the source of the readout gate transistor G3 is connected to the second terminal of the photoelectric conversion element PD, the gate is connected to the signal line LTX3 that propagates the storage drive signal TX3, and the drain is connected to the floating diffusion FD3 and the first terminal of the charge storage capacitor C3.

[0117] Each of the storage drive signals TX1 , TX2 , and TX3 is supplied from the pixel drive circuit 326 via the signal lines LTX1 , LTX2 , and LTX3 .

[0118] The charge storage capacitor C is a capacitor that holds (stores) the charge transferred by the corresponding read gate transistor G.

[0119] The reset gate transistor RT is a transistor for discarding the charge held in the corresponding charge storage unit CS in response to the drive signal input from the vertical scanning circuit 323. In other words, the reset gate transistor RT is a transistor for resetting the charge held in the corresponding charge storage unit CS.

[0120] The source follower gate transistor SF is a transistor for amplifying a voltage signal corresponding to the amount of charge accumulated in the charge storage unit CS connected to the gate terminal and outputting the amplified voltage signal to the corresponding select gate transistor SL.

[0121] The select gate transistor SL is a transistor for outputting, from the corresponding output terminal O, a voltage signal amplified by the corresponding source follower gate transistor SF in response to a drive signal input from the vertical scanning circuit 323 .

[0122] With the above configuration, in the pixel 321 , charges generated by the photoelectric conversion element PD converting incident light are distributed to each of the three charge storage units CS, and voltage signals corresponding to the amount of the distributed charges are output to the pixel signal processing circuit 325 .

[0123] The structure of the pixels arranged in the distance image sensor 32 is not limited to the following. Figure 3 While the illustrated configuration includes three pixel signal readout units RU, any pixel structure may be employed as long as the pixel includes a single photoelectric conversion element PD and multiple pixel signal readout units RU that distribute and accumulate the charge generated by the photoelectric conversion element PD. Specifically, the number of pixel signal readout units RU (charge accumulation units CS) included in a pixel disposed on the range image sensor 32 may be two or four or more.

[0124] In addition, Figure 3 In the pixel 321 of the structure shown, an example is shown in which a charge storage unit CS is composed of a floating diffusion FD and a charge storage capacitor C. However, the charge storage unit CS only needs to be composed of the floating diffusion FD. That is, the pixel 321 can also be a structure that does not have each charge storage capacitor C. In the case of this structure, there is an effect of improving the charge detection sensitivity (charge-voltage conversion gain CG). However, considering the expansion of the dynamic range in the distance measurement in the distance image capture device 1, a structure that can hold (accumulate) more charges is advantageous. Therefore, in the pixel 321, by adopting a structure in which the pixel signal readout unit RU has a charge storage capacitor C and the charge storage unit CS is composed of the floating diffusion FD and the charge storage capacitor C, more charges can be held (accumulated) than a structure in which the charge storage unit CS is composed of only the floating diffusion FD.

[0125] In addition, Figure 3 In the pixel 321 of the structure shown, an example of a structure having a drain-gate transistor GD is shown, but when there is no need to discard the charge accumulated (remaining) in the photoelectric conversion element PD, a structure without a drain-gate transistor GD can also be configured in the pixel of the distance image sensor 32.

[0126] Next, a driving (control) method (timing) of the pixels 321 in the range image capturing device 1 will be described. Figure 4 : is a timing chart showing the driving timing of the pixels 321 arranged in the light receiving pixel section 320 of the imaging element (range image sensor 32) used in the range image capturing device 1 according to the embodiment of the present invention. Figure 4 3 shows the timing of the light pulse PO emitted by the light source section 2 to the subject S, together with the timing of the driving signal of the pixel 321 when the range image sensor 32 outputs the pixel signal of one frame.

[0127] First, the driving (control) of the pixels 321 during the charge accumulation period, during which the charge generated and accumulated by the photoelectric conversion element PD based on the amount of received light (the amount of received light) is distributed to each pixel signal readout unit RU, will be described. During the charge accumulation period, the light source unit 2 irradiates the subject S with a light pulse PO. The pixels 321 are then driven in synchronization with the timing of the irradiation of the light pulse PO, distributing charge corresponding to the received background light and reflected light RL to each charge accumulation unit CS. The pixel drive circuit 326 simultaneously drives all pixels 321 within the light-receiving pixel unit 320, a so-called global shutter drive, thereby distributing and accumulating charge in each charge accumulation unit CS included in all pixels 321. Furthermore, the duration of the pulsed laser light emitted by the light source device 21, namely, the pulse width Tw of the light pulse PO, is a predetermined, very short time, for example, 10 nS. This is because the maximum measurable distance (hereinafter referred to as the "maximum measurable distance") in distance measurement using a pulse modulation method is determined by the pulse width Tw of the light pulse PO. When the pulse width Tw of the light pulse PO is 10 nS, the maximum measurable distance is 1.5 m. Simply increasing the pulse width Tw of the light pulse PO—that is, increasing the laser emission time in the light source device 21—allows the photoelectric conversion element PD to receive more reflected light RL. However, this reduces the resolution of the measured distance to the subject S. On the other hand, when the pulse width Tw of the light pulse PO is shorter, the amount of charge generated by the photoelectric conversion element PD through photoelectric conversion also decreases. Therefore, the distance image capturing device 1 performs multiple irradiations of the light pulse PO and charge distribution to ensure that a sufficient amount of charge is accumulated in each charge storage unit CS during the charge accumulation period.

[0128] Here, the configuration of each pixel 321 driven (controlled) by the vertical scanning circuit 323 and the pixel driving circuit 326 will be described. In the following description, the control circuit 322 outputs clock signals CK1, CK2, CK3, and CKRSTD, which generate the storage driving signals TX1, TX2, and TX3 and the reset driving signal RSTD, to the pixel driving circuit 326. Furthermore, the control circuit 322 outputs clock signals, which generate the selection driving signals SEL1, SEL2, and SEL3 and the reset signals RST1, RST2, and RST3, to the vertical scanning circuit 323.

[0129] exist Figure 4 The charge accumulation period of the timing chart shown in FIG. 1 shows the driving timing of the pixel 321 when irradiation of a plurality of light pulses PO and charge distribution in all pixels 321 are performed. Figure 4In the charge accumulation period of the timing chart shown, the light pulse PO is irradiated at the "H (high)" level (the light source device 21 emits laser light) and stops irradiating the light pulse PO at the "L (low)" level (the light source device 21 is turned off). Figure 4 The timing chart shown illustrates that all pixels 321 are reset, that is, starting from a state where charges have not yet been accumulated in the photoelectric conversion element PD and the charge accumulation portion CS.

[0130] exist Figure 4 The charge accumulation period of the timing chart shown shows the driving timing of the pixel 321 when irradiation of a plurality of light pulses PO and charge distribution in all pixels 321 are performed. Figure 4 In the signal level shown, the higher voltage value of the binary voltage pulse is the "H" level, and the lower voltage value is the "L" level.

[0131] In the following description, times tA1 to tA5 represent the charge distribution accumulation period, and multiple accumulation periods are repeated during the charge accumulation period. Furthermore, for example, the time width between times tA1, tA2, tA3, and tA4, i.e., the pulse widths of the light pulse PO and the accumulation drive signals TX1, TX2, and TX3, are all the same, Tw.

[0132] During the charge accumulation period, first, the pixel driving circuit 326 starts from the moment tA1 which is the same time as the pulse width Tw of the light pulse PO irradiated by the light source unit 2, and transfers and accumulates the charge corresponding to the background light before the light pulse PO is irradiated by the photoelectric conversion element PD to the charge accumulation unit CS1 via the readout gate transistor G1.

[0133] Thereafter, starting at time tA2, which coincides with the timing of the light pulse PO emitted by the light source unit 2, the pixel drive circuit 326 transfers the charge generated by the photoelectric conversion element PD in response to the currently photoelectrically converted light into the charge storage unit CS2 via the readout gate transistor G2. The charge stored in the charge storage unit CS2 corresponds to the reflected light RL reflected from the subject S during the pulse width Tw of the irradiated light pulse PO. This charge, in addition to the charge corresponding to the background light, also includes the charge corresponding to the reflected light RL that enters with a shorter delay, which is proportional to the distance to the subject S (absolute distance). More specifically, for example, when the subject S is relatively close, the irradiated light pulse PO is reflected from the subject S in a shorter time and returns as reflected light RL. Therefore, the charge storage unit CS2 contains a larger amount of charge corresponding to the reflected light RL reflected from the relatively close subject S.

[0134] Thereafter, starting at time tA3, which coincides with the timing at which the light source unit 2 stops emitting the light pulse PO, the pixel drive circuit 326 causes the charge generated by the photoelectric conversion element PD in response to the currently photoelectrically converted light to be transferred to the charge storage unit CS3 via the readout gate transistor G3 and stored therein. The charge stored in the charge storage unit CS3 corresponds to the reflected light RL reflected from the subject S, which occurs outside the duration of the pulse width Tw of the irradiated light pulse PO. This charge, in addition to the charge corresponding to the background light, also includes the charge corresponding to the reflected light RL that enters with a greater delay, proportional to the distance to the subject S (absolute distance). More specifically, for example, when the subject S is located at a relatively distant position, the irradiated light pulse PO takes longer to be reflected from the subject S and return as reflected light RL. Therefore, the charge storage unit CS3 contains a greater amount of charge corresponding to the reflected light RL reflected from the more distant subject S.

[0135] Thereafter, starting at time tA4 after a time equal to the pulse width Tw of the light pulse PO emitted by the light source unit 2 has elapsed, the pixel driving circuit 326 discards the charge generated by the photoelectric conversion element PD in response to the current photoelectric conversion of the light, i.e., the charge not used for distance measurement to the subject S, via the drain-gate transistor GD. In other words, the photoelectric conversion element PD is reset.

[0136] Thereafter, the pixel driving circuit 326 cancels the reset of the photoelectric conversion element PD at time tA5, which is the same time as the pulse width Tw of the next light pulse PO emitted by the light source unit 2. Then, similarly to the timing starting from time tA1, the pixel driving circuit 326 transfers and accumulates the charge generated by the next photoelectric conversion of the photoelectric conversion element PD, i.e., the charge corresponding to the background light before the next light pulse PO is emitted, into the charge storage unit CS1 via the readout gate transistor G1.

[0137] Afterwards, the pixel drive circuit 326 repeats the same drive of the pixel 321 as from time tA1 to time tA5 (hereinafter referred to as "charge sharing drive"). Thus, during the charge accumulation period, an amount of charge corresponding to the amount of repeated charge sharing drive is accumulated and maintained in each charge storage unit CS included in all pixels 321. Furthermore, the maximum number of times the charge sharing drive is repeated during the charge accumulation period depends on the period in which the distance image sensor 32 outputs (acquires) one frame of pixel signals. More specifically, the quotient of the time taken to acquire one frame of pixel signals from the distance image sensor 32 minus the time after the pixel signal readout period is divided by the time the light source device 21 emits the pulsed laser light, i.e., the pulse cycle time To of the light pulse PO, is the number of times. Furthermore, in the distance image sensor 32, the more times the charge sharing drive is repeated, the more charge is accumulated (accumulated) in each charge storage unit CS, and the higher the sensitivity. Consequently, the resolution of the distance to the measured subject S can be improved in the distance image sensor 32.

[0138] Next, the driving (control) of the pixels 321 during the pixel signal readout period, in which, after the charge accumulation period ends, the pixels 321 arranged in the light-receiving pixel section 320 sequentially output, row by row, voltage signals corresponding to the amount of charge allocated to each charge accumulation section CS included in each pixel signal readout section RU. During the pixel signal readout period, a so-called rolling drive is used to drive the pixels 321 arranged in the light-receiving pixel section 320 row by row, thereby outputting, row by row, voltage signals corresponding to the amount of charge accumulated (accumulated) and held in the charge accumulation section CS included in the pixels 321 arranged in the corresponding row to the pixel signal processing circuit 325.

[0139] Furthermore, as described above, in the range image sensor 32, the pixel signal processing circuit 325 performs predetermined signal processing, such as noise suppression or A / D conversion, on the voltage signal output by each pixel 321. The correlated double sampling (CDS) processing performed by the pixel signal processing circuit 325 as noise suppression involves taking the difference between a voltage signal corresponding to the amount of charge stored (accumulated) and held in the charge storage unit CS (hereinafter referred to as the "range pixel voltage signal PS") and a voltage signal corresponding to the amount of charge in the reset state (reset state) of the charge storage unit CS (hereinafter referred to as the "reset voltage signal PR"). Therefore, during the pixel signal readout period, the voltage signals of the range pixel voltage signal PS and the reset voltage signal PR corresponding to each charge storage unit CS included in each pixel 321 are output to the pixel signal processing circuit 325 row by row.

[0140] exist Figure 4During the pixel signal readout period of the timing diagram shown, the driving timing of the pixel 321 when each voltage signal of the distance pixel voltage signal PS(i) and the reset voltage signal PR(i) is output from each pixel 321(i) arranged in the i-th row (1≤i≤y) of the light-receiving pixel section 320 is shown, when a plurality of pixels 321 are arranged in y rows (y is an integer greater than 1) in the horizontal direction (row direction) and x columns (x is an integer greater than 1) in the vertical direction (column direction). In addition, in Figure 4 In the timing chart shown, the charge storage units CS1 ( i ), CS2 ( i ), and CS3 ( i ) included in each pixel 321 ( i ) output the respective voltage signals in this order.

[0141] During the pixel signal readout period, first, from time tR1 to time tR2, the vertical scanning circuit 323 outputs the distance pixel voltage signal PS1(i) from the output terminal O1(i) via the vertical signal line to the pixel signal processing circuit 325. Consequently, the pixel signal processing circuit 325 temporarily holds the distance pixel voltage signal PS1(i) output from the pixel signal readout unit RU1(i) via the vertical signal line.

[0142] Thereafter, during the period from time tR3 to time tR4, the vertical scanning circuit 323 outputs the reset voltage signal PR1(i) from the output terminal O1(i) via the vertical signal line to the pixel signal processing circuit 325. Thus, the pixel signal processing circuit 325 calculates the difference between the temporarily held distance pixel voltage signal PS1(i) and the reset voltage signal PR1(i) output from the pixel signal readout unit RU1(i) via the vertical signal line, thereby suppressing noise included in the voltage signal corresponding to the amount of charge stored (accumulated) and held in the charge storage unit CS1(i).

[0143] Then, during the period from time tR4 to time tR7, similarly to the period from time tR1 to time tR4, the vertical scanning circuit 323 outputs the distance pixel voltage signal PS2(i) and the reset voltage signal PR2(i) from the output terminal O2(i) via the vertical signal line to the pixel signal processing circuit 325. Furthermore, during the period from time tR7 to time tR10, similarly to the period from time tR1 to time tR4, the vertical scanning circuit 323 outputs the distance pixel voltage signal PS3(i) and the reset voltage signal PR3(i) from the output terminal O3(i) via the vertical signal line to the pixel signal processing circuit 325.

[0144] Afterwards, the vertical scanning circuit 323 sequentially drives each pixel 321 of another row arranged in the light-receiving pixel portion 320 (for example, each pixel 321 arranged in the i+1th row) in the same manner as from time tR1 to time tR10 (hereinafter referred to as "pixel signal readout drive"), so that each voltage signal is output sequentially from all pixels 321 arranged in the light-receiving pixel portion 320.

[0145] By using such a driving (control) method (timing), the pixel driving circuit 326 distributes the charge generated and accumulated in the photoelectric conversion element PD to each pixel signal readout unit RU multiple times in each pixel 321 arranged in the light-receiving pixel section 320 .

[0146] Furthermore, the vertical scanning circuit 323 sequentially outputs voltage signals corresponding to the amount of charge accumulated (stored) in the charge storage unit CS included in the pixel signal readout unit RU to the pixel signal processing circuit 325 via the vertical signal lines.

[0147] Furthermore, the pixel signal processing circuit 325 performs A / D conversion processing on each noise-suppressed voltage signal for each row. The horizontal scanning circuit 324 then sequentially outputs the voltage signals (digitized voltage signals) for each row after A / D conversion by the pixel signal processing circuit 325 via horizontal signal lines in the order of the columns of the light-receiving pixel unit 320. As a result, the distance image sensor 32 outputs the pixel signals (pixel signals VQ1, VQ2, VQ3 corresponding to the respective charge amounts Q1, Q2, Q3 of each of the charge storage units CS1, CS2, and CS3) for one frame to the outside. Thus, in the distance image capture device 1, the pixel signals (pixel signals VQ1, VQ2, VQ3) for one frame of captured image are output to the distance calculation unit 42 in what is known as field order.

[0148] In addition, from Figure 4 As can be seen from the driving (control) timing of the pixel 321 shown, each pixel signal in one frame includes three voltage signals corresponding to the three pixel signal readout units RU (charge storage units CS) included in the corresponding pixel 321. The distance calculation unit 42 calculates the distance to the subject S for each pixel signal, that is, for each pixel 321, based on the pixel signal output from the distance image sensor 32 for one frame.

[0149] Here, the method for calculating the distance between the distance image capture device 1 and the subject S in the distance calculation unit 42 is described. Here, the amount of charge corresponding to the background light before the light pulse PO is irradiated, assigned to the charge storage unit CS1 of the pixel signal readout unit RU1, is referred to as charge Q1. Furthermore, the amount of charge corresponding to the background light and the reflected light RL incident with a shorter delay, assigned to the charge storage unit CS2 of the pixel signal readout unit RU2, is referred to as charge Q2. Furthermore, the amount of charge corresponding to the background light and the reflected light RL incident with a longer delay, assigned to the charge storage unit CS3 of the pixel signal readout unit RU3, is referred to as charge Q3. The distance calculation unit 42 calculates the distance L between each pixel 321 and the subject S using the previously described equation (1).

[0150] As described above, the distance image capturing device 1 obtains the distance L between itself and the subject S for each pixel 321 arranged in the light receiving pixel section 320 of the distance image sensor 32 .

[0151] Furthermore, as described above, the structure of the pixels arranged in a grid pattern in the distance image sensor 32 is not limited to the structure of the pixels arranged in a grid pattern. Figure 3 The structure shown as having three pixel signal readout units RU1, RU2, and RU3 can be any structure as long as the pixel 321 has one photoelectric conversion element PD and two or more pixel signal readout units RU that distribute the charge generated and accumulated by the photoelectric conversion element PD. In this case, even in a distance image sensor configured with pixels having a structure having different numbers of pixel signal readout units RU, the pixel driving (control) method (timing) can be controlled by the same method as the above. Figure 4 The driving (control) method (timing) for the pixels 321 in the distance image capture device 1 shown can be easily implemented using the same concept. More specifically, the charge distribution drive to the pixels is repeated at a cycle that maintains a phase relationship, ensuring that the phases of the drive signals input to the readout gate transistor G or drain gate transistor GD included in each pixel signal readout unit RU do not overlap. This allows charge corresponding to the corresponding light to be accumulated in the charge accumulation unit CS included in each pixel signal readout unit RU, similar to the distance image sensor 32. Subsequently, the pixel signal readout drive sequentially outputs voltage signals from all pixels, allowing the pixel signals for one frame to be output to the outside of the distance image sensor, similar to the distance image sensor 32. Consequently, the distance calculation unit 42 can similarly calculate the distance L between the distance image capture device 1 and the subject S for each pixel signal (per pixel) based on the pixel signals for one frame output from a distance image sensor configured with pixels having different numbers of pixel signal readout units RU.

[0152] In this embodiment, in order to remove the voltage component of the non-controlled charge, a correction parameter is pre-set in advance, and a voltage corresponding to the charge amount of the non-controlled charge is obtained in each of the charge storage units CS1, CS2 and CS3 through the following processing as a reference reference voltage, which is first written and stored in the correction parameter storage unit 43.

[0153] Likewise, a voltage corresponding to the amount of charge generated by the background light is acquired in each of the charge storage units CS1 , CS2 , and CS3 as a reference background light voltage, which is previously written and stored in the correction parameter storage unit 43 .

[0154] In this embodiment, the data of the reference base voltage corresponding to the non-controlled charge generated by the background light and the reference background light voltage corresponding to the background light charge generated by the background light are pre-acquired, for example, when the distance image shooting device 1 is shipped from the factory or started up, that is, before shooting the distance image, and are first written and stored in the correction parameter storage unit 43.

[0155] Figure 5 This is a timing diagram illustrating the acquisition process of each of the reference background light voltage and the reference standard voltage in the first embodiment of the present invention. For example, when the distance image capturing device 1 is activated, the distance calculation unit 42 causes the timing control unit 41 to execute the timing output process for each of the reference background light voltage acquisition frame and the reference standard voltage acquisition frame.

[0156] Figure 5 (A) shows the processing of respectively acquiring, in the reference background light voltage acquisition frame, a reference background light voltage VA1 corresponding to the amount of charge generated by the background light accumulated in the charge storage unit CS1, a reference background light voltage VA2 corresponding to the amount of charge generated by the background light accumulated in the charge storage unit CS2, and a reference background light voltage VA3 corresponding to the amount of charge generated by the background light accumulated in the charge storage unit CS3.

[0157] exist Figure 5 In (A), the distance image sensor 32 is connected to Figure 4 The same process is described for the frame period in FIG. 1 , and the storage drive signals TX1, TX2, and TX3 are supplied to the read gate transistors G1, G2, and G3 with pulse widths Tw1, Tw2, and Tw3, respectively. As a result, the charge collected by the photodiode PD by the incident light is distributed to the charge storage units CS1, CS2, and CS3, and background light charges are accumulated. Figure 4 Unlike the processing described above, the timing control unit 41 controls the light source device 21 not to radiate the light pulse PO. Therefore, the incident light input to the photodiode PD of each pixel 321 of the range image sensor 32, more strictly speaking, the incident light input to the silicon region of the pixel including the photodiode PD, is only the background light in the shooting environment.

[0158] Thus, during the accumulation period in the reference background light voltage acquisition frame, the charge accumulation units CS1 , CS2 , and CS3 in each pixel circuit accumulate charges generated by incident light at different angles in each pixel 321 due to the lens 31 .

[0159] Then, with Figure 4 The readout process is performed in the same manner as described above, and a reference background light voltage VA1 corresponding to the amount of background light charges generated solely by background light (i.e., reference background light charges) accumulated in charge storage unit CS1, a reference background light voltage VA2 corresponding to the amount of background light charges accumulated in charge storage unit CS2, and a reference background light voltage VA3 corresponding to the amount of background light charges accumulated in charge storage unit CS3 are output from pixel signal processing circuit 325 as pixel signals. Distance calculation unit 42 writes and stores each of the acquired reference background light voltages VA1, VA2, and VA3, along with identification information for identifying each pixel 321 in light-receiving pixel unit 320, into correction parameter storage unit 43. Alternatively, distance calculation unit 42 may write and store each of the acquired reference background light voltages VA1, VA2, and VA3 in the same order as the pixel signals are read out from pixel signal processing circuit 325 into correction parameter storage unit 43. Here, in the same pixel 321, the charges generated by photoelectric conversion of incident light input to the pixel's silicon region including the photodiode PD are collected by the photodiode PD and controlled and distributed by the readout gate transistors G1, G2, and G3 to be stored in the charge storage units CS1, CS2, and CS3. The difference in the reference background light voltages VA1, VA2, and VA3 is due to the difference in the non-controlled charges flowing into and stored in the charge storage units CS1, CS2, and CS3.

[0160] Figure 5 (B) shows the processing of respectively acquiring, in the reference reference voltage acquisition frame, a reference reference voltage VB1 corresponding to the amount of non-control charges generated by background light and flowing into and accumulated in the charge accumulation section CS1 without passing through the read gate transistor G1, a reference reference voltage VB2 corresponding to the amount of non-control charges flowing into and accumulated in the charge accumulation section CS2 without passing through the read gate transistor G2, and a reference reference voltage VB3 corresponding to the amount of non-control charges flowing into and accumulated in the charge accumulation section CS3 without passing through the read gate transistor G3.

[0161] exist Figure 5 In (B), the distance image sensor 32 is connected to Figure 4 The same process as the frame period in FIG is used to store and read out the charge. Figure 5In the same manner as in the case of (A), the timing control unit 41 controls the light source device 21 not to radiate the light pulse PO, and Figure 5 Different from the case (A), the control circuit 322 is controlled not to output each of the storage drive signals TX1, TX2 and TX3, and each of the charge storage units CS1, CS2 and CS3 does not distribute the charge (ie, the control charge stored in the charge storage unit).

[0162] Furthermore, the timing control unit 41 controls the control circuit 322 to output a reset drive signal RSTD that turns on the drain gate transistor GD at the timing of distributing charge to each of the charge storage units CS1, CS2, and CS3, thereby discarding the control charge that is collected by the photodiode PD and distributed by the read gate transistors G1, G2, and G3 and stored in each charge storage unit. However, the period of distribution by the read gate transistors G1, G2, and G3 is a very short time of approximately 30 ns. Since the amount of charge collected by the photodiode PD is small, it can be retained by the photodiode PD in most cases. Therefore, it is also possible to adopt the same method as the above. Figure 5 (A) Similarly, the drain gate transistor GD is turned off during the period allocated by each read gate transistor G. In this case, when the drain gate transistor GD is turned on immediately after the allocation period, the charge held by the photodiode PD is discarded to the drain.

[0163] Therefore, each of the read gate transistors G1, G2, and G3 remains off, and the control charge caused by the background light is not accumulated in each of the charge storage units CS1, CS2, and CS3. In other words, since the control charge caused by the background light does not propagate through each of the read gate transistors G1, G2, and G3, only the non-control charges QB1, QB2, and QB3 that flow in without passing through each of the read gate transistors G1, G2, and G3 are accumulated in each of the charge storage units CS1, CS2, and CS3.

[0164] Then, in Figure 5 (B) Nakadashi and Figure 5(A) Similarly, readout processing is performed, and the pixel signal processing circuit 325 outputs a reference voltage VB1 corresponding to the charge amount of non-control charges QB1 accumulated in the charge storage unit CS1, a reference voltage VB2 corresponding to the charge amount of non-control charges QB2 accumulated in the charge storage unit CS2, and a reference voltage VB3 corresponding to the charge amount of non-control charges accumulated in the charge storage unit CS3 as pixel signals. The distance calculation unit 42 writes and stores each of the acquired reference voltages VB1, VB2, and VB3, along with identification information for identifying each pixel 321 in the light-receiving pixel unit 320, into the correction parameter storage unit 43. Alternatively, the distance calculation unit 42 may write and store each of the acquired reference voltages VB1, VB2, and VB3 in the same order as the pixel signals are read out from the pixel signal processing circuit 325 into the correction parameter storage unit 43.

[0165] Then, in the frame period for measuring the distance between the distance image sensor 32 and the subject S, as already described, the pixel signal processing circuit 325 outputs the pixel signals VQ1, VQ2, and VQ3 corresponding to each of the charge storage units CS1, CS2, and CS3 to the distance calculation unit 42 for each pixel 321 in one frame.

[0166] At this time, the distance calculation unit 42 corresponds to each pixel signal VQ1, VQ2 and VQ3 of each pixel 321 supplied from the pixel signal processing circuit 325, and reads out each of the reference background light voltages VA1, VA2 and VA3 and the reference reference voltages VB1, VB2 and VB3 corresponding to the pixel 321 from the correction parameter storage unit 43 in sequence.

[0167] Then, the distance calculation unit 42 divides the pixel signal VQ1 by the reference background light voltage VA1 (VQ1 / VA1) to calculate the adjustment ratio β.

[0168] This adjustment ratio β represents the ratio of the background light intensity at the time the correction parameters are acquired to the time the distance calculation is performed. It is the same value for both VQ2' / VA2 and VQ3' / VA3 in the linear region (non-saturated region) of signal intensity. Here, the corrected pixel signal VQ2' is a voltage allocated to the charge storage unit CS2 for the background light and the reflected light RL incident with a relatively short delay, excluding the charge component corresponding to the reflected light RL. Similarly, the corrected pixel signal VQ3' is a voltage allocated to the charge storage unit CS3 for the background light and the reflected light RL incident with a relatively long delay, excluding the charge component corresponding to the reflected light RL.

[0169] The distance calculation unit 42 uses the calculated adjustment ratio β and each of the reference voltages VB1, VB2, and VB3 to calculate the adjustment voltages VP1, VP2, and VP3 corresponding to the pixel signals VQ1, VQ2, and VQ3, respectively, according to the following equations (2), (3), and (4). Here, each of the adjustment voltages VP1, VP2, and VP3 is a voltage generated during distance measurement and is generated by the non-control charge accumulated in the charge storage units CS1, CS2, and CS3, respectively.

[0170] VP1=β×VB1=(VQ1 / VA1)×VB1 (2)

[0171] VP2=β×VB2=(VQ1 / VA1)×VB2 (3)

[0172] VP3=β×VB3=(VQ1 / VA1)×VB3 (4)

[0173] Through the above-mentioned equations (2), (3) and (4), it is possible to obtain each adjustment voltage VP1, VP2, VP3 based on each of the reference voltages VB1, VB2 and VB3 corresponding to the adjustment ratio β, that is, corresponding to the intensity of the background light at the time point of calculating the distance L.

[0174] Then, the distance calculation unit 42 uses each of the obtained adjustment voltages VP1, VP2, and VP3 and calculates corrected pixel signals VQ1', VQ2', and VQ3' by removing the voltage component generated during distance measurement due to the accumulation of non-controlled charges in the charge storage unit CS from each of the pixel signals VQ1, VQ2, and VQ3 using the following equations (5), (6), and (7).

[0175] VQ1'=VQ1-VP1 (5)

[0176] VQ2'=VQ2-VP2 (6)

[0177] VQ3'=VQ3-VP3 (7)

[0178] The distance calculation unit 42 uses each of the obtained corrected pixel signals VQ1 ′, VQ2 ′, and VQ3 ′ to calculate the distance L between the range image sensor 32 and the subject S using the following equation (8) corresponding to the already described equation (1).

[0179] L=[(VQ3'-VQ1') / (VQ2'+VQ3'-2VQ1')]×Dm (8)

[0180] In the above formula (8), Dm is (c / 2)Tw.

[0181] As described above, the distance image capturing device 1 calculates the distance L between the distance image sensor 32 and the subject S for each pixel 321 arranged in the light-receiving pixel portion 320 of the distance image sensor 32. According to this embodiment, by using the corrected pixel signals VQ1', VQ2', and VQ3' obtained by correcting the pixel signals VQ1, VQ2, and VQ3 using the adjustment voltages VP1, VP2, and VP3, the conventionally used equation (1) for calculating the distance can be directly used.

[0182] Figure 6 It is a timing chart explaining the period of non-controlled charge accumulation. Figure 6 is with Figure 4 The timing diagram also shows two frame periods, as shown in Figure 4 As described in

[15] , each frame period includes a charge accumulation period (Integration) during which charge is accumulated in the charge accumulation section CS by irradiating a light pulse PO, and a pixel signal readout period (Read) during which charge is read out from the charge accumulation section CS. Here, the control charge collected by the photodiode PD and controlled and distributed by the readout gate transistors G1, G2, and G3 is accumulated in each of the charge accumulation sections CS1, CS2, and CS3 only during the charge accumulation period (Integration). After the charge is read out, the charge accumulation section CS is temporarily discarded and reset by the reset gate transistor RT, but the charge is not discarded until the next charge readout period. In other words, charge discarding is performed only once per frame (Integration + Read), so non-control charge is accumulated in the charge accumulation section CS for approximately one frame. In other words, the period during which non-control charge is accumulated in the charge accumulation section CS is longer than the period during which control charge is accumulated in the charge accumulation section CS.

[0183] Therefore, in the conventional art, when the area of the pixel 321 is reduced, the ratio of the amount of non-control charge collected by the photodiode PD and accumulated in the charge accumulation unit CS without passing through the readout gate transistor G to the amount of control charge distributed through the readout gate transistor G and accumulated in the charge accumulation unit CS increases, and the accuracy of the calculated distance L decreases. Furthermore, as the area of the pixel 321 decreases, the accumulation ratio of the non-control charge gradually increases, and the accuracy of the calculated distance L gradually decreases.

[0184] However, according to this embodiment, regardless of whether the ratio of the amount of non-control charges accumulated in the charge storage unit CS without passing through the read gate transistor G to the amount of control charges passed through the read gate transistor G increases, or the area of the pixel 321 is reduced, the influence of each non-control charge contained in each charge accumulated in the charge storage unit CS, the charge amount of which varies depending on the incident angle of the incident light, can be eliminated, and the distance L between the subject S and itself can be calculated with similar or higher accuracy.

[0185] In addition, according to this embodiment, after the lens 31 is installed and the distance image capturing device 1 is assembled, the reference background light voltages VA1, VA2, and VA3 and the reference base voltages VB1, VB2, and VB3 can be measured without using special calibration equipment, and each of the pixel signals VQ1, VQ2, and VQ3 can be easily calibrated in accordance with the various characteristics of the installed lens 31, the relative position difference between the installed lens 31 and the distance image sensor 32, etc.

[0186] Furthermore, in this embodiment, when the lens 31 has multiple F-numbers or other characteristics that vary the angle of incidence of incident light on the distance image sensor 32, reference background light voltages VA1, VA2, and VA3 and reference reference voltages VB1, VB2, and VB3 are calculated for each characteristic, and these voltages are pre-assigned to each characteristic and stored in the correction parameter storage unit 43. Correction is then performed on each pixel signal VQ1, VQ2, and VQ3 corresponding to the characteristic. In this case, the distance calculation unit 42 reads correction parameters such as the reference background light voltages VA1, VA2, and VA3 and the reference reference voltages VB1, VB2, and VB3 corresponding to the characteristics of the lens used during the charge accumulation period during the frame cycle from the correction parameter storage unit 43, and corrects each pixel signal VQ1, VQ2, and VQ3 using the read correction parameters.

[0187] In addition, in this embodiment, the reference background light voltages VA1, VA2 and VA3 and the reference reference voltages VB1, VB2 and VB3 corresponding to each of all the pixels 321 in the light-receiving pixel section 320 are stored in the correction parameter storage section 43. However, since the incident light is incident on each of the pixels 321 via the lens 31 provided in front of the light-receiving pixel section 320 relative to the subject S, it is also possible to configure each pixel 321 to be grouped according to the characteristics of the lens 31 so that the adjustment ratio β falls within a predetermined differential range, and the median of each reference background light voltage VA1, VA2 and VA3 and each reference reference voltage VB1, VB2 and VB3 in the pixels 321 in each group is used as the reference background light voltage VA1, VA2 and VA3 and the reference reference voltage VB1, VB2 and VB3 for all the pixels 321 in the group, so as to reduce the storage capacitance of the correction parameter storage section 43.

[0188] Figure 7 3 is a diagram showing a change tendency of the incident angle of incident light entering each pixel in the characteristics of the lens 31 .

[0189] like Figure 7As shown, since the incident angle of the incident light changes continuously and gradually in a concentric circle (concentric circle E) with the area center CO of the chip of the distance image sensor 32 as the center, it can be constructed to group the predetermined pixels 321 with the same incident angle as described above.

[0190] Alternatively, as another configuration, correction parameter values for the reference background light voltages VA1, VA2, and VA3 and the reference base voltages VB1, VB2, and VB3 for pixels 321 at positions of a predetermined radius within a concentric circle E centered on the area center CO may be stored in the correction parameter storage unit 43. In this case, the correction parameter values for the concentric circles E of the predetermined radius are precomputed using a compensating function for the correction parameter values of each pixel 321 in the area enclosed by the concentric circles E, and the function is written and stored in the correction parameter storage unit 43 for each of the aforementioned areas. Then, as needed, the distance calculation unit 42 is configured to read the correction parameter values for the concentric circles E and the compensating function, and to compensating the correction parameter values for the pixels 321 in the area enclosed by the concentric circles E using the compensating function to generate correction parameter values for the pixels 321 corresponding to the concentric circles E enclosing the area.

[0191] In this embodiment, an example is described in which the incident angle of the incident light changes continuously and gradually in a concentric circle shape (concentric circle E) in the characteristics of the lens 31. However, depending on the characteristics of the lens used, it is not necessary to change in a concentric circle shape, and it can also change according to the characteristics of the incident angle of the incident light held by the lens.

[0192] Furthermore, in this embodiment, an example is shown in which the circuits in the distance image sensor 32 are formed using n-channel transistors. However, the polarity of the semiconductor may be reversed and the circuits may be formed using p-channel transistors.

[0193] <Second embodiment>

[0194] According to the second embodiment of the range image capturing device, the Figure 1 The structure is the same as the first embodiment shown, but compared with the first embodiment in which the pixel 321 has three pixel signal readout units RU1, RU2 and RU3, this embodiment has four pixel signal readout units RU1, RU2, RU3 and RU4 (not shown).

[0195] Each of the pixel signal readout units RU1 , RU2 , RU3 , and RU4 includes a charge storage unit CS1 , CS2 , CS3 , and CS4 (not shown), respectively.

[0196] In the configuration of this embodiment, similar to the first embodiment, the background light charge collected by the photodiode PD due to incident background light alone is distributed and accumulated in the charge accumulation unit CS1 during a distribution time Tw1 (which is the same duration as the pulse width Tw, and the same applies to Tw2, Tw3, and Tw4 described below). Distribution times Tw1, Tw2, Tw3, and Tw4 (not shown) are the pulse widths of the accumulation drive signal TX1 applied to the readout gate transistor G1, the accumulation drive signal TX2 applied to the readout gate transistor G2, the accumulation drive signal TX3 applied to the readout gate transistor G3, and the accumulation drive signal TX4 (not shown) applied to the readout gate transistor G4 (not shown), respectively. Tw1, Tw2, Tw3, and Tw4 are the same as the pulse width Tw of the light pulse PO.

[0197] and Figure 4 Similarly, in the allocation time Tw1 before the light pulse PO is irradiated, the background light charge collected by the photodiode PD is allocated to the charge storage unit CS1. Figure 4 Similarly, in the allocated time Tw2 of the light pulse PO, the next allocated time Tw3, and the next allocated time Tw4, the background light charge collected by the photodiode PD and the charge corresponding to the reflected light RL are allocated.

[0198] In this embodiment, similar to the first embodiment, the data of each of the reference base voltages VB1, VB2, VB3 and VB4 corresponding to the non-control charges generated by the background light and the data of each of the reference background light voltages VA1, VA2, VA3 and VA4 corresponding to the background light charges generated by the background light are acquired in advance, for example, when the distance image shooting device 1 is shipped from the factory or when it is started, that is, before shooting the distance image, and are first written and stored in the correction parameter storage unit 43.

[0199] Figure 8 This is a timing diagram illustrating the acquisition process of the reference background light voltage and the reference standard voltage in the second embodiment of the present invention. For example, when the distance image capturing device 1 is activated, the distance calculation unit 42 causes the timing control unit 41 to execute the acquisition process of the reference background light voltage in the reference background light voltage acquisition frame and the acquisition process of the reference standard voltage in the reference standard voltage acquisition frame.

[0200] Figure 8 (A) shows the Figure 5 (A) Similarly, in the reference background light voltage acquisition frame, a process of acquiring the reference background light voltage VA4 accumulated in the charge accumulation unit CS4 together with the reference background light voltages VA1 , VA2 , and VA3 .

[0201] exist Figure 8 In (A), the distance image sensor 32 accumulates and reads charge from the charge storage unit CS. During this charge accumulation, the timing control unit 41 controls the light source device 21 to prevent the emission of light pulses PO. Therefore, the incident light entering each pixel 321 of the distance image sensor 32 is solely the background light in the image capture environment.

[0202] Thus, during the charge accumulation period in the reference background light voltage acquisition frame, only background light is incident on the charge accumulation units CS1 , CS2 , CS3 , and CS4 in each pixel 321 , with the incident angle changed to different angles by the lens 31 .

[0203] Then, with Figure 5 The readout process is performed in the same manner as described in (A), and a reference background light voltage VA1 corresponding to the amount of background light charges generated solely by background light (i.e., reference background light charges) accumulated in charge storage unit CS1, a reference background light voltage VA2 corresponding to the amount of background light charges accumulated in charge storage unit CS2, a reference background light voltage VA3 corresponding to the amount of background light charges accumulated in charge storage unit CS3, and a reference background light voltage VA4 corresponding to the amount of background light charges accumulated in charge storage unit CS4 are output as pixel signals from pixel signal processing circuit 325. Distance calculation unit 42 writes and stores each of the acquired reference background light voltages VA1, VA2, VA3, and VA4 into correction parameter storage unit 43, along with identification information for identifying each pixel 321 in light-receiving pixel unit 320. Alternatively, the distance calculation unit 42 may write and store the reference background light voltages VA1 , VA2 , VA3 , and VA4 sequentially acquired in the same order as the pixel signal reading out from the pixel signal processing circuit 325 in the correction parameter storage unit 43 .

[0204] Figure 8 (B) shows the Figure 5 (B) Similarly, in the reference voltage acquisition frame, a process is performed to acquire each of the reference voltages VB1, VB2, and VB3 and a reference voltage VB4 corresponding to the amount of non-control charges flowing with background light and accumulated in the charge accumulation unit CS4.

[0205] exist Figure 8 In (B), the distance image sensor 32 is connected to Figure 4The same process as described for the frame period in FIG is used to accumulate and read out charges. Meanwhile, the timing control unit 41 controls the light source device 21 so as not to emit the light pulse PO, and controls the control circuit 322 so as not to output each of the accumulation drive signals TX1, TX2, TX3, and TX4, thereby preventing each of the charge accumulation units CS1, CS2, CS3, and CS4 from distributing charges.

[0206] Furthermore, the timing control unit 41 controls the control circuit 322 to output a reset drive signal RSTD to turn on the drain gate transistor GD at the timing of distributing charge to each of the charge storage units CS1, CS2, CS3, and CS4, thereby discarding the charge. However, since the period of distribution by each of the read gate transistors G1, G2, G3, and G4 is a very short time of approximately 40 ns, the charge collected by the photodiode PD is small and can be retained by the photodiode PD in most cases. Therefore, a similar method can also be used. Figure 8 (A) Similarly, in a configuration in which the drain-gate transistor GD is turned off during the allocation period, when the drain-gate transistor GD is turned on immediately after the allocation period, the charge held by the photodiode PD is discarded to the drain.

[0207] As a result, each of the read gate transistors G1, G2, G3, and G4 remains off, and charges caused by background light are not propagated to the charge storage units CS1, CS2, CS3, and CS4 via the read gate transistors G1, G2, G3, and G4. Consequently, each of the charge storage units CS1, CS2, CS3, and CS4 stores only the non-control charges QB1, QB2, QB3, and QB4 that flow in without passing through the read gate transistors G1, G2, G3, and G4.

[0208] Then, with Figure 8 (A) Similarly, readout processing is performed, and a reference voltage VB4 corresponding to the amount of non-control charges accumulated in the charge storage unit CS4 is output from the pixel signal processing circuit 325 as a pixel signal, along with the reference voltages VB1, VB2, and VB3. The distance calculation unit 42 writes and stores each of the acquired reference voltages VB1, VB2, VB3, and VB4, along with identification information for identifying each pixel 321 in the light-receiving pixel unit 320, into the correction parameter storage unit 43. Alternatively, the distance calculation unit 42 may write and store each of the acquired reference voltages VB1, VB2, VB3, and VB4 in the same order as the pixel signals are read out from the pixel signal processing circuit 325 into the correction parameter storage unit 43.

[0209] Then, in the frame period for measuring the distance between the distance image sensor 32 and the subject S, as already described, the pixel signal processing circuit 325 outputs the pixel signals VQ1, VQ2, VQ3, and VQ4 corresponding to each of the charge storage units CS1, CS2, CS3, and CS4 to the distance calculation unit 42 for each pixel 321 in one frame.

[0210] At this time, the distance calculation unit 42 sequentially reads the reference background light voltages VA1 , VA2 , VA3 , and VA4 and the reference base voltages VB1 , VB2 , VB3 , and VB4 corresponding to the read pixel 321 from the correction parameter storage unit 43 .

[0211] Distance calculation unit 42 divides pixel signal VQ1 by reference background light voltage VA1 (VQ1 / VA1) to calculate adjustment ratio β. This adjustment ratio β represents the ratio of the background light intensity at the time the correction parameters are acquired to the time the distance calculation is performed. VQ2' / VA2, VQ3' / VA3, and VQ4' / VA4 are the same value even in the linear region (non-saturated region) of signal intensity. Corrected pixel signal VQ2' is a voltage that excludes the charge component corresponding to the reflected light RL from the charge corresponding to the background light and reflected light RL distributed to charge storage unit CS2. Similarly, corrected pixel signal VQ3' is a voltage that excludes the charge component corresponding to the reflected light RL from the charge corresponding to the background light and reflected light RL distributed to charge storage unit CS3. Corrected pixel signal VQ4' is a voltage that excludes the charge component corresponding to the reflected light RL from the charge corresponding to the background light and reflected light RL distributed to charge storage unit CS4.

[0212] The distance calculation unit 42 uses the adjustment ratio β and the reference voltages VB1, VB2, VB3, and VB4 to calculate the adjustment voltages VP1, VP2, VP3, and VP4 generated when the distance measurement corresponding to each of the pixel signals VQ1, VQ2, VQ3, and VQ4 is performed due to the accumulation of non-control charges in the charge accumulation unit CS through the following equations (9), (10), (11), and (12).

[0213] VP1=β×VB1=(VQ1 / VA1)×VB1 (9)

[0214] VP2=β×VB2=(VQ1 / VA1)×VB2 (10)

[0215] VP3=β×VB3=(VQ1 / VA1)×VB3 (11)

[0216] VP4=β×VB4=(VQ1 / VA1)×VB4 (12)

[0217] Then, the distance calculation unit 42 uses each of the adjustment voltages VP1, VP2, VP3, and VP4 to calculate the corrected pixel signals VQ1', VQ2', VQ3', and VQ4' of the pixel signals VQ1, VQ2, VQ3, and VQ4 using the following equations (13), (14), (15), and (16).

[0218] VQ1'=VQ1-VP1 (13)

[0219] VQ2'=VQ2-VP2 (14)

[0220] VQ3'=VQ3-VP3 (15)

[0221] VQ4'=VQ4-VP4 (16)

[0222] As described above, even when there are four pixel signal readout units (RU1, RU2, RU3, and RU4), the corrected pixel signals VQ1', VQ2', VQ3', and VQ4' can be calculated in the same manner as when there are only three pixel signal readout units (RU1, RU2, and RU3). In other words, even when there are three or more pixel signal readout units, the voltage values of the pixel signals VQ1, VQ2, VQ3, and VQ4 can be accurately corrected using the above-described correction method.

[0223] <Third embodiment>

[0224] In the first and second embodiments, before measuring the distance between the subject S and the distance image sensor 32, the reference background light voltage and the reference base voltage are written and stored in the correction parameter storage unit 43 in advance, but in the third embodiment, a structure is adopted in which the correction parameters are obtained each time the distance measurement process is performed.

[0225] This embodiment adopts Figure 1 The distance image capturing device 1 shown has the same structure as that shown, and operations different from those of the first and second embodiments will be described below.

[0226] In this embodiment, two frames of each of the first frame and the second frame are used to obtain one distance image. Therefore, in the first and second embodiments, for example, distance measurements are performed 60 times per second, while in this embodiment, distance measurements are performed 30 times per second.

[0227] Figure 9 It is a timing chart illustrating the process of acquiring each of the reference background light voltage and the reference standard voltage in the third embodiment of the present invention. Figure 9 (A) shows the timing diagram of the first frame, Figure 9 (B) shows a timing chart of the second frame.

[0228] exist Figure 9 (A) Figure 4 In the same manner as described above, in the first frame, a light pulse PO is emitted, and reflected light RL from the subject S is received (delayed by a time Td from the emission timing of the light pulse PO), thereby capturing a distance image and acquiring pixel signals VQ1, VQ2, and VQ3. The distance calculation unit 42 then writes and stores each of the acquired pixel signals VQ1, VQ2, and VQ3 in the correction parameter storage unit 43.

[0229] Pixel signal VQ1 is a voltage corresponding to the amount of charge collected by photodiode PD due to incident light consisting solely of background light and allocated and accumulated in charge storage unit CS1. Furthermore, pixel signal VQ2 is a voltage corresponding to the amount of charge collected by photodiode PD due to incident light including a portion of background light and reflected light RL and allocated and accumulated in charge storage unit CS2. Pixel signal VQ3 is a voltage corresponding to the amount of charge collected by photodiode PD due to incident light including a portion of background light and reflected light RL and allocated and accumulated in charge storage unit CS3. Each of pixel signals VQ1, VQ2, and VQ3 includes a voltage component of non-control charge corresponding to the amount of charge corresponding to the incident light, namely, reference voltages VB1, VB2, and VB3.

[0230] exist Figure 9 (B) Figure 9 Similarly to the first frame of (A), in the second frame, the light pulse PO is radiated, and thus the reflected light RL from the subject S is received (delayed by a time Td from the radiating timing of the light pulse PO). Figure 5 (B) In the same process, the distance calculation unit 42 obtains the reference voltages VB1, VB2, and VB3 for each of the charge storage units CS1, CS2, and CS3. Figure 9 In the timing diagram (B), when the charge storage units CS1, CS2, and CS3 are each charged, a reset drive signal RSTD is output to turn on the drain gate transistor GD to discard the charge. However, the period of charge distribution by the read gate transistors G1, G2, and G3 is a very short time of about 30ns, so the charge collected by the photodiode PD is small and can be retained by the photodiode PD in most cases. Therefore, the same method can also be used. Figure 9 (A) Similarly, in a configuration in which the drain-gate transistor GD is turned off during the distribution period, when the drain-gate transistor GD is turned on immediately after the distribution period, the charge held by the photodiode PD is discarded to the drain.

[0231] Then, the distance calculation unit 42 reads the pixel signals VQ1 , VQ2 , and VQ3 from the correction parameter storage unit 43 .

[0232] In the distance calculation unit 42, each of the reference voltages VB1, VB2 and VB3 is used as the adjustment voltage VP1, VP2, VP3 of each pixel signal VQ1, VQ2, VQ3, and the corrected pixel signals VQ1', VQ2', VQ3' are calculated using the following equations (17), (18) and (19).

[0233] VQ1'=VQ1-VP1 (17)

[0234] VQ2'=VQ2-VP2 (18)

[0235] VQ3'=VQ3-VP3 (19)

[0236] Then, the distance calculation unit 42 calculates the distance L between the subject S and the range image sensor 32 in the range image capturing device 1 using the obtained corrected pixel signals VQ1 ′, VQ2 ′, and VQ3 ′ according to equation (8).

[0237] According to this embodiment, each time the distance calculation process is performed, the reference base voltages VB1, VB2, and VB3 are obtained as correction parameters. Therefore, there is no need to write and store the reference background light voltages VA1, VA2, VA3 and the reference base voltages VB1, VB2, VB3 corresponding to all pixels 321 in the correction parameter storage unit 43 in advance as in the first and second embodiments. Therefore, the capacitance of the correction parameter storage unit 43 can be reduced.

[0238] In addition, according to this embodiment, since a pixel signal for measurement is obtained in the first frame, and a pixel signal of a reference base voltage for correcting the pixel signal for measurement is obtained in the next second frame, it is applicable not only to the method for measuring the delay of a single light pulse shown in the present invention, but also to a CW (continuous wave) modulated TOF sensor for irradiating continuously modulated light and calculating the phase shift between the irradiated light and the reflected light for use in distance calculation, and can be applicable to various other TOF sensors.

[0239] For example, in a TOF sensor using CW (continuous wave) modulation, two frames are usually used for distance measurement. However, in order to perform the same correction as in this embodiment, two more frames for correction can be added, and four frames can be used to obtain a distance image.

[0240] <Fourth embodiment>

[0241] The first to third embodiments have a configuration in which voltage components corresponding to non-control charges generated by background light and flowing into the charge storage units CS1 , CS2 , and CS3 are removed from the pixel signals VQ1 , VQ2 , and VQ3 .

[0242] On the other hand, the fourth embodiment is a structure in which voltage components corresponding to non-control charges generated by reflected light RL after light pulse PO (pulse width Tw) is reflected by subject S and flows into each charge storage unit CS1, CS2 and CS3 are removed from each pixel signal VQ1, VQ2, and VQ3.

[0243] As described above, any of the first to third embodiments can correct the non-controlled charges caused by background light. In this embodiment, since correction is known to be possible in the fourth embodiment, the description will be made using a state where there is no background light, that is, a state where background light does not enter the image sensor 32 and the ambient light is blocked, such as in a dark room.

[0244] In the present embodiment, for example, when the distance image capturing device 1 is shipped from the factory or started up, that is, before capturing the distance image, a reference reflected light voltage VC1 corresponding to the charge amount of the reflected light charge C1 generated by the reflected light accumulated in the charge storage unit CS1, a reference reflected light voltage VC2 corresponding to the charge amount of the reflected light charge C2 generated by the reflected light accumulated in the charge storage unit CS2, and a reference reflected light voltage VC3 corresponding to the charge amount of the reflected light charge C3 generated by the reflected light accumulated in the charge storage unit CS3, as well as a reference reference voltage VD1 corresponding to the charge amount of the non-control charge D1 generated by the reflected light accumulated in the charge storage unit CS1, a reference reference voltage VD2 corresponding to the charge amount of the non-control charge D2 generated by the reflected light accumulated in the charge storage unit CS2, and a reference reference voltage VD3 corresponding to the charge amount of the non-control charge D3 generated by the reflected light accumulated in the charge storage unit CS3 are pre-acquired and first written into and stored in the correction parameter storage unit 43.

[0245] Figure 10 It is a timing chart illustrating the process of acquiring each of the reference reflected light voltage and the reference standard voltage in the fourth embodiment of the present invention.

[0246] Figure 10 (A) shows the processing of respectively obtaining, in the reference reflected light voltage acquisition frame, a reference reflected light voltage VC1 corresponding to the amount of charge generated by the incident light of only reflected light accumulated in the charge storage unit CS1, a reference reflected light voltage VC2 corresponding to the amount of charge generated by the incident light of only reflected light accumulated in the charge storage unit CS2, and a reference reflected light voltage VC3 corresponding to the amount of charge generated by the incident light of only reflected light accumulated in the charge storage unit CS3.

[0247] exist Figure 10 In (A), the distance image sensor 32 is connected to Figure 4 The operation description of the frame period in the same manner as the distance measurement by the radiation of the light pulse PO is performed, and the charge accumulation in each of the charge accumulation units CS1, CS2 and CS3 and the reading from each charge accumulation unit CS1, CS2, CS3 are performed.

[0248] Thus, during the accumulation period in the reference reflected light voltage acquisition frame, each of the charge storage units CS1, CS2 and CS3 in each pixel 321 will accumulate charges generated by the reflected light that only enters each pixel 321 at different incident angles through the lens 31 and becomes the incident light.

[0249] Then, with Figure 4 The readout processing is performed in the same manner as described above, and a reference reflected light voltage VC1 corresponding to the amount of reflected light charge (i.e., reference reflected light charge) resulting from only the reflected light accumulated in the charge storage unit CS1, a reference reflected light voltage VC2 corresponding to the amount of charge due to only the reflected light accumulated in the charge storage unit CS2, and a reference reflected light voltage VC3 corresponding to the amount of charge due to only the reflected light accumulated in the charge storage unit CS3 are output from the pixel signal processing circuit 325 as pixel signals. The distance calculation unit 42 writes and stores each of the acquired reference reflected light voltages VC1, VC2, and VC3 together with identification information for identifying each pixel 321 in the light-receiving pixel unit 320 into the correction parameter storage unit 43. Alternatively, the distance calculation unit 42 may write and store each of the acquired reference reflected light voltages VC1, VC2, and VC3 in the same order as the pixel signals are read out from the pixel signal processing circuit 325 into the correction parameter storage unit 43.

[0250] Figure 10 (B) shows the processing of respectively acquiring, in the reference reference voltage acquisition frame, a reference reference voltage VD1 corresponding to the charge amount of non-control charges flowing into and accumulated in the charge storage unit CS1 accompanied by reflected light, a reference reference voltage VD2 corresponding to the charge amount of non-control charges flowing into and accumulated in the charge storage unit CS2 accompanied by reflected light, and a reference reference voltage VD3 corresponding to the charge amount of non-control charges flowing into and accumulated in the charge storage unit CS3 accompanied by reflected light.

[0251] exist Figure 10 In (B), the distance image sensor 32 is connected to Figure 5The operation of acquiring a frame with reference to the reference voltage in (B) is similar to the operation described above, and charge accumulation and readout are performed. Specifically, the timing control unit 41 controls the light source device 21 to emit the light pulse PO, while controlling the control circuit 322 to not output the accumulation drive signals TX1, TX2, and TX3, and to prevent the charge accumulation units CS1, CS2, and CS3 from distributing charge.

[0252] Furthermore, the timing control unit 41 controls the control circuit 322 to output a reset drive signal RSTD that turns on the drain gate transistor GD at the timing of distributing the charge to each of the charge storage units CS1, CS2, and CS3, thereby discarding the charge generated by the reflected light. However, the period of distribution by each of the read gate transistors G1, G2, and G3 is a very short time of approximately 30 ns, so the charge collected by the photodiode PD is small and can be retained by the photodiode PD in most cases. Therefore, it is also possible to adopt the same method as the above. Figure 10 (A) Similarly, the drain-gate transistor GD is turned off during the charge sharing period. In this case, when the drain-gate transistor GD is turned on immediately after the charge sharing period, the charge held by the photodiode PD is discarded to the drain.

[0253] As a result, each of the read gate transistors G1, G2, and G3 remains off, and the charge caused by the reflected light does not propagate through the read gate transistors G1, G2, and G3 to the charge storage units CS1, CS2, and CS3. Therefore, each of the charge storage units CS1, CS2, and CS3 stores only the non-control charges QD1, QD2, and QD3 that flow in without passing through the read gate transistors G1, G2, and G3.

[0254] Then, with Figure 10 (A) Similarly, readout processing is performed, and the pixel signal processing circuit 325 outputs a reference voltage VD1 corresponding to the charge amount of non-control charges QD1 accumulated in the charge storage unit CS1, a reference voltage VD2 corresponding to the charge amount of non-control charges QD2 accumulated in the charge storage unit CS2, and a reference voltage VD3 corresponding to the charge amount of non-control charges QD3 accumulated in the charge storage unit CS3 as pixel signals. The distance calculation unit 42 writes and stores each of the acquired reference voltages VD1, VD2, and VD3, along with identification information for identifying each pixel 321 in the light-receiving pixel unit 320, into the correction parameter storage unit 43. Alternatively, the distance calculation unit 42 may write and store each of the acquired reference voltages VD1, VD2, and VD3 in the same order as the pixel signals are read out from the pixel signal processing circuit 325 into the correction parameter storage unit 43.

[0255] Then, in the frame period for measuring the distance between the distance image sensor 32 and the subject S, as already described, the pixel signal processing circuit 325 outputs the pixel signals VQ1, VQ2, and VQ3 corresponding to each of the charge storage units CS1, CS2, and CS3 to the distance calculation unit 42 for each pixel 321 in one frame.

[0256] At this time, the distance calculation unit 42 sequentially reads the reference reflected light voltages VC1 , VC2 , and VC3 and the reference standard voltages VD1 , VD2 , and VD3 corresponding to the read pixel 321 from the correction parameter storage unit 43 .

[0257] Then, the distance calculation unit 42 calculates the reflected light total charge voltage VCALL corresponding to the charge amount of the charge generated by the reflected light in the reference reflected light voltage acquisition frame using the following equation (20).

[0258] VCALL=(VC2-VC1)+(VC3-VC1) (20)

[0259] The distance calculation unit 42 extracts the reflected light charge component generated by the reflected light included in each of the reference reflected light voltages VC1, VC2, and VC3 as the reflected light total charge voltage VCALL using the above-mentioned equation (20).

[0260] Then, the distance calculation unit 42 calculates the reflected light total charge voltage VQALL corresponding to the charge amount of the charges generated by the reflected light in the frame for measuring the distance between the distance image sensor 32 and the subject S using the following equation (21).

[0261] VQALL=(VQ2-VQ1)+(VQ3-VQ1) (21)

[0262] The distance calculation unit 42 extracts the reflected light charge component generated by the reflected light included in each of the pixel signals VQ1 , VQ2 , and VQ3 as the reflected light total charge voltage VQALL using the above-mentioned equation (21).

[0263] The distance calculation unit 42 divides the reflected light total charge voltage VQALL by the reflected light total charge voltage VCALL (VQALL / VCALL) to calculate the adjustment ratio β.

[0264] The adjustment ratio β represents the intensity ratio of the reflected light RL in the reference reflected light voltage acquisition frame and the frame for measuring the distance between the distance image sensor 32 and the subject S.

[0265] Therefore, the distance calculation unit 42 calculates the adjustment voltages VR1, VR2, and VR3 for correcting the pixel signals VQ1, VQ2, and VQ3 using the following equations (22), (23), and (24), respectively.

[0266] VR1=β×VD1=(VQALL / VCALL)×VD1 (22)

[0267] VR2=β×VD2=(VQALL / VCALL)×VD2 (23)

[0268] VR3=β×VD3=(VQALL / VCALL)×VD3 (24)

[0269] Then, the distance calculation unit 42 calculates the corrected pixel signals VQ1 ′, VQ2 ′, and VQ3 ′ using the following equations (25), (26), and (27), respectively.

[0270] VQ1'=VQ1-VR1 (25)

[0271] VQ2'=VQ2-VR2 (26)

[0272] VQ3'=VQ3-VR3 (27)

[0273] The distance calculation unit 42 uses the obtained corrected pixel signals VQ1 ′, VQ2 ′, and VQ3 ′ to calculate the distance L between the range image sensor 32 and the subject S according to the equation (8) in the first embodiment.

[0274] <Fifth embodiment>

[0275] In the fourth embodiment, the reference reflected light voltage and the reference base voltage are written and stored in the correction parameter storage unit 43 before measuring the distance between the subject S and the distance image sensor 32. However, in the fifth embodiment, a structure is adopted in which the correction parameters are obtained each time the distance measurement process is performed.

[0276] This embodiment employs the same configuration as the distance image capturing device 1 shown in the fourth embodiment already described. Hereinafter, operations that differ from those in the fourth embodiment will be described.

[0277] In this embodiment, to obtain one distance image, two frames are used for each of the first frame and the second frame. Therefore, in the fourth embodiment, for example, distance measurements are performed 60 times per second, while in this embodiment, distance measurements are performed 30 times per second.

[0278] In this embodiment, as in the fourth embodiment, the description is made using a state in which there is no background light, that is, a state in which background light does not enter the image sensor 32 and ambient light such as in a dark room is blocked.

[0279] In this embodiment, Figure 10 In the timing diagram of Figure 10(A) shows the timing diagram of the first frame and Figure 10 (B) will be described using a timing chart of the second frame.

[0280] exist Figure 10 In (A), similar to the processing in the fourth embodiment, in the first frame, a light pulse PO is irradiated, and a process is performed to receive reflected light RL from the subject S (delayed by a time Td from the irradiation timing of the light pulse PO), thereby capturing a distance image and acquiring pixel signals VQ1, VQ2, and VQ3. The distance calculation unit 42 then writes and stores each of the acquired pixel signals VQ1, VQ2, and VQ3 in the correction parameter storage unit 43.

[0281] Pixel signal VQ1 is a voltage corresponding to the amount of charge accumulated in charge storage unit CS1. Furthermore, pixel signal VQ2 is a voltage corresponding to the amount of charge accumulated in charge storage unit CS2 by collecting incident light including a portion of reflected light RL on photodiode PD. Pixel signal VQ3 is a voltage corresponding to the amount of charge accumulated in charge storage unit CS3 by collecting incident light including a portion of reflected light RL on photodiode PD. Each of pixel signals VQ1, VQ2, and VQ3 includes a voltage component of non-control charge corresponding to the amount of charge in the incident light, namely, reference voltages VD1, VD2, and VD3.

[0282] exist Figure 10 In the second frame shown in (B), Figure 10 Similarly, in the first frame shown in (A), the light pulse PO is radiated, and thus the reflected light RL from the subject S is received (delayed by a time Td from the radiating timing of the light pulse PO). Figure 5 (B) In the same process, the distance calculation unit 42 obtains the reference voltages VD1 , VD2 , and VD3 for each of the charge storage units CS1 , CS2 , and CS3 .

[0283] Then, the distance calculation unit 42 reads the pixel signals VQ1 , VQ2 , and VQ3 from the correction parameter storage unit 43 .

[0284] In the distance calculation unit 42, each of the reference voltages VD1, VD2 and VD3 is used as the adjustment voltage VO1, VO2, VO3 of each pixel signal VQ1, VQ2, VQ3, and the corrected pixel signals VQ1', VQ2', VQ3' are calculated by the following equations (28), (29) and (30).

[0285] VQ1'=VQ1-VO1 (28)

[0286] VQ2'=VQ2-VO2 (29)

[0287] VQ3'=VQ3-VO3 (30)

[0288] Then, the distance calculation unit 42 calculates the distance L between the subject S and the range image sensor 32 in the range image capturing device 1 using the obtained corrected pixel signals VQ1 ′, VQ2 ′, and VQ3 ′ according to equation (8).

[0289] According to this embodiment, each time the distance calculation process is performed, the reference base voltages VD1, VD2, and VD3 are obtained as correction parameters. Therefore, there is no need to write and store the reference reflected light voltages VC1, VC2, VC3 and the reference base voltages VD1, VD2, VD3 corresponding to all pixels 321 in the correction parameter storage unit 43 in advance as in the fourth embodiment. Therefore, the capacitance of the correction parameter storage unit 43 can be reduced.

[0290] In addition, according to this embodiment, since the measurement pixel signal is obtained in the first frame and the pixel signal of the reference base voltage for correcting the measurement pixel signal is obtained in the next second frame, it is not only applicable to the method of measuring the delay of a single light pulse shown in the present invention, but also can be applied to a CW modulated TOF sensor that irradiates continuously modulated light and calculates the phase shift between the irradiated light and the reflected light for use in distance calculation, as in the fourth embodiment, and can also be applied to various other TOF sensors.

[0291] <Sixth embodiment>

[0292] The distance image capturing device according to the sixth embodiment is Figure 1 The same structure as the first embodiment shown in FIG. Figure 3 As shown, the pixel 321 includes three pixel signal readout units RU1 , RU2 , and RU3 , and each of the pixel signal readout units RU1 , RU2 , and RU3 includes a charge storage unit CS1 , CS2 , and CS3 , respectively.

[0293] In the configuration of this embodiment, similar to the first embodiment, the background light charge collected by the photodiode PD due to incident background light alone is distributed and accumulated in the charge accumulation unit CS1 during a distribution time Tw1 (which is the same time width as the pulse width Tw, and the same applies to Tw2 and Tw3 described below). Distribution times Tw1, Tw2, and Tw3 are the pulse widths of the accumulation drive signal TX1 applied to the readout gate transistor G1, the accumulation drive signal TX2 applied to the readout gate transistor G2, and the accumulation drive signal TX3 applied to the readout gate transistor G3, respectively. Each of Tw1, Tw2, and Tw3 is the same as the pulse width Tw of the light pulse PO.

[0294] and Figure 4Similarly, in the allocation time Tw1 before the light pulse PO is irradiated, the background light charge collected by the photodiode PD is allocated to the charge storage unit CS1. Figure 4 Similarly, in each of the allocated time Tw2 and the next allocated time Tw3 of the light pulse PO, the background light charge collected by the photodiode PD and the charge corresponding to the reflected light RL are allocated.

[0295] In addition, the sixth embodiment differs from the first embodiment in the action of performing distance measurement, that is, the sixth embodiment is configured to calculate and correct the voltage (reflected light control voltages VCL2 and VCL3 described later) corresponding only to the amount of charge (charge amounts QCL2 and QCL3 described later) generated by the reflected light RL and collected in the photoelectric conversion element PD by subtracting the voltage components corresponding to the non-control charges generated by the reflected light RL after the light pulse PO (pulse width Tw) is reflected by the subject S and flows into the respective charge storage units CS1, CS2 and CS3, the voltage components corresponding to the control charges generated by the background light RL and distributed to the respective charge storage units CS1, CS2 and CS3, and the voltage components corresponding to the non-control charges generated by the background light and flowing into the respective charge storage units CS1, CS2 and CS3 from the pixel signals VQ1, VQ2, and VQ3, thereby obtaining the distance between the distance image capturing device 1 and the subject S.

[0296] Figure 15 It is a conceptual diagram showing the relationship between the charge amounts QCB1, QCB2, QCB3 and the charge amounts QFB1, QFB2, QFB3 and the charge amounts QCL1 (not shown, to be described later), QCL2, QCL3 and the charge amounts QFL1, QFL2, QFL3 in each of the charge amounts Q1, Q2, Q3 stored in each charge storage unit CS1, CS2 and CS3.

[0297] Here, each of the charges QCB1, QCB2, and QCB3 represents control charges generated by background light and collected by the photoelectric conversion element PD. These charges are then distributed and accumulated from the photoelectric conversion element PD to the charge accumulation units CS1, CS2, and CS3 via readout gate transistors G1, G2, and G3. Furthermore, each of the charges QFB1, QFB2, and QFB3 represents non-control charges generated by background light and flowing into and accumulated in the charge accumulation units CS1, CS2, and CS3. Each of the charges QCL1 (not shown, described later), QCL2, and QCL3 represents control charges generated by reflected light RL and collected by the photoelectric conversion element PD. These charges are then distributed and accumulated from the photoelectric conversion element PD to the charge accumulation units CS1, CS2, and CS3 via readout gate transistors G1, G2, and G3. The charges QFL1, QFL2, and QFL3 represent non-control charges generated by reflected light RL and flowing into and accumulated in the charge accumulation units CS1, CS2, and CS3. Here, among the electrons (charges) generated by the incident light, the electrons (charges) collected in the photoelectric conversion element PD and distributed to the charge storage unit CS via the readout gate transistor G are control charges. On the other hand, the electrons (charges) that flow into the charge storage unit CS without passing through the readout gate transistor G are non-control charges.

[0298] That is, the control charge caused by the reflected light RL is the charge generated by the reflected light RL and collected in the photoelectric conversion element PD, and is distributed to the charge storage units CS1, CS2, and CS3 via the read gate transistors G1, G2, and G3 (see Figure 3 and Figure 12 ) In addition, the non-control charges caused by the reflected light RL are charges generated by the reflected light RL and are charges that flow into the charge storage units CS1, CS2, and CS3 without passing through the read gate transistors G1, G2, and G3.

[0299] Similarly, the control charge due to background light is generated by the background light and collected in the photoelectric conversion element PD. It is distributed to the charge storage units CS1, CS2, and CS3 via the readout gate transistors G1, G2, and G3. Furthermore, the non-control charge due to background light is generated by the background light and flows into the charge storage units CS1, CS2, and CS3 without passing through the readout gate transistors G1, G2, and G3.

[0300] In this embodiment, the charge amounts QFB1, QFB2, and QFB3 caused by the above-mentioned non-control charges and the non-control voltages corresponding to the charge amounts QFL1, QFL2, and QFL3 are subtracted from the signal voltage corresponding to the charge amounts accumulated in each charge storage unit CS1, CS2, and CS3, and the correction is performed. These parameters are used as necessary parameters for calculating the reflected light control voltages VCL2 and VCL3 generated by the reflected light RL and collected in the photoelectric conversion element PD, and corresponding only to the charge amounts QCL2 and QCL3 caused by the control charges distributed through the readout gate transistors G1, G2, and G3. Before starting the frame for distance measurement, the parameters shown below are pre-acquired and written into and stored in the correction parameter storage unit 43.

[0301] That is, the frame ( Figure 5 (A) In the state where the light pulse PO is not irradiated, the read gate transistor G is turned on and off) and the reference voltage acquisition frame ( Figure 5 (B) shows a frame in which the readout gate transistor G is not driven in a state where the light pulse PO is not irradiated. For example, when the distance image capturing device 1 is shipped from the factory or started up, that is, before capturing a distance image, a reference background light control voltage VPCB1 corresponding to the charge amount (control charge amount) QPCB1 of the control charge generated by the background light is obtained in advance, which is distributed from the photoelectric conversion element PD via the readout gate transistor G1 and accumulated in the charge accumulation section CS1; a reference background light control voltage VPCB2 corresponding to the control charge amount QPCB2 generated by the background light is obtained in advance, which is distributed from the photoelectric conversion element PD via the readout gate transistor G2 and accumulated in the charge accumulation section CS2; and a reference background light control voltage VPCB2 corresponding to the control charge amount QPCB2 generated by the background light is obtained in advance, which is distributed from the photoelectric conversion element PD via the readout gate transistor G3 and accumulated in the charge accumulation section CS4. S3, the reference background light control voltage VPCB3 corresponding to the control charge amount QPCB3 generated by the background light; the reference background light non-control voltage VPFB1 corresponding to the charge amount (non-control charge amount) QPFB1 generated by the background light and flowing into and accumulated in the charge accumulation part CS1 in the non-control state; the reference background light non-control voltage VPFB2 corresponding to the non-control charge amount QPFB2 generated by the background light and flowing into and accumulated in the charge accumulation part CS2 in the non-control state; and the reference background light non-control voltage VPFB3 corresponding to the non-control charge amount QPFB3 generated by the background light and flowing into and accumulated in the charge accumulation part CS3 in the non-control state are first written and stored in the correction parameter storage part 43.

[0302] Alternatively, the reference background light voltage VA acquired in the reference background light voltage acquisition frame and the reference reference voltage VB acquired in the reference reference voltage acquisition frame are acquired in advance before capturing the range image, and are written and stored in the correction parameter storage unit 43 .

[0303] Obviously, the reference background light control voltage VPCB, the reference background light non-control voltage VPFB, the reference background light voltage VA, and the reference base voltage VB may be acquired in advance before shooting the distance image, and written and stored in the correction parameter storage unit 43 .

[0304] The acquisition process of the above-mentioned reference background light control voltages VPCB1, VPCB2, and VPCB3 and the reference background light non-control voltages VPFB1, VPFB2, and VPFB3 will be described later.

[0305] In addition, the frame ( Figure 10 (A) The frame in which the light pulse PO is irradiated in a light-shielded environment and the read gate transistor G is turned on and off) and the frame in which the reference voltage is obtained ( Figure 10 (B) shows a state where the light pulse PO is irradiated in a light-shielded environment without driving the frame of the readout gate transistor G), for example, when the distance image shooting device 1 is shipped or started, that is, before shooting the distance image, a reference reflected light control voltage VPCL1 corresponding to the control charge amount QPCL1 generated by the reflected light is obtained in advance, which is distributed from the photoelectric conversion element PD via the readout gate transistor G1 and accumulated in the charge accumulation section CS1; a reference reflected light control voltage VPCL2 corresponding to the control charge amount QPCL2 generated by the reflected light is obtained in advance, which is distributed from the photoelectric conversion element PD via the readout gate transistor G2 and accumulated in the charge accumulation section CS2; and a reference reflected light control voltage VPCL2 corresponding to the control charge amount QPCL2 generated by the reflected light is obtained in advance, which is distributed from the photoelectric conversion element PD via the readout gate transistor G3 and accumulated in the charge accumulation section CS1. The reference reflected light control voltage VPCL3 of the storage unit CS3, which corresponds to the control charge amount QPCL3 generated by the reflected light; the reference reflected light non-control voltage VPFL1, which is generated by the reflected light and corresponds to the non-control charge amount QPFL1 that flows into and is accumulated in the charge storage unit CS1 in a non-control state; the reference reflected light non-control voltage VPFL2, which is generated by the reflected light and corresponds to the non-control charge amount QPFL2 that flows into and is accumulated in the charge storage unit CS2 in a non-control state; and the reference reflected light non-control voltage VPFL3, which is generated by the reflected light and corresponds to the non-control charge amount QPFL3 that flows into and is accumulated in the charge storage unit CS3 in a non-control state, are first written into and stored in the correction parameter storage unit 43.

[0306] Alternatively, the reference reflected light voltage VC acquired in the reference reflected light voltage acquisition frame and the reference reference voltage VD acquired in the reference reference voltage acquisition frame are acquired in advance before capturing the range image, and are written and stored in the correction parameter storage unit 43 .

[0307] Obviously, the reference reflected light control voltage VPCL, the reference reflected light non-control voltage VPFL, the reference reflected light voltage VC, and the reference base voltage VD may be acquired in advance before capturing the range image, and written and stored in the correction parameter storage unit 43 .

[0308] The acquisition process of the above-mentioned reference reflected light control voltages VPCL1 , VPCL2 , and VPCL3 and the reference reflected light non-control voltages VPFL1 , VPFL2 , and VPFL3 will be described later.

[0309] In addition, the non-controlled charges caused by the background light are as shown in the first embodiment. Figure 6 As described above, during the charge accumulation period (Integration), in which charge is accumulated in the charge accumulation section CS by irradiation with the light pulse PO, and the pixel signal readout period (Read), in which charge is read out from the charge accumulation section CS, background light strikes the range image sensor 32, generating electrons that are accumulated in the charge accumulation section CS. Specifically, non-control charges caused by the background light continue to flow into the charge accumulation section CS and are accumulated for approximately one frame (Integration + Read).

[0310] On the other hand, the non-control charges caused by the reflected light RL are accumulated in the charge accumulation portion CS only during the charge accumulation period (Integration).

[0311] That is, the period during which the non-control charges are accumulated due to the background light is longer than the period during which the non-control charges are accumulated due to the reflected light RL.

[0312] Therefore, the amount of uncontrolled charge flowing into and accumulated in the charge storage unit CS is mainly caused by the background light, but as the background light intensity decreases, the influence of the uncontrolled charge caused by the reflected light RL will become greater. Therefore, in this embodiment, the accuracy of distance measurement is further improved by correcting the uncontrolled voltage equivalent to the uncontrolled charge caused by both the background light and the reflected light RL.

[0313] Next, the following describes a process in which the non-control voltage caused by the non-control charge that flows into the charge storage unit CS without passing through the readout gate transistor G is corrected, and only the reflected light control voltage (reflected light control voltage VCL2 and VCL3) caused by the charge amount of the control charge generated by the reflected light RL and collected in the photoelectric conversion element PD and distributed to the charge storage unit CS via the readout gate transistor G is extracted, and the distance between the subject S and the distance image capture device 1 is calculated using this reflected light control voltage.

[0314] Corresponding to the accumulation Figure 15The charge amounts Q1, Q2, Q3 of each of the charge storage units CS1, CS2, and CS3 are expressed as pixel signals VQ1, VQ2, VQ3 supplied from the pixel circuit 321 of the light receiving unit 3 by the following equations (31) to (33).

[0315] VQ1=VCB1+VCL1+VFB1+VFL1 (31)

[0316] VQ2=VCB2+VCL2+VFB2+VFL2 (32)

[0317] VQ3=VCB3+VCL3+VFB3+VFL3 (33)

[0318] In equations (31) to (33), VCB is a voltage corresponding to the amount of charge generated by background light and distributed from the photoelectric conversion element PD through the readout gate transistor G and accumulated in the charge storage unit CS, that is, a background light control voltage corresponding to the control charge amount QCB due to background light. VCL is a voltage corresponding to the amount of charge generated by reflected light and distributed from the photoelectric conversion element PD through the readout gate transistor G and accumulated in the charge storage unit CS, that is, a reflected light control voltage corresponding to the control charge amount QCL due to the reflected light RL. VFB is a voltage corresponding to the amount of charge generated by background light and flowing into and accumulated in the charge storage unit CS in a non-controlled state, that is, a background light non-control voltage corresponding to the non-control charge amount QFB due to background light. VFL is a voltage corresponding to the amount of charge generated by reflected light and flowing into and accumulated in the charge storage unit CS in a non-controlled state, that is, a reflected light non-control voltage corresponding to the non-control charge amount QFL due to the reflected light RL.

[0319] In equation (31), the light pulse PO is emitted after the charge is distributed from the photoelectric conversion element PD to the charge storage unit CS1. Therefore, at the time the charge is distributed to the charge storage unit CS1, the reflected light RL does not enter the range image sensor 32. Consequently, the charge storage unit CS1 accumulates the control charge QCB1 and non-control charge QFB1 due to the background light, as well as the non-control charge QFL1 due to the reflected light RL, as the charge Q1. However, the control charge QCL1 due to the reflected light RL is not accumulated. Consequently, the reflected light control voltage VCL1 is zero, as shown in equation (34).

[0320] VCL1=0 (34)

[0321] In addition, since the charges generated by the background light and collected in the photoelectric conversion element PD are distributed to the charge storage units CS1, CS2, and CS3 via the readout gate transistors G1, G2, and G3 due to the storage drive signals TX1, TX2, and TX3 with the same pulse width, each of the background light control voltages VCB1, VCB2, and VCB3 has the same voltage value as shown in the following formula (35).

[0322] VCB1=VCB2=VCB3 (35)

[0323] Therefore, based on equations (34) and (35), equations (31), (32), and (33) will each become the following equations (36), (37), and (38), respectively.

[0324] VQ1=VCB1+VFB1+VFL1 (36)

[0325] VQ2=VCB1+VCL2+VFB2+VFL2 (37)

[0326] VQ3=VCB1+VCL3+VFB3+VFL3 (38)

[0327] The above-mentioned pixel signals VQ1 , VQ2 , and VQ3 are actual measurement values actually measured in a distance measurement frame and are known values.

[0328] However, as parameters constituting the pixel signals VQ1, VQ2, and VQ3, the background light control voltage VCB1, the background light non-control voltages VFB1, VFB2, and VFB3, the reflected light control voltages VCL2 and VCL3, and the reflected light non-control voltages VFL1, VFL2, and VFL3 are Figure 15 As shown, they exist as a mixture of charge quantities and are therefore unknown quantities.

[0329] If the background light control voltage VCB1 and the reflected light control voltages VCL2 and VCL3 can be determined among the above unknowns, the distance between the subject S and the range image capturing device 1 can be calculated.

[0330] As already mentioned in the first embodiment, Figure 5 As described above, the frame is acquired based on the reference background photovoltage in the state where the light pulse PO is not irradiated ( Figure 5 In (A), the driving and accumulating driving signals TX1, TX2, and TX3 are driven to obtain reference background light voltages VA1, VA2, and VA3 corresponding to the charges accumulated in the charge accumulating sections CS1, CS2, and CS3. Figure 5In (B)), the storage drive signals TX1, TX2, and TX3 are not driven, and reference voltages VB1, VB2, and VB3 corresponding to the charges stored in the charge storage units CS1, CS2, and CS3 are obtained.

[0331] Here, the reference background light voltage VA obtained in the reference background light control voltage acquisition frame is the sum of the reference background light control voltage VPCB corresponding to the amount of charge generated by the background light and distributed from the photoelectric conversion element PD through the readout gate transistor G and accumulated in the charge storage unit CS, and the reference background light non-control voltage VPFB corresponding to the amount of charge flowing into and accumulated in the charge storage unit CS in a non-controlled state.

[0332] The reference voltage VB acquired in the reference voltage acquisition frame is a reference background light non-control voltage VPFB generated by background light and corresponding to the amount of charge flowing into and accumulated in the charge accumulation section CS in the non-control state.

[0333] Therefore, by referring to the background light voltage VA1 and the reference reference voltage VB1, each of the reference background light control voltages VPCB1, VPCB2, and VPCB3 can be expressed by the following equation (39).

[0334] VPCB1=VPCB2=VPCB3=VA1-VB1 (39)

[0335] In addition, each of the reference background light non-control voltages VPFB1, VPFB2, and VPFB3 can be expressed by the following equations (40) to (42) using the reference voltages VB1, VB2, and VB3.

[0336] VPFB1=VB1 (40)

[0337] VPFB2=VB2 (41)

[0338] VPFB3=VB3 (42)

[0339] Through the above-mentioned equations (39) to (42), the reference background light voltages VA1, VA2 and VA3 obtained in the reference background light voltage acquisition frame and the reference reference voltage acquisition frame can be expressed; the reference background light control voltage VPCB corresponding to the control charge amount caused by the background light through the reference reference voltages VB1, VB2 and VB3; and the reference background light non-control voltage VPFB corresponding to the non-control charge amount caused by the background light can be expressed.

[0340] Frames are acquired by reference to background photovoltage ( Figure 5 (A)) The reference background light voltage VA and the reference reference voltage acquisition frame ( Figure 5The difference between the reference background light control voltage VPCB and the reference background light non-control voltage VPFB obtained from the reference base voltage VB obtained in (B)) and the background light control voltage VCB and the background light non-control voltage VFB obtained in the distance measurement frame is the difference in the intensity of the incident background light under different environments.

[0341] Therefore, the ratio CP1 of the reference background light control voltage VPCB to the background light control voltage VCB obtained in the distance measurement frame, and the ratio CP2 of the reference background light non-control voltage VPFB to the background light non-control voltage VFB obtained in the distance measurement frame, which are obtained using the reference background light voltage acquisition frame and the reference reference voltage acquisition frame, are the same when each voltage has a linear signal level.

[0342] As described above, since the ratio CP1 and the ratio CP2 are the same, the background light correction coefficient R_B between the reference background light non-control voltage VPFB, the background light non-control voltage VFB, the reference background light control voltage VPCB and the background light control voltage VCB can be expressed by the following formula (43).

[0343] R_B = (background light non-controlled voltage of distance measurement frame) / (reference background light non-controlled voltage)

[0344] =VFB / VPFB

[0345] = (background light control voltage of distance measurement frame) / (reference background light control voltage)

[0346] =VCB / VPCB (43)

[0347] In addition, using the relationship between equations (35) and (39), the above equation (43) can also be expressed as the following equation (44).

[0348] R_B=VCB1 / VPCB1 (44)

[0349] Using the above-mentioned equations (39), (43), and (44), the background light correction coefficient R_B can be expressed by the following equation (45).

[0350] R_B=VCB1 / VPCB1

[0351] =VCB1 / (VA1-VB1)

[0352] =VFB1 / VPFB1

[0353] =VFB2 / VPFB2

[0354] =VFB3 / VPFB3 (45)

[0355] According to the above formula (45), each of the background light non-controlled voltages VFB1, VFB2 and VFB3 corresponding to the background light can be expressed by the following formulas (46), (47) and (48), respectively.

[0356] VFB1=R_B×VPFB1=R_B×VB1 (46)

[0357] VFB2=R_B×VPFB2=R_B×VB2 (47)

[0358] VFB3=R_B×VPFB3=R_B×VB3 (48)

[0359] Through each of the above equations (46), (47) and (48), the background light non-control voltages VFB1, VFB2 and VFB3 in the frame of measuring distance can be calculated from the reference background light non-control voltages VPFB1, VPFB2, VPFB3 and the background light correction coefficient R_B.

[0360] In addition, regarding the background light correction coefficient R_B, when focusing on the part "R_B=VCB1 / (VA1-VB1)" in the above formula (45), it can also be obtained by referring to the background light control voltage VCB1 to be determined and the known reference background light voltage VA1 and reference reference voltage VB1 stored in the correction parameter storage unit 43.

[0361] Next, the frame ( Figure 10 (A)) The reference reflected light voltage VC and the reference reference voltage acquisition frame ( Figure 10 (B)) A process of obtaining a reference reflected light control voltage VPCL and a reference reflected light non-control voltage VPFL caused by the reflected light RL obtained with reference to the reference voltage VD, and obtaining a reflected light non-control voltage VFL in a distance measurement frame.

[0362] First, in an environment where light is blocked and background light is not incident, a frame is acquired by referring to the reflected light voltage in a state where no light pulse PO is irradiated ( Figure 10 In (A), the driving and accumulating driving signals TX1, TX2, and TX3 are driven to obtain the reference reflected light voltages VC1, VC2, and VC3 corresponding to the amount of charge accumulated in the charge accumulation units CS1, CS2, and CS3. In addition, the reference voltage acquisition frame ( Figure 10 In (B)), the storage drive signals TX1, TX2, and TX3 are not driven, and reference voltages VD1, VD2, and VD3 corresponding to the charges generated by the reflected light RL flowing into the charge storage unit CS are obtained (acquired in the same manner as in the first embodiment).

[0363] Here, the reference reflected light voltage VC obtained in the reference reflected light control voltage acquisition frame is the sum of the reference reflected light control voltage VPCL corresponding to the amount of charge generated by the reflected light and distributed from the photoelectric conversion element PD through the readout gate transistor G and accumulated in the charge storage unit CS, and the reference background light non-control voltage VPFL corresponding to the amount of charge flowing into and accumulated in the charge storage unit CS in a non-controlled state.

[0364] The reference voltage VD acquired in the reference voltage acquisition frame is a reference reflected light non-control voltage VPFL corresponding to the amount of charge generated by reflected light and flowing into and accumulated in the charge accumulation portion CS in the non-control state.

[0365] Therefore, each of the reference reflected light control voltages VPCL1, VPCL2, and VPCL3 can use the reference reflected light voltages VC1, VC2, and VC3 and the reference reference voltages VD1, VD2, and VD3, and can be expressed by the following equations (49), (50), and (51), respectively.

[0366] VPCL1=VC1-VD1 (49)

[0367] VPCL2=VC2-VD2 (50)

[0368] VPCL3=VC3-VD3 (51)

[0369] In addition, each of the reference reflected light non-control voltages VPFL1, VPFL2 and VPFL3 corresponding to the reflected light can use the reference voltages VD1, VD2 and VD3, and can be expressed by the following equations (52), (53) and (54), respectively.

[0370] VPFL1=VD1 (52)

[0371] VPFL2=VD2 (53)

[0372] VPFL3=VD3 (54)

[0373] Through the above equations (49) to (54), the frame ( Figure 10 (A)) and the reference reflected light voltages VC1, VC2, and VC3 obtained in the reference reference voltage acquisition frame ( Figure 10 The reference voltages VD1, VD2, and VD3 obtained in (B)) represent a reference reflected light control voltage VPCL corresponding to the control charge caused by the reflected light and a reference reflected light non-control voltage VPFL corresponding to the non-control charge caused by the reflected light.

[0374] Then, in the frame for measuring distance, when the charge storage unit CS1 is a charge storage unit that only stores background light, since the control charge generated by the reflected light will be stored in the charge storage units CS2 and CS3, the total control charge voltage VCALL of the reflected light corresponding to the total charge amount of the control charge generated by the reflected light RL can be expressed by the following formula (55).

[0375] VCALL=VCL2+VCL3 (55)

[0376] Similarly, when obtaining the frame ( Figure 10 (A)) and reference voltage acquisition frame ( Figure 10 The reference reflected light total control charge voltage VPCALL corresponding to the total charge amount of the control charges generated by the reflected light RL obtained in (B)) can be expressed by the following equation (56).

[0377] VPCALL=VPCL2+VPCL3

[0378] =(VC2-VD2)+(VC3-VD3) (56)

[0379] In addition, the frame ( Figure 10 (A)) The reference reflected light voltage VC and the reference reference voltage acquisition frame ( Figure 10 The difference between the reference reflected light control voltage VPCL and the reference reflected light non-control voltage VPFL obtained from the reference base voltage VD obtained in (B)) and the reflected light control voltage VCL and the reflected light non-control voltage VFL obtained in the distance measurement frame is the intensity difference of the incident reflected light RL.

[0380] Therefore, the ratio CP3 of the reference reflected light control voltage VPCL to the reflected light control voltage VCL obtained in the distance measurement frame, and the ratio CP4 of the reference reflected light non-control voltage VPFL to the reflected light non-control voltage VFL obtained in the distance measurement frame, which are obtained using the reference reflected light voltage acquisition frame and the reference reference voltage acquisition frame, are the same when each voltage has a linear signal level.

[0381] As described above, since the ratio CP3 and the ratio CP4 are the same, the reflected light correction coefficient R_L between the reference reflected light non-control voltage VPFL, the reflected light non-control voltage VFL, the reference reflected light control voltage VPCL and the reflected light control voltage VCL can be expressed by the following formula (57).

[0382] R_L = (reflected light non-controlled voltage of distance measurement frame) / (reference reflected light non-controlled voltage)

[0383] =VFL / VPFL

[0384] =(Reflected light control voltage of distance measurement frame) / (Reference reflected light control voltage)

[0385] =VCL / VPCL (57)

[0386] Furthermore, by utilizing the relationship between equations (55) and (56) representing the total charge amount of the control charges generated by the reflected light RL, the above equation (57) can also be expressed as the following equation (58).

[0387] R_L = (total control charge voltage of reflected light in distance measurement frame) / (total control charge voltage of reference reflected light)

[0388] =VCALL / VPCALL (58)

[0389] By using the above-mentioned equations (57) and (58), the reflected light correction coefficient R_L can be expressed by the following equation (59).

[0390] R_L=(VCL2+VCL3) / ((VC2-VD2)+(VC3-VD3))

[0391] =VFL1 / VPFL1

[0392] =VFL2 / VPFL2

[0393] =VFL3 / VPFL3 (59)

[0394] According to the above formula (59), each of the reflected light non-control voltages VFL1, VFL2 and VFL3 can be expressed by the following formulas (60), (61) and (62) respectively using the reflected light correction coefficient R_L.

[0395] VFL1=R_L×VPFL1=R_L×VD1 (60)

[0396] VFL2=R_L×VPFL2=R_L×VD2 (61)

[0397] VFL3=R_L×VPFL3=R_L×VD3 (62)

[0398] By using each of the above equations (60), (61) and (62), the reflected light non-control voltages VFL1, VFL2 and VFL3 in the distance measurement frame can be calculated by referring to the reflected light non-control voltage and the reflected light correction coefficient R_L.

[0399] In addition, regarding the reflected light correction coefficient R_L, when focusing on the part of "R_L=(VCL2+VCL3) / ((VC2-VD2)+(VC3-VD3))" in formula (59), it can also be obtained by referring to the desired reflected light control voltages VCL2, VCL3 and the known reference reflected light voltages VC2, VC3 and reference reference voltages VD2, VD3 stored in the correction parameter storage unit 43.

[0400] From the above, it can be seen that each of the background light non-control voltages VFB1, VFB2 and VFB3 and the reflected light non-control voltages VFL1, VFL2 and VFL3, which are unknown variables in equations (36), (37) and (38), constitutes a system of three linear equations, which are formed by the reference background light voltage VA1, reference reference voltages VB1, VB2 and VB3, reference reflected light voltages VC2, VC3, reference reference voltages VB1, VB2 and VB3 stored in the correction parameter storage unit 43, and the background light control voltage VCB1 and reflected light control voltages VCL2 and VCL3 to be determined.

[0401] The only unknowns in this three-variable linear simultaneous equation are the background light control voltage VCB1 and the reflected light control voltages VCL2 and VCL3, so the solution can be obtained strictly.

[0402] That is, the distance calculation unit 42 solves the above-mentioned three-variable linear simultaneous equations to obtain the background light control voltage VCB1 and the reflected light control voltages VCL2 and VCL3, and calculates the distance L between the distance image capturing device 1 and the subject S through the equation (63) described later.

[0403] Alternatively, the solution obtained by solving the above three linear equations is stored in a storage unit or recorded in a program to obtain the background light control voltage VCB1, the reflected light control voltages VCL2 and VCL3, and the distance L between the image capturing device 1 and the subject S is calculated.

[0404] As described above, the frame is acquired with reference to the background photovoltage ( Figure 5 (A)) and obtain the frame with reference to the reference voltage ( Figure 5 (B)) and obtain the frame with reference to the reflected light voltage ( Figure 10 (A)) and reference voltage acquisition frame ( Figure 10 In (B)), the reference background light voltages VA1, VA2 and VA3, the reference reflected light voltages VC1, VC2 and VC3, and the reference reference voltages VB1, VB2, VB3, VD1, VD2 and VD3 are obtained as actual measurement values, and are written and stored in the correction parameter storage unit 43 together with the identification information of each pixel 321 in the light-receiving pixel unit 320.

[0405] Alternatively, using equations (39) to (42) and (49) to (54), the reference background light control voltages VPCB1, VPCB2 and VPCB3 and the reference background light non-control voltages VPFB1, VPFB2 and VPFB3 and the reference reflected light control voltages VPCL1, VPCL2 and VPCL3 and the reference reflected light non-control voltages VPFL1, VPFL2 and VPFL3 are calculated from each of the reference background light voltages VA1, VA2 and VA3 and the reference reflected light voltages VC1, VC2 and VC3 and the reference reference voltages VB1, VB2, VB3, VD1, VD2 and VD3, and are written and stored in the correction parameter storage unit 43 together with the identification information of each pixel 321 in the light-receiving pixel unit 320.

[0406] In addition, each of the reference background light voltages VA1, VA2 and VA3, the reference reflected light voltages VC1, VC2 and VC3, and the reference reference voltages VB1, VB2, VB3, VD1, VD2 and VD3 are first written and stored in the correction parameter storage unit 43 together with the identification information of each pixel 321 in the light-receiving pixel unit 320. Furthermore, a structure can also be adopted in which each of the reference background light control voltages VPCB1, VPCB2 and VPCB3, the reference background light non-control voltages VPFB1, VPFB2 and VPFB3, the reference reflected light control voltages VPCL1, VPCL2 and VPCL3, and the reference reflected light non-control voltages VPFL1, VPFL2 and VPFL3 obtained with reference to these are also first written and stored in the correction parameter storage unit 43.

[0407] Then, each time each pixel signal VQ1, VQ2, and VQ3 is acquired in a distance measurement frame, the distance calculation unit 42 solves the three-variable linear simultaneous equations to calculate the background light control voltage VCB1 and the reflected light control voltages VCL2 and VCL3, referring to the reference background light voltages VA1, VA2, and VA3, the reference reflected light voltages VC1, VC2, and VC3, the reference reference voltages VB1, VB2, VB3, VD1, VD2, and VD3, or the reference background light control voltages VPCB1, VPCB2, and VPCB3, the reference background light non-control voltages VPFB1, VPFB2, and VPFB3, the reference reflected light control voltages VPCL1, VPCL2, and VPCL3, and the reference reflected light non-control voltages VPFL1, VPFL2, and VPFL3 stored in the correction parameter storage unit 43. Alternatively, the solutions to the three-variable linear simultaneous equations are stored in the storage unit or recorded in a program to calculate the background light control voltage VCB1 and the reflected light control voltages VCL2 and VCL3.

[0408] Furthermore, the distance calculation unit 42 calculates the distance L between the image sensor 32 and the subject S using the background light control voltage VCB1 and the reflected light control voltages VCL2 and VCL3 using the following equation (63) corresponding to the already described equation (1).

[0409] L=((V3-V1) / (V2+V3-2V1))×Dm

[0410] =(((VCB1+VCL3)-VCB1) / ((VCB1+VCL2)+(VCB1+VCL3)-2×VCB1))×Dm

[0411] =(VCL3 / (VCL2+VCL3))×Dm (63)

[0412] In the above equation (63), Dm is (c / 2)Tw. Tw is the pulse width of the optical pulse PO.

[0413] According to this embodiment, regardless of whether the ratio of the charge amount of non-control charges accumulated without passing through the readout gate transistor G to the charge amount of control charges accumulated in the charge accumulation unit CS through the readout gate transistor G becomes larger, or the area of the pixel 321 is reduced, or there is a large deviation in the non-control charges contained in each charge accumulated in the charge accumulation unit CS, the charge amount of which varies depending on the incident angle of the incident light, various influences can be eliminated, and the distance L between the image capturing device 1 and the subject S can be calculated with the same or higher accuracy as when the pixel area is not reduced.

[0414] Furthermore, according to the present embodiment, after the lens 31 is installed and the distance image capturing device 1 is assembled, the reference background light voltages VA1, VA2, and VA3, the reference reflected light voltages VC1, VC2, and VC3, and the reference base voltages VB1, VB2, VB3, VD1, VD2, and VD3 can be measured without using special calibration equipment, and the respective characteristics of the installed lens 31, and also the difference in the relative position of the installed lens 31 and the distance image sensor 32, etc., can be made to correspond to each other, thereby making it possible to easily and accurately calculate the respective reflected light control voltages VCL2 and VCL3 corresponding to the amount of charge generated by the reflected light RL, collected in the photoelectric conversion element PD, and distributed to the charge storage units CS2 and CS3 through the readout gate transistors G2 and G3, excluding the background light non-control voltage VFB and the reflected light non-control voltage VFL.

[0415] <Seventh embodiment>

[0416] The distance image capturing device according to the seventh embodiment has the same structure as the sixth embodiment, but Figure 3As shown, the pixel 321 of the sixth embodiment has three pixel signal readout units, namely RU1, RU2 and RU3, while in this embodiment, there are four pixel signal readout units, namely RU1, RU2, RU3 and RU4 (not shown).

[0417] Each of the pixel signal readout units RU1 , RU2 , RU3 , and RU4 includes a charge storage unit CS1 , CS2 , CS3 , and CS4 (not shown), respectively.

[0418] In the configuration of this embodiment, similar to the sixth embodiment, the background light charge collected by the photodiode PD due to incident background light alone is distributed and accumulated in the charge accumulation unit CS1 during a distribution time Tw1 (which is the same duration as the pulse width Tw, and the same applies to each of Tw2, Tw3, and Tw4 described below). Distribution times Tw1, Tw2, Tw3, and Tw4 (not shown) are the pulse widths of the accumulation drive signal TX1 applied to the readout gate transistor G1, the accumulation drive signal TX2 applied to the readout gate transistor G2, the accumulation drive signal TX3 applied to the readout gate transistor G3, and the accumulation drive signal TX4 (not shown) applied to the readout gate transistor G4 (not shown), respectively. Each of Tw1, Tw2, Tw3, and Tw4 is the same as the pulse width Tw of the light pulse PO.

[0419] and Figure 4 Similarly, in the allocation time Tw1 before the light pulse PO is irradiated, the background light charge collected by the photodiode PD is allocated to the charge storage unit CS1. Figure 4 Similarly, in the allocated time Tw2 of the light pulse PO, the next allocated time Tw3, and the next allocated time Tw4, the background light charge collected by the photodiode PD and the charge corresponding to the reflected light RL are allocated.

[0420] Figure 16 It is a conceptual diagram showing the relationship between the charge amounts QCB1, QCB2, QCB3, QCB4 and the charge amounts QFB1, QFB2, QFB3, QFB4 in each of the charge amounts Q1, Q2, Q3, Q4 stored in each charge storage unit CS1, CS2, CS3 and CS4, and the charge amounts QCL1 (not shown, to be described later), QCL2, QCL3, QCL4 (not shown, to be described later) and the charge amounts QFL1, QFL2, QFL3, QFL4.

[0421] Specifically, each of the charges QCB1, QCB2, QCB3, and QCB4 represents control charges generated by background light, collected by the photoelectric conversion element PD, and distributed and accumulated from the photoelectric conversion element PD to the charge accumulation units CS1, CS2, CS3, and CS4 via readout gate transistors G1, G2, G3, and G4. Furthermore, each of the charges QFB1, QFB2, QFB3, and QFB4 represents non-control charges generated by background light, flowing into and accumulated in the charge accumulation units CS1, CS2, CS3, and CS4. Each of the charges QCL1 (not shown, described later), QCL2, QCL3, and QCL4 (not shown, described later) represents control charges generated by reflected light, collected by the photoelectric conversion element PD, and distributed and accumulated from the photoelectric conversion element PD to the charge accumulation units CS1, CS2, CS3, and CS4 via readout gate transistors G1, G2, G3, and G4. Each of the charge amounts QFL1 , QFL2 , QFL3 , and QFL4 is a non-control charge generated by reflected light and flowing into and stored in the charge storage units CS1 , CS2 , CS3 , and CS4 .

[0422] Here, the control charge caused by the reflected light RL is the charge collected by the photodiode PD and distributed to the charge storage units CS1, CS2, CS3, and CS4 via the read gate transistors G1, G2, G3, and G4 (see Figure 3 and Figure 12 ) In addition, the non-control charges caused by the reflected light RL are charges that flow in without passing through the read gate transistors G1, G2, G3, and G4.

[0423] Similarly, control charge due to background light is the charge collected by the photodiode PD and distributed to the charge storage units CS1, CS2, CS3, and CS4 via the readout gate transistors G1, G2, G3, and G4. Furthermore, non-control charge due to background light is the charge that flows in without passing through the readout gate transistors G1, G2, G3, and G4.

[0424] In the present embodiment, in addition to the reference background light control voltages VPCB1, VPCB2, and VPCB3 and the reference background light non-control voltages VPFB1, VPFB2, and VPFB3 in the sixth embodiment, for example, at the time of shipment or startup of the distance image capturing apparatus 1, that is, before capturing a distance image, a reference background light control voltage VPCB4 corresponding to the background light charge amount QPCB4 generated by the background light and a reference background light non-control voltage VPFB4 corresponding to the non-control charge amount QPFB4 generated by the background light and flowing into and accumulated in the charge accumulation unit CS4 in a non-controlled state are pre-acquired from the photoelectric conversion element PD via the readout gate transistor G4 and accumulated in the charge accumulation unit CS4, and are first written into and stored in the correction parameter storage unit 43.

[0425] Alternatively, the reference background light voltage VA acquired in the reference background light voltage acquisition frame and the reference reference voltage VB acquired in the reference reference voltage acquisition frame are acquired in advance before capturing the range image, and are written and stored in the correction parameter storage unit 43 .

[0426] Obviously, the reference background light control voltage VPCB, the reference background light non-control voltage VPFB, the reference background light voltage VA, and the reference base voltage VB may be acquired in advance before shooting the distance image, and written and stored in the correction parameter storage unit 43 .

[0427] Similarly, in addition to the reference reflected light control voltages VPCL1, VPCL2, and VPCL3 and the reference reflected light non-control voltages VPFL1, PFL2, and VPFL3 in the sixth embodiment, for example, when the distance image capturing device 1 is shipped from the factory or started up, that is, before capturing the distance image, a reference reflected light control voltage VPCL4 corresponding to the reflected light charge amount QPCL4 generated by the reflected light RL and collected in the photoelectric conversion element PD and distributed via the readout gate transistor G4, and a reference reflected light non-control voltage VPFL4 corresponding to the non-control charge amount QPFL4 generated by the reflected light and flowing into and accumulated in the charge storage unit CS4 in a non-controlled state are pre-acquired and first written into and stored in the correction parameter storage unit 43.

[0428] Alternatively, the reference reflected light voltage VC acquired in the reference reflected light voltage acquisition frame and the reference reference voltage VD acquired in the reference reference voltage acquisition frame are acquired in advance before capturing the range image, and are written and stored in the correction parameter storage unit 43 .

[0429] Obviously, the reference reflected light control voltage VPCL, the reference reflected light non-control voltage VPFL, the reference reflected light voltage VC, and the reference base voltage VD may be acquired in advance before capturing the range image, and written and stored in the correction parameter storage unit 43 .

[0430] Next, the following describes a process in which the non-control voltage caused by the non-control charge flowing in without passing through the readout gate transistor is corrected, and the control voltage caused by only the charge amount of the control charge generated by the reflected light RL and collected in the photoelectric conversion element PD and distributed to the charge storage unit CS via the readout gate transistor G is extracted, and the distance between the subject S and the distance image capture device is calculated using this charge amount.

[0431] Figure 16 The pixel signals VQ1, VQ2, VQ3, and VQ4 corresponding to the charge amounts Q1, Q2, Q3, and Q4 accumulated in the charge storage units CS1, CS2, CS3, and CS4 can be expressed by the following equations (64) to (67).

[0432] VQ1=VCB1+VCL1+VFB1+VFL1 (64)

[0433] VQ2=VCB2+VCL2+VFB2+VFL2 (65)

[0434] VQ3=VCB3+VCL3+VFB3+VFL3 (66)

[0435] VQ4=VCB4+VCL4+VFB4+VFL4 (67)

[0436] In the above equations (64) to (67), VCB is the background light control voltage corresponding to the control charge amount QCB caused by the background light, distributed from the photoelectric conversion element PD via the readout gate transistor G and accumulated in the charge storage unit CS. VCL is the reflected light control voltage corresponding to the control charge amount QCL caused by the reflected light RL, distributed from the photoelectric conversion element PD via the readout gate transistor G and accumulated in the charge storage unit CS. VFB is the background light non-control voltage corresponding to the non-control charge amount QFB caused by the background light, generated by the background light, flowing into and accumulated in the charge storage unit CS in the non-control state. VFL is the reflected light non-control voltage corresponding to the non-control charge amount QFL caused by the reflected light RL, generated by the reflected light RL, flowing into and accumulated in the charge storage unit CS in the non-control state.

[0437] In equation (64), the light pulse PO is emitted after the charge is distributed from the photoelectric conversion element PD to the charge storage unit CS1. Therefore, at the time the charge is distributed to the charge storage unit CS1, the reflected light RL does not enter the range image sensor 32. Consequently, the charge storage unit CS1 accumulates the control charge QCB1 and non-control charge QFB1 due to the background light, as well as the non-control charge QFL1 due to the reflected light RL, as the charge Q1. However, the control charge QCL1 due to the reflected light RL is not accumulated. Consequently, the reflected light control voltage VCL1 is zero, as shown in equation (68). Furthermore, in this embodiment, an example is described in which reflected light RL enters the range image sensor 32 at the timing when charge is distributed from the photoelectric conversion element PD to the charge storage units CS2 and CS3. Therefore, in the charge storage unit CS4, the control charge amount QCB4 and the non-control charge amount QFB4 due to the background light, as well as the non-control charge amount QFL4 due to the reflected light RL, are also accumulated, while the control charge amount QCL4 due to the reflected light RL is not accumulated. Therefore, in this example, the reflected light control voltage VCL4 becomes zero, similar to the reflected light control voltage VCL1, as shown in the following equation (68).

[0438] VCL1=VCL4=0 (68)

[0439] In addition, since the charges generated by the background light and collected in the photoelectric conversion element PD are distributed to the charge storage units CS1, CS2, CS3, and CS4 via the readout gate transistors G1, G2, G3, and G4 due to the accumulation drive signals TX1, TX2, TX3, and TX4 with the same pulse width, each of the background light control voltages VCB1, VCB2, VCB3, and VCB4 is the same as shown in the following formula (69).

[0440] VCB1=VCB2=VCB3=VCB4 (69)

[0441] Therefore, equations (64) to (67) will become the following equations (70), (71), (72), and (73), respectively.

[0442] VQ1=VCB1+VFB1+VFL1 (70)

[0443] VQ2=VCB1+VCL2+VFB2+VFL2 (71)

[0444] VQ3=VCB1+VCL3+VFB3+VFL3 (72)

[0445] VQ4=VCB1+VFB4+VFL4 (73)

[0446] The above-mentioned pixel signals VQ1 , VQ2 , VQ3 , and VQ4 are actual measurement values actually measured in a distance measurement frame and are known values.

[0447] However, as the parameters constituting the pixel signals VQ1, VQ2, VQ3 and VQ4, the background light control voltage VCB1, background light non-control voltages VFB1, VFB2, VFB3 and VFB4, the reflected light control voltages VCL2 and VCL3, and the reflected light non-control voltages VFL1, VFL2, VFL3 and VFL4 are Figure 16 As shown, they exist as a mixture of charge quantities and are therefore unknown quantities.

[0448] If the background light control voltage VCB1 and the reflected light control voltages VCL2 and VCL3 can be determined among the above unknowns, the distance between the subject S and the range image capturing device 1 can be calculated.

[0449] Then, similarly to the sixth embodiment, by referring to the background light voltage, the frame ( Figure 5 (A)) and reference voltage acquisition frame ( Figure 5 (B)) has the same relationship with the ratio of the background light intensity of the frame for measuring the distance, so the background light correction coefficient R_B between the reference background light non-control voltage VPFB and the background light non-control voltage VFB can be expressed by the following formula (74).

[0450] R_B=VCB1 / (VA1-VB1) (74)

[0451] Therefore, each of the background light non-control voltages VFB1, VFB2, VFB3 and VFB4 corresponding to the background light can be expressed by the following equations (75), (76), (77) and (78), respectively.

[0452] VFB1=R_B×VPFB1=R_B×VB1 (75)

[0453] VFB2=R_B×VPFB2=R_B×VB2 (76)

[0454] VFB3=R_B×VPFB3=R_B×VB3 (77)

[0455] VFB4=R_B×VPFB4=R_B×VB4 (78)

[0456] Through each of the above equations (75), (76), (77) and (78), the background light non-control voltages VFB1, VFB2, VFB3 and VFB4 in the frame of measuring distance can be calculated by referring to the background light non-control voltages VPFB1, VPFB2, VPFB3, VPFB4 and the background light correction coefficient R_B.

[0457] In addition, the background light correction coefficient R_B in equation (74) can also be obtained by referring to the background light control voltage VCB1 to be obtained and the known reference background light voltage VA1 and reference base voltage VB1 stored in the correction parameter storage unit 43.

[0458] In addition, similarly to the sixth embodiment, since the frame ( Figure 10 (A)) and reference voltage acquisition frame ( Figure 10 (B)) has the same relationship with the ratio of the intensity of the reflected light of the frame for measuring the distance, so the reflected light correction coefficient R_L between the reference reflected light non-control voltage VPFL and the reflected light non-control voltage VFL can be expressed by the following formula (79).

[0459] R_L=(VCL2+VCL3) / ((VC2-VD2)+(VC3-VD3)) (79)

[0460] Therefore, each of the reflected light non-control voltages VFL1, VFL2, VFL3 and VFL4 corresponding to the reflected light can be expressed by the following equations (80), (81), (82) and (83), respectively.

[0461] VFL1=R_L×VPFL1=R_L×VD1 (80)

[0462] VFL2=R_L×VPFL2=R_L×VD2 (81)

[0463] VFL3=R_L×VPFL3=R_L×VD3 (82)

[0464] VFL4=R_L×VPFL4=R_L×VD4 (83)

[0465] Similarly, through equations (80) to (83), each of the reflected light non-control voltages VFL1, VFL2, VFL3, and VFL4 can be represented by the reflected light correction coefficient R_L and the reference reflected light non-control voltages VPFL1, VPFL2, VPFL3, and VPFL4, respectively.

[0466] In addition, the background light correction coefficient R_L in equation (79) can also be obtained by referring to the desired reflected light control voltages VCL2, VCL3 and the known reference reflected light voltages VC2, VC3 and reference base voltages VD2, VD3 stored in the correction parameter storage unit 43.

[0467] From the above, it can be seen that each of the background light non-control voltages VFB1, VFB2, VFB3 and VFB4, and the reflected light non-control voltages VFL1, VFL2, VFL3 and VFL4, which were once unknown numbers, is expressed by a linear simultaneous equation formed by the reference background light voltage VA1, reference reference voltages VB1, VB2, VB3, VB4, reference reflected light voltages VC2, VC3 and reference reference voltages VD1, VD2, VD3, VD4 stored in the correction parameter storage unit 43 and the background light control voltage VCB1 and reflected light control voltages VCL2 and VCL3 to be determined.

[0468] That is, the distance calculation unit 42 obtains the background light control voltage VCB1 and the reflected light control voltages VCL2 and VCL3 by solving the aforementioned linear simultaneous equations, and calculates the distance L between the range image capturing device 1 and the subject S.

[0469] Alternatively, the solution obtained by solving the above-mentioned linear simultaneous equations is stored in a storage unit or recorded in a program, and the background light control voltage VCB1 and the reflected light control voltages VCL2 and VCL3 are obtained to calculate the distance L between the distance image capturing device 1 and the subject S.

[0470] As described above, the frame is acquired with reference to the background photovoltage ( Figure 5 (A)) and obtain the frame with reference to the reference voltage ( Figure 5 (B)) and obtain the frame with reference to the reflected light voltage ( Figure 10 (A)) and reference voltage acquisition frame ( Figure 10 In (B)), the reference background light voltages VA1, VA2, VA3 and VA4, the reference reflected light voltages VC1, VC2, VC3 and VC4 and the reference reference voltages VB1, VB2, VB3, VB4, VD1, VD2, VD3 and VD4 are obtained as actual measurement values, and are written and stored in the correction parameter storage unit 43 together with the identification information of each pixel 321 in the light-receiving pixel unit 320.

[0471] Alternatively, the distance calculation unit 42 calculates the reference background light control voltages VPCB1, VPCB2, VPCB3 and VPCB4 and the reference reflected light control voltages VPCL1, VPCL2, VPCL3 and VPCL4 and the reference background light non-control voltages VPFB1, VPFB2, VPFB3 and VPFB4 and the reference reflected light non-control voltages VPFL1, VPFL2, VPFL3 and VPFL4 from each of the reference background light voltages VA1, VA2, VA3 and VA4 and the reference reflected light voltages VC1, VC2, VC3 and VC4 and the reference reference voltages VB1, VB2, VB3, VB4, VD1, VD2, VD3 and VD4, and writes and stores them in the correction parameter storage unit 43 together with the identification information of each pixel 321 in the light-receiving pixel unit 320.

[0472] Then, in the distance measurement frame, each time the distance calculation unit 42 obtains each pixel signal VQ1, VQ2, VQ3 and VQ4, it refers to the reference background light voltages VA1, VA2, VA3 and VA4 and the reference reflected light voltages VC1, VC2, VC3 and VC4 and the reference reference voltages VB1, VB2, VB3, VB4, VD1, VD2, VD3 and VD4 stored in the correction parameter storage unit 43, or the reference background light control voltages VPCB1, VPCB2, VPCB3 and VPCB4 and the reference background light non-control voltages VPFB1, VPFB2, VPFB3 and VPFB4 and the reference reflected light control voltages VPCL1, VPCL2, VPCL3 and VPCL4 and the reference reflected light non-control voltages VPFL1, VPFL2, VPFL3 and VPFL4, respectively, to solve the above-mentioned linear simultaneous equations and calculate the background light control voltage VCB1 and the reflected light control voltages VCL2 and VCL3. Alternatively, the solution obtained by solving the three linear simultaneous equations is stored in a storage unit or recorded in a program to calculate the background light control voltage VCB1 and the reflected light control voltages VCL2 and VCL3.

[0473] According to this embodiment, regardless of whether the ratio of the amount of non-control charges accumulated without passing through the readout gate transistor G to the amount of control charges accumulated in the charge storage unit CS through the readout gate transistor G becomes larger, or the area of the pixel 321 is reduced, or there is a large deviation in the non-control charges contained in each charge accumulated in the charge storage unit CS, the amount of which varies depending on the incident angle of the incident light, various influences can be eliminated, and the distance L between the subject S and the image capturing device 1 can be calculated with the same or higher accuracy as when the pixel area is not reduced.

[0474] According to the sixth embodiment, when the number of pixel signal readout units RU is three, the distance L between the subject S and the image capturing device 1 can be determined with high distance accuracy even if the non-control charge has a large variation. Furthermore, according to the seventh embodiment, when the number of pixel signal readout units RU is four, the distance L between the subject S and the image capturing device 1 can be determined with high distance accuracy even if the non-control charge has a large variation.

[0475] It can be seen from the description of the seventh embodiment that, similar to the case where there are four pixel signal readout units RU, even if there are five, six or more pixel signal readout units RU and the deviation of the non-control charge is large, the distance L between the subject S and the image capturing device 1 can be calculated with a higher distance accuracy.

[0476] That is, according to this embodiment, when there are N pixel signal readout units RU (N is an integer greater than or equal to 3), high distance accuracy can be achieved in distance measurement even if the non-control charge has a large variation.

[0477] <Eighth embodiment>

[0478] Similar to the sixth embodiment, the eighth embodiment is configured to calculate a voltage corresponding to the amount of charge generated only by the reflected light (reflected light control voltages VCL2 and VCL3 to be described later) by subtracting from the pixel signals VQ1, VQ2, and VQ3 the voltage component corresponding to the non-control charge generated by the reflected light RL after the light pulse PO (pulse width Tw) is reflected from the subject S and flows into the respective charge storage units CS1, CS2, and CS3, the voltage component corresponding to the control charge generated by the background light and distributed to the respective charge storage units CS1, CS2, and CS3, and the voltage component corresponding to the non-control charge generated by the background light and flows into the respective charge storage units CS1, CS2, and CS3.

[0479] In addition, in this embodiment, similarly to the third embodiment, two frames, namely the first frame and the second frame, are used to acquire one range image.

[0480] Then, in this embodiment, as described in the sixth embodiment, by referring to the reflected light voltage, the frame ( Figure 10 (A)) and reference voltage acquisition frame ( Figure 10(B)), in a light-shielded environment and in a state where a light pulse PO is irradiated, for example, at the time of shipment or startup of the distance image capturing apparatus 1, that is, before capturing a distance image, a reference reflected light control voltage VPCL1 corresponding to the reflected light charge amount QPCL1 generated by the reflected light RL is obtained in advance, which is distributed from the photoelectric conversion element PD via the readout gate transistor G1 and accumulated in the charge accumulation section CS1; a reference reflected light control voltage VPCL2 corresponding to the reflected light charge amount QPCL2 generated by the reflected light RL is obtained in advance, which is distributed from the photoelectric conversion element PD via the readout gate transistor G2 and accumulated in the charge accumulation section CS2; and a reference reflected light control voltage VPCL2 corresponding to the reflected light charge amount QPCL2 generated by the reflected light RL is obtained in advance, which is distributed from the photoelectric conversion element PD via the readout gate transistor G3 and accumulated in the charge accumulation section CS3. The reference reflected light control voltage VPCL3 corresponding to the reflected light charge amount QPCL3 generated by the reflected light RL; the reference reflected light non-control voltage VPFL1 corresponding to the non-control charge amount QPFL1 generated by the reflected light RL and flowing into and accumulated in the charge accumulation part CS1 in a non-control state; the reference reflected light non-control voltage VPFL2 corresponding to the non-control charge amount QPFL2 generated by the reflected light RL and flowing into and accumulated in the charge accumulation part CS2 in a non-control state; and the reference reflected light non-control voltage VPFL3 corresponding to the non-control charge amount QPFL3 generated by the reflected light RL and flowing into and accumulated in the charge accumulation part CS3 in a non-control state are first written and stored in the correction parameter storage part 43.

[0481] Then, in this embodiment, the distance measurement frames are set to two frames, the first frame and the second frame, and in the first frame, the light source device 21 is controlled not to emit the light pulse PO, and the control circuit 322 is controlled not to output the accumulation drive signals TX1, TX2, and TX3, and the reference reference voltage acquisition frame (corresponding to the reference reference voltage acquisition frame) corresponding to the non-control charge generated by the background light is acquired. Figure 5 (B) Then, in the second frame, the light source device 21 is controlled to emit the light pulse PO, and the control circuit 322 is controlled to output the storage drive signals TX1, TX2, and TX3, and the pixel signals VQ1, VQ2, and VQ3 are acquired to measure the distance.

[0482] In this case, the execution order of the first frame and the second frame may be controlled so that the operations are completely opposite.

[0483] As in the sixth embodiment, each of the pixel signals VQ1, VQ2, and VQ3 can be expressed by equations (36), (37), and (38). In this embodiment, since the background light intensity in the first frame and the second frame for distance measurement, i.e., the reference voltage acquisition frames for acquiring the reference voltages VB1, VB2, and VB3 corresponding to the non-control charges generated by the background light, is the same, equations (40), (41), and (42) can be expressed as follows.

[0484] VB1=VPFB1=VFB1 (84)

[0485] VB2=VPFB2=VFB2 (85)

[0486] VB3=VPFB3=VFB3 (86)

[0487] Therefore, the equation (43) becomes R_B=VFB / VPFB=1. That is, the background light correction coefficient R_B becomes 1 (R_B=1).

[0488] In the second frame, the reference voltage VB1 obtained in the first frame is subtracted from the pixel signal VQ1 , which is a voltage corresponding to the amount of charge accumulated in the charge storage unit CS1 , to obtain the sum of the background light control voltage VCB1 and the reflected light non-control voltage VFL1 .

[0489] Here, the reflected light correction coefficient R_L can be expressed as

[0490] R_L=(reference voltage corresponding to the distance measurement frame (first frame)) / (reference reference voltage)

[0491] =VB / VD

[0492] =VB / VPFL (87).

[0493] After obtaining the reflected light correction coefficient R_L, the reflected light non-control voltage VFL1 can be calculated from "VFL1=R_L×VD1" in formula (60), and only the desired background light control voltage VCB1 can be extracted from the sum of the background light control voltage VCB1 and the reflected light non-control voltage VFL1.

[0494] Therefore, similar to the sixth embodiment, Equations (36), (37) and (38) become three-variable linear simultaneous equations consisting of the background light control voltage VCB1 and the reflected light control voltages VCL2 and VCL3 that are desired to be obtained, and the background light control voltage VCB1 and the reflected light control voltages VCL2 and VCL3 can be strictly obtained as solutions.

[0495] Then, the distance calculation unit 42 solves a linear simultaneous equation using the reference reflected light control voltages VPCL1, VPCL2, and VPCL3, the reference base voltages VPFL1, VPFL2, and VPFL3, the calculated reference background light control voltages VPCB1, VPCB2, and VPCB3, the reference base voltages VPFB1, VPFB2, and VPFB3, and the actually measured pixel signals VQ1, VQ2, and VQ3 stored in the correction parameter storage unit 43 to determine the background light control voltage VCB1 and the reflected light control voltages VCL2 and VCL3. Alternatively, the solutions to the linear simultaneous equations are stored in the storage unit or recorded in a program to determine the background light control voltage VCB1 and the reflected light control voltages VCL2 and VCL3.

[0496] Then, the distance calculation unit 42 uses equation (63) and controls the voltages VCL2 and VCL3 by the reflected light to calculate the distance from the range image capturing device 1 (range image sensor 32) to the subject S.

[0497] As described above, according to this embodiment, the reference background light voltage acquisition frame ( Figure 5 (A)) and reference voltage acquisition frame ( Figure 5 (B)) is performed in two frames when capturing the distance image, rather than before capturing the distance image. This allows the process of calculating the background light correction coefficient R_B (because it is = 1) to be omitted when calculating the distance, and reduces the load on the distance calculation.

[0498] Furthermore, this embodiment has been described using an example in which there are three pixel signal readout units RU. However, similarly to the seventh embodiment, the same solution can be obtained even when there are four pixel signal readout units RU. As described in the seventh embodiment, the same solution can be obtained even when there are five, six, or more pixel signal readout units RU. This allows the distance L between the subject S and the image capturing device 1 to be determined with high distance accuracy. In other words, according to this embodiment, even when there are N pixel signal readout units RU (N is an integer greater than or equal to 3), high distance accuracy can be achieved in distance measurement even when there is a large variation in the non-control charge.

[0499] <Ninth embodiment>

[0500] The ninth embodiment is based on the premise that a distance image is captured in a light-shielded state (i.e., a state without background light) or an extremely dark state (i.e., a state where the background light intensity is negligible), and calculates a voltage (reflected light control voltages VCL2 and VCL3 described later) that corresponds only to the amount of charge generated by the reflected light RL after the light pulse PO (pulse width Tw) is reflected from the subject S, and is corrected by subtracting a voltage component corresponding to the non-controlled charge.

[0501] In addition, in this embodiment, similarly to the eighth embodiment, two frames, namely the first frame and the second frame, are used to acquire one range image.

[0502] Then, in this embodiment, a reference voltage acquisition frame (corresponding to Figure 10 (B) In the first of two frames for distance measurement, the light source device 21 is controlled to emit a light pulse PO, and the control circuit 322 is controlled not to output the storage drive signals TX1, TX2, and TX3, thereby obtaining reference voltages VD1, VD2, and VD3 corresponding to the non-controlled charges generated by the reflected light RL. Then, in the second frame, the light source device 21 is controlled to emit a light pulse PO, and the control circuit 322 is controlled to output the storage drive signals TX1, TX2, and TX3, thereby obtaining pixel signals VQ1, VQ2, and VQ3, thereby measuring the distance.

[0503] In this case, the execution order of the first frame and the second frame may be controlled so that the operations are completely opposite.

[0504] As in the sixth embodiment, each of the pixel signals VQ1, VQ2, and VQ3 can be expressed by equations (36), (37), and (38). In this embodiment, the second frame for distance measurement also serves as a reference reflected light voltage acquisition frame. Therefore, the intensity of reflected light RL in the distance measurement frame and the reference reflected light voltage acquisition frame is the same, and the reflected light correction coefficient R_L becomes 1 (R_L=1).

[0505] That is, the first frame is a reference voltage acquisition frame for calculating the reference reference voltage VD of the reflected light. In addition, the second frame for measuring the distance is used as a reference reflected light voltage acquisition frame. Therefore, the reflected light total control charge voltage VCALL of the distance measurement frame and the reference reflected light total control charge voltage VPCALL are the same, so the numerator and denominator in formula (57) will be the same.

[0506] In this embodiment, similar to the sixth embodiment, the background light correction coefficient R_B can be expressed by formula (45). Since it is in a light-shielded state (i.e., a state without background light) or an extremely dark state (i.e., a state where the background light intensity is so small that it can be ignored), the background light correction coefficient R_B becomes 0 (R_B=0).

[0507] Once the background light correction coefficient R_B and the reflected light correction coefficient R_L are determined, similar to the sixth embodiment, Equations (36), (37) and (38) become three-variable linear simultaneous equations consisting of the background light control voltage VCB1 and the reflected light control voltages VCL2 and VCL3 that are desired to be determined. The background light control voltage VCB1 and the reflected light control voltages VCL2 and VCL3 can be strictly determined, so the distance from the distance image capturing device 1 (distance image sensor 32) to the subject S can be determined with higher accuracy.

[0508] Furthermore, this embodiment has been described using an example in which there are three pixel signal readout units RU. However, similarly to the seventh embodiment, the same solution can be obtained even when there are four pixel signal readout units RU. As described in the seventh embodiment, the same solution can be obtained even when there are five, six, or more pixel signal readout units RU. This allows the distance L between the subject S and the image capturing device 1 to be determined with high distance accuracy. In other words, according to this embodiment, even when there are N pixel signal readout units RU (N is an integer greater than or equal to 3), high distance accuracy can be achieved in distance measurement even when there is a large variation in the non-control charge.

[0509] While the embodiments of the present invention have been described in detail above with reference to the accompanying drawings, the specific configuration is not limited to these embodiments and includes designs and the like that do not depart from the scope of the present invention.

[0510] (Description of labels)

[0511] 1 Distance image capturing device; 2 Light source unit; 3 Light receiving unit; 4 Distance image processing unit; 21 Light source device; 22 Diffuser; 31 Lens; 32 Distance image sensor; 41 Timing control unit; 42 Distance calculation unit; 43 Correction parameter storage unit; 320 Light receiving pixel unit; 321 Pixel; 322 Control circuit; 323 Vertical scanning circuit; 324 Horizontal scanning circuit; 325 Pixel signal processing circuit; 326 Pixel driving circuit; C1, C2, C3 Charge storage capacitors; CS1, CS2, CS3 Charge storage unit; FD1, FD2, FD3 floating diffusions; G1, G2, G3 readout gate transistors; GD drain-gate transistor; O1, O2, O3 output terminals; P measurement space; PD photoelectric conversion element; PO light pulse (irradiation light); RL reflected light; RT1, RT2, RT3 reset gate transistors; RU1, RU2, RU3 pixel signal readout unit; S object; SF1, SF2, SF3 source follower gate transistors; SL1, SL2, SL3 selection gate transistors.

Claims

1. A distance image capturing device, comprising: a light source unit for irradiating irradiation light into a measurement space, wherein the measurement space is a space of a measurement object; a distance image sensor configured to receive light including reflected light from an object in the measurement space as incident light, accumulate charge generated by the incident light in each pixel, and generate a distance image consisting of the charge amount of the charge accumulated in each pixel; as well as A distance image processing unit is configured to store, from the charge amount in the distance image, a signal value based on non-controlled charge included in the charge amount, independently of control for accumulating the charge in the distance image sensor, and to correct the distance to the object in the space using the stored signal value.

2. The distance image capturing device according to claim 1, wherein: The distance image sensor includes a pixel circuit for performing control for accumulating charge in a charge accumulation unit in each pixel. The pixel circuit includes: a photoelectric conversion element for collecting the charge generated in response to the incident light; and the charge accumulation section is configured to accumulate the charge during a frame period; The distance image processing unit subtracts an adjustment voltage corresponding to the amount of the non-control charge from an input voltage corresponding to the amount of charge accumulated in the charge accumulation unit, and measures the distance between the distance image sensor and the object, wherein the non-control charge is charge that flows into the charge accumulation unit independently of control by the pixel circuit.

3. The distance image capturing device according to claim 2, wherein: The charge storage unit includes: At least one first charge accumulation unit is used to accumulate background light charges generated by receiving background light in the space; and two or more second charge accumulation units are used to accumulate reflected light charges generated by receiving the reflected light from the object of the irradiation light.

4. The distance image capturing device according to claim 3, wherein: The distance image capturing device includes a storage unit, which stores: a reference background light voltage obtained from the first charge accumulation section and the second charge accumulation section respectively, which is pre-measured in a state where the irradiation light is not irradiated under specific ambient light and causes each of the first charge accumulation section and the second charge accumulation section to accumulate the charge controlled by the pixel circuit; and a reference base voltage obtained from the first charge accumulation section and the second charge accumulation section respectively, which does not cause each of the first charge accumulation section and the second charge accumulation section to accumulate the charge controlled by the pixel circuit.

5. The distance image capturing device according to claim 4, wherein: The distance image sensor irradiates the measurement space with irradiation light, receives light including reflected light from an object in the measurement space as incident light, and accumulates charge generated by the incident light in each of the first charge accumulation unit and the second charge accumulation unit. The distance image processing unit divides the background light voltage generated by the charge accumulated in the first charge accumulation unit by the reference background light voltage measured in advance and stored in the storage unit to obtain an adjustment ratio, and multiplies each of the reference base voltages by the adjustment ratio to calculate the adjustment voltages respectively relative to the input voltage.

6. The distance image capturing device according to claim 3, wherein: The frame period includes each of a first frame period and a second frame period, In the first frame period, The distance image sensor irradiates the measurement space with irradiation light, receives light including reflected light from an object in the measurement space as incident light, and accumulates charge controlled by the pixel circuit in each of the first charge accumulation section and the second charge accumulation section. The distance image processing unit acquires the input voltage generated by the accumulated charge, In the second frame period, The distance image sensor irradiates the measurement space with irradiation light, receives light including reflected light from an object in the measurement space as incident light, and does not store charge controlled by the pixel circuit in each of the first charge storage unit and the second charge storage unit. The distance image processing section acquires an adjustment voltage corresponding to the non-control charge in each of the first charge accumulation section and the second charge accumulation section.

7. The distance image capturing device according to claim 3, wherein: The distance image capturing device includes a storage unit, which stores: a reference reflected light voltage obtained from the first charge accumulation unit and the second charge accumulation unit respectively, which is pre-measured in a state where the irradiation light is applied in a light-shielding environment, so that each of the first charge accumulation unit and the second charge accumulation unit accumulates the charge controlled by the pixel circuit; and a reference base voltage obtained from the first charge accumulation unit and the second charge accumulation unit respectively, so that each of the first charge accumulation unit and the second charge accumulation unit does not accumulate the charge controlled by the pixel circuit.

8. The distance image capturing device according to claim 7, wherein: The distance image processing unit calculates an adjustment ratio by dividing the result of adding the input voltages generated by the charges accumulated in each of the first charge accumulation unit and the second charge accumulation unit by the result of adding each of the reference reflected light voltages, and multiplies the reference base voltage by the adjustment ratio to calculate the adjustment voltage for each of the input voltages.

9. The distance image capturing device according to claim 3, wherein: The frame period includes each of a first frame period and a second frame period, In the first frame period, The distance image sensor irradiates the measurement space with irradiation light in a light-shielded environment, receives light including reflected light from an object in the measurement space as incident light, and accumulates charge controlled by the pixel circuit in each of the first charge accumulation section and the second charge accumulation section. The distance image processing unit acquires the input voltage generated by the charge accumulated by each of the first charge accumulation unit and the second charge accumulation unit, In the second frame period, The distance image sensor irradiates the measurement space with irradiation light in a light-shielded environment, receives light including reflected light from an object in the measurement space as incident light, and does not accumulate charge controlled by the pixel circuit in each of the first charge accumulation section and the second charge accumulation section. The distance image processing unit acquires an adjustment voltage corresponding to the non-control charge from each of the first charge storage unit and the second charge storage unit through the charge.

10. The distance image capturing device according to claim 3, wherein: The distance image capturing device includes a storage unit, which stores: a reference background light voltage obtained from each of the first charge accumulation section and the second charge accumulation section, respectively, measured in advance under a specific ambient light state without irradiation with the irradiation light, and causing each of the first charge accumulation section and the second charge accumulation section to accumulate the charge controlled by the pixel circuit; and a first reference reference voltage obtained from each of the first charge accumulation section and the second charge accumulation section as a reference reference voltage without causing each of the first charge accumulation section and the second charge accumulation section to accumulate the charge controlled by the pixel circuit; In addition, a reference reflected light voltage is obtained from the first charge accumulation unit and the second charge accumulation unit respectively, which is pre-measured in a state where the irradiation light is irradiated in a light-shielding environment, so that each of the first charge accumulation unit and the second charge accumulation unit accumulates the charge controlled by the pixel circuit; and a second reference reference voltage is obtained from the first charge accumulation unit and the second charge accumulation unit respectively as a reference reference voltage without causing each of the first charge accumulation unit and the second charge accumulation unit to accumulate the charge controlled by the pixel circuit.

11. The distance image capturing device according to claim 10, wherein: The distance image sensor irradiates the measurement space with irradiation light, receives light including reflected light from an object in the measurement space as incident light, and accumulates charge generated by the incident light in each of the pixels. The distance image processing unit divides the input voltage generated by the accumulated charge by the reference background light voltage measured in advance and stored in the storage unit to obtain a first adjustment ratio as an adjustment ratio, and multiplies each of the first reference voltages by the first adjustment ratio to calculate a first adjustment voltage corresponding to each of the input voltages; In addition, the second adjustment ratio is obtained by dividing the result of each addition of the input voltage by the result of each addition of the reference reflected light voltage, and the second reference base voltage is multiplied by the second adjustment ratio to calculate the second adjustment voltage relative to each of the input voltages, and the adjustment voltage is calculated by adding the first adjustment voltage and the second adjustment voltage.

12. The distance image capturing device according to claim 3, wherein: The distance image capturing device includes a storage unit, which respectively obtains: a reference background light voltage obtained from each of the N charge storage units, measured in advance under a specific ambient light condition without irradiation with the irradiation light, and causing each of the N charge storage units to accumulate the charge controlled by the pixel circuit, where N is an integer greater than or equal to 3; and a first reference voltage obtained from each of the charge storage units as a reference reference voltage without causing each of the charge storage units to accumulate the charge controlled by the pixel circuit; a reference reflected light voltage obtained from each of the N charge storage units, measured in advance when the irradiation light is applied in a light-shielded environment, and each of the N charge storage units accumulates the charge controlled by the pixel circuit; and a second reference voltage obtained from each of the charge storage units as a reference reference voltage without accumulating the charge controlled by the pixel circuit in each of the charge storage units; The storage units respectively store: a reference background light control voltage corresponding to the charge amount of the control charges generated by the background light, collected by the photoelectric conversion element, and respectively distributed to the N charge storage units for storage, obtained from each of the reference background light voltage and the first reference reference voltage; a reference background light non-control voltage corresponding to the charge amount of the non-control charges generated by the background light and not distributed to the N charge storage units and flowing in; a reference reflected light control voltage corresponding to the charge amount of the control charges generated by the reflected light, collected by the photoelectric conversion element, and respectively distributed to the N charge storage units and accumulated; and a reference reflected light non-control voltage corresponding to the charge amount of the non-control charges generated by the reflected light and not distributed but flowing into the N charge storage units and accumulated.

13. The distance image capturing device according to claim 12, wherein: The distance image shooting device includes a distance calculation unit, In the shooting state of the distance image, a first ratio is calculated and obtained as the ratio of the light amount of the background light in the pre-measured environment obtained from a part or all of the reference background light voltage, the first reference base voltage, the reference background light control voltage, and the reference background light non-control voltage stored in the storage unit to the light amount of the background light in the shooting state; a second ratio is calculated and obtained as the ratio of the light amount of the reflected light in the pre-measured environment obtained from a part or all of the reference reflected light voltage, the second reference base voltage, the reference reflected light control voltage, and the reference reflected light non-control voltage stored in the storage unit to the light amount of the reflected light in the shooting state; the simultaneous equations obtained using each of the first ratio and the second ratio are solved, the reflected light control voltage corresponding to the charge amount of the control charge generated by the reflected light in the shooting state and accumulated in the charge accumulation unit is calculated, and the distance to the object is obtained.

14. The distance image capturing device according to claim 3, wherein: The distance image capturing device includes a storage unit, which respectively obtains: a reference reflected light voltage corresponding to a charge amount including control charges accumulated by distribution and non-control charges flowing in and accumulated independently of the distribution, which is measured in advance in a state where the irradiation light is irradiated in a light-shielded environment, and which is obtained as a reference reference voltage corresponding to a charge amount of non-control charges flowing in and accumulated without performing the accumulation control on each of the N charge accumulation sections; and a second reference voltage obtained as a reference reference voltage corresponding to a charge amount of non-control charges flowing in and accumulated without performing the accumulation control on each of the charge accumulation sections; The storage units respectively store: A reference reflected light control voltage corresponding to the amount of control charge generated by the reflected light, collected by the photoelectric conversion element, and respectively distributed to the N charge storage units for storage is obtained from each of the reference reflected light voltage and the second reference base voltage; and a reference reflected light non-control voltage corresponding to the amount of non-control charge generated by the reflected light and not distributed but flowing into the N charge storage units and accumulated.

15. The distance image capturing device according to claim 14, wherein: The distance image shooting device includes a distance calculation unit, In an acquisition state of acquiring the distance, the frame period includes each of a first frame period and a second frame period, In one of the first frame period and the second frame period, The distance image sensor acquires a first reference voltage as a reference voltage corresponding to each charge amount of the non-control charges accumulated in each of the N charge accumulation units when the measurement space is not irradiated with the illumination light. In the other of the two frame periods, The distance image sensor irradiates the irradiation light onto the measurement space, controls the charge distribution of each of the N charge storage units, and obtains a voltage for determining the distance from each of the charge storage units, wherein the voltage corresponds to the charge amount including the control charge accumulated by the distribution and the non-control charge that flows in independently of the distribution. The sensor calculates and obtains the ratio of the amount of reflected light under the pre-measured environment obtained from part or all of the reference reflected light voltage, the second reference base voltage, the first reference base voltage, and the voltages for determining the distance stored in the storage unit, and the amount of reflected light in the acquisition state. The sensor solves simultaneous equations to calculate the reflected light control voltage corresponding to the charge amount of the control charge, and thereby determines the distance to the object.

16. The distance image capturing device according to claim 3, wherein: The distance image capturing device includes a distance calculation unit, When the distance is acquired in a light-shielded environment or a dark environment where ambient light is negligible, the frame period includes each of a first frame period and a second frame period. In one of the first frame period and the second frame period, The distance image sensor irradiates the illumination light and receives incident light from a measurement space, accumulates non-controlled charges flowing into each of the N charge accumulation units without accumulating charges controlled by the pixel circuit, and obtains a second reference voltage from each of the charge accumulation units. In the other of the two frame periods, Acquiring a reference reflected light voltage, which is a voltage corresponding to a distance corresponding to the amount of charge included in the non-controlled charge and the control charge controlled and accumulated by the pixel circuit, by irradiating the distance image sensor with the irradiation light and receiving the incident light from the measurement space, and distributing and accumulating the charge from the photoelectric conversion element in each of the charge accumulation sections under the control of the pixel circuit; The simultaneous equations using the second reference standard voltage and the reference reflected light voltage are solved to calculate only the reflected light control voltage corresponding to the charge amount of the control charge, thereby obtaining the distance to the object.

17. The distance image capturing device according to any one of claims 1 to 16, wherein: The distance image capturing device further includes a lens, the lens being configured to receive incident light from the space. The distance image sensor receives the incident light via the lens.

18. The distance image capturing device according to any one of claims 5, 8 or 11, wherein: The distance image capturing device further includes a lens, the lens being configured to receive incident light from the space. Since the incident light enters the pixels respectively through the lens, the pixels whose adjustment ratio is within a predetermined differential range are divided into groups according to the characteristics of the lens, and the middle value of the reference voltage in the group is used as the reference voltage for all pixels in the group.

19. The distance image capturing device according to any one of claims 5, 8, or 11, wherein: Since the incident light is incident on the pixels respectively via the lenses, a plurality of characteristics of the lenses are stored in the storage unit in correspondence with the characteristics of the lenses.

20. The distance image capturing device according to any one of claims 5, 8, or 11, wherein: Since the incident light is incident on the pixels respectively via lenses, an adjustment function is stored in the storage unit in accordance with the characteristics of the lenses, and the adjustment function outputs the adjustment ratio corresponding to the position of each pixel.

21. A method for capturing a range image, comprising: a distance image generating step of irradiating a measurement space as a space to be measured with irradiation light from a light source portion of a distance image sensor, receiving light including reflected light from an object in the measurement space as incident light, accumulating charge generated by the incident light in each pixel, and generating a distance image having a charge amount of the charge accumulated in each pixel; as well as A distance image processing step is performed to obtain the distance to the object in the space by correcting the charge amount, wherein the corrected charge amount removes non-control charges included in the charge amount that are independent of control for accumulating the charge in the distance image sensor from the charge amount in the distance image.

Images