Distance image capturing device and distance image capturing method

Abstract

To provide a distance image pickup device that, even when a distance calculated on the basis of an amount of electric charges accumulated in charge accumulation units has an error, can correct the distance to be close to an actual distance.SOLUTION: A distance image pickup device comprises: a light source unit; a light receiving unit that has pixels each provided with a photoelectric converter and a plurality of charge accumulation units, and a pixel drive circuit distributing electric charges to the charge accumulation units to cause the charge accumulation units to accumulate the electric charges; a distance image processing unit that determines a measured distance to a subject on the basis of an amount of electric charges accumulated in the charge accumulation units; and a storage unit that, for every first variable based on the amount of electric charges accumulated per unit accumulation number, stores table information indicating a relationship between a second variable based on a charge ratio and a corresponding distance. The distance image processing unit calculates the first variable and the second variable on the basis of the amount of electric charges accumulated in the charge accumulation units, selects the table information corresponding to the calculated first variable, acquires the corresponding distance corresponding to the second variable by using the selected table information, and determines the measured distance by using the acquired corresponding distance.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a distance image capturing device and a distance image capturing method. [Background technology]

[0002] Conventionally, there is a technology for measuring the time of flight of a light pulse as a technology for measuring the distance to an object. This technology is called Time of Flight (TOF). Taking advantage of the fact that the speed of light is known, TOF irradiates an object with a light pulse in the near infrared region. Then, it measures the time difference between the time when this light pulse is irradiated and the time when the reflected light of the irradiated light pulse reflected by the object is received. The distance to the object is calculated based on this time difference. Distance measuring sensors that use a photodiode (photoelectric conversion element) to detect light for measuring distance have been put into practical use.

[0003] In recent years, distance measuring sensors that can obtain not only the distance to an object but also depth information for each pixel in a two-dimensional image including the object, that is, three-dimensional information for the object, have been put to practical use. Such distance measuring sensors are also called distance image pickup devices. In a distance image pickup device, a plurality of pixels including photodiodes are arranged in a two-dimensional matrix on a silicon substrate, and the pixel surface receives light reflected by the object. In a distance image pickup device, a photoelectric conversion signal based on the amount of light (electric charge) received by each pixel is output for one image, thereby obtaining a two-dimensional image including the object and information on the distance for each pixel constituting this image. For example, Patent Document 1 discloses a technology in which three charge storage units are provided in one pixel, and the electric charges are distributed in order to calculate the distance. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 4235729 Summary of the Invention [Problem to be solved by the invention]

[0005] In such a distance image capturing device, the pixels receive the light reflected by the object, photoelectrically convert the amount of the received reflected light into an electric charge, accumulate the converted electric charge in the charge accumulation section, and calculate distance information based on the amount of accumulated electric charge. However, there are cases where the distance calculated from the amount of accumulated electric charge has an error with respect to the actual distance (real distance).

[0006] The present invention has been made in light of the above-mentioned problems, and aims to provide a distance image capturing device and a distance image capturing method that can correct the distance calculated based on the amount of charge accumulated in a charge accumulation section to approach the actual distance even if there is an error in the distance calculated based on the amount of charge accumulated in a charge accumulation section. [Means for solving the problem]

[0007] The distance image pickup device of the present invention comprises a light receiving section having a light source section which irradiates a measurement space in which a subject is present with a light pulse, a pixel having a photoelectric conversion element which generates a charge according to the incident light and three or more charge accumulation sections which accumulate the charge, and a pixel drive circuit which distributes and accumulates the charge in each of the charge accumulation sections in the pixel at a predetermined timing synchronized with the irradiation of the light pulse, a distance image processing section which determines a measurement distance to the subject based on the amount of charge accumulated in each of the charge accumulation sections, and a memory section which stores table information indicating, for each first variable based on the amount of charge accumulated per unit accumulation count, a relationship between a second variable based on a charge ratio and a corresponding distance corresponding to the distance to the subject, wherein the first variable is the second variable is a charge ratio indicated using a distance calculation charge amount obtained by subtracting an external light component from the charge amount accumulated in each of two or more distance calculation charge accumulation units among the plurality of charge accumulation units, which are allocated and accumulated with an amount of charge corresponding to the reflected light of the light pulse reflected by the subject, and the distance image processing unit calculates the first variable and the second variable based on the amount of charge accumulated in each of the charge accumulation units, selects the table information corresponding to the calculated first variable, uses the selected table information to obtain the corresponding distance corresponding to the calculated second variable, and determines the measured distance using the obtained corresponding distance.

[0008] In the distance image capturing device of the present invention, the charge ratio is the ratio of the distance calculation charge amount stored in one or more of the distance calculation charge accumulation units to the sum of the distance calculation charge amounts stored in each of the distance calculation charge accumulation units.

[0009] In the distance image capturing device of the present invention, the distance image processing unit calculates the first variable and the second variable based on the amount of charge accumulated in each of the charge accumulation units, selects the table information corresponding to the calculated first variable, extracts from the selected table information a first distance corresponding to a charge ratio smaller than the second variable and a second distance corresponding to a charge ratio larger than the second variable, and determines the measured distance by linearly interpolating the extracted first distance and second distance.

[0010] In the distance image capturing device of the present invention, the pixel has three or more charge accumulation sections, and the distance image processing unit determines the smallest amount of charge among the amounts of charge accumulated in each of the three or more charge accumulation sections as the amount of charge corresponding to the external light component.

[0011] In the distance image capturing device of the present invention, the pixel has three or more charge accumulation sections, and the distance image processing unit controls the timing of accumulating charge in the charge accumulation section for accumulating external light so that charge corresponding to the reflected light is not accumulated in a predetermined charge accumulation section for accumulating external light among the three or more charge accumulation sections, and sets the amount of charge accumulated in the charge accumulation section for accumulating external light to an amount of charge corresponding to the external light component.

[0012] In the distance image capturing device of the present invention, the memory unit stores first table information, which is the table information corresponding to the case where the first variable is smaller than a threshold value, and second table information, which is the table information corresponding to the case where the first variable is larger than the threshold value, the threshold value being a value determined according to a distance shift caused by charge transfer efficiency in the path from the photoelectric conversion element to the charge accumulation unit, and the distance image processing unit calculates the first variable and the second variable based on the amount of charge accumulated in each of the charge accumulation units, and selects the first table information when the calculated first variable is smaller than the threshold value, and selects the second table information when the calculated first variable is larger than the threshold value.

[0013] In the distance image capturing device of the present invention, the memory unit stores the table information for each combination of two of the distance calculation charge accumulation units, and the distance image processing unit determines as the combination of the distance calculation charge accumulation units a combination, among the three or more charge accumulation units, in which the sum of the amount of charge accumulated in each of two of the charge accumulation units that have consecutive timing for accumulating charge is greater than the amount of charge accumulated in each of the other two charge accumulation units, and selects the table information in accordance with the determined combination of the distance calculation charge accumulation units.

[0014] In the distance image capturing device of the present invention, the pixel is provided with three charge accumulation sections, namely a first charge accumulation section, a second charge accumulation section, and a third charge accumulation section, and the distance image processing section controls the charge to be accumulated in the first charge accumulation section, the second charge accumulation section, and the third charge accumulation section in that order, at a timing synchronized with the irradiation of the light pulse, calculates a first calculated value which is the difference between the first charge amount and the third charge amount using a first charge amount accumulated in the first charge accumulation section and a third charge amount accumulated in the third charge accumulation section, and sets the first calculated value as the distance calculation charge amount of either one of the two distance calculation charge accumulation sections.

[0015] In the range image pickup device of the present invention, the pixel is provided with four of the charge accumulation sections, namely a first charge accumulation section, a second charge accumulation section, a third charge accumulation section, and a fourth charge accumulation section, and the range image processing section controls the pixel drive circuit so that the charge is accumulated in the first charge accumulation section, the second charge accumulation section, the third charge accumulation section, and the fourth charge accumulation section in that order, at a timing synchronized with the irradiation of the light pulse, and calculates a difference between the first amount of charge and the third amount of charge by using a first amount of charge accumulated in the first charge accumulation section and a third amount of charge accumulated in the third charge accumulation section. a first calculated value is calculated, and a second calculated value which is the difference between the second amount of charge and the fourth amount of charge is calculated using the second amount of charge accumulated in the second charge accumulation unit and the fourth amount of charge accumulated in the fourth charge accumulation unit; the sum of the absolute value of the first calculated value and the absolute value of the second calculated value is set as the sum of the distance calculation charge amounts in the two distance calculation charge accumulation units; the first calculated value is set as the distance calculation charge amount of one of the two distance calculation charge accumulation units; and the second calculated value is set as the distance calculation charge amount of the other of the two distance calculation charge accumulation units.

[0016] The distance image capturing method of the present invention is a distance image capturing method using a distance image capturing device including: a light source unit that irradiates a measurement space in which a subject is present with a light pulse; a pixel having a photoelectric conversion element that generates a charge according to the incident light and a plurality of charge accumulation units that accumulate the charge; a light receiving unit having a pixel drive circuit that distributes and accumulates the charge in each of the charge accumulation units in the pixel at a predetermined timing synchronized with the irradiation of the light pulse; a distance image processing unit that determines a measurement distance to the subject based on the amount of charge accumulated in each of the charge accumulation units; and a memory unit that stores table information indicating, for each first variable based on the amount of charge accumulated per unit accumulation count, a relationship between a second variable based on a charge ratio and a corresponding distance corresponding to the distance to the subject. The first variable is a variable corresponding to the sum of the amount of charge accumulated in each of the charge accumulation sections per unit accumulation count, and the second variable is a charge ratio indicated using a distance calculation charge amount obtained by subtracting an external light component from the amount of charge accumulated in each of two distance calculation charge accumulation sections among the three or more charge accumulation sections, in which an amount of charge corresponding to reflected light of the light pulse reflected by the subject is allocated and accumulated, and the distance image processing section calculates the first variable and the second variable based on the amount of charge accumulated in each of the charge accumulation sections, selects the table information corresponding to the calculated first variable, obtains the corresponding distance corresponding to the calculated second variable using the selected table information, and determines the measured distance using the obtained corresponding distance. Effect of the Invention

[0017] According to the present invention, even if there is an error in the distance calculated based on the amount of charge accumulated in the charge accumulation section, it is possible to correct the distance so as to approach the actual distance. [Brief description of the drawings]

[0018] [Figure 1] 1 is a block diagram showing a schematic configuration of a distance image capturing device 1 according to an embodiment. [Diagram 2] FIG. 2 is a block diagram showing a schematic configuration of a range image sensor 32 according to the embodiment. [Diagram 3] FIG. 2 is a circuit diagram showing an example of a configuration of a pixel 321 according to an embodiment. [Figure 4] 11 is a timing chart showing an example of timing for driving a pixel 321 according to an embodiment. [Figure 5A] FIG. 4 is a diagram illustrating an example of the configuration of table information 440 according to the embodiment. [Figure 5B] FIG. 4 is a diagram illustrating an example of the configuration of table information 440 according to the embodiment. [Figure 5C] FIG. 4 is a diagram illustrating an example of the configuration of table information 440 according to the embodiment. [Figure 5D] FIG. 4 is a diagram illustrating an example of the configuration of table information 440 according to the embodiment. [Figure 6] FIG. 4 is a diagram illustrating table information 440 according to an embodiment. [Figure 7] 13 is a diagram illustrating a process in which distance image processing unit 4 of the embodiment performs linear interpolation using table information 440. FIG. [Figure 8] 5A to 5C are diagrams illustrating a process in which the distance image processing unit 4 of the embodiment determines an external light component. [Figure 9] 5A to 5C are diagrams illustrating a process in which the distance image processing unit 4 of the embodiment determines an external light component. [Figure 10] 5 is a flowchart showing the flow of processing performed by distance image processing unit 4 of the embodiment. [Figure 11] 11A to 11C are diagrams illustrating the effects of the embodiment. [Figure 12] FIG. 13 is a diagram showing an example of the configuration of table information 440A in the first modification of the embodiment. [Figure 13] FIG. 13 is a diagram showing an example of the configuration of table information 440B in a second modified example of the embodiment. [Figure 14] FIG. 13 is a diagram illustrating two time windows in a third modified example of the embodiment. [Figure 15] FIG. 13 is a diagram showing an example of the configuration of table information 440C in a third modified example of the embodiment. [Figure 16] FIG. 13 is a diagram showing an example of the configuration of table information 440D in a third modified example of the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Hereinafter, a range image pickup device according to an embodiment will be described with reference to the drawings.

[0020] <Embodiment> First, an embodiment will be described. Fig. 1 is a block diagram showing a schematic configuration of a distance image pickup device according to a first embodiment of the present invention. The distance image pickup device 1 shown in Fig. 1 comprises a light source unit 2, a light receiving unit 3, and a distance image processing unit 4. Fig. 1 also shows a subject OB, which is an object to which the distance is measured by the distance image pickup device 1.

[0021] The light source section 2 irradiates a light pulse PO into a space to be photographed, in which a subject OB, the distance of which is to be measured by the distance image pickup device 1, exists, under the control of the distance image processor 4. The light source section 2 is, for example, a surface-emitting semiconductor laser module such as a Vertical Cavity Surface Emitting Laser (VCSEL). The light source section 2 includes a light source device 21 and a diffusion plate 22.

[0022] The light source device 21 is a light source that emits laser light in a near-infrared wavelength band (for example, a wavelength band of 850 nm to 940 nm) that becomes the light pulse PO to be irradiated to the subject OB. The light source device 21 is, for example, a semiconductor laser light emitting element. The light source device 21 emits pulsed laser light in response to control from the timing control unit 41.

[0023] The diffusion plate 22 is an optical component that diffuses the laser light in the near-infrared wavelength band emitted by the light source device 21 to a surface area for irradiating the subject OB. The pulsed laser light diffused by the diffusion plate 22 is emitted as a light pulse PO and irradiated to the subject OB.

[0024] The light receiving unit 3 receives reflected light RL of the light pulse PO reflected by an object OB, the distance of which is to be measured in the range image pickup device 1, and outputs a pixel signal corresponding to the received reflected light RL. The light receiving unit 3 includes a lens 31 and a range image sensor 32.

[0025] The lens 31 is an optical lens that guides the incident reflected light RL to the range image sensor 32. The lens 31 outputs the incident reflected light RL to the range image sensor 32 side, and causes the light to be received (incident) by pixels provided in the light receiving region of the range image sensor 32.

[0026] The distance image sensor 32 is an imaging element used in the distance image capturing device 1. The distance image sensor 32 has a plurality of pixels in a two-dimensional light receiving area. Each pixel of the distance image sensor 32 is provided with one photoelectric conversion element, a plurality of charge accumulation units corresponding to this one photoelectric conversion element, and a component that distributes charge to each charge accumulation unit. In other words, the pixel is an imaging element with a distribution configuration that distributes and accumulates charge in the plurality of charge accumulation units.

[0027] The distance image sensor 32 distributes the charges generated by the photoelectric conversion elements to each charge accumulation section according to the control of the timing control section 41. The distance image sensor 32 also outputs a pixel signal according to the amount of charge distributed to the charge accumulation section. The distance image sensor 32 has a plurality of pixels arranged in a two-dimensional matrix, and outputs a pixel signal for one frame corresponding to each pixel.

[0028] The distance image processing unit 4 controls the distance image pickup device 1 and calculates the distance to the subject OB. The distance image processing unit 4 includes a timing control unit 41, a distance calculation unit 42, a measurement control unit 43, and a storage unit 44. Some of the functional units of the distance image processing unit 4 (the timing control unit 41, the distance calculation unit 42, the measurement control unit 43, and the storage unit 44) may be incorporated into the distance image sensor 32.

[0029] The timing control section 41 controls the timing of outputting various control signals required for measurement in accordance with the control of the measurement control section 43. The various control signals here include, for example, a signal for controlling the irradiation of the light pulse PO, a signal for distributing and storing the reflected light RL in a plurality of charge storage sections, and a signal for controlling the number of distributions (number of accumulations) per frame. The number of distributions is the number of times that the process of distributing electric charges to the charge storage sections CS (see FIG. 3) is repeated. The product of the number of charge distributions and the time (accumulation time Ta, described later) for accumulating electric charges in each charge storage section per charge distribution process is the exposure time.

[0030] The distance calculation unit 42 calculates the distance to the subject OB based on the pixel signal output from the distance image sensor 32, and outputs the calculated distance information. The distance calculation unit 42 calculates the distance to the subject OB based on the amount of charge accumulated in the multiple charge accumulation units.

[0031] In this embodiment, the distance calculation unit 42 determines the distance to the subject OB using table information 440, which will be described later. The table information 440 will be described in detail later. Also, a method in which the distance calculation unit 42 determines the distance to the subject OB using the table information 440 will be described in detail later.

[0032] The measurement control unit 43 controls the timing control unit 41. For example, the measurement control unit 43 sets the number of allocations for one frame, the accumulation time Ta, and the like, and controls the timing control unit 41 so that imaging is performed according to the set contents.

[0033] The storage unit 44 is configured by a storage medium, for example, a hard disk drive (HDD), a flash memory, an electrically erasable programmable read only memory (EEPROM), a random access read / write memory (RAM), a read only memory (ROM), or any combination of these storage media. The storage unit 44 stores, for example, table information 440. The table information 440 will be described in detail later.

[0034] With this configuration, in the distance image capturing device 1, the light source unit 2 irradiates a light pulse PO in the near-infrared wavelength band onto the subject OB, and the light receiving unit 3 receives the reflected light RL reflected by the subject OB, and the distance image processing unit 4 outputs distance information measuring the distance to the subject OB.

[0035] Although FIG. 1 shows the distance image pickup device 1 having the distance image processing section 4 therein, the distance image processing section 4 may be an element provided outside the distance image pickup device 1.

[0036] Next, there will be described the configuration of the distance image sensor 32 used as an imaging element in the distance image pickup device 1. Fig. 2 is a block diagram showing a schematic configuration of the imaging element (distance image sensor 32) used in the distance image pickup device 1 of the embodiment.

[0037] As shown in FIG. 2, the distance image sensor 32 includes, for example, a light receiving area 320 in which a plurality of pixels 321 are arranged, a control circuit 322, a vertical scanning circuit 323 having a distribution operation, a horizontal scanning circuit 324, and a pixel signal processing circuit 325.

[0038] The light receiving area 320 is an area in which a plurality of pixels 321 are arranged, and FIG. 2 shows an example in which the pixels are arranged in a two-dimensional matrix of 8 rows and 8 columns. The pixels 321 accumulate electric charges corresponding to the amount of light received. The control circuit 322 comprehensively controls the distance image sensor 32. The control circuit 322 controls the operation of the components of the distance image sensor 32 in response to, for example, an instruction from the timing control section 41 of the distance image processing section 4. Note that the components of the distance image sensor 32 may be directly controlled by the timing control section 41, in which case the control circuit 322 may be omitted.

[0039] The vertical scanning circuit 323 is a circuit that controls the pixels 321 arranged in the light receiving region 320 for each row in accordance with control from the control circuit 322. The vertical scanning circuit 323 outputs a voltage signal according to the amount of charge accumulated in each charge accumulation section CS of the pixels 321 to the pixel signal processing circuit 325. In this case, the vertical scanning circuit 323 distributes the charges converted by the photoelectric conversion elements to each charge accumulation section of the pixels 321. In other words, the vertical scanning circuit 323 is an example of a "pixel driving circuit."

[0040] The pixel signal processing circuit 325 is a circuit that performs predetermined signal processing (e.g., noise suppression processing, A / D conversion processing, etc.) on the voltage signals output from the pixels 321 in each column to the corresponding vertical signal lines in accordance with control from the control circuit 322.

[0041] Horizontal scanning circuit 324 is a circuit that sequentially outputs signals output from pixel signal processing circuit 325 to horizontal signal lines in accordance with control from control circuit 322. As a result, pixel signals equivalent to the amount of charge accumulated for one frame are sequentially output to distance image processing unit 4 via the horizontal signal lines.

[0042] In the following description, it is assumed that the pixel signal processing circuit 325 performs A / D conversion processing and the pixel signals are digital signals.

[0043] Here, the configuration of the pixel 321 arranged in the light receiving region 320 provided in the distance image sensor 32 will be described. Fig. 3 is a circuit diagram showing an example of the configuration of the pixel 321 arranged in the light receiving region 320 of the distance image sensor 32 of the embodiment. Fig. 3 shows an example of the configuration of one pixel 321 out of the multiple pixels 321 arranged in the light receiving region 320. The pixel 321 is an example of a configuration including three pixel signal readout units.

[0044] 3, the pixel 321 includes one photoelectric conversion element PD, a drain gate transistor GD, and three pixel signal readout units RU that output voltage signals from corresponding output terminals O. Each pixel signal readout unit RU includes a readout gate transistor G, a floating diffusion FD, a charge storage capacitance C, a reset gate transistor RT, a source follower gate transistor SF, and a selection gate transistor SL. In each pixel signal readout unit RU, the floating diffusion FD and the charge storage capacitance C form a charge storage unit CS.

[0045] 3, the three pixel signal readout units RU are distinguished from one another by adding the numbers "1," "2," or "3" after the reference numeral "RU" of each pixel signal readout unit RU. Similarly, the components of the three pixel signal readout units RU are distinguished from one another by adding the number representing each pixel signal readout unit RU after the reference numeral.

[0046] 3, the pixel signal readout unit RU1, which outputs a voltage signal from the output terminal O1, includes a readout gate transistor G1, a floating diffusion FD1, a charge storage capacitance C1, a reset gate transistor RT1, a source follower gate transistor SF1, and a selection gate transistor SL1. In the pixel signal readout unit RU1, the floating diffusion FD1 and the charge storage capacitance C1 form a charge storage unit CS1. The pixel signal readout units RU2 and RU3 have a similar configuration.

[0047] The photoelectric conversion element PD is a buried photodiode that photoelectrically converts incident light to generate electric charges and accumulates the generated electric charges. The structure of the photoelectric conversion element PD may be any. For example, the photoelectric conversion element PD may be a PN photodiode having a structure in which a P-type semiconductor and an N-type semiconductor are joined together, or a PIN photodiode having a structure in which an I-type semiconductor is sandwiched between a P-type semiconductor and an N-type semiconductor. In addition, the photoelectric conversion element PD is not limited to a photodiode, and may be, for example, a photogate type photoelectric conversion element.

[0048] In the pixel 321, the photoelectric conversion element PD photoelectrically converts the incident light to generate electric charges, which are then distributed to each of the three charge accumulation sections CS, and voltage signals corresponding to the amount of the distributed electric charges are output to the pixel signal processing circuit 325.

[0049] The configuration of the pixels arranged in the range image sensor 32 is not limited to the configuration including three pixel signal readout units RU as shown in Fig. 3, but may be any pixel including a plurality of pixel signal readout units RU. In other words, the number of pixel signal readout units RU (charge accumulation units CS) included in the pixels arranged in the range image sensor 32 may be two, or may be four or more.

[0050] 3 shows an example in which the charge storage unit CS is configured by a floating diffusion FD and a charge storage capacitance C. However, the charge storage unit CS only needs to be configured by at least a floating diffusion FD, and the pixel 321 may not have a charge storage capacitance C.

[0051] 3 shows an example of a configuration including a drain gate transistor GD, but is not limited to this. For example, if there is no need to discard the charge remaining in the photoelectric conversion element PD without being stored in the charge storage unit CS, the pixel 321 may be configured without the drain gate transistor GD.

[0052] Next, the driving timing of the pixel 321 will be described with reference to Fig. 4. Fig. 4 is a timing chart showing the timing of driving the pixel 321 of the embodiment.

[0053] In Fig. 4, the timing of irradiating the light pulse PO is indicated by "L", the timing of receiving the reflected light is indicated by "R", the timing of the drive signal TX1 is indicated by "G1", the timing of the drive signal TX2 is indicated by "G2", the timing of the drive signal TX3 is indicated by "G3", and the timing of the drive signal RSTD is indicated by "GD". The drive signal TX1 is a signal that drives the read gate transistor G1. The same applies to the drive signals TX2 and TX3.

[0054] As shown in Fig. 4, a light pulse PO is irradiated for an irradiation time To, and the reflected light RL is received by the range image sensor 32 after a delay time Td. The vertical scanning circuit 323 accumulates electric charges in the charge accumulation units CS1, CS2, and CS3 in this order in synchronization with the irradiation of the light pulse PO. In Fig. 4, the time from irradiating the light pulse PO to accumulating electric charges in the charge accumulation units CS in order in one allocation process is represented as a "unit accumulation period". After the allocation process performed in the "unit accumulation period" is repeated a number of times corresponding to one frame, a process of reading out the amount of electric charge accumulated during that time is performed. The time during which the process of reading out the amount of accumulated electric charge is performed is represented as a "readout period".

[0055] First, a case where reflected light RL from a nearby object is received will be described with reference to Fig. 4. The vertical scanning circuit 323 turns off the drain gate transistor GD and turns on the read gate transistor G1 in synchronization with the timing of irradiation of the light pulse PO. The vertical scanning circuit 323 turns off the read gate transistor G1 after the accumulation time Ta has elapsed since the read gate transistor G1 was turned on. As a result, the charge photoelectrically converted by the photoelectric conversion element PD while the read gate transistor G1 is controlled to be in the on state is accumulated in the charge accumulation section CS1 via the read gate transistor G1.

[0056] Next, the vertical scanning circuit 323 turns the read gate transistor G2 on for the accumulation time Ta at the same time as turning the read gate transistor G1 off. As a result, the charge photoelectrically converted by the photoelectric conversion element PD while the read gate transistor G2 is controlled to be on is accumulated in the charge accumulation section CS2 via the read gate transistor G2.

[0057] Next, the vertical scanning circuit 323 turns on the read gate transistor G3 at the timing when the accumulation of charges in the charge accumulation section CS2 is completed, and turns off the read gate transistor G3 after the accumulation time Ta has elapsed. As a result, the charges photoelectrically converted by the photoelectric conversion element PD while the read gate transistor G3 is controlled to be in the on state are accumulated in the charge accumulation section CS3 via the read gate transistor G3.

[0058] Next, the vertical scanning circuit 323 turns on the drain gate transistor GD to discharge the charge at the timing when the charge accumulation in the charge accumulation unit CS3 is completed, whereby the charge photoelectrically converted by the photoelectric conversion element PD is discarded via the drain gate transistor GD.

[0059] The vertical scanning circuit 323 repeats the above-mentioned driving for a predetermined number of allocations over one frame. After that, the vertical scanning circuit 323 outputs a voltage signal corresponding to the amount of charge allocated to each charge storage unit CS. Specifically, the vertical scanning circuit 323 turns on the selection gate transistor SL1 for a predetermined time, thereby outputting a voltage signal corresponding to the amount of charge accumulated in the charge storage unit CS1 from the output terminal O1 via the pixel signal readout unit RU1. Similarly, the vertical scanning circuit 323 turns on the selection gate transistors SL2 and SL3 in sequence, thereby outputting voltage signals corresponding to the amount of charge accumulated in the charge storage units CS2 and CS3 from the output terminals O2 and O3. Then, an electric signal corresponding to the amount of charge accumulated in each of the charge storage units CS for one frame is output to the distance calculation unit 42 via the pixel signal processing circuit 325 and the horizontal scanning circuit 324.

[0060] In the above, the light source unit 2 irradiates the light pulse PO at the timing when the read gate transistor G1 is turned on. However, the present invention is not limited to this. The light source unit 2 may irradiate at a timing such that the reflected light RL from at least a close object is received across the charge storage units CS1 and CS2. For example, the light source unit 2 may irradiate at a timing just before the read gate transistor G1 is turned on. In the above, the light pulse PO is irradiated for an irradiation time To that is the same length as the storage time Ta. However, the present invention is not limited to this. The irradiation time To and the storage time Ta may have different time intervals.

[0061] In a short-distance light receiving pixel as shown in Fig. 4, the amount of charge corresponding to the reflected light RL and external light components is allocated and stored in the charge storage units CS1 and CS2 due to the relationship between the timing of irradiating the light pulse PO and the timing of storing charges in each of the charge storage units CS. Also, the amount of charge corresponding to external light components such as background light is stored in the charge storage unit CS3. In this case, the charge storage units CS1 and CS2 are an example of a "charge storage unit for distance calculation." In this case, the amount of charge corresponding to the reflected light RL stored in the charge storage units CS1 and CS2 is an example of a "charge amount for distance calculation."

[0062] The distribution (distribution ratio) of the charge amounts distributed to the charge storage units CS1 and CS2 is a ratio according to the delay time Td from when the light pulse PO is reflected by the object OB until it is incident on the range image pickup device 1.

[0063] Using this principle, distance calculation section 42 calculates delay time Td in a conventional short-distance light receiving pixel by the following formula (1).

[0064] Td=To×(Q2-Q3) / (Q1+Q2-2×Q3) …(1)

[0065] Here, To is the period during which the light pulse PO is irradiated, Q1 is the amount of charge accumulated in the charge accumulation unit CS1, Q2 is the amount of charge accumulated in the charge accumulation unit CS2, and Q3 is the amount of charge accumulated in the charge accumulation unit CS3. Note that in formula (1), it is assumed that the amount of charge corresponding to the external light component among the amounts of charge accumulated in the charge accumulation units CS1 and CS2 is the same as the amount of charge accumulated in the charge accumulation unit CS3.

[0066] In the short-distance light receiving pixel, the distance calculation unit 42 calculates the round-trip distance to the subject OB by multiplying the delay time Td calculated by the formula (1) by the speed of light (velocity). Then, the distance calculation unit 42 calculates the distance to the subject OB by dividing the round-trip distance calculated above by 1 / 2.

[0067] Here, a cause of an error occurring in the distance (measured distance) calculated from the amount of accumulated charge in the conventional range image pickup device 1 will be described.

[0068] One of the causes of the error is thought to be a signal delay in the circuit of the distance image pickup device 1, which causes delays in the rising and falling edges of the rectangle, resulting in rounding of the waveform. In other words, such rounding of the light pulse and gate pulse and delays in charge transfer are factors that cause errors in the measured distance. To address this, a method is adopted in which an error is corrected using, for example, a correspondence table (an example of "table information") in which the charge ratio R is associated with the distance D. The distance here is the distance to the subject OB.

[0069] The charge ratio R here is the ratio of the amount of charge accumulated in each of the two charge accumulation units CS (charge accumulation units CS1 and CS2 in FIG. 4) to which the reflected light RL is allocated and accumulated. In this case, the charge ratio R is expressed by, for example, the following formula (2) or (3).

[0070] R = Q1# / (Q1#+Q2#) …(2) R = Q2# / (Q1#+Q2#) …(3) where Q1#=Q1-Qb Q2#=Q2-Qb Q1 is the amount of charge stored in the charge storage section CS1 Q2 is the amount of charge stored in the charge storage section CS2 Qb is the charge amount corresponding to the external light component stored in the charge storage section CS.

[0071] By using a correspondence table in which the charge ratio R is associated with the distance D, even if an error occurs in the distance calculated by the formula (1), it is possible to correct the distance so that the error is reduced. Note that a method for calculating the charge amount Qb corresponding to the external light component will be described in detail later.

[0072] Another possible cause of error is that the efficiency of charge transfer (transfer efficiency) differs depending on the absolute amount of charge generated in one photoelectric conversion. Here, transfer refers to the accumulation of charge from the photoelectric conversion element PD to each charge accumulation unit CS.

[0073] For example, when the intensity (amount of light) of light incident on the photoelectric conversion element PD is low, the transfer efficiency is lower than when the intensity of light incident on the photoelectric conversion element PD is high. This is thought to be because, when a potential pocket (hole) is formed in the transfer path, the electrons related to the charge being transferred are used to fill the pocket.

[0074] For example, when the intensity of light incident on the photoelectric conversion element PD is high, the amount of charge generated by the photoelectric conversion element PD is large. Therefore, even if electrons are used to fill the pockets, the total amount of charge hardly changes, and the transfer efficiency does not decrease significantly.

[0075] On the other hand, when the intensity of the light incident on the photoelectric conversion element PD is low, the amount of charge generated by the photoelectric conversion element PD is small. Therefore, when electrons are used to fill the pockets, the total amount of charge is significantly reduced, and the transfer efficiency decreases.

[0076] If a correspondence table in which the charge ratio R corresponds to the distance D is used, it is not possible to take such transfer efficiency into consideration. This is because the charge ratio R is a value calculated based on the amount of charge Q1 or Q2 accumulated per frame, and the amount of charge accumulated in one accumulation is not known. As a result, the measured distance differs from the actual distance, causing a distance discrepancy.

[0077] To deal with this, in this embodiment, table information 440 is created for each unit amount of accumulated charge Qint. The unit amount of accumulated charge Qint is the sum of the amount of charge accumulated in each charge accumulation section CS per unit accumulation (e.g., one accumulation). For example, the unit amount of accumulated charge Qint is expressed by the following formula (4).

[0078] Qint = QSUM / int …(4) where QSUM=Q1+Q2+Q3 Q1 is the amount of charge stored in the charge storage section CS1 Q2 is the amount of charge stored in the charge storage section CS2 Q3 is the amount of charge stored in the charge storage section CS3 int is the number of accumulations per frame

[0079] Alternatively, the table information 440 may be created for each unit number of electrons Nint instead of the unit amount of accumulated charge Qint. The unit number of electrons Nint is the sum of the number of electrons accumulated in each charge accumulation section CS per unit accumulation (e.g., one accumulation). For example, the unit number of electrons Nint is expressed by the following formula (5).

[0080] Nint = NSUM / int … (5) where NSUM=Q1 / CG1+Q2 / CG2+Q3 / CG3 NSUM is the sum of the number of electrons stored in the charge storage units CS1 to CS3 [e - ] int is the number of accumulations per frame Q1 is the amount of charge stored in the charge storage section CS1 [V] Q2 is the amount of charge stored in the charge storage section CS2 [V] Q3 is the amount of charge stored in the charge storage section CS3 [V] CG1 is the conversion gain [V / e - ] CG2 is the conversion gain [V / e - ] CG3 is the conversion gain [V / e - ]

[0081] The conversion gains CG1 to CG3 are coefficients for converting (transforming) electric charges into the number of electrons, and are values ​​determined in advance in consideration of, for example, the layout, etc. The conversion gains CG1 to CG3 may be different values ​​from one another, or all or some of the conversion gains CG1 to CG3 may be the same value.

[0082] In this manner, by configuring the table information 440 to be created for each unit accumulated charge amount Qint (or each unit accumulated electron number Nint), it becomes possible to select appropriate table information 440 according to the unit accumulated charge amount Qint (or each unit accumulated electron number Nint).

[0083] For example, when the unit accumulated charge amount Qint is large and the transfer efficiency is not reduced, the distance is corrected using normal table information 440. On the other hand, when the unit accumulated charge amount Qint is small and the transfer efficiency is reduced, the distance is corrected using table information 440 corresponding to the case where the transfer efficiency is low. This makes it possible to correct the distance with high accuracy even when the charge transfer efficiency differs. Here, the unit accumulated charge amount Qint is an example of a "first variable." The charge ratio R is an example of a "second variable."

[0084] Here, the table information 440 will be described with reference to Figs. 5A to 5D. Figs. 5A to 5D are diagrams showing an example of the configuration of the table information 440 of the embodiment. Fig. 5A shows table information 440 in which the charge ratio R is associated with the distance when the unit accumulated charge amount is Qint1. Fig. 5B shows table information 440 in which the charge ratio R is associated with the distance when the unit accumulated charge amount is Qint2. Fig. 5C shows table information 440 in which the charge ratio R is associated with the distance when the unit accumulated charge amount is Qint3. Fig. 5D shows table information 440 in which the charge ratio R is associated with the distance when the unit accumulated charge amount is Qint4.

[0085] In this example, even if the charge ratio is the same, that is, charge ratio R1, if the unit accumulated charge amount is Qint1, the distance is D11. On the other hand, if the unit accumulated charge amount is Qint3, the distance is D31. In this way, even if the charge ratio R is the same, the distance after calculation can be a different value depending on the amount of charge accumulated per accumulation process.

[0086] FIG. 6 is a diagram for explaining the table information 440 of the embodiment. FIG. 6 shows the relationship between the charge ratio R and the distance (indicated as the actual distance D in FIG. 6) at a certain unit accumulated charge amount Qint. As shown in FIG. 6, the relationship between the charge ratio R and the distance is not necessarily proportional, and may be nonlinear. In such a case, if the relationship between the charge ratio R and the distance is expressed by a function, it is considered that a complex multidimensional relational expression will be obtained. In such a case, a complex multidimensional relational expression must be calculated every time the distance is corrected, which increases the processing load. By making the relationship between the charge ratio R and the distance into a correspondence table as shown in the table information 440 of FIG. 5, even if the relationship between the charge ratio R and the distance D is nonlinear, it is possible to execute the processing related to the correction without increasing the processing load.

[0087] FIG. 7 is a diagram for explaining the process in which the distance image processing unit 4 of the embodiment performs linear interpolation using the table information 440. FIG. 7 shows a part of the characteristics of FIG. 6 (the range of charge ratios R1 to R2). In the example of this figure, a case is illustrated in which the charge ratio calculated based on the amount of charge accumulated in the photoelectric conversion element PD is a charge ratio R corresponding to an intermediate value between the charge ratios R1 and R2. In this case, the distance calculation unit 42 may calculate the distance D corresponding to the charge ratio R by linearly interpolating the distance D11 corresponding to the charge ratio R1 and the distance D12 corresponding to the charge ratio R2. By performing linear interpolation, it is possible to calculate the distance with higher accuracy.

[0088] Furthermore, the intervals of the charge ratios in the table information 440 may be set so that the accuracy of the interpolated distance falls within a predetermined range when linear interpolation is performed. In this case, the intervals of the charge ratios in the table information 440 are set, for example, within a range that can be regarded as linear.

[0089] A method for calculating the charge amount Qb corresponding to the external light component will now be described with reference to Figures 8 and 9. Figures 8 and 9 are diagrams illustrating the process in which distance image processing unit 4 of the embodiment determines the external light component.

[0090] (Method 1 for calculating the charge Qb equivalent to the external light component) Fig. 8 shows a timing chart when reflected light RL is received from an object at a longer distance than in Fig. 4. In Fig. 8, the timing at which the vertical scanning circuit 323 irradiates the light pulse PO, and the timing at which the readout gate transistors G1 to G3 and the drain gate transistor GD are turned on are the same as in Fig. 4, and therefore description thereof will be omitted.

[0091] As shown in Fig. 8, when the delay time Td is large compared to the timing chart of Fig. 4, if charges are accumulated in each of the charge accumulation units CS at the same timing as in Fig. 4, a charge amount Qb corresponding to the external light component is accumulated in the charge accumulation unit CS1, and the charge amount Qb corresponding to the reflected light RL and the external light component is allocated and accumulated in the charge accumulation units CS2 and CS3. In this case, the charge accumulation units CS2 and CS3 are examples of "distance calculation charge accumulation units".

[0092] That is, when the delay time Td is not large (as in FIG. 4), the charge accumulation unit CS3 accumulates the charge amount Qb corresponding to the external light component, and when the delay time Td is large (as in FIG. 8), the charge accumulation unit CS1 accumulates the charge amount Qb corresponding to the external light component. In either case of FIG. 4 or FIG. 8, the charge accumulation units CS1 to CS3 accumulate the same amount of charge amount Qb corresponding to the external light component. Therefore, the charge accumulation unit CS in which the reflected light RL is distributed and accumulated accumulates a larger amount of charge compared to the charge accumulation unit CS in which only the other external light components are accumulated.

[0093] Using this property, the distance calculation unit 42 determines the smallest amount of charge among the amounts of charge accumulated in the charge accumulation units CS1 to CS3 as the amount of charge Qb corresponding to the external light component.

[0094] The charge ratio R here is the ratio of the amount of charge accumulated in each of the two charge accumulation units CS (charge accumulation units CS2 and CS3 in FIG. 8) to which the reflected light RL is allocated and accumulated. In this case, the charge ratio R is expressed by, for example, the following formula (6) or (7).

[0095] R = Q2# / (Q2#+Q3#) …(6) R = Q3# / (Q2#+Q3#) …(7) where Q2#=Q2-Qb Q3#=Q3-Qb Q2 is the amount of charge stored in the charge storage section CS2 Q3 is the amount of charge stored in the charge storage section CS3 Qb is the charge amount corresponding to the external light component stored in the charge storage section CS.

[0096] (Method 2 for calculating the charge Qb equivalent to the external light component) The distance image capturing device 1 may control the timing so that only the charge amount Qb corresponding to the external light component is accumulated in a predetermined specific charge accumulation unit CS. In this case, the distance calculation unit 42 can determine the charge amount accumulated in the specific charge accumulation unit CS as the charge amount Qb corresponding to the external light component regardless of the length of the delay time Td.

[0097] Fig. 9 shows a timing chart in the case where control is performed so that only the charge amount Qb corresponding to the external light component is accumulated in the charge accumulation section CS1. In Fig. 9, the timing at which the vertical scanning circuit 323 irradiates the light pulse PO, and the timing at which the read gate transistors G1 to G3 and the drain gate transistor GD are turned on are the same as those in Fig. 4, and therefore the description thereof will be omitted.

[0098] As shown in the example of Fig. 9, before the light pulse PO is irradiated, charge is accumulated in the charge accumulation unit CS1, so that only the charge amount Qb corresponding to the external light component is accumulated in the charge accumulation unit CS1. In this case, the distance calculation unit 42 determines the amount of charge accumulated in the charge accumulation unit CS1 as the charge amount Qb corresponding to the external light component. In this case, the charge accumulation unit CS1 is an example of a "predetermined charge accumulation unit for accumulating external light."

[0099] 10 is a flowchart showing the flow of processing performed by distance image processing unit 4 in the embodiment. Distance calculation unit 42 acquires electric charge amounts Q1 to Q3 accumulated in charge accumulation units CS1 to CS3, respectively (step S10). Distance calculation unit 42 calculates electric charge amount Qb corresponding to the external light component using the acquired electric charge amounts Q1 to Q3 (step S11). Distance calculation unit 42 may determine the smallest electric charge amount among electric charge amounts Q1 to Q3 as electric charge amount Qb, or may determine the electric charge amount accumulated in a predetermined specific electric charge accumulation unit CS (for example, charge accumulation unit CS1 in FIG. 9) as electric charge amount Qb.

[0100] The distance calculation unit 42 calculates the charge ratio R using the electric charges Q1 to Q3 and the electric charge Qb (step S12). The distance calculation unit 42 calculates the charge ratio R, for example, by substituting the electric charges Q1 to Q3 and the electric charge Qb into the formula (2) or (3). In this case, the distance calculation unit 42 selects either the formula (2) or the formula (3) so as to match the configuration of the charge ratios shown in the table information 440, and calculates the charge ratio R using the selected formula.

[0101] The distance calculation unit 42 calculates the unit accumulated charge amount Qint using the electric charges Q1 to Q3 and the accumulation count int (step S13). The accumulation count int is a predetermined value, and is stored, for example, in the storage unit 44. In this case, the distance calculation unit 42 reads out the accumulation count int from the storage unit 44, and calculates the unit accumulated charge amount Qint by substituting the electric charges Q1 to Q3 and the read accumulation count int into formula (4).

[0102] The distance calculation unit 42 selects table information 440 corresponding to the unit accumulated charge amount Qint using the unit accumulated charge amount Qint calculated in step S13 (step S14). The distance calculation unit 42 selects a distance corresponding to the charge ratio R calculated in step S12 using the selected table information 440. In this case, the distance calculation unit 42 may select two distances respectively corresponding to the two charge ratios for linear interpolation, or may select one distance corresponding to the charge ratio closest to the charge ratio R calculated in step S12.

[0103] When the distance calculation unit 42 selects two distances (step S16, YES), it determines the distance (measured distance) by linear interpolation (step S17). On the other hand, when the distance calculation unit 42 selects one distance (step S16, NO), it determines the selected distance as the corrected distance (measured distance) (step S18).

[0104] In the above, an example has been described in which the pixel 321 of the distance image pickup device 1 includes three charge accumulation units CS1 to CS3. However, this is not limited to this. The pixel 321 of the distance image pickup device 1 may also include four or more (for example, N, where N≧4) charge accumulation units CS.

[0105] 10, the validity / invalidity of the calculation may be determined. For example, the distance calculation unit 42 invalidates the calculation when the charge ratio R obtained in step S12 exceeds a predetermined upper threshold (e.g., 0.98, etc.). The distance calculation unit 42 invalidates the calculation when the charge ratio R obtained in step S12 is below a predetermined lower threshold (e.g., 0.12, etc.).

[0106] When the pixel 321 of the distance image pickup device 1 has N (N≧4) charge accumulation units CS, in step S10 the distance calculation unit 42 acquires the amounts of electric charges Q1 to QN accumulated in the charge accumulation units CS1 to CSN, respectively. In step S11, the distance calculation unit 42 uses the acquired amounts of electric charges Q1 to Q3 to calculate the amount of electric charge Qb corresponding to the external light component. The method by which the distance calculation unit 42 calculates the amount of electric charge Qb is the same as when the pixel 321 of the distance image pickup device 1 has three charge accumulation units CS1 to CS3.

[0107] In step S12, the distance calculation unit 42 selects two charge accumulation units CS (distance measurement charge accumulation units) in which charges corresponding to the reflected light RL are distributed and accumulated from the charge accumulation units CS1 to CSN. The method of selecting the two charge accumulation units CS by the distance calculation unit 42 is, for example, to select the two charge accumulation units CS in which the sum of the charges accumulated in the charge accumulation units CS is the largest among the combinations of two charge accumulation units CS in which charges corresponding to the reflected light RL are distributed and accumulated (distance measurement charge accumulation units). The distance calculation unit 42 calculates the charge ratio R using the charge amount accumulated in each of the two charge accumulation units CS in which charges corresponding to the reflected light RL are distributed and accumulated, and the charge amount Qb corresponding to the external light component. The processes in steps S13 to S18 are similar to those in the case where the pixel 321 of the distance image pickup device 1 includes three charge accumulation units CS1 to CS3.

[0108] Alternatively, the pixel 321 of the distance image pickup device 1 may be configured to include two charge accumulation units CS. In this case, the distance image pickup device 1 performs two processes related to charge accumulation for each measurement: a process (called a first process) for accumulating only charges corresponding to external light components, and a process (called a second process) for accumulating charges including reflected light RL. For example, the distance image pickup device 1 performs the first process in the first frame, and the second process in the next frame. When performing the first process, the distance image pickup device 1 accumulates charges in each of the charge accumulation units CS1 and CS2 without irradiating the light pulse PO. When performing the second process, the distance image pickup device 1 accumulates charges in each of the charge accumulation units CS1 and CS2 by irradiating the light pulse PO.

[0109] In this case, in step S10, the distance calculation unit 42 acquires the charge amounts Q1f and Q2f accumulated in the charge accumulation units CS1 and CS2, respectively, in the first process. In addition, the distance calculation unit 42 acquires the charge amounts Q1s and Q2s accumulated in the charge accumulation units CS1 and CS2, respectively, in the second process. In step S11, the distance calculation unit 42 sets either the acquired charge amount Q1f or Q2f as the charge amount Qb corresponding to the external light component. In step S12, the distance calculation unit 42 calculates the charge ratio R using the acquired charge amounts Q1s, Q2s, and charge amount Qb. The method of calculating the charge ratio R by the distance calculation unit 42 is the same as when the pixel 321 of the distance image pickup device 1 includes three charge accumulation units CS1 to CS3.

[0110] As described above, the distance image pickup device 1 of the embodiment includes a light source unit 2, a light receiving unit 3, and a distance image processing unit 4. The light source unit 2 irradiates a measurement space in which an object OB exists with a light pulse PO. The light receiving unit 3 includes a pixel 321 and a vertical scanning circuit 323 (an example of a driving circuit). The pixel 321 includes a photoelectric conversion element PD and a plurality of charge accumulation units CS. The vertical scanning circuit 323 distributes and accumulates electric charge in each of the charge accumulation units CS in the pixel 321 at a predetermined timing synchronized with the irradiation of the light pulse PO. The distance image processing unit 4 includes a memory unit 44. The memory unit 44 stores table information 440. The table information 440 is information indicating the relationship between the charge ratio R (an example of the second variable) and the distance (corresponding distance) for each unit accumulated charge amount Qint (an example of the first variable). The unit accumulated charge amount Qint is a variable expressed by equation (4) or equation (5). The unit accumulated charge amount Qint is a variable corresponding to the sum of the charge amounts accumulated in each of the charge accumulation units CS per unit accumulation. The charge ratio R is a variable expressed by formula (2) or (3). The charge ratio R is a ratio indicated by using the charge amount obtained by subtracting the external light component (charge amount Qb) from the charge amount (e.g., charge amounts Q1 and Q2 in FIG. 4) accumulated in each of two or more distance calculation charge accumulation units (e.g., charge accumulation units CS1 and CS2 in FIG. 4). The charge ratio is a ratio of the distance calculation charge amount by any one or a combination of two or more distance calculation charge accumulation units. For example, the charge ratio is a ratio of one of the charge amounts (Q1# or Q2#) to the charge amount (Q1#+Q2#) according to the reflected light RL, that is, a variable expressed by formula (2) or (3). The distance image processing unit 4 calculates the unit accumulated charge amount Qint and the charge ratio R based on the charge amount accumulated in each of the charge accumulation units CS. Distance image processing unit 4 selects table information 440 corresponding to the calculated unit accumulated charge amount Qint. Distance calculation unit 42 uses the selected table information 440 to obtain a distance (corresponding distance) corresponding to the calculated charge ratio R. Distance calculation unit 42 uses the obtained corresponding distance to determine a corrected distance (measured distance).

[0111] As a result, the distance image pickup device 1 of the embodiment can obtain a distance corresponding to the charge ratio R using the table information 440, and even if there is an error in the distance calculated based on the charges Q1 to Q3 stored in the charge storage units CS1 to CS3, it can correct the distance to approach the actual distance. Moreover, in this embodiment, the table information 440 is created for each unit stored charge amount Qint. Therefore, it is possible to select appropriate table information 440 according to the unit stored charge amount Qint. Therefore, even if the transfer efficiency varies greatly depending on the absolute value of the charge amount stored per storage process, the distance can be corrected with high accuracy.

[0112] Furthermore, in the distance image pickup device 1 of the embodiment, the distance image processing unit 4 determines the corrected distance by linear interpolation. The distance image processing unit 4 uses the table information 440 to extract a first distance (e.g., distance D11) associated with a charge ratio (e.g., charge ratio R1) smaller than the calculated charge ratio R, and a second distance (e.g., distance D12) associated with a charge ratio (e.g., charge ratio R2) larger than the charge ratio R. The distance image processing unit 4 determines the distance obtained by linearly interpolating the extracted first distance (e.g., distance D11) and second distance (e.g., distance D12) as the corrected distance (measured distance). This allows the distance to be corrected more accurately in the distance image pickup device 1 of the embodiment.

[0113] Furthermore, pixel 321 of the embodiment includes three or more charge accumulation units CS1-CS3. Distance image processing unit 4 sets the smallest amount of charge among the amounts of charge accumulated in each of charge accumulation units CS1-CS3 as charge amount Qb corresponding to the external light component. This allows distance image capturing device 1 of the embodiment to calculate charge amount Qb corresponding to the external light component by the simple process of comparing the amounts of charge accumulated in each of charge accumulation units CS1-CS3.

[0114] Moreover, pixel 321 of the embodiment includes three or more charge accumulation units CS1-CS3. Distance image processing unit 4 controls the accumulation timing so that only the amount of light corresponding to the external light component is accumulated in a specific charge accumulation unit CS among charge accumulation units CS1-CS3. Distance calculation unit 42 sets the amount of charge accumulated in this specific charge accumulation unit CS as charge amount Qb corresponding to the external light component. As a result, in distance image capturing device 1 of the embodiment, the amount of charge accumulated in a specific charge accumulation unit CS can be calculated as charge amount Qb corresponding to the external light component, and charge amount Qb corresponding to the external light component can be easily determined.

[0115] (Effects of the embodiment) Here, the effect of the distance image pickup device 1 of the embodiment will be described with reference to FIG. 11. FIG. 11 is a diagram for explaining the effect of the embodiment. FIG. 11 shows the relationship between the actual distance (real distance) and the measured distance. The horizontal axis of FIG. 11 shows the actual distance, and the vertical axis shows the measured distance. The distance here is the distance to the object OB. In FIG. 11, the measured distance when the table information 440 is not used (table information not used), which is indicated by a black circle, is the distance calculated by substituting the electric charge amounts Q1 to Q3 into the formula (1). The measured distance when the table information 440 is used (table information used), which is indicated by a black triangle, is the distance calculated using the table information 440. As shown in this figure, the measured distance when the table information is used matches the actual distance. On the other hand, the measured distance when the table information is not used does not match the actual distance, and is a value including an error. That is, in the distance image pickup device 1 of the embodiment, it is possible to calculate a value closer to the actual distance by determining the measured distance using the table information 440.

[0116] (First Modification of the Embodiment) Here, a first modification of the embodiment will be described. This modification differs from the above-described embodiment in that the storage unit 44 stores two pieces of table information 440 separated by the threshold value Th.

[0117] Fig. 12 is a diagram showing an example of the configuration of table information 440A in the first modified example of the embodiment. As shown in Fig. 12, the upper part of the table information 440A shows the relationship between the charge ratio R and the distance D when the unit accumulated charge amount Qint is less than the threshold value Th. The lower part of the table information 440A shows the relationship between the charge ratio R and the distance D when the unit accumulated charge amount Qint is equal to or greater than the threshold value Th. As shown in the example of this figure, the table information 440 (440A) may be a two-dimensional correspondence table with the unit accumulated charge amount Qint and the charge ratio R as variables.

[0118] As already explained, the transfer efficiency depends on the absolute amount of charge transferred in one accumulation process. Therefore, it is considered that the transfer efficiency decreases when a charge amount less than a certain threshold Th is transferred, and the transfer efficiency does not decrease when a charge amount equal to or greater than the threshold is transferred. In this modification, two pieces of table information 440 (upper and lower parts in table information 440A) separated by such a threshold Th are created in advance and stored in the storage unit 44. The distance calculation unit 42 compares the calculated unit accumulated charge amount Qint with the threshold Th, and selects one of the upper and lower parts of the two pieces of table information 440A according to the comparison result. Specifically, when the unit accumulated charge amount Qint is less than the threshold Th, the distance calculation unit 42 selects the correspondence table in the upper part of the table information 440A. When the unit accumulated charge amount Qint is equal to or greater than the threshold Th, the distance calculation unit 42 selects the correspondence table in the lower part of the table information 440A. The distance calculation unit 42 determines the distance using the selected correspondence table.

[0119] As described above, in the distance image pickup device 1 according to the first modification of the embodiment, the storage unit 44 stores two pieces of information: information corresponding to the upper part of the table information 440A (an example of "first table information") and information corresponding to the lower part of the table information 440A (an example of "second table information"). The information corresponding to the upper part of the table information 440A is information corresponding to the case where the unit accumulated charge amount Qint is smaller than the threshold value Th. The information corresponding to the lower part of the table information 440A is information corresponding to the case where the unit accumulated charge amount Qint is larger than the threshold value Th. The threshold value Th is a value determined according to the transfer efficiency of the charge in the path from the photoelectric conversion element PD to the charge accumulation unit CS. The distance image processing unit 4 calculates the unit accumulated charge amount Qint and the charge ratio R based on the amount of charge accumulated in each of the charge accumulation units CS. The distance image processing unit 4 selects the information corresponding to the upper part of the table information 440A when the calculated unit accumulated charge amount Qint is smaller than the threshold value Th. Distance image processing unit 4 selects information corresponding to the lower row of table information 440A when the calculated unit accumulated charge amount Qint is greater than threshold value Th.

[0120] As a result, in the distance image capturing device 1 according to the first variant embodiment, one of two pieces of table information 440 corresponding to good and bad charge transfer efficiency can be selected, making it possible to determine the distance with high accuracy even when the number of pieces of table information 440 stored in the memory unit 44 is small.

[0121] (Modification 2 of the embodiment) Here, a second modification of the embodiment will be described. This modification differs from the above-described embodiment in that the storage unit 44 stores table information 440 corresponding to the range of the unit amount of accumulated charge Qint.

[0122] 13 is a diagram showing an example of the configuration of table information 440B in Modification 2 of the embodiment. The upper part of the table information 440B shows the relationship between the charge ratio R and the distance D when the unit accumulated charge amount Qint is equal to or greater than the threshold value Th1 and less than the threshold value Th2. The lower part of the table information 440B shows the relationship between the charge ratio R and the distance D when the unit accumulated charge amount Qint is equal to or greater than the threshold value Th2 and less than the threshold value Th3.

[0123] In this modification, the range of the unit accumulated charge amount Qint in the table information 440B corresponds to a range having a similar tendency of the transfer efficiency, for example. The distance calculation unit 42 judges which range of the unit accumulated charge amount Qint in the table information 440B the calculated unit accumulated charge amount Qint corresponds to, and selects one of the table information 440B according to the judgment result. The distance calculation unit 42 determines the distance using the selected correspondence table. This makes it possible to select the table information 440 according to the tendency of the charge transfer efficiency, and to determine the distance with high accuracy.

[0124] (Modification 3 of the embodiment) Here, a third modification of the embodiment will be described. This modification is different from the above-described embodiment in that the storage unit 44 stores table information 440 for each of a plurality of time windows. The time windows here correspond to combinations of distance measurement charge accumulation units.

[0125] 4, the combination of the distance measurement charge accumulation units is the charge accumulation units CS1 and CS2, and this combination corresponds to the first time window. The distance D is determined according to the charge ratio R of the charges distributed and accumulated in this first time window.

[0126] 8, the combination of the distance measurement charge accumulation units is the charge accumulation units CS2 and CS3. This combination corresponds to the second time window. The distance D is determined according to the charge ratio R of the charges distributed and accumulated in this second time window.

[0127] FIG. 14 is a diagram for explaining the characteristics of two time windows in the third modified example of the embodiment. The characteristics here are characteristics showing the correspondence between the actual distance and the measured distance. The horizontal axis of FIG. 14 shows the actual distance, and the vertical axis shows the measured distance. The characteristic L0 shows the ideal relationship between the actual distance and the measured distance. The characteristic L1 shows the relationship between the actual distance and the measured distance in the first time window. The characteristic L2 shows the relationship between the actual distance and the measured distance in the second time window. As shown in the example of this figure, the correspondence between the actual distance and the measured distance in one time window is often different from that in another time window. For this reason, when the distance in one time window is corrected using table information 440 that can perform accurate correction in another time window, it is not necessarily possible to perform accurate correction.

[0128] To address this issue, in this modification, table information 440 for each time window is created in advance in the storage unit 44 and stored in the storage unit 44 .

[0129] 15 and 16 are diagrams showing an example of the configuration of table information in the third modified example of the embodiment. FIG. 15 shows table information 440C used for a combination of charge accumulation units for distance measurement corresponding to a certain time window (for example, charge accumulation units CS1 and CS2). FIG. 16 shows table information 440D used for a combination of charge accumulation units for distance measurement corresponding to another time window (for example, charge accumulation units CS2 and CS3). As shown in FIG. 15 and FIG. 16, in the table information 440C (400D), even if the unit accumulated charge amount Qint and the charge ratio R have the same value, different distances D are associated when the time windows are different. In this way, by creating table information 440 for each time window, it is possible to perform correction according to each time window.

[0130] As described above, in distance image pickup device 1 according to the third modification of the embodiment, storage unit 44 stores table information 440C (400D) for each time window (combination of two charge accumulation units for distance calculation). Distance image processing unit 4 determines a combination of two charge accumulation units for distance calculation based on the amount of charge accumulated in each of the charge accumulation units CS. Distance image processing unit 4 determines, as a combination of charge accumulation units for distance calculation, a combination in which the sum (Q1+Q2) of the amount of charge accumulated in each of two charge accumulation units CS (e.g., charge accumulation units CS1 and CS2) that accumulate charge consecutively among charge accumulation units CS1-CS3 is greater than the amount of charge accumulated in each of the other two charge accumulation units CS. Distance image processing unit 4 selects table information 440C (400D) according to the determined combination of charge accumulation units for distance calculation.

[0131] As a result, in the distance image capturing device 1 of the third embodiment, table information 440 can be selected according to the time window, and even if the correspondence between the actual distance and the measured distance tends to differ for each time window, table information 440 suitable for each time window can be selected, making it possible to determine the distance with high accuracy.

[0132] (Fourth Modification of the Embodiment) Here, a fourth modification of the embodiment will be described. This modification is different from the above-described embodiment in that the charge ratio R is calculated without specifying the charge storage unit CS in which only the external light is stored, or the two charge storage units CS (distance calculation charge storage units) in which the reflected light RL is divided and stored.

[0133] In this modified example, the distance calculation unit 42 uses the method described in Patent Document WO2019 / 031510. Patent Document WO2019 / 031510 describes a technology in which an arithmetic equation used for distance calculation is selected depending on whether an index value exceeds a predetermined threshold value. The index value here is the "distance data validity determination signal" in Patent Document WO2019 / 031510. The arithmetic equation here is the "distance reference value" in Patent Document WO2019 / 031510, which corresponds to the "charge ratio R" in this embodiment. Below, a specific method for calculating the charge ratio R will be described separately for a case in which the pixel 321 has three charge storage units CS and a case in which the pixel 321 has four charge storage units CS.

[0134] (When the pixel 321 has three charge storage units CS) In this modification, the distance calculation unit 42 calculates the charge ratio R using the following formula (8) or formula (9). Here, charges are stored in the charge storage units CS1, CS2, and CS3 in this order according to the timing charts shown in FIG. 4 and FIG. 8. That is, the distance calculation unit 42 controls the charge storage units CS1, CS2, and CS3 to store charges in this order at a timing synchronized with the irradiation of the light pulse PO. In this case, the charge storage unit CS1 is an example of a "first charge storage unit." The charge storage unit CS2 is an example of a "second charge storage unit." The charge storage unit CS3 is an example of a "third charge storage unit." The amount of charge stored in the charge storage unit CS1 is an example of a "first charge amount." The amount of charge stored in the charge storage unit CS2 is an example of a "second charge amount." The amount of charge stored in the charge storage unit CS3 is an example of a "third charge amount."

[0135] R=1-(Q1-Q3) / SA…(8) R=(Q1-Q3) / SA…(9) however, SA=|Q1-Q3|+Q2-0.5×SB SB=|Q1+Q3|-|Q1-Q3| Q1 is the amount of charge stored in the charge storage section CS1 Q2 is the amount of charge stored in the charge storage section CS2 Q3 is the amount of charge stored in the charge storage section CS3

[0136] The storage unit 44 stores table information 440 in which the charge ratio R is associated with the distance for each unit amount of accumulated charge Qint. The distance calculation unit 42 calculates the unit amount of accumulated charge Qint, and determines the distance associated with the charge ratio R using the table information 440 corresponding to the calculated unit amount of accumulated charge Qint as the measured distance.

[0137] (When the pixel 321 has four charge storage units CS) First, the driving timing of the pixel 321 in the case where the pixel 321 has four charge storage units CS will be described. In this case, for example, an item of a read gate transistor G4 is further added to FIG. 4 and FIG. 8, and charges are stored in the order of the charge storage units CS1, CS2, CS3, and CS4. In this case, the charge storage unit CS1 is an example of a "first charge storage unit". The charge storage unit CS2 is an example of a "second charge storage unit". The charge storage unit CS3 is an example of a "third charge storage unit". The charge storage unit CS4 is an example of a "fourth charge storage unit". In addition, the amount of charge stored in the charge storage unit CS1 is an example of a "first charge amount". The amount of charge stored in the charge storage unit CS2 is an example of a "second charge amount". The amount of charge stored in the charge storage unit CS3 is an example of a "third charge amount". The amount of charge stored in the charge storage unit CS4 is an example of a "fourth charge amount".

[0138] Specifically, at the timings shown in FIG. 4 and FIG. 8, the light pulse PO is irradiated, the drain gate transistor GD is controlled to be in the OFF state, and the read gate transistors G1 to G3 are controlled to be in the ON state. Next, the vertical scanning circuit 323 turns the read gate transistor G4 to the ON state at the timing when the accumulation of the charge in the charge accumulation section CS3 is completed, and turns the read gate transistor G4 to the OFF state after the accumulation time Ta has elapsed. As a result, the charge photoelectrically converted by the photoelectric conversion element PD while the read gate transistor G4 is controlled to be in the ON state is accumulated in the charge accumulation section CS4 via the read gate transistor G4. Next, the vertical scanning circuit 323 turns the drain gate transistor GD to the ON state at the timing when the accumulation of the charge in the charge accumulation section CS4 is completed, and discharges the charge. As a result, the charge photoelectrically converted by the photoelectric conversion element PD is discarded via the drain gate transistor GD.

[0139] Based on the amount of charge accumulated in each charge accumulation unit CS controlled at the above-mentioned timing, the distance calculation unit 42 calculates the charge ratio XR using the following formula (10) or (11).

[0140] XR=1-(Q1-Q3) / SA…(10) XR=(Q1-Q3) / SA…(11) however, SA=|Q1-Q3|+|Q2-Q4| Q1 is the amount of charge stored in the charge storage section CS1 Q2 is the amount of charge stored in the charge storage section CS2 Q3 is the amount of charge stored in the charge storage section CS3 Q4 is the amount of charge stored in the charge storage section CS4

[0141] Further, the distance calculation unit 42 calculates the charge ratio YR using the following equation (12) or (13).

[0142] YR=2-(Q2-Q4) / SA …(12) YR = 1 + (Q2 - Q4) / SA … (13) however, SA=|Q1-Q3|+|Q2-Q4| Q1 is the amount of charge stored in the charge storage section CS1 Q2 is the amount of charge stored in the charge storage section CS2 Q3 is the amount of charge stored in the charge storage section CS3 Q4 is the amount of charge stored in the charge storage section CS4

[0143] The distance calculation unit 42 compares the charge ratio XR with a threshold ThR. The threshold ThR is set to a value close to the value of the charge ratio XR corresponding to the vicinity of the time window switching. When the charge ratio XR is equal to or less than the threshold ThR, the distance calculation unit 42 selects the charge ratio XR as the charge ratio R. On the other hand, when the charge ratio XR exceeds the threshold ThR, the distance calculation unit 42 selects the charge ratio YR as the charge ratio R.

[0144] The storage unit 44 stores table information 440 in which the charge ratio R is associated with the distance for each unit amount of accumulated charge Qint. The distance calculation unit 42 calculates the unit amount of accumulated charge Qint, and determines the distance associated with the charge ratio R using the table information 440 corresponding to the calculated unit amount of accumulated charge Qint as the measured distance.

[0145] As described above, in the distance image pickup device 1 according to the fourth modification of the embodiment, the pixel 321 includes three charge accumulation units CS1 to CS3. The distance image processing unit 4 controls the charge accumulation units CS1, CS2, and CS3 in this order at a timing synchronized with the irradiation of the light pulse PO. The distance calculation unit 42 determines, as in equation (8) or (9), that (Q1-Q3) is the amount of charge (amount of charge for distance calculation) that corresponds to the reflected light RL and that is accumulated in one of the charge accumulation units CS of the two distance calculation charge accumulation units. Q1 is the amount of charge accumulated in charge accumulation unit CS1. Q3 is the amount of charge accumulated in charge accumulation unit CS3.

[0146] As a result, in the distance image pickup device 1 according to the fourth modification of the embodiment, it is possible to calculate the distance calculation charge amount accumulated in one of the two distance calculation charge accumulation units without specifying the two distance calculation charge accumulation units. Therefore, as shown in SA of the formula (8) or (9), it is possible to calculate the charge ratio R by calculating the sum of the distance calculation charge amounts accumulated in each of the two distance calculation charge accumulation units. Therefore, in the distance image pickup device 1 according to the fourth modification of the embodiment, it is possible to easily calculate the charge ratio R without distinguishing whether to apply the formula (2) or (3) or the formula (6) or (7) depending on the length of the delay time Td, and without specifying the charge accumulation unit CS in which only external light is accumulated and calculating the charge amount Qb corresponding to the external light component. Furthermore, since the same calculation formula (formula (8) or (9)) can be applied even at the boundary between the two time windows, it is possible to improve the discontinuity at the boundary between the time windows.

[0147] In the distance image pickup device 1 according to the fourth modification of the embodiment, the pixel 321 includes four charge accumulation units CS1 to CS4. The distance image processing unit 4 controls the charge accumulation units CS1, CS2, CS3, and CS4 in this order at a timing synchronized with the irradiation of the light pulse PO. The distance calculation unit 42 defines (Q1-Q3) as the signal amount calculated from the charge amount accumulated in one charge accumulation unit CS of the two distance calculation charge accumulation units. The distance calculation unit 42 defines (Q2-Q4) as the signal amount calculated from the charge amount accumulated in the other charge accumulation unit CS of the two distance calculation charge accumulation units. The distance calculation unit 42 defines |Q1-Q3|+|Q2-Q4| as the sum of the signal amounts calculated from the charge amounts accumulated in the respective charge accumulation units CS of the two distance calculation charge accumulation units.

[0148] As a result, in the distance image pickup device 1 according to the fourth modification of the embodiment, it is possible to calculate the sum of the distance calculation charge amounts accumulated in the two distance calculation charge accumulation units, the distance calculation charge amount accumulated in one of the two distance calculation charge accumulation units, and the distance calculation charge amount accumulated in the other one. Therefore, it is possible to calculate the charge ratio R without specifying the two distance calculation charge accumulation units.

[0149] Furthermore, in this case, the charge ratio XR in equation (10) and the charge ratio YR in equation (12) have the same value at the boundary of the time window, which makes it possible to improve the discontinuity at the boundary of the time window.

[0150] In at least one of the above-described embodiments, the charge ratio R is a ratio of the distance calculation charge amount of one of the two distance calculation charge accumulation units to the sum of the distance calculation charge amounts accumulated in each of the two distance calculation charge accumulation units. However, the present invention is not limited to this. The charge ratio R may be a ratio indicated by using the distance calculation charge amounts accumulated in at least two distance calculation charge accumulation units. For example, the charge ratio R may be a ratio of the distance calculation charge amount of one of the two distance calculation charge accumulation units to the distance calculation charge amount of the other one.

[0151] Also, in FIG. 9, an example has been described in which charge is accumulated in the charge accumulation unit CS1 before the light pulse PO is irradiated, thereby controlling so that only the amount of charge Qb corresponding to the external light component is accumulated in the charge accumulation unit CS1. However, this is not limited to this. It is also possible to control so that charge is accumulated in the charge accumulation unit CS3 after irradiating the light pulse PO and receiving the reflected light RL. In this case, only the amount of charge Qb corresponding to the external light component is accumulated in the charge accumulation unit CS3. In this case, the charge accumulation unit CS3 is an example of a "predetermined charge accumulation unit for accumulating external light." Furthermore, in the case where the charge storage unit CS that stores only the charge amount Qb corresponding to the external light component is fixed, the charge storage units CS1 to CS3 may be continuously turned on as shown in FIG. In addition, when the charge storage units CS1 to CS3 are continuously turned on, the driving may be controlled to insert a gap at the timing of switching each of the charge storage units CS to the on state. The gap here is a driving to suppress overlapping in which the charge storage units CS are simultaneously turned on, and is to control all of the charge storage units CS to the off state. For example, when the irradiation time To for irradiating the light pulse PO is 10 [clk], the accumulation time Ta for accumulating charges in the charge storage unit CS is 9 [clk], and the gap is 1 [clk]. Then, the gap is controlled to be provided at the timing of switching the charge storage unit CS from the off state to the on state, or at the timing of switching the charge storage unit CS from the on state to the off state.

[0152] The distance image pickup device 1 and the distance image processing unit 4 in the above-mentioned embodiment may be realized in whole or in part by a computer. In that case, a program for realizing this function may be recorded in a computer-readable recording medium, and the program recorded in the recording medium may be read into a computer system and executed to realize the function. The term "computer system" as used herein includes hardware such as an OS and peripheral devices. The term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into a computer system. The term "computer-readable recording medium" may also include a medium that dynamically holds a program for a short period of time, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line, and a medium that holds a program for a certain period of time, such as a volatile memory inside a computer system that is a server or client in that case. The above-mentioned program may be a program for realizing a part of the above-mentioned function, or may be a program that can realize the above-mentioned function in combination with a program already recorded in the computer system, or may be a program that is realized using a programmable logic device such as an FPGA.

[0153] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and designs that do not deviate from the gist of the present invention are also included. [Explanation of symbols]

[0154] 1...Distance image capture device 2...Light source section 3...Light receiving section 32...Distance image sensor 321…pixels 323...Vertical scanning circuit 4. Distance image processing section 41...Timing control section 42...Distance calculation section 43...Measurement control section 44...Storage section 440…Table information CS…Charge storage section PO: Light pulse

Claims

1. a light source unit that irradiates a measurement space in which a subject is present with a light pulse; a light receiving unit including a pixel including a photoelectric conversion element that generates an electric charge according to incident light and a plurality of charge storage units that store the electric charge, and a pixel drive circuit that distributes and stores the electric charge in each of the charge storage units in the pixel at a predetermined timing synchronized with the irradiation of the light pulse; a distance image processing unit that determines a measured distance to the subject based on the amount of charge accumulated in each of the charge accumulation units; a storage unit that stores table information indicating a relationship between a second variable based on a charge ratio and a corresponding distance corresponding to the distance to the subject for each first variable based on an amount of charge accumulated per unit accumulation; Equipped with the first variable is a variable corresponding to a sum of the amounts of charge accumulated in the charge accumulation units per unit accumulation count, the second variable is a charge ratio indicated by a distance calculation charge amount obtained by subtracting an external light component from the charge amount accumulated in each of two or more distance calculation charge accumulation units in which an amount of charge corresponding to reflected light of the light pulse reflected by the subject is allocated and accumulated among the three or more charge accumulation units, the distance image processing unit calculates the first variable and the second variable based on the amount of charge accumulated in each of the charge accumulation units, selects the table information corresponding to the calculated first variable, obtains the corresponding distance corresponding to the calculated second variable using the selected table information, and determines the measured distance using the obtained corresponding distance. Range imaging device.

2. the charge ratio is a ratio of the distance calculation charge amount of one or a combination of the two or more distance calculation charge accumulation units to the sum of the distance calculation charge amounts accumulated in the distance calculation charge accumulation units, 2. The distance imaging device according to claim 1.

3. the distance image processing unit calculates the first variable and the second variable based on the amount of charge accumulated in each of the charge accumulation units, selects the table information corresponding to the calculated first variable, extracts from the selected table information a first distance associated with a charge ratio smaller than the second variable and a second distance associated with a charge ratio larger than the second variable, and determines the measured distance by linearly interpolating the extracted first distance and second distance.

2. The distance imaging device according to claim 1.

4. The pixel includes three or more of the charge storage units, the distance image processing unit determines the smallest amount of charge among the amounts of charge stored in each of the three or more charge storage units as the amount of charge corresponding to the external light component. The distance imaging device according to claim 1 .

5. The pixel includes three or more of the charge storage units, the distance image processing unit controls a timing for accumulating charge in the charge accumulation unit for accumulating external light so that a charge corresponding to the reflected light is not accumulated in a predetermined charge accumulation unit for accumulating external light among the three or more charge accumulation units, and sets an amount of charge accumulated in the charge accumulation unit for accumulating external light to an amount of charge corresponding to an external light component. The distance imaging device according to claim 1 .

6. the storage unit stores first table information, which is the table information corresponding to a case where the first variable is smaller than a threshold value, and second table information, which is the table information corresponding to a case where the first variable is larger than the threshold value; the threshold value is a value determined according to a charge transfer efficiency in a path from the photoelectric conversion element to the charge accumulation unit, the distance image processing unit calculates the first variable and the second variable based on the amount of charge accumulated in each of the charge accumulation units, and selects the first table information when the calculated first variable is smaller than the threshold value, and selects the second table information when the calculated first variable is larger than the threshold value.

2. The distance imaging device according to claim 1.

7. the storage unit stores the table information for each combination of the two distance calculation charge accumulation units; the distance image processing unit determines, as the combination of the distance calculation charge accumulation units, a combination in which a sum of the amounts of charge accumulated in two of the charge accumulation units that have successive timings for accumulating charge is greater than the amounts of charge accumulated in the other two of the charge accumulation units, among the three or more charge accumulation units, and selects the table information in accordance with the determined combination of the distance calculation charge accumulation units.

2. The distance imaging device according to claim 1.

8. The pixel is provided with three charge accumulation units, that is, a first charge accumulation unit, a second charge accumulation unit, and a third charge accumulation unit, The distance image processing unit Controlling so that the charges are accumulated in the first charge accumulation unit, the second charge accumulation unit, and the third charge accumulation unit in that order at a timing synchronized with the irradiation of the light pulse; a first calculated value that is a difference between the first charge amount and the third charge amount, using a first charge amount stored in the first charge storage unit and a third charge amount stored in the third charge storage unit, and the first calculated value is set as the distance calculation charge amount of one of the two distance calculation charge storage units; 2. The distance imaging device according to claim 1.

9. The pixel is provided with four charge accumulation units, which are a first charge accumulation unit, a second charge accumulation unit, a third charge accumulation unit, and a fourth charge accumulation unit, The distance image processing unit controlling the pixel drive circuit so that the charges are accumulated in the first charge accumulation unit, the second charge accumulation unit, the third charge accumulation unit, and the fourth charge accumulation unit in this order, at a timing synchronized with the irradiation of the light pulse; calculating a first calculated value that is a difference between the first charge amount and the third charge amount by using a first charge amount stored in the first charge storage unit and a third charge amount stored in the third charge storage unit; calculating a second calculated value that is a difference between the second amount of charge and the fourth amount of charge, using a second amount of charge stored in the second charge storage unit and a fourth amount of charge stored in the fourth charge storage unit; a sum of the absolute value of the first calculated value and the absolute value of the second calculated value is set as the sum of the distance calculation charge amounts in the two distance calculation charge accumulation units, the first calculated value is set as the distance calculation charge amount of one of the two distance calculation charge accumulation units, and the second calculated value is set as the distance calculation charge amount of the other of the two distance calculation charge accumulation units.

2. The distance imaging device according to claim 1.

10. a light receiving unit having a light source unit which irradiates a measurement space in which a subject is present with a light pulse, a pixel having a photoelectric conversion element which generates an electric charge according to the incident light and a plurality of charge accumulation units which accumulate the electric charge, and a pixel drive circuit which distributes and accumulates the electric charge in each of the charge accumulation units in the pixel at a predetermined timing synchronized with the irradiation of the light pulse, a distance image processing unit which determines a measurement distance to the subject based on an amount of electric charge accumulated in each of the charge accumulation units, and a memory unit which stores table information which indicates, for each first variable based on an amount of electric charge accumulated per unit accumulation count, a relationship between a second variable based on a charge ratio and a corresponding distance corresponding to the distance to the subject, the first variable is a variable corresponding to a sum of the amounts of charge accumulated in the charge accumulation units per unit accumulation count, the second variable is a charge ratio indicated by a distance calculation charge amount obtained by subtracting an external light component from the charge amount accumulated in each of two or more distance calculation charge accumulation units in which an amount of charge corresponding to reflected light of the light pulse reflected by the subject is allocated and accumulated among the three or more charge accumulation units, The distance image processing unit calculating the first variable and the second variable based on the amount of charge stored in each of the charge storage units; Selecting the table information corresponding to the calculated first variable; Using the selected table information, the corresponding distance corresponding to the calculated second variable is obtained; determining the measured distance using the obtained corresponding distance; A distance imaging method.

Images