Range image capturing device and range image capturing method

Abstract

To provide a distance image capturing device and a distance image capturing method with which it is possible to calculate a distance even when an accumulation count in each subframe is changed.SOLUTION: A distance image capturing device comprises: a light source unit; a light receiving unit that includes a pixel provided with a photoelectric conversion element and a plurality of charge storage units and a pixel drive circuit that allocates the charge to, and has stored in, each of the charge storage units; and a distance image processing unit that calculates a distance to a subject that is present in a measurement space. A plurality of subframes including a first and a second subframe is provided for each frame. The distance image processing unit exercises control so that last storage timing at which a charge is stored in the charge storage unit of the pixel at slowest timing in the first subframe and first storage timing at which a charge is stored in the charge storage unit of the pixel at earliest timing in the second subframe become the same timing, with respect to timing at which irradiation with a light pulse is executed.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a distance image capturing device and a distance image capturing method. [Background technology]

[0002] Taking advantage of the fact that the speed of light is known, a time-of-flight (hereinafter referred to as "TOF") type range imaging device has been realized that measures the distance between a measuring device and an object based on the time of flight of light in space (measurement space) (see, for example, Patent Document 1). In such a range imaging device, the delay time from the time a light pulse is emitted until the light pulse is reflected from the object and returns is found by accumulating charges generated by a photoelectric conversion element in multiple charge accumulation units, and the distance to the object is calculated using the delay time and the speed of light.

[0003] In order to expand the distance measurement range, such distance imaging devices are known to use subframe driving, in which one frame is divided into multiple subframes for measurement. In each subframe, the timing of the gate pulse that accumulates charge in the charge accumulation unit is set relatively differently relative to the timing of the light pulse irradiation. For example, in one subframe, the opening and closing timing of the gate pulse is set so that reflected light arriving from a relatively short distance can be received. In another subframe, the opening and closing timing of the gate pulse is set so that reflected light arriving from a relatively long distance can be received. This allows for a wide range of distances to be measured, from short to long distances. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 4235729 Summary of the Invention [Problem to be solved by the invention]

[0005] However, if the number of integrations for each subframe is different, the amount of charge accumulated in the charge storage unit in each subframe will be different even if the same amount of light is received in one integration. As a result, it is not possible to calculate distance by combining the gate outputs of previous and subsequent subframes, and if the gates of each subframe are opened and closed so that they do not overlap, the distance calculation will be discontinuous at the gate timing when the subframe switches. As a countermeasure, there is a method of overlapping the opening and closing timing of multiple gates in each subframe, driving them so that distance can be calculated in each subframe, and combining the distances calculated in each subframe. However, this method has the problem of shortening the distance measurement range.

[0006] The present invention has been made in light of the above-mentioned problems, and aims to provide a distance image capturing device and a distance image capturing method that can calculate distance even when the number of accumulations in each subframe is changed. [Means for solving the problem]

[0007] The distance image pickup device of the present invention comprises a light source unit that irradiates a measurement space, which is a space to be measured, with a light pulse; a light receiving unit having pixels each including a photoelectric conversion element that generates charge in response to the incident light and a plurality of charge accumulation units that accumulate the charge; and a pixel drive circuit that allocates and accumulates the charge in each of the charge accumulation units at a predetermined timing synchronized with the irradiation of the light pulse; and a distance image processing unit that calculates the distance to a subject present in the measurement space based on the amount of charge accumulated in each of the charge accumulation units, wherein one frame is provided with a plurality of subframes including a first subframe and a second subframe, and the distance image processing unit accumulates charge in each of the charge accumulation units that the pixels have in each subframe in the order of the first subframe and the second subframe, and controls the final accumulation timing, at which charge is accumulated in the charge accumulation unit that the pixel has at the latest timing in the first subframe, and the initial accumulation timing, at which charge is accumulated in the charge accumulation unit that the pixel has at the earliest timing in the second subframe, to be the same timing with respect to the timing of the irradiation of the light pulse. a signal value corresponding to the amount of charge accumulated in each of the charge accumulation units in either the first subframe or the second subframe is multiplied by a constant so that a first signal value corresponding to the amount of charge accumulated in the final charge accumulation unit, which is the charge accumulation unit in which the charge was accumulated at the final accumulation timing, and a second signal value corresponding to the amount of charge accumulated in the initial charge accumulation unit, which is the charge accumulation unit in which the charge was accumulated at the initial accumulation timing, become the same value; and the distance is calculated using the level-adjusted signal value. do.

[0009] In the distance image capturing device of the present invention, the distance image processing unit determines whether or not the final charge storage unit has accumulated charge corresponding to the reflected light of the light pulse, and if the final charge storage unit has accumulated charge corresponding to the reflected light, performs the level adjustment.

[0010] The distance image capturing method of the present invention is a distance image capturing method performed by a distance image capturing device including: a light source unit that irradiates a measurement space, which is a space to be measured, with a light pulse; a light receiving unit having a pixel including a photoelectric conversion element that generates electric charges according to the incident light and a plurality of charge accumulation units that accumulate the electric charges; a pixel drive circuit that distributes and accumulates the electric charges in each of the charge accumulation units at a predetermined timing synchronized with the irradiation of the light pulse; and a distance image processing unit that calculates the distance to a subject present in the measurement space based on the amount of electric charge accumulated in each of the charge accumulation units, wherein one frame includes a first sub-segment, a plurality of subframes including a first subframe and a second subframe are provided, and the distance image processing unit accumulates charges in each charge accumulation unit of the pixel in each subframe in the order of the first subframe and the second subframe, and controls the final accumulation timing, in which the charge accumulation unit of the pixel accumulates charges at the latest timing in the first subframe, and the initial accumulation timing, in which the charge accumulation unit of the pixel accumulates charges at the earliest timing in the second subframe, to be the same timing with respect to the timing of irradiation of the light pulse. a signal value corresponding to the amount of charge accumulated in each of the charge accumulation units in either the first subframe or the second subframe is multiplied by a constant so that a first signal value corresponding to the amount of charge accumulated in the final charge accumulation unit, which is the charge accumulation unit in which the charge was accumulated at the final accumulation timing, and a second signal value corresponding to the amount of charge accumulated in the initial charge accumulation unit, which is the charge accumulation unit in which the charge was accumulated at the initial accumulation timing, become the same value; and the distance is calculated using the level-adjusted signal value. do. [Effects of the Invention]

[0011] According to the present invention, it is possible to calculate the distance even when the number of integration times in each subframe is changed. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a block diagram showing a schematic configuration of a distance image capturing device 1 according to an embodiment. [Figure 2] FIG. 2 is a block diagram showing a schematic configuration of a range image sensor 32 according to the embodiment. [Figure 3] FIG. 3 is a circuit diagram showing an example of the configuration of a pixel 321 according to an embodiment. [Figure 4] 10 is a timing chart showing timings for driving a pixel 321 according to an embodiment. [Figure 5]10 is a timing chart showing timings for driving a pixel 321 according to an embodiment. [Figure 6] 4 is a flowchart showing the flow of processing performed by the distance image capturing device 1 of the embodiment. [Figure 7] 10A and 10B are diagrams illustrating the effects of the embodiment. [Figure 8] 10A and 10B are diagrams illustrating the effects of the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, a distance image capturing device according to an embodiment will be described with reference to the drawings.

[0014] Fig. 1 is a block diagram showing the schematic configuration of a distance image capturing device according to an embodiment. The distance image capturing device 1 includes, for example, a light source unit 2, a light receiving unit 3, and a distance image processing unit 4. Fig. 1 also shows a subject OB, which is an object to which the distance is measured by the distance image capturing device 1.

[0015] The light source unit 2 irradiates a light pulse PO into a space to be measured, in which a subject OB, the distance of which is to be measured by the distance image pickup device 1, is present, under the control of the distance image processor 4. The light source unit 2 is, for example, a surface-emitting semiconductor laser module such as a vertical cavity surface-emitting laser (VCSEL). The light source unit 2 includes a light source device 21 and a diffuser plate 22.

[0016] The light source device 21 is a light source that emits laser light in a near-infrared wavelength band (for example, a wavelength band of 850 nm to 940 nm) that becomes the light pulses PO to be irradiated onto the subject OB. The light source device 21 is, for example, a semiconductor laser light-emitting element. The light source device 21 emits pulsed laser light in response to control from the timing control unit 41.

[0017] The diffusion plate 22 is an optical component that diffuses the laser light in the near-infrared wavelength band emitted by the light source device 21 to the extent of the surface that is irradiated onto the subject OB. The pulsed laser light diffused by the diffusion plate 22 is emitted as a light pulse PO and is irradiated onto the subject OB.

[0018] The light receiving unit 3 receives reflected light RL of the light pulse PO reflected by an object OB, the distance of which is to be measured in the range image pickup device 1, and outputs a pixel signal corresponding to the received reflected light RL. The light receiving unit 3 includes a lens 31 and a range image sensor 32.

[0019] The lens 31 is an optical lens that guides the incident reflected light RL to the range image sensor 32. The lens 31 outputs the incident reflected light RL to the range image sensor 32 side, and causes the light to be received (incident) by pixels provided in the light receiving region of the range image sensor 32.

[0020] The range image sensor 32 is an imaging element used in the range image capturing device 1. The range image sensor 32 has a plurality of pixels in a two-dimensional light receiving area. Each pixel of the range image sensor 32 is provided with one photoelectric conversion element, a plurality of charge accumulation units corresponding to this one photoelectric conversion element, and components that distribute charge to each of the charge accumulation units. In other words, the pixel is an imaging element with a distribution configuration in which charge is distributed and stored in a plurality of charge accumulation units.

[0021] The range image sensor 32 distributes the charges generated by the photoelectric conversion elements to the respective charge accumulation sections under the control of the timing control section 41. The range image sensor 32 also outputs pixel signals according to the amount of charge distributed to the charge accumulation sections. The range image sensor 32 has multiple pixels arranged in a two-dimensional matrix, and outputs pixel signals for one frame corresponding to each pixel.

[0022] The distance image processing unit 4 controls the distance image pickup device 1 and calculates the distance to the subject OB. The distance image processing unit 4 includes a timing control unit 41, a distance calculation unit 42, and a measurement control unit 43.

[0023] The timing control unit 41 controls the timing of outputting various control signals required for measurement in accordance with the control of the measurement control unit 43. The various control signals here include, for example, a signal that controls the irradiation of the light pulse PO, a signal that distributes and accumulates the reflected light RL in multiple charge accumulation units, and a signal that controls the number of accumulations per frame. The number of accumulations is the number of times that the process of distributing and accumulating electric charge in the charge accumulation units CS (see FIG. 3) is repeated. The accumulation time is the product of this number of accumulations and the time (accumulation time) for accumulating electric charge in each charge accumulation unit per process of distributing and accumulating electric charge.

[0024] The distance calculation unit 42 outputs distance information calculated based on the pixel signals output from the distance image sensor 32. The distance calculation unit 42 calculates the delay time from when the light pulse PO is emitted until when the reflected light RL is received based on the amount of charge accumulated in the multiple charge accumulation units. The distance calculation unit 42 calculates the distance to the object OB according to the calculated delay time.

[0025] The measurement control unit 43 controls the timing control unit 41. For example, the measurement control unit 43 sets the number of accumulations and accumulation time for one frame, and controls the timing control unit 41 so that imaging is performed according to the set contents.

[0026] With this configuration, in the distance image capturing device 1, the light source unit 2 irradiates a light pulse PO in the near-infrared wavelength band onto the subject OB, and the light receiving unit 3 receives the reflected light RL reflected by the subject OB, and the distance image processing unit 4 outputs distance information measuring the distance to the subject OB.

[0027] Although Figure 1 shows a distance image capturing device 1 configured such that the distance image processing unit 4 is provided inside the distance image capturing device 1, the distance image processing unit 4 may also be a component provided outside the distance image capturing device 1.

[0028] Here, the configuration of the distance image sensor 32 used as an imaging element in the distance image pickup device 1 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing a schematic configuration of the imaging element (distance image sensor 32) used in the distance image pickup device 1 of the embodiment.

[0029] As shown in FIG. 2, the distance image sensor 32 includes, for example, a light receiving area 320 in which a plurality of pixels 321 are arranged, a control circuit 322, a vertical scanning circuit 323 having a distribution operation, a horizontal scanning circuit 324, and a pixel signal processing circuit 325.

[0030] The light receiving area 320 is an area in which a plurality of pixels 321 are arranged, and FIG. 2 shows an example in which the pixels are arranged in a two-dimensional matrix of 8 rows and 8 columns. The pixels 321 accumulate electric charges corresponding to the amount of light they receive. The control circuit 322 comprehensively controls the range image sensor 32. The control circuit 322 controls the operation of the components of the range image sensor 32 in accordance with instructions from, for example, the timing control unit 41 of the range image processing unit 4. Note that the components of the range image sensor 32 may be directly controlled by the timing control unit 41, in which case the control circuit 322 may be omitted.

[0031] The vertical scanning circuit 323 is a circuit that controls the pixels 321 arranged in the light receiving region 320 for each row in accordance with control from the control circuit 322. The vertical scanning circuit 323 outputs a voltage signal corresponding to the amount of charge accumulated in each charge accumulation unit CS of the pixels 321 to the pixel signal processing circuit 325. In this case, the vertical scanning circuit 323 distributes and accumulates the charge converted by the photoelectric conversion element in each charge accumulation unit of the pixels 321. In other words, the vertical scanning circuit 323 is an example of a "pixel driving circuit."

[0032] The pixel signal processing circuit 325 is a circuit that performs predetermined signal processing (e.g., noise suppression processing, A / D conversion processing, etc.) on the voltage signals output from the pixels 321 in each column to the corresponding vertical signal lines in accordance with control from the control circuit 322.

[0033] Horizontal scanning circuit 324 is a circuit that sequentially outputs signals output from pixel signal processing circuit 325 to horizontal signal lines in accordance with control from control circuit 322. As a result, pixel signals corresponding to the amount of charge accumulated for one frame are sequentially output to distance image processing unit 4 via the horizontal signal lines.

[0034] In the following description, it is assumed that the pixel signal processing circuit 325 performs A / D conversion processing and the pixel signals are digital signals.

[0035] Here, the configuration of the pixel 321 arranged in the light receiving region 320 provided in the range image sensor 32 will be described with reference to Fig. 3. Fig. 3 is a circuit diagram showing an example of the configuration of the pixel 321 arranged in the light receiving region 320 of the range image sensor 32 of the embodiment. Fig. 3 shows an example of the configuration of one pixel 321 out of the multiple pixels 321 arranged in the light receiving region 320. The pixel 321 is an example of a configuration including four pixel signal readout units.

[0036] The pixel 321 includes one photoelectric conversion element PD, a drain gate transistor GD, and four pixel signal readout units RU that output voltage signals from corresponding output terminals O. Each pixel signal readout unit RU includes a readout gate transistor G, a floating diffusion FD, a charge storage capacitance C, a reset gate transistor RT, a source follower gate transistor SF, and a select gate transistor SL. In each pixel signal readout unit RU, the floating diffusion FD and the charge storage capacitance C form a charge storage unit CS.

[0037] 3, the four pixel signal readout units RU are distinguished from one another by adding the numbers "1," "2," "3," or "4" after the symbol "RU" of each pixel signal readout unit RU. Similarly, the components of each of the four pixel signal readout units RU are distinguished from one another by adding the number representing each pixel signal readout unit RU after the symbol.

[0038] 3, the pixel signal readout unit RU1, which outputs a voltage signal from the output terminal O1, includes a readout gate transistor G1, a floating diffusion FD1, a charge storage capacitance C1, a reset gate transistor RT1, a source follower gate transistor SF1, and a select gate transistor SL1. In the pixel signal readout unit RU1, the floating diffusion FD1 and the charge storage capacitance C1 form a charge storage unit CS1. The pixel signal readout units RU2 to RU4 have a similar configuration.

[0039] The photoelectric conversion element PD is a buried photodiode that photoelectrically converts incident light to generate electric charges and accumulates the generated electric charges. The photoelectric conversion element PD may have any structure. For example, the photoelectric conversion element PD may be a PN photodiode having a structure in which a P-type semiconductor and an N-type semiconductor are joined together, or a PIN photodiode having a structure in which an I-type semiconductor is sandwiched between a P-type semiconductor and an N-type semiconductor. Furthermore, the photoelectric conversion element PD is not limited to a photodiode, and may be, for example, a photogate type photoelectric conversion element.

[0040] In pixel 321, the photoelectric conversion element PD photoelectrically converts incident light to generate electric charges, which are then distributed to each of the four charge accumulation units CS, and voltage signals corresponding to the amount of electric charge distributed are output to the pixel signal processing circuit 325.

[0041] The configuration of pixels arranged in the range image sensor 32 is not limited to the configuration including four pixel signal readout units RU as shown in Fig. 3, but may be any pixel configured to include multiple pixel signal readout units RU. In other words, the number of pixel signal readout units RU (charge accumulation units CS) provided in the pixels arranged in the range image sensor 32 may be two, three, five or more.

[0042] 3 shows an example in which the charge storage section CS is configured by a floating diffusion FD and a charge storage capacitance C. However, the charge storage section CS only needs to be configured by at least a floating diffusion FD, and the pixel 321 may not have a charge storage capacitance C.

[0043] Furthermore, in the pixel 321 having the configuration shown in FIG. 3, an example of a configuration including a drain gate transistor GD is shown, but if there is no need to discard the charge accumulated (remaining) in the photoelectric conversion element PD, the pixel 321 may have a configuration not including a drain gate transistor GD.

[0044] In this embodiment, one frame includes multiple subframes. The multiple subframes include a first subframe and a second subframe. The distance image processor 4 controls the accumulation timing of charges in each charge accumulation unit CS of the pixel 321 in the first subframe so that it is earlier than the accumulation timing in the second subframe. This makes it possible to calculate the distance to a relatively close object OB in the first subframe, and to calculate the distance to a relatively far object OB in the second subframe. Therefore, by configuring the subframes in this way, it is possible to expand the range of distances that can be measured.

[0045] Furthermore, in the embodiment, the number of accumulations in each subframe is different. It is known that light intensity is inversely proportional to the square of the distance. For example, when reflected light RL reflected from a relatively close object OB is received in the first subframe, the intensity of the reflected light is high. On the other hand, when reflected light RL reflected from a relatively distant object OB is received in the second subframe, the intensity of the reflected light is low. Therefore, if the first and second subframes have the same number of accumulations, setting the number of accumulations so that the amount of charge accumulated in the charge accumulation unit CS of the pixel 321 in the first subframe is appropriate results in a small amount of charge accumulated in the charge accumulation unit CS of the pixel 321 in the second subframe, which can cause errors. On the other hand, setting the number of accumulations so that the amount of charge accumulated in the charge accumulation unit CS of the pixel 321 in the second subframe is appropriate results in an excessively large amount of charge accumulated in the charge accumulation unit CS of the pixel 321 in the first subframe, resulting in saturation and making it difficult to calculate the distance. To address this issue, in the embodiment, the number of accumulations is controlled to be different for each subframe. Specifically, distance image processing unit 4 controls the number of integrations in the first sub-frame to be smaller than the number of integrations in the second sub-frame.

[0046] Here, the timing for driving the pixel 321 will be described with reference to Fig. 4 and Fig. 5. Fig. 4 and Fig. 5 are timing charts showing the timing for driving the pixel 321 according to the embodiment. Fig. 4 shows an example in which a first sub-frame and a second sub-frame are provided in one frame. That is, one frame includes a first sub-frame, a second sub-frame, and a readout period.

[0047] 4 and 5, the timing of irradiating the light pulse PO is indicated by the item names "L", the timing of the drive signal TX1 is indicated by "G1", the timing of the drive signal TX2 is indicated by "G2", the timing of the drive signal TX3 is indicated by "G3", the timing of the drive signal TX4 is indicated by "G4", and the timing of the drive signal RSTD is indicated by "GD". The drive signal TX1 is a signal that drives the read gate transistor G1. The same applies to the drive signals TX2 to TX4.

[0048] As shown in FIG. 4 , a light pulse PO is irradiated for an irradiation time To. First, the vertical scanning circuit 323 repeats a “first period” in a “first subframe” a predetermined number of times (first accumulation times) in synchronization with the irradiation of the light pulse PO. The “first period” accumulates charges in the charge accumulation units CS1, CS2, CS3, and CS4 of the pixel 321 in that order. Next, the vertical scanning circuit 323 reads out, as signal values, voltage signals corresponding to the amounts of charges accumulated in each pixel 321 in a “first subframe readout period.” Next, the vertical scanning circuit 323 repeats, as signal values, a “second period” in a “second subframe” a predetermined number of times (second accumulation times) in synchronization with the irradiation of the light pulse PO. Then, the vertical scanning circuit 323 reads out, as signal values, voltage signals corresponding to the amounts of charges accumulated in each pixel 321 in a “second subframe readout period.”

[0049] Here, the timing at which the vertical scanning circuit 323 causes the pixels 321 to accumulate electric charges will be described.

[0050] First, the vertical scanning circuit 323 turns off the drain gate transistor GD of the pixel 321 corresponding to the first subframe and turns on the readout gate transistor G1 at the same timing as the irradiation of the light pulse PO. After the accumulation time Ta has elapsed since the readout gate transistor G1 was turned on, the vertical scanning circuit 323 turns off the readout gate transistor G1. As a result, charges photoelectrically converted by the photoelectric conversion element PD while the readout gate transistor G1 is controlled to be in the on state are accumulated in the charge accumulation unit CS1 via the readout gate transistor G1. The accumulation time Ta is a period for accumulating charges in the charge accumulation unit CS and is not limited to being the same period as the gate transistor ON period (the period during which the gate transistor G is in the on state). For example, a gate transistor ON period may be provided after a short gap period (a gate OFF period, i.e., a period during which the gate transistor G is in the off state), and the sum of the gap period and the gate transistor ON period may be controlled to be the accumulation time Ta.

[0051] Next, the vertical scanning circuit 323 turns on the read gate transistor G2 for the accumulation time Ta at the same time as turning off the read gate transistor G1. As a result, the charge photoelectrically converted by the photoelectric conversion element PD while the read gate transistor G2 is controlled to be in the on state is accumulated in the charge accumulation section CS2 via the read gate transistor G2.

[0052] Next, the vertical scanning circuit 323 turns on the read gate transistor G3 at the timing when the accumulation of charges in the charge accumulation unit CS2 is completed, and turns off the read gate transistor G3 after the accumulation time Ta has elapsed. As a result, the charges photoelectrically converted by the photoelectric conversion element PD while the read gate transistor G3 is controlled to be in the on state are accumulated in the charge accumulation unit CS3 via the read gate transistor G3.

[0053] Next, the vertical scanning circuit 323 turns on the read gate transistor G4 at the timing when the accumulation of charges in the charge accumulation unit CS3 is completed, and turns off the read gate transistor G4 after the accumulation time Ta has elapsed. As a result, the charges photoelectrically converted by the photoelectric conversion element PD while the read gate transistor G4 is controlled to be in the on state are accumulated in the charge accumulation unit CS4 via the read gate transistor G4.

[0054] Next, the vertical scanning circuit 323 turns on the drain gate transistor GD to discharge the charge at the timing when the charge accumulation in the charge accumulation unit CS4 is completed, whereby the charge photoelectrically converted by the photoelectric conversion element PD is discarded via the drain gate transistor GD.

[0055] The vertical scanning circuit 323 repeats the driving described above a predetermined number of times (first number of times of accumulation) set in the first sub-frame.

[0056] After the accumulation of charge in the pixel 321 by the first sub-frame driving is completed, the vertical scanning circuit 323 outputs a voltage signal corresponding to the amount of charge allocated to each charge accumulation unit CS. Specifically, the vertical scanning circuit 323 turns on the selection gate transistor SL1 for a predetermined time, causing a voltage signal corresponding to the amount of charge accumulated in the charge accumulation unit CS1 to be output from the output terminal O1 via the pixel signal readout unit RU1. Similarly, the vertical scanning circuit 323 sequentially turns on the selection gate transistors SL2 and SL3, causing voltage signals corresponding to the amounts of charge accumulated in the charge accumulation units CS2 and CS3 to be output from the output terminals O2 and O3. Then, the voltage signals corresponding to the amounts of charge accumulated in each charge accumulation unit CS are output as signal values ​​to the distance calculation unit 42 via the pixel signal processing circuit 325 and the horizontal scanning circuit 324.

[0057] Next, the vertical scanning circuit 323 turns off the drain gate transistor GD of the pixel 321 corresponding to the second sub-frame and turns on the readout gate transistor G1 at a timing delayed by a predetermined delay time (accumulation time Ta×3) from the timing of irradiating the light pulse PO. The operation after turning on the readout gate transistor G1, that is, the timing at which the vertical scanning circuit 323 turns on or off each of the readout gate transistors G1 to G4 and the drain gate transistor GD, is the same as in the first sub-frame, and therefore description thereof will be omitted. The vertical scanning circuit 323 repeats the driving described above a predetermined number of accumulations (second accumulations) set for the second sub-frame.

[0058] After the accumulation of charge in the pixels 321 by the second sub-frame driving is completed, the vertical scanning circuit 323 outputs voltage signals corresponding to the amounts of charge allocated to each charge accumulation unit CS. Specifically, the vertical scanning circuit 323 turns on the selection gate transistor SL1 for a predetermined time, causing a voltage signal corresponding to the amount of charge accumulated in the charge accumulation unit CS1 to be output from the output terminal O1 via the pixel signal readout unit RU1. Similarly, the vertical scanning circuit 323 sequentially turns on the selection gate transistors SL2 and SL3, causing voltage signals corresponding to the amounts of charge accumulated in the charge accumulation units CS2 and CS3 to be output from the output terminals O2 and O3. Then, voltage signals corresponding to the amounts of charge accumulated in each charge accumulation unit CS are output as signal values ​​to the distance calculation unit 42 via the pixel signal processing circuit 325 and the horizontal scanning circuit 324.

[0059] As shown in FIG. 4, in this embodiment, one frame includes multiple subframes. In the example of FIG. 4, one frame includes a first subframe and a second subframe. In the first subframe, charge is accumulated in each charge accumulation unit CS of the pixel 321 in the first subframe at an earlier timing than in the second subframe. In the second subframe, charge is accumulated in each charge accumulation unit CS of the pixel 321 in the second subframe at a later timing than in the first subframe. Using the timing of irradiation of the light pulse PO as a reference, the distance image processing unit 4 delays the accumulation timing of the pixel 321 in the second subframe by a predetermined delay time (accumulation time Ta×3) relative to the accumulation timing of the pixel 321 in the first subframe.

[0060] FIG. 5 shows the timing at which the read gate transistors G1 to G4 and the drain gate transistor GD are turned on or off in the first and second subframes, based on the timing at which the light pulse PO is irradiated.

[0061] 5, in this embodiment, the opening and closing timing of the gate transistor G4 in the first subframe (symbol Tr1) and the opening and closing timing of the gate transistor G1 in the second subframe (symbol Tr2) are controlled to be the same timing with respect to the timing of irradiation of the light pulse PO. Here, the opening and closing timing of the gate transistor G4 in the first subframe (symbol Tr1) is the latest accumulation timing in the first subframe and is an example of the "final accumulation timing." Also, the opening and closing timing of the gate transistor G1 in the second subframe (symbol Tr2) is the earliest accumulation timing in the second subframe and is an example of the "initial accumulation timing."

[0062] Here, the flow of processing performed by the range image pickup device 1 will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the flow of processing performed by the range image pickup device 1 of the embodiment.

[0063] Step S10: In the first sub-frame, the distance image pickup device 1 drives the pixels 321. As a result, charges are accumulated in the charge accumulation units CS of the pixels 321 at accumulation timings according to the first sub-frame.

[0064] Step S11: The range image pickup device 1 reads out signals (Sub1_G1 to G4) in the first sub-frame readout period. Here, signal Sub1_G1 is a signal value corresponding to the amount of charge accumulated in charge accumulation unit CS1 of pixel 321 in the first subframe. Signal Sub1_G2 is a signal value corresponding to the amount of charge accumulated in charge accumulation unit CS2 of pixel 321 in the first subframe. Signal Sub1_G3 is a signal value corresponding to the amount of charge accumulated in charge accumulation unit CS3 of pixel 321 in the first subframe. Signal Sub1_G4 is a signal value corresponding to the amount of charge accumulated in charge accumulation unit CS4 of pixel 321 in the first subframe. Step S12: In the second sub-frame, the distance image pickup device 1 drives the pixels 321. As a result, charges are accumulated in the charge accumulation units CS of the pixels 321 at accumulation timings according to the second sub-frame. Step S13: The range image pickup device 1 reads out signals (Sub2_G1 to G4) in the second sub-frame readout period. Furthermore, signal Sub2_G1 has a signal value corresponding to the amount of charge accumulated in charge accumulation unit CS1 of pixel 321 in the second subframe. Signal Sub2_G2 has a signal value corresponding to the amount of charge accumulated in charge accumulation unit CS2 of pixel 321 in the second subframe. Signal Sub2_G3 has a signal value corresponding to the amount of charge accumulated in charge accumulation unit CS3 of pixel 321 in the second subframe. Signal Sub2_G4 has a signal value corresponding to the amount of charge accumulated in charge accumulation unit CS4 of pixel 321 in the second subframe.

[0065] Step S14: The range image pickup device 1 identifies signals that contain reflected light components from among the signals (Sub1_G1 to G4 and Sub2_G1 to G4).

[0066] The distance image pickup device 1 can use, for example, the following method to identify a signal containing a reflected light component.

[0067] The range image pickup device 1 calculates a signal value (external light signal value) corresponding to the amount of charge of the external light component for each of the first sub-frame and the second sub-frame. For example, the distance image pickup device 1 determines the smallest value among the signal values ​​of the signals (Sub1_G1 to G4) as the ambient light signal value corresponding to the first subframe (ambient light signal value 1). The distance image pickup device 1 identifies signals (Sub1_G1 to G4) that have values ​​greater than ambient light signal value 1 as signals that include a reflected light component. Furthermore, the distance image pickup device 1 determines the smallest value among the signal values ​​of the signals (Sub2_G1 to G4) as the ambient light signal value corresponding to the second subframe (ambient light signal value 2). The distance image pickup device 1 identifies signals (Sub2_G1 to G4) that have values ​​greater than ambient light signal value 2 as signals that include a reflected light component.

[0068] In this case, the distance image capturing device 1 may determine that, among the signal values ​​of the signals (Sub1_G1 to G4), those that are greater than the external light signal value 1 by a threshold value or more are signals that contain reflected light components. Since external light components may contain noise, even if a signal has a value greater than the external light signal value, if the difference between the external light signal value and the external light signal value is not so large, it may not contain reflected light components. By setting a threshold, it becomes possible to separate signals that contain reflected light components from signals that contain noise. Similarly, for the second sub-frame, the distance image capturing device 1 may determine that the signal values ​​of the signals (Sub2_G1 to G4) that are greater than the external light signal value 2 by a threshold value or more are signals that contain reflected light components.

[0069] Alternatively, the distance image pickup device 1 may control the first subframe so that a specific charge accumulation unit CS accumulates only an amount of charge corresponding to the external light component, and set the signal value corresponding to the amount of charge accumulated in the specific charge accumulation unit CS as the external light signal value 1. For example, before irradiating the light pulse PO, the distance image pickup device 1 turns on a specific charge accumulation unit CS, for example, the gate transistor G1 corresponding to the charge accumulation unit CS1, for a predetermined time (accumulation time Ta), thereby causing the specific charge accumulation unit CS to accumulate only an amount of charge corresponding to the external light component. In this case, the distance image pickup device 1 sets the signal value corresponding to the amount of charge accumulated in the charge accumulation unit CS1 of the pixel 321 in the first subframe as the external light signal value 1.

[0070] Step S15: The distance image pickup device 1 determines whether or not the signal Sub1_G4 contains a reflected light component. If the distance image pickup device 1 identifies the signal Sub1_G4 as a signal containing a reflected light component in step S12, it determines that the signal Sub1_G4 contains a reflected light component. On the other hand, if the distance image pickup device 1 does not identify the signal Sub1_G4 as a signal containing a reflected light component in step S12, it determines that the signal Sub1_G4 does not contain a reflected light component. Note that in this case, the distance image pickup device 1 may also determine whether or not the signal Sub2_G1 contains a reflected light component.

[0071] Step S16: If the signal Sub1_G4 contains a reflected light component, the depth image pickup device 1 performs level adjustment to make the signal level corresponding to the amount of charge accumulated in the charge accumulation unit CS in the first subframe equal to the signal level corresponding to the amount of charge accumulated in the charge accumulation unit CS in the second subframe.

[0072] The range image pickup device 1 performs level adjustment using the following equation (1).

[0073] Sub1_G1#=α×Sub1_G1 Sub1_G2#=α×Sub1_G2 Sub1_G3#=α×Sub1_G3 Sub1_G4#=α×Sub1_G4 α=Sub2_G1 / Sub1_G4 …(1)

[0074] In equation (1), the signal Sub1_G1 is a signal value corresponding to the amount of charge accumulated in the charge accumulation unit CS1 of the pixel 321 in the first sub-frame. The signal Sub1_G2 has a signal value corresponding to the amount of charge accumulated in the charge accumulation unit CS2 of the pixel 321 in the first sub-frame. The signal Sub1_G3 has a signal value corresponding to the amount of charge accumulated in the charge accumulation unit CS3 of the pixel 321 in the first sub-frame. The signal Sub1_G4 has a signal value corresponding to the amount of charge accumulated in the charge accumulation unit CS4 of the pixel 321 in the first sub-frame. The signal Sub2_G1 has a signal value corresponding to the amount of charge accumulated in the charge accumulation unit CS1 of the pixel 321 in the second sub-frame. The signal Sub1_G1# is a corrected value of the signal Sub1_G1. The signal Sub1_G2# is a corrected value of the signal Sub1_G2. The signal Sub1_G3# is a corrected value of the signal Sub1_G3. The signal Sub1_G4# is a corrected value of the signal Sub1_G4. α is a constant determined according to the ratio of signal Sub2_G1 to signal Sub1_G4. Note that signals Sub2 and Sub1 in equation (1) may be interchanged. That is, the depth image pickup device 1 may adjust the signal level of signal Sbu2 by matching it to the signal level of signal Sub1.

[0075] Step S17: If the signal Sub1_G4 does not contain a reflected light component, the range image pickup device 1 does not perform level adjustment.

[0076] Step S18: The distance image pickup device 1 calculates the distance. If the level adjustment has been performed in step S14, the distance image pickup device 1 calculates the distance using the level-adjusted signal value.

[0077] After the level adjustment, the distance image pickup device 1 calculates the distance using, for example, equation (2A). Equation (2A) shows an example of a mathematical formula for calculating the distance when the signals Sub1_G3 and Sub1_G4 contain reflected light components.

[0078] d=c0×(1 / 2×Td) Td=To×{1-(AC) / (Abs(AC)+Abs(BD))} +d_offset …(2A)

[0079] In equation (2A), the distance d is the result of the calculation and is the distance to the subject OB. The speed c0 is the speed of light. The time Td is the time (delay time) required from when the light pulse is emitted until the reflected light RL is received. The time To is the time when the light pulse is irradiated. Signal value A is the signal value of signal Sub1_G3#. Signal value B is the signal value of signal Sub1_G4#. The signal value C is the signal value of the signal Sub2_G2. The signal value D is the signal value of the signal Sub2_G3. The distance d_offset is the distance offset. The distance offset is the amount of distance offset that results from the timing of the gate used for distance calculation, i.e., the timing at which the gate transistor that is first controlled to be turned on among the four gate transistors used for distance calculation turns on, being later than the timing at which the light pulse PO is irradiated. For example, in this embodiment, the opening and closing timing of the gate transistor G1 in the first subframe is controlled to coincide with the irradiation timing of the light pulse PO. In this case, when the charges corresponding to the reflected light RL are distributed and received by the gate transistors G1 to G4 in the first subframe, the distance d_offset is set to 0 (zero). On the other hand, in other cases, that is, when the gate transistor that is first controlled to be turned on among the four gate transistors used for distance calculation is controlled to be turned on after the timing of irradiation of the light pulse PO, the distance d_offset is set to a distance corresponding to the timing at which the reflected light is received.

[0080] Note that, of the four signals used in distance calculation, formula (2A) is applied when two of three consecutive gate transistors, including the gate transistor that is first controlled to be turned on, receive reflected light RL. For example, when signals Sub1_G3, Sub1_G4, Sub2_G1, and Sub2_G2 are used in distance calculation, formula (2A) is applied when the charge corresponding to reflected light RL is distributed and accumulated in signals Sub1_G3 and Sub1_G4 or signals Sub1_G4 and Sub2_G1.

[0081] On the other hand, in the case where two of three consecutive gate transistors, including the gate transistor last controlled to be turned on among the four signals used for distance calculation, receive reflected light RL, the following formula (2B) is applied. For example, when signals Sub1_G3, Sub1_G4, Sub2_G1, and Sub2_G2 are used for distance calculation, this applies to the case where the charge equivalent to reflected light RL is distributed to signals Sub1_G4 and Sub2_G1, or signals Sub2, G1, and Sub2_G2.

[0082] d=c0×(1 / 2×Td) Td=To×{2-(BD) / (Abs(AC)+Abs(BD))} +d_offset …(2B)

[0083] In equation (2B), the distance d, the speed c0, the time Td, the time To, and the signal values ​​A to D are the same as in equation (2A).

[0084] Alternatively, the distance image pickup device 1 calculates the distance using, for example, equation (3). Equation (3) shows an example of a mathematical formula for calculating the distance when the signals Sub1_G4 and Sub2_G2 contain reflected light components.

[0085] d=c0×(1 / 2×Td) Td=To×{1-(AC) / (Abs(AC)+Abs(BD))} +d_offset …(3)

[0086] In equation (3), the distance d is the result of the calculation and is the distance to the subject OB. The speed c0 is the speed of light. The time Td is the time (delay time) required from when the light pulse is emitted until the reflected light RL is received. The time To is the time when the light pulse is irradiated. Signal value A is the signal value of signal Sub1_G4#. The signal value B is the signal value of the signal Sub2_G2. The signal value C is the signal value of the signal Sub2_G3. The signal value D is the signal value of the signal Sub2_G4. The distance d_offset is the distance offset.

[0087] On the other hand, if no level adjustment is performed, the range image pickup device 1 calculates the distance in step S16 using the signal value that has not been level adjusted. If level adjustment is not performed, the distance image pickup device 1 calculates the distance using, for example, equation (4). Equation (4) shows an example of a mathematical formula for calculating the distance when the signals Sub1_G1 and Sub1_G2 contain reflected light components.

[0088] d=c0×(1 / 2×Td) Td=To×{1-(AC) / (Abs(AC)+Abs(BD))} …(4)

[0089] In equation (4), the distance d is the result of the calculation and is the distance to the subject OB. The speed c0 is the speed of light. The time Td is the time (delay time) required from when the light pulse is emitted until the reflected light RL is received. The time To is the time when the light pulse is irradiated. The signal value A is the signal value of the signal Sub1_G1. The signal value B is the signal value of the signal Sub1_G2. The signal value C is the signal value of the signal Sub1_G3. The signal value D is the signal value of the signal Sub1_G4.

[0090] As described above, in the distance image capturing device 1 of the embodiment, one frame is provided with multiple subframes, including a first subframe and a second subframe. The distance image processing unit 4 accumulates charge in each charge accumulation unit of the pixel in each subframe, in the order of the first subframe and then the second subframe. The distance image processing unit 4 controls the accumulation timing so that the timings of Tr1 and Tr2 in FIG. 5 are the same with respect to the irradiation time To in FIG. 5. In other words, the distance image processing unit 4 controls the accumulation timing so that the final accumulation timing, which is the latest accumulation timing in the first subframe, and the initial accumulation timing, which is the earliest accumulation timing in the second subframe, are the same with respect to the timing of irradiation of the light pulse PO.

[0091] As a result, in the range image capturing device 1 of the embodiment, charge can be accumulated in the charge accumulation units CS of the first and second subframes at the same timing, based on the irradiation timing of the light pulse PO. This makes it possible to perform level adjustment based on signal values ​​corresponding to the amount of charge accumulated in each of the charge accumulation units CS that accumulate charge at the same timing. Therefore, distance can be calculated even if the number of integrations in each subframe is changed.

[0092] The effects of this embodiment will now be described with reference to Figures 7 and 8. Figures 7 and 8 are diagrams for explaining the effects of this embodiment.

[0093] Fig. 7 shows a timing chart for a comparative example in which pixels are driven at timings different from those of the present embodiment. Similar to Fig. 5, Fig. 7 shows the timings at which the readout gate transistors G1 to G4 and the drain gate transistor GD are turned on or off in the first and second sub-frames, based on the timing at which the light pulse PO is irradiated.

[0094] 7, the opening and closing timing of the gate transistor G4 in the first subframe (symbol Tr3) is followed by the opening and closing timing of the gate transistor G1 in the second subframe (symbol Tr4). That is, the opening and closing timing of the gate transistor G4 in the first subframe (symbol Tr3) and the opening and closing timing of the gate transistor G1 in the second subframe (symbol Tr4) are not the same timing with respect to the timing of irradiation of the light pulse PO.

[0095] In this comparative example, the charge accumulation units CS of the first and second subframes cannot accumulate charges at the same timing based on the irradiation timing of the light pulse PO. Therefore, it is difficult to adjust the level when the accumulation counts of the first and second subframes are different. Therefore, it is difficult to calculate the distance when the accumulation count for each subframe is changed.

[0096] Fig. 8 shows a comparative example in which pixels are driven at a timing different from that of Fig. 7. Similar to Fig. 5, Fig. 8 shows the timing at which the readout gate transistors G1 to G4 and the drain gate transistor GD are turned on or off in the first and second sub-frames, based on the timing at which the light pulse PO is irradiated.

[0097] 8, the opening and closing timings of the gate transistor G3 (symbol Tr5) and the gate transistor G4 (symbol Tr6) in the first subframe, and the opening and closing timings of the gate transistor G1 (symbol Tr7) and the gate transistor G2 (symbol Tr8) in the second subframe are the same with respect to the timing of irradiation of the light pulse PO. That is, in each of the first and second subframes, the accumulation timings of the two gate transistors G are made to overlap.

[0098] In this comparative example, charges can be accumulated at the same timing in each of the first and second subframes, based on the irradiation timing of the light pulse PO. Therefore, even if the number of integrations in each subframe is different, it is possible to calculate the distance without performing level adjustment as in the embodiment. However, in each of the first and second subframes, the accumulation timing of the two gate transistors G must overlap. This shortens the range in which distance measurement is possible.

[0099] In contrast to these comparative examples, in this embodiment, only one gate transistor G is controlled to have the same accumulation timing in each of the first and second subframes. That is, the gate transistor G4 in the first subframe and the gate transistor G1 in the second subframe are controlled to have the same accumulation timing. This minimizes the number of gate transistors G whose accumulation timings overlap. Therefore, it is possible to calculate distance without narrowing the range in which distance measurement is possible, and even if the number of integrations differs between subframes.

[0100] Furthermore, in the range image pickup device 1 of the embodiment, the range image processing unit 4 performs level adjustment and calculates the distance using the level-adjusted signal value. The level adjustment is a process of multiplying a signal value corresponding to the amount of charge accumulated in each charge accumulation unit CS in either the first subframe or the second subframe by a constant so that the first signal value and the second signal value are the same. The first signal value is a signal value corresponding to the amount of charge accumulated in the charge accumulation unit CS4 (last charge accumulation unit), which is the charge accumulation unit CS in which charge is accumulated at the last accumulation timing in the first subframe. The second signal value is a signal value corresponding to the amount of charge accumulated in the charge accumulation unit CS1 (first charge accumulation unit), which is the charge accumulation unit CS in which charge is accumulated at the first accumulation timing in the second subframe. This allows the range image pickup device 1 of the embodiment to calculate the distance using the level-adjusted signal value. Therefore, the correct distance can be calculated even if reflected light RL is received across the first and second subframes.

[0101] Furthermore, in the distance image pickup device 1 of the embodiment, the distance image processing unit 4 determines whether or not a charge corresponding to the reflected light RL (reflected light of the light pulse) has been accumulated in the first signal value. The distance image processing unit 4 performs level adjustment if a charge corresponding to the reflected light has been accumulated in the last charge accumulation unit. This allows the distance image pickup device 1 of the embodiment to perform level adjustment if a reflected light component has been accumulated in the charge accumulation unit CS4 (last charge accumulation unit) in the first subframe. This makes it possible to avoid performing level adjustment if a reflected light component has not been accumulated in the charge accumulation unit CS4 (last charge accumulation unit) in the first subframe. Therefore, if a reflected light component has not been accumulated, i.e., if only an external light component has been accumulated, level adjustment is not performed, thereby preventing level adjustment using a signal value that is prone to fluctuation due to noise contamination, etc. This allows for accurate distance calculation.

[0102] In this embodiment, the combination of subframe 1 and subframe 2 has been described, but the present invention is not limited to this and can be applied to a combination of subframe n and subframe (n+1). Here, n is an arbitrary integer. For example, if subframes 1 and 2 are followed by subframes 3 and 4, it is possible to perform driving in the same manner as in this embodiment using at least one combination of subframes 1 and 2, subframes 2 and 3, and subframes 3 and 4.

[0103] The range image capture device 1 and range image processing unit 4 in the above-described embodiment may be implemented in whole or in part by a computer. In this case, a program for implementing these functions may be recorded on a computer-readable recording medium and then loaded and executed by a computer system. Note that the term "computer system" as used herein includes hardware such as an OS and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into a computer system. Furthermore, the term "computer-readable recording medium" may also include devices that dynamically store programs for a short period of time, such as communication lines used when transmitting programs via networks such as the Internet or telephone lines, or devices that store programs for a fixed period of time, such as volatile memory within a computer system acting as a server or client. The program may also be designed to implement some of the above-described functions, or may be capable of implementing the above-described functions in combination with a program already stored in the computer system, or may be implemented using a programmable logic device such as an FPGA.

[0104] Although an embodiment of the present invention has been described above in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs within the scope of the gist of the present invention. [Explanation of symbols]

[0105] 1...Distance image capturing device 2...Light source section 3...Light receiving section 32...Distance image sensor 321...pixels 323...Vertical scanning circuit 4...Distance image processing section 41...Timing control section 42...Distance calculation section 43...Measurement control section CS…Charge storage section PO...light pulse

Claims

1. a light source unit that irradiates a measurement space, which is a space to be measured, with a light pulse; a light receiving unit including pixels each including a photoelectric conversion element that generates a charge according to incident light and a plurality of charge accumulation units that accumulate the charge, and a pixel drive circuit that distributes and accumulates the charge in each of the charge accumulation units at a predetermined timing synchronized with the irradiation of the light pulse; a distance image processing unit that calculates a distance to a subject present in the measurement space based on the amount of charge accumulated in each of the charge accumulation units; Equipped with One frame is provided with a plurality of subframes including a first subframe and a second subframe, The distance image processing unit causing the charge accumulation unit of the pixel to accumulate the charge in each subframe in the order of the first subframe and the second subframe, and controlling so that a final accumulation timing at which the charge is accumulated in the charge accumulation unit of the pixel at the latest timing in the first subframe and a first accumulation timing at which the charge is accumulated in the charge accumulation unit of the pixel at the earliest timing in the second subframe are the same timing with respect to a timing at which the light pulse is irradiated; performing level adjustment by multiplying by a constant the signal values ​​corresponding to the amounts of charge accumulated in each of the charge accumulation units in either the first subframe or the second subframe so that a first signal value corresponding to the amount of charge accumulated in a final charge accumulation unit, which is the charge accumulation unit in which the charge was accumulated at the final accumulation timing, and a second signal value corresponding to the amount of charge accumulated in an initial charge accumulation unit, which is the charge accumulation unit in which the charge was accumulated at the initial accumulation timing, are the same value; and calculating the distance using the level-adjusted signal values. Range imaging device.

2. the distance image processing unit determines whether or not the charge corresponding to the reflected light of the light pulse has been accumulated in the final charge accumulation unit, and performs the level adjustment if the charge corresponding to the reflected light has been accumulated in the final charge accumulation unit.

2. The distance imaging device according to claim 1.

3. a light receiving unit having a light source unit that irradiates a measurement space, which is a space to be measured, with a light pulse; pixels each having a photoelectric conversion element that generates a charge according to the incident light and a plurality of charge accumulation units that accumulate the charge; and a pixel drive circuit that distributes and accumulates the charge in each of the charge accumulation units at a predetermined timing synchronized with the irradiation of the light pulse; and a distance image processing unit that calculates a distance to a subject present in the measurement space based on the amount of charge accumulated in each of the charge accumulation units, One frame is provided with a plurality of subframes including a first subframe and a second subframe, The distance image processing unit causing the charge accumulation unit of the pixel to accumulate the charge in each subframe in the order of the first subframe and the second subframe, and controlling so that a final accumulation timing at which the charge is accumulated in the charge accumulation unit of the pixel at the latest timing in the first subframe and a first accumulation timing at which the charge is accumulated in the charge accumulation unit of the pixel at the earliest timing in the second subframe are the same timing with respect to a timing at which the light pulse is irradiated; performing level adjustment by multiplying by a constant the signal values ​​corresponding to the amounts of charge accumulated in each of the charge accumulation units in either the first subframe or the second subframe so that a first signal value corresponding to the amount of charge accumulated in a final charge accumulation unit, which is the charge accumulation unit in which the charge was accumulated at the final accumulation timing, and a second signal value corresponding to the amount of charge accumulated in an initial charge accumulation unit, which is the charge accumulation unit in which the charge was accumulated at the initial accumulation timing, are the same value; and calculating the distance using the level-adjusted signal values. Range imaging method.

Images