Range image capturing device and range image capturing method

Abstract

A range image imaging device includes a light source unit, a light receiving unit, and a range image processing unit. The light receiving unit includes a pixel driving circuit and a pixel including a plurality of charge storage units. The distance image processing unit controls the pixel driving circuits so that each of the charge accumulation units accumulates charges, calculates the distance to the subject on the basis of the amount of charge accumulated in each of the charge accumulation units, and causes each of the charge accumulation units to accumulate charges in a plurality of driving modes, and calculates the distance to the subject on the basis of the amount of charge accumulated in each of the charge accumulation units. As a result of causing the charge storage units to store charges in each of the plurality of driving modes, the number of times of light reception by the charge storage unit having a small storage delay time among the charge storage units is less than the number of times of light reception by the charge storage unit having a large storage delay time. The number of times of light reception is the number of times in which the charge accumulation unit accumulates charges in a reflected light reception interval, which is a period from the irradiation timing until a reflected light reception time has elapsed.

Description

Technical Field

[0001] This invention relates to a distance image capturing device and a distance image capturing method.

[0002] This application claims priority based on Japanese Patent Application No. 2023-129765 filed on August 9, 2023, the contents of which are incorporated herein by reference. Background Technology

[0003] A distance image camera device has been developed that uses the speed of light to measure the distance between the measuring device and the object based on the time of flight of light in space (measurement space) (for example, see Patent Document 1).

[0004] Existing technical documents Patent documents Patent Document 1: Japanese Patent No. 4235729 Summary of the Invention

[0005] The problem that the invention aims to solve In such distance imaging devices, when simultaneously measuring objects at close range (near-distance object) and distant range (far-distance object), reflected light from the near-distance object is received at a higher intensity than reflected light from the distant object. In this case, multiple reflections, sometimes referred to as a spotting phenomenon, sometimes occur in optical systems such as lenses. If a spotting phenomenon occurs, the reflected light from the near-distance object overlaps with the pixels that receive the reflected light from the distant object, making it difficult to accurately measure the distance to the distant object. As a means of suppressing this spotting phenomenon, special processing of the lens is considered to reduce the occurrence of spotting, but this sometimes increases manufacturing costs.

[0006] The present invention is made based on the above-mentioned problems, and its object is to provide a distance image imaging device and a distance image imaging method that can suppress the phenomenon of light spots without increasing manufacturing costs.

[0007] Methods for solving problems The distance image capturing device of the present invention includes: a light source unit that irradiates a subject with a light pulse; a light receiving unit having a pixel and a pixel driving circuit, the pixel having a photoelectric conversion element that generates a charge corresponding to the incident light and a plurality of charge storage units that store the charge, the pixel driving circuit distributing and storing charge to the charge storage units at a storage timing synchronized with the irradiation timing of the light pulse; and a distance image processing unit that controls the pixel driving circuit to cause the charge storage units to store charge, and calculates the distance to the subject based on the amount of charge stored in each of the charge storage units. The distance image processing unit is configured to... Multiple driving modes with different numbers of charge accumulation sections allow the charge accumulation sections to accumulate charge. In each of the multiple driving modes, the result of the charge accumulation section accumulating charge is that the charge accumulation section with a smaller accumulation delay time (the difference between the irradiation time and the accumulation time) receives light less often than the charge accumulation section with a larger accumulation delay time. The number of times the charge accumulation section accumulates charge is the number of times it accumulates charge during the period from the irradiation time until the time corresponding to the measurable distance (i.e., the reflected light reception time).

[0008] The distance image capturing method of the present invention is a distance image capturing method performed by a distance image capturing device, which includes: a light source unit that irradiates a light pulse onto a subject; a light receiving unit having a pixel and a pixel driving circuit, the pixel having a photoelectric conversion element that generates a charge corresponding to the incident light, and a plurality of charge accumulating units for accumulating charge, the pixel driving circuit distributing and accumulating charge to the charge accumulating units at an accumulation timing synchronized with the irradiation timing of the light pulse; and a distance image processing unit that controls the pixel driving circuit to cause the charge accumulating units to accumulate charge, and calculates the distance to the subject based on the amount of charge accumulated in each of the charge accumulating units. The distance is determined by the distance image processing unit through multiple driving modes with different numbers of charge accumulation units, causing each charge accumulation unit to accumulate charge. In each of the multiple driving modes, the result of accumulating charge in the charge accumulation units is that the charge accumulation unit with a smaller accumulation delay time (the difference between the illumination time and the accumulation time) receives light less often than the charge accumulation unit with a larger accumulation delay time. The number of times the charge accumulation unit accumulates charge is the number of times it accumulates charge during the period from the illumination time to the time corresponding to the measurable distance (i.e., the reflected light reception time), i.e., the reflected light reception interval.

[0009] Invention Effects According to the present invention, the phenomenon of light spot can be suppressed without increasing manufacturing costs. Attached Figure Description

[0010] Figure 1 This is a block diagram showing the schematic structure of the distance image camera device 1 according to the embodiment.

[0011] Figure 2 This is a block diagram showing the schematic structure of the distance image sensor 32 in the embodiment.

[0012] Figure 3 This is a circuit diagram illustrating an example of the structure of pixel 321 in an embodiment.

[0013] Figure 4 This is a diagram illustrating the frame period of the implementation method.

[0014] Figure 5 This is a diagram illustrating the frame period of the implementation method.

[0015] Figure 6A This is a diagram illustrating the frame period of the implementation method.

[0016] Figure 6B This is a diagram illustrating the frame period of the implementation method.

[0017] Figure 6C This is a diagram illustrating the frame period of the implementation method.

[0018] Figure 7A This is a diagram illustrating the frame period of the implementation method.

[0019] Figure 7B This is a diagram illustrating the frame period of the implementation method.

[0020] Figure 8 This is a flowchart illustrating the process performed by the distance image camera device 1 in the embodiment. Detailed Implementation

[0021] Hereinafter, the distance image camera device according to the embodiment will be described with reference to the accompanying drawings.

[0022] Figure 1 This is a block diagram illustrating the schematic structure of a distance image capturing device according to an embodiment. The distance image capturing device 1 includes, for example, a light source unit 2, a light receiving unit 3, and a distance image processing unit 4. Figure 1 The image also shows the object whose distance is measured in the distance imaging device 1, namely the subject OB.

[0023] The light source unit 2, under the control of the distance image processing unit 4, irradiates a light pulse PO into the space containing the subject OB. The light source unit 2 is, for example, a surface-emitting semiconductor laser module such as a vertically oriented surface-emitting laser (VCSEL). The light source unit 2 includes a light source device 21 and a diffuser plate 22.

[0024] The light source device 21 is a laser light source that emits a near-infrared wavelength (e.g., a wavelength range of 850 nm to 940 nm) that forms a light pulse PO that irradiates the subject OB. The light source device 21 is, for example, a semiconductor laser light-emitting element. The light source device 21 emits pulsed laser light according to the control from the timing control unit 41.

[0025] The diffuser plate 22 is an optical component that diffuses the near-infrared laser emitted by the light source device 21 to the area of ​​the surface irradiating the subject OB. The pulsed laser light diffused by the diffuser plate 22 is emitted as a light pulse PO and irradiates the subject OB.

[0026] The light-receiving unit 3 receives the reflected light RL from the light pulse PO reflected by the subject OB, whose distance is measured in the distance image imaging device 1, and outputs a pixel signal corresponding to the received reflected light RL. The light-receiving unit 3 includes a lens 31 and a distance image sensor 32.

[0027] Lens 31 is an optical lens that guides the incident reflected light RL toward the distance image sensor 32. Lens 31 directs the incident reflected light RL toward the distance image sensor 32, so that the pixels in the light-receiving area of ​​the distance image sensor 32 are illuminated (incidentally).

[0028] The distance image sensor 32 is an imaging element used in the distance image imaging device 1. The distance image sensor 32 includes a plurality of pixels arranged in a two-dimensional matrix. Each pixel of the distance image sensor 32 includes a photoelectric conversion element, a plurality of charge storage sections corresponding to the photoelectric conversion element, and a structural element for distributing charge to each charge storage section. In other words, a pixel is an imaging element with a distribution structure that distributes and stores charge in a plurality of charge storage sections.

[0029] The distance image sensor 32, under control from the timing control unit 41, distributes the charge generated by the photoelectric conversion element to each charge storage unit. Furthermore, the distance image sensor 32 outputs a pixel signal corresponding to the amount of charge distributed to the charge storage units. In the distance image sensor 32, multiple pixels are arranged in a two-dimensional matrix, and each pixel outputs a corresponding one-frame pixel signal.

[0030] The distance image processing unit 4 controls the distance image capturing device 1 to calculate the distance to the subject OB. The distance image processing unit 4 includes a timing control unit 41, a distance calculation unit 42, and a measurement control unit 43.

[0031] The timing control unit 41, under the control of the measurement control unit 43, controls the timing of various control signals required for measurement. These various control signals include, for example, signals controlling the illumination of the light pulse PO, signals distributing and accumulating the reflected light RL in multiple charge accumulation units, and signals controlling the number of accumulations per frame. The number of accumulations refers to the number of times the process of distributing and accumulating charge in the charge accumulation units is repeated in a processing unit (e.g., 1 frame or subframe).

[0032] The distance calculation unit 42 calculates the distance to the subject OB based on the pixel signal output from the distance image sensor 32, and outputs the calculated distance. The distance calculation unit 42 calculates the delay time from the illumination light pulse PO to the reception of the reflected light RL based on the amount of charge stored in the multiple charge storage units, and calculates the distance to the subject OB based on the calculated delay time.

[0033] The measurement control unit 43 controls the timing control unit 41. For example, the measurement control unit 43 sets the cumulative number of frames, etc., and controls the timing control unit 41 in a way that captures images according to the set content.

[0034] With this structure, in the distance image camera device 1, the light receiving unit 3 receives the reflected light RL after the near-infrared light pulse PO irradiated by the light source unit 2 to the subject OB is reflected by the subject OB, and the distance image processing unit 4 outputs distance information obtained by measuring the distance to the subject OB.

[0035] In addition, Figure 1 The diagram shows a distance image camera 1 with a distance image processing unit 4 inside the distance image camera 1, but the distance image processing unit 4 may also be a structural element external to the distance image camera 1.

[0036] Here, use Figure 2 The structure of the distance image sensor 32 will be described. Figure 2 This is a block diagram showing the schematic structure of the imaging element (distance image sensor 32) used in the distance image imaging device 1 of the embodiment.

[0037] like Figure 2 As shown, the distance image sensor 32 includes, for example, a light-receiving area 320 with a plurality of pixels 321, a control circuit 322, a vertical scanning circuit 323 with a distribution operation, a horizontal scanning circuit 324, and a pixel signal processing circuit 325.

[0038] The light-receiving area 320 is an area configured with multiple pixels 321. Figure 2An example is shown with an 8x8 matrix arrangement. Pixels 321 accumulate charge corresponding to the amount of light received. Control circuit 322 uniformly controls distance image sensor 32. For example, control circuit 322 controls the operation of structural elements of distance image sensor 32 according to instructions from timing control unit 41 of distance image processing unit 4. Alternatively, the control of structural elements of distance image sensor 32 may be directly performed by timing control unit 41, in which case control circuit 322 can be omitted.

[0039] The vertical scanning circuit 323 is a circuit that controls the arrangement of pixels 321 in the light-receiving area 320 row by row according to the control from the control circuit 322. The vertical scanning circuit 323 causes the pixel signal processing circuit 325 to output a voltage signal corresponding to the amount of charge stored in each charge storage section CS of the pixel 321. For example, the vertical scanning circuit 323 distributes and stores the charge obtained by the photoelectric conversion element at a timing synchronized with the illumination of the light pulse PO into each charge storage section of the pixel 321. In addition, the vertical scanning circuit 323 discharges the charge obtained by the photoelectric conversion element at a timing different from the storage timing from the charge discharge section (the drain-gate transistor GD described later). In this case, the vertical scanning circuit 323 is an example of a "pixel driving circuit".

[0040] The pixel signal processing circuit 325 is a circuit that performs predetermined signal processing (e.g., noise suppression processing, A / D conversion processing, etc.) on the voltage signals output from each column of pixels 321 to the corresponding vertical signal lines according to the control from the control circuit 322.

[0041] The horizontal scanning circuit 324 is a circuit that, under the control of the control circuit 322, sequentially outputs the signal from the pixel signal processing circuit 325 to the horizontal signal line. Thus, pixel signals equivalent to the charge accumulated for one frame are sequentially output to the distance image processing unit 4 via the horizontal signal line.

[0042] The following explanation assumes that the pixel signal processing circuit 325 performs A / D conversion processing and that the pixel signal is a digital signal.

[0043] Here, use Figure 3 The structure of pixel 321 will be explained. Figure 3 This is a circuit diagram illustrating an example of the structure of a pixel 321 disposed within the light-receiving area 320 of the distance image sensor 32 in the embodiment. Figure 3 An example of the structure of one pixel 321 among a plurality of pixels 321 disposed within the light-receiving area 320 is shown. Pixel 321 is an example of a structure including four pixel signal readout sections.

[0044] Pixel 321 includes one photoelectric conversion element PD, a drain-gate transistor GD, and four pixel signal readout units RU that output voltage signals from corresponding output terminals O. Each pixel signal readout unit RU includes a readout gate transistor G, a floating diffusion region FD, a charge storage capacitor C, a reset gate transistor RT, a source follower gate transistor SF, and a select gate transistor SL. In each pixel signal readout unit RU, the floating diffusion region FD and the charge storage capacitor C constitute a charge storage section CS.

[0045] In addition, Figure 3 In the attached figures, each of the four pixel signal readout units RU is distinguished by assigning any number from "1" to "4" after the reference numeral "RU". Similarly, the various structural elements of the four pixel signal readout units RU are also distinguished by showing the numbers corresponding to each pixel signal readout unit RU after the reference numerals.

[0046] exist Figure 3 In the pixel 321 shown, the pixel signal readout unit RU1, which outputs a voltage signal from the output terminal O1, includes a readout gate transistor G1, a floating diffusion region FD1, a charge storage capacitor C1, a reset gate transistor RT1, a source follower gate transistor SF1, and a select gate transistor SL1. In the pixel signal readout unit RU1, the floating diffusion region FD1 and the charge storage capacitor C1 constitute the charge storage unit CS1. Pixel signal readout units RU2 to RU4 have the same structure.

[0047] A photoelectric conversion element (PD) is an embedded photodiode that converts incident light into electrical charge and stores the resulting charge. The structure of a PD can be arbitrary. For example, a PD can be a PN photodiode with a P-type semiconductor and an N-type semiconductor bonded together, or a PIN photodiode with an I-type semiconductor sandwiched between the P-type and N-type semiconductors. Furthermore, a PD is not limited to a photodiode; for example, it can also be a grating-type photoelectric conversion element.

[0048] In pixel 321, the photoelectric conversion element PD distributes the charge generated by photoelectric conversion of incident light to four charge storage units CS respectively, and outputs each voltage signal corresponding to the amount of charge distributed to the pixel signal processing circuit 325.

[0049] The structure of the pixels configured on the distance image sensor 32 is not limited to Figure 3The structure shown, which has four pixel signal readout units RU, can be used for any pixel having multiple pixel signal readout units RU. That is, the number of pixel signal readout units RU (charge storage units CS) provided by the pixel disposed on the distance image sensor 32 can be two, three, or more than five.

[0050] In addition, Figure 3 In the pixel 321 of the shown structure, an example is shown where the charge storage section CS is composed of a floating diffusion region FD and a charge storage capacitor C. However, the charge storage section CS can be composed of at least the floating diffusion region FD, or the pixel 321 can be a structure without the charge storage capacitor C.

[0051] In addition, Figure 3 In the pixel 321 of the shown structure, an example of a structure with a drain-gate transistor GD is presented. However, if the (residual) charge accumulated in the discarded photoelectric conversion element PD is not required, a structure without a drain-gate transistor GD may also be shown. The drain-gate transistor GD is an example of a "charge discharge section".

[0052] Here, in the distance image camera device 1, if a light spot phenomenon occurs when simultaneously measuring an object at close range (near-distance object) and an object at a distance (far-distance object), then the pixel 321 that receives the reflected light RL from the distant object is superimposed with the reflected light RL from the near-distance object, making it difficult to accurately measure the distance to the distant object.

[0053] As a countermeasure, in this embodiment, multiple driving modes are set. The result of driving multiple driving modes is that the time for receiving reflected light RL from a nearby object (exposure time) is shorter than the time for receiving reflected light RL from a distant object. That is, the result of accumulating charge in the charge accumulation section CS in each of the multiple driving modes is that the charge accumulation section CS with a shorter accumulation delay time receives light less times than the charge accumulation section CS with a longer accumulation delay time.

[0054] The accumulation delay time is the difference between the illumination timing and the accumulation timing. The illumination timing is the timing at which the illumination of the light pulse PO begins. The accumulation timing is the timing at which the operation of accumulating charge in the charge accumulation section CS begins (turning on the readout gate transistor G). In the charge accumulation section CS with a short accumulation delay time, the charge corresponding to the reflected light RL from a nearby object is accumulated. In the charge accumulation section CS with a long accumulation delay time, the charge corresponding to the reflected light RL from a distant object is accumulated.

[0055] The number of light receptions is the number of times the reflected light RL can be received. The number of light receptions is the time Tth from the initial illumination time to the time it takes for the reflected light to be received (refer to...). Figure 6A The number of times the charge accumulation section CS accumulates charge during the period from the irradiation timing to the reflection light reception time Tth is also called the reflection light reception interval. The reflection light reception time Tth is the time during which the reflected light RL is received after the irradiation light pulse PO. The reflection light reception time Tth is arbitrarily determined based on the distance that can be measured by the image imaging device 1. When the charge accumulation section CS has accumulated charge after the reflection light reception time Tth from the irradiation timing, the charge accumulation section CS does not accumulate the charge corresponding to the reflected light RL, but only the charge corresponding to the external light.

[0056] Thus, in this embodiment, pixel 321 is driven in a manner that reduces the exposure time (time for receiving reflected light RL from a nearby object). This reduces the number of times pixel 321, which receives reflected light RL from a distant object, receives light from the nearby object. Therefore, the distance to distant objects can be accurately measured.

[0057] The following explanation illustrates the case where multiple driving modes (first driving mode to fourth driving mode) are driven within a single frame. However, this is not a limitation. Multiple frames can be used, with each frame driving multiple driving modes. Multiple driving modes can also be driven by setting multiple subframes within a single frame.

[0058] Here, use Figure 4 An example of the frame period in the implementation method will be described. Figure 4 This is a diagram illustrating the frame period of the implementation method.

[0059] The diagram shows the timing of the elements corresponding to each item "LI", "G1" through "G4", and "GD". "LI" represents the illumination timing of the light pulse PO, which is the state where light is emitted if the transistor is in the on state and not if it is in the off state. "G1" through "G4" represent the accumulation timing of the readout gate transistors G1 through G4, which is the state where charge is accumulated if the transistor is in the on state and not if it is in the off state. "GD" represents the drive timing of the drain-gate transistor GD, which is the state where charge is discharged if the transistor is in the on state and not discharged if it is in the off state.

[0060] like Figure 4 As shown, the frame period includes, for example, an accumulation period and a readout period. During the accumulation period, the cycle is repeatedly executed a predetermined number of times N. During the cycle, the drive modes corresponding to the first to fourth drive modes are executed respectively.

[0061] Figure 4 The first driving mode is the reference driving mode, which is a driving mode in which all four charge accumulation sections CS accumulate charge sequentially during the accumulation timing synchronized with the irradiation timing. In the first driving mode, at the same timing as the irradiation timing of the irradiation light pulse PO, the drain gate transistor GD is turned off, and the readout gate transistors G1 to G4 are turned on sequentially.

[0062] Specifically, in the first driving mode, the timing control unit 41, via the control circuit 322, turns off the drain-gate transistor GD and turns on the readout-gate transistor G1 during the illumination timing. After a specific accumulation time To (e.g., the same time as the illumination time of the illumination light pulse PO) elapses from when the readout-gate transistor G1 turns on, it turns off the readout-gate transistor G1. During the timing when the readout-gate transistor G1 turns off, the readout-gate transistor G2 turns on. After the accumulation time To elapses from when the readout-gate transistor G2 turns on, it turns off the readout-gate transistor G2. During the timing when the readout-gate transistor G2 turns off, the readout-gate transistor G3 turns on. After the accumulation time To elapses from when the readout-gate transistor G3 turns on, it turns off the readout-gate transistor G3. During the timing when the readout-gate transistor G3 turns off, the readout-gate transistor G4 turns on. After an accumulation time To elapses from when the readout gate transistor G4 is turned on, the readout gate transistor G4 is turned off and the drain gate transistor GD is turned on.

[0063] Therefore, during the period when the readout gate transistors G1 to G4 are controlled to be in the on state, the charge obtained by photoelectric conversion by the photoelectric conversion element PD is stored in the charge storage sections CS1 to CS4. Then, when the readout gate transistor G4 is turned off, the drain gate transistor GD is turned on to discharge the charge. Thus, the charge obtained by photoelectric conversion by the photoelectric conversion element PD is discarded via the drain gate transistor GD. In the first driving mode, the drive that causes the charge storage sections CS1 to CS4 to accumulate charge in sequence is repeatedly executed for a predetermined number of accumulations N.

[0064] Figure 4The second driving mode is a driving mode in which the accumulation timing of each of the readout gate transistors G1 to G4 relative to the irradiation timing is set to the same timing as in the first driving mode, and the readout gate transistor G1 is not turned on. That is, the second driving mode is a driving mode in which, relative to the first driving mode, the charge accumulation section CS1 corresponding to the readout gate transistor G1 is not allowed to accumulate charge. In the second driving mode, this driving process, which sequentially accumulates charge in the charge accumulation sections CS2 to CS4, is repeatedly executed a predetermined number of accumulations N.

[0065] Specifically, in the second driving mode, the timing control unit 41, via the control circuit 322, turns off the drain-gate transistor GD and turns on the readout gate transistor G2 after an accumulation time To elapses from the illumination timing. After an accumulation time To elapses from when the readout gate transistor G2 turns on, it turns off the readout gate transistor G2. At the same time that the readout gate transistor G2 turns off, the readout gate transistor G3 turns on. After an accumulation time To elapses from when the readout gate transistor G3 turns on, the readout gate transistor G3 turns off. At the same time that the readout gate transistor G3 turns off, the readout gate transistor G4 turns on. After an accumulation time To elapses from when the readout gate transistor G4 turns on, the readout gate transistor G4 turns off and the drain-gate transistor GD turns on.

[0066] Figure 4 The third driving mode is a driving mode in which the accumulation timing of each of the readout gate transistors G1 to G4 relative to the irradiation timing is set to the same timing as in the first driving mode, and the readout gate transistors G1 and G2 are not turned on. That is, the third driving mode is a driving mode in which, relative to the first driving mode, the charge accumulation sections CS1 and CS2 corresponding to the readout gate transistors G1 and G2 are not allowed to accumulate charge. In the third driving mode, this driving process that sequentially accumulates charge in the charge accumulation sections CS3 to CS4 is defined as the third accumulation period, and the driving process corresponding to the third accumulation period is repeatedly executed a predetermined number of accumulation times N.

[0067] Specifically, in the third driving mode, the timing control unit 41, via the control circuit 322, turns off the drain-gate transistor GD and turns on the readout-gate transistor G3 after an accumulation time To×2 elapses from the irradiation timing. After an accumulation time To elapses from when the readout-gate transistor G3 turns on, the readout-gate transistor G3 turns off. At the timing when the readout-gate transistor G3 turns off, the readout-gate transistor G4 turns on. After an accumulation time To elapses from when the readout-gate transistor G4 turns on, the readout-gate transistor G4 turns off and the drain-gate transistor GD turns on.

[0068] Figure 4 The fourth driving mode is a driving mode in which the accumulation timing of each of the readout gate transistors G1 to G4 relative to the irradiation timing is set to the same timing as in the first driving mode, and the readout gate transistors G1 to G3 are not turned on. That is, the fourth driving mode is a driving mode in which, relative to the first driving mode, the charge accumulation sections CS1 to CS3 corresponding to the readout gate transistors G1 to G3 are not allowed to accumulate charge. In the fourth driving mode, this driving mode, which only causes the charge accumulation section CS4 to accumulate charge, is repeatedly executed a predetermined number of accumulations N.

[0069] Specifically, in the fourth driving mode, the timing control unit 41, via the control circuit 322, turns off the drain-gate transistor GD and turns on the readout gate transistor G4 after an accumulation time To×3 has elapsed since the irradiation timing. After an accumulation time To has elapsed since the readout gate transistor G4 turned on, the readout gate transistor G4 turns off and the drain-gate transistor GD turns on.

[0070] The readout period is the period during which an accumulation signal corresponding to the amount of charge accumulated in each charge accumulation unit CS is read out and output in each driving mode. For example, the timing control unit 41 turns on the selection gate transistor SL via the control circuit 322, causing the voltage (analog value) corresponding to the amount of charge accumulated in the charge accumulation units CS1 to CS4 to be output from the output terminals O1 to O4 to the pixel signal processing circuit 325. The pixel signal processing circuit 325 converts the analog values ​​output from the output terminals O1 to O4 into digital values ​​through A / D conversion processing and outputs the values ​​as accumulation signals Q1 to Q4.

[0071] Here, the accumulated signal Q1 is the digital signal value obtained by converting the analog value output from output terminal O1 into a digital value through A / D conversion. The accumulated signal Q2 is the digital signal value obtained by converting the analog value output from output terminal O2 into a digital value through A / D conversion. The accumulated signal Q3 is the digital signal value obtained by converting the analog value output from output terminal O3 into a digital value through A / D conversion. The accumulated signal Q4 is the digital signal value obtained by converting the analog value output from output terminal O4 into a digital value through A / D conversion.

[0072] The distance calculation unit 42 uses the accumulated signal output during the readout period to calculate the distance to the subject OB. The distance calculation unit 42 calculates the correction accumulated signal Q# corresponding to the amount of charge accumulated for each of the accumulated signals Q1 to Q4 and the number of accumulations N. For example, the distance D to the subject OB can be calculated using the following formula (1).

[0073] Q1#=Q1 Q2#=Q2 / 2 Q3#=Q3 / 3 Q4#=Q4 / 4 …(1) in, Q1# is the correction accumulation signal corresponding to the accumulation signal Q1.

[0074] Q2# is the correction accumulation signal corresponding to the accumulation signal Q2.

[0075] Q3# is the correction accumulation signal corresponding to the accumulation signal Q3.

[0076] Q4# is the correction accumulation signal corresponding to the accumulation signal Q4.

[0077] The distance calculation unit 42 uses the correction accumulation signal Q# to calculate the distance to the subject OB. For example, the distance to the subject OB can be calculated using the following equation (2). Equation (2) shows an example where the charge accumulation unit CS1 accumulates a charge equivalent to the external light, and the charge accumulation units CS2 and CS3 accumulate charges corresponding to the external light and the reflected light RL.

[0078] D=(Q3#-Q1#) / (Q2#+Q3#-2Q1#)×c / 2×Ta+c / 2×Tc…(2) in, D represents the distance to the subject OB.

[0079] Q1# is the correction accumulation signal corresponding to the accumulation signal Q1.

[0080] Q2# is the correction accumulation signal corresponding to the accumulation signal Q2.

[0081] Q3# is the correction accumulation signal corresponding to the accumulation signal Q3.

[0082] c represents the speed of light.

[0083] Ta represents the duration and amplitude of the irradiation pulse PO.

[0084] Tc represents the time difference from the time the light pulse PO is emitted until the gate transistor G2 is set to the on state.

[0085] In this way, after using Figure 4 During the frame cycle drive, in the first drive mode of the frame cycle, four charge accumulation units CS1 to CS4 accumulate charge respectively. In the second drive mode of the frame cycle, three charge accumulation units CS2 to CS4 accumulate charge respectively. In the third drive mode, two charge accumulation units CS3 to CS4 accumulate charge respectively. In the fourth drive mode, one charge accumulation unit CS4 accumulates charge. Moreover, in the first drive mode, the charge accumulation unit CS with the shortest accumulation delay time is charge accumulation unit CS1, whose illumination timing and accumulation timing are simultaneous. In the second to fourth drive modes, charge accumulation unit CS1 (the first charge accumulation unit) does not accumulate charge.

[0086] Therefore, in Figure 4 In the example, it is possible to control the time (exposed time A, described later) for pixel 321 to accumulate charge in response to reflected light RL from a nearby object to be reduced.

[0087] That is, in Figure 4 In the example, the time (exposure time A) for the charge accumulation unit CS1, which receives reflected light RL from a nearby object, to accumulate charge is the time obtained by multiplying the accumulation time To by the cumulative number N of the first driving mode. That is, exposure time A = To × N. On the other hand, the time (exposure time B) for the charge accumulation unit CS4, which receives reflected light RL from a distant object, to accumulate charge is the time obtained by multiplying the accumulation time To by the sum of the cumulative number of each of the first to fourth driving modes (4N). That is, exposure time B = To × (4N). In other words, the exposure time is controlled to be A < exposure time B.

[0088] In this way, by controlling the exposure time (A) for pixel 321 to receive reflected light RL from a nearby object to be shorter than the exposure time B, the number of times pixel 321, which receives reflected light RL from a distant object, receives light from the nearby object can be reduced. Therefore, the distance to a distant object can be accurately measured.

[0089] Here, use Figure 5 Another example of the frame period in the implementation method will be described. Figure 5 This is a diagram illustrating the frame period of the implementation method. In Figure 5 The diagram shows the time sequence of elements corresponding to each item in “LI”, “G1” through “G4”, and “GD”. The items in “LI”, “G1” through “G4”, and “GD” are... Figure 4 They are the same, therefore their description is omitted. Additionally, Figure 5 The frame period shown is Figure 4 Similarly, it includes an accumulation period and a readout period. During the accumulation period, the cycle is repeatedly executed a predetermined number of times N. During the cycle, the drive mode corresponding to the first to fourth drive modes is executed respectively.

[0090] exist Figure 5 In the frame period shown, a specific non-accumulation time Ts is set. The non-accumulation time Ts is the time during which the charge accumulation section CS does not accumulate charge. Figure 5 In the frame period shown, the control is such that from the irradiation timing until the non-accumulation time Ts, each charge accumulation section CS does not accumulate charge.

[0091] exist Figure 5 In the frame period shown, in each driving mode, after a non-accumulation time Ts elapsed from the illumination timing, the reflected light RL arriving at the distance image camera 1 is illuminated. That is, the distance image camera 1 does not illuminate the reflected light RL arriving before the non-accumulation time Ts elapsed from the illumination timing, i.e., the reflected light RL arriving from a nearby object. Therefore, by not illuminating the reflected light RL arriving from a nearby object, the number of times pixel 321, which illuminates the reflected light RL arriving from a distant object, is illuminated is reduced. Thus, the distance to a distant object can be accurately measured.

[0092] exist Figure 5 In the first driving mode, the accumulation timing of the charge accumulation section CS1 is determined based on the time point Ts elapsed since the irradiation timing, which is different from the non-accumulation time Ts. Figure 4 Different. On the other hand, in Figure 5 In the first driving mode, the principle of sequentially accumulating charge in all four charge accumulation sections CS is similar to... Figure 4 The first drive mode is the same.

[0093] Specifically, in the first driving mode, the timing control unit 41, via the control circuit 322, turns off the drain-gate transistor GD and turns on the readout gate transistor G1 at a time point elapsed after a non-accumulation time Ts from the irradiation timing. After a specific accumulation time To elapsed from when the readout gate transistor G1 turned on, the readout gate transistor G1 is turned off. At the timing when the readout gate transistor G1 is turned off, the readout gate transistor G2 is turned on. After the accumulation time To elapsed from when the readout gate transistor G2 turned on, the readout gate transistor G2 is turned off. At the timing when the readout gate transistor G2 is turned off, the readout gate transistor G3 is turned on. After the accumulation time To elapsed from when the readout gate transistor G3 turned on, the readout gate transistor G3 is turned off. At the timing when the readout gate transistor G3 is turned off, the readout gate transistor G4 is turned on. After an accumulation time To elapses from when the readout gate transistor G4 is turned on, the readout gate transistor G4 is turned off and the drain gate transistor GD is turned on.

[0094] exist Figure 5 In the first driving mode, after a non-accumulation time Ts has elapsed since the irradiation timing, the drive that causes the charge accumulation sections CS1 to CS4 to accumulate charge in sequence is repeatedly executed a predetermined number of accumulation times N.

[0095] exist Figure 5 In the second driving mode, the reference point is the time point at which the non-accumulation time Ts has elapsed since the irradiation timing. Figure 4 The differences lie in the fact that the three charge accumulation sections CS2 to CS4, excluding the charge accumulation section CS1 which has the shortest accumulation delay time, accumulate charge sequentially. Figure 4 The second driving mode is the same. In the second driving mode, after a non-accumulation time Ts from the irradiation timing, and then after an accumulation time To in the first frame for accumulating charge in charge accumulation unit CS1, the drive that causes charge accumulation units CS2 to CS4 to accumulate charge is repeatedly executed a predetermined number of accumulation times N.

[0096] Specifically, in the second driving mode, the timing control unit 41, via the control circuit 322, turns off the drain-gate transistor GD and turns on the readout gate transistor G2 after a non-accumulation time Ts and then an accumulation time To from the irradiation timing. After the accumulation time To from when the readout gate transistor G2 turns on, it turns off the readout gate transistor G2. At the timing when the readout gate transistor G2 turns off, it turns on the readout gate transistor G3. After the accumulation time To from when the readout gate transistor G3 turns on, it turns off the readout gate transistor G3. At the timing when the readout gate transistor G3 turns off, it turns on the readout gate transistor G4. After the accumulation time To from when the readout gate transistor G4 turns on, it turns off the readout gate transistor G4 and turns on the drain-gate transistor GD.

[0097] exist Figure 5 In the third driving mode, the reference point is the time point at which the non-accumulation time Ts has elapsed since the irradiation timing. Figure 4 The difference lies in the fact that the two charge accumulation sections CS3 to CS4, excluding the two charge accumulation sections CS1 to CS2 with short accumulation delay times, accumulate charge sequentially. Figure 4 The third driving mode is the same. In the third driving mode, the drive that causes the charge accumulation sections CS3 to CS4 to accumulate charge is repeatedly executed for a predetermined number of accumulation times N.

[0098] Specifically, in the third driving mode, the timing control unit 41, via the control circuit 322, turns off the drain-gate transistor GD and turns on the readout-gate transistor G3 after a non-accumulation time Ts and then an accumulation time To×2 from the irradiation timing. After an accumulation time To from when the readout-gate transistor G3 turns on, the readout-gate transistor G3 turns off. At the timing when the readout-gate transistor G3 turns off, the readout-gate transistor G4 turns on. After an accumulation time To from when the readout-gate transistor G4 turns on, the readout-gate transistor G4 turns off and the drain-gate transistor GD turns on.

[0099] exist Figure 5 In the fourth driving mode, based on the point in time Ts elapsed since the irradiation timing, it is consistent with... Figure 4 The difference lies in the fact that only the single charge accumulation section CS4, which maximizes the accumulation delay time, accumulates charge. Figure 4The fourth driving mode is the same. In the fourth driving mode, the driving that only causes the charge accumulation section CS4 to accumulate charge is taken as the fourth accumulation period, and the driving corresponding to the fourth accumulation period is repeatedly executed a predetermined number of accumulation times N.

[0100] Specifically, in the fourth driving mode, the timing control unit 41, via the control circuit 322, after a non-accumulation time Ts from the irradiation timing, and then after an accumulation time To×3, turns off the drain-gate transistor GD and turns on the readout-gate transistor G4. After an accumulation time To from turning on the readout-gate transistor G4, it turns off the readout-gate transistor G4 and turns on the drain-gate transistor GD.

[0101] Figure 5 During the readout period and Figure 4 The readout period is the same, therefore its description is omitted. Distance calculation unit 42 and... Figure 4 Similarly, the method described above can be used to calculate the distance to the subject OB using the accumulated signal output during readout.

[0102] In this way, after using Figure 5 During the frame cycle driving, the accumulation timing of the charge accumulation unit CS1 (first charge accumulation unit), which has the smallest accumulation delay time, is set to a timing after a predetermined non-accumulation time Ts from the illumination timing. Therefore, reflected light RL from nearby objects is not received, reducing the number of times pixel 321, which receives reflected light RL from distant objects, receives light from nearby objects. Thus, the distance to distant objects can be accurately measured.

[0103] Here, we use Figure 6 ( Figures 6A to 6C Another example of the frame period in the implementation method will be described below. Figure 6 is a diagram illustrating the frame period of the implementation method. Figure 6 shows a timing diagram of the elements corresponding to each item “LI”, “G1” to “G4”, and “GD”. Each item “LI”, “G1” to “G4”, and “GD” corresponds to… Figure 4 The same applies, therefore its description is omitted. Additionally, the frame period shown in Figure 6 is... Figure 4 Similarly, it includes an accumulation period and a readout period. During the accumulation period, the cycle is repeatedly executed a predetermined number of times N. During the cycle, the drive mode corresponding to the first to fourth drive modes is executed respectively.

[0104] exist Figure 6AIn the frame period shown, a specific reflected light reception time Tth is set. The reflected light reception time Tth is the time, assuming the reflected light RL is received by the image capturing device 1 after the illumination pulse PO, and corresponds to a measurable distance. That is, the reflected light RL arrives during the period from the illumination timing until the reflected light reception time Tth has elapsed (the reflected light reception interval). In other words, the reflected light RL does not arrive during time intervals different from the reflected light reception interval. During these time intervals when the reflected light RL does not arrive, charge accumulation units CS that did not accumulate charge during the reflected light reception interval are charged. By driving in this way, the accumulation count of each charge accumulation unit CS is set to the same number within the frame period.

[0105] exist Figure 6A In the frame period shown, during the period from the illumination timing to the elapsed reflected light reception time Tth (the reflected light reception interval), the process is performed with... Figure 4 The same driving method applies. Furthermore, in the driving mode where there is a charge accumulation section CS that has not accumulated charge in each driving mode, after the reflected light receiving time Tth from the irradiation timing (a time interval different from the reflected light receiving interval), that is, the time interval before the reflected light RL arrives, the charge accumulation section CS that has not accumulated charge in the reflected light receiving interval is charged.

[0106] Therefore, in Figure 6A In the frame period shown, in each driving mode, all charge accumulation sections CS accumulate charge, ensuring that the accumulation count of each charge accumulation section CS is the same within one frame. That is, the external light signal Qb (and the signal value corresponding to the amount of charge accumulated in the charge accumulation section CS corresponding to the amount of external light) contained in any accumulation signal Q is also the same. Therefore, for any accumulation signal Q, the accumulation signal (external light minus the accumulated signal Qa) without external light components can be calculated based on the same external light signal Qb. Furthermore, the number of times the external light component is accumulated is the same as... Figure 4 and Figure 5 The increased driving force allows for the calculation of the external light signal Qb, which suppresses external light deviations. Therefore, the external light signal Qb can be calculated with high precision, as can the value of the external light signal Qb subtracted from the accumulated signal Q. Consequently, the distance to the subject OB can be calculated with high precision.

[0107] Figure 6A First drive mode and Figure 4 The first drive mode is the same, so its description is omitted.

[0108] Figure 6A The second drive mode in relation to Figure 4In the second driving mode, the same accumulation timing is used to accumulate charge in the light-receiving interval of the reflected light, and the charge accumulation section CS2 to CS4 accumulate charge in the time interval different from the light-receiving interval of the reflected light, that is, the time interval when the reflected light RL does not arrive.

[0109] Specifically, in the second driving mode, the timing control unit 41, via the control circuit 322, turns off the drain-gate transistor GD and turns on the readout gate transistor G2 after an accumulation time To elapses from the illumination timing. After an accumulation time To elapses from when the readout gate transistor G2 turns on, it turns off the readout gate transistor G2. At the same time that the readout gate transistor G2 turns off, the readout gate transistor G3 turns on. After an accumulation time To elapses from when the readout gate transistor G3 turns on, the readout gate transistor G3 turns off. At the same time that the readout gate transistor G3 turns off, the readout gate transistor G4 turns on. After an accumulation time To elapses from when the readout gate transistor G4 turns on, the readout gate transistor G4 turns off and the drain-gate transistor GD turns on. Subsequently, after a reflected light reception time Tth from the irradiation timing, the drain-gate transistor GD is turned off and the readout-gate transistor G1 is turned on. After an accumulation time To from when the readout-gate transistor G1 is turned on, the readout-gate transistor G1 is turned off and the drain-gate transistor GD is turned on.

[0110] Thus, in the second driving mode, charge accumulation section CS1 accumulates charge during time intervals different from the reflected light receiving interval. During these time intervals, reflected light RL does not arrive. Therefore, in the second driving mode, only the charge corresponding to external light is accumulated in charge accumulation section CS1. In the second driving mode, this process of sequentially accumulating charge in charge accumulation sections CS2 to CS4 during the reflected light receiving interval and accumulating charge in charge accumulation section CS1 during time intervals different from the reflected light receiving interval is repeatedly executed a predetermined number of accumulations N.

[0111] Figure 6A The third drive mode in conjunction with Figure 4 In the same accumulation timing as the third driving mode, after the charge accumulation sections CS3 to CS4 accumulate charge in the light-receiving interval of the reflected light, the charge accumulation sections CS1 to CS2 accumulate charge in a time interval different from the light-receiving interval of the reflected light, that is, in the time interval when the reflected light RL does not arrive.

[0112] Specifically, in the third driving mode, the timing control unit 41, via the control circuit 322, turns off the drain-gate transistor GD and turns on the readout-gate transistor G3 after an accumulation time To×2 has elapsed since the irradiation timing. After an accumulation time To has elapsed since the readout-gate transistor G3 turned on, it turns off the readout-gate transistor G3. At the timing when the readout-gate transistor G3 turns off, it turns on the readout-gate transistor G4. After an accumulation time To has elapsed since the readout-gate transistor G4 turned on, it turns off the readout-gate transistor G4 and turns on the drain-gate transistor GD. Then, after a reflected light reception time Tth has elapsed since the irradiation timing, it turns off the drain-gate transistor GD and turns on the readout-gate transistor G1. After an accumulation time To has elapsed since the readout-gate transistor G1 turned on, it turns off the readout-gate transistor G1. At the timing when the read gate transistor G1 is turned off, the read gate transistor G2 is turned on. After an accumulation time To elapses from when the read gate transistor G2 is turned on, the read gate transistor G2 is turned off and the drain gate transistor GD is turned on.

[0113] Thus, in the third driving mode, charge accumulation units CS1 to CS2 accumulate charge during time intervals different from the reflected light receiving interval. During time intervals different from the reflected light receiving interval, the reflected light RL does not arrive. Therefore, in the third driving mode, only the charge corresponding to external light is accumulated in charge accumulation units CS1 to CS2. In the third driving mode, this drive, which sequentially accumulates charge in charge accumulation units CS3 to CS4 during the reflected light receiving interval and accumulates charge in charge accumulation units CS1 to CS2 during time intervals different from the reflected light receiving interval, is repeatedly executed a predetermined number of accumulations N.

[0114] Figure 6A The fourth drive mode in conjunction with Figure 4 The same accumulation timing as the fourth driving mode, after the charge accumulation section CS4 accumulates charge in the light-receiving interval of the reflected light, the charge accumulation sections CS1 to CS3 accumulate charge in a time interval different from the light-receiving interval of the reflected light.

[0115] Specifically, in the fourth driving mode, the timing control unit 41, via the control circuit 322, turns off the drain-gate transistor GD and turns on the readout gate transistor G4 after an accumulation time To×3 from the irradiation timing. After an accumulation time To from when the readout gate transistor G4 turns on, the readout gate transistor G4 turns off and the drain-gate transistor GD turns on. Then, after a reflected light reception time Tth from the irradiation timing, the drain-gate transistor GD turns off and the readout gate transistor G1 turns on. After an accumulation time To from when the readout gate transistor G1 turns on, the readout gate transistor G1 turns off. At the timing when the readout gate transistor G1 turns off, the readout gate transistor G2 turns on. After an accumulation time To from when the readout gate transistor G2 turns on, the readout gate transistor G2 turns off. At the timing when the read gate transistor G2 is turned off, the read gate transistor G3 is turned on. After an accumulation time To elapses from when the read gate transistor G3 is turned on, the read gate transistor G3 is turned off and the drain gate transistor GD is turned on.

[0116] Thus, in the fourth driving mode, charge accumulation sections CS1 to CS3 accumulate charge during time intervals different from the reflected light receiving interval. During time intervals different from the reflected light receiving interval, reflected light RL does not arrive. Therefore, in the fourth driving mode, only the charge corresponding to external light is accumulated in charge accumulation sections CS1 to CS3. In the fourth driving mode, this drive, which accumulates charge only in charge accumulation section CS4 during the reflected light receiving interval and in charge accumulation sections CS1 to CS3 during time intervals different from the reflected light receiving interval, is repeatedly executed a predetermined number of accumulations N.

[0117] Figure 6A During the readout period and Figure 4 The readout period is the same, so its description is omitted. The distance calculation unit 42 uses the accumulated signal output during the readout period to calculate the distance to the subject OB.

[0118] First, the distance calculation unit 42 calculates the external optical signal Qb. For example, the distance calculation unit 42 sets the minimum signal value among the accumulated signals Q1 to Q4 as the value of the external optical signal Qb. That is, the external optical signal Qb = min(Q1, Q2, Q3, Q4).

[0119] The distance calculation unit 42 calculates the external light subtraction storage signal Qa obtained by subtracting the external light signal Qb from the storage signals Q1 to Q4 respectively. Specifically, the external light subtraction storage signal Qa can be calculated using the following equation (3).

[0120] Qa1=Q1-Qb Qa2=Q2-Qb Qa3 = Q3 - Qb Qa4 = Q4 - Qb … (3) in, Qa1 is the external light minus the accumulated signal corresponding to the accumulated signal Q1.

[0121] Qa2 is the external light minus the accumulated signal corresponding to the accumulated signal Q2.

[0122] Qa3 is the external light minus the accumulated signal corresponding to the accumulated signal Q3.

[0123] Qa4 is the external light minus the accumulated signal corresponding to the accumulated signal Q4.

[0124] Qb is the external optical signal.

[0125] The distance calculation unit 42 calculates the distance D to the subject OB using the external light subtracted from the accumulated signal Qa (external light subtracted from the accumulated signal Qa1 to Qa4). Based on the sum of the external light subtracted from the accumulated signal Qa1 to Qa4 and the number of accumulations N, the distance calculation unit 42 calculates a correction signal Qa# corresponding to the amount of reflected light accumulated for each of the same number of accumulations.

[0126] Qa1#=Q1a Qa2#=Q2a / 2 Qa3#=Q3a / 3 Qa4#=Q4a / 4 …(4) in, Qa1# is the correction signal corresponding to the accumulated signal Qa1 after the external light is subtracted.

[0127] Qa2# is the correction signal corresponding to the accumulated signal Qa2 after the external light is subtracted.

[0128] Qa3# is the correction signal corresponding to the accumulated signal Qa3 after subtracting the external light.

[0129] Qa4# is the correction signal corresponding to the accumulated signal Qa4 after the external light is subtracted.

[0130] The distance calculation unit 42 uses the correction signal Qa# (correction signal Qa1#~Qa4#) to calculate the distance to the subject OB. For example, the distance D to the subject OB can be calculated using the following equation (5). Equation (5) shows an example of a case where the charge corresponding to the reflected light RL is stored in the charge storage units CS2 and CS3.

[0131] D=(Qa3#) / (Qa2#+Qa3#)×c / 2×Ta+c / 2×Tc…(5) in, D represents the distance to the subject OB.

[0132] Qa2# is the correction signal corresponding to the accumulated signal Qa2 after the external light is subtracted.

[0133] Qa3# is the correction signal corresponding to the accumulated signal Qa3 after subtracting the external light.

[0134] c represents the speed of light.

[0135] Ta represents the duration and amplitude of the irradiation pulse PO.

[0136] Tc represents the time difference from the time the light pulse PO is emitted until the gate transistor G2 is set to the on state.

[0137] In this way, after using Figure 6A During the frame cycle driving, in the first driving mode of the frame cycle, four charge accumulation units CS1 to CS4 accumulate charge in the reflected light receiving area. In the second driving mode of the frame cycle, three charge accumulation units CS2 to CS4 accumulate charge in the reflected light receiving area. In the third driving mode, two charge accumulation units CS3 to CS4 accumulate charge in the reflected light receiving area. In the fourth driving mode, one charge accumulation unit CS4 accumulates charge in the reflected light receiving area. In the first driving mode, the charge accumulation unit CS with the shortest accumulation delay time is charge accumulation unit CS1, whose illumination timing and accumulation timing are simultaneous. In the second to fourth driving modes, charge accumulation unit CS1 (the first charge accumulation unit) does not accumulate charge in the reflected light receiving area. Furthermore, in the second to fourth driving modes, the charge accumulation section CS1 (first charge accumulation section), which does not accumulate charge in the reflected light receiving region, accumulates charge in a time interval different from the reflected light receiving region. At this time, within a frame period, the result of the charge accumulation section CS1 accumulating charge is that the sum of the first and second counts is the same as the third count. The first count is the number of times the charge accumulation section CS1 (first charge accumulation section) accumulates charge in the reflected light receiving region in the first to fourth driving modes. The second count is the number of times the charge accumulation section CS1 (first charge accumulation section) accumulates charge in a time interval different from the reflected light receiving region in the first to fourth driving modes. The third count is the number of times the charge accumulation section CS4 (second charge accumulation section) accumulates charge in the reflected light receiving region in the first to fourth driving modes.

[0138] That is, in Figure 6AIn the example, the number of times each charge accumulation unit CS accumulates within one frame period can be controlled to be the same. The time (exposure time X) for charge accumulation unit CS1 is obtained by multiplying the accumulation time To by the sum of the cumulative counts N for each of the first drive mode to the fourth drive mode (4N). That is, exposure time X = To × (4N). On the other hand, the time (exposure time Y) for charge accumulation unit CS4 is obtained by multiplying the accumulation time To by the sum of the cumulative counts N for each of the first drive mode to the fourth drive mode (4N). That is, exposure time Y = To × (4N). That is, exposure time X = exposure time Y. Furthermore, in the reflected light receiving area, with Figure 4 Similarly, pixel 321 is driven, so the exposure time (A) is shorter than the exposure time (B) for receiving reflected light RL from a nearby object than for receiving reflected light RL from a distant object.

[0139] In this way, by controlling the time for receiving reflected light RL from a nearby object to be reduced, and by setting the number of accumulation cycles for each charge accumulation unit CS in the frame period to the same number, the amount of external light signal Qb contained in any accumulation signal Q can be set to the same value. Furthermore, by increasing the number of accumulation cycles for the external light component, an external light signal Qb that suppresses external light deviations can be calculated. Therefore, the distance to the subject OB can be calculated with high accuracy.

[0140] exist Figure 6B It shows Figure 6A Modification 1. In this modification, after performing the first to fourth driving modes, a fifth driving mode (external optical accumulation driving mode) is implemented. In the fifth driving mode, in Figure 6A In each driving mode, the driving involved in the accumulation of charge corresponding to the external light (only) is implemented in a time interval different from the light-receiving interval of the reflected light (the time interval during which the reflected light RL does not arrive).

[0141] exist Figure 6B In the middle, the first to fourth drive modes and Figure 4 The first to fourth drive modes in the frame period are the same.

[0142] Moreover, in Figure 6B Within the frame period, after the reflected light reception time Tth elapses from the start of the fourth driving mode, the fifth driving mode is implemented. In the fifth driving mode, the charge accumulation unit CS1 accumulates three charges, the charge accumulation unit CS2 accumulates two charges, and the charge accumulation unit CS3 accumulates one charge.

[0143] The figure shows an example of accumulating charge in the order of charge accumulation section CS1, charge accumulation section CS2, charge accumulation section CS3, charge accumulation section CS1, charge accumulation section CS2, and charge accumulation section CS1.

[0144] Specifically, in the fifth driving mode, the timing control unit 41, via the control circuit 322, turns off the drain-gate transistor GD and turns on the readout-gate transistor G1 after a reflected light reception time Tth elapsed since the start of the fourth driving mode. After an accumulation time To elapsed since turning on the readout-gate transistor G1, the readout-gate transistor G1 is turned off. At the same time that the readout-gate transistor G1 is turned off, the readout-gate transistor G2 is turned on. After an accumulation time To elapsed since turning on the readout-gate transistor G2, the readout-gate transistor G2 is turned off. At the same time that the readout-gate transistor G2 is turned off, the readout-gate transistor G3 is turned on. After an accumulation time To elapsed since turning on the readout-gate transistor G3, the readout-gate transistor G3 is turned off. At the same time that the readout-gate transistor G3 is turned off, the readout-gate transistor G1 is turned on. After an accumulation time To elapses since the read gate transistor G1 was turned on, the read gate transistor G1 is turned off. During the timing when the read gate transistor G1 is turned off, the read gate transistor G2 is turned on. After an accumulation time To elapses since the read gate transistor G2 was turned on, the read gate transistor G2 is turned off. During the timing when the read gate transistor G2 is turned off, the read gate transistor G1 is turned on. After an accumulation time To elapses since the read gate transistor G1 was turned on, the read gate transistor G1 is turned off and the drain gate transistor GD is turned on.

[0145] Thus, in Figure 6B In the fifth driving mode, no light pulse PO is irradiated, but charge corresponding only to external light is accumulated, thereby ensuring that the accumulation count of each charge accumulation unit CS is the same within the frame period. By setting the fifth driving mode, the time required to implement each driving mode (first driving mode to fourth driving mode) can be reduced compared to... Figure 6A Shortened. Therefore, with Figure 6A In comparison, it can shorten Figure 6B The cycle period. As a result, in Figure 6A and Figure 6B If the time required to drive one frame is set to be the same, Figure 6BIt has a shorter cycle time, thus allowing for a longer accumulation time or an increased number of accumulations. That is, it allows for a longer exposure time (time during which each charge accumulation section CS accumulates charge within one frame). By extending the exposure time, the signal strength (SN) of the accumulated charge can be improved, thereby increasing the accuracy of the distance D to the subject OB. Furthermore, in Figure 6A and Figure 6B Without changing the accumulation time and the number of times, and Figure 6A In comparison, Figure 6B The shorter loop period reduces the time required to drive one frame. That is, in Figure 6B This can shorten the measurement time, thus enabling high-precision measurement of fast-moving subjects.

[0146] exist Figure 6C It shows Figure 6A Variation 2. In this variation, firstly, we perform... Figure 6A The first to third driving modes are among the various driving modes. Then, after performing the first to third driving modes, the driving involved in the accumulation of charge corresponding to external light in different time intervals of the reflected light receiving interval (the time interval when the reflected light RL does not arrive) is uniformly implemented as the fourth driving mode (external light accumulation driving mode).

[0147] In this variation 2, during the drive to accumulate charge in the light-receiving region of the reflected light within one frame period, it is sufficient to execute multiple drives with different drive modes, or as follows: Figure 6C As shown, execute the first drive mode to the third drive mode.

[0148] Alternatively, it can be executed. Figure 4 The structure of at least two of the first to fourth drive modes shown. Figure 4 The first to fourth drive modes shown include at least two drive modes, such as any combination of the first drive mode and the second drive mode, the first drive mode and the third drive mode, the first drive mode and the fourth drive mode, the second drive mode and the third drive mode, the second drive mode and the fourth drive mode, and the third drive mode and the fourth drive mode.

[0149] exist Figure 6C In the middle, the first drive mode to the third drive mode and Figure 4 The first to third driving modes in the frame period are the same.

[0150] Moreover, in Figure 6CWithin the frame period, after the reflected light reception time Tth has elapsed since the start of the third driving mode, the fourth driving mode is implemented. In the fourth driving mode, the charge accumulation unit CS1 accumulates two charges, and the charge accumulation unit CS2 accumulates one charge.

[0151] The figure shows an example of accumulating charge in the order of charge accumulation section CS1, charge accumulation section CS2, and charge accumulation section CS1.

[0152] Specifically, in Figure 6C In the fourth driving mode, the timing control unit 41, via the control circuit 322, turns off the drain-gate transistor GD and turns on the readout gate transistor G1 after a reflected light reception time Tth elapsed since the start of the third driving mode. After an accumulation time To elapsed since turning on the readout gate transistor G1, the readout gate transistor G1 is turned off. At the same time that the readout gate transistor G1 is turned off, the readout gate transistor G2 is turned on. After an accumulation time To elapsed since turning on the readout gate transistor G2, the readout gate transistor G2 is turned off. At the same time that the readout gate transistor G2 is turned off, the readout gate transistor G1 is turned on. After an accumulation time To elapsed since turning on the readout gate transistor G1, the readout gate transistor G1 is turned off and the drain-gate transistor GD is turned on.

[0153] Thus, in Figure 6C In the fourth driving mode, no light pulse PO is irradiated; instead, only the charge corresponding to the external light is accumulated. Therefore, compared to... Figure 6A Compared to the comparison example that omits the fourth drive mode, Figure 6C The cycle period is relatively short. As a result, in the comparative example and Figure 6C If the time required to drive one frame is set to be the same, Figure 6C The shorter cycle time allows for a longer accumulation time or an increased number of accumulation cycles. That is, the exposure time of each charge accumulation section CS can be extended within the frame cycle. By extending the exposure time, the SN of the accumulation signal can be improved, thus improving the accuracy of the distance D to the subject OB. Furthermore, in the comparative examples and... Figure 6C Without changing the accumulation time and the number of accumulations, compared with the comparative example, in Figure 6C The shorter loop period reduces the time required to drive one frame. That is, in Figure 6C This can shorten the measurement time, thus enabling high-precision measurement of fast-moving subjects.

[0154] Here, Figure 7 is used ( Figures 7A-7BAnother example of the frame period in the implementation method will be described below. Figure 7 is a diagram illustrating the frame period of the implementation method. Figure 7 shows a timing diagram of the elements corresponding to each item “LI”, “G1” to “G4”, and “GD”. Each item “LI”, “G1” to “G4”, and “GD” corresponds to… Figure 4 The same applies, therefore its description is omitted. Additionally, the frame period shown in Figure 7 is... Figure 4 Similarly, it includes an accumulation period and a readout period. During the accumulation period, the cycle is repeatedly executed a predetermined number of times N. During the cycle, the drive mode corresponding to the first to fourth drive modes is executed respectively.

[0155] exist Figure 7A In the frame period shown, Figure 5 The non-accumulation time Ts and Figure 6A The reflected light reception time Tth shown is set.

[0156] exist Figure 7A In the frame period shown, from the irradiation timing until the non-accumulation time Ts, the charge accumulation sections CS are not allowed to accumulate charge, and the process is carried out in the reflected light receiving region. Figure 5 The same driver. And, with... Figure 6A Similarly, in each driving mode, in the driving mode where there is a charge accumulation section CS that has not accumulated charge, the charge accumulation section CS that has not accumulated charge in the reflected light receiving interval is charged in a time interval different from the reflected light receiving interval.

[0157] Therefore, in Figure 7A In the frame period shown, in each driving mode, the distance image imaging device 1 does not receive light from reflected light RL arriving before the illumination timing has elapsed for a non-accumulation time Ts, i.e., reflected light RL arriving from a nearby object. Furthermore, in one frame, the charge accumulation units CS accumulate the same number of times. Therefore, the number of times the pixel 321 that receives light from reflected light RL arriving from a distant object receives light from reflected light RL arriving from a nearby object can be reduced, and the distance calculation process can be simplified.

[0158] Figure 7A First drive mode and Figure 5 The first drive mode is the same, so its description is omitted.

[0159] exist Figure 7A In the second driving mode, based on the time point Ts after the non-accumulation time from the irradiation timing, the charge accumulation sections CS2 to CS4 accumulate charge in the reflected light receiving interval, and the charge accumulation section CS1 accumulates charge in a time interval different from the reflected light receiving interval.

[0160] Specifically, in the second driving mode, the timing control unit 41, via the control circuit 322, turns off the drain-gate transistor GD and turns on the readout gate transistor G2 after a non-accumulation time Ts and then an accumulation time To from the irradiation timing. After the accumulation time To from when the readout gate transistor G2 turns on, it turns off the readout gate transistor G2. At the timing when the readout gate transistor G2 turns off, it turns on the readout gate transistor G3. After the accumulation time To from when the readout gate transistor G3 turns on, it turns off the readout gate transistor G3. At the timing when the readout gate transistor G3 turns off, it turns on the readout gate transistor G4. After the accumulation time To from when the readout gate transistor G4 turns on, it turns off the readout gate transistor G4 and turns on the drain-gate transistor GD. Subsequently, after a reflected light reception time Tth from the irradiation timing, the drain-gate transistor GD is turned off and the readout-gate transistor G1 is turned on. After an accumulation time To from when the readout-gate transistor G1 is turned on, the readout-gate transistor G1 is turned off and the drain-gate transistor GD is turned on.

[0161] In the second driving mode, the driving process that takes the non-accumulation time Ts elapsed from the irradiation timing as a reference, in which the charge accumulation units CS2 to CS4 sequentially accumulate charge in the reflected light receiving interval, and in the time interval different from the reflected light receiving interval, the charge accumulation unit CS1 accumulates charge is defined as the second accumulation period. The driving process corresponding to the second accumulation period is repeatedly executed a predetermined number of times N.

[0162] exist Figure 7A In the third driving mode, based on the time point Ts after the non-accumulation time from the irradiation timing, the charge accumulation sections CS3 to CS4 accumulate charge in the reflected light receiving interval, and the charge accumulation sections CS1 to CS2 accumulate charge in a time interval different from the reflected light receiving interval.

[0163] Specifically, in the third driving mode, the timing control unit 41, via the control circuit 322, turns off the drain-gate transistor GD and turns on the readout-gate transistor G3 after a non-accumulation time Ts and then an accumulation time To×2 from the irradiation timing. After an accumulation time To from when the readout-gate transistor G3 turns on, the readout-gate transistor G3 turns off. At the timing when the readout-gate transistor G3 turns off, the readout-gate transistor G4 turns on. After an accumulation time To from when the readout-gate transistor G4 turns on, the readout-gate transistor G4 turns off and the drain-gate transistor GD turns on. Then, after a reflected light reception time Tth from the irradiation timing, the drain-gate transistor GD turns off and the readout-gate transistor G1 turns on. After an accumulation time To from when the readout-gate transistor G1 turns on, the readout-gate transistor G1 turns off. At the timing when the read gate transistor G1 is turned off, the read gate transistor G2 is turned on. After an accumulation time To elapses from when the read gate transistor G2 is turned on, the read gate transistor G2 is turned off and the drain gate transistor GD is turned on.

[0164] In the third driving mode, the drive, which takes the non-accumulation time Ts elapsed from the irradiation timing as a reference, and causes the charge accumulation parts CS3 to CS4 to accumulate charge sequentially in the reflected light receiving interval, and causes the charge accumulation parts CS1 to CS2 to accumulate charge in a time interval different from the reflected light receiving interval, is repeatedly executed for a predetermined number of accumulation times N.

[0165] exist Figure 7A In the fourth driving mode, based on the time point Ts after the non-accumulation time from the irradiation timing, the charge accumulation section CS4 accumulates charge in the reflected light receiving interval, and the charge accumulation sections CS1 to CS3 accumulate charge in time intervals different from the reflected light receiving interval.

[0166] Specifically, in the fourth driving mode, the timing control unit 41, via the control circuit 322, turns off the drain-gate transistor GD and turns on the readout-gate transistor G4 after a non-accumulation time Ts and then an accumulation time To×3 from the irradiation timing. After an accumulation time To from when the readout-gate transistor G4 turns on, the readout-gate transistor G4 turns off and the drain-gate transistor GD turns on. Then, after a reflected light reception time Tth from the irradiation timing, the drain-gate transistor GD turns off and the readout-gate transistor G1 turns on. After an accumulation time To from when the readout-gate transistor G1 turns on, the readout-gate transistor G1 turns off. At the timing when the readout-gate transistor G1 turns off, the readout-gate transistor G2 turns on. After an accumulation time To from when the readout-gate transistor G2 turns on, the readout-gate transistor G2 turns off. At the timing when the read gate transistor G2 is turned off, the read gate transistor G3 is turned on. After an accumulation time To elapses from when the read gate transistor G3 is turned on, the read gate transistor G3 is turned off and the drain gate transistor GD is turned on.

[0167] In the fourth driving mode, the driving process is based on the time point Ts after the irradiation timing, in which the charge accumulation section CS4 accumulates charge in the reflected light receiving interval and the charge accumulation sections CS1 to CS3 accumulate charge in time intervals different from the reflected light receiving interval, and the driving process is repeatedly executed for a predetermined number of accumulation times N.

[0168] Figure 7A During the readout period and Figure 4 The readout period is the same, therefore its description is omitted. Distance calculation unit 42 and... Figure 6A Similarly, the method described above can be used to calculate the distance to the subject OB using the accumulated signal output during readout.

[0169] In this way, after using Figure 7A In the frame cycle drive, in multiple drive modes, it is controlled so that reflected light RL arriving at the image capturing device 1 before the non-accumulation time Ts has elapsed since the illumination timing is started is not exposed to light. Based on this, it is controlled so that the exposure time A < exposure time B in the reflected light exposure interval, and the exposure time X = exposure time Y in the frame cycle.

[0170] Therefore, the distance image capturing device 1 does not receive reflected light RL that arrives before the non-accumulation time Ts from the illumination timing, and the exposure time A < exposure time B in the reflected light receiving interval. This reduces the number of times pixel 321, which receives reflected light RL from distant objects, receives reflected light RL from nearby objects. Furthermore, by controlling the exposure time X = exposure time Y in the frame period, the distance calculation process can be simplified.

[0171] exist Figure 7B It shows Figure 7A Modification 1. In this modification, after performing the first to fourth driving modes, a fifth driving mode (external optical accumulation driving mode) is implemented. In the fifth driving mode, in Figure 7A In each driving mode, the driving involved in the accumulation of charge corresponding to the external light (only) is implemented in a time interval different from the light-receiving interval of the reflected light (the time interval during which the reflected light RL does not arrive).

[0172] exist Figure 7B In the middle, the first to fourth drive modes and Figure 5 The first to fourth drive modes in the frame period are the same.

[0173] Moreover, in Figure 7B Within the frame period, the fifth driving mode is implemented after the reflected light reception time Tth has elapsed since the start of the fourth driving mode.

[0174] Figure 7B The fifth drive mode and Figure 6B The fifth drive mode is the same.

[0175] Thus, in Figure 7B In the fifth driving mode, without irradiating the light pulse PO, charge corresponding only to external light is accumulated, thereby ensuring that the accumulation counts of each charge accumulation unit CS are the same within the frame period. By setting the fifth driving mode, and... Figure 7A Compared to other methods, this reduces the time required to implement each drive mode (first drive mode to fourth drive mode). Therefore, compared to... Figure 7A In comparison, it can shorten Figure 7B The cycle period. As a result, in Figure 7A and Figure 7B If the time required to drive one frame is set to be the same, Figure 7B It has a shorter cycle time, thus allowing for a longer accumulation time or an increased number of accumulations. That is, it allows for a longer exposure time (time during which each charge accumulation section CS accumulates charge within one frame). By extending the exposure time, the signal strength (SN) of the accumulated charge can be improved, thereby increasing the accuracy of the distance D to the subject OB. Furthermore, in Figure 7Aand Figure 7B Without changing the accumulation time and the number of times, and Figure 7A In comparison, Figure 6B The shorter loop period reduces the time required to drive one frame. That is, in Figure 7B This can shorten the measurement time, thus enabling high-precision measurement of fast-moving subjects.

[0176] Additionally, as in Figure 6C As illustrated in the example, in Figure 7A and Figure 7B As shown in the example, it can also be a structure that executes at least two of the first to fourth drive modes. At least two of the first to fourth drive modes include, for example, any combination of the first and second drive modes, the first and third drive modes, the first and fourth drive modes, the second and third drive modes, the second and fourth drive modes, and the third and fourth drive modes.

[0177] Here, use Figure 8 The process of processing performed by the distance image camera device 1 is explained. Figure 8 This is a flowchart illustrating the processing flow performed by the distance image capturing device 1 in the embodiment. Figure 8 The diagram illustrates the process of calculating the distance to the subject OB after a drive corresponding to the frame period.

[0178] Step S10: Read the accumulated signals Q1 to Q4 from the distance image camera device 1.

[0179] Step S11: The distance image camera device 1 calculates the accumulated signals Qa1 to Qa4 after subtracting the external light. For example, the distance image camera device 1 sets the smallest signal value among the accumulated signals Q1 to Q4 as the external light signal Qb, which is equivalent to the amount of light from the external light alone. As shown in equation (3), the distance image camera device 1 calculates the accumulated signals Qa1 to Qa4 after subtracting the external light signal Qb from the accumulated signals Q1 to Q4 respectively.

[0180] Step S12: The distance image camera device 1 calculates the corrected accumulation signals (Qa1#~Qa4a#). The distance image camera device 1 calculates the corrected accumulation signals (Qa1#~Qa4#) by normalizing the accumulation signals Qa1~Qa4 after subtracting the external light according to the accumulation number N. For example, as shown in Equation (4), the distance image camera device 1 calculates the corrected accumulation signals (Qa1#~Q4a#) by normalizing the accumulation signals Qa1~Qa4 after subtracting the external light by a value corresponding to the accumulation number N.

[0181] Step S13: The distance image capturing device 1 calculates the distance to the subject OB using the calibration accumulation signals (Qa1# to Qa4#).

[0182] Step S14: The distance image capturing device 1 determines whether the drive executed during the frame period is a drive with a non-accumulation time Ts set.

[0183] Step S15: In the case of a drive with a non-accumulation time Ts set, the distance image capturing device 1 corrects the distance calculated in Step S13 based on the non-accumulation time Ts.

[0184] As described above, the distance image capturing device 1 of the embodiment includes a light source unit 2, a light receiving unit 3, and a distance image processing unit 4. The light source unit 2 irradiates the subject OB with a light pulse PO. The light receiving unit 3 includes pixels 321 and a control circuit 322 (pixel drive circuit). Each pixel 321 includes a photoelectric conversion element PD and a plurality of charge accumulation units CS. The photoelectric conversion element PD generates charges corresponding to the incident light. The charge accumulation units CS accumulate charges. The vertical scanning circuit 323 distributes and accumulates charges to the charge accumulation units CS respectively at an accumulation timing synchronized with the irradiation of the light pulse PO through the frame period. The distance image processing unit 4 controls the control circuit 322 (pixel drive circuit) to cause the charge accumulation units CS to accumulate charges respectively. The distance image processing unit 4 calculates the distance D to the subject OB based on the amounts of charges accumulated in the charge accumulation units CS respectively. The distance image processing unit 4 sets a plurality of drive modes. As a result of driving the plurality of drive modes, the number of light reception times of the charge accumulation units CS with a smaller accumulation delay time in the charge accumulation units CS is less than that of the charge accumulation units CS with a larger accumulation delay time. The number of light reception times is the number of times the charge accumulation units CS accumulate charges in the reflected light reception section.

[0185] Thus, in the distance image capturing device 1 of the embodiment, it is possible to control the time (exposure time A) for the pixels 321 that receive the reflected light RL from a nearby object to accumulate charges to become smaller, and it is possible to reduce the number of times the pixels 321 that receive the reflected light RL from a nearby object receive the reflected light RL from a distant object. Therefore, it is possible to accurately measure the distance to a distant object.

[0186] In addition, in the distance image capturing device 1 of the embodiment, in the first drive mode among the plurality of drive modes, the plurality of N (N≥2) charge accumulation units CS accumulate charges respectively. In the second drive mode, M (M < N) charge accumulation units CS accumulate charges. And, the first charge accumulation unit, which is the charge accumulation unit CS with the smallest accumulation delay time in the first drive mode, does not accumulate charges in the second drive mode.

[0187] Therefore, in the distance image camera device 1 of the embodiment, the time (exposure time A) for the pixel 321 to accumulate charge in order to receive reflected light RL from a nearby object can be reduced, achieving the same effect as described above.

[0188] Furthermore, in the distance image camera device 1 of the embodiment, the accumulation timing of the charge accumulation unit CS1 (first charge accumulation unit) with the smallest accumulation delay time is set to a timing after a predetermined non-accumulation time Ts from the irradiation timing.

[0189] Therefore, in the distance image camera device 1 of the embodiment, the reflected light RL from a nearby object is not received, which reduces the number of times the pixel 321 that receives the reflected light RL from a nearby object can receive the light, achieving the same effect as described above.

[0190] Furthermore, in the distance image camera device 1 of the embodiment, in the second driving mode, the charge accumulation section CS1 (first charge accumulation section), which has not accumulated charge in the reflected light receiving area, accumulates charge in a time interval different from the reflected light receiving area. At this time, in each of the multiple driving modes, the result of accumulating charge in the charge accumulation section CS1 is that the sum of the first and second counts is the same as the third count. The first count is the number of times the charge accumulation section CS1 (first charge accumulation section) accumulates charge in the reflected light receiving area. The second count is the number of times the charge accumulation section CS1 (first charge accumulation section) accumulates charge in a time interval different from the reflected light receiving area. The third count is the number of times the charge accumulation section CS4 (second charge accumulation section) accumulates charge in the reflected light receiving area.

[0191] Therefore, in the distance image camera device 1 of the embodiment, the number of times the pixel 321 that receives reflected light RL from distant objects receives light from nearby objects can be reduced, and the distance calculation process can be simplified.

[0192] Furthermore, in the distance image camera device 1 of the embodiment, after multiple driving modes, there is an external light accumulation driving mode in which no light pulse PO is irradiated and charge corresponding only to external light is accumulated. In the external light accumulation driving mode, charge is accumulated in the charge accumulation section CS1 (first charge accumulation section) that has not accumulated charge in the second driving mode. The accumulation timing of the first charge accumulation section in the external light accumulation driving mode is a timing after the irradiation timing of the last driving mode among the multiple driving modes, elapsed after the reflected light reception time Tth.

[0193] This improves the signal SN of the accumulated signal, thus increasing the accuracy of the distance D to the subject OB. Alternatively, it enables high-precision measurement of fast-moving subjects.

[0194] All or part of the distance image capturing device 1 and the distance image processing unit 4 in the above embodiments can also be implemented by a computer. In this case, it can also be implemented by recording the program for implementing this function on a computer-readable recording medium, and having the computer system read and execute the program recorded on the recording medium. Furthermore, the term "computer system" here includes hardware such as an operating system and peripheral devices. In addition, "computer-readable recording medium" refers to removable media such as floppy disks, optical disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into the computer system. Moreover, "computer-readable recording medium" can also include a medium that dynamically holds a program for a short period of time, such as a communication line in the case of transmitting a program via a network such as the Internet or a communication line such as a telephone line, or a medium that holds a program for a certain period of time, such as volatile memory inside a computer system that serves as a server or client in this case. In addition, the above-mentioned program can be a program for implementing a part of the above-mentioned function, or a program that can implement the above-mentioned function by combining it with a program already recorded in the computer system, or a program implemented using a programmable logic device such as an FPGA.

[0195] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the specific structure is not limited to these embodiments and may include designs that do not depart from the spirit of the present invention.

[0196] Explanation of reference numerals in the attached figures 1…distance image camera device; 2…Light source section; 3…light-receiving part; 32… distance image sensor; 321… pixels (pixel circuit); 4… Distance Image Processing Unit; 42… Distance calculation unit; CS…charge storage section; PO…light pulse; RL…reflected light.

Claims

1. A distance image camera device, comprising: The light source irradiates the subject with light pulses; The light-receiving section includes a pixel and a pixel driving circuit. The pixel comprises a photoelectric conversion element that generates a charge corresponding to the incident light, and multiple charge storage sections for accumulating charge. The pixel driving circuit distributes and accumulates charge to the charge storage sections at an accumulation timing synchronized with the irradiation timing of the light pulse. The distance image processing unit controls the pixel driving circuit to cause the charge accumulation sections to accumulate charge, and calculates the distance to the subject based on the amount of charge accumulated in each charge accumulation section. The distance image processing unit is, By employing multiple driving modes with varying numbers of charge accumulation sections, the charge accumulation sections can accumulate charge individually. In each of the plurality of driving modes, the result of accumulating charge in the charge accumulation section is that the charge accumulation section with a smaller accumulation delay time (the difference between the irradiation timing and the accumulation timing) receives light less often than the charge accumulation section with a larger accumulation delay time. The number of times the light is received refers to the number of times the charge accumulation portion accumulates charge during the period from the irradiation time to the time corresponding to the measurable distance, i.e., the reflected light reception time, i.e., the reflected light reception interval.

2. The distance image camera device according to claim 1, wherein, The distance image processing unit is, In the first driving mode of the plurality of driving modes, N of the charge accumulation portions are respectively charged, where N≥2. In the second driving mode of the plurality of driving modes, M of the charge accumulation sections accumulate charge, wherein M <N, The charge accumulation section that minimizes the accumulation delay time in the first driving mode, i.e., the first charge accumulation section, does not accumulate charge in the second driving mode.

3. The distance image camera device according to claim 2, wherein, The distance image processing unit sets the accumulation timing of the first charge accumulation unit to a timing period after a predetermined non-accumulation time has elapsed since the irradiation timing.

4. The distance image capturing device according to claim 2 or 3, wherein, The distance image processing unit is, In the second driving mode, the first charge accumulation section accumulates charge in a time interval different from the light-receiving interval of the reflected light. In each of the plurality of driving modes, the result of accumulating charge in the charge accumulation section is such that the sum of the first number of times the first charge accumulation section accumulates charge in the reflected light receiving interval and the second number of times the first charge accumulation section accumulates charge in a time interval different from the reflected light receiving interval is the same as the third number of times the second charge accumulation section, which is different from the first charge accumulation section, accumulates charge in the reflected light receiving interval.

5. The distance image camera device according to claim 2 or 3, wherein, Following the multiple driving modes, there is also an external light accumulation driving mode, which accumulates charges corresponding only to external light without irradiating the light pulse. In the external light accumulation driving mode, the first charge accumulation section, which did not accumulate charge in the second driving mode, accumulates charge. The accumulation timing of the first charge accumulation unit in the external light accumulation driving mode is the timing after the reflection light reception time, starting from the irradiation timing of the last driving mode among the plurality of driving modes.

6. A distance image capturing method, performed by a distance image capturing device, the distance image capturing device comprising: a light source unit that irradiates a subject with a light pulse; a light receiving unit having a pixel and a pixel driving circuit, the pixel having a photoelectric conversion element that generates a charge corresponding to the incident light, and a plurality of charge accumulating units for accumulating charge, the pixel driving circuit distributing and accumulating charge to the charge accumulating units at an accumulation timing synchronized with the irradiation timing of the light pulse; and a distance image processing unit that controls the pixel driving circuit to cause the charge accumulating units to accumulate charge, and calculates the distance to the subject based on the amount of charge accumulated in each of the charge accumulating units. The distance image processing unit, By employing multiple driving modes with varying numbers of charge accumulation sections, the charge accumulation sections can accumulate charge individually. In each of the plurality of driving modes, the result of accumulating charge in the charge accumulation section is that the charge accumulation section with a smaller accumulation delay time (the difference between the irradiation timing and the accumulation timing) receives light less often than the charge accumulation section with a larger accumulation delay time. The number of times the light is received refers to the number of times the charge accumulation portion accumulates charge during the period from the irradiation time to the time corresponding to the measurable distance, i.e., the reflected light reception time, i.e., the reflected light reception interval.

Images