Range image camera device and range image camera method

By introducing a detailed measurement mode into the distance image sensor, adjusting the phase and pulse width of the irradiated light, and dividing the distance range, the problem of insufficient measurement accuracy of the distance image sensor in the wide distance range in the prior art is solved, and the proximity is achieved High-precision measurement of distance and long-distance areas.

CN114026460BActive Publication Date: 2024-11-29TOPPAN HOLDINGS INC
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN201980097886.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2019-07-04
Publication Date
2024-11-29
Estimated Expiration
2039-07-04

AI Technical Summary

Technical Problem

Existing distance image sensors are difficult to achieve optimal measurement accuracy for close and long distance areas within a wide distance range, resulting in a decrease in distance resolution.

Method used

By introducing a detailed measurement mode into the distance image imaging device, the phase and pulse width of the irradiated light are adjusted, the distance range is divided into multiple sub-measurement ranges, and charge accumulation and distribution are performed according to the charge amount of reflected light, so as to improve the close and long distances measurement accuracy.

Benefits of technology

It is achieved to improve measurement accuracy for close and long-distance areas within a wide distance range, and improve distance resolution.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114026460B_ABST
    Figure CN114026460B_ABST
Patent Text Reader

Abstract

The distance image camera device of the present invention includes: a light source unit, which irradiates the space of the measurement object, that is, the measurement space, with irradiation light; a light receiving pixel unit, which includes: a photoelectric conversion element, which receives reflected light reflected by the irradiation light on the object in the measurement space and the background light in the environment of the measurement space, and generates charges corresponding to the received reflected light and background light; and a charge accumulation unit, which accumulates the charges when irradiated with the irradiation light in a frame period, and the light receiving pixel unit has a pixel circuit that accumulates the charges in the charge accumulation unit synchronously with the irradiation of the irradiation light; and a distance image processing unit, which measures the distance in a normal mode of a prescribed width of the irradiation light when using an input voltage corresponding to the amount of charge accumulated in the charge accumulation unit to measure the distance, and changes to a detailed measurement mode corresponding to the distance to the object measured in the normal mode to adjust the phase of the irradiation light irradiated from the light source unit.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a distance image capturing device and a distance image capturing method. Background Art

[0002] In the past, a time of flight (hereinafter referred to as "TOF") type distance image sensor was realized, which measures the distance between a measuring instrument and an object based on the flight time of light in a space (measurement space) by utilizing the fact that the speed of light is known. In a TOF type distance image sensor, a light pulse (for example, near-infrared light, etc.) is irradiated to a measurement object, and the distance between the measuring instrument and the object is measured based on the difference between the time of the irradiated light pulse and the time when the light pulse (reflected light) reflected by the object in the measurement space returns, that is, the flight time of light between the measuring instrument and the object (for example, refer to Patent Document 1).

[0003] The TOF distance image sensor can set the measurable distance range by using the width of the light pulse. Therefore, the TOF distance image sensor can measure the distance in a wider distance range as the width of the light pulse increases.

[0004] In terms of usage, the TOF distance image sensor is used by setting the light pulse width in accordance with the required distance measurement range.

[0005] Furthermore, in order to eliminate the influence of background light in the measurement space environment, a period during which no light is irradiated is set, only the amount of background light received is accumulated, and the component caused by the background light is subtracted from the signal containing information on the delay of the reflected light, thereby eliminating the influence of the background light.

[0006] Prior art literature

[0007] Patent Literature

[0008] Patent Document 1: Japanese Patent Application Publication No. 2004-294420 Summary of the invention

[0009] Problem that the invention aims to solve

[0010] As described above, in Patent Document 1, in the measurement space, the distance between the object and the distance image sensor in the measurement range corresponding to the width of the prescribed light pulse is measured. The measurement range represents the distance from the distance image sensor from the minimum distance (short distance) that can be measured by the reflected light from the object after the light pulse is irradiated to the maximum distance (long distance) that can be measured using the pulse width, for example, the distance from the distance image sensor to the minimum of 0.1m to the maximum of 4m.

[0011] However, since a wide distance range is measured, it is not possible to optimize the measurement accuracy (distance resolution) in each of the short distance and the long distance corresponding to each of the short distance and the long distance.

[0012] In view of the above-mentioned problems, the object of the present invention is to provide a range image camera device and a range image camera method, which can improve the distance measurement accuracy (distance resolution) in the respective areas of short distance and long distance in the range from the minimum distance (short distance) that can be measured by the reflected light from the object after irradiating a light pulse to the maximum distance (long distance) that can be measured using the pulse width.

[0013] Solutions for solving problems

[0014] The distance image camera device of the present invention includes: a light source unit, which irradiates irradiation light to a space of a measurement object, namely, a measurement space; a light receiving pixel unit, which includes: a photoelectric conversion element, which receives reflected light reflected by the irradiation light on an object in the measurement space and background light in the environment of the measurement space, and generates electric charges corresponding to the received reflected light and the background light; and a charge accumulation unit, which accumulates the electric charges when irradiated with the irradiation light in a frame period, and the light receiving pixel unit has a pixel circuit that accumulates the electric charges in the charge accumulation unit in synchronization with the irradiation of the irradiation light; and a distance image processing unit, which measures the distance in a normal mode of a prescribed width of the irradiation light when performing distance measurement using an input voltage corresponding to the amount of charge accumulated in the charge accumulation unit, and changes to a detailed measurement mode corresponding to the distance to the object measured in the normal mode to adjust the phase of the irradiation light irradiated from the light source unit.

[0015] In the distance image camera device of the present invention, the distance image processing unit measures the distance in the normal mode of the width of the irradiation light specified, and adjusts the width together with the phase of the irradiation light irradiated from the light source unit in the detailed measurement mode according to the distance to the object.

[0016] For the distance image camera device of the present invention, in the detailed measurement mode, the range of distances that can be measured in the normal mode, i.e., the distance range, is divided into multiple sub-measurement ranges of the same width, the width of the irradiation light is set corresponding to the sub-measurement range, and the phase of the irradiation light is set corresponding to the minimum distance in the sub-measurement range.

[0017] In the distance image camera device of the present invention, after measuring the distance in the normal mode, the distance image processing unit performs distance measurement in the detailed measurement mode based on the width and phase of the irradiation light set corresponding to the sub-measurement range including the distance obtained by the measurement.

[0018] For the distance image camera device of the present invention, the distance image processing unit distributes the charge with a predetermined fixed number of charge distributions in the normal mode, and calculates the distance to the object existing in the measurement space based on the amount of charge accumulated in the multiple distributed charge storage units in the charge storage unit, that is, the charge amount. After the distance measurement in the normal mode, the distance measurement in the detailed measurement mode is performed based on the charge amount in the charge distribution number set corresponding to the sub-measurement range including the distance calculated by the measurement.

[0019] In the distance image capturing device of the present invention, the distance image processing unit adjusts the intensity of the irradiation light irradiated from the light source unit according to the distance to the object.

[0020] With respect to the distance image camera device of the present invention, when the distance image processing unit calculates the distance to the object based on the amount of the charge accumulated in the two charge accumulation units that accumulate the charge of the reflected light, namely, the first allocated charge accumulation unit and the second allocated charge accumulation unit, that is, the ratio of the charge amounts, after switching to the detailed measurement mode, if the amount of charge generated by the reflected light in either the first allocated charge accumulation unit or the second allocated charge accumulation unit is below a predetermined charge amount threshold, the processing for calculating the distance is not performed and the mode is switched to the normal mode.

[0021] In the distance image capturing device of the present invention, the charge amount threshold is the charge amount accumulated in the background light charge accumulation unit using background light.

[0022] In the distance image camera device of the present invention, when the distance to the object is calculated based on the amount of the charge accumulated in the two charge storage units that accumulate the charge of the reflected light, namely, the first allocated charge storage unit and the second allocated charge storage unit, that is, the ratio of the charge amounts, the distance image processing unit adjusts the phase of the irradiated light after measuring the distance in the normal mode so that the respective charge amounts of the first allocated charge storage unit and the second allocated charge storage unit are the same, and calculates the distance based on the charge amounts and the adjustment amounts of the phases.

[0023] For the distance image camera device of the present invention, after adjusting the phase of the irradiated light to make the charge amount the same, the distance image processing unit adjusts the width of the storage drive signal for distributing the charge to the first distributed charge storage unit and the second distributed charge storage unit respectively, so that there is no area that does not contain the charge generated by the reflected light.

[0024] The distance image capturing method of the present invention comprises the following steps: irradiating a space of a measurement object, i.e., a measurement space, with irradiation light from a light source unit; causing a photoelectric conversion element to receive reflected light reflected by the irradiation light on an object in the measurement space and background light in the environment of the measurement space, and generating electric charges corresponding to the received reflected light and the background light; accumulating the electric charges generated by the reflected light in a charge accumulation unit in synchronization with the irradiation of the irradiation light in a frame cycle; measuring the distance in a normal mode of a prescribed width of the irradiation light when performing distance measurement using an input voltage corresponding to the amount of charge accumulated in the charge accumulation unit; and switching to a detailed measurement mode to adjust the phase of the irradiation light irradiated from the light source unit in accordance with the distance to the object measured in the normal mode.

[0025] Effects of the Invention

[0026] The present invention provides a distance image camera device and a distance image camera method, which respectively improve the distance measurement accuracy (distance resolution) in the respective areas of short distance and long distance in the range from the minimum distance (short distance) that can be measured by reflected light from an object after irradiating a light pulse to the maximum distance (long distance) that can be measured using the pulse width. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a block diagram showing a schematic configuration of a distance image capturing device according to a first embodiment of the present invention.

[0028] Figure 2 This is a block diagram showing a schematic configuration example of a distance image sensor used in the distance image capturing device 1 according to the embodiment of the present invention.

[0029] Figure 3 This is a circuit diagram showing an example of the configuration of a pixel circuit 321 disposed in a light receiving pixel unit 320 of an image pickup element (range image sensor 32) used in the range image pickup device 1 according to the embodiment of the present invention.

[0030] Figure 4 This is a timing chart showing the timing of driving the pixel circuit 321 disposed in the light receiving pixel unit 320 of the imaging element (range image sensor 32) used in the range image capturing device 1 according to the embodiment of the present invention.

[0031] Figure 5 This is a conceptual diagram showing a configuration example of an AD conversion circuit that performs AD conversion on an input voltage supplied from a pixel signal processing circuit according to an embodiment of the present invention.

[0032] Figure 6 It is a block diagram showing a configuration example of the distance calculation unit 42 in the distance image processing unit 4 according to the first embodiment.

[0033] Figure 7 It is a timing chart for explaining the phase of the light pulse PO in each of the normal mode, the short distance mode, and the long distance mode during the charge accumulation period of the first embodiment.

[0034] Figure 8 This is a flowchart showing an operation example of the distance measurement process performed by the distance image capturing device 1 according to the first embodiment.

[0035] Fig. 9 A diagram for explaining the effect of distance measurement in the detail measurement mode in the distance image capturing device 1 according to the present embodiment.

[0036] Fig.10 It is a timing chart for explaining the phase of the light pulse PO in each of the normal mode, the short distance mode, and the long distance mode during the charge accumulation period of another configuration of the present embodiment.

[0037] Fig.11 It is a block diagram showing a configuration example of the distance calculation unit 42A in the distance image processing unit 4 according to the second embodiment.

[0038] Fig.12 This is a diagram for explaining adjustment of the phase of the emitted light pulse PO by the variation adjustment unit 428 according to the second embodiment.

[0039] Fig.13 This is a flow chart showing an operation example of a process of adjusting the phase of the light pulse PO by the range image capturing device 1 according to the second embodiment.

[0040] Fig.14 It is a block diagram showing a configuration example of the distance calculation unit 42B in the distance image processing unit 4 according to the third embodiment.

[0041] Fig.15 It is a diagram for explaining adjustment of the pulse widths of the storage drive signals TX1 , TX2 , and TX3 according to the third embodiment.

[0042] Fig.16 This is a flowchart showing an operation example of the adjustment process of the pulse widths of the storage drive signals TX1 , TX2 , and TX3 by the distance image capturing device 1 according to the third embodiment. DETAILED DESCRIPTION

[0043] <First embodiment>

[0044] Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. Figure 1 is a block diagram showing a schematic structure of a distance image capturing device according to this embodiment. Figure 1 2 also shows a subject S which is a subject whose distance is measured by the distance image capturing device 1 .

[0045] Figure 1 The distance image capturing device 1 of the structure shown includes a light source unit 2 , a light receiving unit 3 , and a distance image processing unit 4 .

[0046] The light source unit 2 irradiates intermittent light pulses PO at a predetermined period to a space where a subject S, whose distance is to be measured in the distance image pickup device 1, exists, based on control from the distance image processing unit 4. The light source unit 2 is, for example, a surface emitting semiconductor laser module such as a vertical cavity surface emitting laser (VCSEL: Vertical Cavity Surface Emitting Laser).

[0047] The light source device 21 is, for example, a light source that emits a laser beam in a near-infrared wavelength range (for example, a wavelength range of 850 nm to 940 nm) as a light pulse PO irradiated to the subject S. The light source device 21 is, for example, a semiconductor laser light emitting element. The light source device 21 emits a pulsed laser beam (light pulse PO) under control from the timing control unit 41.

[0048] The diffuser plate 22 is an optical component that diffuses the laser beam in the near-infrared wavelength range emitted by the light source device 21 to a predetermined cross-sectional area for irradiation to the measurement space P where the subject S exists. The pulsed laser beam diffused by the diffuser plate 22 is emitted from the light source unit 2 as a light pulse PO and irradiates the subject S in the measurement space P.

[0049] The light receiving unit 3 receives reflected light RL of the light pulse PO reflected by the subject S as a target for distance measurement in the distance image capturing apparatus 1 , and outputs a pixel signal corresponding to the received reflected light RL.

[0050] The lens 31 is an optical lens that guides the incident reflected light RL toward the distance image sensor 32. The lens 31 emits the incident reflected light RL toward the distance image sensor 32, and causes the pixels provided in the light receiving area of ​​the distance image sensor 32 to receive the light (enter).

[0051] The distance image sensor 32 is an imaging element used in the distance image imaging device 1. The distance image sensor 32 is an imaging element of a distribution structure, and the distance image sensor 32 has a plurality of pixels in a two-dimensional light receiving area, and each pixel has a photoelectric conversion element, a plurality of charge storage units corresponding to the photoelectric conversion element, and a structural element that distributes charges to each charge storage unit. The distance image sensor 32 distributes the charges generated by the photoelectric conversion elements constituting the pixels to each charge storage unit under the control of the timing control unit 41, and outputs a pixel signal corresponding to the amount of charge distributed to each charge storage unit.

[0052] In addition, in the distance image sensor 32 , a plurality of pixels are arranged in a two-dimensional grid (row and column), and pixel signals corresponding to each pixel for one frame are output.

[0053] The distance image processing unit 4 is a control unit that controls the entire distance image capturing apparatus 1 and is also a calculation unit that calculates the distance to the subject S measured in the distance image capturing apparatus 1. The distance image processing unit 4 includes a timing control unit 41 and a distance calculation unit 42.

[0054] The timing control unit 41 controls the timing at which the light source unit 2 irradiates the light pulse PO to the subject S, the timing at which the distance image sensor 32 included in the light receiving unit 3 receives the reflected light RL, and the like.

[0055] The distance calculation unit 42 outputs distance information obtained by calculating the distance between the distance image capturing apparatus 1 and the subject S based on the pixel signal output from the distance image sensor 32 .

[0056] According to such a structure, in the distance image camera 1, the light receiving unit 3 receives the reflected light RL reflected by the subject S from the light pulse PO in the near-infrared wavelength range irradiated by the light source unit 2 to the subject S, and the distance image processing unit 4 outputs the distance information measured to the subject S.

[0057] In addition, Figure 1 , the distance image capturing device 1 is shown to include the distance image processing unit 4 therein, but the distance image processing unit 4 may be a component included outside the distance image capturing device 1 .

[0058] Next, the configuration of the distance image sensor 32 used as an imaging element in the distance image imaging device 1 will be described. Figure 2 1 is a block diagram showing a schematic configuration of an imaging element (range image sensor 32) used in the range image imaging device 1 according to the embodiment of the present invention. Figure 2In the embodiment, the distance image sensor 32 includes a light receiving pixel unit 320 configured with a plurality of pixel circuits 321, a control circuit 322, a vertical scanning circuit 323, a horizontal scanning circuit 324, a pixel signal processing circuit 325, and a pixel driving circuit 326. Figure 2 The illustrated distance image sensor 32 shows an example of a light-receiving pixel unit 320 in which a plurality of pixel circuits 321 are arranged in a two-dimensional grid pattern of 8 rows and 8 columns.

[0059] The control circuit 322 controls the components of the distance image sensor 32, such as the vertical scanning circuit 323, the horizontal scanning circuit 324, the pixel signal processing circuit 325, and the pixel driving circuit 326. The control circuit 322 controls the operation of the components of the distance image sensor 32, for example, based on the control from the distance image processing unit 4 (more specifically, the timing control unit 41) of the distance image camera device 1. In addition, the following structure may be adopted: the control of the components of the distance image sensor 32 by the control circuit 322 is directly performed by the distance image processing unit 4 (more specifically, the timing control unit 41). In this case, the distance image sensor 32 may also be a structure not having the control circuit 322.

[0060] The pixel driving circuit 326 outputs the storage driving signal (storage driving signals TX1, TX2, TX3 and reset driving signal RSTD described later) in units of columns of pixel circuits 321 arranged in a grid pattern in the light-receiving pixel section 320. The storage driving signal distributes and accumulates the charges generated by the photoelectric conversion elements (photoelectric conversion elements PD described later) of the pixel circuits 321 arranged in a grid pattern in the pixel circuits 321 in a plurality of charge storage sections (charge storage sections CS1, CS2 and CS3 described later) in the pixel circuits 321.

[0061] The vertical scanning circuit 323 is a driving circuit that controls each pixel circuit 321 arranged in the light-receiving pixel section 320 according to the control from the control circuit 322, and outputs (reads out) a signal of a voltage corresponding to the amount of charge after the incident light is photoelectrically converted (hereinafter referred to as a "voltage signal") from each pixel circuit 321 to the corresponding vertical signal line 327. The vertical scanning circuit 323 outputs a control signal (hereinafter described selection drive signal SEL1, SEL2, SEL3) for driving (controlling) and reading the pixel circuit 321 in units of rows of pixel circuits 321 arranged in a grid shape in the light-receiving pixel section 320.

[0062] Thus, a voltage signal corresponding to the amount of charge distributed to each charge storage unit in the pixel circuit 321 is read out from each vertical signal line 327 corresponding to each row of the light receiving pixel unit 320 and output to the pixel signal processing circuit 325 .

[0063] In the light-receiving pixel section 320, the pixel circuit 321 receives the reflected light RL reflected by the subject S from the light pulse PO irradiated by the light source section 2, and generates a charge corresponding to the amount of light (the amount of light received) of the received reflected light RL. In each pixel circuit 321, the pixel driving circuit 326 distributes and accumulates the charge corresponding to the amount of light (the amount of light received) of the received reflected light RL in any one of the plurality of charge accumulation sections by outputting an accumulation driving signal. Moreover, in the pixel circuit 321, the vertical scanning circuit 323 outputs a voltage signal of a size corresponding to the amount of charge distributed and accumulated in each charge accumulation section to the corresponding vertical signal line 327 by outputting a reading driving signal. In addition, a detailed description of the structure and driving (control) method of the pixel circuit 321 will be described later.

[0064] The pixel signal processing circuit 325 is a signal processing circuit that performs predetermined signal processing on the voltage signal output from the pixel circuit 321 of each column to the corresponding vertical signal line 327 according to the control from the vertical scanning circuit 323. The predetermined signal processing includes, for example, noise suppression processing for suppressing noise included in the voltage signal using correlated double sampling (CDS).

[0065] The AD conversion circuit 329 performs AD conversion on the analog voltage signal for each column supplied from the pixel signal processing circuit 325 via the vertical signal line 330 , and converts the signal into a digital value.

[0066] In addition, the pixel signal processing circuit 325 may be a pixel signal processing circuit group including a plurality of pixel signal processing circuits corresponding to the columns of the light-receiving pixel units 320. In this case, the pixel signal processing circuit 325 outputs a voltage signal after a predetermined signal processing to the AD conversion circuit 329 via the vertical signal line 330 according to the control from the control circuit 322.

[0067] Then, the AD conversion circuit 329 outputs each row of the light receiving pixel unit 320 to the horizontal signal line 338 according to the control of the horizontal scanning circuit 324 .

[0068] The horizontal scanning circuit 324 is a driving circuit that outputs (reads out) digital values ​​of the voltage signal after the signal processing from the AD conversion circuit 329 via the signal line 328 according to the control from the control circuit 322. The horizontal scanning circuit 324 sequentially outputs the readout driving signal for outputting the voltage signal corresponding to the pixel circuit 321 of each column to the pixel signal processing circuit 325. As a result, the voltage signal of one frame after the signal processing output by the pixel signal processing circuit 325 is sequentially output to the outside of the distance image sensor 32 via the horizontal signal line 338 as the pixel signal of one frame. At this time, the distance image sensor 32 outputs the voltage signal after the signal processing as the pixel signal to the outside of the distance image sensor 32 from the output circuit (not shown) such as the output amplifier.

[0069] In the following description, the following situation is used for explanation: the pixel signal processing circuit 325 of the distance image sensor 32 performs noise suppression processing on the voltage signal output from the pixel circuit 321, and then performs A / D conversion processing in the AD conversion circuit 329 and outputs it, that is, the voltage signal converted into a digital value is output from the horizontal signal line 338.

[0070] Next, the configuration of the pixel circuit 321 disposed in the light receiving pixel unit 320 included in the distance image sensor 32 will be described. Figure 3 1 is a circuit diagram showing an example of the structure of a pixel circuit 321 arranged in a light receiving pixel section 320 of an imaging element (range image sensor 32) used in the range image imaging device 1 according to the embodiment of the present invention. Figure 3 2 shows an example of the structure of one pixel circuit 321 among a plurality of pixel circuits 321 arranged in the light receiving pixel section 320. The pixel circuit 321 is an example of a structure including three pixel signal reading sections.

[0071] The pixel circuit 321 includes one photoelectric conversion element PD, a drain gate transistor GD, and three pixel signal readout units RU that output voltage signals from corresponding output terminals O. Each pixel signal readout unit RU includes a readout gate transistor G, a floating diffusion FD, a charge storage capacitor C, a reset gate transistor RT, a source follower gate transistor SF, and a select gate transistor SL. In each pixel signal readout unit RU, a charge storage unit CS is formed by the floating diffusion FD and the charge storage capacitor C. The drain gate transistor GD, the readout gate transistor G, the reset gate transistor RT, the source follower gate transistor SF, and the select gate transistor SL are N-channel MOS transistors.

[0072] In addition, Figure 3In the figure, each pixel signal readout unit RU is distinguished by marking the number "1", "2" or "3" after the reference mark "RU" of the three pixel signal readout units RU. In addition, similarly, each structural element of the three pixel signal readout units RU is distinguished by indicating the number representing each pixel signal readout unit RU after the reference mark, so as to distinguish the pixel signal readout unit RU corresponding to each structural element. Figure 3 In the pixel circuit 321 shown, the pixel signal readout section RU1 that outputs a voltage signal from the output terminal O1 includes a readout gate transistor G1, a floating diffusion FD1, a charge storage capacitor C1, a reset gate transistor RT1, a source follower gate transistor SF1, and a select gate transistor SL1. In the pixel signal readout section RU1, the floating diffusion FD1 and the charge storage capacitor C1 constitute a charge storage section CS1. The pixel signal readout section RU2 and the pixel signal readout section RU3 also have the same structure.

[0073] The photoelectric conversion element PD is a buried photodiode that generates charges by photoelectrically converting incident light and accumulates the generated charges. In addition, in the present invention, the structure of the photoelectric conversion element PD provided in the pixel circuit 321 is not particularly specified. Therefore, the photoelectric conversion element PD may be, for example, a PN photodiode having a structure in which a P-type semiconductor and an N-type semiconductor are joined together, or a PIN photodiode having a structure in which an I-type semiconductor is sandwiched between a P-type semiconductor and an N-type semiconductor. In addition, the photoelectric conversion element provided in the pixel circuit 321 is not limited to a photodiode, and may be, for example, a photoelectric conversion element of a photogate type.

[0074] The drain-gate transistor GD is a transistor for discarding the charge that is generated and accumulated by the photoelectric conversion element PD according to the drive signal input from the pixel drive circuit 326 and is not transmitted to each pixel signal readout unit RU. In other words, the drain-gate transistor GD is a transistor for resetting the charge that is generated by the photoelectric conversion element PD and is not used in the measurement of the distance to the subject S.

[0075] The read gate transistor G is a transistor for transferring the charge generated and accumulated by the photoelectric conversion element PD according to the drive signal input from the pixel drive circuit 326 to the corresponding charge accumulation section CS. The charge transferred by the read gate transistor G is held (accumulated) in the corresponding charge accumulation section CS.

[0076] Here, in the pixel signal readout unit RU1, the readout gate transistor G1 has a drain connected to the second terminal of the photoelectric conversion element PD, a gate connected to the signal line LTX1 for transmitting and storing the drive signal TX1, and a source connected to the floating diffusion FD1 and the first terminal of the charge storage capacitor C1.

[0077] Similarly, in the pixel signal readout unit RU2, for the readout gate transistor G2, the drain is connected to the second terminal of the photoelectric conversion element PD, the gate is connected to the signal line LTX2 that transmits and stores the drive signal TX2, and the source is connected to the floating diffusion FD2 and the first terminal of the charge storage capacitor C2.

[0078] In addition, similarly, in the pixel signal readout unit RU3, for the readout gate transistor G3, the drain is connected to the third terminal of the photoelectric conversion element PD, the gate is connected to the signal line LTX3 that transmits the storage drive signal TX3, and the source is connected to the floating diffusion FD3 and the first terminal of the charge storage capacitor C3.

[0079] The above-mentioned storage driving signal TX1 , storage driving signal TX2 , and storage driving signal TX3 are supplied from the pixel driving circuit 326 via the signal line LTX1 , the signal line LTX2 , and the signal line LTX3 , respectively.

[0080] The charge storage capacitor C is a capacitor that holds (stores) the charge transferred by the corresponding read gate transistor G.

[0081] The reset gate transistor RT is a transistor for discarding the charge held in the corresponding charge storage unit CS in response to the drive signal input from the vertical scanning circuit 323. That is, the reset gate transistor RT is a transistor for resetting the charge held in the corresponding charge storage unit CS.

[0082] The source follower gate transistor SF is a transistor for amplifying a voltage signal corresponding to the amount of charge accumulated in the charge storage unit CS connected to the gate terminal and outputting the amplified voltage signal to the corresponding selection gate transistor SL.

[0083] The selection gate transistor SL is a transistor for outputting, from the corresponding output terminal O, a voltage signal amplified by the corresponding source follower gate transistor SF in accordance with a drive signal input from the vertical scanning circuit 323 .

[0084] According to the above configuration, in the pixel circuit 321 , charges generated by photoelectric conversion of incident light into the photoelectric conversion element PD are distributed to the three charge storage units CS, and voltage signals corresponding to the amount of the distributed charges are output to the pixel signal processing circuit 325 .

[0085] The structure of the pixels arranged in the distance image sensor 32 is not limited to the following. Figure 3The structure including three pixel signal readout sections RU as shown in the figure may be a pixel of any structure as long as it includes one photoelectric conversion element PD and a plurality of pixel signal readout sections RU for distributing the charge generated and accumulated by the photoelectric conversion element PD. In other words, the number of pixel signal readout sections RU (charge accumulation section CS) provided in the pixel arranged in the range image sensor 32 may be two or more.

[0086] In addition, Figure 3 In the pixel circuit 321 of the structure shown, an example is shown in which the charge storage unit CS is composed of a floating diffusion FD and a charge storage capacitor C. However, the charge storage unit CS only needs to be composed of the floating diffusion FD. That is to say, the pixel circuit 321 may also be a structure that does not include the respective charge storage capacitors C. In the case of this structure, there is an effect of improving the charge detection sensitivity. However, considering the situation of expanding the dynamic range in the distance measurement in the distance image camera device 1, a structure that can hold (accumulate) more charges is advantageous. Therefore, in the pixel circuit 321, the charge storage capacitor C is included in the pixel signal readout unit RU, and the charge storage unit CS is composed of the floating diffusion FD and the charge storage capacitor C, so that it is also configured to hold (accumulate) more charges compared to the case where the charge storage unit CS is composed of only the floating diffusion FD.

[0087] In addition, Figure 3 In the pixel circuit 321 having the structure shown, an example of a structure including a drain-gate transistor GD is shown, but when there is no need to discard the (residual) charge accumulated in the photoelectric conversion element PD, it is also possible that the pixel arranged at the distance image sensor 32 does not have a structure including the drain-gate transistor GD.

[0088] Next, a driving (control) method (timing) of the pixel circuit 321 in the distance image capturing device 1 will be described. Figure 4 1 is a timing chart showing the timing of driving the pixel circuit 321 in the light receiving pixel unit 320 of the imaging element (range image sensor 32) used in the range image imaging device 1 according to the embodiment of the present invention. Figure 4 , the timing of the driving signal of the pixel circuit 321 when outputting one frame of pixel signals to the distance image sensor 32 and the timing of the light pulse PO irradiated to the subject S by the light source unit 2 are shown.

[0089] First, the photoelectric conversion element PD generates and stores light according to the amount of light received (the amount of light received).

[0090] The driving (control) of the pixel circuit 321 during the charge accumulation period in which the accumulated charge is distributed to each pixel signal readout unit RU is described. During the charge accumulation period, the light source unit 2 irradiates the light pulse PO to the subject S. Then, the pixel circuit 321 is driven synchronously with the timing of the irradiation light pulse PO, so that the charge corresponding to the received background light and reflected light RL is distributed to each charge accumulation unit CS. The pixel driving circuit 326 simultaneously drives all the pixel circuits 321 configured in the light receiving pixel unit 320, that is, through global shutter driving, the charge is distributed and accumulated in each charge accumulation unit CS of all the pixel circuits 321. In addition, the time when the light source device 21 emits a pulsed laser beam, that is, the pulse width Tw of the light pulse PO is a predetermined very short time such as 10ns. The reason is that in the distance measurement of the pulse modulation method, the maximum distance that can be measured (hereinafter referred to as the "maximum measurement distance") is determined by the pulse width Tw of the light pulse PO. When the pulse width Tw of the above-mentioned light pulse PO is 10ns, the maximum measurement distance is 1.5m. In addition, if the pulse width Tw of the light pulse PO is simply increased, that is, the emission time of the laser beam in the light source device 21 is extended, the photoelectric conversion element PD can receive more reflected light RL, but the resolution of the distance to the measured object S is reduced. On the other hand, if the pulse width Tw of the light pulse PO is shorter, the amount of charge generated by the photoelectric conversion element PD through photoelectric conversion is also reduced. Therefore, in the distance image camera 1, multiple light pulses PO are irradiated and the charge is distributed in a manner that a sufficient amount of charge is accumulated in each charge accumulation unit CS during the charge accumulation period.

[0091] Here, the structure of each vertical scanning circuit 323 and the pixel driving circuit 326 driving (controlling) the pixel circuit 321 is described. In the following description, the control circuit 322 outputs clock signals CK1, CK2, CK3, and CKRSTD for generating each accumulation driving signal TX1, TX2, and TX3 and the reset driving signal RSTD to the pixel driving circuit 326. In addition, the control circuit 322 outputs clock signals for generating each selection driving signal SEL1, SEL2, and SEL3 and the reset signal RST1, RST2, and RST3 to the vertical scanning circuit 323.

[0092] exist Figure 4 The charge accumulation period of the timing chart shown shows the driving timing of the pixel circuit 321 in the case where multiple light pulses PO are irradiated and the charges are distributed in all the pixel circuits 321. In addition, the following description is made: Figure 4In the charge accumulation period of the timing diagram shown, the light pulse PO is irradiated at the "H (High)" level (the light source device 21 emits a laser beam), and the irradiation of the light pulse PO is stopped at the "L (Low)" level (the light source device 21 is turned off). Figure 4 The timing chart shown explains a timing chart starting from resetting all pixel circuits 321, that is, in a state where the photoelectric conversion element PD and the charge storage unit CS do not store charges.

[0093] In the following description, the time from tA1 to tA5 is the accumulation period for distributing the charge, and a plurality of accumulation periods are repeated during the charge accumulation period. In addition, for example, the time width between the time tA1, tA2, tA3, and tA4, that is, the pulse width of the light pulse PO and the accumulation drive signals TX1, TX2, and TX3 is the same Tw.

[0094] During the charge accumulation period, first, the pixel driving circuit 326 transmits and accumulates the charge corresponding to the background light before the irradiation light pulse PO by the photoelectric conversion element PD from the moment tA1 before the pulse width Tw of the light pulse PO irradiated by the light source unit 2 via the readout gate transistor G1 to the charge accumulation unit CS1.

[0095] After that, the pixel driving circuit 326 transfers and accumulates the charge generated by the photoelectric conversion element PD according to the current photoelectric converted light to the charge storage unit CS2 via the read gate transistor G2 from the time tA2 which is the same as the timing of the light pulse PO irradiated by the light source unit 2. Here, the charge accumulated in the charge storage unit CS2 is the charge corresponding to the reflected light RL reflected by the subject S during the pulse width Tw of the irradiated light pulse PO. In addition to the charge corresponding to the background light, the charge also includes the charge corresponding to the reflected light RL incident with a shorter delay time proportional to the distance to the subject S (absolute distance). More specifically, for example, when the subject S is at a relatively close position, the irradiated light pulse PO is reflected by the subject S in a relatively short time and returns as the reflected light RL, so the charge storage unit CS2 includes more charges corresponding to the reflected light RL reflected by the subject S at a relatively close position.

[0096] After that, the pixel driving circuit 326 transfers and accumulates the charge generated by the photoelectric conversion element PD according to the current photoelectric conversion light via the read gate transistor G3 to the charge storage unit CS3 from the same time tA3 as the timing when the light source unit 2 stops irradiating the light pulse PO. Here, the charge accumulated in the charge storage unit CS3 is the charge corresponding to the reflected light RL reflected by the subject S outside the time of the pulse width Tw of the irradiated light pulse PO. In addition to the charge corresponding to the background light, the charge also includes the charge corresponding to the reflected light RL incident with a longer delay time proportional to the distance to the subject S (absolute distance). More specifically, for example, when the subject S is at a relatively far position, the irradiated light pulse PO takes a longer time to be reflected by the subject S and returned as the reflected light RL, so the charge storage unit CS3 contains more charges corresponding to the reflected light RL reflected by the subject S at a relatively far position.

[0097] After that, the pixel driving circuit 326 discards the charge generated by the photoelectric conversion element PD based on the light currently photoelectrically converted, that is, the charge not used in the measurement of the distance to the subject S, via the drain-gate transistor GD from the time tA4 when the same time as the pulse width Tw of the light pulse PO irradiated by the light source unit 2 has passed. In other words, the photoelectric conversion element PD is reset.

[0098] Thereafter, the pixel driving circuit 326 releases the resetting of the photoelectric conversion element PD at time tA5 before the same time as the pulse width Tw of the next light pulse PO irradiated by the light source unit 2. Then, the pixel driving circuit 326 transmits and accumulates the charge generated by the next photoelectric conversion of the photoelectric conversion element PD, that is, the charge corresponding to the background light before the next light pulse PO is irradiated, to the charge accumulation unit CS1 via the readout gate transistor G1 in the same manner as the timing from time tA1.

[0099] Afterwards, the pixel driving circuit 326 repeats the driving of the pixel circuit 321 in the same manner as that at time tA1 to time tA5 (hereinafter referred to as "charge sharing driving"). Thus, during the charge accumulation period, the charge amount of the repeated charge sharing driving is accumulated and maintained in each charge accumulation unit CS of all pixel circuits 321. In addition, the maximum number of times the charge sharing driving is repeated during the charge accumulation period is determined by the period of outputting (obtaining) a pixel signal of one frame by the distance image sensor 32. More specifically, the distance image sensor 32 obtains the time after the pixel signal readout period from the time of obtaining a pixel signal of one frame and divides it by the time when the light source device 21 emits a pulsed laser beam, that is, the pulse cycle time To of the light pulse PO. In addition, in the distance image sensor 32, the more times the charge sharing driving is performed, the more the charge amount accumulated (accumulated) in each charge accumulation unit CS is, and the higher the sensitivity is. Thus, in the distance image sensor 32, the resolution of the distance to the measured subject S can be improved.

[0100] Next, the following describes the driving (control) of the pixel circuits 321 during the pixel signal readout period, in which, after the charge accumulation period ends, the voltage signals corresponding to the charge amounts allocated to the charge accumulation units CS provided in the pixel signal readout units RU are sequentially output to each row of the pixel circuits 321 arranged in the light-receiving pixel unit 320. During the pixel signal readout period, the pixel circuits 321 arranged in the light-receiving pixel unit 320 are driven row by row, that is, the voltage signals corresponding to the charge amounts accumulated (accumulated) and held in the charge accumulation units CS provided in the pixel circuits 321 arranged in the corresponding rows are sequentially output to the pixel signal processing circuit 325 row by row.

[0101] In addition, as described above, in the distance image sensor 32, the pixel signal processing circuit 325 performs predetermined signal processing such as noise suppression processing on the voltage signal output by each pixel circuit 321. Here, the correlated double sampling (CDS) processing performed by the pixel signal processing circuit 325 as the noise suppression processing is a processing that takes the difference between the voltage signal corresponding to the amount of charge accumulated (accumulated) and held in the charge storage unit CS (hereinafter referred to as "distance pixel voltage signal PS") and the voltage signal corresponding to the amount of charge in the reset state (reset state) of the charge storage unit CS (hereinafter referred to as "reset voltage signal PR"). Therefore, during the pixel signal readout period, each voltage signal between the distance pixel voltage signal PS and the reset voltage signal PR corresponding to each charge storage unit CS of each pixel circuit 321 is output to the pixel signal processing circuit 325 row by row.

[0102] Shown in Figure 4During the pixel signal readout period of the timing diagram shown, when a plurality of pixel circuits 321 are configured with the vertical direction (row arrangement direction) of the light-receiving pixel portion 320 being set to y rows (y is an integer greater than 1) and the horizontal direction (column arrangement direction) being set to x columns (x is an integer greater than 1), the driving timing of the pixel circuit 321 is when each pixel circuit 321(i) configured in the i-th row (1≤i≤y) of the light-receiving pixel portion 320 outputs each voltage signal between the distance pixel voltage signal PS(i) and the reset voltage signal PR(i). In addition, Figure 4 In the timing chart shown, the charge storage unit CS1 (i), the charge storage unit CS2 (i), and the charge storage unit CS3 (i) included in each pixel circuit 321 (i) output respective voltage signals in this order.

[0103] During the pixel signal readout period, first, during the period from time tR1 to time tR2, the vertical scanning circuit 323 outputs the distance pixel voltage signal PS1(i) from the output terminal O1(i) via the vertical signal line to the pixel signal processing circuit 325. Thus, the pixel signal processing circuit 325 temporarily holds the distance pixel voltage signal PS1(i) output from the pixel signal readout unit RU1(i) via the vertical signal line.

[0104] Then, during the period from time tR2 to time tR3 , the vertical scanning circuit 323 supplies the reset signal RST1 ( i ) to discharge the charge of the charge storage unit CS1 ( i ) included in the pixel circuit 321 ( i ) and perform the reset.

[0105] Thereafter, during the period from time tR3 to time tR4, the vertical scanning circuit 323 outputs the reset voltage signal PR1(i) from the output terminal O1(i) via the vertical signal line to the pixel signal processing circuit 325. Thus, the pixel signal processing circuit 325 takes the difference between the temporarily held distance pixel voltage signal PS1(i) and the reset voltage signal PR1(i) output from the pixel signal readout section RU1(i) via the vertical signal line, that is, suppresses the noise included in the voltage signal corresponding to the amount of charge accumulated (stored) and held in the charge storage section CS1(i).

[0106] Thereafter, during the period from time tR4 to time tR7, the vertical scanning circuit 323 outputs the distance pixel voltage signal PS2(i) and the reset voltage signal PR2(i) from the output terminal O2(i) via the vertical signal line to the pixel signal processing circuit 325, similarly to the period from time tR1 to time tR4. Furthermore, during the period from time tR7 to time tR10, the vertical scanning circuit 323 also outputs the distance pixel voltage signal PS3(i) and the reset voltage signal PR3(i) from the output terminal O3(i) via the vertical signal line to the pixel signal processing circuit 325, similarly to the period from time tR1 to time tR4.

[0107] Thereafter, the vertical scanning circuit 323 sequentially drives each pixel circuit 321 arranged in other rows of the light-receiving pixel portion 320 (for example, each pixel circuit 321 arranged in the i+1th row) in the same manner as from time tR1 to time tR10 (hereinafter referred to as “pixel signal readout drive”), and sequentially outputs each voltage signal from all the pixel circuits 321 arranged in the light-receiving pixel portion 320.

[0108] According to such a driving (control) method (timing), the pixel driving circuit 326 performs distribution multiple times to each pixel signal readout unit RU that generates and accumulates charges in the photoelectric conversion element PD of each pixel circuit 321 arranged in the light receiving pixel unit 320.

[0109] Furthermore, the vertical scanning circuit 323 sequentially outputs a voltage signal corresponding to the amount of charge accumulated (stored) in the charge storage section CS included in the pixel signal readout section RU to the pixel signal processing circuit 325 via the vertical signal lines.

[0110] In addition, the AD conversion circuit 329 performs A / D conversion processing on each voltage signal that suppresses noise on a row basis. Then, the horizontal scanning circuit 324 sequentially outputs the voltage signals of each row after the A / D conversion processing by the AD conversion circuit 329 via the horizontal signal line in the order of the columns of the light-receiving pixel unit 320, so that the distance image sensor 32 outputs the pixel signals of all the pixel circuits 321 of one frame to the outside. Thus, in the distance image camera 1, the pixel signals of one frame are output to the distance calculation unit 42 in the so-called raster order.

[0111] In addition, from Figure 4As can be seen from the driving (control) timing of the pixel circuit 321 shown, each pixel signal for one frame includes three voltage signals corresponding to the three pixel signal readout sections RU (charge storage sections CS) provided in the corresponding pixel circuit 321. The distance calculation section 42 calculates the distance to the subject S for each pixel signal, that is, for each pixel circuit 321, based on the pixel signal for one frame output from the distance image sensor 32.

[0112] Here, a method for calculating the distance between the distance image camera 1 and the subject S in the distance calculation unit 42 is described. Here, the charge amount of the electric charge corresponding to the background light before the light pulse PO is irradiated by the charge storage unit CS1 assigned to the pixel signal readout unit RU1 is set as the charge amount Q1. In addition, the charge amount of the electric charge corresponding to the background light assigned to the charge storage unit CS2 of the pixel signal readout unit RU2 and the reflected light RL incident with a shorter delay time is set as the charge amount Q2. In addition, the charge amount of the electric charge corresponding to the background light assigned to the charge storage unit CS3 of the pixel signal readout unit RU3 and the reflected light RL incident with a longer delay time is set as the charge amount Q3. The distance calculation unit 42 calculates the distance D between each pixel circuit 321 and the subject S using the following formula (1).

[0113] D=(Q3-Q1) / (Q2+Q3-2Q1)×Dm···(1)

[0114] In the above formula (1), Dm is the maximum distance (maximum measurement distance) that can be measured by irradiation with the light pulse PO. Here, the maximum measurement distance Dm is represented by the following formula (2).

[0115] Dm=(c / 2)Tw···(2)

[0116] In the above formula (2), c is the speed of light, and Tw is the pulse width of the light pulse PO.

[0117] As described above, the distance image capturing device 1 obtains the distance D between itself and the subject S for each pixel circuit 321 arranged in the light receiving pixel section 320 of the distance image sensor 32 .

[0118] In addition, as described above, the structure of the pixel circuits arranged in a grid pattern in the distance image sensor 32 is not limited to the structure of Figure 3The structure including three pixel signal readout units RU1, RU2 and RU3 as shown in the figure can be any pixel circuit 321 having a structure including one photoelectric conversion element PD and two or more pixel signal readout units RU for distributing the charge generated and accumulated by the photoelectric conversion element PD. In this case, that is, in a distance image sensor having a pixel configuration including a structure having different numbers of pixel signal readout units RU, Figure 4 The driving (control) method (timing) of the pixel circuit 321 in the distance image camera device 1 shown in FIG. 1 can be considered in the same way, and the driving (control) method (timing) of the pixel can also be easily realized. More specifically, in a period in which the phases of the driving signals input to the read gate transistor G and the drain gate transistor GD of each pixel signal readout unit RU are maintained in a phase relationship in a manner that they do not overlap with each other, by repeating the charge distribution drive of the pixel, the charge corresponding to the corresponding light can be accumulated (accumulated) in the charge accumulation unit CS of each pixel signal readout unit RU, similarly to the distance image sensor 32. Then, each voltage signal is sequentially output from all pixels by the pixel signal readout drive, so that the pixel signal of one frame can be output to the outside of the distance image sensor, similarly to the distance image sensor 32. Thus, the distance calculation unit 42 can calculate the distance D between the distance image camera device 1 and the subject S according to each pixel signal (each pixel), based on the pixel signal of one frame output from the distance image sensor having a pixel configuration with a different number of pixel signal readout units RU.

[0119] Generally, in order to accurately measure the distance between the distance image sensor and the object, all pixel circuits 321 in the light-receiving pixel unit 320 are driven at the same timing during the accumulation period according to the global shutter method. That is, each accumulation drive signal TX1, TX2, TX3 and reset drive signal RSTD are supplied to all columns of the pixel circuits 321 at the same timing in the grid-shaped arrangement of the pixel circuits 321.

[0120] In the above Figure 2 In each column of the pixel circuits 321, the four timing adjustment circuits 326C and the drive circuit 326D supply the storage drive signals TX1, TX2, TX3 and the reset drive signal RSTD to each pixel circuit 321 in the column.

[0121] Then, the above-mentioned storage drive signals TX1, TX2 and TX3 are used to control the Figure 3 The read gate transistors G1, G2, and G3 shown in the figure accumulate electric charges in the accumulation sections CS1, CS2, and CS3, respectively, in each electric storage period within a frame period.

[0122] The vertical scanning circuit 323 outputs a voltage corresponding to the amount of charge accumulated in each of the accumulation units CS1 , CS2 , and CS3 to the pixel signal processing circuit 325 from the source follower gate transistors SF1 , SF2 , and SF3 .

[0123] The vertical scanning circuit 323 controls the selection gate transistors SL1, SL2, and SL3 by outputting the selection drive signals SEL1, SEL2, and SEL3, respectively. As a result, the selection gate transistors SL1, SL2, and SL3 output voltages corresponding to the amount of charge accumulated in the storage units CS1, CS2, and CS3 from the output terminals O1, O2, and O3 as distance pixel voltage signals PS1, PS2, and PS3 (in order to make it clear that they are analog voltages, the input voltages VA (CS1), VA (CS2), and VA (CS3) are shown below) to the pixel signal processing circuit 325.

[0124] Figure 5 This is a conceptual diagram showing a configuration example of an AD conversion circuit that performs AD conversion on an input voltage supplied from a pixel signal processing circuit according to an embodiment of the present invention.

[0125] The AD conversion circuit 329 has a column AD conversion unit 329j for each column j of the pixel circuits 321 arranged in a grid. Figure 2 ) includes three vertical signal lines. For example, the vertical signal lines 330j corresponding to the columns j of the pixel circuits 321 arranged in a grid form include vertical signal lines 330j (CS1), 330j (CS2), and 330j (CS3).

[0126] The column AD conversion unit 329j includes column AD conversion circuits 329j (CS1), 329j (CS2), 329j (CS3) which are arranged corresponding to the output terminals O1, O2 and O3 in column j and connected via the vertical signal lines 330j (CS1), 330j (CS2), 330j (CS3).

[0127] In each column AD conversion circuit 329j (CS1), 329j (CS2), and 329j (CS3), analog voltages corresponding to the amount of charge accumulated in the charge accumulation section CS1 after the signal processing supplied from the pixel signal processing circuit 325 are supplied as input voltages VA (CS1), VA (CS2), and VA (CS3) via vertical signal lines 330j (CS1), 330j (CS2), and 330j (CS3). Each vertical signal line 330j (CS1), 330j (CS2), and 330j (CS3) is connected to Figure 3 The output terminals O1, O2, and O3 in the pixel circuit 321 are connected.

[0128] Moreover, each column AD conversion circuit 329j (CS1), 329j (CS2) and 329j (CS3) performs AD conversion on the input voltages VA (CS1), VA (CS2), VA (CS3), respectively, thereby outputting digital values ​​OD (CS1), OD (CS2), OD (CS3).

[0129] In addition, in the present embodiment, when measuring the distance between the subject S and the distance image sensor 32 , the distance is measured using either the normal mode or the detailed measurement mode as the measurement mode.

[0130] In the normal mode, it is determined whether the distance D to the subject S to be measured is in the short distance range or in the long distance range.

[0131] Furthermore, by dividing the maximum measurement distance of the distance image sensor 32 into two, two sub-measurement ranges are generated, the sub-measurement range with a smaller distance to the subject S is set as the short-distance range, and the sub-measurement range with a larger distance to the subject S is set as the long-distance range (described later). Fig. 9 (a)).

[0132] The detailed measurement mode includes a short distance mode and a long distance mode. The short distance mode is a detailed measurement mode for measuring the distance D to the subject S in the short distance range. The long distance mode is a detailed measurement mode for measuring the distance D to the subject S in the long distance range.

[0133] Figure 6 : is a block diagram showing a configuration example of the distance calculation unit 42 in the distance image processing unit 4 according to the present embodiment.

[0134] The distance calculation section 42 includes a charge amount determination section 420 , a distance calculation section 421 , a distance determination section 422 , a mode setting section 423 , a measurement enable section 424 , and a mode table 425 .

[0135] When the measurement mode is the short-distance mode, the charge amount determination unit 420 compares the output digital value OD (CS2) with the output digital value OD (CS1) in each pixel circuit 321 to determine whether the output digital value OD (CS2) exceeds the output digital value OD (CS1). Here, when the output digital value OD (CS2) exceeds the output digital value OD (CS1), the charge amount Q2 includes the charge amount generated by the reflected light RL in addition to the charge amount Q1, which means that the distance to the subject S can be measured by the charge amounts Q2 and Q3. On the other hand, when the output digital value OD (CS2) is the same as or less than the output digital value OD (CS1), the charge amount Q2 does not include the charge amount generated by the reflected light RL in addition to the charge amount Q1, which means that the distance to the subject S cannot be measured by the charge amounts Q2 and Q3. When the output digital value OD( CS2 ) is equal to or smaller than the output digital value OD( CS1 ), the subject S is not located in the short distance range which is the short distance sub-measurement range, but is located in the long distance range which is the long distance sub-measurement range.

[0136] On the other hand, in the case where the measurement mode is the long-distance mode, the charge amount determination unit 420 compares the output digital value OD (CS3) with the output digital value OD (CS1) in each pixel circuit 321 to determine whether the output digital value OD (CS1) exceeds the output digital value OD (CS3). Here, when the output digital value OD (CS1) is less than the output digital value OD (CS3), the charge amount Q3 includes the charge amount generated by the reflected light RL in addition to the charge amount Q1, which means that the distance to the subject S can be measured by the charge amounts Q2 and Q3. On the other hand, when the output digital value OD (CS1) does not exceed the output digital value OD (CS3), the charge amount Q1 includes the charge amount generated by the reflected light RL in addition to the charge amount generated by the background light, which means that the distance to the subject S cannot be measured by the charge amounts Q2 and Q3. When the output digital value OD( CS1 ) exceeds the output digital value OD( CS3 ), the subject S is not located in the long-distance sub-measurement range but is located in the short-distance sub-measurement range, that is, the short-distance range.

[0137] Then, after the above determination is made, the charge amount determination unit 420 controls the distance calculation unit 421 to calculate the distance when the result of the charge amount determination indicates that the distance can be measured.

[0138] On the other hand, when the charge amount determination result indicates that the distance cannot be measured, the charge amount determination unit 420 does not calculate the distance to the distance calculation unit 421, and outputs control information that uses a preset value as the calculation result. At this time, for example, the distance calculation unit 421 outputs the maximum distance value as the calculation result in the short distance mode, and outputs the minimum value as the calculation result in the long distance mode.

[0139] Furthermore, when the measurement mode is the normal mode, the charge amount determination unit 420 does not perform charge amount determination processing, and outputs control information that a predetermined calculation processing has been performed to the distance calculation unit 421 .

[0140] In the normal mode, the distance calculation unit 421 calculates the distance D between each pixel circuit 321 and the subject S, which has been described in the description of the distance calculation process of the distance calculation unit 42 , using the above equation (1).

[0141] In addition, when the charge amount determination unit 420 performs control without performing calculation, the distance calculation unit 421 does not perform calculation using the corresponding output digital values ​​OD(CS1), OD(CS2), OD(CS3), but outputs a preset numerical value as the calculation result.

[0142] The distance determination unit 422 determines, for example, whether the distance D of the pixels in the target area calculated by the distance calculation unit 421 is included in the short distance range or the long distance range. The target area shows in detail the area to be captured in the captured range image (distance measurement target).

[0143] In this embodiment, the distance range, which is the range from the distance image camera 1 to the maximum measurement distance (Dm) that the distance image camera 1 can measure, is divided into a plurality of sub-measurement ranges of the same width. For example, when the distance range is from 0m to 4m, the distance range 2 is divided into two sub-measurement ranges, namely, a short distance range of more than 0m and less than 2m and a long distance range of more than 2m and less than 4m. In this embodiment, the distance range is divided into two, but it can also be configured to be divided into three or more.

[0144] The pulse width Tsw of each optical pulse PO and the accumulation drive signals TX1 , TX2 , and TX3 is set in accordance with the sub-measurement range.

[0145] That is, using the above formula (2), when the number of divisions is n, the width Dsm of each sub-measurement range is Dm / n, and the pulse width Tsw is (Dm / n)×(2 / c).

[0146] The distance determination unit 422 determines in which sub-measurement range the distance D of the pixel in the target area is included, for example, the short distance range or the long distance range, and outputs measurement range information of the determination result to the mode setting unit 423 .

[0147] Furthermore, when the distance D is unknown, the distance determination unit 422 outputs measurement range information indicating that the distance is within the distance range corresponding to the maximum measurement distance Dm to the mode setting unit 423 .

[0148] Therefore, the distance calculation unit 421 calculates the distance D using the following formula (3) in the case of the short distance mode, and calculates the distance D using the following formula (4) in the case of the long distance mode. In the following formulas (3) and (4), Dsm represents the maximum measurement distance in the width of the sub-measurement range. The maximum measurement distance Dsm is calculated by formula (5). Tsw is the pulse width of the light pulse PO in the detailed measurement mode.

[0149] D=(Q3-Q1) / (Q2+Q3-2Q1)×Dsm, (3)

[0150] D=Dsm+(Q3-Q1) / (Q2+Q3-2Q1)×Dsm, (4)

[0151] Dsm=(c / 2)Tsw、 (5)

[0152] The mode setting unit 423 determines whether the measurement mode is to be set to the normal mode or the detailed measurement mode, based on the measurement range information supplied from the distance determination unit 422 .

[0153] Here, in this embodiment, the above-mentioned normal mode and detailed measurement mode are provided as measurement modes.

[0154] When the measurement range information indicating that the measurement is in the measurement range corresponding to the maximum measurement distance Dm is supplied from the distance determination unit 422 , the mode setting unit 423 reads the mode setting value of the normal mode from the mode table 425 .

[0155] On the other hand, when the distance determination unit 422 supplies measurement range information indicating that the mode setting is performed in the sub-measurement range corresponding to the distance measured in the normal mode, the mode setting unit 423 reads the mode setting value corresponding to the sub-measurement range from the mode table 425 .

[0156] The mode setting values ​​are each combination of the above-mentioned pulse width Tw and the phase (output timing) of the optical pulse PO. The mode setting values ​​are set in the mode table 425 in correspondence with the normal mode in the distance range and the detailed measurement mode in each sub-measurement range.

[0157] The measurement unit 424 determines whether the distance of the pixels in the target area is included in the short distance range in the case of the short distance mode or in the long distance range in the case of the long distance mode according to the current detailed measurement mode.

[0158] Figure 7 1 is a timing chart for explaining the phase of the light pulse PO in each of the normal mode, the short distance mode, and the long distance mode during the charge accumulation period of the present embodiment.

[0159] Figure 7 (a) shows the pulse width Tw of the mode setting value and the phase of the light pulse PO in the normal mode. Figure 7 (b) shows the pulse width Tsw of the mode setting value and the phase of the light pulse PO in the close distance mode. Figure 7 (c) shows the pulse width Tsw of the mode setting value and the phase of the light pulse PO in the long distance mode.

[0160] Figure 7 (a) is a timing diagram showing the timing of driving the pixel circuit 321 configured in the light receiving pixel section 320 of the distance image sensor 32 in the normal mode. The pulse width Tw of each drive signal in the normal mode is set to Dm×(2 / c). In addition, the phase of the light pulse PO is the same as that of the storage drive signal TX2. Figure 7 (a) shows the timing of the driving signal of the pixel circuit 321 when the distance image sensor 32 outputs the pixel signal for one frame in the normal mode and the timing of the light pulse PO irradiated to the subject S by the light source unit 2 .

[0161] Before time tA1, the reset drive signal RSTD is supplied to the pixel circuit 321. At time tA1, the pixel drive circuit 326 distributes the charge generated by the photoelectric conversion element PD using the background light to the charge storage unit CS1 by accumulating the drive signal TX1. At time tA2, the pixel drive circuit 326 distributes the charge generated by the photoelectric conversion element PD using the reflected light RL reflected from the subject S to the charge storage unit CS2 by accumulating the drive signal TX2. At time tA3, the pixel drive circuit 326 distributes the charge generated by the photoelectric conversion element PD using the reflected light RL reflected from the subject S to the charge storage unit CS3 by accumulating the drive signal TX3. At time tA4, the charge generated by the photoelectric conversion element PD is eliminated (reset) by supplying the reset drive signal RSTD.

[0162] Figure 7(b) is a timing diagram showing the timing of driving the pixel circuit 321 configured in the light-receiving pixel section 320 of the distance image sensor 32 in the close-range mode. The pulse width Tsw of each drive signal in the close-range mode is set to (Dsm)×(2 / c), Dm / c. Here, Dsm is Dm / n, n is the number of divisions, for example, when Dm is 4m and the number of divisions n is 2, Dsm is 2m for the close-range range and the long-range range, respectively. In addition, since the distance range 2 is divided into sub-measurement ranges of the close-range range and the long-range range, the pulse width Tsw is 1 / 2 of the pulse width Tw. The phase of the light pulse PO is the same as that of the accumulation drive signal TX2. Figure 7 (b) shows the timing of the drive signal of the pixel circuit 321 when outputting one frame of pixel signals in the close-range mode in the distance image sensor 32 and the timing of the light pulse PO irradiated by the light source unit 2 to the subject S. Here, the phase of the light pulse PO is made equal to the accumulation drive signal TX2 because the sub-measurement range (close-range range) in the close-range mode is 0 (m) < L < 2 (m).

[0163] Before time tB1, the reset drive signal RSTD is supplied to the pixel circuit 321. At time tB1, the pixel drive circuit 326 distributes the charge generated by the photoelectric conversion element PD using the background light to the charge storage unit CS1 by accumulating the drive signal TX1. At time tB2, the pixel drive circuit 326 distributes the charge generated by the photoelectric conversion element PD using the reflected light RL reflected from the subject S to the charge storage unit CS2 by accumulating the drive signal TX2. At time tB3, the pixel drive circuit 326 distributes the charge generated by the photoelectric conversion element PD using the reflected light RL reflected from the subject S to the charge storage unit CS3 by accumulating the drive signal TX3. In addition, at time tB4, the charge generated by the photoelectric conversion element PD is eliminated by supplying the reset drive signal RSTD. Through the above-mentioned processing, the reflected light RL from the subject S located at a distance less than 2m relative to the pixel circuit 321 is distributed and accumulated in the charge accumulation units CS2 and CS3 respectively through the respective accumulation drive signals TX2 and TX3, and the distance image processing unit 4 uses formula (3) to calculate the distance D between the pixel circuit 321 and the subject S.

[0164] Figure 7 (c) is a timing diagram showing the timing of driving the pixel circuit 321 configured in the light receiving pixel unit 320 of the distance image sensor 32 in the long-distance mode. The pulse width Tsw of each drive signal in the long-distance mode is set to Dsm / c, the same as in the short-distance mode. In addition, the phase of the light pulse PO is the same as that of the storage drive signal TX1.

[0165] Here, the phase of the light pulse PO is not equal to the accumulation drive signal TX2, but equal to the accumulation drive signal TX1, because the sub-measurement range L (long-distance range) in the long-distance mode is 2 (m) ≤ L < 4 (m). Therefore, at the timing of 4m (= Dm) / c before the time of the light round-trip distance of 2m, that is, at the timing of the accumulation drive signal TX2 to distribute the charge, the distance image processing unit 4 irradiates the light pulse PO to the light source unit 2 at the timing of the accumulation drive signal TX1, so that the reflected light RL from the subject S located at a distance of more than 2m is incident.

[0166] Moreover, with Figure 7 Similarly, in the case of the close-distance mode of (b), the reset drive signal RSTD is supplied to the pixel circuit 321 before the time tB1. At the time tB1, the pixel drive circuit 326 distributes the charge generated by the photoelectric conversion element PD using the background light to the charge storage unit CS1 by accumulating the drive signal TX1. At the time tB2, the pixel drive circuit 326 distributes the charge generated by the photoelectric conversion element PD using the reflected light RL reflected from the subject S to the charge storage unit CS2 by accumulating the drive signal TX2. At the time tB3, the pixel drive circuit 326 distributes the charge generated by the photoelectric conversion element PD using the reflected light RL reflected from the subject S to the charge storage unit CS3 by accumulating the drive signal TX3. Through the above-mentioned processing, the reflected light RL from the subject S located at a distance of more than 2m relative to the pixel circuit 321 is distributed and accumulated in the charge accumulation units CS2 and CS3 respectively through the respective accumulation drive signals TX2 and TX3, and the distance image processing unit 4 uses formula (4) to calculate the distance D between the pixel circuit 321 and the subject S.

[0167] Below, refer to Figure 8 , the distance measurement process of the distance image processing unit 4 according to the present embodiment will be described. Figure 8 1 is a flowchart showing an example of the operation of the distance measurement process of the distance image capturing device 1 of this embodiment. The following distance measurement is performed on the pixel signals (for each frame of the captured image) of all the pixel circuits 321 in the light receiving pixel unit 320 of the distance image sensor 32.

[0168] Step SA1: The mode setting unit 423 initially sets the measurement mode for distance measurement, and reads the pulse width Tw and phase in the normal mode from the mode table 425. Then, the mode setting unit 423 outputs the read pulse width Tw and phase to the timing control unit 41. As a result, the timing control unit 41 supplies the respective accumulation drive signals TX1, TX2, TX3 and the reset drive signal RSTD to the distance image sensor 32, and outputs the drive signal (RSTD) for emitting the light pulse PO to the light source device 21. Figure 7 The waveform shown in (a)).

[0169] Pixel signals of output digital values ​​OD( CS1 ), OD( CS2 ), and OD( CS3 ) corresponding to the charge amounts Q1 , Q2 , and Q3 are supplied from the distance image sensor 32 to the distance calculation unit 421 .

[0170] The distance calculation unit 421 substitutes the output digital values ​​OD(CS1), OD(CS2), and OD(CS3) as charge amounts Q1, Q2, and Q3, respectively, into equation (1) to calculate the distance from the distance image sensor 32 to the subject S (distance measurement).

[0171] Step SA2 : The user uses, for example, an input unit (not shown) to input information of a region of interest (distance measurement target) that the user wants to capture in detail in the range image to the range image capturing apparatus 1 .

[0172] In addition, the information of the above-mentioned area of ​​interest in the distance image may be set in advance for the distance image camera 1, and the area of ​​interest set in the distance image may be surrounded by an image frame. In this case, the user adjusts the shooting direction of the distance image camera 1 so that the object whose distance is to be measured in detail enters the above-mentioned image frame.

[0173] Step SA3: The distance determination unit 422 compares the input pixel distance of the target area with the sub-measurement ranges of the short-distance mode and the long-distance mode. Here, the distance determination unit 422 determines whether the pixel distance of the target area is included in the detailed sub-measurement range of the short-distance mode.

[0174] At this time, when the distance of the pixel of the target area is included in the short distance range which is the sub-measurement range of the short distance mode, the distance determination unit 422 advances the process to step SA4 .

[0175] On the other hand, when the distance of the pixel of the target area is not included in the short distance range of the short distance mode (when included in the long distance range which is the sub-measurement range of the long distance mode), the distance determination unit 422 advances the process to step SA8.

[0176] Step SA4: The mode setting unit 423 reads the pulse width Tsw and phase in the short-distance mode from the mode table 425 in order to change the measurement mode for distance measurement from the normal mode to the short-distance mode of the detailed measurement mode. The mode setting unit 423 outputs the read pulse width Tsw and phase to the timing control unit 41. As a result, the timing control unit 41 supplies the respective accumulation drive signals TX1, TX2, TX3 and the reset drive signal RSTD corresponding to the supplied pulse width Tsw and phase to the distance image sensor 32, and outputs the drive signal (RSTD) for emitting the light pulse PO to the light source device 21. Figure 7The waveform shown in (b)).

[0177] Step SA5: For the distance image sensor 32, in the short-distance mode, after completing the allocation times for one frame, the pixel signals of the output digital values ​​OD(CS1), OD(CS2), OD(CS3) corresponding to the respective charge amounts Q1, Q2, and Q3 are output to the distance image processing unit 4.

[0178] Thus, the distance calculation unit 421 inputs output digital values ​​OD( CS1 ), OD( CS2 ), and OD( CS3 ) corresponding to the charge amounts Q1 , Q2 , and Q3 obtained in the short-distance mode.

[0179] Step SA6: Since the current measurement mode is the short-distance mode, the charge amount determination unit 420 compares the output digital value OD (CS2) of each pixel circuit 321 with the output digital value OD (CS1) to determine whether the output digital value OD (CS2) exceeds the output digital value OD (CS1).

[0180] Furthermore, when the distance to the subject S can be measured by the charge amounts Q2 and Q3 , the charge amount determination unit 420 outputs control information for calculating the distance to the distance calculation unit 421 .

[0181] When the distance calculation unit 421 receives control information for calculating the distance, it substitutes the input charge amounts Q1, Q2, and Q3 into equation (3) and outputs the calculated distance D (distance measurement in the short-distance mode).

[0182] On the other hand, when the distance to the subject S cannot be measured using the charges Q2 and Q3 , the charge amount determination unit 420 outputs control information indicating a predetermined numerical value (out-of-range value) as a calculation result to the distance calculation unit 421 .

[0183] The distance calculation unit 421 does not calculate the distance D using the respective input charge amounts Q1 , Q2 , and Q3 , but outputs a preset numerical value as a calculation result.

[0184] The charge amount determination unit 420 compares the charge amounts Q1 and Q2 for all pixels of the captured image, and the distance calculation unit 421 calculates the distance D according to the comparison result.

[0185] Step SA7 : The measurement capability unit 424 determines whether the distance D of the pixels in the target area can be measured within the short-distance range in the short-distance mode, that is, whether the distance D is an out-of-range value.

[0186] At this time, if the distance D is not an out-of-range value, the subject S in the target area is included in the short-distance range and can be measured in the short-distance mode. Therefore, the measurement availability unit 424 advances the processing to step SA5.

[0187] On the other hand, when the distance D is an out-of-range value, the subject S in the target area is not within the short-distance range and cannot be measured in the short-distance mode. Therefore, the measurement availability unit 424 advances the process to step SA1.

[0188] Step SA8: The mode setting unit 423 reads the pulse width Tsw and phase in the long-distance mode from the mode table 425 in order to change the measurement mode for distance measurement from the normal mode to the long-distance mode of the detailed measurement mode. The mode setting unit 423 outputs the read pulse width Tsw and phase to the timing control unit 41. As a result, the timing control unit 41 supplies the respective accumulation drive signals TX1, TX2, TX3 and the reset drive signal RSTD corresponding to the supplied pulse width Tsw and phase to the distance image sensor 32, and outputs the drive signal (RSTD) for emitting the light pulse PO to the light source device 21. Figure 7 The waveform shown in (c)).

[0189] Step SA9: For the distance image sensor 32, in the long-distance mode, after completing the allocation times for one frame, the pixel signals of the output digital values ​​OD(CS1), OD(CS2), OD(CS3) corresponding to the respective charge amounts Q1, Q2, Q3 are output to the distance image processing unit 4.

[0190] Thus, the distance calculation unit 421 receives pixel signals of output digital values ​​OD( CS1 ), OD( CS2 ), and OD( CS3 ) corresponding to the charge amounts Q1 , Q2 , and Q3 obtained in the long-distance mode from the distance image sensor 32 .

[0191] Step SA10: Since the current measurement mode is the long-distance mode, the charge amount determination unit 420 compares the output digital value OD(CS3) with the output digital value OD(CS1) to determine whether the output digital value OD(CS3) exceeds the output digital value OD(CS1).

[0192] Furthermore, when the distance to the subject S can be measured by the charge amounts Q2 and Q3, the charge amount determination unit 420 outputs control information for calculating the distance to the distance calculation unit 421. When the distance calculation unit 421 receives control information for calculating the distance, it substitutes the input charge amounts Q1, Q2, and Q3 into equation (4) and outputs the calculated distance D (distance measurement in the long-distance mode).

[0193] On the other hand, when the distance to the subject S cannot be measured using the charges Q2 and Q3 , the charge amount determination unit 420 outputs control information indicating a predetermined numerical value (out-of-range value) as a calculation result to the distance calculation unit 421 .

[0194] The distance calculation unit 421 does not calculate the distance D using the respective input charge amounts Q1 , Q2 , and Q3 , but outputs a preset numerical value as a calculation result.

[0195] The charge amount determination unit 420 compares the charge amounts Q1 and Q2 for all pixels of the captured image, and the distance calculation unit 421 calculates the distance D according to the comparison result.

[0196] Step SA11: The measurement capability unit 424 determines whether the distance D of the pixel in the target area can be measured within the long-distance range in the long-distance mode, that is, whether the distance D is an out-of-range value. At this time, if the distance D is not an out-of-range value, the subject S in the target area is included in the distance within the long-distance range and can be measured by the long-distance mode, so the measurement capability unit 424 causes the process to proceed to step SA9.

[0197] On the other hand, when the distance D is an out-of-range value, the subject S in the target area is not included in the long-distance range and cannot be measured in the long-distance mode. Therefore, the measurement availability unit 424 advances the processing to step SA1.

[0198] Fig. 9 It is a diagram for explaining the effect of distance measurement in the detail measurement mode in the distance image capturing device 1 according to the present embodiment. Fig. 9 (a) shows the measurement ranges of the normal mode, the short-distance mode, and the long-distance mode of the present embodiment. Fig. 9 (b) shows the charge amount corresponding to the distance to the subject S in each of the normal mode and the long distance mode.

[0199] In this embodiment, as an example, the measurement range L in the normal mode, i.e., 0.1m to 4m (0.1[m]<L<4[m]), is divided into sub-measurement ranges of a short distance range of 0.1m to 2m (0.1[m]<L<2[m]) and a long distance range of 2m to 4m (2[m]≤L<4[m]).

[0200] Moreover, the measurable sub-measurement range is set narrower than the measurement range in the normal mode. In order to measure the distance in the sub-measurement range, the pulse width is shortened from Tw to Tsw. In order to correspond to the distance measurement in the sub-measurement range, the width of the light pulse PO is shortened.

[0201] With the above-described configuration, in the present embodiment, when switching from the normal mode to the detailed measurement mode, by shortening the pulse width Tw to the pulse width Tsw, the intensity of the light pulse can be increased without changing the power consumption of the light source device 21. By increasing the intensity of the light pulse, the ratio of the charge amount of the reflected light RL to the charge amount of the background light can be increased in the charge amounts Q2 and Q3, and the accuracy of the measured distance can be improved by increasing the S / N ratio of the intensity of the reflected light RL.

[0202] Furthermore, in the present embodiment, by setting the detailed measurement mode, when measuring the distance to the subject S in the long distance range, the reflected light RL from the subject S in the short distance range is not input, so that the S / N ratio of the reflected light RL from the subject S in the long distance range can be improved. That is, in the normal mode, since the intensity of the reflected light RL from the subject S in the short distance range is large, the relative intensity of the reflected light RL in the long distance is reduced, so the S / N ratio of the distance D obtained from the formula (1) is increased.

[0203] However, by using the charge amounts Q2 and Q3 of the reflected light RL from the subject S at a long distance to perform calculations using equation (4), the charge amount generated by the high-intensity reflected light RL at a short distance is not included, thereby improving the accuracy (resolution) of distance measurement.

[0204] In addition, in the present embodiment, the number of allocations per frame of the distance image is set so that the storage amount of the charge storage unit CS is not saturated.

[0205] like Fig. 9 As shown in (b), in the normal mode, the number of allocations at which the storage amount of the charge storage unit CS is not saturated is set corresponding to the charge generated by the reflected light RL from the subject S located at a close distance, for example, 0.1 m from the distance image sensor 32. Therefore, the intensity of the reflected light RL from the subject S located at a distance of, for example, 2 m and 4 m from the distance image sensor 32 is low, and therefore, the charge amount is suppressed to a relatively small amount, and the S / N ratio decreases as the distance increases.

[0206] In the present embodiment, by dividing the measurement range into a short-distance range and a long-distance range, when measuring in the long-distance mode, the charge generated by the reflected light RL from the subject S in the short-distance range is not accumulated in the charge accumulation unit CS in the distance image sensor 32. Therefore, in order to accumulate only the charge generated by the reflected light RL with low intensity in the long-distance range, the accumulation amount of the charge accumulation unit CS is suppressed, and the pulse width Tw is shortened to be set to the pulse width Tsw, so that the number of allocations in the frame period for capturing the same frame of the distance image can be increased compared with the normal mode. As a result, the amount of charge generated by the reflected light RL in the long-distance range of 2m or 4m, etc. can be increased, the S / N ratio can be improved, and the accuracy (resolution) of distance measurement can be improved.

[0207] Fig.10 It is a timing chart for explaining the phase of the light pulse PO in each of the normal mode, the short distance mode, and the long distance mode during the charge accumulation period of another configuration of the present embodiment.

[0208] Fig.10 (a) and Figure 7 (a) shows the pulse width Tw of the mode setting value and the phase of the light pulse PO in the normal mode. Fig.10 (b) shows the pulse width Tsw of the mode setting value and the phase of the light pulse PO in the close distance mode. Fig.10 (c) shows the pulse width Tsw of the mode setting value and the phase of the light pulse PO in the long-distance mode.

[0209] In addition, due to Fig.10 (a) and Figure 7 (a) is the same, therefore, the description is omitted.

[0210] Fig.10 (b) is a timing diagram showing the timing of driving the pixel circuit 321 configured in the light-receiving pixel section 320 of the distance image sensor 32 in the close-range mode. The pulse width Tsw of each drive signal in the close-range mode is set to (Dsm)×(2 / c), Dm / c. Here, Dsm is Dm / n, n is the number of divisions, for example, when Dm is 4m and the number of divisions n is 2, Dsm is 2m for the close-range range and the long-range range, respectively. In addition, since the distance range 2 is divided into sub-measurement ranges of the close-range range and the long-range range, the pulse width Tsw is 1 / 2 of the pulse width Tw. The phase of the light pulse PO is the same as that of the accumulation drive signal TX2. Figure 7(b) shows the timing of the drive signal of the pixel circuit 321 when outputting one frame of pixel signals in the close-range mode in the distance image sensor 32 and the timing of the light pulse PO irradiated by the light source unit 2 to the subject S. Here, the phase of the light pulse PO is made equal to the accumulation drive signal TX2 because the sub-measurement range (close-range range) in the close-range mode is 0 (m) < L < 2 (m).

[0211] Before time tC1, the reset drive signal RSTD is supplied to the pixel circuit 321. At time tC1, the pixel drive circuit 326 distributes the charge generated by the photoelectric conversion element PD using the background light to the charge storage unit CS1 by accumulating the drive signal TX1. At time tC2, the pulse of the reset drive signal RSTD is inserted to eliminate the charge generated by the photoelectric conversion element PD. Before the charge generation for calculating the distance D, the charge generated by the background light is completely eliminated from the photoelectric conversion element PD. As a result, the charge amount generated by the background light in the charge amount Q2, which becomes noise, is reduced, and the accuracy of the distance D calculated by the charge amounts Q2 and Q3 is improved. At time tC3, the pixel drive circuit 326 distributes the charge generated by the photoelectric conversion element PD using the reflected light RL reflected from the subject S to the charge storage unit CS2 by accumulating the drive signal TX2. At time tC4, the pixel drive circuit 326 distributes the charge generated by the photoelectric conversion element PD using the reflected light RL reflected from the subject S to the charge storage unit CS3 by accumulating the drive signal TX3. In addition, at time tC5, the charge generated by the photoelectric conversion element PD is eliminated by supplying the reset drive signal RSTD. Through the above-mentioned processing, the reflected light RL from the subject S located at a distance less than 2 m relative to the pixel circuit 321 is distributed and accumulated in the charge accumulation units CS2 and CS3 respectively through the respective accumulation drive signals TX2 and TX3, and the distance image processing unit 4 calculates the distance D between the pixel circuit 321 and the subject S using equation (3).

[0212] Fig.10 (c) is a timing chart showing the timing of driving the pixel circuit 321 arranged in the light receiving pixel section 320 of the distance image sensor 32 in the long-distance mode. The pulse width Tsw of each driving signal in the long-distance mode is set to Dsm / c similarly to the short-distance mode.

[0213] Before time tC1, the reset drive signal RSTD is supplied to the pixel circuit 321. At time tC1, the pixel drive circuit 326 distributes the charge generated by the photoelectric conversion element PD using the background light to the charge storage unit CS1 by accumulating the drive signal TX1. At time tC2, a pulse with a pulse width Tsw of the reset drive signal RSTD is inserted to eliminate the charge generated by the photoelectric conversion element PD. At time tC3, the pixel drive circuit 326 distributes the charge generated by the photoelectric conversion element PD using the reflected light RL reflected from the subject S to the charge storage unit CS2 by accumulating the drive signal TX2. At time tC4, the pixel drive circuit 326 distributes the charge generated by the photoelectric conversion element PD using the reflected light RL reflected from the subject S to the charge storage unit CS3 by accumulating the drive signal TX3. In addition, at time tC5, the charge generated by the photoelectric conversion element PD is eliminated by supplying the reset drive signal RSTD. In addition, the phase of the light pulse PO is the same as the pulse of the reset drive signal RSTD at time tC2.

[0214] Here, the reason why the phase of the light pulse PO is set to the pulse of the reset drive signal RSTD at time tC2 instead of the accumulation drive signal TX2 is that the sub-measurement range L (long-distance range) in the long-distance mode is 2 (m) ≤ L < 4 (m). Therefore, at the timing before the time of 4 m (= Dm) / c of the round-trip distance of 2 m of the light, that is, at the timing of the accumulation drive signal TX2 to distribute the charge, the distance image processing unit 4 irradiates the light pulse PO to the light source unit 2 at the timing of the pulse of the reset drive signal RSTD at time tC2, so that the reflected light RL from the subject S located at a distance of more than 2 m is incident.

[0215] Through the above-mentioned processing, the reflected light RL from the subject S located at a distance of more than 2m relative to the pixel circuit 321 is distributed and accumulated in the charge accumulation units CS2 and CS3 respectively through the respective accumulation drive signals TX2 and TX3, and the distance image processing unit 4 uses formula (4) to calculate the distance D between the pixel circuit 321 and the subject S.

[0216] Furthermore, by inserting a pulse of the reset drive signal RSTD at time tC2, the charge generated by the background light is completely eliminated from the photoelectric conversion element PD before the charge generation for calculating the distance D. Thus, the charge amount generated by the background light in the charge amount Q2, which becomes noise, is reduced, and the accuracy of the distance D calculated by the charge amounts Q2 and Q3 is improved.

[0217] <Second embodiment>

[0218] Hereinafter, a second embodiment of the present invention will be described with reference to the drawings. Fig.111 is a block diagram showing a configuration example of the distance calculation unit 42A in the distance image processing unit 4 of the present embodiment. Hereinafter, the configuration and operation different from the first embodiment will be described.

[0219] Fig.11 The distance calculation section 42A shown includes a charge amount comparison section 426 , a distance calculation section 427 , and a change amount adjustment section 428 .

[0220] The charge amount comparison unit 426 calculates the difference between the charge amounts Q2 and Q3 , obtains the difference charge amount, and outputs it to each of the distance calculation unit 427 and the change amount adjustment unit 428 .

[0221] That is, the charge amount comparison unit 426 obtains the difference voltage of the output digital values ​​OD(CS2) and OD(CS3) from the output digital values ​​OD(CS1), OD(CS2), and OD(CS3) corresponding to the charge amounts Q1, Q2, and Q3 supplied from the distance image sensor 32. In addition, in the present embodiment, the difference voltage of the output digital values ​​OD(CS2) and OD(CS3) is obtained from the pixel signals of predetermined pixels (e.g., the center or centroid pixel of the abstract area) in the abstract area set by the user in the distance image, or from the average value of the pixel signals of the pixels in the abstract area.

[0222] The distance calculation unit 427 calculates the distance D between each pixel circuit 321 in the light-receiving pixel unit 320 and the object S using equation (1) or equation (6) shown below, similarly to the first embodiment. In equation (6) below, Tc is a phase adjustment time for adjusting the phase of the light pulse PO (details will be described later).

[0223] D=(c / 2)Tc+(Q3'-Q1) / (Q2'+Q3'-2Q1)×Dm, (6)

[0224] The distance calculation unit 427 selects which of the equations (1) and (2) to use to calculate the distance D, based on the difference voltage supplied from the charge amount comparison unit 426. Here, the distance calculation unit 427 calculates the distance D using equation (6) when the difference voltage is "0", and calculates the distance D using equation (1) when the difference voltage is "0".

[0225] The change amount adjustment section 428 obtains the adjustment amount of the phase (timing) of the emission light pulse PO based on the difference voltage supplied from the charge amount comparison section 426 .

[0226] Fig.12 This is a diagram for explaining adjustment of the phase of the emitted light pulse PO by the variation adjustment unit 428 according to the present embodiment.

[0227] exist Fig.12 In (a), when the respective charge amounts Q2 and Q3 are set to be the same, when the charge amount Q2 is less than the charge amount Q3, the charge amount Q2 needs to be increased. Therefore, in order to increase the charge amount Q2, it is considered to shorten the distance between the distance image sensor 32 and the subject S. That is, it is equivalent to actually shortening the distance between the distance image sensor 32 and the subject S by advancing the timing of emitting the light pulse PO by a predetermined time (advancing the phase). In area 501, the phase of the light pulse PO emitted before the timing adjustment (time tD2-tD5, the same phase as the accumulation drive signal TX2) and the phase of the reflected light RL from the subject S (time tD4-tD7) are shown. In addition, in area 502, the phase of the light pulse PO emitted after the timing adjustment (time tD1) and the phase of the reflected light RL from the subject S (time tD3-tD6) are shown.

[0228] Therefore, the change amount adjuster 428 obtains the phase adjustment amount corresponding to the value of the difference voltage and the phase advancement adjustment time Tc. Then, the change amount adjuster 428 outputs a timing control signal instructing the timing control unit 41 to emit the light pulse PO including the adjustment time Tc.

[0229] Thus, the timing control unit 41 adjusts the timing of outputting the light pulse PO earlier by the time Tc from the time tD2 to the time tD1.

[0230] As described above, by advancing the phase, the charge amount Q2 is increased and the charge amount Q3 is decreased, thereby obtaining charge amounts Q2' and Q3' that are equal to each other.

[0231] By increasing the charge amount Q2, when using formula (1), the distance D from the subject S is calculated to be relatively small. Therefore, as a correction value for advancing the phase of the irradiation of the light pulse PO by the adjustment time Tc, in formula (6), the distance (c / 2) Tc of the light reciprocation is added to the adjustment time Tc. At this time, in order to perform the correction of increasing the distance, the polarity of the adjustment time Tc is "+".

[0232] By substituting the equal charge amounts Q2' and Q3' into equation (6) through the above-described processing, the distance D to the subject S when the phase of the irradiation of the light pulse PO is advanced by the adjustment time Tc can be calculated.

[0233] exist Fig.12In (b), when the respective charge amounts Q2 and Q3 are set to be the same, when the charge amount Q2 exceeds the charge amount Q3, the charge amount Q2 needs to be reduced. Therefore, in order to reduce the charge amount Q2, it is considered to lengthen the distance between the distance image sensor 32 and the subject S. That is, it is equivalent to actually lengthening the distance between the distance image sensor 32 and the subject S by delaying the timing of emitting the light pulse PO by a predetermined time (delay phase). In area 511, the phase of the light pulse PO emitted before the timing adjustment (time tE1-tE5, the same phase as the accumulation drive signal TX2) and the phase of the reflected light RL from the subject S (time tE3-tE6) are shown. In addition, in area 512, the phase of the light pulse PO emitted after the timing adjustment (time tE2) and the phase of the reflected light RL from the subject S (time tE4-tE7) are shown.

[0234] Therefore, the change amount adjuster 428 obtains the phase adjustment amount corresponding to the value of the difference voltage and the adjustment time Tc of the delayed phase. The change amount adjuster 428 outputs a control signal to the timing control unit 41 to instruct the timing control of the emission of the light pulse PO including the adjustment time Tc.

[0235] Thus, the timing control unit 41 delays the timing of outputting the optical pulse PO from the time tE1 to the time tE2 by the adjustment time Tc.

[0236] As described above, by advancing the phase, the charge amount Q2 is reduced and the charge amount Q3 is increased, thereby obtaining charge amounts Q2' and Q3' that are equal to each other.

[0237] By reducing the charge amount Q2, when using formula (1), the distance D from the subject S is calculated to be relatively large. Therefore, as a correction value for delaying the phase of the irradiation of the light pulse PO by the adjustment time Tc, in formula (6), the distance (c / 2) Tc of the light reciprocation is added to the adjustment time Tc. At this time, in order to perform the correction to reduce the distance, the polarity of the adjustment time Tc is "-".

[0238] By substituting the equal charge amounts Q2' and Q3' into equation (6) through the above-described processing, the distance D to the subject S when the phase of the irradiation of the light pulse PO is delayed by the adjustment time Tc can be calculated.

[0239] For the noise (shot noise) generated by the background light, the greater the signal intensity, the higher the S / N ratio and the better the distance resolution. However, when the ratio of the signal intensity of one of the charge amounts Q2 and Q3 to the other is significantly different, as shown below, the noise increases and the distance resolution of the one with the smaller signal intensity ratio decreases.

[0240] When the ratio of the charge amounts is Q2:Q3=1:3

[0241] Noise=(12+32)1 / 2=(10)1 / 2≈3.16

[0242] When the ratio of the charge amounts is Q2:Q3=2:2

[0243] Noise=(22+22)1 / 2=(8)1 / 2≈2.83

[0244] By making the respective charge amounts Q2 and Q3 equal in this manner, noise can be reduced.

[0245] As described above, according to this embodiment, when the distance D to the subject is measured, the phase of the light pulse PO is adjusted so that the charge amounts Q2 and Q3 are equal in the distance D, thereby improving the distance resolution and obtaining a more accurate distance to the subject S.

[0246] In addition, in this embodiment, it is described that the charge amounts Q2 and Q3 are set to be the same, that is, the difference voltage is set to 0, but it can also be constructed so that a difference voltage range of a specified width is set according to the size of the allowable noise, and an adjustment time Tc is generated so that the difference voltage falls within the difference voltage range.

[0247] Below, refer to Fig.13 Next, a process of adjusting the phase of the light pulse PO in distance measurement by the distance image processing unit 4 according to the present embodiment will be described. Fig.13 1 is a flowchart showing an example of the operation of the distance image camera device 1 of the present embodiment to adjust the phase of the light pulse PO. In the following distance measurement, the distance image sensor 32 performs phase adjustment of the irradiation of the light pulse PO by using the pixel signal of the target area in the distance image captured in units of frames, and measures the pixel signals of all the pixel circuits 321 (for each frame of the captured image).

[0248] Step SB1 : The user uses, for example, an input unit (not shown) to input information of a region of interest (distance measurement target) that the user wants to capture in detail in the range image to the range image capturing apparatus 1 .

[0249] In addition, it is also possible to set the information of the above-mentioned area of ​​interest in the distance image in advance for the distance image camera 1, and surround the area of ​​interest set in the distance image with an image frame. In this case, the user adjusts the camera direction of the distance image camera 1 so that the object whose distance is to be measured in detail enters the above-mentioned image frame. In the case of this structure, when the user changes the object of interest, he or she performs an operation indicating the change. Fig.13 processing.

[0250] Step SB2 : After obtaining pixel signals for one frame, the distance image sensor 32 sequentially outputs pixel signals to the distance image processing unit 4 .

[0251] Then, the charge amount comparison unit 426 extracts a pixel signal corresponding to the set target area from the distance image (pixel signals of all pixel circuits 321 for one frame) supplied from the distance image sensor 32 (obtains pixel signals).

[0252] Step SB3: The charge amount comparison unit 426 compares the charge amounts Q2 and Q3 in the pixel signal of the target area, that is, compares the output digital values ​​OD(CS2) and OD(CS3). Here, the charge amount comparison unit 426 determines whether the output digital value OD(CS2) exceeds the output digital value OD(CS3).

[0253] At this time, when the output digital value OD (CS2) exceeds the output digital value OD (CS3), the charge amount comparison unit 426 advances the processing to step SB4. On the other hand, when the output digital value OD (CS2) does not exceed the output digital value OD (CS3) (either the output digital value OD (CS2) is equal to the output digital value OD (CS3) or the output digital value OD (CS2) is less than the output digital value OD (CS3)), the charge amount comparison unit 426 advances the processing to step SB5.

[0254] Step SB4 : The charge amount comparison unit 426 obtains the difference voltage (positive polarity in this case) between the output digital value OD ( CS2 ) and the output digital value OD ( CS3 ), and outputs it to the change amount adjustment unit 428 .

[0255] The change amount adjustment unit 428 refers to, for example, a correspondence table in which correspondence between the difference voltage and the adjustment time Tc is recorded, and extracts and obtains the adjustment time Tc corresponding to the difference voltage supplied from the charge amount comparison unit 426 .

[0256] Alternatively, a formula for calculating the adjustment time Tc from a predetermined difference voltage may be generated in advance based on an experiment, and the change amount adjustment unit 428 may calculate the adjustment time Tc by substituting the difference voltage supplied from the charge amount comparison unit 426 into the formula.

[0257] At this time, since the output digital value OD( CS2 ) exceeds the output digital value OD( CS3 ), the difference voltage has a positive polarity, and the variation adjustment unit 428 obtains the adjustment time Tc as the time for delaying the phase of the optical pulse PO.

[0258] Then, the change amount adjustment unit 428 adds the adjustment time Tc to the control signal instructing the timing control of the emission of the light pulse PO, and outputs the signal to the timing control unit 41 .

[0259] Thus, the timing control unit 41 delays the pulse light PO from the light source device 21 by the adjustment time Tc. In addition, the change amount adjustment unit 428 outputs the adjustment time Tc to the distance calculation unit 427.

[0260] Step SB5: The charge amount comparison unit 426 compares the output digital values ​​OD(CS2) and OD(CS3) in the pixel signal of the target area. Here, the charge amount comparison unit 426 determines whether the output digital value OD(CS3) exceeds the output digital value OD(CS2).

[0261] At this time, when the output digital value OD (CS3) exceeds the output digital value OD (CS2), the charge amount comparison unit 426 advances the process to step SB6. On the other hand, when the output digital value OD (CS3) does not exceed the output digital value OD (CS2) (when the output digital value OD (CS3) is equal to the output digital value OD (CS2)), the charge amount comparison unit 426 advances the process to step SB7.

[0262] Step SB6 : The charge amount comparison unit 426 obtains the difference voltage (negative polarity in this case) between the output digital value OD ( CS2 ) and the output digital value OD ( CS3 ), and outputs it to the change amount adjustment unit 428 .

[0263] The change amount adjustment unit 428 refers to, for example, a correspondence table in which correspondence between the difference voltage and the adjustment time Tc is recorded, and extracts and obtains the adjustment time Tc corresponding to the difference voltage supplied from the charge amount comparison unit 426 .

[0264] Alternatively, a formula for calculating the adjustment time Tc from a predetermined difference voltage may be generated in advance based on an experiment, and the change amount adjustment unit 428 may calculate the adjustment time Tc by substituting the difference voltage supplied from the charge amount comparison unit 426 into the formula.

[0265] At this time, since the output digital value OD(CS3) exceeds the output digital value OD(CS2), the difference voltage has a negative polarity, and the change amount adjustment unit 428 obtains the adjustment time Tc as the time for advancing the phase of the light pulse PO.

[0266] Then, the change amount adjustment unit 428 adds the adjustment time Tc to the control signal instructing the timing control of the emission of the light pulse PO, and outputs the signal to the timing control unit 41 .

[0267] Thus, the timing control unit 41 causes the pulse light PO to be emitted from the light source device 21 in advance by the adjustment time Tc. In addition, the change amount adjustment unit 428 outputs the adjustment time Tc to the distance calculation unit 427.

[0268] Step SB7: The distance calculation unit 427 substitutes the adjustment time Tc of the phase of the pulse light PO in the pixel signals of all frames, the charge amount Q1 (output digital value OD(CS1)), the charge amount Q2' (output digital value OD(CS2)) and Q3' (output digital value OD(CS3)) into formula (6) to calculate the distance D.

[0269] Furthermore, the phase adjustment of the emission light pulse PO in the present embodiment may be performed after the mode selection of the short-distance mode and the long-distance mode of the region of interest in the first embodiment.

[0270] That is, the charge amount comparison unit 426 and the change amount adjustment unit 428 of this embodiment are added to the distance calculation unit 42 of the first embodiment.

[0271] Furthermore, the distance calculation unit 42 of the first embodiment is configured to select a sub-measurement range as a detailed measurement mode, and to perform measurement when the subject S in the target area is located in either the short distance range or the long distance range.

[0272] Then, the charge amount comparison unit 426 compares the charge amounts Q2 and Q3 and adjusts the phase of emission of the light pulse PO according to the detailed measurement mode.

[0273] For example, in Figure 7 In the case of the close distance mode shown in (b), the phase of the light pulse PO is advanced or delayed by the adjustment time Tc starting from the time tB2. Figure 7 In the case of the long-distance mode (c), the phase of the light pulse PO is advanced or delayed by the adjustment time Tc starting from time tB1. At this time, (c / 2)Tc is added as the correction distance to each equation (3) and equation (4) used in each detailed measurement mode, as well as equation (6).

[0274] In this way, by adding the configuration of adjusting the phase of the light pulse PO of the second embodiment to the distance calculation unit 42 of the first embodiment, the distance D between the range image capturing device 1 and the subject S can be measured with a higher resolution than that of the first embodiment.

[0275] <Third embodiment>

[0276] Hereinafter, a third embodiment of the present invention will be described with reference to the drawings. Fig.14 1 is a block diagram showing a configuration example of the distance calculation unit 42B in the distance image processing unit 4 of the present embodiment. Hereinafter, the configuration and operation different from the second embodiment will be described.

[0277] Fig.14The distance calculation unit 42B shown includes a charge amount comparison unit 426, a distance calculation unit 427, a change amount adjustment unit 428, and a pulse width adjustment unit 429. Here, each of the charge amount comparison unit 426, the distance calculation unit 427, and the change amount adjustment unit 428 has the same configuration as the distance calculation unit 42A of the second embodiment.

[0278] The pulse width adjustment unit 429 adjusts the pulse width (including the phase) of the storage drive signals TX1, TX2, and TX3 according to the ratio between the charge amounts Q2 and Q3. In addition, the width of the optical pulse PO is fixed to the pulse width Tw.

[0279] Fig.15 It is a diagram for explaining adjustment of the pulse widths of the storage drive signals TX1 , TX2 , and TX3 according to the present embodiment.

[0280] Fig.15 (a) shows the charge generated by the background light and the charge generated by the reflected light RL when the respective charge amounts Q2 and Q3 are set to be the same.

[0281] Fig.15 (b) shows the charges generated by the background light and the charges generated by the reflected light RL after the pulse widths of the storage drive signals TX1 , TX2 , and TX3 are adjusted.

[0282] exist Fig.15 In (a), the storage drive signal TX2 rises at time tF1 and falls after time tF3. In addition, the storage drive signal TX3 rises at time tF3 and falls after time tF5.

[0283] Here, the pulse width adjustment unit 429 determines the phase (time tF2-tF4) of the reflected pulse PL based on the ratio of the charge amounts Q2 and Q3, that is, the ratio of the output digital values ​​OD(CS2) and OD(CS3). Thus, the pulse widths TX1 (time tF2-tF3) and TX2 (time tF3-tF4) are obtained based on the time tF3. Here, Tw=Tw1+Tw2.

[0284] exist Fig.15 In (b), the pulse width adjustment unit 429 adjusts the pulse width of the storage drive signals TX1, TX2, and TX3 according to the phase of the reflected light RL. In addition, as described in the second embodiment, by adjusting the phase of the pulse light PO so that the respective charge amounts Q2 and Q3 are the same, Tw1=Tw2=Tw / 2 is obtained.

[0285] Thus, the pulse width adjustment unit 429 sets the pulse width of the storage drive signals TX1, TX2, and TX3 to Tw / 2 and outputs a control signal for controlling the timing control unit 41 to adjust the phases of the storage drive signals TX1, TX2, and TX3 with reference to the time tF3.

[0286] Thus, the pulse widths of the storage drive signals TX1, TX2, and TX3 are set to Tw / 2, and the timing control unit 41 controls the distance image sensor 32 as follows: the storage drive signal TX1 rises at time tF1 and falls at time tF2, the storage drive signal TX2 rises at time tF2 and falls at time tF3, and the storage drive signal TX3 rises at time tF3 and falls at time tF4.

[0287] Through the above structure, such as Fig.15 As shown, the amount of charge generated by the background light can be reduced before and after the pulse width is adjusted without reducing the amount of charge generated by the reflected light RL. That is, the shot noise can be reduced, and even when the background light is strong, the reduction in the accuracy of distance measurement caused by the reflected light RL can be suppressed.

[0288] In addition, by reducing the amount of charge generated by the background light, the charge accumulation unit CS has a surplus of charge, and shortening the pulse width of the accumulation drive signals TX1, TX2, and TX3. Therefore, the number of allocations per frame can be increased without changing the frame period, and the proportion of the charge generated by the reflected light RL in the charge can be increased, thereby improving the S / N ratio and improving the accuracy of the measurement of the distance D.

[0289] Below, refer to Fig.16 Next, a process of adjusting the pulse widths of the storage drive signals TX1 , TX2 , and TX3 in distance measurement by the distance image processing unit 4 according to the present embodiment will be described. Fig.16 1 is a flowchart showing an example of the operation of adjusting the pulse width of the storage drive signals TX1, TX2, and TX3 by the distance image camera device 1 of this embodiment. In the following distance measurement, the distance image sensor 32 adjusts the phase of the light pulse PO irradiation by the pixel signal of the target area in the captured distance image, adjusts the pulse width of the storage drive signals TX1, TX2, and TX3, and measures the pixel signals of all pixel circuits 321 (each frame of the captured image). In addition, steps SB1 to SB6 are the same operations as those in the second embodiment, and therefore, the description thereof is omitted.

[0290] Step SB8 : The pulse width adjustment unit 429 extracts the pixel signal in the target area from the pixel signal supplied from the distance image sensor 32 in each frame period.

[0291] Furthermore, the pulse width adjustment unit 429 compares the output digital values ​​OD(CS2) and OD(CS3). Fig.15 As described above, the processing of adjusting the pulse width and phase of the storage drive signals TX1 , TX2 , and TX3 is performed.

[0292] Furthermore, the timing control unit 41 causes the distance image sensor 32 to adjust the pulse width and phase of the storage drive signals TX1, TX2, and TX3 so that the storage drive signals TX2 and TX3 are as follows: Fig.15 (b) shows the same phase as the reflected light RL and includes the pulse width Tw of the reflected light RL.

[0293] Step SB9 : After obtaining pixel signals for one frame, the distance image sensor 32 sequentially outputs pixel signals to the distance image processing unit 4 .

[0294] Thus, the distance calculation unit 427 inputs the distance image (pixel signals of all pixel circuits 321 for one frame) supplied from the distance image sensor 32 (obtaining pixel signals).

[0295] Step SB10: The distance calculation unit 427 substitutes the output digital values ​​OD(CS1), OD(CS2), OD(CS3) in the input pixel signal as the charge amounts Q1, Q2', and Q3' into equation (6) to calculate the distance D to the subject S.

[0296] Step SB11: The pulse width adjustment unit 429 determines whether the user has performed an operation to change the region of interest through the input unit.

[0297] At this time, if a change operation is performed, the pulse width adjustment unit 429 advances the process to step SB1. On the other hand, if a change operation is not performed, the pulse width adjustment unit 429 advances the process to step SB9.

[0298] Alternatively, the pulse width adjustment unit 429 may be configured to return the process to step SB1 when the measured distance D has a change exceeding a predetermined threshold value compared to the distance D measured previously, assuming that the subject S in the region of interest has moved.

[0299] As mentioned above, although the embodiment of the present invention is described in detail with reference to the drawings, the specific structure is not limited to this embodiment, and also includes the design etc. within the scope which does not deviate from the summary of the present invention.

[0300] Description of Reference Numerals

[0301] 1. Distance image camera device; 2. Light source unit; 3. Light receiving unit; 4. Distance image processing unit; 21. Light source device; 22. Diffuser; 31. Lens; 32. Distance image sensor; 41. Timing control unit; 42, 42A, 42B, Distance calculation unit; 320, Light receiving pixel unit; 321, Pixel circuit; 322, Control circuit; 323, Vertical scanning circuit; 324, Horizontal scanning circuit; 325, Pixel signal processing circuit; 326, Pixel driving circuit; 329, AD conversion circuit; 329j, Column AD conversion unit; 329j (CS1), 329j (CS2), 329j (CS3), Column AD conversion circuit; 420 , charge quantity determination unit; 421, 427, distance calculation unit; 422, distance determination unit; 423, mode setting unit; 424, measurement capability unit; 425, mode table; 426, charge quantity comparison unit; 428, change adjustment unit; 429, pulse width adjustment unit; C, charge storage capacitor; CS, charge storage unit; FD, floating diffusion; G, readout gate transistor; GD, drain gate transistor; O, output terminal; P, measurement space; PD, photoelectric conversion element; PO, light pulse; RL, reflected light; RT, reset gate transistor; RU, pixel signal readout unit; S, object; SF, source follower gate transistor; SL, select gate transistor.

Claims

1. A distance image camera device, characterized in that: The distance image camera device comprises: a light source unit for irradiating irradiation light to a space to be measured, that is, a measurement space; a light-receiving pixel section, comprising: a photoelectric conversion element, which receives reflected light reflected by the irradiation light on an object in the measurement space and background light in the environment of the measurement space, and generates electric charges corresponding to the received reflected light and the background light; and a charge accumulation section, which accumulates the electric charges when the irradiation light is irradiated in a frame period, the light-receiving pixel section having a pixel circuit that accumulates the electric charges in the charge accumulation section in synchronization with the irradiation of the irradiation light; and a distance image processing unit that measures the distance in a normal mode of a predetermined width of irradiation light when performing distance measurement using an input voltage corresponding to the amount of charge accumulated in the charge accumulation unit, and switches to a detailed measurement mode and adjusts the width together with the phase of the irradiation light irradiated from the light source unit in accordance with the distance to the object measured in the normal mode; In the detailed measurement mode, the range of distances that can be measured in the normal mode, i.e., the distance range, is divided into multiple sub-measurement ranges of the same width, the width of the irradiation light is set corresponding to the sub-measurement range, and the phase of the irradiation light is set corresponding to the minimum distance in the sub-measurement range.

2. The distance image camera device according to claim 1, characterized in that: After measuring the distance in the normal mode, the distance image processing unit performs distance measurement in the detailed measurement mode based on the width and phase of the irradiation light set corresponding to the sub-measurement range including the distance obtained by the measurement.

3. The distance image camera device according to claim 2, characterized in that: The distance image processing unit distributes the charge at a predetermined fixed number of times in the normal mode, and obtains the distance to the object existing in the measurement space based on the amount of the charge accumulated in the plurality of distributed charge storage units in the charge storage unit, i.e., the charge amount. After the distance is measured in the normal mode, the distance is measured in the detailed measurement mode based on the charge amount in the charge distribution number set corresponding to the sub-measurement range including the distance obtained by the measurement.

4. The distance image camera device according to claim 1, characterized in that: The distance image processing unit adjusts the intensity of the irradiation light emitted from the light source unit according to the distance to the object.

5. The distance image camera device according to claim 1, characterized in that: When the distance image processing unit obtains the distance to the object based on the ratio of the amounts of the charges accumulated in the first distributed charge accumulation unit and the second distributed charge accumulation unit, which are two charge accumulation units that accumulate the charges of the reflected light, After switching to the detailed measurement mode, when the charge amount generated by the reflected light is less than a preset charge amount threshold, either the first distributed charge storage unit or the second distributed charge storage unit switches to the normal mode without performing a process of obtaining the distance.

6. The distance image camera device according to claim 5, characterized in that: The charge amount threshold is the charge amount accumulated in the background light charge accumulation unit using background light.

7. A distance image camera device, characterized in that: a light source unit for irradiating irradiation light to a space to be measured, that is, a measurement space; a light-receiving pixel section, comprising: a photoelectric conversion element, which receives reflected light reflected by the irradiation light on an object in the measurement space and background light in the environment of the measurement space, and generates electric charges corresponding to the received reflected light and the background light; and a charge accumulation section, which accumulates the electric charges when the irradiation light is irradiated in a frame period, the light-receiving pixel section having a pixel circuit that accumulates the electric charges in the charge accumulation section in synchronization with the irradiation of the irradiation light; and a distance image processing unit that measures the distance in a normal mode of a predetermined width of irradiation light when performing distance measurement using an input voltage corresponding to the amount of charge accumulated in the charge accumulation unit, and switches to a detailed measurement mode and adjusts the width together with the phase of the irradiation light irradiated from the light source unit in accordance with the distance to the object measured in the normal mode; When the distance image processing unit obtains the distance to the object based on the ratio of the amounts of the charges accumulated in the first distributed charge accumulation unit and the second distributed charge accumulation unit, which are two charge accumulation units that accumulate the charges of the reflected light, After measuring the distance in the normal mode, the phase of the irradiation light is adjusted so that the charge amounts of the first distributed charge accumulation section and the second distributed charge accumulation section are equal, and the distance is obtained based on the charge amounts and the amount of phase adjustment.

8. The distance image camera device according to claim 7, characterized in that: After adjusting the phase of the irradiation light to make the charge amounts the same, the distance image processing unit adjusts the width of the accumulation driving signal for respectively distributing the charge to the first distribution charge accumulation unit and the second distribution charge accumulation unit so that there is no region that does not include the charge generated by the reflected light.

9. A distance image capturing method, characterized in that: The distance image capturing method comprises the following steps: irradiating the irradiation light from the light source unit to the measurement space, which is the space of the measurement object; causing a photoelectric conversion element to receive reflected light of the irradiated light reflected on an object in the measurement space and background light in an environment of the measurement space, and to generate electric charges corresponding to the received reflected light and background light; accumulating the charge generated by the reflected light in a charge accumulation section in synchronization with the irradiation of the irradiation light in a frame period; When distance measurement is performed using an input voltage corresponding to the amount of charge accumulated in the charge accumulation section, measuring the distance in a normal mode with a predetermined width of irradiated light; and In accordance with the distance to the object measured in the normal mode, switching to the detailed measurement mode and adjusting the width together with the phase of the irradiation light irradiated from the light source unit; In the detailed measurement mode, the range of distances that can be measured in the normal mode, i.e., the distance range, is divided into multiple sub-measurement ranges of the same width, the width of the irradiation light is set corresponding to the sub-measurement range, and the phase of the irradiation light is set corresponding to the minimum distance in the sub-measurement range.

10. A distance image capturing method, characterized in that: The distance image capturing method comprises the following steps: irradiating the irradiation light from the light source unit to the measurement space, which is the space of the measurement object; causing a photoelectric conversion element to receive reflected light of the irradiated light reflected on an object in the measurement space and background light in an environment of the measurement space, and to generate electric charges corresponding to the received reflected light and background light; accumulating the charge generated by the reflected light in a charge accumulation section in synchronization with the irradiation of the irradiation light in a frame period; When distance measurement is performed using an input voltage corresponding to the amount of charge accumulated in the charge accumulation section, the distance is measured in a normal mode with a predetermined width of irradiated light; and switching to a detailed measurement mode to adjust a phase of the irradiation light irradiated from the light source section in accordance with the distance to the object measured in the normal mode; When the distance to the object is determined based on the ratio of the amount of the charge accumulated in the first distributed charge accumulation section and the second distributed charge accumulation section, which respectively accumulate the charge of the reflected light, After measuring the distance in the normal mode, the phase of the irradiation light is adjusted so that the charge amounts of the first distributed charge accumulation section and the second distributed charge accumulation section are equal, and the distance is obtained based on the charge amounts and the amount of phase adjustment.

Citation Information

Patent Citations

  • Distance image sensor

    JP2004294420A

  • Imaging apparatus and driving method of imaging apparatus

    JP2015210176A