Distance image capturing apparatus and distance image capturing method

Abstract

To reduce power consumption while maintaining ranging accuracy equivalent to ranging by sub-frame driving.SOLUTION: A distance image capturing apparatus measures a distance to an object based on a time until light emitted from a light source is reflected by the object and returns. The distance image capturing apparatus has a first distance measurement mode in which a distance to at least a part of a distance-measurable range is measured in one frame, and a second distance measurement mode in which the distance-measurable range is divided into a plurality of distance measurement ranges in a depth direction, and a distance to at least one of the divided distance measurement ranges is measured in at least one of subframes obtained by dividing one frame according to the divided distance measurement ranges. A control unit included in the distance image capturing apparatus switches between the first distance measurement mode and the second distance measurement mode.SELECTED DRAWING: Figure 10

Description

Technical Field

[0001] The present invention relates to a distance image capturing device and a distance image capturing method.

Background Art

[0002] A time-of-flight (TOF) type distance image capturing device that measures the distance between a measuring instrument and an object based on the flight time of light in space (measurement space) by utilizing the fact that the speed of light is known has been realized (see, for example, Patent Document 1). In such a distance image capturing device, the delay time from the time when a light pulse is irradiated until the light pulse reflected by the subject returns is obtained by accumulating the charges generated by a photoelectric conversion element in a plurality of charge accumulation units, and the distance to the subject is calculated using the delay time and the speed of light.

[0003] In such a distance image capturing device, in order to widen the distance measurement range, distance measurement by sub-frame driving in which one frame is divided into a plurality of sub-frames for distance measurement is known. In each sub-frame, the timing of the gate pulse for accumulating charges in the charge accumulation unit is made relatively different with respect to the timing of irradiating the light pulse. For example, in a certain sub-frame, the opening and closing timing of the gate pulse is set so as to be able to receive reflected light arriving from a relatively short distance. In another sub-frame, the opening and closing timing of the gate pulse is set so as to be able to receive reflected light arriving from a relatively long distance. By performing distance measurement by sub-frame driving in this way, it is possible to make a wide range from a short distance to a long distance the range that can be measured, and to accurately measure the distance with relatively fine distance resolution.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, much of the power consumption of the TOF camera is occupied by the light pulse irradiation light source. As the frame rate increases, the number of light emissions from the light source also increases, and the power consumption increases. Therefore, in the distance measurement by sub-frame driving, since the light pulse is irradiated multiple times within one frame, the power consumption per frame increases. As the power consumption increases, the camera generates heat, which deteriorates the noise of the sensor. This poses a problem because it has an adverse effect on the distance measurement performance of the TOF camera.

[0006] The present invention has been made in view of the above problems, and one of its objects is to provide a distance image capturing device and a distance image capturing method capable of reducing power consumption while maintaining the same distance measurement accuracy as that of distance measurement by sub-frame driving.

Means for Solving the Problems

[0007] A distance image capturing device according to an aspect of the present invention is a distance image capturing device that measures the distance to an object based on the time it takes for light irradiated from a light source to be reflected back from the object, and includes a first distance measurement mode for measuring the distance within at least a part of the distance measurement range in one frame, and a second distance measurement mode for dividing the distance measurement range into a plurality of distance measurement ranges in the depth direction, and measuring at least one of the divided distance measurement ranges with at least one of the sub-frames obtained by dividing the one frame according to the divided distance measurement ranges, and a control unit for switching between the first distance measurement mode and the second distance measurement mode.

[0008] In addition, a distance image capturing method performed by a distance image capturing apparatus that measures the distance to an object based on the time it takes for light emitted from a light source to be reflected back from the object includes a first distance measurement mode in which at least a part of the measurable distance range is measured in one frame, and the measurable distance range is divided into a plurality of distance measurement ranges in the depth direction, and at least one of the divided distance measurement ranges is measured with at least one sub-frame obtained by dividing the one frame according to the divided distance measurement ranges, that is, a second distance measurement mode. A control unit switches between the first distance measurement mode and the second distance measurement mode.

Advantages of the Invention

[0009] According to the present invention, it is possible to reduce power consumption while maintaining the same distance measurement accuracy as that of distance measurement by sub-frame driving.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Mode for Carrying Out the Invention

[0011] Hereinafter, the distance image capturing device of the present embodiment will be described with reference to the drawings.

[0012] FIG. 1 is a block diagram showing a schematic configuration of a distance image capturing device according to the present embodiment. The distance image capturing device 1 is a distance image capturing device that measures the distance to an object (distance measurement) using the TOF method, and includes, for example, a light source unit 2, a light receiving unit 3, and a distance image processing unit 4. FIG. 1 also shows a subject OB, which is an object whose distance is measured by the distance image capturing device 1.

[0013] The light source unit 2 irradiates the space to be measured with an optical pulse PO in accordance with control from the distance image processing unit 4. The light source unit 2 is, for example, a surface-emitting semiconductor laser module such as a vertical cavity surface emitting laser (VCSEL). The light source unit 2 includes a light source device 21 and a diffuser plate 22.

[0014] The light source device 21 is a light source that emits laser light in the near-infrared wavelength band (for example, the wavelength band with a wavelength of 850 nm to 940 nm) that becomes the light pulse PO for irradiating the space to be measured. The light source device 21 is, for example, a semiconductor laser light-emitting element. The light source device 21 emits pulsed laser light in response to control from the distance image processing unit 4.

[0015] The diffusion plate 22 is an optical component that diffuses the laser light in the near-infrared wavelength band emitted by the light source device 21 to the area of the surface for irradiating the space to be measured. The pulsed laser light diffused by the diffusion plate 22 is emitted as the light pulse PO and irradiates the space to be measured.

[0016] When the subject OB exists in the space to be measured where the distance is measured in the distance image capturing device 1, the light receiving unit 3 receives the reflected light RL of the light pulse PO reflected by the subject OB from the light pulse PO irradiated from the light source unit 2, and outputs a pixel signal corresponding to the received reflected light RL. The light receiving unit 3 includes a lens 31 and a distance image sensor 32.

[0017] The lens 31 is an optical lens that guides the incident reflected light RL to the distance image sensor 32. The lens 31 emits the incident reflected light RL toward the distance image sensor 32 side and causes it to be received (incident) by the pixels provided in the light receiving area of the distance image sensor 32.

[0018] The distance image sensor 32 is an imaging element used in the distance image capturing device 1. The distance image sensor 32 includes a plurality of pixels in a two-dimensional light receiving area. In each pixel of the distance image sensor 32, one photoelectric conversion element, a plurality of charge storage units corresponding to this one photoelectric conversion element, and a component for distributing charges to each charge storage unit are provided. That is, the pixel is an imaging element having a distribution configuration for distributing and accumulating charges in a plurality of charge storage units.

[0019] The distance image sensor 32 distributes the charges generated by the photoelectric conversion elements to the respective charge storage units according to the control from the timing control unit 41. Further, the distance image sensor 32 outputs a pixel signal corresponding to the amount of charge distributed to the charge storage unit. In the distance image sensor 32, a plurality of pixels are arranged in a two-dimensional matrix, and a pixel signal for one frame corresponding to each pixel is output. For example, the distance image sensor 32 is an IR image sensor that outputs a pixel signal of an IR image (infrared image).

[0020] Here, in the distance image capturing device 1, the range (distance range) in the depth direction measurable in the space of the measurement target for measuring the distance is mainly determined by the light intensity of the light pulse PO irradiated from the light source unit 2 and the light receiving sensitivity of the light receiving unit 3. Further, the range in the surface direction measurable for distance measurement is determined by the irradiation angle (spread of light) of the light pulse PO irradiated from the light source unit 2 and the light receiving angle (angle at which light can be received) of the light receiving unit 3.

[0021] In the present embodiment, there are a method of measuring the distance of the space of the measurement target (measurable range) in one frame and a method of dividing the space of the measurement target (measurable range) into a plurality of distance measurement ranges in the depth direction and measuring the distance for each of the plurality of distance measurement ranges with a plurality of sub-frames (frames obtained by dividing one frame into a plurality), and switching between the methods is possible.

[0022] The distance image processing unit 4 controls the distance image capturing device 1 and calculates the distance to the subject OB. The distance image processing unit 4 includes a timing control unit 41, a distance calculation unit 42, and a measurement control unit 43.

[0023] The timing control unit 41 controls the timing for outputting various control signals required for measurement according to the control of the measurement control unit 43. The various control signals here are, for example, a signal for controlling the irradiation of the optical pulse PO, a signal for distributing and accumulating the reflected light RL to a plurality of charge accumulation units, a signal for controlling the number of accumulations per frame, and the like. The number of accumulations is the number of times the process of distributing and accumulating charges to the charge accumulation unit CS (see FIG. 3) is repeated. The product of this number of accumulations and the time (accumulation time) for accumulating charges in each charge accumulation unit per one process of distributing and accumulating charges is the accumulation time.

[0024] The distance calculation unit 42 outputs distance information obtained by calculating the distance to the subject OB based on the pixel signals output from the distance image sensor 32. The distance calculation unit 42 calculates the delay time from the irradiation of the optical pulse PO to the reception of the reflected light RL based on the amount of charge accumulated in the plurality of charge accumulation units. The distance calculation unit 42 calculates the distance to the subject OB according to the calculated delay time.

[0025] The measurement control unit 43 controls the timing control unit 41. For example, the measurement control unit 43 sets the number of accumulations and the accumulation time for one frame, and controls the timing control unit 41 so that imaging is performed with the set content.

[0026] With such a configuration, in the distance image capturing device 1, the light receiving unit 3 receives the reflected light RL of the near-infrared wavelength band optical pulse PO irradiated by the light source unit 2 to the subject OB, and the distance image processing unit 4 outputs distance information obtained by measuring the distance to the subject OB.

[0027] In FIG. 1, the distance image capturing device 1 having a configuration in which the distance image processing unit 4 is provided inside the distance image capturing device 1 is shown, but the distance image processing unit 4 may be a component provided outside the distance image capturing device 1.

[0028] Here, with reference to FIG. 2, the configuration of the distance image sensor 32 used as an image sensor in the distance image capturing device 1 will be described. FIG. 2 is a block diagram showing the schematic configuration of the image sensor (distance image sensor 32) used in the distance image capturing device 1 according to the present embodiment.

[0029] As shown in FIG. 2, the distance image sensor 32 includes, for example, a light receiving region 320 in which a plurality of pixels 321 are arranged, a control circuit 322, a vertical scanning circuit 323 having a distribution operation, a horizontal scanning circuit 324, and a pixel signal processing circuit 325.

[0030] The light receiving region 320 is a region in which a plurality of pixels 321 are arranged. In FIG. 2, an example of a two-dimensional matrix arrangement in 8 rows and 8 columns is shown. The pixel 321 accumulates charges corresponding to the amount of received light. The control circuit 322 comprehensively controls the distance image sensor 32. The control circuit 322 controls the operations of the components of the distance image sensor 32, for example, in response to an instruction from the timing control unit 41 of the distance image processing unit 4. Note that the control of the components provided in the distance image sensor 32 may be directly performed by the timing control unit 41. In this case, the control circuit 322 may be omitted.

[0031] The vertical scanning circuit 323 is a circuit that controls the pixels 321 arranged in the light receiving region 320 row by row in response to control from the control circuit 322. The vertical scanning circuit 323 causes the pixel signal processing circuit 325 to output a voltage signal corresponding to the amount of charge accumulated in each charge storage unit CS of the pixel 321. In this case, the vertical scanning circuit 323 distributes and accumulates the charges converted by the photoelectric conversion element in each charge storage unit of the pixel 321. That is, the vertical scanning circuit 323 is an example of a "pixel driving circuit".

[0032] The pixel signal processing circuit 325 is a circuit that performs predetermined signal processing (for example, noise suppression processing, A / D conversion processing, etc.) on the voltage signals output from the pixels 321 of each column to the corresponding vertical signal lines in response to control from the control circuit 322.

[0033] The horizontal scanning circuit 324 is a circuit that sequentially outputs the signals output from the pixel signal processing circuit 325 to the horizontal signal lines in accordance with the control from the control circuit 322. As a result, the pixel signals corresponding to the amount of charge accumulated for one frame are sequentially output to the distance image processing unit 4 via the horizontal signal lines.

[0034] Hereinafter, it will be described on the assumption that the pixel signal processing circuit 325 performs A / D conversion processing and the pixel signals are digital signals.

[0035] Here, with reference to FIG. 3, the configuration of the pixel 321 disposed in the light-receiving region 320 provided in the distance image sensor 32 will be described. FIG. 3 is a circuit diagram showing an example of the configuration of the pixel 321 disposed in the light-receiving region 320 of the distance image sensor 32 according to the present embodiment. FIG. 3 shows an example of the configuration of one pixel 321 among a plurality of pixels 321 disposed in the light-receiving region 320. The pixel 321 is an example of a configuration including four pixel signal readout units.

[0036] The pixel 321 includes one photoelectric conversion element PD, a drain gate transistor GD, and four pixel signal readout units RU that output voltage signals from the corresponding output terminals O. Each of the pixel signal readout units RU includes a readout gate transistor G, a floating diffusion FD, a charge storage capacitor C, a reset gate transistor RT, a source follower gate transistor SF, and a selection gate transistor SL. In each pixel signal readout unit RU, a charge storage unit CS is configured by the floating diffusion FD and the charge storage capacitor C.

[0037] In FIG. 3, the respective pixel signal readout units RU are distinguished by assigning numbers "1", "2", "3", or "4" after the symbol "RU" of the four pixel signal readout units RU. Similarly, each component provided in the four pixel signal readout units RU is also represented by indicating the number representing the respective pixel signal readout unit RU after the symbol, thereby distinguishing and representing the pixel signal readout unit RU to which each component corresponds.

[0038] In the pixel 321 shown in FIG. 3, the pixel signal reading unit RU1 that outputs a voltage signal from the output terminal O1 includes a read gate transistor G1, a floating diffusion FD1, a charge storage capacitor C1, a reset gate transistor RT1, a source follower gate transistor SF1, and a selection gate transistor SL1. In the pixel signal reading unit RU1, a charge storage unit CS1 is configured by the floating diffusion FD1 and the charge storage capacitor C1. The pixel signal reading units RU2 to RU4 have the same configuration.

[0039] The photoelectric conversion element PD is an embedded photodiode that photoelectrically converts incident light to generate charges and accumulates the generated charges. The structure of the photoelectric conversion element PD may be arbitrary. The photoelectric conversion element PD may be, for example, a PN photodiode having a structure in which a P-type semiconductor and an N-type semiconductor are joined, or a PIN photodiode having a structure in which an I-type semiconductor is sandwiched between a P-type semiconductor and an N-type semiconductor. Further, the photoelectric conversion element PD is not limited to a photodiode, and may be, for example, a photoelectric conversion element of a photogate method.

[0040] In the pixel 321, the charges generated by the photoelectric conversion element PD photoelectrically converting the incident light are distributed to each of the four charge storage units CS, and each voltage signal corresponding to the amount of the distributed charges is output to the pixel signal processing circuit 325.

[0041] The configuration of the pixels arranged in the distance image sensor 32 is not limited to the configuration including four pixel signal reading units RU as shown in FIG. 3, and any pixel having a configuration including a plurality of pixel signal reading units RU may be used. That is, the number of pixel signal reading units RU (charge storage units CS) provided in the pixels arranged in the distance image sensor 32 may be two, three, or five or more.

[0042] In addition, in the pixel 321 having the configuration shown in FIG. 3, an example in which the charge storage unit CS is configured by the floating diffusion FD and the charge storage capacitor C was shown. However, the charge storage unit CS only needs to be configured by at least the floating diffusion FD, and the pixel 321 may have a configuration without the charge storage capacitor C.

[0043] In addition, in the pixel 321 having the configuration shown in FIG. 3, an example of a configuration including the drain gate transistor GD was shown. However, when it is not necessary to discard the charges accumulated (remaining) in the photoelectric conversion element PD, a configuration without the drain gate transistor GD may be used.

[0044] In this embodiment, a plurality of sub-frames are provided in one frame. The plurality of sub-frames include a first sub-frame and a second sub-frame. The distance image processing unit 4 controls so that the charge storage timing for accumulating charges in each of the charge storage units CS included in the pixel 321 in the first sub-frame is earlier than the charge storage timing in the second sub-frame. Thereby, the distance to the subject OB relatively close in the first sub-frame can be calculated, and the distance to the subject OB relatively far in the second sub-frame can be calculated. Therefore, by adopting such a sub-frame configuration, it becomes possible to widen the measurable distance.

[0045] Also, in this embodiment, the number of accumulations in each sub-frame is different. It is known that the intensity of light is inversely proportional to the square of the distance. For example, when receiving the reflected light RL reflected by the subject OB that is relatively close in the first sub-frame, the intensity of the reflected light is large. On the other hand, when receiving the reflected light RL reflected by the subject OB that is relatively far in the second sub-frame, the intensity of the reflected light becomes small. For this reason, when the first sub-frame and the second sub-frame have the same number of accumulations, if the number of accumulations is set so that the amount of charge accumulated in the charge accumulation portion CS of the pixel 321 in the first sub-frame becomes an appropriate amount, the amount of charge accumulated in the charge accumulation portion CS of the pixel 321 in the second sub-frame will be reduced and become a factor causing an error. On the other hand, if the number of accumulations is set so that the amount of charge accumulated in the charge accumulation portion CS of the pixel 321 in the second sub-frame becomes an appropriate amount, the amount of charge accumulated in the charge accumulation portion CS of the pixel 321 in the first sub-frame will become too large and saturated, making it difficult to calculate the distance. As a countermeasure, in the embodiment, control is performed so that the number of accumulations in each sub-frame is different. Specifically, the distance image processing unit 4 controls so that the number of accumulations in the first sub-frame is smaller than the number of accumulations in the second sub-frame.

[0046] Here, the timing for driving the pixel 321 will be described with reference to FIGS. 4, 5, and 6. FIGS. 4, 5, and 6 are timing charts showing the timing for driving the pixel 321 according to this embodiment. FIG. 4 shows an example in which a first sub-frame and a second sub-frame are provided in one frame. That is, one frame includes a first sub-frame, a second sub-frame, and a read period.

[0047] In FIGS. 4, 5, and 6, the timing of irradiating the optical pulse PO is indicated by the item name "L", the timing of the drive signal TX1 is indicated by "G1", the timing of the drive signal TX2 is indicated by "G2", the timing of the drive signal TX3 is indicated by "G3", the timing of the drive signal TX4 is indicated by "G4", and the timing of the drive signal RSTD is indicated by "GD". Note that the drive signal TX1 is a signal for driving the read gate transistor G1. The same applies to the drive signals TX2 to TX4.

[0048] As shown in FIG. 4, the optical pulse PO is irradiated for the irradiation time To. First, the vertical scanning circuit 323 repeats a "first period" of accumulating charges in the charge storage portions CS1, CS2, CS3, and CS4 in the pixel 321 in the order of the "first sub-frame" in synchronization with the irradiation of the optical pulse PO for a predetermined number of accumulation times (first accumulation times). Next, the vertical scanning circuit 323 reads out, as a signal value, a voltage signal corresponding to the amount of charge accumulated in each pixel 321 during the "first sub-frame read period". Next, the vertical scanning circuit 323 repeats a "second period" of accumulating charges in the charge storage portions CS1, CS2, CS3, and CS4 in the pixel 321 in the order of the "second sub-frame" in synchronization with the irradiation of the optical pulse PO for a predetermined number of accumulation times (second accumulation times). Then, the vertical scanning circuit 323 reads out, as a signal value, a voltage signal corresponding to the amount of charge accumulated in each pixel 321 during the "second sub-frame read period".

[0049] Here, the timing at which the vertical scanning circuit 323 accumulates charges in the pixel 321 will be described.

[0050] First, the vertical scanning circuit 323 turns off the drain gate transistor GD of the pixel 321 corresponding to the first sub-frame and turns on the readout gate transistor G1 at the same timing as the timing for irradiating the optical pulse PO. After the accumulation time Ta has elapsed since the vertical scanning circuit 323 turned on the readout gate transistor G1, the vertical scanning circuit 323 turns off the readout gate transistor G1. As a result, the charges photoelectrically converted by the photoelectric conversion element PD while the readout gate transistor G1 is controlled to be in the on state are accumulated in the charge accumulation section CS1 via the readout gate transistor G1. Note that the accumulation time Ta is a period for accumulating charges in the charge accumulation section CS, and is not limited to a configuration that is the same as the gate transistor ON period (the period during which the gate transistor G is in the on state). For example, a gate transistor ON period may be provided after a short gap period (gate OFF period, that is, the period during which the gate transistor G is in the off state), and control may be performed such that the total period of the gap period and the gate transistor ON period becomes the accumulation time Ta.

[0051] Next, the vertical scanning circuit 323 turns on the readout gate transistor G2 for the accumulation time Ta at the timing when the readout gate transistor G1 is turned off. As a result, the charges photoelectrically converted by the photoelectric conversion element PD while the readout gate transistor G2 is controlled to be in the on state are accumulated in the charge accumulation section CS2 via the readout gate transistor G2.

[0052] Next, the vertical scanning circuit 323 turns on the readout gate transistor G3 at the timing when the accumulation of charges in the charge accumulation section CS2 is completed, and turns off the readout gate transistor G3 after the accumulation time Ta has elapsed. As a result, the charges photoelectrically converted by the photoelectric conversion element PD while the readout gate transistor G3 is controlled to be in the on state are accumulated in the charge accumulation section CS3 via the readout gate transistor G3.

[0053] Next, the vertical scanning circuit 323 turns on the read gate transistor G4 at the timing when the charging of the charge storage section CS3 is completed, and turns off the read gate transistor G4 after the accumulation time Ta has elapsed. As a result, the charges photoelectrically converted by the photoelectric conversion element PD while the read gate transistor G4 is controlled to be in the on state are accumulated in the charge storage section CS4 via the read gate transistor G4.

[0054] Next, the vertical scanning circuit 323 turns on the drain gate transistor GD at the timing when the charging of the charge storage section CS4 is completed to discharge the charges. As a result, the charges photoelectrically converted by the photoelectric conversion element PD are discarded via the drain gate transistor GD.

[0055] The vertical scanning circuit 323 repeats the above-described driving for a predetermined number of accumulations (first accumulation number) set in the first sub-frame.

[0056] After the charging of the pixels 321 by the first sub-frame driving is completed, the vertical scanning circuit 323 outputs a voltage signal corresponding to the amount of charge distributed to each charge storage section CS. Specifically, the vertical scanning circuit 323 turns on the selection gate transistor SL1 for a predetermined time to output, from the output terminal O1, a voltage signal corresponding to the amount of charge accumulated in the charge storage section CS1 via the pixel signal reading section RU1. Similarly, the vertical scanning circuit 323 sequentially turns on the selection gate transistors SL2 and SL3 to output, from the output terminals O2 and O3, voltage signals corresponding to the amounts of charge accumulated in the charge storage sections CS2 and CS3. Then, voltage signals corresponding to the amounts of charge accumulated in each of the charge storage sections CS are output to the distance calculation section 42 as signal values via the pixel signal processing circuit 325 and the horizontal scanning circuit 324.

[0057] Next, the vertical scanning circuit 323 turns off the drain gate transistor GD of the pixel 321 corresponding to the second sub-frame at a timing delayed by a predetermined delay time (accumulation time Ta × 3) from the timing of irradiating the optical pulse PO, and turns on the readout gate transistor G1. The operations after turning on the readout gate transistor G1, that is, the timings at which the vertical scanning circuit 323 turns on or off each of the readout gate transistors G1 to G4 and the drain gate transistor GD are the same as those in the case of the first sub-frame, so the description thereof is omitted. The vertical scanning circuit 323 repeats the above-described driving for a predetermined number of accumulations (second accumulation number) set in the second sub-frame.

[0058] After the charge accumulation in the pixel 321 by the second sub-frame driving is completed, the vertical scanning circuit 323 outputs a voltage signal corresponding to the amount of charge distributed to each charge accumulation unit CS. Specifically, the vertical scanning circuit 323 turns on the selection gate transistor SL1 for a predetermined time, and outputs a voltage signal corresponding to the amount of charge accumulated in the charge accumulation unit CS1 from the output terminal O1 via the pixel signal readout unit RU1. Similarly, the vertical scanning circuit 323 sequentially turns on the selection gate transistors SL2 and SL3, and outputs voltage signals corresponding to the amounts of charge accumulated in the charge accumulation units CS2 and CS3 from the output terminals O2 and O3. Then, voltage signals corresponding to the amounts of charge accumulated in each of the charge accumulation units CS are output as signal values to the distance calculation unit 42 via the pixel signal processing circuit 325 and the horizontal scanning circuit 324.

[0059] As shown in FIG. 4, in the present embodiment, a plurality of sub-frames are provided in one frame. In the example of FIG. 4, a first sub-frame and a second sub-frame are provided in one frame. In the first sub-frame, charges are accumulated in each of the charge storage parts CS included in the pixel 321 at a timing earlier than that in the second sub-frame. In the second sub-frame, charges are accumulated in each of the charge storage parts CS included in the pixel 321 at a timing later than that in the first sub-frame. The distance image processing unit 4 makes the accumulation timing of the pixel 321 in the second sub-frame lag behind the accumulation timing of the pixel 321 in the first sub-frame by a predetermined delay time (accumulation time Ta × 3) with respect to the timing of irradiating the optical pulse PO.

[0060] FIG. 5 shows the timing for turning on or off each of the read gate transistors G1 to G4 and the drain gate transistor GD of the first sub-frame and the second sub-frame with respect to the timing of irradiating the optical pulse PO.

[0061] As shown in FIG. 5, in the present embodiment, the opening / closing timing (reference numeral Tr1) of the gate transistor G4 in the first sub-frame and the opening / closing timing (reference numeral Tr2) of the gate transistor G1 in the second sub-frame are controlled to be the same timing with respect to the timing of irradiating the optical pulse PO. Here, the opening / closing timing (reference numeral Tr1) of the gate transistor G4 in the first sub-frame is the latest accumulation timing in the first sub-frame and is an example of the "last accumulation timing". Also, the opening / closing timing (reference numeral Tr2) of the gate transistor G1 in the second sub-frame is the earliest accumulation timing in the second sub-frame and is an example of the "first accumulation timing".

[0062] As shown in FIG. 5, the distance image processing unit 4 measures a part of the space (rangable range) of the measurement target in each sub-frame. Such distance measurement by sub-frame driving is called "sub-frame distance measurement". Sub-frame distance measurement requires a plurality of sub-frames when measuring the entire space of the measurement target, so the number of light source irradiations is large and the power consumption is relatively high. However, since the distance is calculated from the ratio of the charges accumulated in a relatively short distance range, it can be expected to measure the distance with a relatively fine distance resolution. A method of measuring the space of the measurement target by combining these sub-frame distance measurements is called "precision distance measurement mode". Note that the number of sub-frames in one frame is not limited to two.

[0063] Next, the case where the distance image processing unit 4 measures the space (rangable range) of the measurement target in one frame will be described. FIG. 6 shows the timing of turning on or off each of the read gate transistors G1 to G4 and the drain gate transistor GD when measuring the space of the measurement target in a plurality of sub-frames in one frame with reference to the timing of irradiating the optical pulse PO.

[0064] As shown in FIG. 6, when the distance image processing unit 4 measures the space (rangable range) of the measurement target in one frame, it extends the pulse width of the optical pulse PO to a length twice that of the irradiation time T0 (2 × T0), and also extends each gate transistor to a length twice that of the accumulation time Ta (2 × Ta) accordingly. Here, the reason for doubling the respective times T0 and Ta is to measure the space of the measurement target in one frame, and it is not necessarily limited to doubling. Thereby, the space of the measurement target in a plurality of sub-frames can be acquired in one frame. This method is called "one-frame distance measurement". Compared with the precision distance measurement mode, one-frame distance measurement calculates the distance from the ratio of the charges accumulated in a relatively long distance, so it can measure the space of the measurement target in one frame, resulting in fewer light source irradiations and relatively low power consumption. However, it can be expected to measure the distance with a relatively coarse distance resolution. A method of measuring the space of the measurement target by one-frame distance measurement is called "scan distance measurement mode".

[0065] Next, the precision distance measurement mode will be described with reference to FIG. 7. FIG. 7 is a diagram showing a first example of the precision distance measurement mode according to the present embodiment. When measuring the space (distance measurable range) of the measurement target using three sub-frames as shown in FIG. 7, the distance image processing unit 4 divides the space of the measurement target into three distance measurement ranges (Zone1 to Zone3) in the depth direction. The distance image processing unit 4 obtains all the distance information of the space of the measurement target by combining the distance information of the three distance measurement ranges (Zone1 to Zone3) obtained by each of the sub-frame distance measurements in the three sub-frames (sf1 to sf3).

[0066] FIG. 8 is a diagram showing a second example of the precision distance measurement mode according to the present embodiment. When limited to two of the three distance measurement ranges (Zone1 and Zone2) within the space of the measurement target as shown in FIG. 8, the distance image processing unit 4 may stop or skip the sub-frame distance measurement in the sub-frame (sf3) responsible for obtaining the distance information of the distance measurement range (Zone3) outside the measurement target within the space of the measurement target. Also, when another Zone becomes a distance measurement range outside the measurement target, the distance image processing unit 4 may similarly stop or skip the sub-frame distance measurement responsible for obtaining the distance information of the distance measurement range outside the measurement target.

[0067] Next, the scan distance measurement mode will be described with reference to FIG. 9. FIG. 9 is a diagram showing an example of the scan distance measurement mode according to the present embodiment. As shown in FIG. 9, in the scan distance measurement mode, the distance image processing unit 4 measures the entire space (distance measurable range) of the measurement target in one frame (one-frame distance measurement). Note that when the short distance saturates or the signal of the long distance cannot be obtained sufficiently, the distance image processing unit 4 may combine the sub-frame distance measurement for the short distance (in this figure, the distance measurement range Zone1 is sub-frame distance measured) and the one-frame distance measurement to perform distance measurement in the scan distance measurement mode while suppressing the saturation of the short distance.

[0068] Note that, in addition to, for example, a surface-emitting semiconductor laser module, the light source unit 2 may be equipped with a point-emitting semiconductor laser module (for example, mounted in parallel). At this time, the point-emitting semiconductor laser module may be mounted as a dedicated light source for the scanning distance measurement mode.

[0069] Next, an example of the distance measurement control process in which the distance image processing unit 4 switches between the scanning distance measurement mode and the precise distance measurement mode to perform distance measurement will be described.

[0070] (First Example of Distance Measurement Control Process) FIG. 10 is a diagram showing a first example of the distance measurement control process according to the present embodiment. As shown in FIG. 10, normally (for example, by default), the distance image processing unit 4 continuously measures the distance of the space to be measured in the scanning distance measurement mode. For example, when there is a change in the distance information during the scanning distance measurement mode, that is, when there is a subject OB to be measured in the space to be measured, the distance image processing unit 4 shifts to the precise distance measurement mode triggered by the change in the distance information. The trigger for shifting to this precise distance measurement mode is not limited to a change in the distance information, but also includes a change in the IR image or a command from an external signal. Hereinafter, the occurrence of a trigger when shifting to the precise distance measurement mode due to a change in the distance information, the presence of a subject OB to be measured, or a command from an external signal is referred to as "event occurrence". That is, when an "event occurs" in the scanning distance measurement mode, the distance image processing unit 4 shifts to the precise distance measurement mode. Note that when an "event occurs" in the precise distance measurement mode, the distance image processing unit 4 continues the precise distance measurement mode.

[0071] Here, the change in the IR image refers to a change in the pixel signal (before calculating the distance) output from the distance image sensor 32. The external signal is a trigger signal for shifting to the precise distance measurement mode from a switch, sound, external device, or the like. Hereinafter, the occurrence of a trigger when shifting to the precise distance measurement mode due to a change in the distance information, the presence of a subject OB to be measured, or a command from an external signal is referred to as "event occurrence". That is, when an "event occurs" in the scanning distance measurement mode, the distance image processing unit 4 shifts to the precise distance measurement mode. Note that when an "event occurs" in the precise distance measurement mode, the distance image processing unit 4 continues the precise distance measurement mode.

[0072] After the distance image processing unit 4 has shifted to the precise distance measurement mode and there is no longer any change in the distance information, that is, when there is no longer a subject OB to be measured in the space of the measurement target, the distance image processing unit 4 uses the fact that there is no change in the distance information as a trigger to shift to the scan distance measurement mode. Similarly, the trigger for shifting to the scan distance measurement mode is not limited to the absence of a change in the distance information, but also includes the absence of a change in the IR image or the absence of a command from an external signal. That is, when the distance image processing unit 4 determines that "no event has occurred" in the precise distance measurement mode, it shifts to the scan distance measurement mode. In addition, when the distance image processing unit 4 determines that "no event has occurred" in the scan distance measurement mode, it continues the scan distance measurement mode.

[0073] Next, with reference to FIG. 11, a specific drive flow of the distance image capturing device 1 and a method for creating distance information for the first example of the distance measurement control process will be described. FIG. 11 is a schematic diagram showing a drive example of the first example of the distance measurement control process according to the present embodiment. In FIG. 11, the horizontal axis represents time, and the processing from the first frame to the fifth frame is shown in order from left to right.

[0074] At the start of driving (first frame), it is set to the normal (default) scan distance measurement mode, and the distance image capturing device 1 monitors the space of the measurement target by one-frame distance measurement. Since the distance image capturing device 1 can obtain all the distance information of the space of the measurement target by one-frame distance measurement, the distance information of all Zones can be created from the distance measurement results by one-frame distance measurement.

[0075] Based on the distance measurement results and the external signal, when the distance image capturing device 1 determines that "no event has occurred" in all Zones of the space of the measurement target, it continues to monitor the space of the measurement target by one-frame distance measurement in the next frame (second frame). On the other hand, as shown in FIG. 11, when the distance image capturing device 1 determines that "an event has occurred" in a certain Zone (Zone 2 in the figure) of the space of the measurement target based on the distance measurement results by one-frame distance measurement and the external signal in the second frame, it shifts to the precise distance measurement mode by sub-frame distance measurement in the next frame (third frame).

[0076] After switching to the precise distance measurement mode, the distance image capturing device 1 performs sub-frame distance measurement driving in all Zones from Zone1 to Zone3 to obtain distance information of the space of the measurement target. As a result, detailed distance information of the subject OB can be obtained. In the precise distance measurement mode, the distance image capturing device 1 continues to monitor whether the subject OB exists in the space of the measurement target for each Zone by sub-frame distance measurement.

[0077] As shown in FIG. 11, when the distance image capturing device 1 determines that "event occurs" in a certain Zone (Zone1 and Zone2 in the figure) of the space of the measurement target based on the distance measurement result by sub-frame distance measurement or an external signal in the third frame, the distance image capturing device 1 continues to monitor the space of the measurement target by sub-frame distance measurement in the next frame (the fourth frame).

[0078] On the other hand, as shown in FIG. 11, when the distance image capturing device 1 determines that "no event occurs" in all Zones of the space of the measurement target in the fourth frame, the distance image capturing device 1 switches to the scan distance measurement mode by one-frame distance measurement. Thereafter, the previous driving flow is repeated.

[0079] Next, with reference to the flowchart shown in FIG. 12, the operation of the distance measurement control process by the distance image capturing device 1 described with reference to FIG. 11 will be described. FIG. 12 is a flowchart showing an example of the distance measurement control process according to the present embodiment.

[0080] (Step S10) When the distance image capturing device 1 starts shooting (for example, shooting an IR image for distance measurement), it proceeds to step S11. (Step S11) The distance image capturing device 1 performs one-frame distance measurement on Zones 1 to 3 of the space of the measurement target in the scan distance measurement mode. Then, it proceeds to step S12. (Step S12) The distance image capturing device 1 outputs the distance measurement result of the space of the measurement target obtained in step S11. Then, it proceeds to step S13.

[0081] (Step S13) The distance image capturing device 1 performs distance measurement for Zones 1 to 3 of the space of the measurement target in one frame in the scan distance measurement mode. Then, it proceeds to Step S14. (Step S14) The distance image capturing device 1 outputs the distance measurement result of the space of the measurement target obtained in Step S13. Then, it proceeds to Step S15. (Step S15) The distance image capturing device 1 compares the distance measurement results between the previously output two frames. For example, the distance image capturing device 1 compares the distance information, the results of the pixel values (IR values) of the IR image, etc.

[0082] (Step S16) The distance image capturing device 1 determines whether or not a subject OB (measurement target object) exists in the space of the measurement target based on the comparison result in Step S15. For example, when the distance image capturing device 1 determines that there is a change in the distance information between the two frames in Step S15 and a subject OB (measurement target object) exists in the space of the measurement target (YES), it proceeds to Step S17 and performs sub-frame distance measurement in the precise distance measurement mode. Note that the distance image capturing device 1 also proceeds to Step S17 and performs sub-frame distance measurement in the precise distance measurement mode when there is a request from an external signal (request to shift to the precise distance measurement mode). That is, when the distance image capturing device 1 "detects an event" as described with reference to FIG. 10, it proceeds to Step S17 and performs sub-frame distance measurement in the precise distance measurement mode.

[0083] On the other hand, in Step S16, when the distance image capturing device 1 determines that there is no change in the distance information between the two frames in Step S15 and a subject OB (measurement target object) does not exist in the space of the measurement target (YES), it returns to Step S13 and performs one-frame distance measurement in the scan distance measurement mode.

[0084] (Step S17) The distance image capturing device 1 performs sub-frame distance measurement for each of Zones 1 to 3 of the space of the measurement target in the precise distance measurement mode. Then, the distance image capturing device 1 outputs the distance measurement results for each of Zones 1 to 3 in Step S14.

[0085] Thereafter, the distance image capturing device 1 repeats steps S13 to S17 according to whether or not a subject OB (measurement target object) exists in the space of the measurement target (whether or not an event has occurred) by comparing the distance measurement results between two frames, and performs distance measurement driving. That is, when the distance image capturing device 1 determines that a subject OB (measurement target object) exists in the space of the measurement target (an event has occurred), it performs sub-frame distance measurement in the precise distance measurement mode, and when it determines that the subject OB (measurement target object) does not exist (no event has occurred), it performs one-frame distance measurement in the scan distance measurement mode.

[0086] (Second Example of Distance Measurement Control Processing) Next, a second example of distance measurement control processing for performing distance measurement by switching between the scan distance measurement mode and the precise distance measurement mode will be described. FIG. 13 is a diagram showing a second example of distance measurement control processing according to the present embodiment. As shown in FIG. 13, normally (for example, by default), the distance image processing unit 4 continuously measures the distance of the space of the measurement target in the scan distance measurement mode. For example, when there is a change in the distance information in a Zone during the scan distance measurement mode, that is, when a subject OB serving as a measurement target object exists in a Zone that is a part of the measurement target, the distance image processing unit 4 shifts to the precise distance measurement mode triggered by the change in the distance information. The trigger for shifting to this precise distance measurement mode is not limited to a change in the distance information, and also includes a change in the IR image or a command from an external signal. That is, when an "event occurs" in the scan distance measurement mode, the distance image processing unit 4 shifts to the precise distance measurement mode. Note that when an "event occurs" in the precise distance measurement mode, the distance image processing unit 4 continues the precise distance measurement mode.

[0087] On the other hand, for a Zone where there is no change in the distance information, that is, a Zone that is a part of the measurement target where the subject OB to be measured does not exist, the sub-frame distance measurement is stopped or skipped by the distance image processing unit 4. When the precision distance measurement mode has ended for one cycle for all Zones in the space of the measurement target, the distance image processing unit 4 shifts to the scan distance measurement mode and determines the presence or absence of the subject OB again. In the second example of the distance measurement control process shown in FIG. 13, since it is possible to select to perform, stop, or skip sub-frame distance measurement for each Zone in the precision distance measurement mode, a further power consumption reduction effect can be expected compared to the first example described above. Furthermore, by alternately switching between the scan distance measurement mode and the precision distance measurement mode, it is possible to prevent the measurement target from being missed.

[0088] Next, with reference to FIG. 14, a specific drive flow and a method for creating distance information when the distance image capturing device 1 stops sub-frame distance measurement in the precision distance measurement mode in the second example of the distance measurement control process will be described. FIG. 14 is a schematic diagram showing a drive example in which sub-frame distance measurement is stopped in the precision distance measurement mode in the second example of the distance measurement control process according to the present embodiment. In FIG. 14, the horizontal axis represents time, and the processing from the first frame to the fifth frame is shown in order from left to right.

[0089] At the start of driving (the first frame), it is set to the normal (default) scan distance measurement mode, and the distance image capturing device 1 monitors the space of the measurement target by one-frame distance measurement. Since the distance image capturing device 1 can obtain all the distance information of the space of the measurement target by one-frame distance measurement, the distance information of all Zones can be created from the distance measurement results by one-frame distance measurement.

[0090] When the distance image capturing device 1 determines that "no event occurs" in all Zones of the space of the measurement target based on the distance measurement result and the external signal, it continues to monitor the space of the measurement target by one-frame distance measurement in the next frame (the second frame). On the other hand, as shown in FIG. 14, when the distance image capturing device 1 determines that "an event occurs" in a certain Zone (Zone 2 in the figure) of the space of the measurement target based on the distance measurement result by one-frame distance measurement and the external signal in the second frame, it shifts to the precise distance measurement mode by sub-frame distance measurement in the next frame (the third frame).

[0091] In the third frame after shifting to the precise distance measurement mode, the distance image capturing device 1 performs sub-frame distance measurement drive only in the Zone (Zone 2 in the figure) where the subject OB to be the measurement target is determined to exist, and acquires the distance information of the space of the measurement target. On the other hand, the distance image capturing device 1 stops sub-frame distance measurement outside the Zone where the subject OB to be the measurement target is determined to exist. Thereby, detailed distance information can be acquired in the Zone where the subject OB exists. When one cycle (Zone 1 to Zone 3) of the precise distance measurement mode ends, the distance image capturing device 1 shifts to the scan distance measurement mode, and in the next frame (the fourth frame), it performs distance measurement and monitoring of the entire area again in the scan distance measurement mode.

[0092] As shown in FIG. 14, when the distance image capturing device 1 determines that "an event occurs" in a certain Zone (Zone 1 and Zone 2 in the figure) of the space of the measurement target based on the distance measurement result by one-frame distance measurement and the external signal in the fourth frame, it shifts to the precise distance measurement mode by sub-frame distance measurement in the next frame (the fifth frame).

[0093] In the fifth frame after transitioning to the precise distance measurement mode, the distance image capturing device 1 performs sub-frame distance measurement drive only in the Zones (in the illustration, Zone 1 and Zone 2) where it is determined that the subject OB exists, and acquires distance information of the space to be measured. On the other hand, the distance image capturing device 1 stops sub-frame distance measurement outside the Zones where it is determined that the subject OB serving as the measurement target exists. As a result, detailed distance information can be obtained in the Zones where the subject OB exists. When one cycle (Zone 1 to Zone 3) of the precise distance measurement mode ends, the distance image capturing device 1 transitions to the scan distance measurement mode, and in the next frame, performs distance measurement and monitoring of the entire area again in the scan distance measurement mode.

[0094] The distance image capturing device 1 repeats this scan distance measurement mode and precise distance measurement mode until the subject OB disappears. As a result, the distance image capturing device 1 can accurately obtain distance information only for the area where the subject OB exists while measuring the entire area. Also, when stopping sub-frame distance measurement in the precise distance measurement mode, the frame rate does not change between the scan distance measurement mode and the precise distance measurement mode. In the precise distance measurement mode, since sub-frame distance measurement is performed only in the Zones where the subject OB exists, the number of light emissions of the light source unit 2 decreases, and an effect of suppressing power consumption can be expected.

[0095] Next, with reference to FIG. 15, a method for generating a distance image when sub-frame distance measurement is stopped in the precise distance measurement mode will be described. FIG. 15 is a schematic diagram showing a method for generating a distance image in a drive example of a second example of the distance measurement control process according to the present embodiment. In this FIG. 15, the drive of the distance measurement control process is as shown in FIG. 14, and the method for generating a distance image is additionally shown in the figure.

[0096] In the precise distance measurement mode, the distance information of the Zones where the subject OB serving as the measurement target does not exist is missing, and the distance information at that frame time is in a state of being absent. Therefore, the distance image capturing device 1 generates distance images of all the spaces to be measured by diverting the distance information measured in the scan distance measurement mode in the immediately preceding frame for the Zones where it is determined that the subject OB does not exist.

[0097] On the other hand, similarly in the scan ranging mode, when it is determined that the subject OB exists and distance information measured in the precise ranging mode exists in the immediately preceding frame, the distance image capturing apparatus 1 may generate a distance image in the scan ranging mode as a more accurate distance image by diverting the distance information.

[0098] Next, with reference to FIG. 16, a specific drive flow and a method for creating distance information when the distance image capturing apparatus 1 skips sub-frame ranging in the precise ranging mode in the second example of the ranging control process will be described. FIG. 16 is a schematic diagram showing a drive example in which sub-frame ranging is skipped in the precise ranging mode in the second example of the ranging control process according to the present embodiment. In FIG. 16, the horizontal axis represents time, and the processing from the first frame to the fifth frame is shown in order from left to right. Although the occurrence status of the events in FIG. 16 is not shown, similar to FIG. 14, it is assumed that "event occurs" in Zone 2 of the second frame and Zones 1 and 2 of the fourth frame.

[0099] At the start of driving (first frame), it is set to the normal (default) scan ranging mode, and the distance image capturing apparatus 1 monitors the space of the measurement target by one-frame ranging. Since the distance image capturing apparatus 1 can obtain all the distance information of the space of the measurement target by one-frame ranging, the distance information of all Zones can be created from the ranging result by one-frame ranging.

[0100] When the distance image capturing apparatus 1 determines that "no event occurs" in all Zones of the space of the measurement target based on the ranging result and the external signal, it continues to monitor the space of the measurement target by one-frame ranging in the next frame (second frame). On the other hand, as shown in FIG. 16, when the distance image capturing apparatus 1 determines that "an event occurs" in a certain Zone (Zone 2 in the figure) of the space of the measurement target based on the ranging result by one-frame ranging and the external signal in the second frame, it shifts to the precise ranging mode by sub-frame ranging in the next frame (third frame).

[0101] In the third frame after transitioning to the precise distance measurement mode, the distance image capturing device 1 skips the sub-frame distance measurement of Zones other than the Zone (Zone2 in the figure) where it is determined that the subject OB, which is the object to be measured, exists, and instead performs sub-frame distance measurement of the Zone (Zone2 in the figure) where the subject OB exists. That is, the distance image capturing device 1 performs sub-frame distance measurement drive of the Zone (Zone2 in the figure) where it is determined that the subject OB exists three times, and acquires distance information each time. As a result, detailed distance information can be obtained in the Zone (Zone2 in the figure) where the subject OB exists. At this time, the distance information of the skipped Zone is missing, and there is no distance information at that frame time. Therefore, the distance image capturing device 1 generates distance images of all spaces to be measured by diverting the distance information measured in the scan distance measurement mode in the immediately preceding frame for the skipped Zone.

[0102] When the precise distance measurement mode ends one cycle (distance measurement of Zone2 three times), the distance image capturing device 1 transitions to the scan distance measurement mode, and in the next frame (the fourth frame), it performs distance measurement and monitoring of the entire area again in the scan distance measurement mode. In the scan distance measurement mode, as described with reference to FIG. 15, when it is determined that the subject OB exists and there is distance information measured in the precise distance measurement mode in the immediately preceding frame, the distance image capturing device 1 may divert the distance information and generate a distance image in the scan distance measurement mode as a more accurate distance image.

[0103] As shown in FIG. 16, when the distance image capturing device 1 determines that "event occurs" in a certain Zone (Zone1 and Zone2 in the figure) of the space to be measured based on the distance measurement result by one-frame distance measurement and an external signal in the fourth frame, it shifts to the precise distance measurement mode by sub-frame distance measurement in the next frame (the fifth frame). Thus, when "event occurs" in two Zones (Zone1 and Zone2 in the figure), similar to the fifth frame shown in FIGS. 14 and 15, the distance image capturing device 1 performs sub-frame distance measurement drive only in the Zones (Zone1 and Zone2 in the figure) where the subject OB is determined to exist, and may stop sub-frame distance measurement outside the Zones where the subject OB is determined to exist.

[0104] The distance image capturing device 1 repeats this scan distance measurement mode and precise distance measurement mode until the subject OB disappears. Thereby, the distance image capturing device 1 can accurately obtain distance information only for the area where the subject OB exists while measuring the entire area.

[0105] (The third example of distance measurement control processing) Next, a third example of the distance measurement control processing in which the distance image processing unit 4 switches between the scan distance measurement mode and the precise distance measurement mode to perform distance measurement will be described. With reference to FIG. 17, the specific drive flow of the distance image capturing device 1 and the method of creating distance information for the third example of the distance measurement control processing will be described.

[0106] FIG. 17 is a schematic diagram showing a drive example of the third example of the distance measurement control processing according to the present embodiment. In FIG. 17, the horizontal axis represents time, and the processing from the first frame to the fifth frame is shown in order from left to right. The drive flow and distance information from the first frame to the fifth frame shown in this FIG. 17 are basically the same as the example shown in FIG. 11, but the measurement range (distance measurement range) of each sub-frame distance measurement responsible for distance measurement of each Zone in the precise distance measurement mode is different in that it is not limited to each Zone.

[0107] Here, the precise distance measurement mode in the third frame will be described as an example. As shown in FIG. 17, the measurement ranges of the sub-frame distance measurements can be overlapped with those of adjacent sub-frame distance measurements. For example, as shown in FIG. 17, assuming that the depth direction of the measurable range is 21 m, and it is divided into Zone1 from 1 to 7 m, Zone2 from 7 to 14 m, and Zone3 from 14 to 21 m. At this time, the measurement range of the sub-frame distance measurement in the sub-frame (sf1) responsible for Zone1 is set to, for example, 1 to 8.5 m, and overlapped with the measurement range of the sub-frame distance measurement in the sub-frame (sf2). Also, the measurement range of the sub-frame distance measurement in the sub-frame (sf2) responsible for Zone2 is set to, for example, 5.5 to 15.5 m, and overlapped with the measurement range of the sub-frame distance measurement in the sub-frame (sf1) and the measurement range of the sub-frame distance measurement in the sub-frame (sf3). Further, the measurement range of the sub-frame distance measurement in the sub-frame (sf3) responsible for Zone3 is set to, for example, 12.5 to 21 m, and overlapped with the measurement range of the sub-frame distance measurement in the sub-frame (sf2).

[0108] Thereby, the variation in the distance measurement results for each sub-frame can be corrected, so that the distance measurement results obtained by each sub-frame distance measurement can be smoothly combined, and the distance deviation at the Zone end can be reduced.

[0109] As described above, the distance image capturing device 1 according to the present embodiment measures the distance to the subject OB based on the time until the light emitted from the light source is reflected by the subject OB (an example of the object) and returns. The distance image capturing device 1 has a scan distance measurement mode (an example of the first distance measurement mode) for measuring the distance within at least a part of the measurable range in one frame, and a precise distance measurement mode (an example of the second distance measurement mode) for dividing the measurable range into a plurality of Zones (distance measurement ranges) in the depth direction, and measuring at least one of the divided distance measurement ranges with at least one of the sub-frames obtained by dividing one frame according to the divided distance measurement ranges. Further, the distance image capturing device 1 includes a distance image processing unit 4 (an example of the control unit) for switching between the scan distance measurement mode and the precise distance measurement mode.

[0110] As a result, the distance image capturing device 1 can switch between the scan distance measurement mode for measuring distance in one frame and the sub-frame distance measurement for measuring distance using sub-frame driving as needed. Therefore, power consumption can be reduced while maintaining distance measurement accuracy equivalent to that of sub-frame distance measurement.

[0111] For example, in the scan distance measurement mode, the entire range in the depth direction of the measurable distance range is measured in one frame.

[0112] As a result, in the normal state (for example, default), the distance image capturing device 1 measures the measurable distance range in one frame, so power consumption can be reduced compared to sub-frame distance measurement.

[0113] For example, in the precise distance measurement mode, one frame is divided into the number of sub-frames corresponding to the number of divided distance measurement ranges in the depth direction of the measurable distance range, and distance measurement is performed for each corresponding sub-frame for each divided distance measurement range.

[0114] As a result, the distance image capturing device 1 can measure distance using sub-frames for each divided distance measurement range in the depth direction of the measurable distance range, so the distance measurement accuracy can be improved compared to one-frame distance measurement.

[0115] Also, the precise distance measurement mode has higher power consumption and higher distance measurement accuracy in the depth direction of the measurable distance range compared to the scan distance measurement mode.

[0116] As a result, when the distance image capturing device 1 prioritizes distance measurement accuracy despite high power consumption, it selects the precise distance measurement mode to perform sub-frame distance measurement. When less distance measurement accuracy is required, it selects the scan distance measurement mode for one-frame distance measurement to reduce power consumption.

[0117] Also, the scan distance measurement mode and the precise distance measurement mode have different frame rates.

[0118] As a result, in the precision distance measurement mode, the distance image capturing device 1 can increase the frame rate with respect to the scan distance measurement mode to improve the distance measurement accuracy.

[0119] In addition, the scan distance measurement mode and the precision distance measurement mode have different distance measurement ranges.

[0120] As a result, the distance image capturing device 1 can, for example, switch from the scan distance measurement mode to the precision distance measurement mode, enabling distance measurement drive only in the limited range where the measurement object exists, thereby suppressing power consumption.

[0121] In addition, the distance image processing unit 4 switches between the scan distance measurement mode and the precision distance measurement mode based on the detection of a predetermined event within the measurable distance range. For example, the predetermined event may be that the distance information changes, that a subject OB to be measured exists, or that there is a command from an external signal.

[0122] As a result, when the measurement object exists, the distance image capturing device 1 can measure the distance to the measurement object with high accuracy in the precision distance measurement mode. On the other hand, when the measurement object does not exist, the distance image capturing device 1 can reduce power consumption by setting it to the scan distance measurement mode.

[0123] In addition, the distance image processing unit 4 switches between the scan distance measurement mode and the precision distance measurement mode based on at least one external signal.

[0124] As a result, the distance image capturing device 1 can switch between performing distance measurement with high distance measurement accuracy or performing distance measurement while suppressing power consumption according to a user's instruction or the like.

[0125] In addition, in the precision distance measurement mode, distance measurement is stopped or skipped in some of the sub - frames obtained by dividing one frame (see, for example, the examples shown in FIGS. 13 to 16).

[0126] As a result, even in the precise distance measurement mode, the distance image capturing device 1 can expect an effect of suppressing power consumption by stopping or skipping distance measurement in some sub-frames.

[0127] For example, as the distance information of the sub-frame that has been stopped or skipped in the precise distance measurement mode, the distance information obtained in the scan distance measurement mode is used.

[0128] As a result, even when the distance measurement is stopped or skipped in some sub-frames in the precise distance measurement mode, the distance image capturing device 1 can interpolate with the distance information obtained in the scan distance measurement mode.

[0129] Also, the distance information obtained in the precise distance measurement mode may be used as the distance information in the scan distance measurement mode.

[0130] As a result, the distance image capturing device 1 can improve the distance measurement accuracy of a part of the distance measurement range even in the scan distance measurement mode by using the distance information obtained in the precise distance measurement mode.

[0131] Also, in the precise distance measurement mode, the distance image processing unit 4 may continuously measure the same distance measurement range in a plurality of sub-frames obtained by dividing one frame (see, for example, the example shown in FIG. 16).

[0132] As a result, the distance image capturing device 1 can continuously measure the distance measurement range where the measurement object exists a plurality of times within one frame, so that the distance measurement accuracy can be improved.

[0133] Also, in the precise distance measurement mode, a part of the distance measurement ranges of a plurality of sub-frames obtained by dividing one frame may overlap.

[0134] As a result, the distance image capturing device 1 can correct the variation in the distance measurement results for each sub-frame, so that it becomes possible to smoothly combine the distance measurement results obtained by each sub-frame distance measurement, and the distance deviation at the Zone end can be reduced.

[0135] In addition, the scan distance measurement mode and the precise distance measurement mode may differ in the intensity of the irradiation light or the irradiation profile. The irradiation profile refers to the specifications of the light source unit 2 (for example, surface emission type, point emission type, type of light source, irradiation angle, etc.).

[0136] Thereby, the distance image capturing device 1 can irradiate light suitable for each distance measurement method of the scan distance measurement mode and the precise distance measurement mode.

[0137] Moreover, in the distance image capturing method performed by the distance image capturing device 1 that measures the distance to the subject OB (an example of an object) based on the time until the light irradiated from the light source is reflected back by the subject OB, there is a scan distance measurement mode (an example of a first distance measurement mode) that measures the distance of at least a part of the measurable range in one frame, and the measurable range is divided into a plurality of Zones (distance measurement ranges) in the depth direction, and at least one of the divided sub-frames obtained by dividing one frame according to the divided distance measurement ranges is used to measure at least one of the divided distance measurement ranges. There is a precise distance measurement mode (an example of a second distance measurement mode), and the distance image processing unit 4 (an example of a control unit) switches between the scan distance measurement mode and the precise distance measurement mode.

[0138] Thereby, the distance image capturing method performed by the distance image capturing device 1 can switch between the scan distance measurement mode that measures the distance in one frame and the sub-frame distance measurement that measures the distance using sub-frame driving as needed. Therefore, the power consumption can be reduced while maintaining the distance measurement accuracy equivalent to that of the sub-frame distance measurement.

[0139] In addition, in this embodiment, the description has been made only with respect to the combination of Zone1, Zone2, and Zone3, but the present invention is not limited thereto, and the combination of Zone(n) and Zone(n + 1) is applicable. Here, n is an arbitrary integer. For example, if there are Zone4 and Zone5 following Zone1, Zone2, and Zone3, it is possible to drive in the same manner as in this embodiment by performing one-frame distance measurement and sub-frame distance measurement for five Zones. Further, the number of sub-frames obtained by dividing one frame can also be an arbitrary number.

[0140] All or part of the distance image capturing device 1 and the distance image processing unit 4 in the above-described embodiment may be realized by a computer. In that case, a program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed to realize it. Here, the “computer system” shall include hardware such as an OS and peripheral devices. Further, the “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a storage device such as a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” also includes, like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line, a medium that dynamically holds a program for a short time, and also includes, like a volatile memory inside a computer system serving as a server or a client in that case, a medium that holds a program for a certain period of time. Also, the above program may be for realizing a part of the above-described functions, and may further be realized in combination with a program already recorded in a computer system for realizing the above-described functions, or may be realized using a programmable logic device such as an FPGA.

[0141] As described above in detail with reference to the drawings for the embodiments of the present invention, the specific configuration is not limited to this embodiment, and designs and the like within the scope not departing from the gist of the present invention are also included.

Explanation of Reference Numerals

[0142] 1…Distance image capturing device 2…Light source unit 21…Light source device 22…Diffusion plate 3…Light receiving unit 31…Lens 32…Distance image sensor 320…Light receiving area 321…Pixel 322…Control circuit 323…Vertical scanning circuit 324…Horizontal scanning circuit 325…Image signal processing circuit 4…Distance image processing unit 41…Timing control unit 42…Distance calculation unit 43…Measurement control unit CS…Charge storage unit PO…Light pulse RL…Reflected light OB…Subject

Claims

1. A distance image capturing device that measures the distance to an object based on the time it takes for light irradiated from a light source to be reflected back by the object, having a first distance measurement mode for measuring the distance within at least a part of the measurable distance range in one frame, and a second distance measurement mode for dividing the measurable distance range into a plurality of distance measurement ranges in the depth direction, and measuring at least one of the divided distance measurement ranges with at least one of the sub-frames obtained by dividing the one frame according to the divided distance measurement ranges, and having, a control unit for switching between the first distance measurement mode and the second distance measurement mode, a distance image capturing device comprising the same.

2. In the first distance measurement mode, all ranges in the depth direction of the measurable distance range are measured in the one frame, The distance image capturing device according to claim 1.

3. In the second distance measurement mode, the one frame is divided into a number of sub-frames corresponding to the number of distance measurement ranges obtained by dividing the measurable distance range in the depth direction, and distance measurement is performed for each of the divided distance measurement ranges using the corresponding sub-frame, The distance image capturing device according to claim 1.

4. The second distance measurement mode has higher power consumption and higher depth direction distance measurement accuracy within the measurable distance range compared to the first distance measurement mode, The distance image capturing device according to claim 1.

5. The first distance measurement mode and the second distance measurement mode have different frame rates, The distance image capturing device according to claim 1.

6. The first distance measurement mode and the second distance measurement mode have different distance measurement ranges, The distance image capturing device according to claim 1.

7. The control unit, Switching between the first distance measurement mode and the second distance measurement mode based on detection of a predetermined event within the measurable distance range. The distance image capturing device according to claim 1.

8. The control unit switches between the first distance measurement mode and the second distance measurement mode based on at least one external signal. The distance image capturing device according to claim 1.

9. In the second distance measurement mode, distance measurement is stopped or skipped in some of the sub-frames obtained by dividing one frame. The distance image capturing device according to claim 1.

10. Using the distance information obtained in the first distance measurement mode as the distance information of the sub-frame stopped or skipped in the second distance measurement mode. The distance image capturing device according to claim 9.

11. Using the distance information obtained in the second distance measurement mode as the distance information in the first distance measurement mode. The distance image capturing device according to claim 1.

12. The control unit In the second distance measurement mode, continuously measures the same distance range with a plurality of sub-frames obtained by dividing one frame. The distance image capturing device according to claim 1.

13. In the second distance measurement mode, a part of the distance measurement ranges of a plurality of sub-frames obtained by dividing one frame overlaps. The distance image capturing device according to claim 1.

14. The first distance measurement mode and the second distance measurement mode have different intensities of irradiation light or irradiation profiles. The distance image capturing device according to claim 1.

15. A distance image capturing method performed by a distance image capturing device that measures the distance to an object based on the time it takes for light emitted from a light source to be reflected back from the object until it returns, a first distance measurement mode in which at least a part of the measurable distance range is measured in one frame, and a second distance measurement mode in which the measurable distance range is divided into a plurality of distance measurement ranges in the depth direction, and at least one of the divided distance measurement ranges is measured with at least one of sub-frames obtained by dividing the one frame according to the divided distance measurement ranges, wherein a control unit switches between the first distance measurement mode and the second distance measurement mode, a distance image capturing method.

Images