Distance image measuring device

Abstract

The distance image sensor (10) comprises: a light source (11) generating pulse light (L P ); a light source control unit (31) for controlling the light source (11); a pixel circuit (13) comprising: a photoelectric conversion region (21), a charge readout region (221-224), a charge discharge region (23), and control electrodes (241-244, 25); a charge forwarding control unit (32) for outputting control pulses (G1-G3, G D ); and a distance calculation unit, which reads the voltage of the charge readout region (221-224) as a detection signal (S1-S4), and repeatedly calculates the distance based on the detection signal (S1-S4). The charge forwarding control unit (32) sets the timing of the output control pulse (G1) at the pulse light (L P ) before the timing is generated.

Description

Technical Field

[0001] The present invention relates to a distance image measuring device that generates a distance image including distance information for each pixel. Background Art

[0002] Patent document 1 discloses a sensor device that uses the flight time of light to generate a distance image containing distance information. The sensor device irradiates a target object with a series of first to fifth pulses as irradiation pulses in the first to fifth frames arranged on the time axis. The sensor device generates a distance image using reflected light from the target object caused by the irradiated light. The distance image contains information related to the distance to the target object in the pixel array.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2010-32425. Summary of the invention

[0006] Technical problem to be solved by the invention

[0007] When an object exists near a measuring device, strong reflected light may enter the measuring device from the object. After being reflected inside the measuring device, the reflected light enters the photodetector at a position different from the image position of the object. Such incident light may generate so-called flare. In the measuring device, the flare may become noise.

[0008] The present invention provides a distance image measuring device capable of suppressing the influence of light spots.

[0009] Means for solving technical problems

[0010] A distance image measuring device according to one embodiment of the present invention comprises: a light source for generating pulsed light; a light source control unit for controlling the light source in a manner of periodically and repeatedly generating pulsed light; a pixel circuit unit having: a photoelectric conversion region for converting light into electric charge; first to Mth (M is an integer greater than or equal to 3) charge readout regions, which are arranged close to and separated from the photoelectric conversion region; a charge discharge region for discharging electric charge; first to Mth control electrodes, which are arranged corresponding to the photoelectric conversion region and the first to Mth charge readout regions and receive first to Mth control pulses for forwarding electric charge between the photoelectric conversion region and the first to Mth charge readout regions; and an M+1th control electrode, which is arranged corresponding to the photoelectric conversion region and the first to Mth charge readout regions. The charge forwarding control unit is provided corresponding to the photoelectric conversion region and the charge discharge region, and receives the M+1th control pulse for charge forwarding between the photoelectric conversion region and the charge discharge region; the charge forwarding control unit outputs the first to Mth control pulses to the first to Mth control electrodes corresponding to the generation of the pulse light of the light source control unit, and outputs the M+1th control pulse to the M+1th control electrode during the period other than the output period of the first to Mth control pulses; the voltage detection unit reads the voltage of the first to Mth charge readout regions of the pixel circuit part as the first to Mth detection signals after the charge forwarding control unit outputs the first to Mth control pulses; and the distance calculation unit repeatedly calculates the distance based on the first to Mth detection signals. The charge forwarding control unit sets the timing of outputting the first control pulse to at least one of before the timing of generating the pulse light and after the timing of outputting the second to Mth control pulses.

[0011] The above-mentioned distance image measuring device outputs the second to M-th control pulses to the second to M-th control electrodes. As a result, the charge generated in the photoelectric conversion region due to the light from the object is forwarded to the second to M-th charge readout region. The charge of the second to M-th charge readout region is read out by the voltage detection unit as the second to M-th detection signal. In the distance calculation unit, the read charge is used for distance calculation. On the other hand, the above-mentioned distance image measuring device outputs the first control pulse to the first control electrode. Among them, the timing of outputting the first control pulse is set to at least one of before the timing of pulse light generation and after the second to M-th control pulses are output. The charge forwarded to the first charge readout region according to the output of the first control pulse is caused by background light. That is, at these timings, the light caused by the pulse light will not be incident on the photoelectric conversion region. Or, the light incident on the photoelectric conversion region is extremely weak. Therefore, the charge forwarded to the first charge readout region can be regarded as not including the influence of the light spot. As a result, according to the correction of the first detection signal using the component representing the background light, the corrected signal can suppress the influence of the light spot. Therefore, the distance image measuring device can suppress the influence of the light spot.

[0012] In the above-mentioned manner, the charge forwarding control unit can also set the timing of the second to Mth control pulses in a manner that the delay time of the second to Mth control pulses relative to the timing of the pulse light generation is offset to different times for each of the first to Nth (N is an integer greater than 2) subframe periods contained in one frame period. According to this setting, it is possible to detect charges using multiple time windows with a small duty cycle. As a result, even when objects within various distance measurement ranges are used as the object, the influence of background light in the detection signal can be reduced. As a result, distance calculation with high distance resolution can be achieved. Moreover, it is also possible to prevent the reduction in the accuracy of distance calculation caused by saturation of the detection signal caused by strong background light.

[0013] In the above-mentioned manner, the charge forwarding control unit may also be set to reduce the ratio of the number of output times of at least the second control pulse among the second to M control pulses as the delay time of the second to M control pulses relative to the timing of pulse light generation is shorter between the second to N subframe periods. According to this setting, saturation of the detection signal can be prevented when an object within the short-distance measurement range is used as the object. As a result, the dynamic range of distance measurement can be improved.

[0014] In the above-mentioned manner, the charge forwarding control unit may also perform the following actions: a first action, setting the timing of outputting the first control pulse after outputting the second to M-th control pulses; and a second action, setting the timing of outputting the first control pulse after outputting the second to M-th control pulses, and setting the timing of outputting the first control pulse at the same time as or after the timing of pulse light generation and before outputting the second to M-th control pulses. According to this setting, through the first action, a first detection signal containing a component of background light can be obtained. Through the second action, a first detection signal containing a component of background light and a component of reflected light from an object can also be obtained. Therefore, since the signal component that can be used for distance calculation increases, the accuracy of distance calculation can be improved.

[0015] In the above-mentioned manner, the charge forwarding control unit may also set the timing of outputting the second control pulse to be after a standby time shorter than the duration of the pulse light from the timing of the pulse light generation. According to this setting, the second control pulse is delayed by the corresponding standby time from the timing of the pulse light generation. Therefore, the component of the light spot incident on the photoelectric conversion area during the standby time will not be forwarded to the second charge readout area as a signal component. Therefore, the influence of the light spot on the second detection signal can be reduced.

[0016] A distance image measuring device according to another embodiment of the present invention comprises: a light source for generating pulsed light; a light source control unit for controlling the light source in a manner of periodically and repeatedly generating pulsed light; a pixel circuit unit having: a photoelectric conversion region for converting light into electric charge; first to Mth (M is an integer greater than or equal to 3) charge readout regions, which are arranged close to and separated from the photoelectric conversion region; a charge discharge region for discharging electric charge; first to Mth control electrodes, which are arranged corresponding to the photoelectric conversion region and the first to Mth charge readout regions and receive first to Mth control pulses for forwarding electric charge between the photoelectric conversion region and the first to Mth charge readout regions; and an M+1th control electrode, which is arranged corresponding to the photoelectric conversion region and the first to Mth charge readout regions. The charge forwarding control unit is provided corresponding to the photoelectric conversion region and the charge discharge region, and receives the M+1th control pulse for charge forwarding between the photoelectric conversion region and the charge discharge region; the charge forwarding control unit outputs the first to Mth control pulses to the first to Mth control electrodes corresponding to the generation of the pulse light of the light source control unit, and outputs the M+1th control pulse to the M+1th control electrode during the period other than the output period of the first to Mth control pulses; the voltage detection unit reads the voltage of the first to Mth charge readout regions of the pixel circuit part as the first to Mth detection signals after the charge forwarding control unit outputs the first to Mth control pulses; and the distance calculation unit repeatedly calculates the distance based on the first to Mth detection signals. The charge forwarding control unit performs the following actions: a third action, setting the timing of outputting the first control pulse to at least one of before the timing of generating the pulse light and after the outputting of the second to Mth control pulses; and a fourth action, setting the timing of outputting the first control pulse to be simultaneously with or after the timing of generating the pulse light and before the outputting of the second to Mth control pulses.

[0017] The distance image measuring device as another embodiment can suppress the influence of speckle in the corrected signal, similarly to the distance image measuring device as one embodiment, when the correction is performed using the first detection signal representing the component of background light. Therefore, the distance image measuring device can suppress the influence of speckle.

[0018] In the above-mentioned other aspects, the charge transfer control means may set the number of times the third operation is performed in one frame to be different from the number of times the fourth operation is performed. According to this setting, an operation that reduces the influence of light spots can be performed.

[0019] In the other above-mentioned embodiments, the charge forwarding control unit may also be set so that the delay time of the second to Mth control pulses in the third action relative to the timing of the pulse light generation is the same as the delay time in the fourth action, and the number of times the third action is performed in one frame is greater than the number of times the fourth action is performed. According to the third action, a first detection signal that can ignore the influence of the light spot can be obtained. According to the fourth action, a first detection signal that can be used for distance calculation can be obtained. By performing more third actions than the fourth action, more first detection signals caused by the third action can be obtained. As a result, a calculation result that reduces the influence of the light spot can be obtained.

[0020] In the other embodiments described above, the charge forwarding control unit may also be set so that the delay time of the second to Mth control pulses in the third action relative to the timing of the pulse light generation is different from the delay time in the fourth action, and the number of the third action performed in one frame is less than the number of the fourth action. According to the third action, a first detection signal that can ignore the influence of the light spot can be obtained. According to the fourth action, a first detection signal that can be used for distance calculation can be obtained. In addition, by performing more fourth actions than the third action, more first detection signals caused by the fourth action can be obtained. As a result, since the signal component used for distance calculation increases, the accuracy of distance calculation can be improved.

[0021] In the above-mentioned other aspects, the charge transfer control unit may be set so that the number of times the third operation is performed in one frame is equal to the number of times the fourth operation is performed. This setting can also suppress the influence of light spots.

[0022] Effects of the Invention

[0023] According to the present invention, there is provided a distance image measuring device capable of reducing the influence of speckle. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a block diagram showing a schematic configuration of a distance image sensor according to the first embodiment.

[0025] Figure 2 It is used to illustrate Figure 1 A timing diagram showing the principle of distance calculation of a distance image sensor.

[0026] Figure 3 Yes means through Figure 1 A timing diagram of various signals processed by the distance image sensor, and a graph of changes in various values ​​calculated by the distance image sensor relative to delay time.

[0027] Figure 4It is a diagram for explaining the operation of the distance image sensor according to the first embodiment and the distance image sensor according to the second embodiment.

[0028] Figure 5 1 is a timing chart showing various signals processed by the distance image sensor according to the second embodiment, and a graph showing changes in various values ​​calculated by the distance image sensor with respect to delay time.

[0029] Figure 6 1 is a timing chart showing various signals processed by the distance image sensor according to the second embodiment, and a graph showing changes in various values ​​calculated by the distance image sensor with respect to delay time.

[0030] Figure 7 This is a block diagram showing a schematic configuration of a distance image sensor according to a third embodiment.

[0031] Figure 8 1 is a timing chart of various signals processed by the distance image sensor according to the third embodiment.

[0032] Fig. 9 Yes means Figure 8 A portion of a timing diagram of various signals is shown, as well as a graph of the variation of various values ​​calculated by the range image sensor with respect to the delay time.

[0033] Fig.10 1 is a timing chart of various signals processed by the distance image sensor according to the fourth embodiment.

[0034] Fig.11 It is a diagram for explaining the operation of the distance image sensor according to the fourth embodiment.

[0035] Fig.12 1 is a timing chart of various signals processed by the distance image sensor according to the fifth embodiment.

[0036] Fig.13 It is a part of a timing chart showing various signals processed by the distance image sensor according to the fifth embodiment, and a graph showing changes in various values ​​calculated by the distance image sensor with respect to delay time.

[0037] Fig.14 1 is a timing chart of various signals processed by the distance image sensor according to the sixth embodiment.

[0038] Fig.15 1 is a timing chart of various signals processed by the distance image sensor according to the seventh embodiment.

[0039] Fig.16It is a part of a timing chart showing various signals processed by the distance image sensor according to the seventh embodiment, and a graph showing changes in various values ​​calculated by the distance image sensor with respect to delay time.

[0040] Fig.17 1 is a timing chart showing various signals processed by the distance image sensor according to the eighth embodiment, and a graph showing changes in various values ​​calculated by the distance image sensor with respect to delay time. DETAILED DESCRIPTION

[0041] Hereinafter, the mode for implementing the present invention will be described in detail with reference to the drawings. In the description of the drawings, the same elements are given the same reference numerals, and repeated descriptions are omitted.

[0042] [First embodiment]

[0043] Reference Figure 1 , the function and structure of the distance image sensor 10 of the first embodiment of the distance image measuring device of the present invention are described. The distance image sensor 10 generates a distance image containing distance information for each pixel using the time of flight method. The distance image sensor 10 includes: a light source 11, an operation circuit 12, and a plurality of pixel circuits 13 (pixel circuit unit). The distance image sensor 10 performs distance measurement based on the time of flight (TOF) method. The light source 11 generates a pulse light L irradiated to the object S. P . The light source 11 has: a semiconductor light emitting element such as a light emitting diode or a laser diode, and a driving circuit for driving the semiconductor light emitting element. As the light source 11, an element that generates light in a wavelength range such as a near-infrared range and a visible light range can be used. The distance image sensor 10 has a plurality of pixel circuits 13. The plurality of pixel circuits 13 are arranged in a two-dimensional array in a two-dimensional direction (for example, a column direction and a row direction). The plurality of pixel circuits 13 constitute an image sensor. The plurality of pixel circuits 13 are formed by reflecting the pulse light L by the object S. P The incident pulse light L R Photoelectric conversion is performed to generate a detection signal. The distance image sensor 10 also includes an operation circuit 12. The operation circuit 12 uses the detection signals generated by the plurality of pixel circuits 13 to calculate the distance information related to the object S for each pixel. The operation circuit 12 generates and outputs a distance image including two-dimensional image information reflecting the distance information of each pixel. The operation circuit 12 can also be composed of a dedicated integrated circuit such as a single-chip microcomputer including a CPU, RAM, ROM, and input and output devices. In addition, the operation circuit 12 can also be composed of a general-purpose computer such as a personal computer.

[0044] Hereinafter, the configurations of the pixel circuit 13 and the calculation circuit 12 will be described in detail.

[0045] The structure of the pixel circuit 13 is described below. The pixel circuit 13 is composed of a semiconductor element. The pixel circuit 13 includes a photoelectric conversion region 21 and a charge readout region 22. 1 ~22 3 , charge discharge region 23, control electrode 24 1 ~24 3 , control electrode 25, and voltage detection unit 26 1 ~26 3 The photoelectric conversion region 21 receives the incident pulse light L R Converted into electric charge. Charge readout region 22 1 ~22 3 The charge discharge region 23 is close to the photoelectric conversion region 21. 1 ~22 3 The control electrode 24 is separated from the charge discharge region 23. 1 ~24 3 and the control electrode 25 and the charge readout region 22 1 ~22 3 The control electrode 24 is provided correspondingly to the charge discharge region 23. 1 ~24 3 And the control electrode 25 receives a control pulse for charge transfer between the photoelectric conversion region 21 and the respective regions. 1 ~26 3 From the charge readout region 22 1 ~22 3 Read out the detection signal. Voltage detection unit 26 1 ~26 3 For example, the amplifier includes a source follower amplifier. The voltage detection unit 26 1 ~26 3 Under the control from the operation circuit 12, the charge readout area 22 is selectively detected and amplified. 1 ~22 3 The reference potential is the reference voltage. Voltage detection unit 26 1 ~26 3 The amplified voltage is output to the operation circuit 12 as a detection signal.

[0046] The pixel circuit 13 is formed on a p-type semiconductor substrate such as a silicon substrate. The photoelectric conversion region 21 is provided in the center of the pixel formation region. The pixel formation region is formed on a p-type semiconductor substrate. The pixel formation region has an active region formation layer composed of a p-type semiconductor, an n-type surface embedding region, a p-type fixed layer, and an insulating film. An n-type charge readout region 22 having an impurity concentration higher than that of the n-type surface embedding region is formed at a position separated from each other in a manner close to the photoelectric conversion region 21. 1 ~22 3 and the charge discharge region 23. From the photoelectric conversion region 21 on the insulating film to the charge readout region 22 1 ~22 3 A control electrode 24 is provided on the charge transfer path of each of the charge discharge regions 23. 1 ~24 3 , 25. Control electrode 24 1 ~24 3 , 25 can be respectively arranged on the charge moving path. Control electrode 24 1 ~24 3 25 may be separately provided on a plurality of electrode portions so as to sandwich the charge transfer path from both sides.

[0047] The calculation circuit 12 controls the electrode 24 1 ~24 3 , 25 output control pulses with different phases. As a result, the depletion potential of the surface embedding region changes sequentially. Therefore, a potential gradient is formed in order to transfer charges to any of the charge transfer paths. As a result, the majority carriers (charges) generated in the surface embedding region of the photoelectric conversion region 21 are transferred to the charge readout region 22. 1 ~22 3 And any one of the charge discharge regions 23 moves.

[0048] Next, the structure of the operation circuit 12 is described. The operation circuit 12 includes, as functional structural elements, a light source control unit 31, a charge forwarding control unit 32, a distance data validity determination signal generating unit 33, a distance data validity determination unit 34, an invalid pixel identification value generating unit 35, a distance calculation reference signal generating unit 36, a distance calculation reference signal selecting unit 37, and a distance image generating unit 38. The distance data validity determination signal generating unit 33, the distance data validity determination unit 34, the invalid pixel identification value generating unit 35, the distance calculation reference signal generating unit 36, the distance calculation reference signal selecting unit 37, and the distance image generating unit 38 constitute the distance calculation unit of the present embodiment.

[0049] The light source control unit 31 of the operation circuit 12 is used to control the pulse light L based on the light source 11.P Timing, pulse light L P The intensity of the pulse light L P Specifically, the light source control unit 31 periodically and repeatedly generates a pulse of duration T within a preset distance calculation repetition period, that is, a frame period. 0 (first duration) of pulse light L P The charge forwarding control unit 32 controls the electrode 24 1 ~24 3 , 25 respectively output control pulse G 1 ~G 3 And the control pulse G D That is, the charge forwarding control unit 32 is 0 The corresponding duration T 1 During the (second duration) period, the control electrode 24 1 ~24 3 Output control pulse G 1 ~G 3 . Output control pulse G 1 ~G 3 The timing of the pulse light L in one frame period P In this embodiment, the duration T 1 It can also be compared with the duration T 0 The charge forwarding control unit 32 removes the output control pulse G 1 ~G 3 During the period other than the timing of D Output to the control electrode 25. Control pulse G D The charges accumulated in the photoelectric conversion region 21 are discharged to the charge discharge region 23 .

[0050] In the case of using pulsed light L P When the resolution of the distance calculation of the flight time is set to 15 cm, if the resolution is converted into the flight time, it is equivalent to 1 nanosecond. In order to achieve this resolution, a control unit with a time accuracy of the order of picoseconds is required as a reference. Therefore, the charge forwarding control unit 32 and the light source control unit 31 need to consider the design of the wiring capacitance and the like formed integrally with the pixel circuit 13. Therefore, it is preferred that the charge forwarding control unit 32 and the light source control unit 31 are formed on a semiconductor that is integrally formed with the pixel circuit 13. In addition, the "integrated semiconductor" also includes different semiconductors in a plurality of semiconductor layers stacked using SOI (Silicon On Insulator) technology or TSV (Through Silicon Via) technology. Specifically, the charge forwarding control unit 32, the charge forwarding control unit 32 and the control electrode 24 1 ~243 The circuits between the pixel circuit 13 and the pixel circuit 13 may be formed on the same semiconductor as the pixel circuit 13. Alternatively, these may be formed on a semiconductor layer stacked together with the pixel circuit 13.

[0051] The distance calculation unit of the calculation circuit 12 repeatedly performs the distance calculation of each pixel circuit 13 for each frame as a preset distance calculation repetition period. The distance calculation unit repeatedly generates a distance image including distance information. The distance data validity determination signal generation unit 33 generates a distance image including a distance information based on the distance information of the pulse light L. P The detection signal S output from the pixel circuit 13 corresponds to the timing of 1 ~S 3 , generate distance data validity determination signal S A12 . Distance data validity judgment signal S A12 is removed from the detection signal S 2 , S 3 The incident pulse light L R The total value of the signal component of the generated charge. Distance data validity judgment signal S A12 Indicates the detection signal S 2 , S 3 Is it a strong reflection of the incident pulse light L R Signal. Distance data validity judgment signal S A12 It is used to determine based on the detection signal S 2 , S 3 The distance data validity determination unit 34 determines whether the calculation of the distance is valid based on the distance data validity determination signal S A12 , to determine based on the detection signal S 2 , S 3 Specifically, the distance data validity determination unit 34 determines whether the distance data validity determination signal S A12 and a predetermined threshold value (first threshold value) Th 1 The distance data validity determination unit 34 generates a distance data validity determination signal S based on the comparison result. A12 Exceeding the threshold Th 1 On the other hand, the distance data validity determination unit 34 determines that the distance data validity determination signal S is valid when the comparison result is A is the threshold value Th 1 In the following cases, the calculation of the distance is determined to be invalid. The invalid pixel identification value generating unit 35 generates an identification value based on the determination result of the distance data validity determining unit 34. The identification value indicates whether the calculation of the distance for each pixel corresponding to the pixel circuit 13 is invalid.

[0052] The distance calculation reference signal generating unit 36 ​​generates a reference signal based on the pulse light L P The detection signal S output from the pixel circuit 13 corresponds to the timing of 2 , S 3 , generating a distance calculation reference signal D as a basis for distance calculation A12 Specifically, the distance calculation reference signal generation unit 36 ​​uses two detection signals S 2 , S 3 The difference between the distance data and the validity judgment signal S A12 The ratio of the distance calculation reference signal D is generated. A12 The distance calculation reference signal selection unit 37 selects the distance calculation reference signal D based on the distance calculation reference signal D A12 The distance calculation reference signal selection unit 37 determines whether the position of the object S is included in the measurable range. If the result of the determination is that the position of the object S is included in the measurable range, the distance calculation reference signal selection unit 37 sets the distance calculation reference signal D A12 The distance calculation reference signal selection unit 37 outputs the distance calculation reference signal D as a valid value to the distance image generation unit 38. A12 The value of Th is different from the specified threshold value (second and third threshold value) 2 ,Th 3 The distance calculation reference signal selection unit 37 determines the distance calculation reference signal D according to the comparison result. A12 When the distance calculation is determined to be valid, the distance image generation unit 38 refers to the distance calculation reference signal D A12 The distance information is calculated. Then, the distance image generation unit 38 generates a distance image including the distance information corresponding to each pixel circuit 13. The distance image is output to an external device. Examples of the external device as the output destination include output devices such as a display device and a communication interface device. The distance image generation unit 38 can embed an invalid value in the pixel generated by the identification value indicating that the distance information is invalid. In addition, the distance image generation unit 38 determines that the distance calculation reference signal D A12 Pixels that are not included in the measurable range may be buried with invalid values ​​in the range image.

[0053] Figure 2 1 is a timing chart for explaining the principle of distance calculation by the distance image sensor 10. Figure 2 In FIG. 1 , the timing of various signals controlled by the distance image sensor 10 is shown. Figure 2 , the timing of charge accumulation in each region of the pixel circuit 13 is shown. Figure 2 The pulse light L is shown in order from the top P Timing and control pulse G1 ~G 3 Timing and control pulse G D Timing and charge readout region 22 1 ~22 3 In this way, the pulse light L P Duration T 0 The timing of the control pulse G 2 ~G 3 In a non-overlapping manner for a duration T 0 In addition, when the control pulse G is not output 1 , G 2 , G 3 The timing output control pulse G D According to this control, by R The charge accumulated in the photoelectric conversion region 21 is converted into a photoelectric charge to react with the incident pulse light L R Relative to pulse light L P The delay time T D The corresponding ratio is allocated to the two charge readout regions 22 2 ,twenty two 3 On the other hand, the control pulse G 1 The timing ratio of pulse light L P As a result, in the charge readout region 22 1 In the control pulse G 1 Only the charge N caused by noise such as background light and dark current is transmitted within the specified time window. B In this regard, in the charge readout region 22 2 In the control pulse G 2 In the specified time window, the amount of charge N delivered B Plus the delay time T D The corresponding amount of charge N sm1 On the other hand, in the charge readout region 22 3 In the control pulse G 3 In the specified time window, the amount of charge N delivered B Plus the delay time T D The corresponding amount of charge N sm2 By using this phenomenon, in the calculation circuit 12 of the distance image sensor 10, according to the pulse light L P The amount of charge N removed is calculated at the respective timings. B The charge N sm1 With the charge removed N B The charge N sm2As a result, the calculation circuit 12 of the distance image sensor 10 can calculate the ratio of the delay time T D The distance of the corresponding object S.

[0054] Next, the procedure of distance calculation by the distance image sensor 10 will be described in detail. The distance image measurement method described below is also referred to as a "3-tap 1-block method".

[0055] Figure 3 is a timing diagram showing various signals processed by the distance image sensor 10, and various values ​​calculated with respect to the delay time T D Graph of the changes in . Figure 3 (a) to Figure 3 (f) is based on the actual time (t). On the other hand, Figure 3 (g) to Figure 3 (n) with delay time T D Delay time T D The pulse light L P Duration T 0 Normalized value. Delay time T D It is pulse light L P The charge is reflected by the object S and enters the photoelectric conversion region 21, and the charge enters the charge readout region 22 from the photoelectric conversion region 21. 1 ~22 3 In other words, Figure 3 (g) to Figure 3 (n) is based on the time obtained by dividing the round-trip distance to the object S by the speed of light. In other words, the delay time T D The axis of is the axis of time which is proportional to the distance to the object S. Figure 3 (a) to Figure 3 (d) shows the control pulse G 1 ~G 3 , G D Timing. Figure 3 (e) shows the pulse light L P This timing is called "pulse light L P In addition, Figure 3 (e) shows the incident pulse light L R The timing at which the incident pulse light L enters the photoelectric conversion region 21. This timing is referred to as "the timing at which the incident pulse light L R Timing". Figure 3 (f) shows the stray light L F The timing at which the stray light L enters the photoelectric conversion region 21. This timing is referred to as "stray light L F Timing". Figure 3 (g) to Figure 3 (i) shows the detection signal S 1 ~S 3 The value of . Figure 3 In (j), the difference value S is shown. 3-2 The value of Figure 3 (k) shows the distance data validity determination signal S A12 The value of Figure 3 (l) shows the distance calculation reference signal D A12 The value of Figure 3 In (m), the data valid range is shown. Figure 3 In (n), the measurable range is shown.

[0056] The distance image sensor 10 starts the process of generating the distance image. The light source control unit 31 controls the pulse light L to P In addition, the charge transfer control unit 32 controls the output control pulse G during one frame. 1 ~G 3 , G D Specifically, first, the charge forwarding control unit 32 controls the pulse G 1 ~G 3 , G D The duration is set to T 1 =T 0 Next, the charge transfer control unit 32 outputs the control pulse G during the period from t=-2 to -1. 1 In addition, the charge forwarding control unit 32 can make the start control pulse G 1 The timing of the output is strictly consistent with t = -2, and may be slightly delayed from t = -2. Next, the charge transfer control unit 32 outputs the control pulse G during the period from t = -1 to 0. D Next, the charge transfer control unit 32 outputs the control pulse G during the period t=0-1. 2 Then, the light source control unit 31 makes the pulse light L P More specifically, the light source control unit 31 starts to emit pulse light L P The timing of emission of the pulse light L is set to be slightly earlier than t=0. P The timing of the injection to start the control pulse G 2 There is a waiting time T between the output timing and the output timing. SB Next, the charge transfer control unit 32 outputs the control pulse G during the period from t=+1 to +2. 3 Furthermore, the charge transfer control unit 32 outputs the control pulse G during the period after t=+2.D After that, the voltage detection unit 26 of each pixel circuit 13 1 ~26 3 Read out the detection signal S 1 ~S 3 . These detection signals S 1 ~S 3 The voltage is output to the calculation circuit 12 (voltage detection step).

[0057] In addition, the charge forwarding control unit 32 outputs a control pulse G 2 The timing is set at the pulse light L P After the timing is generated, the waiting time T SB After that. Standby time T SB Pulse light L P The duration T 0 According to this setting, the control pulse G 2 From the pulse light L P The generated timing delay waiting time T SB Therefore, during the standby time T SB During this period, the stray light L incident on the photoelectric conversion region 21 F Will not be used as a detection signal S 2 A portion of the charge is forwarded to the charge readout region 22 2 Therefore, the detection signal S can be reduced 2 The stray light L F impact.

[0058] Next, the arithmetic circuit 12 calculates the detection signal S output from the pixel circuit 13. 1 ~S 3 , calculate the distance information of each pixel (distance calculation step). That is, the distance data validity determination signal generation unit 33 calculates the distance information of each pixel based on the detection signal S 1 ~S 3 , use the following formulas (1) and (2) to calculate the distance data validity judgment signal S A12 The value of .

[0059] S 3-2 =S 3 -S 2 …(1)

[0060] S A12 =S 3 +S 2 -2S 1 …(2)

[0061] In the above equations (1) and (2), the detection signal S 1 ~S 3 The signal value (voltage value) shown is represented by the symbol "S1 ”, “S 2 ”, “S 3 " indicates. The same applies to the following description. Distance data validity determination signal S A12 The value is from the detection signal S 2 , S 3 The total value of the value is divided by the charge N B The value after the component. Charge N B The component is caused by noise such as background light and dark current. Charge N B The components and detection signal S 1 Corresponding. Distance data validity judgment signal S A12 The value of the detection signal S 2 , S 3 The incident pulse light L R The distance data validity determination unit 34 determines the distance data validity determination signal S A12 The value and threshold Th 1 The distance data validity determination unit 34 determines whether the detection signal S is used. 2 , S 3 Specifically, the distance data validity determination unit 34 determines whether the distance data has a value greater than the threshold value Th 1 The value of the distance data validity judgment signal S A12 Therefore, Figure 3 The ranges shown in (m) are determined to be ranges in which distance calculation is valid. In other words, these ranges are determined to be "data valid ranges".

[0062] Next, the distance calculation reference signal generation unit 36 ​​calculates the distance calculation reference signal D using the following equation (3): A12 The value of .

[0063] D A12 =0.5+S 3-2 / (2S A12 )…(3)

[0064] Next, the distance calculation reference signal selection unit 37 determines the distance calculation reference signal D A12 In other words, the distance calculation reference signal selection unit 37 determines whether the object S is within the measurable range. The distance calculation reference signal selection unit 37 sets the distance calculation reference signal D for determining whether the object S is within the measurable range. A12 The threshold value Th 2 ,Th 3 For example, the threshold Th 2 is 0.01. In addition, the threshold Th 3Then, the distance calculation reference signal selection unit 37 obtains the distance calculation reference signal D A12 is the threshold value Th 2 Above the threshold Th 3 The following time range is the measurable range (Th 2 ≤D A12 ≤Th 3 As a result, t = 0.31 to 1.29 is obtained as the measurable range. By making such a determination, it is possible to exclude from the distance calculation the case where the object S is too close to the incident pulse light L. R From the detection signal S 2 The time window deviation causes the distance calculation reference signal D A12 The value of does not reflect the distance. In addition, it is also possible to remove the object S from the distance calculation because the incident pulse light L is too far away. R From the detection signal S 3 The time window deviation causes the distance calculation reference signal D A12 The value does not reflect the distance.

[0065] Finally, the distance image generation unit 38 calculates the reference signal D based on the distance determined to be in the "data valid range" and in the "measurable range". A12 , to calculate the distance of the object S. As a result, a distance image including the distance information of each pixel is generated and output.

[0066] The above-mentioned distance image sensor 10 controls the pulse G 2 , G 3 To control electrode 24 2 ,twenty four 3 As a result, the incident pulse light L from the object S R The charge generated in the photoelectric conversion region 21 is transferred to the charge readout region 22. 2 ,twenty two 3 . Charge readout region 22 2 ,twenty two 3 The charge is detected by the voltage detection unit 26 2 , 26 3 Read out as the detection signal S 2 , S 3 The read detection signal S 2 , S 3 The calculation circuit 12 uses this to calculate the distance.

[0067] Among them, in the pulse light L P The light that causes the pulsed light L P The incident pulse light L is reflected by the object S and enters the distance image sensor 10. R . The incident pulse light LR The incident pulse light L directly enters the photoelectric conversion region 21 of the pixel at the position where the image is formed. R There is no further reflection from the object S to the photoelectric conversion region 21. Therefore, the incident pulse light L R This can be used as a basis for calculating the distance from the object S to the distance image sensor 10 .

[0068] In the pulsed light L P In the light caused by the stray light L F . Stray light L F Stray light L refers to light reflected by the object S and then further reflected by an object different from the object S. F For example, stray light L may be generated by reflection from a lens disposed between the object S and the distance image sensor 10. F Sometimes, the stray light L is generated by reflection from the housing 51 that houses the lens and the distance image sensor 10. F It is pulse light L P The stray light L is reflected from an object at a very close distance from the image sensor 10, so the effect is particularly large. F The incident position does not correspond to the position of the object S. Therefore, the stray light L F It cannot be used as a basis for calculating the distance from the object S to the image sensor 10. F When incident on the photoelectric conversion region 21 of a pixel located at a position different from the position of the imaging object S, a spot of light is generated. F It refers to noise light that cannot be used as a basis for distance calculation and may become noise in distance calculation.

[0069] Therefore, the distance image sensor 10 of the first embodiment sets the time window of one tap among the multiple taps (control pulse G 1 ) is set to the pulse light L P The position shifted in time from the generation timing of the control pulse G 1 ) obtained in the detection signal S 1 Used to correct background light. In other time windows (control pulse G 2 , G 3 ) obtained in the detection signal S 2 , S 3 Used to measure distance.

[0070] In more detail, the distance image sensor 10 controls the pulse G 1 To control electrode 241 Output. Output control pulse G 1 The timing is earlier than the pulse light L P The timing of the generation. According to this timing, the pulse light L P The incident pulse light L R The stray light L F It will not be incident on the photoelectric conversion region 21. That is, according to the control pulse G 1 The output is forwarded to the charge readout region 22 1 The charge is caused by the background light. Therefore, it is forwarded to the charge readout region 22 1 The charge of can be regarded as not including the influence of light spots. As a result, the detection signal S using the component representing the background light is 1 The correction can be regarded as that the signal after correction is not affected by the light speckle. Therefore, the distance image sensor 10 can suppress the influence of the light speckle.

[0071] [Second Embodiment]

[0072] The distance image sensor 10 of the first embodiment is based on Figure 3 The action is performed at the timing shown. Figure 4 As shown in FIG. 2( a ), the distance image sensor 10 of the first embodiment performs a distance calculation by the calculation circuit 12 during one frame period (T f ), repeatedly execute Figure 3 The operation shown. According to this operation, the measurement range of the distance image sensor 10 is only one range. The distance image sensor 10 of the second embodiment expands the measurement range to two or more. The expansion of the measurement range can be achieved by introducing a sub-frame method.

[0073] That is, the distance image sensor 10 of the second embodiment uses a subframe for a short distance and a subframe for a long distance. The distance image sensor 10 of the second embodiment measures the distance by synthesizing the results obtained for each subframe.

[0074] like Figure 4 (b) and Figure 4 As shown in (c), during one frame (T f ) may also include more than two subframe periods S Z1 , S Z2 , S Z3 And the readout periods R corresponding to them respectively. Z1 The object S which is relatively close to the image sensor 10 is used as the measurement object. Z3 The object S which is relatively far from the image sensor 10 is used as the measurement object.P The control pulse G is based on the generation timing of 2 , G 3 The timing is used to control the measurement range.

[0075] For example, during the subframe S Z1 In the example, the distance image sensor 10 is Figure 3 The operation is performed at the timing shown. Z1 The subframe period S during which the object S at a farther position is the object Z2 In the example, the distance image sensor 10 is Figure 5 The operation is performed at the timing shown. Z2 The subframe period S during which the object S at a farther position is the object Z3 In the example, the distance image sensor 10 is Figure 6 The operation is performed at the timing shown.

[0076] Figure 5 as well as Figure 6 is a timing diagram showing various signals processed by the distance image sensor 10, and various values ​​calculated relative to the delay time T D The curve diagram of the change of Figure 5 (a) to Figure 5 (d) and Figure 6 (a) to Figure 6 (d) shows the control pulse G 1 ~G 3 , G D Timing. Figure 5 (e) and Figure 6 (e) shows the pulse light L P The timing and incident pulse light L R Timing. Figure 5 (f) and Figure 6 (f) shows the stray light L F Timing. Figure 5 (g) to Figure 5 (i) and Figure 6 (g) to Figure 6 (i) shows the detection signal S 1 ~S 3 The value of Figure 5 (j) and Figure 6 In (j), the difference value S is shown. 3-2 The value of Figure 5 (k) and Figure 6 (k) shows the distance data validity determination signal S A12 The value of Figure 5 (l) and Figure 6(l) shows the distance calculation reference signal D A12 The value of Figure 5 (m) and Figure 6 In (m), the data valid range is shown. Figure 5 (n) and Figure 6 In (n), the measurable range is shown.

[0077] exist Figure 5 In the period from t = +1 to +3, the incident pulse light L R is incident on the photoelectric conversion region 21. That is, the incident pulse light L R The timing is later than Figure 3 The incident pulse light L R In other words, Figure 5 The delay time T D Compare Figure 3 The delay time T D Therefore, in order to capture the incident pulse light L R , control pulse G 2 The timing of the control pulse G is set in the period of t = +1 to +2. 3 The timing is set to the period of t = +2 to +3. Then, the detection signal S 1 ~S 3 , obtain the distance data validity judgment signal S A12 And the distance calculation reference signal D A12 The calculation contents are the same as those in the first embodiment. Then, the distance calculation reference signal selection unit 37 determines the distance calculation reference signal D A12 Is the value within the threshold Th 2 ,Th 3 In this embodiment, the threshold value Th 2 is 1.01. In addition, the threshold Th 3 is 1.99. That is, the predetermined range is 1.01 or more and 1.99 or less. Therefore, the distance calculation reference signal selection unit 37 determines that the distance calculation reference signal D A12 Is the value of 1.01 or more and 1.99 or less (Th 2 ≤D A12 ≤Th 3 ). As a result, t=1.21 to 2.19 was obtained as the measurable range.

[0078] exist Figure 6 In the example above, the incident pulse light L caused by the object S at a greater distance is captured. R If the distance between the object S and the distance image sensor 10 increases, the pulse light L P From the start of emission to the incident pulse light LR The time until the incident light reaches the photoelectric conversion region 21 becomes longer. Figure 6 In the example, during the period from t = +2 to +4, the incident pulse light L R The incident pulse light L R The timing is later than Figure 3 as well as Figure 5 The incident pulse light L R In other words, Figure 6 The delay time T D Compare Figure 3 as well as Figure 5 The delay time T D Therefore, in order to capture the incident pulse light L R , control pulse G 2 The timing of the control pulse G is set to the period of t = +2 to +3. 3 The timing is set to the period of t = +3 to +4. 1 ~S 3 , obtain the distance data validity judgment signal S A12 And the distance calculation reference signal D A12 The distance calculation reference signal selection unit 37 determines the distance calculation reference signal D A12 Is the value within the threshold Th 2 ,Th 3 In this embodiment, the threshold value Th 2 is 2.01. Threshold Th 3 is 2.99. The prescribed range is 2.01 or more and 2.99 or less. Therefore, the distance calculation reference signal selection unit 37 determines that the distance calculation reference signal D A12 Is the value of 2.01 or more and 2.99 or less (Th 2 ≤D A12 ≤Th 3 ). As a result, t=2.11 to 3.09 was obtained as the measurable range.

[0079] Finally, the distance image generation unit 38 determines that the distance calculation reference signal D A12 If it is in the "data valid range" and is determined to be in the "measurable range", the reference signal D is calculated based on the distance A12 , to calculate the distance of object S.

[0080] According to the operation of the second embodiment, the incident pulse light L R The generated charge is distributed to multiple time windows. Multiple time windows and three subframe periods S Z1 , S Z2 , SZ3 The detection signal S 2 , S 3 As a result, the delay time T can be extended. D The corresponding calculable range. Moreover, the time width of the time window can also be reduced. On this basis, the distance can be calculated using the value of the appropriate distance data reference signal corresponding to the range where the object S is located. Therefore, regardless of the position of the object S, a high-precision image signal can be generated.

[0081] According to the operation of the second embodiment, it is possible to detect the charge using a plurality of time windows with a small duty cycle. As a result, even when the object S is within a range of distance measurement, the detection signal S can be used to detect the charge. 1 Reduce the detection signal S 2 , S 3 As a result, distance calculation with high distance resolution is achieved. In addition, the detection signal S caused by strong background light can be prevented. 2 , S 3 The saturation of the image will cause errors in distance calculation.

[0082] [Third Embodiment]

[0083] The distance image sensor 10 of the first embodiment adopts a 3-tap 1-area 1-subframe method. The distance image sensor 10A of the third embodiment further adopts a 4-tap 2-time window method as another method. Figure 7 As shown, the distance image sensor 10A has four charge readout regions 22 as a physical structure. 1 ~22 4 , four control electrodes 24 1 ~24 4 and four voltage detection units 26 1 ~26 4 . Moreover, the distance image sensor 10A may also use more than two subframes. That is, the distance image sensor 10A uses a multi-tap of more than four taps. Moreover, the distance image sensor 10A processes each distance range that is range-shifted by the combination of taps. Then, the distance image sensor 10A performs distance measurement by synthesizing the results obtained for each distance range.

[0084] Figure 8 is a timing diagram of various signals processed by the distance image sensor 10A. Figure 8 (a) to Figure 8 (f) shows an operation in which a short distance is used as the measurement target range. Figure 8 (g) to Figure 8(l) shows the operation of taking the middle distance as the measurement target range. Figure 8 (m) to Figure 8 (r) shows an operation in which a long distance is set as the measurement target range.

[0085] exist Figure 8 (a) shows the pulse light L P The timing of the incident pulse light L caused by the close object S R Timing. Figure 8 (b) to Figure 8 (f) shows the control pulse G 1 ~G 4 , G D The charge transfer control unit 32 outputs the control pulse G during the period from t = -2 to -1. 1 .like Figure 8 As shown in parts (c) to (e) of FIG. 1 , the charge forwarding control unit 32 captures the incident pulse light L caused by the object S at a close distance. R , output control pulse G during the period of t = 0 to +1 2 , output control pulse G during t = +1 to +2 3 , output control pulse G during t = +2 to +3 4 The charge forwarding control unit 32 does not output the control pulse G 1 ~G 4 During the period, the output control pulse G D .like Figure 8 As shown in (a), according to this operation, it is possible to capture the incident pulse light L incident during the period of t=0 to +3. R .

[0086] exist Figure 8 (g) shows the pulse light L P The timing of the incident pulse light L caused by the medium-distance object S R Timing. Figure 8 (h) to Figure 8 (l) shows the control pulse G 1 ~G 4 , G D The charge forwarding control unit 32 outputs the control pulse G during the period from t = -2 to -1. 1 .like Figure 8 As shown in parts (i) to (k) of FIG. 1 , the charge forwarding control unit 32 captures the incident pulse light L caused by the medium-distance object S. R , output control pulse G during t = +2 ~ +3 2 , output control pulse G during t = +3 ~ +4 3, output control pulse G during t = +4 to +5 4 The charge forwarding control unit 32 does not output the control pulse G 1 ~G 4 Output control pulse G during D .like Figure 8 As shown in (g), according to this operation, it is possible to capture the incident pulse light L incident during the period t = +2 to +5. R .

[0087] exist Figure 8 (m) shows the pulse light L P The timing of the incident pulse light L caused by the distant object S R Timing of incidence. Figure 8 (n) to Figure 8 (r) shows the output control pulse G 1 ~G 4 , G D The charge forwarding control unit 32 outputs the control pulse G during the period from t = -2 to -1. 1 .like Figure 8 (o) to Figure 8 As shown in (q), the charge transfer control unit 32 captures the incident pulse light L caused by the distant object S. R , output control pulse G during t = +4 to +5 2 , output control pulse G during t = +5 ~ +6 3 , output control pulse G during t = +6 ~ +7 4 The charge forwarding control unit 32 does not output the control pulse G 1 ~G 4 Output control pulse G during D .like Figure 8 As shown in (g), according to this operation, it is possible to capture the incident pulse light L incident during the period of t = +4 to +7. R .

[0088] Fig. 9 is a timing diagram showing various signals processed by the distance image sensor 10A, and various values ​​calculated relative to the delay time T D Graph of the changes in . Fig. 9 It is detailed Figure 8 A diagram showing the action of the sub-frame used for close distance. Fig. 9 (a) to Fig. 9 (g) Based on actual time (t). Fig. 9 (h) to Fig. 9 (p) with delay time T D As a benchmark. Fig. 9 (a) to Fig. 9 (e) shows the output control pulse G 1 ~G 4 , G D Timing. Fig. 9 (f) shows the pulse light L P The timing and incident pulse light L R Timing. Fig. 9 (g) shows the stray light L F Timing. Fig. 9 (h) to Fig. 9 (k) shows the detection signal S 1 ~S 4 The value of Fig. 9 (l) shows the distance data validity determination signal S A The value of Fig. 9 (m) shows the difference value S 24 .exist Fig. 9 In (n), the value of the distance calculation reference signal D is shown. Fig. 9 In (o), the data valid range is shown. Fig. 9 In (p), the measurable range is shown.

[0089] The distance image sensor 10A starts the process of generating the distance image. The light source control unit 31 and the charge transfer control unit 32 of the calculation circuit 12 control the output control pulse G during one frame. 1 ~G 4 , G D Timing and pulse light L P Specifically, first, the charge forwarding control unit 32 controls the pulse G 1 ~G 4 , G D The duration is set to T 1 =T 0 Next, the charge transfer control unit 32 outputs the control pulse G during the period from t=-2 to -1. 1 Next, the charge transfer control unit 32 outputs the control pulse G during the period from t=-1 to 0. D Next, the charge transfer control unit 32 outputs the control pulse G during the period from t=0 to +1. 2 Next, the charge transfer control unit 32 outputs the control pulse G during the period from t=+1 to +2. 3 Then, the charge transfer control unit 32 outputs the control pulse G during the period from t=+2 to +3. 4 Then, the charge transfer control unit 32 outputs the control pulse G in the period after t=+3. DThen, the voltage detection unit 26 of each pixel circuit 13 1 ~26 4 Read out the detection signal S 1 ~S 4 . Detection signal S 1 ~S 4 The voltage is output to the calculation circuit 12 (voltage detection step).

[0090] Next, the calculation circuit 12 calculates the detection signal S output from each pixel circuit 13. 1 ~S 4 , to calculate the distance information of each pixel (distance calculation step). The step of calculating the distance information includes: obtaining the distance data validity determination signal S A Steps; Get the difference value S 24 step of obtaining a distance calculation reference signal D; a step of determining a valid range of data; a step of determining a measurable range; and a step of obtaining a distance of an object S.

[0091] First, obtain the distance data validity judgment signal S A The distance data validity determination signal generation unit 33 generates a signal based on the detection signal S 1 ~S 4 , use the following formula (4) to obtain the distance data validity judgment signal S A .

[0092] S A =S 4 +S 3 +S 2 -3S 1 …(4)

[0093] Next, the difference value S is obtained using the following formula (5): 24 .

[0094] S 24 =S 2 -S 4 …(5)

[0095] Next, the distance data validity determination unit 34 determines the distance data validity by A The value and threshold Th 1 Compare and obtain the valid range of data. For example, the threshold Th 1 The distance data validity determination unit 34 obtains the value greater than the threshold value Th 1 The value of the distance data validity judgment signal S A The range is regarded as the valid range of data. Fig. 9 (l) and Fig. 9(o) The lower limit of the data valid range is the period from t = -1 to 0. The upper limit of the data valid range is the period from t = 2 to 3.

[0096] Next, the distance calculation reference signal generation unit 36 ​​obtains the distance calculation reference signal D based on the following equation (6).

[0097] D=1-S 24 / S A …(6)

[0098] The distance calculation reference signal selection unit 37 determines whether the value of the distance calculation reference signal D is within the range determined by the threshold value Th. 2 ,Th 3 In this embodiment, the threshold value Th 2 is 0.01. Threshold Th 3 The predetermined range is 0.01 or more and 1.99 or less. Therefore, the distance calculation reference signal selection unit 37 determines whether the value of the distance calculation reference signal D is 0.01 or more and 1.99 or less (Th 2 ≤D≤Th 3 ). As a result, t=0.21 to 2.19 was obtained as the measurable range.

[0099] The distance image sensor 10A also outputs the control pulse G similarly to the distance image sensor 10. 1 The timing is set to be earlier than the pulse light L P As a result, the distance image sensor 10A can suppress the influence of the light spot.

[0100] [Fourth Embodiment]

[0101] The distance image sensor 10A of the fourth embodiment adopts the 4-tap 2-time window method similarly to the distance image sensor 10A of the third embodiment. The distance image sensor 10A of the third embodiment always provides the control pulse G for measuring the distance. 2 ~G 4 However, the control pulse G 2 ~G 4 It is not always necessary to provide this. For example, when repeatedly executing Figure 8 In the operation shown, the control pulse G may be omitted in part of the operation. 2 ~G 4 In this way, the control pulse G will be omitted 2 ~G 4 The action of is called "thinning action". Fig.10 as well as Fig.11 , a thinning operation performed by the range image sensor 10A according to the fourth embodiment will be described in detail.

[0102] exist Fig.10 (a) to Fig.10 In (g), the control timing of waveform P1 is shown. Fig.10 (h) to Fig.10 In (n), the control timing of waveform P2 is shown. Fig.10 (o) to Fig.10 In (u), the control timing of waveform P3 is shown. Fig.10 (a) to Fig.10 (e) Fig.10 (h) to Fig.10 (l) and Fig.10 (o) to Fig.10 (s) shows the control pulse G 1 ~G 4 , G D Timing. Fig.10 (f) Fig.10 (m) and Fig.10 (t) shows the pulse light L P The timing and incident pulse light L R Timing. Fig.10 (g) Fig.10 (n) and Fig.10 In (u), the stray light L is shown. F Timing.

[0103] like Fig.10 (a) to Fig.10 As shown in (d), the charge forwarding control unit 32 outputs all control pulses G 1 ~G 4 As waveform P1. On the other hand, Fig.10 (h) to Fig.10 As shown in (k), the charge forwarding control unit 32 outputs a control pulse G 1 , G 3 , G 4 As waveform P2. The charge forwarding control unit 32 does not output the control pulse G in waveform P2. 2 The charge forwarding control unit 32 outputs the control pulse G 2 During the period (t = 0 ~ +1), the control pulse G is output D .like Fig.10 (o) to Fig.10 As shown in (r), the charge forwarding control unit 32 outputs a control pulse G 1 , G 4 As waveform P3. The charge forwarding control unit 32 does not output the control pulse G in waveform P3. 2 , G 3 The charge forwarding control unit 32 outputs the control pulse G2 During the period (t = 0 ~ +1), the control pulse G is output D Moreover, the charge forwarding control unit 32 also outputs the control pulse G 3 During the period (t = +1 to +2), the control pulse G is output D .

[0104] For example, Fig.11 As shown in (a), the distance image sensor 10A can also perform an operation including the waveform P1 and the waveform P2. Fig.11 In the example shown in (a), one of the eight waveforms is set as waveform P1, and the remaining seven waveforms are set as waveform P2. In this case, the distance calculation reference signal generating unit 36 ​​obtains the distance calculation reference signal D based on the following equations (7) and (8): 1 , D 2 .

[0105] D 1 =(S 3 -S 1 ) / (S 3 +8S 2- 2S 1 )…(7)

[0106] D 2 =(S 4 -S 1 ) / (S 4 +S 3 -2S 1 )…(8)

[0107] In addition, as another example, Fig.11 As shown in part (b), the distance image sensor 10A may also perform an operation including waveforms P1, P2, and P3. Fig.11 In the example shown in (b), among the eight waveforms, one waveform is set as waveform P1, one waveform is set as waveform P2, and the remaining six waveforms are set as waveform P3. In this case, the distance calculation reference signal generating unit 36 ​​obtains the distance calculation reference signal D based on the following equations (9) and (10): 1 , D 2 .

[0108] D 1 =(4S 3 -S 1 ) / (4S 3 +8S 2 -2S 1 )…(9)

[0109] D 2 =(S 4 -S1 ) / (S 4 +4S 3 -2S 1 )…(10)

[0110] According to the operation of the fourth embodiment, when the object S in the short-distance measurement range is the target, it is possible to prevent the detection signal S from being 1 ~S 4 As a result, the dynamic range of distance measurement can be improved.

[0111] More specifically, according to the operation of the fourth embodiment, when the object S in the short-distance measurement range is the target, it is possible to prevent the detection signal S from being 1 ~S 4 As a result, the dynamic range of distance measurement can be improved. That is, when the object S in the close range is the object, due to the strong incident pulse light L R This causes the number of electrons generated to increase, and therefore it is possible to exceed the saturated number of electrons in the pixel circuit 13. In the operation of the fourth embodiment, during the subframe period of the time window for measuring the close-range area, the number of interval eliminations of the charge based on the time window is reduced. Therefore, the number of electrons generated can be suppressed. As a result, the number of electrons generated in the pixel circuit 13 can be converged to the range of the saturated number of electrons. Therefore, the distance resolution in the case of the object S in the long-range area can be fully improved. Moreover, the distance resolution in the case of the object S in the close-range area can also be maintained at a sufficient value.

[0112] [Fifth Embodiment]

[0113] The distance image sensor 10A of the fifth embodiment adopts a 4-tap 4-time window method. The physical structure of the distance image sensor 10A is the same as that of the distance image sensor 10A of the third embodiment. Furthermore, the distance image sensor 10A of the fifth embodiment may use two or more subframes.

[0114] The control pulse G of the distance image sensor 10A according to the fifth embodiment 1 The method is different from the distance image sensor 10A of the third embodiment. For example, in the third embodiment, the control pulse G 1 It is always output during the period of t = -2 to -1. That is, the control pulse G 1 The timing is 1.

[0115] On the other hand, Fig.12 As shown, in the fifth embodiment, the control pulse G 1 The timing of the control pulse G includes two waveforms P4 and P5. 1It is output during the period of t=0 to +1 (waveform P4: fourth operation). 1 During the period of t=+4 to +5 (waveform P5: third operation), the control pulse G is output. In the operation of the fifth embodiment, the waveforms P4 and P5 can be selectively executed. 1 The timing of is determined by the preset output ratio. Now, a variable K is used as a value representing the output ratio. The variable K represents the number of waveforms.

[0116] For example, as shown in waveform P4, the control pulse G is output during the period t=0 to +1. 1 The ratio is 1 / K. That is, when the variable K is 8, during the period of t=0 to +1, the control pulse G is output only once in 8 times. 1 On the other hand, as shown in waveform P5, the control pulse G is output during t=+4 to +5. 1 The ratio is 1-1 / K. That is, when the variable K is 8, during the period of t = +4 to +5, the control pulse G is output only 7 times out of 8 times. 1 Therefore, as described above, in the fifth embodiment, in one waveform, the control pulse G 1 The control pulse G is not output during both the period of t=0 to +1 and the period of t=+4 to +5. This operation mode can also be said to output the control pulse G during the period of t=0 to +1. 1 The ratio (1 / K) and the control pulse G output during t = +4 to +5 1 The sum of the ratios (1-1 / K) is 1.

[0117] Fig.13 is a timing diagram showing various signals processed by the distance image sensor 10A, and the calculated values ​​with respect to the delay time T D Graph of the changes in . Fig.13 (a) to Fig.13 (g) Based on actual time (t). Fig.13 (h) to Fig.13 (u) with delay time T D As a benchmark. Fig.13 (a) to Fig.13 (e) shows the output control pulse G 1 ~G 4 , G D Timing. Fig.13 (f) shows the pulse light L P The timing and incident pulse light L R Timing. Fig.13 (g) shows the stray light L F Timing. Fig.13 (h) to Fig.13 (k) shows the detection signal S 1 ~S 4 The value of Fig.13 (l) and Fig.13 (m) shows the difference value S 13 , S 24 The value of Fig.13 (n) and Fig.13 In part (o), the distance data validity determination signal S is shown. A1 The value of and distance calculation reference signal D 1 The value of Fig.13 (p) and Fig.13 (q) shows the distance data validity determination signal S A2 The value of and distance calculation reference signal D 2 The value of Fig.13 (r) shows the distance calculation reference signal D 3 The value of Fig.13 (s) shows the comparison signal E 1 ~E 4 The value of Fig.13 In (t), the data valid range is shown. Fig.13 In (u), the measurable range is shown.

[0118] The distance image sensor 10A starts the process of generating the distance image. The light source control unit 31 and the charge transfer control unit 32 of the calculation circuit 12 control the output control pulse G during one frame period. 1 ~G 4 , G D Timing and pulse light L P Specifically, first, the charge forwarding control unit 32 controls the pulse G 1 ~G 4 , G D The duration is set to T 1 =T 0 Next, the charge transfer control unit 32 outputs the control pulse G during either the period from t=0 to +1 or the period from t=+4 to +5. 1 Next, the charge transfer control unit 32 outputs the control pulse G during the period from t=+1 to +2. 2 Next, the charge transfer control unit 32 outputs the control pulse G during the period from t=+2 to +3. 3 Then, the charge transfer control unit 32 outputs the control pulse G during the period from t=+3 to +4. 4 Then, the charge transfer control unit 32 outputs the control pulse G during the period after t=+5.D In addition, the charge forwarding control unit 32 does not output the control pulse G 1 ~G 4 Output control pulse G during D Then, the voltage detection unit 26 of each pixel circuit 13 1 ~26 4 Read out the detection signal S 1 ~S 4 The read detection signal S 1 ~S 4 The voltage is output to the calculation circuit 12 (voltage detection step).

[0119] Next, the calculation circuit 12 calculates the detection signal S output from each pixel circuit 13. 1 ~S 4 , to calculate the distance information of each pixel (distance calculation step). The step of calculating the distance information includes: obtaining the distance data validity determination signal S A1 , S A2 Step: Obtain distance calculation reference signal D 1 , D 2 , D 3 Step: Select the distance calculation reference signal D 1 , D 2 , D 3 Any one of the steps; and a step of obtaining the distance of the object S based on the selected distance calculation reference signal.

[0120] First, obtain the distance data validity judgment signal S A1 , S A2 That is, the distance data validity determination signal generation unit 33 generates the distance data validity determination signal based on the detection signal S 1 ~S 4 , using the following equations (11) to (14), the distance data validity determination signal S is obtained A1 , S A2 .

[0121] S 13 =S 1 -S 3 …(11)

[0122] S 24 =S 2 -S 4 …(12)

[0123] S A1 =K|S 13 |+|S 24 |…(13)

[0124] S A2=(K / (K-1))|S 13 |+|S 24 |…(14)

[0125] Next, the distance data validity determination unit 34 determines the distance data validity by A2 The value and threshold Th 1 For example, the threshold Th 1 is set to near 0. As a result, the distance data validity determination signal S is obtained. A2 With threshold Th 1 The intersection between is taken as the upper and lower limits of the valid range of the data. Fig.13 The range shown by (t) is the “data valid range”.

[0126] Next, the distance calculation reference signal generating unit 36 ​​obtains the distance calculation reference signal D based on the following equations (15) to (17): 1 , D 2 , D 3 .

[0127] D 1 =1-KS 13 / SA 1 …(15)

[0128] D 2 =2-S 24 / S A2 …(16)

[0129] D 3 =3+(K / (K-1))S 13 / S A2 …(17)

[0130] Next, the distance calculation reference signal selection unit 37 selects the distance calculation reference signal D 1 , D 2 , D 3 First, the distance calculation reference signal selection unit 37 uses equations (18) to (21) to obtain the comparison signal E 1 ~E 4 .

[0131] [Mathematical formula 1]

[0132]

[0133] [Mathematical formula 2]

[0134]

[0135] [Mathematical formula 3]

[0136]

[0137] [Formula 4]

[0138]

[0139] Next, the distance calculation reference signal selection unit 37 uses the comparison signal E 1 ~E 4 And the judgment formula shown in formula (22) selects the distance calculation reference signal D 1 , D 2 , D 3 Any one of .

[0140] [Mathematical formula 5]

[0141]

[0142] In equation (22), when the determination result is "100", the distance calculation reference signal selection unit 37 selects the distance calculation reference signal D 1 When the determination result is "010", the distance calculation reference signal selection unit 37 selects the distance calculation reference signal D 2 When the determination result is "001", the distance calculation reference signal selection unit 37 selects the distance calculation reference signal D 3 .

[0143] Then, the distance image generation unit 38 determines that the selected “data valid range” is in the selected “data valid range” and calculates the reference signal D based on the distance selected according to equation (22). 1 , D 2 , D 3 Use any one of the following to calculate the distance of the object S.

[0144] In the first embodiment, as described above, the stray light L F Sometimes the stray light L is generated by the lens or the housing 51. These objects are located at a relatively close distance from the light source 11 and the image sensor 10. Therefore, the stray light L F When the pulse light L is emitted from the light source 11 P There is a high probability that the stray light L will enter the photoelectric conversion region 21 immediately thereafter. F During the period with a high probability of incident light (t = 0 to +1), the output control pulse G is reduced. 1 In the operation of the fifth embodiment, the control pulse G is increased. 4 After that (t=+4~+5) output control pulse G 1 Frequency. In the control pulse G4 After that (t = +4 to +5), due to the pulse light L P Sufficient time has passed since the emission started, so the stray light L F The intensity of the control pulse G 4 The detection signal S obtained during the period after (t = +4 to +5) 1 Suppress stray light L F As a result, the stray light L is suppressed. F The impact of the detection signal S 1 The ratio of L increases, so the calculation result can be suppressed by the stray light L F impact.

[0145] In summary, the operation of the fifth embodiment is to reduce the control pulse G output at the timing corresponding to the short distance. 1 The operation of the fifth embodiment is performed by outputting a control pulse G at a timing corresponding to a long distance. 1 As a result, the light spot component included in the detection signal can be made relatively smaller than the background light component.

[0146] [Sixth Embodiment]

[0147] As in the fifth embodiment, the combined output control pulse G 1 The operation of waveforms with different timings is set to another mode. Originally, in order to suppress stray light L F The waveform P5 is far away from the pulse light L P The timing time (t = +4 to +5) of the generated control pulse G is set 1 As described in the second embodiment, when the measurement range is a medium distance or a long distance, the control pulse G is output. 1 ~G 4 The timing is later than the pulse light L P That is, when the medium and long distances are measured, even if the control pulse G 1 ~G 4 The order of output, stray light L F The detection signal S 1 The impact is also relatively small.

[0148] Fig.14 The operation of the distance image sensor according to the sixth embodiment is shown in FIG. Fig.14 (a)~ Fig.14 (f) shows the operation of the first subframe. Fig.14 (h)~ Fig.14(m) shows the operation of the second subframe. Fig.14 (o)~ Fig.14 (t) shows the action of the third subframe. Fig.14 The "L" shown is an integer. 1 、N 2 、N 3 are the pulse light L in the subframe P The number of repetitions.

[0149] The action of the first subframe includes waveform P4 a For example, when the pulse light L P When the K times of the two action waveforms are repeated as one cycle, waveform P4 a Waveform P4 is performed once, while waveform P5 is performed (K-1) times. a During the period t=0~+1, the control pulse G is output. 1 According to waveform P4 a The action is controlled by the control pulse G 1 The obtained result can be used as information for background light correction and distance measurement. P The period is repeated, so there may also be a control pulse G 1 The result includes stray light L F On the other hand, waveform P5 outputs control pulse G during t = +4 to +5. 1 During this period, the output control pulse G 2 ~G 4 That is, since the pulse light L is sufficiently far away from the P The timing of stray light L F Therefore, according to the operation of waveform P5, information that can accurately correct the background light can be obtained. In addition, according to the operation of the first subframe, the incident pulse light L input during the period of t = 0 to +4 can be captured. R .

[0150] The action of the second subframe only includes waveform P4 b . Control pulse G 1 It is output during the period of t=+3 to +4. Control pulse G 2 Output during t = +4 to +5. Control pulse G 3 Output during t = +5 to +6. Control pulse G 4 It is output during the period of t=+6 to +7. 1 ~G 4 During the period, the output control pulse G DAccording to the operation of the second subframe, the incident pulse light L input during the period of t=+3 to +7 can be captured. R .

[0151] The action of the third subframe only includes waveform P4 c . Control pulse G 1 It is output during t=+6 to +7. Control pulse G 2 Output during t = +7 to +8. Control pulse G 3 Output during t = +8 to +9. Control pulse G 4 It is output during the period of t = +9 to +10. 1 ~G 4 During the period, the output control pulse G D According to the operation of the second subframe, the incident pulse light L input during the period of t=+6 to +10 can be captured. R .

[0152] That is, when the short distance is used as the measurement range, the pulse light L P When K times is set as one cycle and two action waveforms are repeated, one time can be used as waveform P4. a The remaining (K-1) times are taken as waveform P5. Then, with the middle distance as the measurement range, waveform P4 is repeated. b When the measurement range is long, repeat the waveform P4. b .

[0153] [Seventh Embodiment]

[0154] Reference Fig.15 as well as Fig.16 The operation of the distance image sensor 10A according to the seventh embodiment will be described. The operation of the distance image sensor 10A can also be the following operation: when the control pulse G is output during the period t=0 to +1, 1 The ratio of the control pulse G output during t = +4 to +5 1 When the ratios of are added, the ratio is not 1.

[0155] Specifically, if Fig.15 As shown, the operation of the seventh embodiment includes: outputting the control pulse G only during the period t=+4 to +5 1 In the case of (waveform P5: first action), the control pulse G is output during both the period of t = 0 to +1 and the period of t = +4 to +5. 1 (waveform P6: second action). The period t = 0 to +1 refers to the period between the pulse light L P The timing of the generation is simultaneous with or after the output control pulse G2 ~G 4 According to waveforms P5 and P6, the control pulse G is always output during the period of t = +4 to +5. 1 In other words, the control pulse G is output during the period of t = +4 to +5. 1 The ratio is 1. On the other hand, the control pulse G is output during the period t = 0 to +1. 1 The ratio is 1 / K. Therefore, in the operation of the seventh embodiment, the control pulse G is output during the period from t=0 to +1. 1 The ratio (1 / K) and the control pulse G output during t = +4 to +5 1 The sum of the ratios (1) is 1+1 / K, not 1.

[0156] like Fig.16 As shown, in the operation of the seventh embodiment, the reference signal D for distance calculation is 1 , D 2 , D 3 And the distance data validity determination signal S A1 , S A2 The calculation formula of is different from that of the sixth embodiment. The other operations of the seventh embodiment are the same as those of the sixth embodiment. In the seventh embodiment, the following formulas (23) to (27) are used to calculate the distance reference signal D 1 , D 2 , D 3 .

[0157] S A1 =K|S 13 |+|S 24 |…(23)

[0158] S A2 =|S 13 |+|S 24 |…(24)

[0159] D 1 =1-KS 13 / S A1 …(25)

[0160] D 2 =2-S 24 / S A2 …(26)

[0161] D 3 =3+S 13 / S A2 …(27)

[0162] According to the operation of the seventh embodiment, when the stray light L FThe detection signal S is obtained during the period (t = +4 to +5) when the influence of 1 The frequency of is further increased. As a result, the detection signal S that can be used for distance calculation 1 The number of increases can further improve the accuracy of distance measurement.

[0163] [Eighth Embodiment]

[0164] The distance image sensor 10A of the third embodiment adopts a 4-tap 2-time window method. For example, the distance image sensor of the eighth embodiment may also adopt a 5-tap 3-time window method. The distance image sensor of the eighth embodiment has five charge readout regions, five control electrodes, and five voltage detection units as a physical structure.

[0165] Fig.17 is a timing chart showing various signals processed by the distance image sensor of the eighth embodiment, and various values ​​calculated with respect to the delay time T D Graph of the changes in . Fig.17 (a) to Fig.17 (h) is based on actual time (t). Fig.17 (i) to Fig.17 (t) with delay time T D As a benchmark. Fig.17 (a) to Fig.17 (f) shows the output control pulse G 1 ~G 5 , G D Timing. Fig.17 (g) shows the pulse light L P The timing of the incident pulse light L R Timing. Fig.17 (h) shows the stray light L F Timing. Fig.17 (i) to Fig.17 (m) shows the detection signal S 1 ~S 5 The value of Fig.17 (n) shows the distance data validity determination signal S A The value of Fig.17 In (o), the difference value S is shown. 24 The value of Fig.17 In (p), the difference value S is shown. 35 The value of Fig.17 (q) shows the distance calculation reference signal D 2 The value of Fig.17 (r) shows the distance calculation reference signal D 3 The value of Fig.17 In (s), the data valid range is shown. Fig.17 In (t), the measurable range is shown.

[0166] The distance image sensor starts the process of generating the distance image. The light source control unit 31 and the charge transfer control unit 32 of the calculation circuit 12 control the output control pulse G during one frame period. 1 ~G 5 , G D The timing and pulse light L P Specifically, first, the charge forwarding control unit 32 controls the pulse G 1 ~G 5 , G D The duration is set to T 1 =T 0 Next, the charge transfer control unit 32 outputs the control pulse G during the period from t=-2 to -1. 1 Next, the charge transfer control unit 32 outputs the control pulse G during the period from t=-1 to 0. D Next, the charge transfer control unit 32 outputs the control pulse G during the period from t=0 to +1. 2 Next, the charge transfer control unit 32 outputs the control pulse G during the period t=+1 to +2. 3 Next, the charge transfer control unit 32 outputs the control pulse G during the period t=+2 to +3. 4 Next, the charge transfer control unit 32 outputs the control pulse G during the period t=+3 to +4. 5 Then, the charge transfer control unit 32 outputs the control pulse G after t=+4. D Then, the voltage detection unit of the pixel circuit 13 reads out the detection signal S 1 ~S 5 The read detection signal S 1 ~S 5 The voltage is output to the calculation circuit 12 (voltage detection step).

[0167] Next, the calculation circuit 12 calculates the detection signal S output from each pixel circuit 13. 1 ~S 5 , to calculate the distance information of each pixel (distance calculation step). The step of calculating the distance information includes: obtaining the distance data validity determination signal S A Steps; Get the difference value S 24 , S 35 Step: Obtain distance calculation reference signal D 2 , D 3step; a step of determining a valid range of data; a step of determining a measurable range; and a step of obtaining a distance of an object S.

[0168] First, obtain the distance data validity judgment signal S A The distance data validity determination signal generation unit 33 generates a signal based on the detection signal S 1 ~S 5 , use the following formula (28) to obtain the distance data validity judgment signal S A .

[0169] S A =S 5 +S 4 +S 3 +S 2 -4S 1 …(28)

[0170] Next, the distance data validity determination unit 34 determines the distance data validity by A The value and threshold Th 1 Compare and obtain the valid range of data. For example, the threshold Th 1 The distance data validity determination unit 34 obtains the value greater than the threshold value Th 1 The value of the distance data validity judgment signal S A The range is regarded as the valid range of data. Fig.17 (l) and Fig.17 (o) The lower limit of the data valid range is between t = -1 and 0. In addition, the upper limit of the data valid range is between t = +3 and +4.

[0171] Next, the difference value S is obtained using the following equations (29) and (30): 24 , S 35 .

[0172] S 24 =S 2 -S 4 …(29)

[0173] S 35 =S 3 -S 5 …(30)

[0174] Next, the distance calculation reference signal generation unit 36 ​​obtains the distance calculation reference signal D based on the following equations (31) and (32): 2 , D 3 .

[0175] D 2 =1-S 24 / S A …(31)

[0176] D 3 =2-S 35 / S A …(32)

[0177] The distance calculation reference signal selection unit 37 uses the threshold value Th 2 ,Th 3 ,Th 4 ,Th 5 , select the distance calculation reference signal D 2 , D 3 Specifically, when Th is satisfied 2 ≤D 2 ≤Th 3 When the distance calculation reference signal selection unit 37 selects the distance calculation reference signal D 2 On the other hand, in satisfying Th 4 ≤D 3 ≤Th 5 When the distance calculation reference signal selection unit 37 selects the distance calculation reference signal D 3 .

[0178] For example, at the threshold Th 2 ,Th 3 ,Th 4 ,Th 5 When the following numerical values ​​are obtained, t=0.21 to 3.19 is obtained as the measurable range.

[0179] Threshold Th 2 : 0.01

[0180] Threshold Th 3 :1.5

[0181] Threshold Th 4 :1.5

[0182] Threshold Th 5 :2.99

[0183] The distance image sensor according to the eighth embodiment can also achieve the same effects as those of the distance image sensors 10 and 10A according to the first and third embodiments.

[0184] The distance image sensors 10 and 10A are described above, but the distance image sensors 10 and 10A are not limited to the above-described embodiments, and can be implemented in various forms.

[0185] In order to reduce the influence of light spots, the distance image sensor 10, 10A adopts at least one of two actions. The first action is to correct the background light and convert the detection signal S 1Set to be away from the pulse light L in time P The second action is to adjust the detection signal S to correct the background light. 1 That is, reduce the number of control pulses G output at the timing that is easily affected by the light spot. 1 The number of times the control pulse G is outputted is increased at a timing that is less susceptible to the influence of the light spot. 1 The number of times.

[0186] When the distance image sensor 10 or 10A operates, it only needs to include one or both of the first operation and the second operation. Therefore, the physical structure of the distance image sensor 10 or 10A is not particularly limited, as long as it has a so-called three-tap structure or more. In addition, according to the specifications of the distance image sensor 10 or 10A, methods such as subframe operation, thinning operation, and multi-zone operation or a combination thereof can be appropriately adopted.

[0187] Description of reference numerals:

[0188] 10: distance image sensor (distance image measuring device), 11: light source, 12: calculation circuit, 13: pixel circuit (pixel circuit unit), 21: photoelectric conversion area, 22 1 ~22 4 : Charge readout area, 24 1 ~24 4 :Control electrode, 26 1 ~26 4 : voltage detection unit, 31: light source control unit, 32: charge forwarding control unit, 33: distance data validity determination signal generation unit, 34: distance data validity determination unit, 35: invalid pixel identification value generation unit, 36: distance calculation reference signal generation unit, 37: distance calculation reference signal selection unit, 38: distance image generation unit, G 1 : Control pulse (first control pulse), G 2 : Control pulse (second control pulse), G 3 : Control pulse (third control pulse), G 4 : Control pulse (fourth control pulse), G 5 : Control pulse (fifth control pulse), G D : Control pulse (M+1th control pulse), S 1 : Detection signal (first detection signal), S 2 : Detection signal (second detection signal), S 3 : Detection signal (third detection signal), S 4 : Detection signal (fourth detection signal), S 5 : Detection signal (fifth detection signal).

Claims

1. A distance image measuring device, wherein: have: A light source, generating pulsed light; A light source control unit, configured to control the light source in such a manner as to periodically and repeatedly generate the pulsed light; The pixel circuit portion includes: a photoelectric conversion region that converts light into electric charges; First to Mth charge readout regions, which are arranged close to and separated from the photoelectric conversion region; a charge discharge region for discharging the charge; first to Mth control electrodes, which are arranged corresponding to the photoelectric conversion region and the first to Mth charge readout regions, and receive first to Mth control pulses for forwarding the charge between the photoelectric conversion region and the first to Mth charge readout regions; and an M+1th control electrode, which is disposed corresponding to the photoelectric conversion region and the charge discharge region and receives an M+1th control pulse for charge forwarding between the photoelectric conversion region and the charge discharge region; a charge forwarding control unit, which outputs the first to Mth control pulses to the first to Mth control electrodes corresponding to the generation of the pulse light by the light source control unit, and outputs the M+1th control pulse to the M+1th control electrode during a period other than an output period of the first to Mth control pulses; a voltage detection unit, which reads out voltages of the first to Mth charge readout regions of the pixel circuit portion as first to Mth detection signals after the charge forwarding control unit outputs the first to Mth control pulses; as well as a distance calculation unit, which repeatedly calculates the distance based on the first to Mth detection signals, The charge forwarding control unit performs the following actions: a first action, setting the timing of outputting the first control pulse to at least one of before the timing of generating the pulse light and after outputting the second to Mth control pulses and before outputting the next pulse light generated after the second to Mth control pulses; and a second action, setting the timing of outputting the first control pulse after outputting the second to M-th control pulses and before the next pulse light generated after outputting the second to M-th control pulses, and setting the timing of outputting the first control pulse at the same time as or after the timing of generating the pulse light and before outputting the second to M-th control pulses, When two action waveforms are repeated with K times of the pulse light as one cycle, the total number of times the waveform is executed in each of the two action waveforms is represented by a variable K. The charge forwarding control unit performs the first action K-1 times and the second action 1 time in the waveform performed K times, The distance calculation unit multiplies the calculated value including the signal obtained by the first control pulse by the variable K as a weight coefficient in the distance calculation. M is an integer greater than or equal to 3.

2. The distance image measuring device according to claim 1, wherein: The charge forwarding control unit sets the timing of the second to M-th control pulses in a manner that the delay time of the second to M-th control pulses relative to the timing of the pulse light generation is shifted to different times for each of the first to N-th subframe periods included in one frame period, N is an integer greater than or equal to 2.

3. The distance image measuring device according to claim 2, wherein: The charge forwarding control unit is set to reduce the ratio of the output number of at least the second control pulse among the second to Mth control pulses as the delay time of the second to Mth control pulses relative to the timing of the pulse light generation is shorter between the second to Nth subframes.

4. The distance image measuring device according to any one of claims 1 to 3, wherein: The charge transfer control unit sets the timing of outputting the second control pulse to a time after a waiting time shorter than a duration of the pulse light has passed since the timing of the pulse light generation.

5. A distance image measuring device, wherein: have: A light source, generating pulsed light; A light source control unit, configured to control the light source in such a manner as to periodically and repeatedly generate the pulsed light; The pixel circuit portion includes: a photoelectric conversion region that converts light into electric charges; First to Mth charge readout regions, which are arranged close to and separated from the photoelectric conversion region; a charge discharge region for discharging the charge; first to Mth control electrodes, which are arranged corresponding to the photoelectric conversion region and the first to Mth charge readout regions, and receive first to Mth control pulses for forwarding the charge between the photoelectric conversion region and the first to Mth charge readout regions; and an M+1th control electrode, which is disposed corresponding to the photoelectric conversion region and the charge discharge region and receives an M+1th control pulse for charge forwarding between the photoelectric conversion region and the charge discharge region; a charge forwarding control unit, which outputs the first to Mth control pulses to the first to Mth control electrodes corresponding to the generation of the pulse light by the light source control unit, and outputs the M+1th control pulse to the M+1th control electrode during a period other than an output period of the first to Mth control pulses; a voltage detection unit, which reads out voltages of the first to Mth charge readout regions of the pixel circuit portion as first to Mth detection signals after the charge forwarding control unit outputs the first to Mth control pulses; as well as a distance calculation unit, which repeatedly calculates the distance based on the first to Mth detection signals, The charge forwarding control unit performs the following actions: a third action, setting the timing of outputting the first control pulse to at least one of before the timing of generating the pulse light and after outputting the second to Mth control pulses and before outputting the next pulse light generated after the second to Mth control pulses; and a fourth action, setting the timing of outputting the first control pulse to be simultaneously with or after the timing of generating the pulse light and before outputting the second to Mth control pulses, When the pulse light is repeated K times as one cycle to repeat the two action waveforms of the third action and the fourth action, the total number of times the waveform is executed in each of the two action waveforms is represented by the variable K. The charge forwarding control unit performs the third action K-1 times and the fourth action 1 time in the waveform performed K times, The distance calculation unit multiplies the calculated value including the signal obtained by the first control pulse by the variable K as a weight coefficient in the distance calculation. M is an integer greater than or equal to 3.

6. The distance image measuring device according to claim 5, wherein: The charge forwarding control unit is set so that the delay time of the second to Mth control pulses in the third action relative to the timing of the pulse light generation is the same as the delay time in the fourth action, and the number of times the third action is performed in one frame is greater than the number of times the fourth action is performed.

Images