Imaging device and difference information calculation method

Abstract

A camera (1) includes an emission section (4) that irradiates light to an object, and a solid-state camera (2) that has a plurality of pixel sections (100) that convert received light into signal charges, and a signal processing section (20) that calculates distance information that shows a distance to the object, the signal processing section (20) calculating the distance information for each of the plurality of pixel sections (100) by a TOF (Time of Flight) method using a plurality of signal charges converted in the plurality of pixel sections (100) in a case where reflected light reflected by the object is received by at least a part of the plurality of pixel sections (100) by the emission section (4), and calculating difference information that shows an amount of difference relating to the distance shown by the distance information for one or more pixel sections (100) of at least a part of the plurality of pixel sections (100).

Description

TECHNICAL FIELD

[0001] The present disclosure relates to an imaging device that obtains distance information of a subject. BACKGROUND

[0002] Conventionally, there is known an imaging device that performs distance measurement using a TOF (Time Of Flight) method and calculates distance information that shows a distance to a subject (see, for example, Patent Literature 1).

[0003] (Patent Literature)

[0004] (Patent Literature)

[0005] Patent Literature 1: Japanese Patent Application Publication No. 2018-136123 SUMMARY

[0006] PROBLEMS TO BE SOLVED BY THE INVENTION

[0007] In processing that is performed using a distance shown by calculated distance information, there are some processes that can obtain more appropriate results if the reliability of the distance can be discriminated (for example, filtering processing performed on a distance map, and the like).

[0008] Therefore, an object of the present disclosure is to provide an imaging device and the like that can discriminate the reliability of a distance shown by calculated distance information.

[0009] MEANS FOR SOLVING THE PROBLEMS

[0010] One aspect of the present disclosure relates to an imaging device including: a light emitting section that irradiates a subject with light; and a solid-state imaging device that has a plurality of pixel sections each of which converts received light into signal charge and a signal processing section that calculates distance information that shows a distance to the subject, wherein the signal processing section calculates the distance information for each of the plurality of pixel sections using a TOF (Time Of Flight) method using a plurality of the signal charges converted in the plurality of pixel sections in a case where reflected light reflected by the subject from the light irradiated by the light emitting section is received by at least a part of the plurality of pixel sections, and calculates difference information that shows an amount of difference related to the distance shown by the distance information for one or more pixel sections of at least a part of the plurality of pixel sections.

[0011] One aspect of the present disclosure relates to a difference information calculating method, which is a difference information calculating method performed by an imaging device including an emission unit configured to emit light to an object, and a solid-state imaging device including a plurality of pixel units each configured to convert received light into signal charge, and a signal processing unit configured to calculate distance information indicating a distance to the object, the difference information calculating method including: a first step of calculating, by the signal processing unit, the distance information for each of the plurality of pixel units by using a plurality of the signal charge converted in the plurality of pixel units by a TOF method in a case where reflected light, which is light emitted by the emission unit and reflected by the object, is received by at least a part of the plurality of pixel units; and a second step of calculating, by the signal processing unit, difference information indicating an amount of difference with respect to the distance information for one or more pixel units of at least a part of the plurality of pixel units in a case where the reflected light is received by at least a part of the plurality of pixel units.

[0012] Effects of Invention

[0013] The present disclosure provides an imaging device or the like capable of determining a reliability of a distance indicated by calculated distance information. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 is a block diagram illustrating a configuration example of an imaging device according to Embodiment 1.

[0015] Figure 2 is a block diagram illustrating a configuration example of a signal processing unit according to Embodiment 1.

[0016] Figure 3 is a block diagram illustrating a configuration example of a pixel unit according to Embodiment 1.

[0017] Figure 4 is a block diagram illustrating a configuration example of a pixel unit according to Embodiment 1.

[0018] Figure 5 is a timing chart of a first exposure sequence according to Embodiment 1.

[0019] Figure 6 is a timing chart of a first exposure sequence according to Embodiment 1.

[0020] Figure 7 is a timing chart of a second exposure sequence according to Embodiment 1.

[0021] Figure 8 is a block diagram illustrating a configuration example of an imaging device according to Embodiment 2.

[0022] Figure 9 FIG. 2 is a block diagram showing a configuration example of a signal processing section according to Embodiment 2. DETAILED DESCRIPTION

[0023] A configuration according to the present disclosure is a camera device including: an emission section that emits light to an object; and a solid-state camera device that has a plurality of pixel sections each of which converts received light into signal charge and a signal processing section that calculates distance information indicating a distance to the object, wherein the signal processing section calculates the distance information for each of the plurality of pixel sections by a TOF (Time of Flight) method using a plurality of the signal charges converted in the plurality of pixel sections in a case where at least a part of the plurality of pixel sections receives reflected light reflected by the object from the light emitted by the emission section, and calculates difference information indicating an amount of difference in the distance indicated by the distance information for one or more of the at least a part of the plurality of pixel sections.

[0024] With the camera device according to the above-described configuration, the difference information indicating the amount of difference in the distance indicated by the calculated distance information is calculated. By using the difference information, it is possible to determine the reliability of the distance indicated by the distance information. As described above, with the camera device according to the above-described configuration, it is possible to determine the reliability of the distance indicated by the calculated distance information.

[0025] Further, the plurality of pixel sections can constitute a pixel array configured in a matrix shape, and the signal processing section can calculate the difference information for each of the plurality of pixel sections.

[0026] Further, the difference information can indicate a standard deviation of the distance to the object indicated by the distance information.

[0027] Further, the signal processing section can calculate, for each of the plurality of pixel sections, an emission component light amount indicating a light component of the reflected light within the signal charge of the pixel, and calculate the distance information for the pixel section in accordance with the emission component light amount, and the difference information can indicate a standard deviation of the emission component light amount.

[0028] Further, the light emitting section can emit light in a pulse shape as the light, and the signal processing section can calculate the distance information for each of the plurality of pixel sections based on first signal charges that are converted by each of the plurality of pixel sections from light received in a first period before the light in the pulse shape stops being emitted and second signal charges that are converted by each of the plurality of pixel sections from light received in a second period that is a period following the first period, and a time interval from the start of the first period to the end of the second period is longer than a period in which the light in the pulse shape is emitted.

[0029] One aspect of the present disclosure relates to a difference information calculating method. The difference information calculating method is a difference information calculating method performed by an imaging device that includes a light emitting section configured to emit light toward a subject and a solid-state imaging device including a plurality of pixel sections each configured to convert received light into a signal charge and a signal processing section configured to calculate distance information indicating a distance to the subject. The difference information calculating method includes a first step of calculating, by the signal processing section, the distance information for each of the plurality of pixel sections using a TOF method using a plurality of the signal charges converted in the plurality of pixel sections in a case where reflected light reflected by the subject is received by at least a part of the plurality of pixel sections, and a second step of calculating, by the signal processing section, difference information indicating an amount of difference with respect to the distance information for one or more of the at least a part of the plurality of pixel sections in a case where the reflected light is received by the at least a part of the plurality of pixel sections.

[0030] The difference information calculating method calculates difference information indicating an amount of difference with respect to a distance indicated by calculated distance information. By using the difference information, it is possible to determine a reliability of a distance indicated by distance information. As described above, the difference information calculating method makes it possible to determine a reliability of a distance indicated by calculated distance information.

[0031] A specific example of an imaging device according to one aspect of the present disclosure will be described below with reference to the drawings. The embodiments shown in the drawings are merely one example of the present disclosure. Therefore, the numerical values, shapes, constituent elements, arrangement of constituent elements, and connection forms shown in the following embodiments are one example, and the gist is not limited to the present disclosure. In addition, each drawing is a schematic view, and is not a strict illustration.

[0032] (Embodiment 1)

[0033] The following describes the imaging device according to Embodiment 1. The imaging device performs distance measurement by a TOF method, which is a method of performing distance measurement using the time of flight of light to an object. The imaging device is sometimes referred to as a range imaging device because it measures the distance to an object.

[0034] Figure 1 is a block diagram showing a configuration example of the imaging device 1 according to Embodiment 1.

[0035] As shown in Figure 1 , the imaging device 1 includes a solid-state imaging device 2, a control section 3, and a light emitting section 4.

[0036] The control section 3 outputs a light emitting signal and an exposure signal. The light emitting signal is a signal indicating the irradiation of light to an object. The exposure signal is a signal indicating the exposure of background light, which is light generated based on reflected light from the object and sunlight or the like. The control section 3 is implemented by, for example, a memory and a processor that executes a program stored in the memory.

[0037] The light emitting section 4 has a light emitting element that irradiates pulsed light to an object in accordance with the light emitting signal output from the control section 3. The light emitting element is implemented by, for example, a laser diode, a vertical cavity surface emitting laser (VCSEL), a light emitting diode (LED), or the like. The irradiated light is, for example, infrared light.

[0038] The solid-state imaging device 2 has an imaging section 10 and a signal processing section 20. The solid-state imaging device 2 is implemented by, for example, a CMOS image sensor.

[0039] The imaging section 10 has a pixel array 30 that is configured in an array shape by a plurality of pixel sections 100 Figure 1 (not shown in the drawing). The plurality of pixel sections each convert received light into signal charge. The pixel array 30 is implemented by, for example, a CCD image sensor. Figure 3 Figure 4 The imaging section 10 outputs a plurality of pixel signals corresponding to each of the plurality of pixel sections 100 that configure the pixel array 30 when the pixel array 30 is exposed.

[0040] The pixel array 30 receives reflected light that is reflected by an object from the light irradiated by the light emitting section 4. In addition, the pixel array 30 also receives background light based on sunlight or the like. The pixel array 30 is exposed in accordance with the exposure signal output from the control section 3.

[0041] The imaging section 10 outputs a plurality of pixel signals corresponding to each of the plurality of pixel sections 100 that configure the pixel array 30 when the pixel array 30 is exposed.

[0042] ​Figure 2 is a block diagram showing a configuration example of the signal processing section 20.

[0043] As shown in Figure 2 , the signal processing section 20 includes a coefficient decision section 201, a distance calculation section 202, and a distance difference calculation section 203. The signal processing section 20 is realized by, for example, a logic circuit. Or it is realized by, for example, a memory and a processor that executes a program stored in the memory.

[0044] The coefficient decision section 201 decides a coefficient that is multiplied by a plurality of pixel signals output from the imaging section 10. How the coefficient decision section 201 decides the coefficient will be described later.

[0045] The distance calculation section 202 calculates, for each of a plurality of pixel sections, a distance signal that shows a distance to an object, based on the plurality of pixel signals output from the imaging section 10 and the coefficient decided by the coefficient decision section 201. How the distance calculation section 202 calculates the distance signal will be described later.

[0046] The distance difference calculation section 203 calculates a distance difference signal that shows an amount of difference in the distance shown by the distance signal, based on the plurality of pixel signals output from the imaging section 10 and the coefficient decided by the coefficient decision section 201. How the distance difference calculation section 203 calculates the distance difference signal will be described later.

[0047] Figure 3 is a block diagram showing a configuration example of the imaging section 10.

[0048] Figure 3 As shown in

[0049] The pixel array 30 includes a plurality of pixel sections 100 arranged in a matrix shape, and a plurality of vertical signal lines 16 arranged per column.

[0050] In the pixel array 30, the transfer channels 104 (to be described later) included in the pixel sections 100 arranged in the column direction are connected in a straight line in the column direction, thereby forming a transfer channel 17 per column.

[0051] Figure 4 is a mode diagram showing a configuration example of the pixel section 100.

[0052] As shown in Figure 4As shown, the pixel section 100 includes a photoelectric conversion section 101, a plurality of charge accumulation sections 102 (for example, a first charge accumulation section 102a, a second charge accumulation section 102b, and a third charge accumulation section 102c), a plurality of readout gates 106 (for example, a first readout gate 106a, a second readout gate 106b, and a third readout gate 106c), an output control gate 113, a floating diffusion layer 114, a reset gate 115, a reset drain 116, a readout circuit 117, a plurality of exposure control gates 108 (for example, an exposure control gate 108a and an exposure control gate 108b), and a plurality of overflow drains 109 (for example, an overflow drain 109a and an overflow drain 109b).

[0053] The photoelectric conversion section 101 converts received light into signal charges.

[0054] The readout gate 106 reads out the signal charges from the photoelectric conversion section 101.

[0055] The charge accumulation section 102 accumulates the signal charges read out from the readout gate 106. The charge accumulation section 102 is configured by a transfer channel 104 (CCD channel 104) for transferring the signal charges under a gate insulating film and a transfer electrode 105 (for example, any one of a transfer electrode 105a, a transfer electrode 105b, a transfer electrode 105c, a transfer electrode 105d, a transfer electrode 105e, and a transfer electrode 105f) on the gate insulating film. That is, as shown in FIG. 1, the charge accumulation section 102 includes a portion of the transfer channel 104 and a portion of the transfer electrode 105 overlapping the portion of the transfer channel 104 in a plan view of the semiconductor substrate. Figure 4 As shown, the charge accumulation section 102 includes a portion of the transfer channel 104 and a portion of the transfer electrode 105 overlapping the portion of the transfer channel 104 in a plan view of the semiconductor substrate. Further, as shown in FIG. 1, the charge accumulation section 102 includes a portion of the transfer channel 104 and a portion of the transfer electrode 105 overlapping the portion of the transfer channel 104 in a plan view of the semiconductor substrate. Figure 4 As shown, the charge accumulation section 102 includes a portion of the transfer channel 104 and a portion of the transfer electrode 105 overlapping the portion of the transfer channel 104 in a plan view of the semiconductor substrate. Further, as shown in FIG. 1, the charge accumulation section 102 includes a portion of the transfer channel 104 and a portion of the transfer electrode 105 overlapping the portion of the transfer channel 104 in a plan view of the semiconductor substrate.

[0056] In the present embodiment, the voltage to be applied to the transfer electrode 105a, the transfer electrode 105b, the transfer electrode 105c, the transfer electrode 105d, the transfer electrode 105e, and the transfer electrode 105f is set to VG1, VG2, VG3, VG4, VG5, and VG6, respectively.

[0057] The first charge accumulation section 102a, the second charge accumulation section 102b, and the third charge accumulation section 102c are subjected to six-phase driving.

[0058] When the high voltages VG1, VG3, and VG5 are applied to the first, second, and third charge accumulation sections 102a, 102b, and 102c from the pixel array control section 11, they are respectively formed at positions adjacent to the first, second, and third readout gates 106a, 106b, and 102c, respectively, and below the transfer electrode 105 in the depth direction (here, below the transfer electrodes 105a, 105c, and 105e in the depth direction).

[0059] The overflow drain 109 is used to discharge signal charges from the photoelectric conversion unit 101 .

[0060] The exposure control gate 108 controls the discharge to the overflow drain 109 .

[0061] The floating diffusion layer 114 receives and holds the signal charge stored in one of the plurality of charge storage units 102 (here, the first charge storage unit 102a, the second charge storage unit 102b, and the third charge storage unit 102c).

[0062] The output control gate 113 controls the transmission to the floating diffusion layer 114 .

[0063] The readout circuit 117 converts the signal charge held in the floating diffusion layer 114 into a voltage and reads it from the pixel portion 100 to the vertical signal line 16. For example, the readout circuit 117 includes a source-follower transistor having a gate connected to the floating diffusion layer 114 and a select transistor connected in series with the source-follower transistor. For example, when the select transistor selects the readout circuit 117, the signal charge held in the floating diffusion layer 114 is converted into a voltage signal by the readout circuit 117 and read out to the vertical signal line 16.

[0064] From the pixel array control section 11, a drive pulse ODG is applied to the exposure control gate 108a and the exposure control gate 108b, drive pulses TG1, TG2, and TG3 are applied to the 1st readout gate 106a, the 2nd readout gate 106b, and the 3rd readout gate 106c, and drive pulses VG1 to VG6 are applied to the transfer electrodes 105a to 105f, respectively. At the time of exposure, a High voltage is applied to VG1 and VG3 and VG5, and a Low voltage is applied to the other electrodes, so that electric charges can be accumulated under the transfer electrodes 105 to which the High voltage is applied. That is, the charge accumulation sections 102 (here, the 1st charge accumulation section 102a, the 2nd charge accumulation section 102b, and the 3rd charge accumulation section 102c) are formed by the transfer electrodes 105 (here, the transfer electrode 105a, the transfer electrode 105c, and the transfer electrode 105e) to which the High voltage is applied and the transfer channels 104 overlapping thereunder.

[0065] As an initial state, ODG is in a High state, and the photoelectric conversion section 101 is in a reset state. Further, the 1st readout gate 106a and the 2nd readout gate 106b and the 3rd readout gate 106c are in a Low state (a state in which they are not activated), and the transfer electrode 105a and the transfer electrode 105c and the transfer electrode 105e are held in the 1st charge accumulation section 102a and the 2nd charge accumulation section 102b and the 3rd charge accumulation section 102c in a High state (a state in which they are activated), and the photoelectric conversion section 101 is in a state in which it is electrically cut off. In this state, signal charges generated in the photoelectric conversion section 101 are discharged to the overflow drain 109 via the exposure control gate 108.

[0066] Returning to Figure 3 , the structure of the imaging section 10 will be further described.

[0067] The pixel array control section 11 controls the plurality of pixel sections 100 constituting the pixel array 30 in accordance with an exposure signal output from the control section 3.

[0068] The vertical scanning section 12 scans signal charges read from the plurality of pixel sections 100 constituting the pixel array 30 in units of rows, in other words, sequentially selects each row, and outputs to the plurality of vertical signal lines 16 arranged for each column.

[0069] The column processing section 13 receives signal charges output to the plurality of vertical signal lines 16, performs correlated double sampling (CDS), and outputs as a pixel signal.

[0070] The horizontal scanning section 14 scans the pixel signal output from the column processing section 13, in other words, selects each one in turn and outputs it. The column processing section 13 can have an A / D conversion circuit for conversion to a digital signal for each column of a plurality of vertical signal lines 16.

[0071] The output buffer 15 outputs the pixel signal received from the horizontal scanning section 14.

[0072] The pixel section 100 having the above structure is a unit cell that outputs a signal required to calculate one distance signal.

[0073] The operation of the imaging device 1 having the above structure will be described below with reference to the drawings.

[0074] Figure 5 、 Figure 6 is a timing chart showing the light emission timing of the light emitting section 4, the exposure of the pixel section 100 and the signal accumulation timing, and the exposure state of the signal charge accumulated to the charge accumulation section 102 via the first readout gate 106a and the second readout gate 106b and the third readout gate 106c in the first exposure sequence.

[0075] The first exposure sequence is composed of a first exposure period, a second exposure period, and a third exposure period. The first exposure period is a period synchronized with the time Tp from the start to the end of the irradiation of the pulsed light of the light emitting section 4 controlled by the control section 3. The second exposure period is a period from the end of the irradiation of the pulsed light until the time Tp elapses. The third exposure period is a period from the end of the second exposure period until the time Tp elapses.

[0076] At the start of the first exposure sequence, pulsed light is irradiated from the light emitting section 4 at intervals of the time Tp by the instruction of the control section 3. The reflected light of the pulsed light irradiated and reflected by the subject reaches the pixel section 100 after a delay of the time Td corresponding to the distance from the imaging device 1, and is converted to signal charge at the photoelectric conversion section 101.

[0077] Figure 5 is a timing chart in the case of example 1 in which the pulsed reflected light is received by the photoelectric conversion section 101 in the first exposure period and the second exposure period, that is, a timing chart in the case where the subject is located at a relatively close position. Figure 6 is a timing chart in the case of example 2 in which the pulsed reflected light is received by the photoelectric conversion section 101 in the second exposure period and the third exposure period, that is, a timing chart in the case where the subject is located at a relatively distant position.

[0078] By the instruction of the control section 3, the pixel array control section 11, in synchronization with the timing tll at which the first exposure period is started, causes the ODG to shift from the High state (activated state) to the Low state (non-activated state), and concurrently causes the first read gate 106a and the second read gate 106b and the third read gate 106c to all shift from the Low state (non-activated state) to the High state (activated state).

[0079] By the above action of the pixel array control section 11, the signal charges generated at the photoelectric conversion section 101 by accepting the reflected light of the pulse-shaped light irradiated from the light emitting section 4 during the first exposure period and the background light other than the reflected light reaching the photoelectric conversion section 101 during the first exposure period are accumulated to the first charge accumulation section 102a via the first read gate 106a, to the second charge accumulation section 102b via the second read gate 106b, and to the third charge accumulation section 102c via the third read gate 106c. Thus, the signal charges generated at the photoelectric conversion section 101 are divided into approximately 1 / 3 and accumulated to the first charge accumulation section 102a, the second charge accumulation section 102b, and the third charge accumulation section 102c, respectively.

[0080] Next, in the second exposure period, by the instruction of the control section 3, the pixel array control section 11 causes the first read gate 106a to shift from the High state (activated state) to the Low state (non-activated state) at the timing t12 at which the period is started. Thus, the accumulation of signal charges to the first charge accumulation section 102a is stopped.

[0081] By the above action of the pixel array control section 11, the signal charges generated at the photoelectric conversion section 101 by accepting the reflected light reaching the photoelectric conversion section 101 during the second exposure period and the background light reaching during the second exposure period are accumulated to the second charge accumulation section 102b via the second read gate 106b and to the third charge accumulation section 102c via the third read gate 106c. Thus, the signal charges generated at the photoelectric conversion section 101 are divided into approximately half and accumulated to the second charge accumulation section 102b and the third charge accumulation section 102c, respectively.

[0082] Next, in the third exposure period, by the instruction of the control section 3, the pixel array control section 11 causes the second read gate 106b to shift from the High state (activated state) to the Low state (non-activated state) at the timing t13 at which the period is started. Thus, the accumulation of signal charges to the second charge accumulation section 102b is stopped.

[0083] Through the above action of the pixel array control section 11, the signal charge generated at the photoelectric conversion section 101 by accepting the reflected light reaching the photoelectric conversion section 101 during the 3rd exposure period and the background light reaching during the 3rd exposure period, is all accumulated to the 3rd charge accumulation section 102c via the 3rd read gate 106c.

[0084] At the time t14 when the 3rd exposure period ends, by the instruction of the control section 3, the pixel array control section 11 shifts the 3rd read gate 106c from the High state (activated state) to the Low state (non-activated state), thereby stopping the accumulation of signal charge to the 3rd charge accumulation section 102c, and shifts the ODG from the Low state to the High state, thereby making the exposure control gate 108 into the on state. Thereby the photoelectric conversion section 101 returns to the reset state.

[0085] Hereinafter, at the time when the 1st exposure sequence ends, the signal charge accumulated to the 1st charge accumulation section 102a via the 1st read gate 106a is set as P0, the signal charge accumulated to the 2nd charge accumulation section 102b via the 2nd read gate 106b is set as P1, and the signal charge accumulated to the 3rd charge accumulation section 102c via the 3rd read gate 106c is set as P2.

[0086] The signal charge P0, the signal charge P1, the signal charge P2, are respectively outputted to the signal processing section 20 as the pixel signal P0, the pixel signal P1, the pixel signal P2 in the order of the raster scan from the respective pixel sections 100 by the readout action of the signal charge, and held at the signal processing section 20.

[0087] Next, the readout action of the signal charge is explained. Here, the case of reading out the signal charge P0 is explained as an example, and the same is applied to the other signal charges.

[0088] By the instruction of the control section 3, the pixel array control section 11, in all the pixel sections 100, moves the signal charge P0 held at the 1st charge accumulation section 102a to under the transfer electrode 105f to which VG6 is applied, by the charge transfer applied by the six-phase drive pulses accompanying VG1 to VG6 applied to the transfer electrodes 105a to 105f.

[0089] By the instruction of the pixel array control section 11, the vertical scanning section 12, with respect to the pixel section 100 of the prescribed 1 line, in the state where the output control gate 113 is not activated, activates the reset gate 115 to discharge the unnecessary charge remaining at the floating diffusion layer 114 to the reset drain 116, and then activates the readout circuit 117 to output the no-signal voltage output to the vertical signal line 16.

[0090] The column processing section 13, by the instruction of the vertical scanning section 12, holds the non-signal voltage output to the vertical signal line 16 of each column.

[0091] Next, the vertical scanning section 12, with respect to the pixel section 100 of the same row, causes the reset gate 115 to be activated in a state where the output control gate 113 is not activated, causes the non-desired charge remaining in the floating diffusion layer 114 to be discharged to the reset drain 116, and then causes the output control gate 113 to be activated, thereby causing the signal charge P0 held under the transfer electrode 105f to be transferred to the floating diffusion layer 114, and then causes the readout circuit 117 to be activated, causing the signal voltage output of the signal charge P0 to be output to the vertical signal line 16.

[0092] The column processing section 13, by the instruction of the vertical scanning section 12, performs correlated double sampling based on the non-signal voltage held at the front end of each column and the signal voltage of the signal charge P0, and outputs and holds the pixel signal P0 of each column to the horizontal scanning section 14.

[0093] The horizontal scanning section 14, by the instruction of the vertical scanning section 12, sequentially scans the group of pixel signals P0 of one row output and held from the column processing section 13, thereby sequentially selecting the pixel signal P0 corresponding to the pixel section 100 of a prescribed row in the horizontal direction, and outputs the pixel signal P0 to the signal processing section 20 via the output buffer 15.

[0094] The vertical scanning section 12 sequentially performs the series of operations from the output control of the non-signal voltage to the output control of the pixel signal P0 of one row to the horizontal scanning section 14, and causes all of the pixel signals P0 corresponding to each pixel section 100 to be output from the imaging section 10 in a raster scan via the output buffer 15.

[0095] Figure 7 is a timing chart showing the exposure of the pixel section 100 in the 2nd exposure sequence and the signal accumulation timing, and the exposure state of the signal charge accumulated to the charge accumulation section 102 via the 1st readout gate 106a and the 2nd readout gate 106b and the 3rd readout gate 106c, respectively.

[0096] The 2nd exposure sequence is composed of a 1st exposure period, a 2nd exposure period, and a 3rd exposure period, the 1st exposure period is a period of the same length as the time Tp from the start of the pulsed light irradiation from the light emitting section 4 to the end of the irradiation in the 1st exposure sequence, the 2nd exposure period is a period from the end of the 1st exposure period until the time Tp elapses, and the 3rd exposure period is a period from the end of the 2nd exposure period until the time Tp elapses.

[0097] In the 2nd exposure sequence, the pulsed light is not irradiated from the light emitting section 4.

[0098] By the instruction of the control section 3, the pixel array control section 11, in synchronization with the timing t21 at which the 1st exposure period is started, causes the ODG to shift from the High state to the Low state, and in parallel therewith, causes the 1st read gate 106a and the 2nd read gate 106b and the 3rd read gate 106c to all shift from the Low state (non-activated state) to the High state (activated state).

[0099] By the above action of the pixel array control section 11, the discharge of the signal charge from the photoelectric conversion section 101 to the overflow drain 109 is stopped, and the signal charge generated in the photoelectric conversion section 101 by the acceptance of the background light is accumulated to the 1st charge accumulation section 102a via the 1st read gate 106a, to the 2nd charge accumulation section 102b via the 2nd read gate 106b, and to the 3rd charge accumulation section 102c via the 3rd read gate 106c. Thus, the signal charge generated in the photoelectric conversion section 101 is divided into approximately 1 / 3 and accumulated to the 1st charge accumulation section 102a, the 2nd charge accumulation section 102b, and the 3rd charge accumulation section 102c, respectively.

[0100] Next, in the 2nd exposure period, by the instruction of the control section 3, the pixel array control section 11, at the timing t22 at which the period is started, causes the 1st read gate 106a to shift from the High state (activated state) to the Low state (non-activated state). Thus, the accumulation of the signal charge to the 1st charge accumulation section 102a is stopped.

[0101] By the above action of the pixel array control section 11, the signal charge generated in the photoelectric conversion section 101a by the acceptance of the background light arriving during the 2nd exposure period is accumulated to the 2nd charge accumulation section 102b via the 2nd read gate 106b and to the 3rd charge accumulation section 102c via the 3rd read gate 106c. Thus, the signal charge generated in the photoelectric conversion section 101a is divided into approximately half and accumulated to the 2nd charge accumulation section 102b and the 3rd charge accumulation section 102c, respectively.

[0102] Next, in the 3rd exposure period, by the instruction of the control section 3, the pixel array control section 11, at the timing t23 at which the period is started, causes the 2nd read gate 106b to shift from the High state (activated state) to the Low state (non-activated state). Thus, the accumulation of the signal charge to the 2nd charge accumulation section 102b is stopped.

[0103] By the above action of the pixel array control section 11, the signal charge generated in the photoelectric conversion section 101 by the acceptance of the background light arriving during the 3rd exposure period is all accumulated to the 3rd charge accumulation section 102c via the 3rd read gate 106c.

[0104] At the time t24 at which the 3rd exposure period ends, the pixel array control section 11, by the instruction of the control section 3, causes the 3rd read gate 106c to shift from the High state (activated state) to the Low state (non-activated state), thereby stopping the accumulation of the signal charge to the 3rd charge accumulation section 102c, and causes the ODG to shift from the Low state to the High state, thereby causing the exposure control gate 108 to become the on state. Thereby, the photoelectric conversion section 101 returns to the reset state.

[0105] The signal charge accumulated to the 1st charge accumulation section 102a via the 1st read gate 106a at the time of the end of the 2nd exposure sequence is set to B0, the signal charge accumulated to the 2nd charge accumulation section 102b via the 2nd read gate 106b is set to B1, and the signal charge accumulated to the 3rd charge accumulation section 102c via the 3rd read gate 106c is set to B2.

[0106] The signal charge B0, the signal charge B1, and the signal charge B2, by the readout operation of the signal charge, are outputted to the signal processing section 20 as the pixel signal B0, the pixel signal B1, and the pixel signal B2, respectively, in the order of the optical raster scan from the respective pixel sections 100, and are held in the signal processing section 20.

[0107] The pixel signal B0 is equal to the signal charge S1Xb contained in the background light component (S1Xb) in the pixel signal P0, the pixel signal B1 is equal to the signal charge S1Yb contained in the background light component (S1Yb) in the pixel signal P1, and the pixel signal B2 is equal to the signal charge S1Zb contained in the background light component (S1Zb) in the pixel signal P2. Figure 5 Or Figure 6 The pixel signal B1 is equal to the signal charge S1Yb contained in the background light component (S1Yb) in the pixel signal P1, and the pixel signal B2 is equal to the signal charge S1Zb contained in the background light component (S1Zb) in the pixel signal P2. Figure 5 Or Figure 6 The pixel signal B2 is equal to the signal charge S1Zb contained in the background light component (S1Zb) in the pixel signal P2. Figure 5 Or Figure 6 The pixel signal B0 is equal to the signal charge S1Xb contained in the background light component (S1Xb) in the pixel signal P0, the pixel signal B1 is equal to the signal charge S1Yb contained in the background light component (S1Yb) in the pixel signal P1, and the pixel signal B2 is equal to the signal charge S1Zb contained in the background light component (S1Zb) in the pixel signal P2.

[0108] While the pixel signal P0, the pixel signal P1, the pixel signal P2, the pixel signal B0, the pixel signal B1, and the pixel signal B2 are held in the signal processing section 20 in all the pixel sections 100, the coefficient decision section 201 decides, for each of the pixel sections 100, the coefficients used by the distance calculation section 202 and the distance difference calculation section 203, in accordance with the pixel signal P0, the pixel signal P1, the pixel signal P2, the pixel signal B0, the pixel signal B1, and the pixel signal B2. More specifically, the coefficient decision section 201 decides, for each of the pixel sections 100, the pixel signal coefficient values k01, k02, k03, k04, k05, k06, k11, k12, k13, k14, k15, and k16 defined by the following (Formula 1) or (Formula 2), in accordance with the following (Condition 1) and (Condition 2).

[0109] P0-B0 < P2-B2-P1+B1 (Condition 2)

[0110] k01 = -2

[0111] k02 = +1

[0112] k03 = +1

[0113] k04 = +2

[0114] k05 = -1

[0115] k06 = -1

[0116] k11 = +1

[0117] k12 = +1

[0118] k13 = +1

[0119] k14 = -1

[0120] k15 = -1

[0121] K16 = -1 (Formula 1)

[0122] P0-B0 < P2-B2-P1+B1 (Condition 2)

[0123] k01 = 0

[0124] k02 = 0

[0125] k03 = +2

[0126] k04 = 0

[0127] k05 = 0

[0128] k06 = -2

[0129] k11 = 0

[0130] k12 = +1

[0131] k13 = +1

[0132] k14 = 0

[0133] k15 = -1

[0134] K16 = -1 (Formula 2)

[0135] Here, in the case where (Condition 1) is true, it corresponds to the case 1 in which the subject is located at a relatively close position, and in the case where (Condition 2) is true, it corresponds to the case 2 in which the subject is located at a relatively distant position.

[0136] When the pixel signal coefficient value is decided by the coefficient decision section 201, the distance calculation section 202 calculates the distance signal Dout with respect to each pixel section 100 in accordance with (Formula 3).

[0137] Dout = K' x (k01 x P0 + k02 x P1 + k03 x P2 + k04 x B0 + k05 x B1 + k06 x B2) / (k11 x P0 + k12 x P1 + k13 x P2 + k14 x B0 + k15 x B1 + k16 x B2)...(Formula 3)

[0138] Here, K' is a constant equivalent to c x Tp / 2 when the speed of light (299,792,458 m / s) is set as c and the time width of the pulse irradiation light of the light emitting section 4 is set as Tp.

[0139] When (Formula 1) is substituted into (Formula 3),

[0140] Dout = K' x (P1 + P2 - 2P0) - (B1 + B2 - 2B0) / ((P0 + P1 + P2) - (B0 + B1 + B2)).

[0141] In this formula, the distance signal Dout shows the distance to the object in Case 1.

[0142] When (Formula 2) is substituted into (Formula 3),

[0143] Dout = K' x 2 x (P2 - B2) / ((P1 + P2) - (B1 + B2)).

[0144] In this formula, the distance signal Dout shows the distance to the object in Case 2.

[0145] The distance calculation section 202 thus calculates the distance signal Dout showing the distance to the object.

[0146] The distance difference calculation section 203 calculates the distance difference signal DV showing the standard deviation of the distance shown by the distance signal Dout. The following describes how the distance difference calculation section 203 calculates the distance difference signal DV.

[0147] When the pixel signal coefficient value is decided by the coefficient decision section 201, the distance difference calculation section 203 calculates the intermediate signal S1 (the numerator component of Dout) and the light receiving amount signal IRR (the denominator component of Dout) with respect to each pixel section 100 in accordance with (Formula 4) and (Formula 5). The light receiving amount signal IRR here is a signal showing the light receiving amount of only the reflected light component excluding the background light component in the light accepted by the photoelectric conversion section 101.

[0148] S1 = k01 x P0 + k02 x P1 + k03 x P2 + k04 x B0 + k05 x B1 + k06 x B2 · · · (Equation 4)

[0149] IRR = k11 x P0 + k12 x P1 + k13 x P2 + k14 x B0 + k15 x B1 + k16 x B2 · · · (Equation 5)

[0150] Here, in order to calculate the difference amount of the distance signal Dout, the propagation formula of the error shown by (Equation 6) can be used.

[0151] Z = f(x1, x2, x3, x4,..., xn)

[0152] [Equation 1]

[0153]

[0154] Further, the pixel signal P0, when the signal charge is set to e(P0) and the conversion coefficient from the signal charge to the pixel signal is set to Kh, is represented by the following equation (Equation 7).

[0155] P0 = Kh x e(P0) · · · (Equation 7)

[0156] Here, when the standard deviation of the shot noise that becomes the difference reason of the pixel signal P0 is set to σ(P0) (Equation 8) is established, it can be deformed to (Equation 9).

[0157] [Equation 2]

[0158]

[0159] [Equation 3]

[0160] σ(P0] 2 = Kh 2 x e(P0) = Kh x P0 · · · (Equation 9)

[0161] The same also applies to the other pixel signals P1, P2, B0, B1, B2.

[0162] The distance difference calculating section 203 calculates the difference of the distance shown by the distance signal Dout calculated in (Equation 3) as the standard deviation. That is, the distance difference calculating section 203, for each pixel section 100, calculates the distance difference signal DV showing the standard deviation of the distance shown by the distance signal Dout using the following equation (Equation 10) according to the error propagation formula (Equation 6), (Equation 4), (Equation 5), (Equation 9).

[0163] [Equation 4]

[0164]

[0165] The distance difference signal DV calculated using Equation 10 adds pixel signals B0, B1, and B2, which correspond to background light components. This appropriately reflects the noise propagation state in both Case 1, when the subject is relatively close, and Case 2, when the subject is relatively far away. Therefore, the distance difference signal DV can be used to determine the reliability of the distance indicated by the distance signal Dout.

[0166] As described above, the imaging device 1 can determine the reliability of the distance indicated by the distance signal Dout, which is the calculated distance information.

[0167] (Implementation Method 2)

[0168] Next, the imaging device according to the second embodiment will be described.

[0169] Figure 8 This is a block diagram showing an example of the configuration of an imaging device 1 a according to the second embodiment.

[0170] like Figure 8 As shown, the imaging device 1a differs from the imaging device 1 according to the first embodiment in that the signal processing unit 20 of the imaging device 1 is replaced with a signal processing unit 20a. Furthermore, with this replacement, the imaging device 1a differs from the imaging device 1 according to the first embodiment in that the solid-state imaging device 2 of the imaging device 1 is replaced with a solid-state imaging device 2a.

[0171] The signal processing unit 20 according to the first embodiment is configured to calculate the distance signal Dout and the distance difference signal DV. In contrast, the signal processing unit 20a calculates the distance signal Dout, the received light amount signal IRR, and a received light amount difference signal indicating the difference in the received light amount indicated by the received light amount signal IRR.

[0172] Figure 9 It is a block diagram showing a configuration example of the signal processing unit 20a.

[0173] like Figure 9 As shown, the signal processing unit 20a includes a coefficient determination unit 201, a distance calculation unit 202, a received light amount calculation unit 204, and a received light amount difference calculation unit 205. The signal processing unit 20a can be implemented, for example, by a logic circuit or a memory and a processor that executes a program stored in the memory.

[0174] The light-receiving-amount calculating section 204 calculates the light-receiving-amount signal IRR from the plurality of pixel signals output from the imaging section 10 and the coefficients determined by the coefficient determining section 201. More specifically, the light-receiving-amount calculating section 204 calculates the light-receiving-amount signal IRR in (Formula 5).

[0175] The light-receiving-amount difference calculating section 205 calculates the light-receiving-amount difference signal IRV, which shows the amount of difference in the light-receiving-amount shown by the light-receiving-amount signal IRR, from the plurality of pixel signals output from the imaging section 10 and the coefficients determined by the coefficient determining section 201.

[0176] The following describes how the light-receiving-amount difference calculating section 205 calculates the light-receiving-amount difference signal.

[0177] The light-receiving-amount difference calculating section 205 calculates the light-receiving-amount difference signal IRV, which shows the standard deviation in the light-receiving-amount shown by the light-receiving-amount signal IRR.

[0178] The light-receiving-amount difference signal IRV is calculated in (Formula 11) derived from the error propagation formula (Formula 6) and (Formula 5).

[0179] [Formula 5]

[0180]

[0181] Therefore, the light-receiving-amount difference calculating section 205, when the pixel signal coefficient values are determined by the coefficient determining section 201 for each pixel section 100, calculates the light-receiving-amount difference signal IRV in (Formula 12) for each corresponding pixel section 100, which is a formula in which (Formula 11) is transformed using (Formula 9).

[0182] [Formula 6]

[0183]

[0184] The light-receiving-amount difference signal IRV calculated in (Formula 12) has added thereto the pixel signal B0, the pixel signal B1, and the pixel signal B2, which correspond to the background light components. Therefore, in both the case 1 in which the subject is located at a relatively close position and the case 2 in which the subject is located at a relatively far position, the propagation state of the noise can be appropriately reflected for each pixel section 100.

[0185] Here, the ratio of the light-receiving-amount signal IRR, which shows the light-receiving-amount of the reflected light component, and the light-receiving-amount difference signal IRV, which has added thereto the background light components, corresponds to the SNR of the distance shown by the distance signal Dout. Therefore, the light-receiving-amount difference signal IRV can be used to determine the reliability of the distance shown by the distance signal Dout.

[0186] As described above, by the imaging device 1a, it is possible to determine the reliability of the distance indicated by the distance information, that is, the distance signal Dout, which is calculated.

[0187] Further, the amount of operation of (Formula 12) is less than that of (Formula 10). Therefore, the signal processing section 20a has an advantage that the burden of the processing by the logic circuit or the processor is light compared to the signal processing section 20 related to Embodiment 1.

[0188] (Supplement)

[0189] The imaging device related to one aspect of the present disclosure described above according to Embodiment 1 and Embodiment 2, but the present disclosure is not limited to these embodiments. The configurations obtained by applying various modifications that can be thought of by those skilled in the art to these embodiments, the configurations obtained by combining the constituent elements in different embodiments, and the like are also included in the scope of one or more aspects of the present disclosure without departing from the spirit of the present disclosure.

[0190] In Embodiment 1, the imaging device 1 is described to calculate the distance difference signal DV for each pixel section 100. However, the imaging device 1 does not necessarily have to be configured to calculate the distance difference signal DV for each pixel section 100, as long as it is possible to calculate the distance difference signal DV for one or more pixel sections 100 of at least a part of the plurality of pixel sections 100. For example, the imaging device 1 can be configured to leave one or more pixel sections 100 from all of the pixel sections 100, to exclude the other pixel sections 100, and to calculate the distance difference signal DV for the one or more pixel sections 100 that are not excluded, respectively.

[0191] Further, in Embodiment 2, the imaging device 1a is described to calculate the light-receiving amount difference signal IRV for each pixel section 100. However, the imaging device 1a does not necessarily have to be configured to calculate the light-receiving amount difference signal IRV for each pixel section 100, as long as it is possible to calculate the light-receiving amount difference signal IRV for one or more pixel sections 100 of at least a part of the plurality of pixel sections 100. For example, the imaging device 1a can be configured to leave one or more pixel sections 100 from all of the pixel sections 100, to exclude the other pixel sections 100, and to calculate the light-receiving amount difference signal IRV for the one or more pixel sections 100 that are not excluded, respectively.

[0192] Industrial Applicability

[0193] The present application can be widely used for an imaging device that obtains distance information of an object.

[0194] Legend of symbols

[0195] 1 Imaging device

[0196] 2 solid-state imaging device

[0197] 3 control section

[0198] 4 light emitting section

[0199] 10 imaging section

[0200] 11 pixel array control section

[0201] 12 vertical scanning section

[0202] 13 column processing section

[0203] 14 horizontal scanning section

[0204] 15 output buffer

[0205] 16 vertical signal line

[0206] 17, 104 transfer channel

[0207] 20, 20a signal processing section

[0208] 30 pixel array

[0209] 100 pixel section

[0210] 101 photoelectric conversion section

[0211] 102 charge accumulation section

[0212] 102a first charge accumulation section

[0213] 102b second charge accumulation section

[0214] 102c third charge accumulation section

[0215] 105, 105a, 105b, 105c, 105d, 105e, 105f transfer electrode

[0216] 106 readout gate

[0217] 106a first readout gate

[0218] 106b second readout gate

[0219] 106c third readout gate

[0220] 108, 108a, 108b exposure control gate

[0221] 109, 109a, 109b overflow drain

[0222] 113 output control gate

[0223] 114 floating diffusion layer

[0224] 115 reset gate

[0225] 116 reset drain

[0226] 117 readout circuit

[0227] 201 coefficient determination unit

[0228] 202 distance calculation unit

[0229] 203 distance difference calculation unit

[0230] 204 light reception amount calculation unit

[0231] 205 light reception amount difference calculation unit

Claims

1. A camera device, The camera device comprises: a light emitting unit for emitting a plurality of pulsed lights to a subject; and A solid-state imaging device includes an exposure control unit, a signal processing unit, and a plurality of pixel units, each of which converts received light into a signal charge. The exposure control unit controls the accumulation of the signal charge in the pixel unit, and the signal processing unit calculates distance information indicating the distance to the subject. The exposure control section causes the pixel section to accumulate the signal charge during a plurality of different exposure periods corresponding to the emission timing of the pulsed light in one frame. The signal processing unit multiplies each signal related to the signal charge accumulated during a plurality of different exposure periods generated by the pixel unit by a coefficient to perform calculations, and calculates distance information of the subject projected onto the pixel unit using a TOF (time of flight) method. The signal processing unit calculates difference information indicating a difference amount with respect to the distance indicated by the distance information of the pixel unit using each signal associated with the signal charge accumulated during the plurality of different exposure periods and a coefficient applied to the signal associated with the signal charge in calculating the distance information. The solid-state imaging device outputs the distance information and difference information indicating a difference amount related to the distance.

2. The imaging device according to claim 1, The signal processing section includes a coefficient determination section that determines a coefficient to be applied to each signal related to the signal charge in calculating distance information, based on a mutual magnitude relationship between the signal charges accumulated in the plurality of different exposure periods.

3. A camera device, The camera device comprises: a light emitting unit for emitting a plurality of pulsed lights to a subject; and A solid-state imaging device includes an exposure control unit, a signal processing unit, and a plurality of pixel units, each of which converts received light into a signal charge. The exposure control unit controls the accumulation of the signal charge in the pixel unit, and the signal processing unit calculates distance information indicating the distance to the subject. The exposure control section causes the pixel section to accumulate the signal charge during a plurality of different exposure periods corresponding to the emission timing of the pulsed light in one frame. The signal processing unit calculates distance information of the subject projected onto the pixel unit using a TOF (time of flight) method based on each signal related to the signal charge accumulated during a plurality of different exposure periods generated in the pixel unit, and calculates information on the amount of light received by the luminescent component by multiplying each signal related to the signal charge accumulated during the plurality of different exposure periods by a coefficient. The information on the amount of light received by the luminescent component indicates the amount of light received based on the light irradiated by the luminescent unit. The signal processing unit calculates difference information indicating a difference in light reception amount indicated by the information on the light reception amount of the luminescent component in the pixel unit, using each signal associated with the signal charge accumulated during the plurality of different exposure periods and a coefficient applied to the signal associated with the signal charge in calculating the light reception amount of the luminescent component. The solid-state imaging device outputs information on the amount of light received by the luminescent component and difference information indicating a difference in the amount of light received.

4. The imaging device according to claim 3, The signal processing unit includes a coefficient determination unit that determines coefficients to be applied to respective signals related to the signal charges in calculating information on the amount of light received by the luminescent components, based on a mutual magnitude relationship between the signal charges accumulated in the plurality of different exposure periods.

5. A method for calculating difference information, the method being performed by an imaging device, the imaging device comprising: a light emitting unit configured to emit a plurality of pulsed lights toward a subject; and a solid-state imaging device comprising an exposure control unit, a signal processing unit, and a plurality of pixel units, each of the plurality of pixel units converting received light into a signal charge, the exposure control unit controlling the accumulation of the signal charge in the pixel units, and the signal processing unit calculating distance information indicating the distance to the subject. The difference information calculation method includes: In a first step, the exposure control unit causes the pixel unit to accumulate the signal charge during a plurality of different exposure periods corresponding to the emission timing of the pulsed light in one frame. In a second step, the signal processing unit multiplies each signal related to the signal charge accumulated during a plurality of different exposure periods generated by the pixel unit by a coefficient to calculate distance information of the subject projected onto the pixel unit using a TOF (time of flight) method; In a third step, the signal processing unit calculates difference information indicating a difference amount with respect to the distance indicated by the distance information of the pixel unit using each signal associated with the signal charge accumulated during the plurality of different exposure periods and a coefficient applied to the signal associated with the signal charge in calculating the distance information; and In the fourth step, the solid-state imaging device outputs the distance information and difference information indicating a difference amount related to the distance.

6. A method for calculating difference information, the method being performed by an imaging device, the imaging device comprising: a light emitting unit configured to emit a plurality of pulsed lights toward a subject; and a solid-state imaging device comprising an exposure control unit, a signal processing unit, and a plurality of pixel units, each of the plurality of pixel units converting received light into a signal charge, the exposure control unit controlling the accumulation of the signal charge in the pixel units, and the signal processing unit calculating distance information indicating the distance to the subject. The difference information calculation method includes: In a first step, the exposure control unit causes the pixel unit to accumulate the signal charge during a plurality of different exposure periods corresponding to the emission timing of the pulsed light in one frame. In a second step, the signal processing unit calculates distance information of the subject projected onto the pixel unit using a TOF (time of flight) method based on each signal related to the signal charge accumulated during a plurality of different exposure periods generated in the pixel unit, and calculates information on the amount of light received by the luminescent component by multiplying each signal related to the signal charge accumulated during the plurality of different exposure periods by a coefficient, wherein the information on the amount of light received by the luminescent component indicates the amount of light received based on the light irradiated by the luminescent unit; In a third step, the signal processing unit calculates difference information indicating a difference in light reception amount indicated by the information on the light reception amount of the light-emitting component in the pixel unit, using each signal associated with the signal charge accumulated during the plurality of different exposure periods and a coefficient applied to the signal associated with the signal charge in calculating the light reception amount of the light-emitting component; and In the fourth step, the solid-state imaging device outputs information on the amount of light received by the luminescent component and difference information indicating a difference in the amount of light received.

Images