Distance measuring device and distance measuring method

Abstract

The present technology relates to a distance measuring device and a distance measuring method that make it possible to suppress the occurrence of radiation noise. The distance measuring device comprises a plurality of light emitting units including a first light emitting unit and a second light emitting unit that emit light at a subject, a light-emission control unit that controls the light emission of each of the plurality of light emitting units, a light receiving unit that receives reflected light from the subject, and a distance measuring unit that calculates the distance to the subject on the basis of a signal obtained by the light reception, wherein the light-emission control unit drives the first light emitting unit on the basis of a first drive signal, drives the second light emitting unit on the basis of a second drive signal, and controls the first drive signal and the second drive signal to have a prescribed phase difference. The present technology is applicable, for example, to distance measuring devices.

Description

Distance measuring device, ranging method

[0001] The present technology relates to a distance measuring device and a distance measuring method, and more particularly to a distance measuring device and a distance measuring method that can suppress the generation of radiation noise, for example.

[0002] In recent years, advances in semiconductor technology have led to the miniaturization of distance measuring devices that measure the distance to an object. This has made it possible to incorporate distance measuring devices into mobile terminals such as smartphones, which are compact information processing devices with communication functions. Examples of distance measuring devices (sensors) that measure the distance to an object include time-of-flight (TOF) sensors (see, for example, Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2016-090268

[0004] When measuring the distance to a subject by irradiating the subject with light from multiple light-emitting elements, the measurement accuracy can be improved, but there is a possibility that the radiation noise emitted from the distance measuring device to the outside will be large. It is desired to improve the measurement accuracy while suppressing the generation of radiation noise.

[0005] The present technology has been made in view of such circumstances, and is intended to make it possible to suppress the generation of radiation noise.

[0006] A distance measuring device according to one aspect of the present technology includes a plurality of light-emitting units including a first light-emitting unit and a second light-emitting unit that irradiate light onto a subject, a light-emitting control unit that controls the light emission of each of the plurality of light-emitting units, a light-receiving unit that receives light reflected from the subject, and a distance measuring unit that calculates a distance to the subject based on a signal obtained by the light-receiving, wherein the light-emitting control unit drives the first light-emitting unit based on a first drive signal and drives the second light-emitting unit based on a second drive signal, and controls the first drive signal and the second drive signal to have a predetermined phase difference.

[0007] A ranging method according to one aspect of the present technology is a ranging device including a plurality of light-emitting units, including a first light-emitting unit and a second light-emitting unit, that irradiate light onto a subject, a light-emitting control unit that controls the light emission of each of the plurality of light-emitting units, a light-receiving unit that receives light reflected from the subject, and a ranging unit that calculates a distance to the subject based on a signal obtained by the light-receiving, wherein the light-emitting control unit drives the first light-emitting unit based on a first drive signal and drives the second light-emitting unit based on a second drive signal, and controls the first drive signal and the second drive signal to have a predetermined phase difference.

[0008] A ranging device and ranging method according to one aspect of the present technology include a plurality of light-emitting units including a first light-emitting unit and a second light-emitting unit that irradiate light onto a subject, a light-emitting control unit that controls the light emission of each of the plurality of light-emitting units, a light-receiving unit that receives reflected light from the subject, and a ranging unit that calculates the distance to the subject based on a signal obtained by the received light, and the light-emitting control unit drives the first light-emitting unit based on a first drive signal and drives the second light-emitting unit based on a second drive signal, and controls the first drive signal and the second drive signal to have a predetermined phase difference.

[0009] The distance measuring device may be an independent device or an internal block constituting a single device.

[0010] 1 is a diagram illustrating an example of the configuration of a distance measuring device; FIG. 2 is a diagram illustrating an example of the configuration of a light receiving unit; FIG. 3 is a diagram illustrating an example of the configuration of a pixel; FIG. 4 is a diagram illustrating distribution of charges in a pixel; FIG. 5 is a diagram illustrating an example of four types of light reception with a phase delay of 90 degrees each; FIG. 6 is a diagram illustrating a detection signal in one frame; FIG. 7 is a diagram illustrating a relationship between a detection period and a detection signal; FIG. 8 is a diagram illustrating a detection signal in one frame; FIG. 9 is a diagram illustrating a configuration of an embodiment of a distance measuring system to which the present technology is applied; FIG. 10 is a diagram illustrating an imaging area; FIG. 11 is a diagram illustrating a drive signal; FIG. 12 is a diagram illustrating a drive signal; FIG. 13 is a diagram illustrating a drive signal; FIG. 14 is a diagram illustrating a correction area in an imaging area; FIG. 15 is a diagram illustrating correction during readout driving; FIG. 16 is a diagram illustrating an imaging area; FIG. 17 is a diagram illustrating an imaging area; FIG. 18 is a diagram illustrating an imaging area; FIG. 1 is a diagram for explaining the configuration of a light emitting section. FIG. 1 is a diagram for explaining the configuration of a light emitting section. FIG. 1 is a diagram for explaining the configuration of a light emitting section. FIG. 1 is a diagram for explaining the configuration of a light emitting section. FIG. 2 is a diagram for explaining the configuration of a light emission control section. FIG. 2 is a diagram for explaining the configuration of a light emission control section. FIG. 3 is a diagram for explaining an example configuration related to detection of different wavelengths. FIG. 4 is a diagram for explaining an example configuration related to detection of different wavelengths. FIG. 5 is a diagram for explaining an example pixel arrangement in which light with different phase delays is received. FIG. 6 is a diagram for explaining an example arrangement of a charge accumulation section. FIG. 7 is a diagram for explaining an example arrangement of a charge accumulation section and a charge transfer section. FIG. 8 is a diagram for explaining another example configuration of an imaging region and an overlapping region.Fig. 1 is a diagram showing an example of a schematic configuration of an endoscopic surgery system. Fig. 2 is a block diagram showing an example of a functional configuration of a camera head and a CCU. Fig. 3 is a block diagram showing an example of a schematic configuration of a vehicle control system. Fig. 4 is an explanatory diagram showing an example of the installation positions of an outside-vehicle information detection unit and an imaging unit.

[0011] Hereinafter, modes for carrying out the present technology (hereinafter referred to as embodiments) will be described.

[0012] The present technology can be applied to, for example, a light receiving element that constitutes a distance measurement system that performs distance measurement using an indirect TOF method, and an imaging device that has such a light receiving element.

[0013] For example, the distance measurement system can be applied to an in-vehicle system that is installed in a vehicle and measures the distance to an object outside the vehicle, or a gesture recognition system that measures the distance to an object such as a user's hand and recognizes the user's gesture based on the measurement result, etc. In this case, the result of the gesture recognition can be used for operating a car navigation system, for example.

[0014] <Outline of Indirect TOF> An indirect TOF to which the present technology can be applied will now be described. Fig. 1 shows an example of the configuration of a distance measuring device that performs distance measurement using an indirect TOF method.

[0015] The distance measuring device 10 includes a lens 11, a light receiving unit 12, a signal processing unit 13, a light emitting unit 14, a light emission control unit 15, and a filter unit 16. The distance measuring device 10 in Fig. 1 measures the distance to an object by irradiating the object with light and receiving light (reflected light) that is reflected from the object (irradiated light).

[0016] The light-emitting system of the distance measuring device 10 is composed of a light-emitting unit 14 and a light-emitting control unit 15. In the light-emitting system, the light-emitting control unit 15 controls the light-emitting unit 14 to emit infrared light (IR) under the control of the signal processing unit 13. An IR bandpass filter may be provided between the lens 11 and the light-receiving unit 12, and the light-emitting unit 14 may emit infrared light corresponding to the transmission wavelength band of the IR bandpass filter.

[0017] The light emitting unit 14 may be disposed inside the housing of the distance measuring device 10, or may be disposed outside the housing of the distance measuring device 10. The light emission control unit 15 causes the light emitting unit 14 to emit light at a predetermined frequency.

[0018] The signal processing unit 13 functions as a calculation unit that calculates the distance (depth value) from the distance measuring device 10 to an object based on, for example, a detection signal (pixel data) supplied from the light receiving unit 12. The signal processing unit 13 generates a depth map in which depth values ​​(depth information) are stored as pixel values ​​of each pixel 50 ( FIG. 2 ) of the light receiving unit 12, and outputs the depth map to the filter unit 16. The signal processing unit 13 also calculates the reliability of the calculated depth value for each pixel 50 of the light receiving unit 12, and generates a reliability map in which reliability (luminance information) is stored as pixel values ​​of each pixel 50 of the light receiving unit 12, and outputs the reliability map to the filter unit 16.

[0019] 2 is a block diagram showing an example of the configuration of the light receiving unit 12. The light receiving unit 12 can be a CMOS (Complementary Metal Oxide Semiconductor) image sensor.

[0020] The light receiving unit 12 includes a pixel array unit 41, a vertical drive unit 42, a column processing unit 43, a horizontal drive unit 44, and a system control unit 45. The pixel array unit 41, the vertical drive unit 42, the column processing unit 43, the horizontal drive unit 44, and the system control unit 45 are formed on a semiconductor substrate (chip) not shown.

[0021] In the pixel array section 41, unit pixels (for example, pixels 50 in FIG. 3 ) each having a photoelectric conversion element that generates and internally accumulates photocharges whose amount corresponds to the amount of incident light are arranged two-dimensionally in a matrix. Note that, hereinafter, the photocharges whose amount corresponds to the amount of incident light may be simply referred to as "charges," and unit pixels may be simply referred to as "pixels."

[0022] The pixel array unit 41 further includes pixel drive lines 46 formed in the left-right direction (the direction in which the pixels in the pixel rows are arranged) for each row of the matrix-like pixel arrangement, and vertical signal lines 47 formed in the up-down direction (the direction in which the pixels in the pixel columns are arranged) for each column. One end of each pixel drive line 46 is connected to an output terminal of the vertical drive unit 42 corresponding to each row.

[0023] The vertical drive unit 42 is a pixel drive unit that is configured with a shift register, an address decoder, etc., and drives each pixel of the pixel array unit 41, either all pixels at the same time or row by row, etc. Pixel signals output from each unit pixel of a pixel row selected and scanned by the vertical drive unit 42 are supplied to the column processing unit 43 through each vertical signal line 47. The column processing unit 43 performs predetermined signal processing on the pixel signals output from each unit pixel of the selected row through the vertical signal line 47 for each pixel column of the pixel array unit 41, and temporarily holds the pixel signals after signal processing.

[0024] Specifically, the column processing unit 43 performs at least noise removal processing, such as correlated double sampling (CDS) processing, as signal processing. This correlated double sampling by the column processing unit 43 removes pixel-specific fixed pattern noise such as reset noise and threshold voltage variations of the amplification transistor. In addition to the noise removal processing, the column processing unit 43 can also be provided with an AD (analog-digital) conversion function, for example, to output the signal level as a digital signal.

[0025] The horizontal drive unit 44 is configured with a shift register, an address decoder, etc., and sequentially selects unit circuits corresponding to pixel columns in the column processing unit 43. By selective scanning by this horizontal drive unit 44, pixel signals that have been signal-processed by the column processing unit 43 are output sequentially to the signal processing unit 48.

[0026] The system control unit 45 is composed of a timing generator that generates various timing signals, and controls the driving of the vertical driving unit 42, column processing unit 43, horizontal driving unit 44, etc. based on the various timing signals generated by the timing generator.

[0027] In the pixel array unit 41, a pixel drive line 46 is wired in the row direction for each pixel row in the matrix-like pixel arrangement, and two vertical signal lines 47 are wired in the column direction for each pixel column. For example, the pixel drive line 46 transmits a drive signal for driving the pixels when reading out a signal. Note that although FIG. 2 shows the pixel drive line 46 as a single line, the number of pixel drive lines 46 is not limited to one. One end of the pixel drive line 46 is connected to an output terminal of the vertical drive unit 42 corresponding to each row.

[0028] <Structure of Unit Pixel> Next, a specific structure of the unit pixels 50 arranged in a matrix in the pixel array section 41 will be described.

[0029] The pixel 50 includes a photodiode 61 (hereinafter referred to as PD61) which is a photoelectric conversion element, and is configured so that charges generated by the PD 61 are distributed to taps 51-1 and 51-2. Of the charges generated by the PD 61, the charges distributed to the tap 51-1 are read out from the vertical signal line 47-1 and output as a detection signal SIG1. Meanwhile, the charges distributed to the tap 51-2 are read out from the vertical signal line 47-2 and output as a detection signal SIG2.

[0030] The tap 51-1 is configured with a transfer transistor 62-1, a floating diffusion (FD) 63-1, a reset transistor 64, an amplification transistor 65-1, and a selection transistor 66-1. Similarly, the tap 51-2 is configured with a transfer transistor 62-2, a floating diffusion (FD) 63-2, a reset transistor 64, an amplification transistor 65-2, and a selection transistor 66-2.

[0031] As shown in FIG. 3, the reset transistor 64 may be shared by the FD 63-1 and the FD 63-2, or may be provided for each of the FD 63-1 and the FD 63-2.

[0032] When a configuration is adopted in which the reset transistor 64 is provided for each of the FD63-1 and FD63-2, the reset timing can be controlled individually for each of the FD63-1 and FD63-2, enabling finer control.When a configuration is adopted in which a common reset transistor 64 is provided for both the FD63-1 and FD63-2, the reset timing can be made the same for the FD63-1 and FD63-2, simplifying control and simplifying the circuit configuration.

[0033] In the following description, an example in which a common reset transistor 64 is provided for the FD 63-1 and the FD 63-2 will be described.

[0034] 4, the distribution of charges in the pixel 50 will be described. Here, distribution means that charges accumulated in the pixel 50 (PD 61) are read out at different timings, thereby reading out the charges for each tap.

[0035] As shown in FIG. 4, the light emitter 14 outputs irradiated light that has been modulated (one period = Tp) so as to repeatedly turn on and off within the irradiation time, and the reflected light is received by the PD 61 after a delay time Td that corresponds to the distance to the object.

[0036] The transfer control signal TRT_A controls the on / off state of the transfer transistor 62-1, and the transfer control signal TRT_B controls the on / off state of the transfer transistor 62-2. As shown in the figure, the transfer control signal TRT_A has the same phase as the irradiated light, while the transfer control signal TRT_B has an inverted phase of the transfer control signal TRT_A.

[0037] Therefore, the charge generated by the photodiode 61 receiving reflected light is transferred to the FD section 63-1 while the transfer transistor 62-1 is turned on in accordance with the transfer control signal TRT_A. Also, the charge is transferred to the FD section 63-2 while the transfer transistor 62-2 is turned on in accordance with the transfer control signal TRT_B. As a result, during a predetermined period in which irradiation light is periodically applied for the irradiation time T, the charge transferred via the transfer transistor 62-1 is sequentially accumulated in the FD section 63-1, and the charge transferred via the transfer transistor 62-2 is sequentially accumulated in the FD section 63-2.

[0038] Then, after the charge accumulation period ends, when the selection transistor 66-1 is turned on in accordance with the selection signal SELm1, the charge accumulated in the FD section 63-1 is read out via the vertical signal line 47-1, and a detection signal A corresponding to the amount of charge is output from the light-receiving section 12. Similarly, when the selection transistor 66-2 is turned on in accordance with the selection signal SELm2, the charge accumulated in the FD section 63-2 is read out via the vertical signal line 47-2, and a detection signal B corresponding to the amount of charge is output from the light-receiving section 12.

[0039] The charge stored in the FD section 63-1 is discharged when the reset transistor 64 is turned on in accordance with the reset signal RST. Similarly, the charge stored in the FD section 63-2 is discharged when the reset transistor 64 is turned on in accordance with the reset signal RST.

[0040] In this way, the pixel 50 can distribute the charge generated by the reflected light received by the photodiode 61 to the tap 51-1 and the tap 51-2 according to the delay time Td, and output the detection signal A and the detection signal B. The delay time Td corresponds to the time it takes for the light emitted by the light-emitting unit 14 to travel to the object, reflect off the object, and then travel to the light-receiving unit 12, i.e., the delay time Td corresponds to the distance to the object. Therefore, the distance measuring device 10 can determine the distance (depth) to the object based on the detection signal A and the detection signal B according to the delay time Td.

[0041] <Range Measurement with Phase Shift on Light Receiving Side> With reference to FIG. 5, a case where light is received with a phase shift on the receiving side and a detection signal is obtained will be described.

[0042] For example, as shown in Fig. 5, four types of light reception are performed with a phase delay of 90 degrees each. That is, based on light reception with a phase delay of 0 degrees, which is light reception without any phase shift from the phase on the irradiating side, light reception with a phase delay of 90 degrees, light reception with a phase delay of 180 degrees, and light reception with a phase delay of 270 degrees are performed, and four periods (quads) are provided for detecting detection signals A and B, respectively.

[0043] For example, a detection period Q0 in which reflected light is detected by receiving light with a phase delay of 0 degrees, a detection period Q1 in which reflected light is detected by receiving light with a phase delay of 90 degrees, a detection period Q2 in which reflected light is detected by receiving light with a phase delay of 180 degrees, and a detection period Q3 in which reflected light is detected by receiving light with a phase delay of 270 degrees are successively provided.

[0044] At the same timing as the start of irradiation light emission (phase delay of 0 degrees), the transfer control signal TRT_A of the tap 51-1 is turned on, and light reception (transfer) begins at the tap 51-1. At the same timing as the transfer control signal TRT_A is turned off, the transfer control signal TRT_B is turned on, and light reception (transfer) begins at the tap 51-2. By receiving light with a phase delay of 0 degrees in this manner, electric charges are distributed to the taps 51-1 and 51-2 in amounts according to the delay time TR, and the electric charges are accumulated respectively. The accumulated electric charges are read out, and detection signals A90 and B90 are output during the detection period Q1.

[0045] At a timing 90 degrees phase-delayed from the start of irradiation light emission, the transfer control signal TRT_A of tap 51-1 is turned on, and light reception (transfer) begins at tap 51-1. Furthermore, at the timing when the transfer control signal TRT_A is turned off, the transfer control signal TRT_B is turned on, and light reception (transfer) begins at tap 51-2. By receiving light with a phase delay of 90 degrees in this manner, electric charges are distributed to tap 51-1 and tap 51-2 in amounts according to the delay time TR, and the electric charges are accumulated respectively. The accumulated electric charges are read out, and detection signals A90 and B90 for detection period Q1 are output.

[0046] At a timing 180 degrees phase delayed from the start of irradiation light emission, the transfer control signal TRT_A of the tap 51-1 is turned on, and light reception (transfer) begins at the tap 51-1. At the timing when the transfer control signal TRT_A is turned off, the transfer control signal TRT_B is turned on, and light reception (transfer) begins at the tap 51-2. By receiving light with a phase delay of 180 degrees in this manner, electric charges are distributed to the taps 51-1 and 51-2 in amounts according to the delay time TR, and the electric charges are accumulated respectively. The accumulated electric charges are read out, and detection signals A180 and B180 are output during the detection period Q2.

[0047] At a timing 270 degrees phase delayed from the start of irradiation light emission, the transfer control signal TRT_A of tap 51-1 is turned on, and light reception (transfer) begins at tap 51-1. Furthermore, at the timing when the transfer control signal TRT_A is turned off, the transfer control signal TRT_B is turned on, and light reception (transfer) begins at tap 51-2. By receiving light with a phase delay of 270 degrees in this manner, electric charges are distributed to tap 51-1 and tap 51-2 in amounts according to the delay time TR, and the electric charges are accumulated respectively. The accumulated electric charges are read out, and detection signal A270 and detection signal B270 are output during detection period Q3.

[0048] Thus, in detection period Q0, detection signals A0 and B0 are detected when light is received with a phase delay of 0 degrees, and in detection period Q1, detection signals A90 and B90 are detected when light is received with a phase delay of 90 degrees. Similarly, in detection period Q2, detection signals A180 and B180 are detected when light is received with a phase delay of 180 degrees, and in detection period Q3, detection signals A270 and B270 are detected when light is received with a phase delay of 270 degrees.

[0049] On the irradiating side, regardless of the phase delay at which light reception starts, there is no phase delay, in other words, irradiation of the irradiating light always starts at the same timing.

[0050] In this way, when receiving light with four phase delays using two taps 51 and measuring the distance to a specified object, processing is performed in the signal processing unit 13 (Figure 1) based on the signals obtained from the four detection periods.

[0051] 6, the detection period of one frame is made up of detection periods Q0, Q1, Q2, and Q3. During detection period Q0, detection signals A0 and B0 are acquired, and during detection period Q1, detection signals A90 and B90 are acquired. During detection period Q2, detection signals A180 and B180 are acquired, and during detection period Q3, detection signals A270 and B270 are acquired.

[0052] The signal processing unit 13 (FIG. 1) uses these detection signals to calculate the phase difference θ based on the following equation (1), calculate the distance D based on the following equation (2), and calculate the reliability c based on the following equation (3).

[0053]

[0054]

[0055]

[0056] In equation (1), I represents the value obtained by subtracting value C180, obtained by subtracting detection signal B180 from detection signal A180, from value C0, obtained by subtracting detection signal B0 from detection signal A0. Q represents the value obtained by subtracting value C270, obtained by subtracting detection signal B270 from detection signal A270, from value C90, obtained by subtracting detection signal B90 from detection signal A90. The phase difference θ is calculated by calculating the arc tangent of (Q / I).

[0057] In equation (2), C is the speed of light, and Tp is the pulse width. The delay time Td can be calculated based on the phase difference θ, and the distance D to the target object can be calculated from the delay time Td.

[0058] Equation (3) is an equation for calculating a value representing the reliability of the calculated distance. The reliability c is found by calculating the square root of the sum of the squares of I and Q. Note that the calculation of the reliability c is not an essential element in calculating the distance D, and it can be omitted. Furthermore, the reliability c may be calculated using an equation other than Equation (3). For example, the sum of the absolute values ​​of I and Q can be used as the reliability c.

[0059] This case of measuring the distance to a predetermined object using irradiation light with four phase differences using two taps 51 is referred to as 2-tap 4-phase. The present technology can also be applied to a case of measuring the distance to a predetermined object using irradiation light with two phase differences using two taps 51, or a case of measuring the distance to a predetermined object using irradiation light with four phase differences using one tap 51.

[0060] <Regarding 2-tap 2-phase> A simple explanation will be given of a case where the distance to a predetermined object is measured by using irradiation light with two phase differences at two taps 51 or by receiving light with two phase differences (hereinafter referred to as 2-tap 2-phase as appropriate). Here, the explanation will continue by taking as an example a case where the distance to a predetermined object is measured by receiving light with two phase differences.

[0061] FIG. 7 is a diagram showing exposure timings of four phases, 0 degrees, 90 degrees, 180 degrees, and 270 degrees, with the beginnings aligned so that the phase differences can be easily seen.

[0062] In practice, as shown in FIG. 6, imaging to acquire detection signals A0 and B0 is performed in detection period Q0, imaging to acquire detection signals A90 and B90 is performed in detection period Q1, imaging to acquire detection signals A180 and B180 is performed in detection period Q2, and imaging to acquire detection signals A270 and B270 is performed in detection period Q3.

[0063] If the imaging operations performed sequentially in the time direction are arranged vertically with the beginnings of the detection periods aligned, it will look like Figure 7. From the beginning of the detection period Q0, exposure for imaging the detection signal A0 is performed, and then exposure for imaging the detection signal B0 is performed.

[0064] From a point 90 degrees out of phase with the beginning of the detection period Q1, exposure for capturing the detection signal A90 is performed, and then exposure for capturing the detection signal B90 is performed.

[0065] From a point 180 degrees out of phase with respect to the beginning of the detection period Q2, exposure for capturing the detection signal A180 is performed, and then exposure for capturing the detection signal B180 is performed.

[0066] From a point 270 degrees out of phase with respect to the beginning of the detection period Q3, exposure for capturing the detection signal A270 is performed, and then exposure for capturing the detection signal B270 is performed.

[0067] Comparing the exposure time of the detection signal B0 during the detection period Q0 and the exposure time of the detection signal A180 during the detection period Q2 reveals that the exposures are performed at the same timing. Therefore, the detection signal A180 during the detection period Q2 can be substituted with the detection signal B0 during the detection period Q0. Similarly, the detection signal B180 during the detection period Q2 can be substituted with the detection signal A0 during the detection period Q0.

[0068] Similarly, comparing the exposure time of the detection signal B90 in the detection period Q1 and the exposure time of the detection signal A270 in the detection period Q3 reveals that the exposures are performed at the same timing. Therefore, the detection signal A270 in the detection period Q3 can be substituted with the detection signal B90 in the detection period Q1. Similarly, the detection signal B270 in the detection period Q3 can be substituted with the detection signal A90 in the detection period Q1.

[0069] For this reason, as shown in FIG. 8, the detection period Q0 and the detection period Q1 are set as the detection period of one frame, and the detection signal A0 and the detection signal B0 are acquired during the detection period Q0.

[0070] The detection signal A0 acquired during this detection period Q0 can be used as the detection signal B180. Furthermore, the detection signal B0 acquired during the detection period Q0 can be used as the detection signal A180. Therefore, in this case, the detection signals A0, B0, A180, and B180 can be treated as if they were acquired during the detection period Q0.

[0071] Furthermore, the detection signal A90 acquired during the detection period Q1 can be used as the detection signal B270. Furthermore, the detection signal B90 acquired during the detection period Q1 can be used as the detection signal A270. Therefore, in this case, the detection signals A90, B90, A270, and B270 can be treated as if they were acquired during the detection period Q1.

[0072] Therefore, the 2-tap 2-phase case described with reference to FIG. 8 can be handled in the same manner as the 2-tap 4-phase case described with reference to FIG.

[0073] The values ​​I and Q in equation (1) for 2 taps and 4 phases described with reference to FIG. 6 are expressed as in the following equation (4).

[0074]

[0075] The value I is obtained by subtracting the detection signal B0 from the detection signal A0, and the value Q is obtained by subtracting the detection signal B90 from the detection signal A90. Since the values ​​I and Q are obtained, the phase difference θ can be calculated by equation (1), and the distance D can be calculated by equation (2), just as in the case of the two-tap method described above.

[0076] <Regarding 1 Tap 4 Phase> A simple explanation will be given below of the case where the distance to a predetermined object is measured by using irradiation light with four phase differences or receiving light with four phase differences with one tap 51 (the configuration of the pixel 50 having one tap 51 is not shown) (hereinafter, referred to as 1 tap 4 phase as appropriate).

[0077] If the imaging order in the time direction for one tap and four phases is expressed in the same way as in Figure 6, it will look like Figure 9. In the detection period Q0, the value C0 in the above-mentioned equation (1) is acquired. In the detection period Q1, the value C90 in the above-mentioned equation (1) is acquired. In the detection period Q2, the value C180 in the above-mentioned equation (1) is acquired. In the detection period Q3, the value C270 in the above-mentioned equation (1) is acquired.

[0078] In the case of the one-tap method, the values ​​I and Q in the above equation (1) are expressed as in the following equation (5).

[0079] Since the values ​​I and Q are obtained, the phase difference θ can be calculated using equation (1) and the distance D can be calculated using equation (2), just as in the case of the two-tap method described above.

[0080] This technology can be applied to the above-mentioned 2-tap 4-phase, 2-tap 2-phase, and 1-tap 4-phase. This technology can also be applied to systems using numbers of taps and phases not shown here, and can be applied regardless of the number of taps or phases.

[0081] 10 is a diagram showing an example of the configuration of an embodiment of a distance measuring device 100 to which the present technology is applied. The distance measuring device 100 is made up of a camera head unit 101 and a signal processing unit 102.

[0082] The camera head unit 101 includes a light-emitting unit 121-1, a light-emitting unit 121-2, a light-receiving unit 122, a lens 123, a light-emitting control unit 124-1, a light-emitting control unit 124-2, a power supply unit 125, a control unit 126, a non-volatile memory unit 127, and a communication unit 128. The camera head unit 101 corresponds to the lens 11, the light-receiving unit 12, the light-emitting unit 14, and the light-emitting control unit 15 of the distance measuring device 10 shown in FIG.

[0083] The signal processing unit 102 includes a communication unit 131, a distance calculation unit 132, a distance correction unit 133, a control signal transmission unit 134, and a communication unit 135. The signal processing unit 13 corresponds to the signal processing unit 13 of the distance measuring device 10 shown in FIG.

[0084] The camera head unit 101 is configured to have two light-emitting units 121. The light emission of the light-emitting unit 121-1 is controlled by a light-emitting control unit 124-1, and the light emission of the light-emitting unit 121-2 is controlled by a light-emitting control unit 124-2. The light emitted by the light-emitting unit 121 strikes the subject, and the reflected light is received by the light-receiving unit 122 via a lens 123. The light-receiving unit 122 has a configuration such as that shown in FIG. 2, for example.

[0085] The power supply unit 125 supplies power to each unit in the camera head unit 101. The power supply unit 125 is connected to the light emission control unit 124-1, the light emission control unit 124-2, and the light receiving unit 122, and supplies the power necessary for each unit to operate. The control unit 126 controls each unit in the camera head unit 101. The non-volatile memory unit 127 stores control information and the like necessary for the control unit 126 to perform control. A signal from the light receiving unit 122 is supplied to the communication unit 128 by the control unit of the control unit 126, and is supplied to the signal processing unit 102 via the communication unit 128.

[0086] The signal processing unit 102 processes the signal supplied from the camera head unit 101. The communication unit 131 of the signal processing unit 102 receives the signal from the camera head unit 101 and supplies it to the distance calculation unit 132. The distance calculation unit 132 calculates the distance to the subject based on the supplied signal information and supplies the calculation result to the distance correction unit 133. The distance correction unit 133 corrects the calculation result of the distance calculation and supplies the corrected distance measurement data to the communication unit 135.

[0087] The communication unit 135 outputs the corrected distance measurement data to the outside as distance information, and transmits and receives control signals from external devices. The control signal transmission unit 134 transmits the control signal input from the outside via the communication unit 135 to the signal processing unit 102.

[0088] In the example shown in Figure 10, the light receiving unit 122 and the light emission control unit 124 have the same reference signal, and for example, the light receiving unit 122 is configured to control the light emission control unit 124 based on the reference signal generated on the light receiving unit 122 side.

[0089] The light receiving unit 122 can be applied to a distance measuring sensor with a structure in which charge is distributed to two charge storage units, such as a distance measuring sensor with a CAPD (Current Assisted Photonic Demodulator) structure or a gate type distance measuring sensor in which photodiode charge is alternately pulsed to two gates. Of course, the light receiving unit 122 can also be applied to a distance measuring sensor with a structure in which charge is distributed to two or more charge storage units.

[0090] The distance measuring device 100 may have a two-tap structure that distributes the signal to two taps as described above, or may have a pixel structure with another number of taps, such as a one-tap structure or a four-tap structure.

[0091] <Regarding the Imaging Area> The distance measuring device 100 shown in Fig. 10 has two light sources, light-emitting unit 121-1 and light-emitting unit 121-2. The imaging area captured by the light emitted from light-emitting unit 121 is, for example, as shown in Fig. 11. In Fig. 11, the imaging area is indicated by a rectangle. On the left side of the figure is illumination area 121-1 illuminated with light from light-emitting unit 121-1, and on the right side of the figure is illumination area 121-2 illuminated with light from light-emitting unit 121-2. In the center of the imaging area is overlapping area 121-12, where illumination area 121-1 and illumination area 121-2 overlap.

[0092] In this way, the imaging area includes three areas: the projection area 121-1, the projection area 121-2, and the overlap area 121-12. Processing to be described later is executed for each area, and distance measurement information is generated.

[0093] <Regarding Driving of Light Emitting Unit> Driving of light emitting unit 121-1, in other words, a method of controlling light emission control unit 124-1 and light emission control unit 124-2 will be described with reference to FIGS. 12 and 13. FIG.

[0094] 12 is a diagram showing the light emitting interval of the light emitting unit 121 and the drive signal of the light receiving unit 122. In FIG. 12, an example is shown in which the light emitting period of the light emitting unit 121 (light source) is set based on the drive signal of the light receiving unit 122.

[0095] The light receiving unit 122 has an integration period and a readout period, during which charge is accumulated, and during the readout period, the amount of charge accumulated during the integration period is read out, and a detection signal is output during the detection period. This process is performed during each of the detection periods Q0, Q90, Q180, and Q270.

[0096] The light emission intervals of the multiple light sources, here light emitters 121-1 and 121-2, are set in accordance with the integration period of light receiver 122, and are controlled by light emission controller 124 so that they emit light in synchronization with the integration period of light receiver 122. The drive pulses of the light sources in one detection period Q, for example detection period Q0 (the period indicated by a circle around the light emission interval in FIG. 12), are shown in FIG.

[0097] During the light emission section of detection period Q0, the light emission of light-emitting unit 121-1 and light-emitting unit 121-2 is controlled by drive signals that are 180 degrees out of phase. During the period when light-emitting unit 121-1 is emitting light, light-emitting unit 121-2 is controlled not to emit light, and during the period when light-emitting unit 121-2 is emitting light, light-emitting unit 121-1 is controlled not to emit light. Control is performed so that the timing of the start of light emission (rising edge) of light-emitting unit 121-1 and light-emitting unit 121-2 is shifted.

[0098] By controlling the light-emitting units 121-1 and 121-2 so that there is a phase shift between them, it is possible to suppress radiation noise emitted to the outside. If the light-emitting units 121-1 and 121-2 were driven in phase, there is a high possibility that radiation noise would be generated due to the superposition of the drive currents of the light sources. In particular, if the light-emitting units 121-1 and 121-2 start emitting light at the same timing, the radiation noise would be large.

[0099] According to this technology, by controlling the drive signal of light-emitting unit 121-1 and the drive signal of light-emitting unit 121-2 so that there is a phase difference, the timing of the start of light emission is shifted and the drive currents are prevented from overlapping, thereby reducing radiation noise.

[0100] The light emission of the light-emitting unit 121 is controlled with this phase difference (predetermined phase amount). The light-receiving unit 122 is also driven in accordance with the driving of the light-emitting unit 121. When so-called two-tap driving and calculation processing is performed to eliminate ambient light and variations in the charge storage units, an operation is performed to distribute signal charges of opposite phases to the charge storage units, as with the gate drive signal shown in Figure 13. In the example shown in Figure 13, when the gate drive signal supplied to the light-receiving unit 122 is high, the charge is distributed to tap A, and when it is low, the charge is distributed to tap B.

[0101] The drive signals of the light-emitting units 121 shown in FIG. 13 each have a duty ratio of 50%, and the phase difference between the drive signal of the light-emitting unit 121-1 and the drive signal of the light-emitting unit 121-2 is 180 degrees.

[0102] The drive signal for light-emitting unit 121-1 and the drive signal for light-emitting unit 121-2 are controlled to have different amplitudes. In the example shown in Fig. 13, the amplitude of the drive signal for light-emitting unit 121-1 is amplitude A+α, and the amplitude of the drive signal for light-emitting unit 121-2 is amplitude A. In other words, the amplitude of the drive signal for light-emitting unit 121-1 is controlled to be larger by α than the amplitude of the drive signal for light-emitting unit 121-2.

[0103] 11, the imaging area includes an overlapping area 121-12 where light from the light-emitting unit 121-1 and light from the light-emitting unit 121-2 overlap. By providing a difference in the emission intensity between the light-emitting unit 121-1 and the light-emitting unit 121-2, the signal in the overlapping area 121-12 can be processed in the same manner as the signal in the irradiation area 121-1, as will be described later, and a ranging signal can also be obtained in the overlapping area 121-12.

[0104] <Other embodiments of drive signals> When the light-emitting units 121-1 and 121-2 are controlled by drive signals having a phase difference, the drive signals may have a phase difference of 180 degrees as shown in FIG. 13, or may have a phase difference as described below.

[0105] 14 to 20 are diagrams showing other drive signals for light-emitting unit 121. The drive signals shown in Fig. 14 show a case where the phase difference between the drive signal for light-emitting unit 121-1 and the drive signal for light-emitting unit 121-2 is Π+β.

[0106] 15 shows a case where the phase difference between the drive signal of light-emitting unit 121-1 and the drive signal of light-emitting unit 121-2 is Π-γ. Even with such a phase difference, radiation noise can be reduced by shifting the timing of light emission start.

[0107] The drive signals shown in FIG. 16 represent a case where the duty ratio of the drive signal for the light-emitting unit 121-1 and the drive signal for the light-emitting unit 121-2 are both 30% and the phase difference is 0.3×Π.

[0108] 17 shows a case where the duty ratio of the drive signal for light-emitting unit 121-1 and the drive signal for light-emitting unit 121-2 are each 70% and the phase difference is 0.7×Π. Even with such duty ratios and a phase difference corresponding to the duty ratios, staggering the timing at which light emission starts can reduce radiation noise.

[0109] The drive signals shown in FIG. 18 are signals in which a square wave and a slope are combined, and the drive signals of the light-emitting unit 121-1 and the light-emitting unit 121-2 have a phase difference.

[0110] The drive signals shown in FIG. 19 are sine waves, and illustrate a case where the drive signal for the light-emitting unit 121-1 and the drive signal for the light-emitting unit 121-2 are signals having a phase difference.

[0111] The drive signals shown in FIG. 20 are triangular waves, and illustrate a case where the drive signal for the light-emitting unit 121-1 and the drive signal for the light-emitting unit 121-2 have a phase difference.

[0112] The waveforms shown in Figures 18, 19, and 20 can be applied to the drive signals with the phase difference described above. The two drive signals shown in Figures 18, 19, and 20 are controlled so that when one drive signal falls, the other drive signal rises. By using a waveform with such a phase difference, the timing at which light emission starts can be shifted, thereby reducing radiation noise.

[0113] The phase difference, duty ratio, and waveform shown here are merely examples, and phase difference, duty ratio, and waveform other than the examples are also within the scope of application of the present technology.

[0114] 13 to 17 show a case where the amplitude of the drive signal for light-emitting unit 121-1 is amplitude A+α, and the amplitude of the drive signal for light-emitting unit 121-2 is amplitude A. An explanation will now be given of the processing performed when distance measurement is performed using light from light-emitting units 121 driven by drive signals with different amplitudes and phase differences.

[0115] As described with reference to Fig. 11, the imaging area is divided into three areas: an illumination area 121-1, an illumination area 121-2, and an overlap area 121-12. Because there is a phase difference in the light irradiated between the illumination area 121-1 and the illumination area 121-2, processing is performed taking this phase difference into consideration. Because the overlap area 121-12 is dominated by light from the light-emitting unit 121-1, which has a large amplitude, processing equivalent to that performed for the illumination area 121-1 is performed. As a result, as shown in Fig. 21, the area can be divided into an area where correction is not performed and an area where correction is performed.

[0116] Here, the explanation will be continued by taking as an example a case where the light from the light-emitting section 121-1 shown in FIG. 13 is used as a reference and the light from the light-emitting section 121-2 has a phase difference of 180 degrees.

[0117] The imaging area shown in Fig. 21 is divided into an area that requires correction due to irradiation with light having a phase difference, and an area that does not require correction. The area that does not require correction includes the irradiation area 121-1 and the overlap area 121-12. The area that requires correction is an area equivalent to the irradiation area 121-2.

[0118] The illumination area 121-1 is an area illuminated with light from the light-emitting unit 121-1. The light from the light-emitting unit 121-1 is emitted based on a drive signal synchronized with the drive timing of the light-receiving unit 122, so that distance measurement data can be calculated by the processing described with reference to FIGS.

[0119] In the overlapping region 121-12, the signal with the larger light amplitude becomes dominant due to tap calculations in the normal iTOF system, and therefore, as an area irradiated with light from the light-emitting unit 121-1, ranging data can be calculated using the same processing as that for the irradiation region 121-1. The overlapping region 121-12 is irradiated with light from the light-emitting unit 121-1, which is emitted based on a drive signal synchronized with the drive timing of the light-receiving unit 122, and ranging data can be calculated using the processing described with reference to FIGS. 1 to 10, as in the processing within the irradiation region 121-1.

[0120] Illumination area 121-2 is an area illuminated with light from light-emitting unit 121-2. The light from light-emitting unit 121-2 has a phase difference with respect to the light from light-emitting unit 121-1, and is emitted based on a drive signal that also has a phase difference with respect to the drive timing of light-receiving unit 122. Therefore, distance measurement data can be calculated by performing the processes described with reference to FIGS. 1 to 10 and a correction process to compensate for the phase difference.

[0121] When the light from light-emitting unit 121-2 is shifted in phase by 180 degrees relative to the light from light-emitting unit 121-1, the signal in illumination area 121-2 is corrected by shifting the phase by 180 degrees, and distance measurement data is calculated.

[0122] In this way, when multiple light-emitting units 121 are provided and a phase difference is set in the light-emitting drive signal, normal processing is performed on the area where light from the light-emitting unit 121 that is synchronized with the drive signal of the light-receiving unit 122 is irradiated, and in addition to normal processing, correction processing to correct the phase difference is performed on the area where light from the light-emitting unit 121 that is not synchronized with the drive signal of the light-receiving unit 122 is irradiated.

[0123] By performing such correction, even in distance measurement using multiple lights with phase differences in the light emission drive signal, it is possible to generate a distance measurement image by processing the image obtained in a single image capture.

[0124] To perform such correction, the distance measuring device 100 holds a table (lookup table) that can distinguish between areas where correction is to be performed and areas where correction is not to be performed, as shown in Fig. 21. The lookup table is stored, for example, in the nonvolatile memory unit 127 (Fig. 10) or the distance correction unit 133 (Fig. 10). The lookup table is created and stored by acquiring the areas in advance through a correction operation. "Advance" refers to, for example, when the distance measuring device 100 is manufactured, when an image is captured for the first time, when the power is turned on, or after a predetermined number of images have been captured.

[0125] 22 is a diagram illustrating another correction method. By performing corrected or uncorrected readout when the light receiving unit 122 (sensor) is driven, it is possible to perform distance measurement processing that absorbs the phase difference during light emission by the light emitting unit 121.

[0126] The light receiving unit 122 is composed of a logic circuit 201 and a photodetector 202. A gate drive signal adjustment circuit 211 is provided within the logic circuit 201. The gate drive signal adjustment circuit 211 is a circuit that adjusts gate drive for each region within the photodetector 202. A gate drive signal is supplied to the gate drive signal adjustment circuit 211 from, for example, the control unit 126 (FIG. 10). This gate drive signal is, for example, the gate drive signal supplied to the light receiving unit 122 shown in FIG. 13, and is a signal synchronized with the drive signal of the light emitting unit 121-1.

[0127] 2, for example, and within the pixel array section 41, a region where the correction amount of the gate drive signal is 0 and a region where the correction amount is +Π are set. When reading from the region where the correction amount of the gate drive signal is 0, the gate drive signal adjustment circuit 211 drives the gate without adjusting the supplied gate drive signal. When reading from the region where the correction amount of the gate drive signal is +Π, the gate drive signal adjustment circuit 211 corrects the phase of the supplied gate drive signal by +Π and drives the gate.

[0128] In this way, by adjusting the gate drive signal of the light receiving section 122, distance measurement can be performed with the phase difference of the light emitting section 121 corrected.

[0129] <Regarding Illumination Region> The above-described embodiment has been described by taking as an example a case where there are two light-emitting units 121 and light is emitted from the two light-emitting units 121. The present technology is not limited to application to two light-emitting units 121, and can also be applied to a case where, for example, light is emitted from three light-emitting units 121, 121-1 to 121-3, and the three illumination regions 121-1 to 121-3 are located within an imaging region, as shown in FIG.

[0130] The example shown in Figure 23 shows a state in which distance measuring device 100 is provided with three light-emitting units 121-1 to 121-3, and the imaging area includes illumination area 121-1 illuminated with light from light-emitting unit 121-1, illumination area 121-2 illuminated with light from light-emitting unit 121-2, and illumination area 121-3 illuminated with light from light-emitting unit 121-3. Illumination area 121 shown in Figure 23 is shown as a square for the sake of explanation, but the shape of the actual illumination area may differ. The same applies below.

[0131] 23, the present technology can be applied to a case where there are three light sources. The present technology can also be applied to a case where four or more light sources are provided in the distance measuring device 100.

[0132] The distance measuring device 100 may also be configured such that multiple light sources are arranged in a state in which there is no area in the imaging area where the light irradiated from the multiple light sources overlaps. Fig. 24 is a diagram showing the imaging area when the distance measuring device 100 includes a light emitting unit 121-1 and a light emitting unit 121-2, and the light from the light emitting units 121 is irradiated so as to avoid an area where the light overlaps. The irradiation of light from the light emitting unit 121-1 and the irradiation of light from the light emitting unit 121-2 are controlled so that there is no area where the irradiation area 121-1 irradiated with light from the light emitting unit 121-1 and the irradiation area 121-2 irradiated with light from the light emitting unit 121-2 overlap.

[0133] As such, the present technology can be applied even when there is no overlapping area. Even when there is no overlapping area, the present technology can be applied even when three light-emitting units 121-1 to 121-3 are provided in the ranging device 100, and the light sources (light-emitting units 121) are arranged in the ranging device 100 so that the imaging area includes an irradiation area 121-1 irradiated with light from the light-emitting unit 121-1, an irradiation area 121-2 irradiated with light from the light-emitting unit 121-2, and an irradiation area 121-3 irradiated with light from the light-emitting unit 121-3, as shown in Fig. 25. This case can also be applied to a case where four or more light sources are provided in the ranging device 100.

[0134] When the arrangement and light emission of the light-emitting units 121 are controlled so that the irradiation areas 121 do not overlap, the amplitudes of the multiple light-emitting units 121 may be the same. For example, the amplitude of light-emitting unit 121-1 shown in FIG. 13 is amplitude A+α, and the amplitude of light-emitting unit 121-2 is amplitude A. However, if there are no overlapping areas in the irradiation areas, the amplitudes of light-emitting unit 121-1 and light-emitting unit 121-2 can both be amplitude A. Even if the light intensities of the multiple light sources are the same, if there are no overlapping areas, the signals in the imaging area can be used to calculate ranging data by normal processing.

[0135] When a configuration is used in which light is irradiated from a plurality of light sources, the wavelengths of the light sources can be the same or different.

[0136] 26 shows a state in which distance measuring device 100 is provided with two light-emitting units, 121-1 and 121-2, and an imaging area includes an illumination area 121-1 illuminated with light from light-emitting unit 121-1 and an illumination area 121-2 illuminated with light from light-emitting unit 121-2. The wavelengths of light-emitting units 121-1 and 121-2 are different.

[0137] 26, the irradiation of light of different wavelengths is represented graphically, with a rectangle representing the area irradiated with light from light-emitting unit 121-1 driven by a drive signal of wavelength A, and a circle representing the area irradiated with light from light-emitting unit 121-2 driven by a drive signal of wavelength B different from wavelength A. Here, the irradiation areas are represented by rectangles and circles to clarify that they are of different wavelengths, but they do not represent the actual shapes of the areas.

[0138] 27 shows a state in which distance measuring device 100 is provided with three light-emitting units 121-1, 121-2, and 121-3, and the imaging area includes illumination area 121-1 illuminated with light from light-emitting unit 121-1, illumination area 121-2 illuminated with light from light-emitting unit 121-2, and illumination area 121-3 illuminated with light from light-emitting unit 121-3. The wavelength of light-emitting unit 121-1 is wavelength A, the wavelength of light-emitting unit 121-2 is wavelength B, and the wavelength of light-emitting unit 121-3 is wavelength C, with wavelengths A, B, and C being different from each other. Alternatively, wavelengths A and B may be the same wavelength, but wavelength C may be a different wavelength from wavelengths A and B.

[0139] 26 and 27 , the present technology can be applied to a distance measuring device 100 including a light-emitting unit 121 that emits light of different wavelengths, and to a case where light of different wavelengths is used. By using different wavelengths, it becomes easier to distinguish areas irradiated with light from different light sources (light-emitting units 122).

[0140] This technology can also be applied to a distance measuring device 100 equipped with an light-emitting unit 121 that emits light of different wavelengths, when the position and light emission of the light-emitting unit 121 are controlled so that there are no overlapping irradiation areas, as shown in Figures 28 and 29.

[0141] 28 is a diagram showing an imaging area when distance measuring device 100 includes light-emitting unit 121-1 and light-emitting unit 121-2, and light of different wavelengths is emitted from these light-emitting units 121 so that there is no overlapping area. The emission of light from light-emitting unit 121-1 and the emission of light from light-emitting unit 121-2 are controlled so that there is no overlapping area between illumination area 121-1 illuminated with light from light-emitting unit 121-1 and illumination area 121-2 illuminated with light from light-emitting unit 121-2, and the emitted light is controlled to be light of different wavelengths.

[0142] In this way, the present technology can be applied even when different wavelengths are used and there is no overlapping area in the irradiation area. Even when there is no overlapping area, the present technology can be applied even when, as shown in Fig. 29, three light-emitting units 121-1 to 121-3 are provided in the distance measuring device 100, the wavelengths of light emitted from these three light-emitting units 122 are controlled to be different, and the light source (light-emitting unit 121) in the distance measuring device 100 is arranged and controlled so that there is no overlapping area in the imaging area among the irradiation area 121-1 illuminated with light from the light-emitting unit 121-1, the irradiation area 121-2 illuminated with light from the light-emitting unit 121-2, and the irradiation area 121-3 illuminated with light from the light-emitting unit 121-3. This case can also be applied to a case where four or more light sources are provided in the distance measuring device 100.

[0143] As shown in Figures 28 and 29, this technology can also be applied to a distance measuring device 100 that has a light emitting unit 121 that emits light of different wavelengths, and in which the light emission is controlled so that the areas irradiated by these lights do not overlap.

[0144] <Configuration of Light Emitting Unit 121> The light emitting unit 121 may be configured to use a light emitting diode (LED), a laser diode (LD), a solid-state laser, a gas laser, a discharge lamp, a light bulb, or the like.

[0145] 30 and 31, for example, the light-emitting unit 121 may be composed of a plurality of LDs. In the example shown in Fig. 30, one light-emitting unit 121 is configured to include three LDs 221-1 to 221-3. The LDs 221-1 to 221-3 are arranged in an array.

[0146] 31, one light-emitting unit 121 is configured to include three LDs 221-1 to 221-3, including LD 221-3, which is driven in a phase different from LD 221-1 and LD 221-2, and LDs 221-1 to 221-3 are arranged in an array. Note that, although an example in which the light-emitting unit 121 is configured with three LDs 221 has been described here, the light-emitting unit 121 may be configured with LDs other than three, or may be configured with LDs 221 arranged in both vertical and horizontal directions.

[0147] When a configuration is adopted in which one light-emitting unit 121 is provided with a plurality of light sources, the light sources may be of the same type as shown in Figures 30 and 31, or may be configured with light sources of different types as shown in Figure 32. The light-emitting unit 121 shown in Figure 32 is configured with an LD 221 and an LED 222. The LED 222 can irradiate the LD 221 with diffused light, and the light-emitting unit 121 may be configured by combining such a diffused light source with the LD 221.

[0148] The light-emitting unit 121 may also be configured by combining a light source with a reflecting member that reflects light. The light-emitting unit 121 shown in FIG. 33 is configured to include an LD 221 and a reflecting member 241. Light from the LD 221 is reflected by the reflecting member 241 and irradiated onto the subject. The reflecting member 241 may be configured with a mirror, a DMD (Digital Micromirror Device), or the like. The reflecting member 241 may be configured to be movable, and the light from the LD 221 may be diffused and irradiated over a predetermined range by scanning the reflecting member 241.

[0149] Fig. 34 is a diagram showing the configuration of a light-emitting unit 121 having another diffusion structure. In the example shown in Fig. 34, the configuration is a combination of an LD 221 and a diffusion member 251 that diffuses light. The light from the LD 221 is diffused and irradiated within a predetermined range by the diffusion member 251. The diffusion member can be formed of a lens, a diffuser, or the like.

[0150] The type of light source of the light emitting unit 121, the combination of multiple light sources, the diffusion method, and the like can be applied to the present technology in addition to those exemplified here.

[0151] <Configuration Related to Control of Light Source> Fig. 35 is a diagram showing an example configuration of the light-emitting unit 121 and the light-emitting control unit 124. The example shown in Fig. 35A shows an example configuration in which there are two light sources, light-emitting unit 121-1 and light-emitting unit 121-2, and two light-emitting control units 124-1 and 124-2 that control the respective light sources. In this case, the light-emitting unit 121 and the light-emitting control unit 124 have a one-to-one relationship. The light-emitting control unit 124 is a control circuit such as a driver IC.

[0152] 35B shows an example of a configuration in which there are two light sources, light-emitting unit 121-1 and light-emitting unit 121-2, and one light-emitting control unit 124 that controls these light sources. In this case, there is a 2-to-1 relationship between light-emitting unit 121 and light-emitting control unit 124.

[0153] 35C shows an example of a configuration in which there are two light sources, light-emitting unit 121-1 and light-emitting unit 121-2, and one control board 271 that controls these light sources. It is also possible to use a configuration in which a control board 271 equipped with a control circuit is used instead of a driver IC.

[0154] It is also possible to have a configuration in which the light emitting unit 121 and the light emission control unit 124 are integrated. The configuration of the light source 301 shown in Fig. 36 includes a phase control unit 311 and a light emitter 312. It is also possible to have a configuration in which the light source 301 itself includes a phase control unit 311 that controls the phase of a drive signal that controls the light emission of the light emitter 312, and the light emitter 312 emits light in response to the drive signal controlled by the phase control unit 311. It is also possible to have a configuration in which the phase control unit 311 is included in the light emitter 312.

[0155] <Embodiment of Wavelength Detection> As described with reference to FIGS. 26 to 29 , when a light-emitting unit 121 that emits light of different wavelengths is used, the light of different wavelengths can be detected by a configuration such as that shown in FIGS. 37 to 38 .

[0156] FIG. 37 is a diagram for explaining a configuration related to the detection of light of different wavelengths. The configuration example shown in FIG. 37 is a configuration example in which the wavelength of light to be received is set for each pixel 50. FIG. 37 shows nine pixels in a 3×3 array on the pixel array unit 41 (FIG. 1). When focusing on four pixels in a 2×2 array, the pixel 50-1 in the upper left receives light with a wavelength of 940 nm, the pixel 50-2 in the upper right receives light with a wavelength of 850 nm, the pixel 50-4 in the lower left receives light with a wavelength of 850 nm, and the pixel 50-5 in the lower right is set to receive light with a wavelength of 940 nm.

[0157] In this case, pixels 50 that receive 950 nm light and pixels 50 that receive 850 nm light are arranged alternately in the vertical and horizontal directions. The wavelength of the light received can be adjusted, for example, by a color filter arranged on the pixel 50.

[0158] Fig. 38 is a diagram for explaining another configuration related to detection of light of different wavelengths. In the configuration example shown in Fig. 38, a filter 501 is provided between the light receiving unit 122 (pixel array unit 41) and the lens 123. The filter 501 includes, for example, a bandpass filter (BPF) 511 that transmits light with a wavelength of 850 nm and a bandpass filter 512 that transmits light with a wavelength of 940 nm.

[0159] For example, the bandpass filter 511 is provided between the pixel array section 41 and the lens 123 so as to cover the pixels in the left half of the pixel array section 41 , and the bandpass filter 512 is provided between the pixel array section 41 and the lens 123 so as to cover the pixels in the right half of the pixel array section 41 .

[0160] The size of the bandpass filters 511 and 512 is not limited to a size that covers half of the pixel array section 41. For example, they may be large enough to cover a plurality of pixels, for example, four pixels, and the bandpass filters 511 and 512 may be arranged in a lattice pattern, with the entire filter 501 covering the pixel array section 41. In this way, the wavelength of the light to be received can be set by dividing the area.

[0161] 39 is a diagram illustrating yet another configuration related to detection of light of different wavelengths. In the configuration example shown in FIG. 39, a filter 531 is provided between the light receiving unit 122 (pixel array unit 41) and the lens 123.

[0162] The filter 531 includes a bandpass filter (BPF) 541 that transmits light with a wavelength of 850 nm and a bandpass filter 542 that transmits light with a wavelength of 940 nm, and is configured to be movable, for example, rotatable.

[0163] For example, the bandpass filter 541 and the bandpass filter 542 are provided between the pixel array unit 41 and the lens 123 so as to cover the entire pixel array unit 41. At a predetermined time t1, the bandpass filter 541 covers the pixel array unit 41, enabling it to receive light of 850 nm, and at the time t2 following time t1, the filter 531 is rotated, allowing the bandpass filter 542 to cover the pixel array unit 41, enabling it to receive light of 940 nm. In this way, it is also possible to configure the system so that the wavelength of the light to be received is set in a time-division manner.

[0164] 5, the distance measuring device 100 receives light in four ways with a phase delay of 90 degrees each as a process related to distance measurement. That is, based on light reception with a phase delay of 0 degrees, which is light reception without any phase shift from the phase on the irradiating side, light reception with a phase delay of 90 degrees, light reception with a phase delay of 180 degrees, and light reception with a phase delay of 270 degrees are performed, and a detection signal is generated at each phase.

[0165] As shown in Figure 40, a configuration may be adopted in which light with different phase delays is received (accumulated) in pixels 50 on the pixel array unit 41. Figure 40 shows nine 3 x 3 pixels on the pixel array unit 41 (Figure 1), and if we focus on four 2 x 2 pixels among them, the pixel 50-1 on the upper left receives light with a phase delay of 0 degrees, the pixel 50-2 on the upper right receives light with a phase delay of 90 degrees, the pixel 50-4 on the lower left receives light with a phase delay of 180 degrees, and the pixel 50-5 on the lower right receives light with a phase delay of 270 degrees. In this way, a configuration may be adopted in which a phase delay for receiving light is set for each pixel, and the pixels are driven based on that setting.

[0166] 41 is a diagram showing an example of the arrangement of the charge accumulation unit 611. The charge accumulation unit 611 is, for example, the FD 63 (FIG. 2) or a memory (not shown). The charge accumulation unit 611-1 is located at the center of the 2×2 pixels 50-1, 50-2, 50-4, and 50-5, and is provided between the pixels. Similarly, the charge accumulation unit 611-2, charge accumulation unit 611-3, and charge accumulation unit 611-4 are also located at the center of the 2×2 four pixels, and are provided between the pixels.

[0167] The present technology can also be applied to a configuration in which the charge storage unit 611 is provided between pixels in this way. The charge storage unit 611 provided between pixels can also be configured to be shared by a plurality of adjacent pixels 50. Although not shown in the figure, the charge storage unit 611 can also be configured to be provided in each pixel 50, and the present technology can also be applied to such a configuration.

[0168] 42 is a diagram showing an example of the arrangement of the charge accumulation unit 611, the charge transfer unit 631, and the ground (GND) unit 633. The charge accumulation unit 611-1 is located at the center of the 2×2 pixels 50-1, 50-2, 50-4, and 50-5, and is provided between the pixels so as to be shared by these pixels.

[0169] Charge transfer units 631-1 to 631-4 are provided in each of the pixels 50-1, 50-2, 50-4, and 50-5, and are configured to transfer charges read from the PD to the charge accumulation unit 611-1. The ground unit 633-1 is located at the center of the 2×2 pixels 50-2, 50-3, 50-5, and 50-6, and is provided between the pixels.

[0170] 43 is a diagram showing an example of the arrangement of the charge transfer section 631 and the charge accumulation section 651. A charge transfer section 631-1 and a charge accumulation section 651-1 that temporarily accumulates the charge transferred from the charge transfer section 631-1 or accumulates the charge that overflows from the PD are arranged in the pixel 50-1. Similarly, a charge transfer section 631-2 and a charge accumulation section 651-2 are arranged in the pixel 50-2. The charge accumulation section 651 is a memory. The present technology can also be applied to a configuration of the pixel 50 in which a memory is arranged.

[0171] This technology can be applied regardless of the position of the charge storage unit 611, and regardless of whether it is shared or not. This technology is not dependent on the position of the charge transfer unit 631 or the position of the ground unit 633, and can be applied regardless of whether it is shared or not. This technology can also be applied to a case where a memory is provided as the charge storage unit.

[0172] This technology can be applied regardless of whether a color filter is used or not. This technology can be applied regardless of the rolling shutter method or global shutter method. This technology can be applied regardless of coding.

[0173] <Other forms of imaging area and overlapping area> The above-described embodiment has been described with reference to FIG. 11 as an example in which there are three areas in the imaging area: the projection area 121-1, the projection area 121-2, and the overlapping area 121-12.

[0174] As shown in FIG. 44, the present technology can also be applied to a case where there is only one overlapping area 121-12 within the imaging area.

[0175] It is assumed that distance measuring device 100 has two light sources, light-emitting unit 121-1 and light-emitting unit 121-2. In Fig. 44, the imaging area is shown as a rectangle. On the left side of the figure is illumination area 121-1 illuminated with light from light-emitting unit 121-1, and on the right side of the figure is illumination area 121-2 illuminated with light from light-emitting unit 121-2. Part of the imaging area includes overlap area 121-12 where illumination area 121-1 and illumination area 121-2 overlap.

[0176] In this way, when there is only the overlapping area 121-12 within the imaging area, the above-described processing is performed on the overlapping area 121-12, and distance measurement information can be generated in the same way as in the above-described embodiment.

[0177] According to the present technology, radiation noise can be reduced.

[0178] <Application Example to Endoscopic Surgery System> The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology according to the present disclosure may be applied to an endoscopic surgery system.

[0179] FIG. 45 is a diagram showing an example of a schematic configuration of an endoscopic surgery system to which the technology according to the present disclosure (the present technology) can be applied.

[0180] Figure 45 shows an operator (doctor) 11131 performing surgery on a patient 11132 on a patient bed 11133 using an endoscopic surgery system 11000. As shown in the figure, the endoscopic surgery system 11000 is composed of an endoscope 11100, other surgical tools 11110 such as an insufflation tube 11111 and an energy treatment tool 11112, a support arm device 11120 that supports the endoscope 11100, and a cart 11200 on which various devices for endoscopic surgery are mounted.

[0181] The endoscope 11100 is composed of a lens barrel 11101, a region of a predetermined length from the tip of which is inserted into a body cavity of a patient 11132, and a camera head 11102 connected to the base end of the lens barrel 11101. In the example shown, the endoscope 11100 is configured as a so-called rigid scope having a rigid lens barrel 11101, but the endoscope 11100 may also be configured as a so-called flexible scope having a flexible lens barrel.

[0182] An opening into which an objective lens is fitted is provided at the tip of the lens barrel 11101. A light source device 11203 is connected to the endoscope 11100, and light generated by the light source device 11203 is guided to the tip of the lens barrel by a light guide extending inside the lens barrel 11101, and is irradiated via the objective lens toward an object to be observed inside the body cavity of the patient 11132. The endoscope 11100 may be a direct-viewing endoscope, an oblique-viewing endoscope, or a side-viewing endoscope.

[0183] An optical system and an image sensor are provided inside the camera head 11102, and light reflected from the object of observation (observation light) is collected onto the image sensor by the optical system. The observation light is photoelectrically converted by the image sensor to generate an electrical signal corresponding to the observation light, i.e., an image signal corresponding to the observed image. The image signal is sent to a camera control unit (CCU) 11201 as RAW data.

[0184] The CCU 11201 is configured with a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc., and comprehensively controls the operations of the endoscope 11100 and the display device 11202. Furthermore, the CCU 11201 receives an image signal from the camera head 11102 and performs various types of image processing on the image signal, such as development processing (demosaic processing), to display an image based on the image signal.

[0185] Under the control of the CCU 11201, the display device 11202 displays an image based on an image signal that has been subjected to image processing by the CCU 11201.

[0186] The light source device 11203 is composed of a light source such as an LED (light emitting diode), and supplies irradiation light to the endoscope 11100 when photographing the surgical site, etc.

[0187] The input device 11204 is an input interface for the endoscopic surgery system 11000. A user can input various information and instructions to the endoscopic surgery system 11000 via the input device 11204. For example, the user inputs an instruction to change the imaging conditions (type of irradiation light, magnification, focal length, etc.) of the endoscope 11100.

[0188] The treatment tool control device 11205 controls the driving of the energy treatment tool 11112 for cauterizing tissue, incising, sealing blood vessels, etc. The insufflation device 11206 inflates the body cavity of the patient 11132 through the insufflation tube 11111 in order to ensure a clear field of view for the endoscope 11100 and a working space for the surgeon. The recorder 11207 is a device capable of recording various types of information related to the surgery. The printer 11208 is a device capable of printing various types of information related to the surgery in various formats such as text, images, or graphs.

[0189] The light source device 11203, which supplies illumination light to the endoscope 11100 when photographing the surgical site, can be configured from a white light source, such as an LED, a laser light source, or a combination of these. When the white light source is configured from a combination of RGB laser light sources, the output intensity and output timing of each color (each wavelength) can be controlled with high precision, allowing the light source device 11203 to adjust the white balance of the captured image. In this case, it is also possible to time-share images corresponding to each RGB by irradiating the object of observation with laser light from each RGB laser light source and controlling the drive of the image sensor of the camera head 11102 in synchronization with the irradiation timing. According to this method, color images can be obtained without providing a color filter to the image sensor.

[0190] Furthermore, the light source device 11203 may be controlled to change the intensity of light it outputs at predetermined time intervals. By controlling the driving of the image sensor of the camera head 11102 in synchronization with the timing of the change in light intensity to acquire images in a time-division manner and combining the images, it is possible to generate an image with a high dynamic range that is free from so-called blocked-up shadows and blown-out highlights.

[0191] The light source device 11203 may also be configured to supply light in a predetermined wavelength band corresponding to special light observation. Special light observation, for example, utilizes the wavelength dependence of light absorption in body tissues to irradiate light with a narrower band than the light irradiated during normal observation (i.e., white light), thereby capturing high-contrast images of specific tissues, such as blood vessels on the surface of mucous membranes, known as narrow-band imaging. Alternatively, special light observation may involve fluorescence observation, in which images are obtained using fluorescence generated by irradiating excitation light. Fluorescence observation may involve irradiating excitation light onto body tissues and observing the fluorescence from the tissue (autofluorescence observation), or by locally injecting a reagent such as indocyanine green (ICG) into the body tissue and irradiating the tissue with excitation light corresponding to the fluorescent wavelength of the reagent to obtain a fluorescent image. The light source device 11203 may be configured to supply narrow-band light and / or excitation light corresponding to such special light observation.

[0192] FIG. 46 is a block diagram showing an example of the functional configuration of the camera head 11102 and the CCU 11201 shown in FIG.

[0193] The camera head 11102 has a lens unit 11401, an imaging unit 11402, a drive unit 11403, a communication unit 11404, and a camera head control unit 11405. The CCU 11201 has a communication unit 11411, an image processing unit 11412, and a control unit 11413. The camera head 11102 and the CCU 11201 are connected to each other via a transmission cable 11400 so that they can communicate with each other.

[0194] The lens unit 11401 is an optical system provided at the connection portion with the lens barrel 11101. Observation light taken in from the tip of the lens barrel 11101 is guided to the camera head 11102 and enters the lens unit 11401. The lens unit 11401 is composed of a combination of multiple lenses including a zoom lens and a focus lens.

[0195] The imaging unit 11402 may include one imaging element (a so-called single-chip type) or multiple imaging elements (a so-called multi-chip type). When the imaging unit 11402 is configured as a multi-chip type, for example, each imaging element may generate an image signal corresponding to each of RGB, and a color image may be obtained by combining these signals. Alternatively, the imaging unit 11402 may be configured to have a pair of imaging elements for acquiring image signals for the right eye and the left eye corresponding to a 3D (dimensional) display. The 3D display allows the surgeon 11131 to more accurately grasp the depth of the biological tissue at the surgical site. Note that when the imaging unit 11402 is configured as a multi-chip type, multiple lens units 11401 may be provided corresponding to each imaging element.

[0196] Furthermore, the imaging unit 11402 does not necessarily have to be provided in the camera head 11102. For example, the imaging unit 11402 may be provided inside the lens barrel 11101, immediately after the objective lens.

[0197] The driving unit 11403 is configured by an actuator, and moves the zoom lens and focus lens of the lens unit 11401 by a predetermined distance along the optical axis under the control of the camera head control unit 11405. This allows the magnification and focus of the image captured by the imaging unit 11402 to be adjusted appropriately.

[0198] The communication unit 11404 is configured by a communication device for transmitting and receiving various types of information to and from the CCU 11201. The communication unit 11404 transmits the image signal obtained from the imaging unit 11402 to the CCU 11201 via the transmission cable 11400 as RAW data.

[0199] Furthermore, the communication unit 11404 receives a control signal for controlling the driving of the camera head 11102 from the CCU 11201 and supplies the control signal to the camera head control unit 11405. The control signal includes information on the imaging conditions, such as information specifying the frame rate of the captured image, information specifying the exposure value at the time of imaging, and / or information specifying the magnification and focus of the captured image.

[0200] The image capturing conditions such as the frame rate, exposure value, magnification, and focus may be appropriately specified by the user, or may be automatically set by the control unit 11413 of the CCU 11201 based on the acquired image signal. In the latter case, the endoscope 11100 is equipped with a so-called AE (Auto Exposure) function, AF (Auto Focus) function, and AWB (Auto White Balance) function.

[0201] The camera head control unit 11405 controls the driving of the camera head 11102 based on a control signal received from the CCU 11201 via the communication unit 11404 .

[0202] The communication unit 11411 is configured by a communication device for transmitting and receiving various information to and from the camera head 11102. The communication unit 11411 receives an image signal transmitted from the camera head 11102 via the transmission cable 11400.

[0203] Furthermore, the communication unit 11411 transmits to the camera head 11102 a control signal for controlling the driving of the camera head 11102. The image signal and the control signal can be transmitted by electrical communication, optical communication, or the like.

[0204] The image processing unit 11412 performs various image processing operations on the image signal, which is RAW data transmitted from the camera head 11102 .

[0205] The control unit 11413 performs various controls related to the imaging of the surgical site, etc. by the endoscope 11100 and the display of the captured image obtained by imaging the surgical site, etc. For example, the control unit 11413 generates a control signal for controlling the driving of the camera head 11102.

[0206] Furthermore, the control unit 11413 displays the captured image showing the surgical site, etc., on the display device 11202 based on the image signal subjected to image processing by the image processing unit 11412. At this time, the control unit 11413 may recognize various objects in the captured image using various image recognition technologies. For example, the control unit 11413 can recognize surgical tools such as forceps, specific biological parts, bleeding, mist generated when using the energy treatment tool 11112, etc., by detecting the shape and color of the edges of objects included in the captured image. When displaying the captured image on the display device 11202, the control unit 11413 may use the recognition results to superimpose various surgical support information on the image of the surgical site. By superimposing the surgical support information and presenting it to the surgeon 11131, the burden on the surgeon 11131 can be reduced and the surgeon 11131 can proceed with the surgery reliably.

[0207] The transmission cable 11400 connecting the camera head 11102 and the CCU 11201 is an electrical signal cable for electrical signal communication, an optical fiber for optical communication, or a composite cable of these.

[0208] In the illustrated example, communication is performed wired using a transmission cable 11400, but communication between the camera head 11102 and the CCU 11201 may also be performed wirelessly.

[0209] <Application to a Mobile Body> The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology according to the present disclosure may be realized as a device mounted on any type of mobile body, such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, personal mobility, an airplane, a drone, a ship, or a robot.

[0210] FIG. 47 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile object control system to which the technology of the present disclosure can be applied.

[0211] The vehicle control system 12000 includes a plurality of electronic control units connected via a communication network 12001. In the example shown in Fig. 47, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside-vehicle information detection unit 12030, an inside-vehicle information detection unit 12040, and an integrated control unit 12050. Also shown as functional components of the integrated control unit 12050 are a microcomputer 12051, an audio / video output unit 12052, and an in-vehicle network I / F (Interface) 12053.

[0212] The drivetrain control unit 12010 controls the operation of devices related to the drivetrain of the vehicle in accordance with various programs. For example, the drivetrain control unit 12010 functions as a control device for a drive force generating device for generating a drive force of the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, and a braking device for generating a braking force of the vehicle.

[0213] The body system control unit 12020 controls the operation of various devices equipped in the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as headlamps, backup lamps, brake lamps, turn signals, and fog lamps. In this case, radio waves transmitted from a portable device that serves as a key or signals from various switches can be input to the body system control unit 12020. The body system control unit 12020 receives these radio waves or signals and controls the vehicle's door lock device, power window device, lamps, etc.

[0214] The outside-vehicle information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the outside-vehicle information detection unit 12030. The outside-vehicle information detection unit 12030 causes the imaging unit 12031 to capture images outside the vehicle and receives the captured images. The outside-vehicle information detection unit 12030 may perform object detection processing or distance detection processing for people, cars, obstacles, signs, characters on the road surface, etc. based on the received images.

[0215] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 can output the electrical signal as an image or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0216] The in-vehicle information detection unit 12040 detects information inside the vehicle. For example, a driver state detection unit 12041 that detects the state of the driver is connected to the in-vehicle information detection unit 12040. The driver state detection unit 12041 includes, for example, a camera that captures an image of the driver, and the in-vehicle information detection unit 12040 may calculate the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041, or may determine whether the driver is dozing off.

[0217] The microcomputer 12051 can calculate control target values ​​for the driving force generating device, steering mechanism, or braking device based on the information inside and outside the vehicle acquired by the outside-vehicle information detection unit 12030 or the inside-vehicle information detection unit 12040, and output control commands to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing the functions of an ADAS (Advanced Driver Assistance System), including vehicle collision avoidance or impact mitigation, following driving based on the distance between vehicles, maintaining vehicle speed, vehicle collision warning, vehicle lane departure warning, etc.

[0218] In addition, the microcomputer 12051 can perform cooperative control for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on driver operation, by controlling the driving force generating device, steering mechanism, braking device, etc. based on information about the surroundings of the vehicle obtained by the outside vehicle information detection unit 12030 or the inside vehicle information detection unit 12040.

[0219] Furthermore, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information outside the vehicle acquired by the outside information detection unit 12030. For example, the microcomputer 12051 can control the headlamps according to the position of a preceding vehicle or an oncoming vehicle detected by the outside information detection unit 12030, and perform cooperative control aimed at preventing glare, such as switching from high beams to low beams.

[0220] The audio / video output unit 12052 transmits at least one of audio and video output signals to an output device capable of visually or audibly notifying information to vehicle occupants or the outside of the vehicle. In the example of Fig. 47, the output devices are exemplified by an audio speaker 12061, a display unit 12062, and an instrument panel 12063. The display unit 12062 may include, for example, at least one of an on-board display and a head-up display.

[0221] FIG. 48 is a diagram showing an example of the installation position of the imaging unit 12031.

[0222] In FIG. 48, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

[0223] The imaging units 12101, 12102, 12103, 12104, and 12105 are provided, for example, at positions such as the front nose, side mirrors, rear bumper, back door, and the top of the windshield inside the vehicle cabin of the vehicle 12100. The imaging unit 12101 provided on the front nose and the imaging unit 12105 provided on the top of the windshield inside the vehicle cabin mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 provided on the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 provided on the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 provided on the top of the windshield inside the vehicle cabin is mainly used to detect preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, lanes, etc.

[0224] 48 shows an example of the imaging ranges of the imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of the imaging unit 12101 provided on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of the imaging units 12102 and 12103 provided on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of the imaging unit 12104 provided on the rear bumper or back door. For example, by overlaying the image data captured by the imaging units 12101 to 12104, an overhead image of the vehicle 12100 viewed from above can be obtained.

[0225] At least one of the image capturing units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the image capturing units 12101 to 12104 may be a stereo camera made up of multiple image capturing elements, or may be an image capturing element having pixels for phase difference detection.

[0226] For example, based on the distance information obtained from the imaging units 12101 to 12104, the microcomputer 12051 can calculate the distance to each three-dimensional object within the imaging ranges 12111 to 12114 and the change in this distance over time (relative speed with respect to the vehicle 12100), thereby extracting as a preceding vehicle, in particular, the three-dimensional object that is the closest three-dimensional object on the path of the vehicle 12100 and traveling in approximately the same direction as the vehicle 12100 at a predetermined speed (e.g., 0 km / h or higher). Furthermore, the microcomputer 12051 can set a vehicle-to-vehicle distance to be maintained in advance in front of the preceding vehicle, and perform automatic braking control (including follow-up stop control), automatic acceleration control (including follow-up start control), etc. In this way, cooperative control can be performed for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on driver operation.

[0227] For example, the microcomputer 12051 classifies and extracts three-dimensional object data regarding three-dimensional objects into two-wheeled vehicles, ordinary vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects based on distance information obtained from the imaging units 12101 to 12104, and can use the data for automatic obstacle avoidance. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 then determines a collision risk that indicates the risk of collision with each obstacle, and when the collision risk is equal to or greater than a set value and a collision is possible, the microcomputer 12051 can provide driving assistance for collision avoidance by outputting an alarm to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or avoidance steering via the drive system control unit 12010.

[0228] At least one of the image capturing units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether a pedestrian is present in the images captured by the image capturing units 12101 to 12104. Such pedestrian recognition is performed, for example, by extracting feature points from the images captured by the image capturing units 12101 to 12104 as infrared cameras and performing pattern matching on a series of feature points that indicate the outline of an object to determine whether the object is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the images captured by the image capturing units 12101 to 12104 and recognizes the pedestrian, the audio / image output unit 12052 controls the display unit 12062 to superimpose a rectangular outline on the recognized pedestrian for emphasis. The audio / image output unit 12052 may also control the display unit 12062 to display an icon or the like indicating the pedestrian at a desired position.

[0229] In this specification, a system refers to an entire device that is made up of multiple devices.

[0230] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0231] It should be noted that the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible within the scope of the present technology.

[0232] The present technology may also be configured as follows. (1) A distance measuring device including a plurality of light-emitting units including a first light-emitting unit and a second light-emitting unit that irradiate a subject with light; an emission control unit that controls the emission of each of the plurality of light-emitting units; a light-receiving unit that receives light reflected from the subject; and a distance measuring unit that calculates a distance to the subject based on a signal obtained by the received light, wherein the emission control unit drives the first light-emitting unit based on a first drive signal and drives the second light-emitting unit based on a second drive signal, and controls the first drive signal and the second drive signal to have a predetermined phase difference. (2) The distance measuring device according to (1), wherein the first drive signal is synchronized with a drive signal that drives the light-receiving unit. (3) The distance measuring device according to (1), wherein there is a phase difference of 180 degrees between the first drive signal and the second drive signal. (4) The distance measuring device according to (1), wherein the amplitude of the first drive signal and the amplitude of the second drive signal are different. (5) The distance measuring device according to (4), wherein the imaging area includes a first illumination area illuminated with light from the first light-emitting unit, a second illumination area illuminated with light from the second light-emitting unit, and an overlapping area where the first illumination area and the second illumination area overlap. (6) The distance measuring device according to (5), wherein the first illumination area and the overlapping area calculate the distance without correction, and the second illumination area calculates the distance by correcting the phase difference. (7) The distance measuring device according to (6), wherein a lookup table indicating the area where the distance is calculated without correction and the area where the distance is calculated by correction is stored. (8) The distance measuring device according to (5), wherein the light-receiving unit drives the first illumination area and the overlapping area without correction, and drives the second illumination area by correcting the phase difference. (9) The distance measuring device according to (1), wherein the emission wavelength of the first light-emitting unit and the emission wavelength of the second light-emitting unit are different. (10) The distance measuring device according to (9) above, in which the wavelength of light received varies on a pixel-by-pixel basis. (11) The distance measuring device according to (9) above, in which the wavelength of light received varies for each predetermined region. (12) The distance measuring device according to (9) above, in which the wavelength of light received is changed in a time-division manner.(13) A distance measuring method in which the distance measuring device includes a plurality of light-emitting units including a first light-emitting unit and a second light-emitting unit that irradiate light onto a subject, a light-emitting control unit that controls the light emission of each of the plurality of light-emitting units, a light-receiving unit that receives light reflected from the subject, and a distance measuring unit that calculates the distance to the subject based on a signal obtained by the received light, the light-emitting control unit drives the first light-emitting unit based on a first drive signal, and drives the second light-emitting unit based on a second drive signal, and controls the first drive signal and the second drive signal to have a predetermined phase difference.

[0233] REFERENCE SIGNS LIST 10 Range finding device, 11 Lens, 12 Light receiving unit, 13 Signal processing unit, 14 Light emitting unit, 15 Light emission control unit, 16 Filter unit, 41 Pixel array unit, 42 Vertical driving unit, 43 Column processing unit, 44 Horizontal driving unit, 45 System control unit, 46 Pixel driving line, 47 Vertical signal line, 48 Signal processing unit, 50 Pixel, 51 Tap, 61 Photodiode, 62 Transfer transistor, 63 FD unit, 64 Reset transistor, 65 Amplification transistor, 66 Selection transistor, 100 Range finding system, 101 Camera head unit, 102 Signal processing unit, 121 Light emitting unit, 122 Light receiving unit, 123 Lens, 124 Light emission control unit, 125 Power supply unit, 126 Control unit, 127 Non-volatile memory unit, 128 communication unit, 131 communication unit, 132 distance calculation unit, 133 distance correction unit, 134 control signal transmission unit, 135 communication unit, 201 logic circuit, 202 photodetector, 211 gate drive signal adjustment circuit, 222 LED, 241 reflecting member, 251 diffusion member, 271 control board, 301 light source, 311 phase control unit, 312 light emitter, 501 filter, 511, 512 band pass filter, 531 filter, 541, 542 band pass filter, 611 charge storage unit, 631 charge transfer unit, 633 ground unit, 651 charge storage unit

Claims

1. A distance measuring device comprising: a plurality of light-emitting units including a first light-emitting unit and a second light-emitting unit that irradiate a subject with light; a light-emitting control unit that controls the light emission of each of the plurality of light-emitting units; a light-receiving unit that receives light reflected from the subject; and a distance measuring unit that calculates the distance to the subject based on a signal obtained by the received light, wherein the light-emitting control unit drives the first light-emitting unit based on a first drive signal and drives the second light-emitting unit based on a second drive signal, and controls the first drive signal and the second drive signal to have a predetermined phase difference.

2. The distance measuring device according to claim 1, wherein the first drive signal is synchronized with a drive signal that drives the light receiving section.

3. The distance measuring device according to claim 1, wherein there is a phase difference of 180 degrees between the first drive signal and the second drive signal.

4. The distance measuring device according to claim 1, wherein the amplitude of the first drive signal and the amplitude of the second drive signal are different.

5. A distance measuring device as described in claim 4, wherein the imaging area includes a first illumination area illuminated by light from the first light-emitting unit, a second illumination area illuminated by light from the second light-emitting unit, and an overlap area where the first illumination area and the second illumination area overlap.

6. The distance measuring device according to claim 5, wherein the distance is calculated without correction for the first illumination area and the overlapping area, and the distance is calculated by correcting the phase difference for the second illumination area.

7. The distance measuring device according to claim 6, further comprising a lookup table indicating an area for which the distance is calculated without correction and an area for which the distance is calculated with correction.

8. The distance measuring device according to claim 5, wherein the light receiving section is driven without correction for the first irradiation area and the overlapping area, and the second irradiation area is driven with correction for the phase difference.

9. The distance measuring device according to claim 1, wherein the emission wavelength of the first light emitting unit and the emission wavelength of the second light emitting unit are different.

10. The distance measuring device according to claim 9, wherein the wavelength of light received varies on a pixel-by-pixel basis.

11. The distance measuring device according to claim 9, wherein the wavelength of light received varies for each predetermined area.

12. The distance measuring device according to claim 9, wherein the wavelength of the light received is changed in a time-division manner.

13. A distance measuring method in which a distance measuring device includes: a plurality of light-emitting units, including a first light-emitting unit and a second light-emitting unit, that irradiate a subject with light; a light-emitting control unit that controls the light emission of each of the plurality of light-emitting units; a light-receiving unit that receives light reflected from the subject; and a distance measuring unit that calculates the distance to the subject based on a signal obtained by the light-receiving; the light-emitting control unit drives the first light-emitting unit based on a first drive signal, and drives the second light-emitting unit based on a second drive signal, and controls the first drive signal and the second drive signal to have a predetermined phase difference.

Images