Distance measuring device and distance measuring method

Abstract

To suppress generation of radiation noise.SOLUTION: A distance measuring device comprising: a plurality of light emitting units including a first light emitting unit and a second light emitting unit that emit light at a subject; a light-emission control unit that controls light emission of each of the plurality of light emitting units; a light receiving unit that receives reflected light from the subject; and a distance measuring unit that calculates a distance to the subject on the basis of a signal obtained by the light reception, wherein the light-emission control unit drives the first light emitting unit on the basis of a first drive signal, drives the second light emitting unit on the basis of a second drive signal, and controls the first drive signal and the second drive signal to have a prescribed phase difference. The present technology is applicable, for example, to distance measuring devices.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] The present technology relates to a distance measuring device and a distance measuring method, and more particularly to a distance measuring device and a distance measuring method that can suppress the generation of radiation noise, for example. [Background technology]

[0002] In recent years, advances in semiconductor technology have led to the miniaturization of distance measuring devices that measure the distance to an object. This has made it possible to incorporate distance measuring devices into mobile terminals such as smartphones, which are small information processing devices equipped with communication functions. Examples of distance measuring devices (sensors) that measure the distance to an object include TOF (Time Of Flight) sensors (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-090268 Summary of the Invention [Problem to be solved by the invention]

[0004] When measuring the distance to a subject by irradiating the subject with light from multiple light-emitting elements, the measurement accuracy can be improved, but there is a possibility that the radiation noise emitted from the distance measuring device to the outside will be large. It is desired to improve the measurement accuracy while suppressing the generation of radiation noise.

[0005] The present technology has been made in view of such circumstances, and is intended to make it possible to suppress the generation of radiation noise. [Means for solving the problem]

[0006] A distance measuring device according to one aspect of the present technology includes a plurality of light-emitting units including a first light-emitting unit and a second light-emitting unit that irradiate light onto a subject, a light-emitting control unit that controls the light emission of each of the plurality of light-emitting units, a light-receiving unit that receives light reflected from the subject, and a distance measuring unit that calculates a distance to the subject based on a signal obtained by the received light, wherein the light-emitting control unit drives the first light-emitting unit based on a first drive signal and drives the second light-emitting unit based on a second drive signal, and controls the first drive signal and the second drive signal to have a predetermined phase difference.

[0007] A ranging method according to one aspect of the present technology is a ranging device including a plurality of light-emitting units, including a first light-emitting unit and a second light-emitting unit, that irradiate light onto a subject, a light-emitting control unit that controls the light emission of each of the plurality of light-emitting units, a light-receiving unit that receives light reflected from the subject, and a ranging unit that calculates a distance to the subject based on a signal obtained by the light-receiving, wherein the light-emitting control unit drives the first light-emitting unit based on a first drive signal and drives the second light-emitting unit based on a second drive signal, and controls the first drive signal and the second drive signal to have a predetermined phase difference.

[0008] A ranging device and ranging method according to one aspect of the present technology include a plurality of light-emitting units including a first light-emitting unit and a second light-emitting unit that irradiate light onto a subject, a light-emitting control unit that controls the light emission of each of the plurality of light-emitting units, a light-receiving unit that receives reflected light from the subject, and a ranging unit that calculates the distance to the subject based on a signal obtained by receiving the light, and the light-emitting control unit drives the first light-emitting unit based on a first drive signal and drives the second light-emitting unit based on a second drive signal, and controls the first drive signal and the second drive signal to have a predetermined phase difference.

[0009] The distance measuring device may be an independent device or an internal block constituting a single device. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 illustrates an example of the configuration of a distance measuring device. [Figure 2] FIG. 2 is a diagram illustrating an example of the configuration of a light receiving unit. [Figure 3] FIG. 2 is a diagram illustrating an example of a pixel configuration. [Figure 4] FIG. 2 is a diagram illustrating distribution of charges in a pixel. [Figure 5] FIG. 10 is a diagram showing an example of four types of received light with a phase delay of 90 degrees each. [Figure 6] FIG. 2 is a diagram for explaining a detection signal in one frame. [Figure 7] 10A and 10B are diagrams for explaining the relationship between a detection period and a detection signal. [Figure 8] FIG. 2 is a diagram for explaining a detection signal in one frame. [Figure 9] FIG. 2 is a diagram for explaining a detection signal in one frame. [Figure 10] 1 is a diagram illustrating a configuration of an embodiment of a ranging system to which the present technology is applied. [Figure 11] FIG. 2 is a diagram for explaining an imaging area. [Figure 12] FIG. 4 is a diagram for explaining a drive signal. [Figure 13] FIG. 4 is a diagram for explaining a drive signal. [Figure 14] FIG. 4 is a diagram for explaining a drive signal. [Figure 15] FIG. 4 is a diagram for explaining a drive signal. [Figure 16] FIG. 4 is a diagram for explaining a drive signal. [Figure 17] FIG. 4 is a diagram for explaining a drive signal. [Figure 18] FIG. 4 is a diagram for explaining a drive signal. [Figure 19] FIG. 4 is a diagram for explaining a drive signal. [Figure 20] FIG. 4 is a diagram for explaining a drive signal. [Figure 21] FIG. 10 is a diagram for explaining a correction area within an imaging area. [Figure 22]FIG. 10 is a diagram for explaining correction during read driving. [Figure 23] FIG. 2 is a diagram for explaining an imaging area. [Figure 24] FIG. 2 is a diagram for explaining an imaging area. [Figure 25] FIG. 2 is a diagram for explaining an imaging area. [Figure 26] FIG. 2 is a diagram for explaining an imaging area. [Figure 27] FIG. 2 is a diagram for explaining an imaging area. [Figure 28] FIG. 2 is a diagram for explaining an imaging area. [Figure 29] FIG. 2 is a diagram for explaining an imaging area. [Figure 30] FIG. 2 is a diagram illustrating a configuration of a light-emitting unit. [Figure 31] FIG. 2 is a diagram illustrating a configuration of a light-emitting unit. [Figure 32] FIG. 2 is a diagram illustrating a configuration of a light-emitting unit. [Figure 33] FIG. 2 is a diagram illustrating a configuration of a light-emitting unit. [Figure 34] FIG. 2 is a diagram illustrating a configuration of a light-emitting unit. [Figure 35] FIG. 2 is a diagram illustrating a configuration of a light emission control unit. [Figure 36] FIG. 2 is a diagram illustrating a configuration of a light emission control unit. [Figure 37] FIG. 10 is a diagram illustrating an example of a configuration related to detection of different wavelengths. [Figure 38] FIG. 10 is a diagram illustrating an example of a configuration related to detection of different wavelengths. [Figure 39] FIG. 10 is a diagram illustrating an example of a configuration related to detection of different wavelengths. [Figure 40] 10A and 10B are diagrams illustrating examples of pixel arrangements in which light beams with different phase delays are received. [Figure 41] FIG. 10 is a diagram illustrating an example of the arrangement of charge accumulation units. [Figure 42] FIG. 10 is a diagram showing an example of the arrangement of a charge storage section and a charge transfer section. [Figure 43] 10A and 10B are diagrams illustrating an example of the arrangement of charge accumulation units and charge accumulation units. [Figure 44] 10A and 10B are diagrams for explaining other configuration examples of the imaging area and the overlap area. [Figure 45] FIG. 1 is a diagram illustrating an example of a schematic configuration of an endoscopic surgery system. [Figure 46] FIG. 2 is a block diagram showing an example of the functional configuration of a camera head and a CCU. [Figure 47] 1 is a block diagram showing an example of a schematic configuration of a vehicle control system; [Figure 48] FIG. 2 is an explanatory diagram showing an example of the installation positions of an outside-vehicle information detection unit and an imaging unit. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, modes for carrying out the present technology (hereinafter referred to as embodiments) will be described.

[0012] This technology can be applied to, for example, a light receiving element that constitutes a distance measurement system that performs distance measurement using an indirect TOF method, or to an imaging device that has such a light receiving element.

[0013] For example, the distance measurement system can be applied to an in-vehicle system that is installed in a vehicle and measures the distance to an object outside the vehicle, or a gesture recognition system that measures the distance to an object such as a user's hand and recognizes the user's gesture based on the measurement result, etc. In this case, the result of the gesture recognition can be used for operating a car navigation system, for example.

[0014] <Outline of Indirect TOF> We will now explain indirect TOF, to which this technology can be applied. Figure 1 shows an example of the configuration of a distance measuring device that performs distance measurement using the indirect TOF method.

[0015] The distance measuring device 10 includes a lens 11, a light receiving unit 12, a signal processing unit 13, a light emitting unit 14, a light emission control unit 15, and a filter unit 16. The distance measuring device 10 in Fig. 1 measures the distance to an object by irradiating light onto the object and receiving light (reflected light) that is reflected from the object.

[0016] The light-emitting system of the distance measuring device 10 is composed of a light-emitting unit 14 and a light-emitting control unit 15. In the light-emitting system, the light-emitting control unit 15 causes the light-emitting unit 14 to emit infrared light (IR) under the control of the signal processing unit 13. An IR bandpass filter may be provided between the lens 11 and the light-receiving unit 12, and the light-emitting unit 14 may emit infrared light corresponding to the transmission wavelength band of the IR bandpass filter.

[0017] The light emitting unit 14 may be disposed inside the housing of the distance measuring device 10, or may be disposed outside the housing of the distance measuring device 10. The light emission control unit 15 causes the light emitting unit 14 to emit light at a predetermined frequency.

[0018] The signal processing unit 13 functions as a calculation unit that calculates the distance (depth value) from the distance measuring device 10 to an object based on, for example, a detection signal (pixel data) supplied from the light receiving unit 12. The signal processing unit 13 generates a depth map in which depth values ​​(depth information) are stored as pixel values ​​of each pixel 50 ( FIG. 2 ) of the light receiving unit 12, and outputs the depth map to the filter unit 16. The signal processing unit 13 also calculates the reliability of the calculated depth value for each pixel 50 of the light receiving unit 12, and generates a reliability map in which reliability (luminance information) is stored as pixel values ​​of each pixel 50 of the light receiving unit 12, and outputs the reliability map to the filter unit 16.

[0019] <Image sensor configuration> 2 is a block diagram showing an example of the configuration of the light receiving unit 12. The light receiving unit 12 can be a CMOS (Complementary Metal Oxide Semiconductor) image sensor.

[0020] The light receiving unit 12 includes a pixel array unit 41, a vertical drive unit 42, a column processing unit 43, a horizontal drive unit 44, and a system control unit 45. The pixel array unit 41, the vertical drive unit 42, the column processing unit 43, the horizontal drive unit 44, and the system control unit 45 are formed on a semiconductor substrate (chip) not shown.

[0021] In the pixel array section 41, unit pixels (for example, pixels 50 in FIG. 3) each having a photoelectric conversion element that generates and internally accumulates photocharges whose amount corresponds to the amount of incident light are arranged two-dimensionally in a matrix. Note that, hereinafter, the photocharges whose amount corresponds to the amount of incident light may be simply referred to as "charges," and unit pixels may be simply referred to as "pixels."

[0022] The pixel array unit 41 further includes pixel drive lines 46 formed in the left-right direction of the drawing (the direction in which the pixels in the pixel rows are arranged) for each row of the matrix-like pixel arrangement, and vertical signal lines 47 formed in the up-down direction of the drawing (the direction in which the pixels in the pixel columns are arranged). One end of each pixel drive line 46 is connected to an output terminal of the vertical drive unit 42 corresponding to each row.

[0023] The vertical drive unit 42 is a pixel drive unit that is configured with a shift register, an address decoder, etc., and drives each pixel of the pixel array unit 41, either all pixels at the same time or row by row, etc. Pixel signals output from each unit pixel of a pixel row selected and scanned by the vertical drive unit 42 are supplied to the column processing unit 43 through each vertical signal line 47. The column processing unit 43 performs predetermined signal processing on the pixel signals output from each unit pixel of the selected row through the vertical signal line 47 for each pixel column of the pixel array unit 41, and temporarily stores the pixel signals after signal processing.

[0024] Specifically, the column processing unit 43 performs at least noise removal processing, such as CDS (Correlated Double Sampling) processing, as signal processing. This correlated double sampling by the column processing unit 43 removes pixel-specific fixed pattern noise such as reset noise and threshold variation of the amplification transistor. Note that in addition to noise removal processing, the column processing unit 43 can also be provided with, for example, an AD (analog-digital) conversion function, so as to output the signal level as a digital signal.

[0025] The horizontal driving unit 44 is configured with a shift register, an address decoder, etc., and sequentially selects unit circuits corresponding to pixel columns in the column processing unit 43. By selective scanning by this horizontal driving unit 44, pixel signals that have been signal-processed in the column processing unit 43 are sequentially output to the signal processing unit 48.

[0026] The system control unit 45 is composed of a timing generator that generates various timing signals, and controls the driving of the vertical driving unit 42, column processing unit 43, horizontal driving unit 44, etc. based on the various timing signals generated by the timing generator.

[0027] In the pixel array unit 41, pixel drive lines 46 are wired in the row direction for each pixel row in the matrix-like pixel arrangement, and two vertical signal lines 47 are wired in the column direction for each pixel column. For example, the pixel drive lines 46 transmit drive signals for driving the pixels when reading out signals. Note that although FIG. 2 shows one pixel drive line 46, the number of pixel drive lines 46 is not limited to one. One end of the pixel drive line 46 is connected to an output terminal of the vertical drive unit 42 corresponding to each row.

[0028] <Structure of unit pixel> Next, a specific structure of the unit pixels 50 arranged in a matrix in the pixel array section 41 will be described.

[0029] The pixel 50 includes a photodiode 61 (hereinafter referred to as PD61) which is a photoelectric conversion element, and is configured so that charges generated in the PD61 are distributed to taps 51-1 and 51-2. Of the charges generated in the PD61, the charges distributed to the tap 51-1 are read out from the vertical signal line 47-1 and output as a detection signal SIG1. Meanwhile, the charges distributed to the tap 51-2 are read out from the vertical signal line 47-2 and output as a detection signal SIG2.

[0030] The tap 51-1 is configured with a transfer transistor 62-1, a floating diffusion (FD) 63-1, a reset transistor 64, an amplification transistor 65-1, and a selection transistor 66-1. Similarly, the tap 51-2 is configured with a transfer transistor 62-2, a floating diffusion (FD) 63-2, a reset transistor 64, an amplification transistor 65-2, and a selection transistor 66-2.

[0031] As shown in FIG. 3, the reset transistor 64 may be configured to be shared by the FD63-1 and FD63-2, or may be configured to be provided for each of the FD63-1 and FD63-2.

[0032] When a configuration is adopted in which the reset transistor 64 is provided for each of the FD63-1 and FD63-2, the reset timing can be controlled individually for each of the FD63-1 and FD63-2, enabling finer control.When a configuration is adopted in which a common reset transistor 64 is provided for both the FD63-1 and FD63-2, the reset timing can be made the same for the FD63-1 and FD63-2, simplifying control and simplifying the circuit configuration.

[0033] In the following description, an example in which a common reset transistor 64 is provided for the FD63-1 and FD63-2 will be described.

[0034] 4, the distribution of charges in the pixel 50 will be described. Here, distribution means that charges accumulated in the pixel 50 (PD 61) are read out at different timings, thereby reading out for each tap.

[0035] As shown in Figure 4, the light emitting unit 14 emits light modulated to repeatedly turn on and off within the irradiation time (one period = Tp), and the reflected light is received by the PD 61 after a delay time Td that corresponds to the distance to the object.

[0036] The transfer control signal TRT_A controls the on / off of the transfer transistor 62-1, and the transfer control signal TRT_B controls the on / off of the transfer transistor 62-2. As shown in the figure, the transfer control signal TRT_A has the same phase as the irradiated light, while the transfer control signal TRT_B has an inverted phase of the transfer control signal TRT_A.

[0037] Therefore, the charges generated by the photodiode 61 receiving reflected light are transferred to the FD section 63-1 while the transfer transistor 62-1 is turned on in accordance with the transfer control signal TRT_A. Also, the charges are transferred to the FD section 63-2 while the transfer transistor 62-2 is turned on in accordance with the transfer control signal TRT_B. As a result, during a predetermined period in which irradiation light is periodically applied for the irradiation time T, the charges transferred via the transfer transistor 62-1 are sequentially accumulated in the FD section 63-1, and the charges transferred via the transfer transistor 62-2 are sequentially accumulated in the FD section 63-2.

[0038] Then, after the charge accumulation period ends, when the selection transistor 66-1 is turned on in accordance with the selection signal SELm1, the charge accumulated in the FD unit 63-1 is read out via the vertical signal line 47-1, and a detection signal A corresponding to the amount of charge is output from the light receiving unit 12. Similarly, when the selection transistor 66-2 is turned on in accordance with the selection signal SELm2, the charge accumulated in the FD unit 63-2 is read out via the vertical signal line 47-2, and a detection signal B corresponding to the amount of charge is output from the light receiving unit 12.

[0039] The charge stored in the FD unit 63-1 is discharged when the reset transistor 64 is turned on in accordance with the reset signal RST. Similarly, the charge stored in the FD unit 63-2 is discharged when the reset transistor 64 is turned on in accordance with the reset signal RST.

[0040] In this way, the pixel 50 can distribute the charge generated by the reflected light received by the photodiode 61 to the tap 51-1 and the tap 51-2 according to the delay time Td, and output the detection signal A and the detection signal B. The delay time Td corresponds to the time it takes for the light emitted by the light-emitting unit 14 to travel to the object, reflect off the object, and then travel to the light-receiving unit 12, i.e., the delay time Td corresponds to the distance to the object. Therefore, the distance measuring device 10 can determine the distance (depth) to the object based on the detection signal A and the detection signal B according to the delay time Td.

[0041] <Distance measurement with phase shift on the light receiving side> With reference to FIG. 5, a case where light is received with a phase shift on the receiving side and a detection signal is obtained will be described.

[0042] For example, as shown in Fig. 5, four types of light reception are performed with a phase delay of 90 degrees each. That is, based on light reception with a phase delay of 0 degrees, which is light reception without any phase shift from the irradiated side, light reception with a phase delay of 90 degrees, light reception with a phase delay of 180 degrees, and light reception with a phase delay of 270 degrees are performed, and four periods (quads) are provided for detecting detection signal A and detection signal B, respectively.

[0043] For example, a detection period Q0 in which reflected light is detected by receiving light with a phase delay of 0 degrees, a detection period Q1 in which reflected light is detected by receiving light with a phase delay of 90 degrees, a detection period Q2 in which reflected light is detected by receiving light with a phase delay of 180 degrees, and a detection period Q3 in which reflected light is detected by receiving light with a phase delay of 270 degrees are successively provided.

[0044] At the same timing (phase delay 0 degrees) as the start of emitting irradiation light, the transfer control signal TRT_A of the tap 51-1 is turned on, and light reception (transfer) begins at the tap 51-1. At the same timing as the transfer control signal TRT_A is turned off, the transfer control signal TRT_B is turned on, and light reception (transfer) begins at the tap 51-2. By receiving light with a phase delay of 0 degrees in this manner, charges are distributed to the taps 51-1 and 51-2 in amounts according to the delay time TR, and the charges are accumulated respectively. The accumulated charges are read out, and detection signals A90 and B90 for the detection period Q1 are output.

[0045] At a timing 90 degrees phase delayed from the start of irradiation light emission, the transfer control signal TRT_A of tap 51-1 is turned on, and light reception (transfer) begins at tap 51-1. At the timing when the transfer control signal TRT_A is turned off, the transfer control signal TRT_B is turned on, and light reception (transfer) begins at tap 51-2. By receiving light with a phase delay of 90 degrees in this way, electric charges are distributed to tap 51-1 and tap 51-2 in amounts according to the delay time TR, and the electric charges are accumulated respectively. The accumulated electric charges are read out, and detection signals A90 and B90 for detection period Q1 are output.

[0046] At a timing 180 degrees phase delayed from the start of irradiation light emission, the transfer control signal TRT_A of tap 51-1 is turned on, and light reception (transfer) begins at tap 51-1. At the timing when the transfer control signal TRT_A is turned off, the transfer control signal TRT_B is turned on, and light reception (transfer) begins at tap 51-2. By receiving light with a phase delay of 180 degrees in this manner, charges are distributed to tap 51-1 and tap 51-2 in amounts according to delay time TR, and the charges are accumulated respectively. The accumulated charges are read out, and detection signal A180 and detection signal B180 for detection period Q2 are output.

[0047] At a timing 270 degrees phase delayed from the start of irradiation light emission, the transfer control signal TRT_A of tap 51-1 is turned on, and light reception (transfer) begins at tap 51-1. At the timing when the transfer control signal TRT_A is turned off, the transfer control signal TRT_B is turned on, and light reception (transfer) begins at tap 51-2. By receiving light with a phase delay of 270 degrees in this manner, electric charges are distributed to tap 51-1 and tap 51-2 in amounts according to the delay time TR, and the electric charges are accumulated respectively. The accumulated electric charges are read out, and detection signal A270 and detection signal B270 for detection period Q3 are output.

[0048] Thus, in detection period Q0, detection signals A0 and B0 are detected when light is received with a phase delay of 0 degrees, and in detection period Q1, detection signals A90 and B90 are detected when light is received with a phase delay of 90 degrees. Similarly, in detection period Q2, detection signals A180 and B180 are detected when light is received with a phase delay of 180 degrees, and in detection period Q3, detection signals A270 and B270 are detected when light is received with a phase delay of 270 degrees.

[0049] On the irradiating side, regardless of the phase delay at which light reception starts, there is no phase delay, in other words, irradiation of the irradiating light always starts at the same timing.

[0050] In this way, when receiving light with four phase delays using two taps 51 and measuring the distance to a predetermined object, the signal processing unit 13 (FIG. 1) performs processing based on signals obtained from the four detection periods.

[0051] <About distance calculation> 6, the detection period of one frame is made up of detection periods Q0, Q1, Q2, and Q3. In detection period Q0, detection signals A0 and B0 are acquired, and in detection period Q1, detection signals A90 and B90 are acquired. In detection period Q2, detection signals A180 and B180 are acquired, and in detection period Q3, detection signals A270 and B270 are acquired.

[0052] Using these detection signals, the signal processing unit 13 (FIG. 1) calculates the phase difference θ based on the following equation (1), calculates the distance D based on the following equation (2), and calculates the reliability c based on the following equation (3).

[0053]

number

[0054]

number

[0055]

number

[0056] In equation (1), I represents the value obtained by subtracting C180, the value obtained by subtracting detection signal B180 from detection signal A180, from C0, the value obtained by subtracting detection signal B0 from detection signal A0. Q represents the value obtained by subtracting C270, the value obtained by subtracting detection signal B270 from detection signal A270, from C90, the value obtained by subtracting detection signal B90 from detection signal A90. The phase difference θ is calculated by taking the arc tangent of (Q / I).

[0057] In equation (2), C is the speed of light, and Tp is the pulse width. The delay time Td can be calculated based on the phase difference θ, and the distance D to the target object can be calculated from the delay time Td.

[0058] Equation (3) is an equation for calculating a value representing the reliability of the calculated distance. The reliability c is found by calculating the square root of the sum of the squares of I and Q. Note that the calculation of the reliability c is not an essential element in calculating the distance D, and it can be omitted. Furthermore, the reliability c may be calculated using an equation other than equation (3). For example, the sum of the absolute values ​​of I and Q can be used as the reliability c.

[0059] This case of measuring the distance to a predetermined object using irradiation light with four phase differences at two taps 51 is called 2-tap 4-phase. The present technology can also be applied to a case of measuring the distance to a predetermined object using irradiation light with two phase differences at two taps 51, or a case of measuring the distance to a predetermined object using irradiation light with four phase differences at one tap 51.

[0060] <About 2 Taps and 2 Phases> A simple explanation will be given below about measuring the distance to a predetermined object by using two taps 51 to emit light with two phase differences or by receiving light with two phase differences (hereinafter referred to as "2 tap 2 phase" as appropriate). Here, the explanation will continue by taking as an example a case where the distance to a predetermined object is measured by receiving light with two phase differences.

[0061] FIG. 7 is a diagram showing exposure timings of four phases, 0 degrees, 90 degrees, 180 degrees, and 270 degrees, with the beginnings aligned so that the phase differences can be easily seen.

[0062] In practice, as shown in FIG. 6, imaging to acquire detection signals A0 and B0 is performed in detection period Q0, imaging to acquire detection signals A90 and B90 is performed in detection period Q1, imaging to acquire detection signals A180 and B180 is performed in detection period Q2, and imaging to acquire detection signals A270 and B270 is performed in detection period Q3.

[0063] If the imaging operations performed sequentially in this time direction are arranged vertically with the beginnings of the detection periods aligned, it will look like Figure 7. From the beginning of detection period Q0, exposure is performed to capture detection signal A0, and then exposure is performed to capture detection signal B0.

[0064] From a point 90 degrees out of phase with the beginning of the detection period Q1, exposure for capturing the detection signal A90 is performed, and then exposure for capturing the detection signal B90 is performed.

[0065] From a point 180 degrees out of phase with respect to the beginning of the detection period Q2, exposure for capturing the detection signal A180 is performed, and then exposure for capturing the detection signal B180 is performed.

[0066] From a point 270 degrees out of phase with respect to the beginning of the detection period Q3, exposure for capturing the detection signal A270 is performed, and then exposure for capturing the detection signal B270 is performed.

[0067] Here, comparing the exposure time of the detection signal B0 in the detection period Q0 and the exposure time of the detection signal A180 in the detection period Q2, it can be seen that the exposures are performed at the same timing. Therefore, the detection signal A180 in the detection period Q2 can be substituted with the detection signal B0 in the detection period Q0. Similarly, the detection signal B180 in the detection period Q2 can be substituted with the detection signal A0 in the detection period Q0.

[0068] Similarly, comparing the exposure time of the detection signal B90 in the detection period Q1 and the exposure time of the detection signal A270 in the detection period Q3 reveals that the exposures are performed at the same timing. Therefore, the detection signal A270 in the detection period Q3 can be substituted with the detection signal B90 in the detection period Q1. Similarly, the detection signal B270 in the detection period Q3 can be substituted with the detection signal A90 in the detection period Q1.

[0069] For this reason, as shown in FIG. 8, the detection period Q0 and the detection period Q1 are set as the detection period of one frame, and the detection signal A0 and the detection signal B0 are acquired during the detection period Q0.

[0070] The detection signal A0 acquired during this detection period Q0 can be used as the detection signal B180. Furthermore, the detection signal B0 acquired during the detection period Q0 can be used as the detection signal A180. Therefore, in this case, the detection signals A0, B0, A180, and B180 can be treated as if they were acquired during the detection period Q0.

[0071] Furthermore, the detection signal A90 acquired during the detection period Q1 can be used as the detection signal B270. Furthermore, the detection signal B90 acquired during the detection period Q1 can be used as the detection signal A270. Therefore, in this case, the detection signals A90, B90, A270, and B270 can be treated as if they were acquired during the detection period Q1.

[0072] Therefore, the 2-tap 2-phase case described with reference to FIG. 8 can be handled in the same way as the 2-tap 4-phase case described with reference to FIG.

[0073] The values ​​I and Q in the equation (1) for 2 taps and 4 phases described with reference to FIG. 6 are expressed as in the following equation (4).

[0074]

number

[0075] The value I is obtained by subtracting the detection signal B0 from the detection signal A0, and the value Q is obtained by subtracting the detection signal B90 from the detection signal A90. Because the values ​​I and Q are obtained, the phase difference θ can be calculated using equation (1), and the distance D can be calculated using equation (2), just as in the case of the two-tap method described above.

[0076] <About 1 tap 4 phases> A simple explanation will be given below of the case where the distance to a predetermined object is measured by using irradiation light with four phase differences or receiving light with four phase differences with one tap 51 (the configuration of a pixel 50 having one tap 51 is not shown) (hereinafter referred to as 1 tap 4 phases as appropriate).

[0077] If the imaging order in the time direction for one tap and four phases is expressed in the same way as in Figure 6, it will look like Figure 9. In the detection period Q0, the value C0 in the above formula (1) is acquired. In the detection period Q1, the value C90 in the above formula (1) is acquired. In the detection period Q2, the value C180 in the above formula (1) is acquired. In the detection period Q3, the value C270 in the above formula (1) is acquired.

[0078] In the case of the one-tap method, the values ​​I and Q in the above equation (1) are expressed as in the following equation (5).

number

[0079] Since the values ​​I and Q are obtained, the phase difference θ can be calculated by equation (1) and the distance D can be calculated by equation (2), just like in the case of the two-tap method described above.

[0080] This technology can be applied to the above-mentioned 2-tap 4-phase, 2-tap 2-phase, and 1-tap 4-phase. This technology can also be applied to systems using numbers of taps and phases not shown here, and can be applied regardless of the number of taps or phases.

[0081] <Configuration example of distance measuring device> 10 is a diagram showing an example of the configuration of an embodiment of a distance measuring device 100 to which the present technology is applied. The distance measuring device 100 is made up of a camera head unit 101 and a signal processing unit 102.

[0082] The camera head unit 101 includes a light-emitting unit 121-1, a light-emitting unit 121-2, a light-receiving unit 122, a lens 123, a light-emitting control unit 124-1, a light-emitting control unit 124-2, a power supply unit 125, a control unit 126, a non-volatile memory unit 127, and a communication unit 128. The camera head unit 101 corresponds to the lens 11, the light-receiving unit 12, the light-emitting unit 14, and the light-emitting control unit 15 of the distance measuring device 10 shown in FIG.

[0083] The signal processing unit 102 includes a communication unit 131, a distance calculation unit 132, a distance correction unit 133, a control signal transmission unit 134, and a communication unit 135. The signal processing unit 13 corresponds to the signal processing unit 13 of the distance measuring device 10 shown in FIG.

[0084] The camera head unit 101 is configured to have two light-emitting units 121. The light emission of light-emitting unit 121-1 is controlled by light-emitting control unit 124-1, and the light emission of light-emitting unit 121-2 is controlled by light-emitting control unit 124-2. Light emitted by light-emitting unit 121 strikes a subject, and the reflected light is received by light-receiving unit 122 via lens 123. Light-receiving unit 122 has a configuration such as that shown in FIG. 2, for example.

[0085] The power supply unit 125 supplies power to each unit in the camera head unit 101. The power supply unit 125 is connected to the light emission control unit 124-1, the light emission control unit 124-2, and the light receiving unit 122, and supplies the power necessary for each unit to operate. The control unit 126 controls each unit in the camera head unit 101. The non-volatile memory unit 127 stores control information and the like necessary for the control unit 126 to perform control. A signal from the light receiving unit 122 is supplied to the communication unit 128 by the control unit of the control unit 126, and is supplied to the signal processing unit 102 via the communication unit 128.

[0086] The signal processing unit 102 processes the signal supplied from the camera head unit 101. The communication unit 131 of the signal processing unit 102 receives the signal from the camera head unit 101 and supplies it to the distance calculation unit 132. The distance calculation unit 132 calculates the distance to the subject based on the supplied signal information and supplies the calculation result to the distance correction unit 133. The distance correction unit 133 corrects the calculation result of the distance calculation and supplies the corrected distance measurement data to the communication unit 135.

[0087] The communication unit 135 outputs the corrected distance measurement data to the outside as distance information, and transmits and receives control signals from external devices. The control signal transmission unit 134 transmits the control signal input from the outside via the communication unit 135 to the signal processing unit 102.

[0088] In the example shown in FIG. 10, the light receiving unit 122 and the light emission control unit 124 have the same reference signal, and for example, the light receiving unit 122 is configured to control the light emission control unit 124 based on the reference signal generated on the light receiving unit 122 side.

[0089] The light receiving unit 122 can be applied to a distance measuring sensor with a structure that distributes charge to two charge storage units, such as a distance measuring sensor with a CAPD (Current Assisted Photonic Demodulator) structure or a gate type distance measuring sensor that applies pulses alternately to two gates to receive charge from a photodiode.Of course, it can also be applied to a distance measuring sensor with a structure that distributes charge to two or more charge storage units.

[0090] The distance measuring device 100 may have a two-tap structure that distributes the signal to two taps as described above, or may have a pixel structure with other numbers of taps, such as a one-tap structure or a four-tap structure.

[0091] <About the imaging area> The distance measuring device 100 shown in FIG. 10 has two light sources, light-emitting unit 121-1 and light-emitting unit 121-2. The imaging area captured by the light emitted from light-emitting unit 121 is, for example, as shown in FIG. 11. In FIG. 11, the imaging area is indicated by a rectangle. On the left side of the figure is illumination area 121-1 illuminated with light from light-emitting unit 121-1, and on the right side of the figure is illumination area 121-2 illuminated with light from light-emitting unit 121-2. In the center of the imaging area is overlapping area 121-12, where illumination area 121-1 and illumination area 121-2 overlap.

[0092] In this way, the imaging area includes three areas: the projection area 121-1, the projection area 121-2, and the overlap area 121-12. Processing, which will be described later, is executed for each area to generate distance measurement information.

[0093] <About the light-emitting unit drive> The driving of the light-emitting unit 121-1 and the light-emitting unit 121-2, in other words, the control method of the light-emitting control unit 124-1 and the light-emitting control unit 124-2, will be described with reference to FIGS.

[0094] 12 is a diagram showing the light emitting interval of the light emitting unit 121 and the drive signal of the light receiving unit 122. In FIG. 12, a case where the light emitting period of the light emitting unit 121 (light source) is set based on the drive signal of the light receiving unit 122 is shown as an example.

[0095] The light receiving unit 122 has an integration period and a readout period, during which charge is accumulated, and during the readout period, the charge accumulated during the integration period is read out, and a detection signal is output during the detection period. This process is performed during each of the detection periods Q0, Q90, Q180, and Q270.

[0096] The light emission intervals of the multiple light sources, here light-emitting units 121-1 and 121-2, are set in accordance with the integration period of light-receiving unit 122, and are controlled by light-emission control unit 124 so that the light is emitted in synchronization with the integration period of light-receiving unit 122. The drive pulses of the light sources in one detection period Q, for example, detection period Q0 (the period indicated by a circle around the light-emission interval in FIG. 12), are shown in FIG.

[0097] During the light emission section of detection period Q0, the light emission of light-emitting unit 121-1 and light-emitting unit 121-2 is controlled by drive signals that are 180 degrees out of phase. During the period when light-emitting unit 121-1 is emitting light, light-emitting unit 121-2 is controlled not to emit light, and during the period when light-emitting unit 121-2 is emitting light, light-emitting unit 121-1 is controlled not to emit light. Control is performed so that the timing of the start of light emission (rising edge) of light-emitting unit 121-1 and light-emitting unit 121-2 is shifted.

[0098] By controlling the light-emitting units 121-1 and 121-2 so that there is a phase shift, it is possible to suppress radiation noise emitted to the outside. If the light-emitting units 121-1 and 121-2 were driven in phase, there is a high possibility that radiation noise would be generated due to the superposition of the drive currents of the light sources. In particular, if the light-emitting units 121-1 and 121-2 start emitting light at the same timing, the radiation noise would be large.

[0099] According to the present technology, a phase difference is applied between the drive signal of the light-emitting unit 121-1 and the drive signal of the light-emitting unit 121-2. By controlling the difference, the timing of the start of light emission is shifted and the driving currents are prevented from overlapping, thereby reducing radiation noise.

[0100] The light emission of the light-emitting unit 121 is controlled with this phase difference (predetermined phase amount). The light-receiving unit 122 is also driven in accordance with the driving of the light-emitting unit 121. When performing driving and calculation processing called two taps to eliminate ambient light and variations in the charge storage units, an operation is performed to distribute signal charges of opposite phases to the charge storage units, as with the gate drive signal shown in FIG. 13. In the example shown in FIG. 13, when the gate drive signal supplied to the light-receiving unit 122 is high, the charge is distributed to tap A, and when it is low, the charge is distributed to tap B.

[0101] The drive signals of light-emitting units 121 shown in FIG. 13 each have a duty ratio of 50%, and the phase difference between the drive signal of light-emitting unit 121-1 and the drive signal of light-emitting unit 121-2 is 180 degrees.

[0102] The drive signal for light-emitting unit 121-1 and the drive signal for light-emitting unit 121-2 are controlled to have different amplitudes. In the example shown in Fig. 13, the amplitude of the drive signal for light-emitting unit 121-1 is amplitude A+α, and the amplitude of the drive signal for light-emitting unit 121-2 is amplitude A. In other words, the amplitude of the drive signal for light-emitting unit 121-1 is controlled to be larger by α than the amplitude of the drive signal for light-emitting unit 121-2.

[0103] 11, the imaging area includes an overlapping area 121-12 where light from light-emitting unit 121-1 and light from light-emitting unit 121-2 overlap. By providing a difference in the emission intensity between light-emitting unit 121-1 and light-emitting unit 121-2, the signal in overlapping area 121-12 can be processed in the same manner as the signal in irradiation area 121-1, as will be described later, and a ranging signal can also be obtained in overlapping area 121-12.

[0104] <Other embodiments of the drive signal> When light-emitting unit 121-1 and light-emitting unit 121-2 are controlled by drive signals having a phase difference, the drive signals may have a phase difference of 180 degrees as shown in FIG. 13, or may have a phase difference as described below.

[0105] 14 to 20 are diagrams illustrating other drive signals for light-emitting unit 121. The drive signals shown in Fig. 14 illustrate a case where the phase difference between the drive signal for light-emitting unit 121-1 and the drive signal for light-emitting unit 121-2 is Π+β.

[0106] 15 shows a case where the phase difference between the drive signal of light-emitting unit 121-1 and the drive signal of light-emitting unit 121-2 is Π-γ. Even with this phase difference, the radiation noise can be reduced by shifting the timing at which light emission starts.

[0107] The drive signals shown in FIG. 16 represent a case where the duty ratio of the drive signal for light-emitting unit 121-1 and the drive signal for light-emitting unit 121-2 are each 30% and the phase difference is 0.3×Π.

[0108] 17 shows a case where the duty ratio of the drive signal for light-emitting unit 121-1 and the duty ratio of the drive signal for light-emitting unit 121-2 are both 70%, and the phase difference is 0.7×Π. Even with such duty ratios and a phase difference corresponding to the duty ratios, staggering the timing at which light emission starts can reduce radiation noise.

[0109] The drive signals shown in FIG. 18 are signals in which a square wave and a slope are combined, and the drive signals for light-emitting unit 121-1 and light-emitting unit 121-2 have a phase difference.

[0110] The drive signals shown in FIG. 19 are sine waves, and illustrate a case where the drive signal for light-emitting unit 121-1 and the drive signal for light-emitting unit 121-2 are signals having a phase difference.

[0111] The drive signals shown in FIG. 20 are triangular waves, and illustrate a case where the drive signal for light-emitting unit 121-1 and the drive signal for light-emitting unit 121-2 have a phase difference.

[0112] The waveforms shown in Figures 18, 19, and 20 can be applied to the drive signals with the phase difference described above. The two drive signals shown in Figures 18, 19, and 20 are controlled so that when one drive signal falls, the other drive signal rises. By using a waveform with such a phase difference, the timing at which light emission starts can be shifted, thereby reducing radiation noise.

[0113] The phase difference, duty ratio, and waveform shown here are merely examples, and phase difference, duty ratio, and waveform other than the examples are also within the scope of application of the present technology.

[0114] <About correction processing> 13 to 17 show a case where the amplitude of the drive signal for light-emitting unit 121-1 is amplitude A+α, and the amplitude of the drive signal for light-emitting unit 121-2 is amplitude A. An explanation will now be given of the processing performed when distance measurement is performed using light from light-emitting units 121 driven by drive signals with different amplitudes and a phase difference.

[0115] As described with reference to FIG. 11, the imaging area is divided into three areas: illumination area 121-1, illumination area 121-2, and overlap area 121-12. Because there is a phase difference in the light irradiated between illumination area 121-1 and illumination area 121-2, processing is performed taking this phase difference into consideration. Because overlap area 121-12 is dominated by light from light-emitting unit 121-1, which has a large amplitude, processing similar to that performed for illumination area 121-1 is performed. As a result, as shown in FIG. 21, the imaging area can be divided into areas where correction is not performed and areas where correction is performed.

[0116] Here, the explanation will be continued by taking as an example a case where the light from the light-emitting section 121-1 shown in FIG. 13 is used as a reference and the light from the light-emitting section 121-2 has a phase difference of 180 degrees.

[0117] 21 is divided into an area that requires correction due to irradiation with light having a phase difference and an area that does not require correction. The area that does not require correction includes irradiation area 121-1 and overlap area 121-12. The area that requires correction is an area equivalent to irradiation area 121-2.

[0118] Illumination area 121-1 is an area illuminated with light from light-emitting unit 121-1. Light from light-emitting unit 121-1 is emitted based on a drive signal synchronized with the drive timing of light-receiving unit 122, so that ranging data can be calculated by the processing described with reference to FIGS.

[0119] In the overlapping region 121-12, the signal with the larger light amplitude becomes dominant due to tap calculations in the normal iTOF system, and therefore, as an area irradiated with light from the light-emitting unit 121-1, distance measurement data can be calculated by processing similar to that for the irradiation region 121-1. The overlapping region 121-12 is irradiated with light from the light-emitting unit 121-1, which is emitted based on a drive signal synchronized with the drive timing of the light-receiving unit 122, and distance measurement data can be calculated by the processing described with reference to FIGS. 1 to 10, similar to the processing in the irradiation region 121-1.

[0120] Illumination area 121-2 is an area illuminated with light from light-emitting unit 121-2. The light from light-emitting unit 121-2 has a phase difference with respect to the light from light-emitting unit 121-1 and is emitted based on a drive signal that also has a phase difference with respect to the drive timing of light-receiving unit 122. Therefore, distance measurement data can be calculated by performing the processes described with reference to FIGS. 1 to 10 and a correction process to compensate for the phase difference.

[0121] When the light from light-emitting unit 121-2 is shifted in phase by 180 degrees relative to the light from light-emitting unit 121-1, the signal in illumination area 121-2 is corrected by shifting the phase by 180 degrees, and distance measurement data is calculated.

[0122] In this way, when multiple light-emitting units 121 are provided and a phase difference is set in the light-emitting drive signal, normal processing is performed on the area where light from the light-emitting unit 121 that is synchronized with the drive signal of the light-receiving unit 122 is irradiated, and in addition to normal processing, correction processing to correct the phase difference is performed on the area where light from the light-emitting unit 121 that is not synchronized with the drive signal of the light-receiving unit 122 is irradiated.

[0123] By performing such correction, even in distance measurement using multiple lights with phase differences in the light emission drive signal, it is possible to generate a distance measurement image by processing the image obtained in a single capture.

[0124] To perform such correction, the distance measuring device 100 holds a table (lookup table) that can distinguish between areas where correction is to be performed and areas where correction is not to be performed, as shown in Fig. 21. The lookup table is stored, for example, in the nonvolatile memory unit 127 (Fig. 10) or the distance correction unit 133 (Fig. 10). The lookup table is created and stored by acquiring the areas in advance through a correction operation. "In advance" refers to, for example, when the distance measuring device 100 is manufactured, when an image is captured for the first time, when the power is turned on, or after a predetermined number of images have been captured.

[0125] 22 is a diagram for explaining another correction method. By performing corrected or uncorrected reading when the light receiving unit 122 (sensor) is driven, it is possible to perform distance measurement processing that absorbs the phase difference when the light emitting unit 121 emits light.

[0126] The light receiving unit 122 is composed of a logic circuit 201 and a photodetector 202. A gate drive signal adjustment circuit 211 is provided within the logic circuit 201. The gate drive signal adjustment circuit 211 is a circuit that adjusts gate drive for each region within the photodetector 202. A gate drive signal is supplied to the gate drive signal adjustment circuit 211 from, for example, the control unit 126 (FIG. 10). This gate drive signal is, for example, the gate drive signal supplied to the light receiving unit 122 shown in FIG. 13, and is a signal synchronized with the drive signal of the light emitting unit 121-1.

[0127] 2, for example, and within the pixel array section 41, a region where the correction amount of the gate drive signal is 0 and a region where the correction amount is +Π are set. When reading from the region where the correction amount of the gate drive signal is 0, the gate drive signal adjustment circuit 211 drives the gate without adjusting the supplied gate drive signal. When reading from the region where the correction amount of the gate drive signal is +Π, the gate drive signal adjustment circuit 211 corrects the phase of the supplied gate drive signal by +Π and drives the gate.

[0128] In this way, by adjusting the gate drive signal of the light receiving section 122, distance measurement can be performed with the phase difference of the light emitting section 121 corrected.

[0129] <About the irradiation area> The above-described embodiment has been described by taking as an example a case where there are two light-emitting units 121 and light is emitted from the two light-emitting units 121. The present technology is not limited to application to two light-emitting units 121, and can also be applied to a case where, for example, light is emitted from three light-emitting units 121, light-emitting units 121-1 to 121-3, and three illumination areas 121-1 to 121-3 are within an imaging area, as shown in FIG.

[0130] The example shown in Fig. 23 shows a state in which distance measuring device 100 is provided with three light-emitting units 121-1 to 121-3, and the imaging area includes illumination area 121-1 illuminated with light from light-emitting unit 121-1, illumination area 121-2 illuminated with light from light-emitting unit 121-2, and illumination area 121-3 illuminated with light from light-emitting unit 121-3. Illumination area 121 shown in Fig. 23 is shown as a square for the sake of explanation, but the shape of the actual illumination area may differ. The same applies below.

[0131] The present technology can also be applied to a case where there are three light sources, as shown in Fig. 23. The present technology can also be applied to a case where four or more light sources are provided in the distance measuring device 100.

[0132] The distance measuring device 100 may also be configured so that multiple light sources are arranged in a state in which there is no area in the imaging area where light irradiated from the multiple light sources overlaps. Fig. 24 is a diagram showing the imaging area when the distance measuring device 100 includes light emitting unit 121-1 and light emitting unit 121-2 and light from these light emitting units 121 is irradiated so as to avoid an area where light overlaps. The irradiation of light from light emitting unit 121-1 and the irradiation of light from light emitting unit 121-2 are controlled so that there is no area where irradiation area 121-1 is irradiated with light from light emitting unit 121-1 and irradiation area 121-2 is irradiated with light from light emitting unit 121-2 overlap.

[0133] As such, the present technology can be applied even when there is no overlapping region. Even when there is no overlapping region, the present technology can be applied even when three light-emitting units 121-1 to 121-3 are provided in the distance measuring device 100, and light sources (light-emitting units 121) are arranged in the distance measuring device 100 so that an imaging region includes an irradiation region 121-1 illuminated with light from the light-emitting unit 121-1, an irradiation region 121-2 illuminated with light from the light-emitting unit 121-2, and an irradiation region 121-3 illuminated with light from the light-emitting unit 121-3, as shown in Fig. 25. This case can also be applied to a case where four or more light sources are provided in the distance measuring device 100.

[0134] When the arrangement and light emission of the light-emitting units 121 are controlled so that the irradiation areas 121 do not overlap, the amplitudes of the multiple light-emitting units 121 may be the same. For example, the amplitude of light-emitting unit 121-1 shown in FIG. 13 is amplitude A+α, and the amplitude of light-emitting unit 121-2 is amplitude A. However, if there are no overlapping areas in the irradiation areas, the amplitudes of light-emitting unit 121-1 and light-emitting unit 121-2 can both be amplitude A. Even if the light intensities of the multiple light sources are the same, if there are no overlapping areas, the signals in the imaging area can be processed normally to calculate ranging data.

[0135] When a configuration is used in which light is irradiated from a plurality of light sources, the wavelengths of the light sources can be the same or different.

[0136] 26 shows a state in which distance measuring device 100 is provided with two light-emitting units, light-emitting unit 121-1 and light-emitting unit 121-2, and an imaging area includes illumination area 121-1 illuminated with light from light-emitting unit 121-1 and illumination area 121-2 illuminated with light from light-emitting unit 121-2. The wavelengths of light-emitting unit 121-1 and light-emitting unit 121-2 are different.

[0137] 26, the irradiation of light of different wavelengths is represented by a diagram, with a rectangle representing the area irradiated with light from light-emitting unit 121-1 driven by a drive signal of wavelength A, and a circle representing the area irradiated with light from light-emitting unit 121-2 driven by a drive signal of wavelength B different from wavelength A. Here, the irradiation areas are represented by a rectangle and a circle to clarify that they are of different wavelengths, but this does not represent the actual shape of the areas.

[0138] 27 shows a state in which distance measuring device 100 is provided with three light-emitting units 121-1, 121-2, and 121-3, and the imaging area includes illumination area 121-1 illuminated with light from light-emitting unit 121-1, illumination area 121-2 illuminated with light from light-emitting unit 121-2, and illumination area 121-3 illuminated with light from light-emitting unit 121-3. The wavelength of light-emitting unit 121-1 is wavelength A, the wavelength of light-emitting unit 121-2 is wavelength B, and the wavelength of light-emitting unit 121-3 is wavelength C, and wavelengths A, B, and C are all different wavelengths. Alternatively, wavelengths A and B may be the same wavelength, but wavelength C may be a different wavelength from wavelengths A and B.

[0139] 26 and 27, the present technology can be applied to a distance measuring device 100 including a light emitting unit 121 that emits light of different wavelengths, and to a case where light of different wavelengths is used. By using different wavelengths, it becomes easier to distinguish areas illuminated with light from different light sources (light emitting units 122).

[0140] This technology can also be applied to a distance measuring device 100 equipped with a light emitting unit 121 that emits light of different wavelengths, when the position and light emission of the light emitting unit 121 are controlled so that there are no overlapping irradiation areas, as shown in Figures 28 and 29.

[0141] 28 is a diagram showing an imaging area when distance measuring device 100 includes light-emitting unit 121-1 and light-emitting unit 121-2, and light of different wavelengths is emitted from these light-emitting units 121 so that there is no overlapping area. The emission of light from light-emitting unit 121-1 and the emission of light from light-emitting unit 121-2 are controlled so that there is no overlapping area between illumination area 121-1 illuminated with light from light-emitting unit 121-1 and illumination area 121-2 illuminated with light from light-emitting unit 121-2, and the emitted light is controlled to be light of different wavelengths.

[0142] In this way, the present technology can be applied even when different wavelengths are used and there is no overlapping area in the irradiation area. Even when there is no overlapping area, the present technology can be applied even when, as shown in Fig. 29, three light-emitting units 121-1 to 121-3 are provided in the distance measuring device 100, the wavelengths of light emitted from the three light-emitting units 122 are controlled to be different, and the light sources (light-emitting units 121) in the distance measuring device 100 are arranged and controlled so that there is no overlapping area in the imaging area among the irradiation area 121-1 illuminated with light from the light-emitting unit 121-1, the irradiation area 121-2 illuminated with light from the light-emitting unit 121-2, and the irradiation area 121-3 illuminated with light from the light-emitting unit 121-3. This case can also be applied to a case where four or more light sources are provided in the distance measuring device 100.

[0143] As shown in Figures 28 and 29, this technology can also be applied to a distance measuring device 100 that has a light emitting unit 121 that emits light of different wavelengths, and the light emission is controlled so that the areas irradiated with these lights do not overlap.

[0144] <Configuration of Light Emitting Unit 121> The light emitting unit 121 can be configured to use an LED (Light Emitting Diode), a laser diode (LD: Laser Diode), a solid-state laser, a gas laser, a discharge lamp, a light bulb, or the like.

[0145] 30 and 31, for example, the light-emitting section 121 may be composed of a plurality of LDs. In the example shown in Fig. 30, one light-emitting section 121 is configured to include three LDs, 221-1 to 221-3. The LDs 221-1 to 221-3 are arranged in an array.

[0146] 31, one light-emitting unit 121 is configured to include three LDs 221-1 to 221-3, including LD 221-3, which is driven in a phase different from that of LD 221-1 and LD 221-2, and LDs 221-1 to 221-3 are arranged in an array. Note that, although an example in which the light-emitting unit 121 is configured with three LDs 221 has been described here, the light-emitting unit 121 may also be configured with LDs other than three, or may be configured with LDs 221 arranged in both vertical and horizontal directions.

[0147] When a configuration is adopted in which one light-emitting unit 121 is provided with a plurality of light sources, the light sources may be of the same type as shown in Figures 30 and 31, or may be configured with light sources of different types as shown in Figure 32. The light-emitting unit 121 shown in Figure 32 is configured with an LD 221 and an LED 222. The LED 222 can irradiate light by diffusing it relative to the LD 221, and the light-emitting unit 121 may be configured by combining such a diffused light source with the LD 221.

[0148] The light-emitting unit 121 can also be configured by combining a light source with a reflecting member that reflects light. The light-emitting unit 121 shown in FIG. 33 is configured to include an LD 221 and a reflecting member 241. Light from the LD 221 is reflected by the reflecting member 241 and is irradiated onto the subject. The reflecting member 241 can be configured with a mirror, a DMD (Digital Micromirror Device), or the like. The reflecting member 241 can be configured to be movable, and by scanning the reflecting member 241, the light from the LD 221 can be diffused and irradiated over a predetermined range.

[0149] Fig. 34 is a diagram showing the configuration of a light-emitting unit 121 having another diffusion structure. In the example shown in Fig. 34, the configuration is a combination of an LD 221 and a diffusion member 251 that diffuses light. Light from the LD 221 is diffused and irradiated within a predetermined range by the diffusion member 251. The diffusion member can be formed of a lens, a diffuser, or the like.

[0150] The type of light source of the light emitting unit 121, the combination of multiple light sources, the diffusion method, and the like can be applied to the present technology in addition to those exemplified here.

[0151] <Configuration related to light source control> Fig. 35 is a diagram showing an example configuration of the light-emitting unit 121 and the light-emitting control unit 124. The example shown in Fig. 35A shows an example configuration in which there are two light sources, light-emitting unit 121-1 and light-emitting unit 121-1, and two light-emitting control units 124-1 and 124-2 that control the respective light sources. In this case, the light-emitting unit 121 and the light-emitting control unit 124 have a one-to-one relationship. The light-emitting control unit 124 is a control circuit such as a driver IC.

[0152] 35B shows an example of a configuration in which there are two light sources, light-emitting unit 121-1 and light-emitting unit 121-1, and one light-emitting control unit 124 that controls these light sources. In this case, there is a 2-to-1 relationship between light-emitting unit 121 and light-emitting control unit 124.

[0153] 35C shows an example of a configuration in which there are two light sources, light-emitting unit 121-1 and light-emitting unit 121-1, and one control board 271 that controls these light sources. It is also possible to use a configuration in which control board 271 equipped with a control circuit is used instead of a driver IC.

[0154] It is also possible to have a configuration in which the light emitting unit 121 and the light emission control unit 124 are integrated together. The configuration of the light source 301 shown in Fig. 36 includes a phase control unit 311 and a light emitter 312. It is also possible to have a configuration in which the light source 301 itself includes a phase control unit 311 that controls the phase of a drive signal that controls the light emission of the light emitter 312, and the light emitter 312 emits light in response to the drive signal controlled by the phase control unit 311. It is also possible to have a configuration in which the phase control unit 311 is included in the light emitter 312.

[0155] <Embodiment of Wavelength Detection> As explained with reference to Figures 26 to 29, when a light emitting unit 121 that emits light of different wavelengths is used, the light of different wavelengths can be detected by a configuration such as that shown in Figures 37 to 38.

[0156] Fig. 37 is a diagram for explaining a configuration related to detection of light of different wavelengths. The configuration example shown in Fig. 37 is a configuration example in which the wavelength of light to be received is set for each pixel 50. Fig. 37 shows 9 pixels in a 3 x 3 array on the pixel array unit 41 (Fig. 1). When focusing on 4 pixels in a 2 x 2 array, pixel 50-1 on the upper left receives light with a wavelength of 940 nm, pixel 50-2 on the upper right receives light with a wavelength of 850 nm, pixel 50-4 on the lower left receives light with a wavelength of 850 nm, and pixel 50-5 on the lower right is set as a pixel that receives light with a wavelength of 940 nm.

[0157] In this case, pixels 50 that receive 950 nm light and pixels 50 that receive 850 nm light are arranged alternately in the vertical and horizontal directions. The wavelength of the light received can be adjusted, for example, by a color filter arranged on the pixel 50.

[0158] Fig. 38 is a diagram for explaining another configuration related to detection of light of different wavelengths. In the configuration example shown in Fig. 38, a filter 501 is provided between the light receiving unit 122 (pixel array unit 41) and the lens 123. The filter 501 includes, for example, a bandpass filter (BPF) 511 that transmits light with a wavelength of 850 nm and a bandpass filter 512 that transmits light with a wavelength of 940 nm.

[0159] For example, the bandpass filter 511 is provided between the pixel array section 41 and the lens 123 so as to cover the pixels in the left half of the pixel array section 41, and the bandpass filter 512 is provided between the pixel array section 41 and the lens 123 so as to cover the pixels in the right half of the pixel array section 41.

[0160] The size of the bandpass filters 511 and 512 is not limited to a size that covers half of the pixel array section 41. For example, they may be large enough to cover a plurality of pixels, for example, four pixels, and the bandpass filters 511 and 512 may be arranged in a lattice pattern, with the entire filter 501 covering the pixel array section 41. In this way, the wavelength of the light to be received can be set by dividing the area.

[0161] Fig. 39 is a diagram for explaining yet another configuration related to detection of light of different wavelengths. In the configuration example shown in Fig. 39, a filter 531 is provided between the light receiving section 122 (pixel array section 41) and the lens 123.

[0162] The filter 531 includes a bandpass filter (BPF) 541 that transmits light with a wavelength of 850 nm and a bandpass filter 542 that transmits light with a wavelength of 940 nm, and is configured to be movable, for example, rotatable.

[0163] For example, bandpass filter 541 and bandpass filter 542 are provided between pixel array unit 41 and lens 123 so as to cover the entire pixel array unit 41. At a predetermined time t1, bandpass filter 541 covers pixel array unit 41 and is ready to receive light of 850 nm, and at the time t2 following time t1, filter 531 is rotated and bandpass filter 542 covers pixel array unit 41 and is ready to receive light of 940 nm. In this way, a configuration can be adopted in which the wavelength of light to be received is set in a time-division manner.

[0164] <Other configurations> 5, the distance measuring device 100 receives light in four ways with a phase delay of 90 degrees each as a process related to distance measurement. That is, based on light reception with a phase delay of 0 degrees, which is light reception without any phase shift from the phase on the irradiating side, light reception with a phase delay of 90 degrees, light reception with a phase delay of 180 degrees, and light reception with a phase delay of 270 degrees are performed, and a detection signal is generated at each phase.

[0165] As shown in Fig. 40, a configuration may be adopted in which pixels 50 on the pixel array unit 41 receive (accumulate) light with different phase delays. Fig. 40 shows nine 3 x 3 pixels on the pixel array unit 41 (Fig. 1), and focusing on four 2 x 2 pixels among them, pixel 50-1 on the upper left receives light with a phase delay of 0 degrees, pixel 50-2 on the upper right receives light with a phase delay of 90 degrees, pixel 50-4 on the lower left receives light with a phase delay of 180 degrees, and pixel 50-5 on the lower right receives light with a phase delay of 270 degrees. In this way, a configuration may be adopted in which a phase delay for receiving light is set for each pixel, and the pixels are driven based on that setting.

[0166] 41 is a diagram showing an example of the arrangement of the charge accumulation unit 611. The charge accumulation unit 611 is, for example, the FD 63 (FIG. 2) or a memory (not shown). The charge accumulation unit 611-1 is located at the center of the 2×2 pixels 50-1, 50-2, 50-4, and 50-5, and is provided between the pixels. Similarly, the charge accumulation unit 611-2, charge accumulation unit 611-3, and charge accumulation unit 611-4 are also located at the center of the 2×2 four pixels and are provided between the pixels.

[0167] The present technology can also be applied to a configuration in which the charge storage unit 611 is provided between pixels in this way. The charge storage unit 611 provided between pixels can also be configured to be shared by a plurality of adjacent pixels 50. Although not shown in the figure, the charge storage unit 611 can also be configured to be provided in each pixel 50, and the present technology can also be applied to such a configuration.

[0168] 42 is a diagram showing an example of the arrangement of the charge accumulation unit 611, the charge transfer unit 631, and the ground (GND) unit 633. The charge accumulation unit 611-1 is located at the center of the 2×2 pixels 50-1, 50-2, 50-4, and 50-5, and is provided between the pixels so as to be shared by these pixels.

[0169] Charge transfer units 631-1 to 631-4 are provided in each of pixels 50-1, 50-2, 50-4, and 50-5, and are configured to transfer charges read from the PD to the charge accumulation unit 611-1. A ground unit 633-1 is provided at the center of the 2×2 pixels 50-2, 50-3, 50-5, and 50-6, and is located between the pixels.

[0170] 43 is a diagram showing an example of the arrangement of a charge transfer unit 631 and a charge accumulation unit 651. A pixel 50-1 is provided with a charge transfer unit 631-1 and a charge accumulation unit 651-1 that temporarily accumulates charges transferred from the charge transfer unit 631-1 or accumulates charges that overflow from the PD. Similarly, a pixel 50-2 is provided with a charge transfer unit 631-2 and a charge accumulation unit 651-2. The charge accumulation unit 651 is a memory. The present technology can also be applied to a configuration of a pixel 50 in which a memory is arranged.

[0171] This technology can be applied regardless of the position of the charge storage unit 611, and regardless of whether it is shared or not. This technology is not dependent on the position of the charge transfer unit 631 or the position of the ground unit 633, and can be applied regardless of whether it is shared or not. This technology can also be applied to a case where a memory is provided as the charge storage unit.

[0172] This technology can be applied regardless of whether a color filter is used or not. This technology can be applied regardless of the rolling shutter method or global shutter method. This technology can be applied regardless of coding.

[0173] <Other forms of imaging area and superimposed area> As described with reference to FIG. 11, the above-described embodiment has been described by taking as an example a case where there are three areas, ie, the projection area 121-1, the projection area 121-2, and the overlap area 121-12, within the imaging area.

[0174] As shown in FIG. 44, the present technology can also be applied to a case where there is only one overlapping area 121-12 within the imaging area.

[0175] It is assumed that distance measuring device 100 has two light sources, light-emitting unit 121-1 and light-emitting unit 121-2. In Fig. 44, the imaging area is shown as a rectangle. On the left side of the figure is illumination area 121-1 illuminated with light from light-emitting unit 121-1, and on the right side of the figure is illumination area 121-2 illuminated with light from light-emitting unit 121-2. Part of the imaging area includes overlap area 121-12 where illumination area 121-1 and illumination area 121-2 overlap.

[0176] In this way, when there is only the overlapping area 121-12 in the imaging area, the above-described processing is performed on the overlapping area 121-12, and distance measurement information can be generated in the same way as in the above-described embodiment.

[0177] According to the present technology, radiation noise can be reduced.

[0178] <Application example to endoscopic surgery system> The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology according to the present disclosure may be applied to an endoscopic surgery system.

[0179] FIG. 45 is a diagram showing an example of a schematic configuration of an endoscopic surgery system to which the technology according to the present disclosure (the present technology) can be applied.

[0180] Figure 45 shows an operator (doctor) 11131 performing surgery on a patient 11132 on a patient bed 11133 using an endoscopic surgery system 11000. As shown in the figure, the endoscopic surgery system 11000 is composed of an endoscope 11100, other surgical instruments 11110 such as an insufflation tube 11111 and an energy treatment instrument 11112, a support arm device 11120 that supports the endoscope 11100, and a cart 11200 on which various devices for endoscopic surgery are mounted.

[0181] The endoscope 11100 is composed of a lens barrel 11101, a region of a predetermined length from the tip of which is inserted into a body cavity of a patient 11132, and a camera head 11102 connected to the base end of the lens barrel 11101. In the example shown, the endoscope 11100 is configured as a so-called rigid scope having a rigid lens barrel 11101, but the endoscope 11100 may also be configured as a so-called flexible scope having a flexible lens barrel.

[0182] An opening into which an objective lens is fitted is provided at the tip of the lens barrel 11101. A light source device 11203 is connected to the endoscope 11100, and light generated by the light source device 11203 is guided to the tip of the lens barrel by a light guide extending inside the lens barrel 11101, and is irradiated via the objective lens towards an object to be observed inside the body cavity of the patient 11132. The endoscope 11100 may be a direct-viewing endoscope, an oblique-viewing endoscope, or a side-viewing endoscope.

[0183] An optical system and an image sensor are provided inside the camera head 11102, and light reflected from the object of observation (observation light) is collected onto the image sensor by the optical system. The observation light is photoelectrically converted by the image sensor to generate an electrical signal corresponding to the observation light, i.e., an image signal corresponding to the observed image. The image signal is sent to a camera control unit (CCU) 11201 as RAW data.

[0184] The CCU 11201 is configured with a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc., and comprehensively controls the operations of the endoscope 11100 and the display device 11202. Furthermore, the CCU 11201 receives an image signal from the camera head 11102, and performs various image processing on the image signal, such as development processing (demosaic processing), to display an image based on the image signal.

[0185] Under the control of the CCU 11201, the display device 11202 displays an image based on an image signal that has been subjected to image processing by the CCU 11201.

[0186] The light source device 11203 is configured from a light source such as an LED (light emitting diode), and supplies irradiation light to the endoscope 11100 when photographing an operation site or the like.

[0187] The input device 11204 is an input interface for the endoscopic surgery system 11000. A user can input various information and instructions to the endoscopic surgery system 11000 via the input device 11204. For example, the user inputs an instruction to change the imaging conditions (type of irradiating light, magnification, focal length, etc.) of the endoscope 11100.

[0188] The treatment tool control device 11205 controls the driving of the energy treatment tool 11112 for cauterizing tissue, incising, sealing blood vessels, etc. The insufflation device 11206 inflates the body cavity of the patient 11132 through the insufflation tube 11111 in order to ensure a clear field of view for the endoscope 11100 and a working space for the surgeon. The recorder 11207 is a device capable of recording various types of information related to the surgery. The printer 11208 is a device capable of printing various types of information related to the surgery in various formats such as text, images, or graphs.

[0189] The light source device 11203 that supplies illumination light to the endoscope 11100 when photographing the surgical site can be configured from a white light source configured from, for example, an LED, a laser light source, or a combination of these. When the white light source is configured from a combination of RGB laser light sources, the output intensity and output timing of each color (each wavelength) can be controlled with high precision, making it possible to adjust the white balance of the captured image in the light source device 11203. In this case, it is also possible to capture images corresponding to each RGB in a time-division manner by irradiating the object of observation with laser light from each RGB laser light source in a time-division manner and controlling the drive of the image sensor of the camera head 11102 in synchronization with the irradiation timing. According to this method, a color image can be obtained without providing a color filter to the image sensor.

[0190] Furthermore, the light source device 11203 may be controlled to change the intensity of light it outputs at predetermined time intervals. By controlling the driving of the image sensor of the camera head 11102 in synchronization with the timing of the change in light intensity to acquire images in a time-division manner and combining the images, it is possible to generate an image with a high dynamic range that is free of so-called blocked-up shadows and blown-out highlights.

[0191] The light source device 11203 may also be configured to supply light in a predetermined wavelength band corresponding to special light observation. Special light observation, for example, utilizes the wavelength dependence of light absorption in body tissues to irradiate light with a narrower band than the light irradiated during normal observation (i.e., white light), thereby capturing high-contrast images of specific tissues, such as blood vessels on the surface of mucous membranes, known as narrow-band imaging. Alternatively, special light observation may be performed using fluorescence observation, in which images are obtained using fluorescence generated by irradiating excitation light. Fluorescence observation can involve irradiating excitation light onto body tissues and observing the fluorescence from the tissues (autofluorescence observation), or locally injecting a reagent such as indocyanine green (ICG) into the body tissue and irradiating the tissue with excitation light corresponding to the fluorescent wavelength of the reagent to obtain a fluorescent image. The light source device 11203 may be configured to supply narrow-band light and / or excitation light corresponding to such special light observation.

[0192] FIG. 46 is a block diagram showing an example of the functional configuration of the camera head 11102 and the CCU 11201 shown in FIG.

[0193] The camera head 11102 has a lens unit 11401, an imaging unit 11402, a drive unit 11403, a communication unit 11404, and a camera head control unit 11405. The CCU 11201 has a communication unit 11411, an image processing unit 11412, and a control unit 11413. The camera head 11102 and the CCU 11201 are connected to each other by a transmission cable 11400 so that they can communicate with each other.

[0194] The lens unit 11401 is an optical system provided at the connection point with the lens barrel 11101. Observation light taken in from the tip of the lens barrel 11101 is guided to the camera head 11102 and enters the lens unit 11401. The lens unit 11401 is composed of a combination of multiple lenses including a zoom lens and a focus lens.

[0195] The imaging unit 11402 may include one imaging element (a so-called single-chip type) or multiple imaging elements (a so-called multi-chip type). When the imaging unit 11402 is configured as a multi-chip type, for example, each imaging element may generate an image signal corresponding to each of RGB, and a color image may be obtained by combining these signals. Alternatively, the imaging unit 11402 may be configured to have a pair of imaging elements for acquiring image signals for the right eye and the left eye corresponding to 3D (dimensional) display. 3D display allows the surgeon 11131 to more accurately grasp the depth of the biological tissue at the surgical site. Note that when the imaging unit 11402 is configured as a multi-chip type, multiple lens units 11401 may be provided corresponding to each imaging element.

[0196] Furthermore, the imaging unit 11402 does not necessarily have to be provided in the camera head 11102. For example, the imaging unit 11402 may be provided inside the lens barrel 11101, immediately after the objective lens.

[0197] The driving unit 11403 is configured by an actuator, and moves the zoom lens and focus lens of the lens unit 11401 by a predetermined distance along the optical axis under the control of the camera head control unit 11405. This allows the magnification and focus of the image captured by the imaging unit 11402 to be adjusted appropriately.

[0198] The communication unit 11404 is configured by a communication device for transmitting and receiving various types of information to and from the CCU 11201. The communication unit 11404 transmits the image signal obtained from the imaging unit 11402 to the CCU 11201 via the transmission cable 11400 as RAW data.

[0199] Furthermore, the communication unit 11404 receives a control signal for controlling the driving of the camera head 11102 from the CCU 11201 and supplies the control signal to the camera head control unit 11405. The control signal includes information on the imaging conditions, such as information specifying the frame rate of the captured image, information specifying the exposure value at the time of imaging, and / or information specifying the magnification and focus of the captured image.

[0200] The image capturing conditions such as the frame rate, exposure value, magnification, and focus may be appropriately specified by the user, or may be automatically set by the control unit 11413 of the CCU 11201 based on the acquired image signal. In the latter case, the endoscope 11100 is equipped with so-called AE (Auto Exposure) function, AF (Auto Focus) function, and AWB (Auto White Balance) function.

[0201] The camera head control unit 11405 controls the driving of the camera head 11102 based on a control signal received from the CCU 11201 via the communication unit 11404 .

[0202] The communication unit 11411 is configured by a communication device for transmitting and receiving various information to and from the camera head 11102. The communication unit 11411 receives an image signal transmitted from the camera head 11102 via the transmission cable 11400.

[0203] Furthermore, the communication unit 11411 transmits to the camera head 11102 a control signal for controlling the driving of the camera head 11102. The image signal and the control signal can be transmitted by electrical communication, optical communication, or the like.

[0204] The image processing unit 11412 performs various image processing operations on the image signal, which is RAW data sent from the camera head 11102 .

[0205] The control unit 11413 performs various controls related to the imaging of the surgical site, etc. by the endoscope 11100 and the display of the captured image obtained by imaging the surgical site, etc. For example, the control unit 11413 generates a control signal for controlling the driving of the camera head 11102.

[0206] Furthermore, the control unit 11413 causes the display device 11202 to display a captured image showing the surgical site, etc., based on the image signal that has been image processed by the image processing unit 11412. At this time, the control unit 11413 may recognize various objects in the captured image using various image recognition technologies. For example, the control unit 11413 can recognize surgical tools such as forceps, specific biological parts, bleeding, mist generated when using the energy treatment tool 11112, etc., by detecting the shape and color of the edges of objects included in the captured image. When displaying the captured image on the display device 11202, the control unit 11413 may use the recognition results to superimpose various surgical support information on the image of the surgical site. By superimposing the surgical support information and presenting it to the surgeon 11131, the burden on the surgeon 11131 can be reduced and the surgeon 11131 can proceed with the surgery reliably.

[0207] The transmission cable 11400 connecting the camera head 11102 and the CCU 11201 is an electrical signal cable for communication of electrical signals, an optical fiber for optical communication, or a composite cable of these.

[0208] In the illustrated example, communication is performed by wire using the transmission cable 11400, but communication between the camera head 11102 and the CCU 11201 may also be performed wirelessly.

[0209] <Application to moving objects> The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology according to the present disclosure may be realized as a device mounted on any type of moving body, such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, personal mobility, an airplane, a drone, a ship, or a robot.

[0210] FIG. 47 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile object control system to which the technology according to the present disclosure can be applied.

[0211] The vehicle control system 12000 includes a plurality of electronic control units connected via a communication network 12001. In the example shown in Fig. 47, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside-vehicle information detection unit 12030, an inside-vehicle information detection unit 12040, and an integrated control unit 12050. Also shown as functional components of the integrated control unit 12050 are a microcomputer 12051, an audio / video output unit 12052, and an in-vehicle network I / F (Interface) 12053.

[0212] The drivetrain control unit 12010 controls the operation of devices related to the drivetrain of the vehicle in accordance with various programs. For example, the drivetrain control unit 12010 functions as a control device for a drive force generating device for generating a drive force of the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, a braking device for generating a braking force of the vehicle, etc.

[0213] The body system control unit 12020 controls the operation of various devices equipped in the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as headlamps, backup lamps, brake lamps, turn signals, and fog lamps. In this case, radio waves transmitted from a portable device that serves as a key or signals from various switches may be input to the body system control unit 12020. The body system control unit 12020 receives these radio waves or signals and controls the vehicle's door lock device, power window device, lamps, etc.

[0214] The outside-vehicle information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the outside-vehicle information detection unit 12030. The outside-vehicle information detection unit 12030 causes the imaging unit 12031 to capture images outside the vehicle and receives the captured images. The outside-vehicle information detection unit 12030 may perform object detection processing or distance detection processing for people, cars, obstacles, signs, characters on the road surface, etc., based on the received images.

[0215] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal according to the amount of light received. The imaging unit 12031 can output the electrical signal as an image, or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0216] The in-vehicle information detection unit 12040 detects information inside the vehicle. For example, a driver state detection unit 12041 that detects the state of the driver is connected to the in-vehicle information detection unit 12040. The driver state detection unit 12041 includes, for example, a camera that captures an image of the driver, and the in-vehicle information detection unit 12040 may calculate the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041, or may determine whether the driver is dozing off.

[0217] The microcomputer 12051 can calculate control target values ​​for the driving force generating device, steering mechanism, or braking device based on the information inside and outside the vehicle acquired by the outside-vehicle information detection unit 12030 or the inside-vehicle information detection unit 12040, and output control commands to the drivetrain control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing the functions of an ADAS (Advanced Driver Assistance System), including avoiding or mitigating collisions between vehicles, following based on the distance between vehicles, maintaining vehicle speed, warning of vehicle collisions, or warning of vehicle lane departure.

[0218] In addition, the microcomputer 12051 can perform cooperative control for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on driver operation, by controlling the driving force generating device, steering mechanism, braking device, etc. based on information about the surroundings of the vehicle obtained by the outside vehicle information detection unit 12030 or the inside vehicle information detection unit 12040.

[0219] Furthermore, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information about the outside of the vehicle acquired by the outside information detection unit 12030. For example, the microcomputer 12051 can control the headlamps according to the position of a preceding vehicle or an oncoming vehicle detected by the outside information detection unit 12030, and perform cooperative control for the purpose of preventing glare, such as switching from high beams to low beams.

[0220] The audio / video output unit 12052 transmits at least one of audio and video output signals to an output device capable of visually or audibly notifying information to passengers in the vehicle or to the outside of the vehicle. In the example of Fig. 47, an audio speaker 12061, a display unit 12062, and an instrument panel 12063 are exemplified as output devices. The display unit 12062 may include, for example, at least one of an on-board display and a head-up display.

[0221] FIG. 48 is a diagram showing an example of the installation position of the imaging unit 12031.

[0222] In FIG. 48, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

[0223] The imaging units 12101, 12102, 12103, 12104, and 12105 are provided, for example, at positions such as the front nose, side mirrors, rear bumper, back door, and the top of the windshield inside the vehicle cabin of the vehicle 12100. The imaging unit 12101 provided at the front nose and the imaging unit 12105 provided at the top of the windshield inside the vehicle cabin mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 provided at the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 provided at the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 provided at the top of the windshield inside the vehicle cabin is mainly used to detect preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, lanes, etc.

[0224] 48 shows an example of the imaging ranges of the imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of the imaging unit 12101 provided on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of the imaging units 12102 and 12103 provided on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of the imaging unit 12104 provided on the rear bumper or back door. For example, by overlaying the image data captured by the imaging units 12101 to 12104, a bird's-eye view image of the vehicle 12100 viewed from above can be obtained.

[0225] At least one of the imaging units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera made up of multiple imaging elements, or may be an imaging element having pixels for phase difference detection.

[0226] For example, the microcomputer 12051 can calculate the distance to each three-dimensional object within the imaging ranges 12111 to 12114 and the change in this distance over time (relative speed with respect to the vehicle 12100) based on the distance information obtained from the imaging units 12101 to 12104, thereby extracting as a preceding vehicle, in particular, the three-dimensional object that is the closest three-dimensional object on the path of the vehicle 12100 and traveling in approximately the same direction as the vehicle 12100 at a predetermined speed (for example, 0 km / h or higher). Furthermore, the microcomputer 12051 can set a vehicle-to-vehicle distance to be maintained in advance in front of the preceding vehicle, and perform automatic braking control (including follow-up stop control), automatic acceleration control (including follow-up start control), etc. In this way, cooperative control can be performed for the purpose of automatic driving, which runs autonomously without relying on driver operation.

[0227] For example, the microcomputer 12051 classifies and extracts three-dimensional object data regarding three-dimensional objects into two-wheeled vehicles, ordinary vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects based on distance information obtained from the imaging units 12101 to 12104, and can use the data for automatic obstacle avoidance. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 then determines the collision risk, which indicates the degree of risk of collision with each obstacle, and when the collision risk is equal to or greater than a set value and a collision is possible, the microcomputer 12051 can provide driving assistance for collision avoidance by outputting an alarm to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or avoidance steering via the drivetrain control unit 12010.

[0228] At least one of the image capturing units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether or not a pedestrian is present in the images captured by the image capturing units 12101 to 12104. The pedestrian recognition is performed, for example, by extracting feature points from the images captured by the image capturing units 12101 to 12104, which are infrared cameras, and then performing pattern matching on a series of feature points that indicate the outline of an object to determine whether or not the object is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the images captured by the image capturing units 12101 to 12104 and recognizes the pedestrian, the audio / image output unit 12052 controls the display unit 12062 to superimpose a rectangular outline on the recognized pedestrian for emphasis. The audio / image output unit 12052 may also control the display unit 12062 to display an icon or the like indicating the pedestrian at a desired position.

[0229] In this specification, a system refers to an entire device made up of multiple devices.

[0230] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0231] It should be noted that the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible within the scope of the present technology.

[0232] The present technology can also be configured as follows. (1) a plurality of light emitting units including a first light emitting unit and a second light emitting unit that irradiate a subject with light; a light emission control unit that controls light emission of each of the plurality of light emitting units; a light receiving unit that receives reflected light from the subject; a distance measuring unit that calculates the distance to the subject based on a signal obtained by the light reception; Equipped with The light emission control unit Driving the first light-emitting unit based on a first drive signal; driving the second light-emitting unit based on a second drive signal; The first drive signal and the second drive signal are controlled to have a predetermined phase difference. Ranging device. (2) The first drive signal is synchronized with the drive signal that drives the light receiving unit. The distance measuring device according to (1) above. (3) The first drive signal and the second drive signal have a phase difference of 180 degrees. The distance measuring device according to (1) above. (4) The amplitude of the first drive signal is different from the amplitude of the second drive signal. The distance measuring device according to (1) above. (5) The imaging area includes a first illumination area illuminated with light from the first light-emitting unit, a second illumination area illuminated with light from the second light-emitting unit, and an overlap area where the first illumination area and the second illumination area overlap. The distance measuring device according to (4) above. (6) The distance is calculated without correction between the first irradiation area and the overlapping area; The second illumination area corrects the phase difference to calculate the distance. The distance measuring device according to (5) above. (7) A lookup table is held that indicates the area in which the distance is calculated without correction and the area in which the distance is calculated with correction. The distance measuring device according to (6) above. (8) the light receiving unit is driven without correcting the first irradiation area and the overlapping area, The second illumination area is driven by correcting the phase difference. The distance measuring device according to (5) above. (9) The emission wavelength of the first light-emitting portion is different from the emission wavelength of the second light-emitting portion. The distance measuring device according to (1) above. (10) Each pixel receives different wavelengths of light The distance measuring device according to (9) above. (11) The wavelength of light received varies for each predetermined area. The distance measuring device according to (9) above. (12) The wavelength of the light received is changed in a time-division manner The distance measuring device according to (9) above. (13) a plurality of light emitting units including a first light emitting unit and a second light emitting unit that irradiate a subject with light; a light emission control unit that controls light emission of each of the plurality of light emitting units; a light receiving unit that receives reflected light from the subject; a distance measuring unit that calculates the distance to the subject based on a signal obtained by the light reception; The light emission control unit of the distance measuring device is Driving the first light-emitting unit based on a first drive signal; driving the second light-emitting unit based on a second drive signal; The first drive signal and the second drive signal are controlled to have a predetermined phase difference. Distance measurement method. [Explanation of symbols]

[0233] 10 Range finding device, 11 Lens, 12 Light receiving section, 13 Signal processing section, 14 Light emitting section, 15 Light emission control section, 16 Filter section, 41 Pixel array section, 42 Vertical driving section, 43 Column processing section, 44 Horizontal driving section, 45 System control section, 46 Pixel driving line, 47 Vertical signal line, 48 Signal processing section, 50 Pixel, 51 Tap, 61 Photodiode, 62 Transfer transistor, 63 FD section, 64 Reset transistor, 65 Amplification transistor, 66 Selection transistor, 100 Range finding system, 101 Camera head section, 102 Signal processing section, 121 Light emitting section, 122 Light receiving section, 123 Lens, 124 Light emission control section, 125 Power supply section, 126 Control section, 127 Non-volatile memory section, 128 communication section, 131 communication section, 132 distance calculation section, 133 distance correction section, 134 control signal transmission section, 135 communication section, 201 logic circuit, 202 photodetector, 211 gate drive signal adjustment circuit, 222 LED, 241 reflective member, 251 diffusion member, 271 control board, 301 light source, 311 phase control section, 312 light emitter, 501 filter, 511, 512 band-pass filter, 531 filter, 541, 542 band-pass filter, 611 charge storage section, 631 charge transfer section, 633 ground section, 651 charge storage section

Claims

1. a plurality of light emitting units including a first light emitting unit and a second light emitting unit that irradiate a subject with light; a light emission control unit that controls light emission of each of the plurality of light emitting units; a light receiving unit that receives reflected light from the subject; a distance measuring unit that calculates the distance to the subject based on a signal obtained by the light reception; Equipped with The light emission control unit Driving the first light-emitting unit based on a first drive signal; driving the second light-emitting unit based on a second drive signal; The first drive signal and the second drive signal are controlled to have a predetermined phase difference. Ranging device.

2. The first drive signal is synchronized with a drive signal that drives the light receiving unit.

2. The distance measuring device according to claim 1.

3. The first drive signal and the second drive signal have a phase difference of 180 degrees.

2. The distance measuring device according to claim 1.

4. The amplitude of the first drive signal is different from the amplitude of the second drive signal.

2. The distance measuring device according to claim 1.

5. The imaging area includes a first illumination area irradiated with light from the first light-emitting unit, a second illumination area irradiated with light from the second light-emitting unit, and an overlap area where the first illumination area and the second illumination area overlap.

5. The distance measuring device according to claim 4.

6. The distance is calculated without correction for the first irradiation area and the overlapping area; The second illumination area corrects the phase difference to calculate the distance.

6. The distance measuring device according to claim 5.

7. A lookup table is held that indicates the area in which the distance is calculated without correction and the area in which the distance is calculated with correction.

7. The distance measuring device according to claim 6.

8. the light receiving unit is driven without correcting the first irradiation area and the overlapping area, The second illumination area is driven by correcting the phase difference.

6. The distance measuring device according to claim 5.

9. The emission wavelength of the first light-emitting portion is different from the emission wavelength of the second light-emitting portion.

2. The distance measuring device according to claim 1.

10. Each pixel receives different wavelengths of light 10. The distance measuring device according to claim 9.

11. The wavelength of light received varies for each predetermined area.

10. The distance measuring device according to claim 9.

12. The wavelength of the light received is changed in a time-division manner 10. The distance measuring device according to claim 9.

13. a plurality of light emitting units including a first light emitting unit and a second light emitting unit that irradiate a subject with light; a light emission control unit that controls light emission of each of the plurality of light emitting units; a light receiving unit that receives reflected light from the subject; a distance measuring unit that calculates the distance to the subject based on a signal obtained by the light reception; The light emission control unit of the distance measuring device is Driving the first light-emitting unit based on a first drive signal; driving the second light-emitting unit based on a second drive signal; The first drive signal and the second drive signal are controlled to have a predetermined phase difference. Distance measurement method.

Images