semiconductor devices

Abstract

One object is to effectively take measures against erroneous measurements of distance measurement information based on a laser light source. Therefore, a semiconductor device according to the present technology includes an imaging unit including a photoelectric conversion element that receives reflected light emitted from a specific laser light source reflected from an object and performs photoelectric conversion; and a control unit that performs validity determination processing for determining whether the light received by the photoelectric conversion element is light emitted from the specific laser light source.

Description

Technical Field

[0001] The present technology relates to a semiconductor device that performs distance measurement. Background Art

[0002] There is a distance measurement device that can prevent the output of erroneous distance measurement information with a simple structure (for example, see Patent Document 1). This distance measurement device uses a hardware logic circuit to superimpose a side channel signal on the main channel signal used to measure the relative distance to the object. This hardware logic circuit generates a random number that is difficult to copy or imitate. A comparison and verification unit compares the transmitted side channel data with the received side channel data and verifies the similarity to confirm the validity of the received signal.

[0003] Reference List

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2018-194297 Summary of the Invention

[0006] Problems to be solved by the present invention

[0007] However, the invention described in Patent Document 1 uses an LED as the light source for distance measurement, making it unsuitable for use with a laser resonator. In a real-world environment with various noises, extensive measurement and signal processing are required. Therefore, high-speed and accurate similarity verification from weak side channel signals is difficult.

[0008] The aim is therefore to take effective measures against distance measurement information based on erroneous measurements of laser light sources.

[0009] Problem Solution

[0010] A semiconductor device according to the present technology includes an imaging unit, which includes a photoelectric conversion element that receives reflected light of light emitted from a specific laser light source reflected from an object and performs photoelectric conversion; and a control unit that performs validity determination processing for determining whether the light received by the photoelectric conversion element is light emitted from the specific laser light source.

[0011] For example, in the case of performing distance measurement using a specific laser light source, the control unit determines that the received light is reflected light of the light emitted from the specific laser light source.

[0012] The photoelectric conversion element in the above semiconductor device includes a first photoelectric conversion element, and the first photoelectric conversion element may be connected to at least four charge accumulation units, and charges accumulated in different time periods in the first photoelectric conversion element are transferred to the charge accumulation units.

[0013] As a result, a light reception waveform based on the waveform of the light emitted from the laser light source can be obtained.

[0014] The photoelectric conversion element in the above-mentioned semiconductor device includes a second photoelectric conversion element, and the number of charge accumulation units connected to the second photoelectric conversion element may be smaller than the number of charge accumulation units connected to the first photoelectric conversion element.

[0015] For example, the second photoelectric conversion element is used for distance measurement.

[0016] The number of second photoelectric conversion elements in the semiconductor device may be greater than the number of first photoelectric conversion elements.

[0017] As a result, the number of charge accumulation units connected to the photoelectric conversion element is further reduced.

[0018] In the above-described semiconductor device, the first photoelectric conversion element may be arranged outside a group of second photoelectric conversion elements.

[0019] As a result, the second photoelectric conversion elements can be densely arranged.

[0020] The first photoelectric conversion element in the semiconductor device described above can be used for effectiveness determination processing and can also be used for distance measurement.

[0021] As a result, light reception data received by the first photoelectric conversion element can be effectively used.

[0022] The first photoelectric conversion element in the above-described semiconductor device may be used for effectiveness determination processing, and the second photoelectric conversion element may be used for distance measurement.

[0023] That is, some photoelectric conversion elements are used for effectiveness determination processing.

[0024] In the above-described semiconductor device validity determination process, determination based on the rising waveform of the laser light source may be performed.

[0025] The rising waveform of light emitted from a laser light source varies depending on the laser resonator that generates the laser light, making it difficult for anyone other than the laser light source manufacturer to replicate the rising waveform.

[0026] In the effectiveness determination process of the semiconductor device described above, determination based on the shape of a light-receiving spot of light emitted from a laser light source may be performed.

[0027] The light receiving point shape is determined based on the spatial positional relationship among the laser light source, the object, and the imaging unit.

[0028] The above-mentioned semiconductor device may include an irradiation unit (including a specific laser light source).

[0029] By integrating the specific laser light source and the imaging unit, the positional relationship between the specific laser light source and the imaging unit is fixed.

[0030] The irradiation unit in the above-mentioned semiconductor device is capable of emitting laser light having a first waveform and laser light having a second waveform different from the first waveform.

[0031] As a result, effectiveness determination processing can be performed using both the first waveform and the second waveform.

[0032] In the above-described semiconductor device, light intensities at the time of emission of the first waveform and the second waveform may be different from each other.

[0033] As a result, effectiveness determination processing taking light intensity into consideration can be performed.

[0034] In the above semiconductor device, rising shapes of the first waveform and the second waveform may be different from each other.

[0035] As a result, the validity determination process can be performed using the rising shapes of both the first waveform and the second waveform.

[0036] The irradiation unit in the above semiconductor device may emit the first waveform and the second waveform in a random order.

[0037] As a result, the validity determination process can be performed in consideration of the order of appearance of the first waveform and the second waveform.

[0038] In the above-described semiconductor device, the light emission time lengths of the first waveform and the second waveform may be different from each other.

[0039] As a result, the effectiveness determination process can be performed in consideration of the light emission time lengths of the first waveform and the second waveform.

[0040] In the above-described semiconductor device, the non-light emitting time lengths of the first waveform and the second waveform may be different from each other.

[0041] As a result, the effectiveness determination process can be performed in consideration of the non-light emitting time lengths of the first waveform and the second waveform.

[0042] The irradiation unit in the semiconductor device may emit spot-shaped laser light having an emission range equal to or smaller than a quarter of an imaging range of the imaging unit.

[0043] As a result, even if the spot shape of light emitted on the object is modified and the width is doubled, the spot shape can be within the imaging range.

[0044] The irradiation unit in the semiconductor device can emit laser light having a mirror-symmetrical or point-symmetrical dot shape.

[0045] This makes it easy to realize point-like lasing.

[0046] The irradiation unit in the semiconductor device may emit laser light having a non-mirror-symmetrical or non-point-symmetrical spot shape.

[0047] This makes it difficult to mimic point-like lasers.

[0048] The irradiation unit in the above-mentioned semiconductor device may emit a dot pattern in which a plurality of dot-shaped laser beams are irregularly arranged.

[0049] As a result, a plurality of point-like laser beams can be detected by one imaging by the imaging unit.

[0050] The irradiation unit in the above-mentioned semiconductor device may emit a dot pattern in which a plurality of dot-shaped laser beams are regularly arranged.

[0051] This makes it easy to generate dot patterns.

[0052] The irradiation unit in the above-mentioned semiconductor device may emit a dot pattern in which the dot laser beams are arranged in a number equal to or less than one-fourth the number of the photoelectric conversion elements.

[0053] As a result, the effectiveness determination process can be performed in consideration of the light-receiving point shape according to each of the spot light beams forming the dot pattern.

[0054] The irradiation unit in the above-mentioned semiconductor device may emit a specific laser light source so that laser light having a spot shape smaller than an imaging range of the imaging unit draws a specific trajectory within the imaging range.

[0055] As a result, the validity determination process using a specific trajectory, ie, the scanning trajectory, can be performed. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a diagram illustrating a system configuration example of an imaging device according to an embodiment of the present technology.

[0057] Figure 2 is a diagram illustrating a configuration example of an imaging element.

[0058] Figure 3 is a diagram showing a configuration example of a unit pixel as an equivalent circuit.

[0059] Figure 4 is a schematic diagram illustrating a configuration example of a unit pixel.

[0060] Figure 5 This is an example of a timing diagram for range image generation.

[0061] Figure 6 is a graph schematically showing an example of a waveform of a pixel signal.

[0062] Figure 7is a graph showing an example of the relationship between the wavelength and light intensity of a laser resonator.

[0063] Figure 8 is a graph showing an example of the relationship between the wavelength and light intensity of the laser resonator in the first light emission mode.

[0064] Figure 9 Graph showing an example of temporal changes in the light intensity of the laser resonator in the first emission mode.

[0065] Figure 10 Graph showing an example of the relationship between the wavelength and the light intensity of the laser resonator in the second emission mode.

[0066] Figure 11 Graph showing an example of temporal changes in the light intensity of the laser resonator in the second emission mode.

[0067] Figure 12 is a flowchart of a first example of validity determination processing.

[0068] Figure 13 is a graph showing reference conditions in the first example of the effectiveness determination process.

[0069] Figure 14 is a graph showing an example of a normal light reception waveform determined to be valid in the first example of the validity determination process.

[0070] Figure 15 is a graph showing an example of an abnormal light reception waveform determined to be invalid in the first example of the effectiveness determination process.

[0071] Figure 16 is a graph showing another example of an abnormal light reception waveform determined to be invalid in the first example of the effectiveness determination process.

[0072] Figure 17 is a graph showing still another example of an abnormal light reception waveform determined to be invalid in the first example of the effectiveness determination process.

[0073] Figure 18 : is a diagram showing a system configuration of an imaging device in a second example of the effectiveness determination process.

[0074] Figure 19 is a flowchart of a second example of the validity determination process.

[0075] Figure 20 is a flowchart of a modification of the second example of the validity determination process.

[0076] Figure 21: is a graph showing an example of the waveform of light emitted in the third example of the effectiveness determination process.

[0077] Figure 22 is a graph illustrating another example of the waveform of light emitted in the third example of the effectiveness determination process.

[0078] Figure 23 is a graph showing still another example of the waveform of emitted light in the third example of the effectiveness determination process.

[0079] Figure 24 are diagrams illustrating examples of transmission signals and transmission waveforms in the third example of effectiveness determination processing.

[0080] Figure 25 is a schematic block diagram in a fourth example of the validity determination process.

[0081] Figure 26 : is a diagram showing the relationship between the transmission waveform, the light reception waveform, and the accumulated charge amount in the fourth example of the effectiveness determination process.

[0082] Figure 27 is a diagram showing an example of a light reception waveform and an electric charge accumulation amount.

[0083] Figure 28 : is a diagram showing the relationship among the emission signal, the reference signal, and the charge amount for each charge accumulation period in the fourth example of the effectiveness determination process.

[0084] Figure 29 is a diagram illustrating a relationship between a transmission signal and a reference signal in a fifth example of the validity determination process.

[0085] Figure 30 is a diagram illustrating a relationship between a transmission signal and a reference signal in a sixth example of effectiveness determination processing.

[0086] Figure 31 : is a diagram showing the relationship between the transmission signal and the reference signal in a modification of the sixth example of the effectiveness determination process.

[0087] Figure 32 : is a diagram showing an arrangement example of the first photoelectric conversion element and the second photoelectric conversion element in the seventh example of the effectiveness determination process.

[0088] Figure 33 : is a diagram showing another arrangement example of the first photoelectric conversion element and the second photoelectric conversion element in the seventh example of the effectiveness determination process.

[0089] Figure 34 is a flowchart of a first example of an eighth example of the validity determination process.

[0090] Figure 35 is a flowchart of a second example of the eighth example of the validity determination process.

[0091] Figure 36 is a flowchart of a third example of the eighth example of the validity determination process.

[0092] Figure 37 is a flowchart of a fourth example of the eighth example of the validity determination process.

[0093] Figure 38 : is a diagram showing a system configuration of an imaging device in a ninth example of the effectiveness determination process.

[0094] Figure 39 : is a diagram showing a configuration example of a signal processing circuit in a ninth example of validity determination processing.

[0095] Figure 40 : is a diagram illustrating an example of an emission pattern of laser light in a ninth example of effectiveness determination processing.

[0096] Figure 41 is a diagram illustrating an example of an emission pattern from another light source in a ninth example of the effectiveness determination process.

[0097] Figure 42 : is a diagram showing an example of the spot shape of laser light in the ninth example of the effectiveness determination process.

[0098] Figure 43 is a diagram illustrating an example of a dot pattern emitted from an irradiation unit.

[0099] Figure 44 is a diagram illustrating another example of a dot pattern emitted from an irradiation unit.

[0100] Figure 45 is a diagram illustrating still another example of a dot pattern emitted from an irradiation unit.

[0101] Figure 46 : is a diagram showing an example of a light-receiving dot pattern when laser light is emitted from an effective light source.

[0102] Figure 47 : is a diagram showing an example of a light-receiving dot pattern when laser light is emitted from an ineffective light source.

[0103] Figure 48 is a flowchart of a tenth example of the effectiveness determination process.

[0104] Figure 49 is an example of a light-receiving dot pattern determined to be invalid in the eleventh example of the effectiveness determination process.

[0105] Figure 50 is another example of a light-receiving dot pattern determined to be invalid in the eleventh example of the validity determination process.

[0106] Figure 51 : is a diagram showing the shape of effective light-receiving point light in the twelfth example of the effectiveness determination process.

[0107] Figure 52 : is a diagram showing the shape of invalid light-receiving point light in the twelfth example of the effectiveness determination process.

[0108] Figure 53 is a diagram illustrating an example of an effective light-receiving dot pattern.

[0109] Figure 54 : is a diagram illustrating an example of an invalid light-receiving dot pattern in the thirteenth example of the effectiveness determination process.

[0110] Figure 55 is a diagram illustrating another example of the invalid light-receiving dot pattern in the thirteenth example of the effectiveness determination process.

[0111] Figure 56 is a diagram illustrating another example of the invalid light-receiving dot pattern in the fourteenth example of the effectiveness determination process.

[0112] Figure 57 is a diagram illustrating another example of the invalid light-receiving dot pattern in the fifteenth example of the effectiveness determination process.

[0113] Figure 58 : is a diagram showing an example of the positions and postures of the imaging device and another imaging device in the embodiment in the fifteenth example of the effectiveness determination process.

[0114] Figure 59 is a diagram illustrating another example of the invalid light-receiving dot pattern in the sixteenth example of the effectiveness determination process.

[0115] Figure 60 : is a diagram showing a configuration example of an imaging element using a SPAD element in a seventeenth example of the effectiveness determination process.

[0116] Figure 61 is a cross-sectional view of a SPAD pixel in a seventeenth example of effectiveness determination processing.

[0117] Figure 62 is a plan view of a SPAD pixel in a seventeenth example of the validity determination process.

[0118] Figure 63: is a diagram showing the locus of the effective laser light source in the seventeenth example of the effectiveness determination process.

[0119] Figure 64 : is a diagram showing the locus of an invalid laser light source in the seventeenth example of the effectiveness determination process.

[0120] Figure 65 is a diagram showing another example of the locus of the effective laser light source in the seventeenth example of the effectiveness determination process.

[0121] Figure 66 is a diagram showing still another example of the locus of the effective laser light source in the seventeenth example of the effectiveness determination process. DETAILED DESCRIPTION

[0122] Hereinafter, the embodiments will be described in the following order.

[0123] <1. Configuration of Imaging Device>

[0124] <1-1. Overall Configuration>

[0125] <1-2. Configuration of Imaging Element>

[0126] <1-3. Configuration of Unit Pixels>

[0127] <2. Range Image Generation Method>

[0128] <3. Validity Determination Process>

[0129] <3-1. Laser Rising Waveform>

[0130] <3-2. First Example of Validity Determination Processing>

[0131] <3-3. Second Example of Validity Determination Processing>

[0132] <3-4. Third Example of Validity Determination Processing>

[0133] <3-5. Fourth Example of Validity Determination Processing>

[0134] <3-6. Fifth Example of Validity Determination Processing>

[0135] <3-7. Sixth Example of Validity Determination Processing>

[0136] <3-8. Seventh Example of Validity Determination Processing>

[0137] <3-9. Eighth Example of Validity Determination Processing>

[0138] <3-10. Ninth Example of Validity Determination Processing>

[0139] <3-11. Tenth Example of Validity Determination Processing>

[0140] <3-12. Eleventh Example of Validity Determination Processing>

[0141] <3-13. Twelfth Example of Validity Determination Processing>

[0142] <3-14. Thirteenth Example of Validity Determination Processing>

[0143] <3-15. Fourteenth Example of Validity Determination Processing>

[0144] <3-16. Fifteenth Example of Validity Determination Processing>

[0145] <3-17. Sixteenth Example of Validity Determination Processing>

[0146] <3-18. Seventeenth Example of Validity Determination Processing>

[0147] <4. Summary>

[0148] <5. This technology>

[0149] <1. Configuration of Imaging Device>

[0150] <1-1. Overall Configuration>

[0151] Figure 1 is a block diagram of an imaging device 1 according to this embodiment. The imaging device 1 is a device that captures a range image using, for example, a time-of-flight (ToF) method. Here, the range image represents an image including a range pixel signal detected for each pixel based on the distance of an object from the imaging device 1 in the depth direction.

[0152] The imaging device 1 includes an irradiation unit 2 , an imaging unit 3 , a control unit 4 , a display unit 5 , and a storage unit 6 .

[0153] The irradiation unit 2 includes an irradiation control unit 2a and a laser light source 2b. The irradiation control unit 2a controls the pattern in which the laser light source 2b emits the emission light (laser light) based on the control of the control unit 4. Specifically, the irradiation control unit 2a controls the pattern in which the laser light source 2b emits the emission light based on the emission code included in the emission signal provided by the control unit 4.

[0154] For example, the emission code includes two values, one (High) and zero (Low), and the irradiation control unit 2a controls to turn on the laser light source 2b when the value of the emission code is 1, and to turn off the laser light source 2b when the value of the emission code is 0.

[0155] The laser light source 2b emits light using a laser resonator (optical resonator) under the control of the irradiation control unit 2a. It is desirable to use near-infrared light, which is invisible or almost invisible to the naked eye, as the laser light emitted from the laser light source 2b. For this purpose, a light receiving sensor made of inexpensive silicon can be used. However, far-infrared light, visible light, ultraviolet light, or infrared light can also be used.

[0156] As the laser resonator included in the laser light source 2b, for example, a semiconductor laser (diode laser) is used. However, a solid-state laser, a liquid laser, a gas laser, or the like may be used.

[0157] As the semiconductor laser, the following can be used: an edge emitting laser (EEF), which has a structure in which a resonator is formed parallel to a semiconductor substrate and emits light on a cleavage side surface; a surface emitting laser (SEL), which has a structure in which light is emitted in a direction perpendicular to the semiconductor substrate; a vertical cavity surface emitting laser (VCSEL), which is a surface emitting laser in which a resonator is formed perpendicular to a semiconductor substrate; a vertical external cavity surface emitting laser (VECSEL) having a resonator outside, etc.

[0158] The imaging unit 3 includes a lens 3 a , an imaging element 3 b , and a signal processing circuit 3 c .

[0159] The lens 3a forms an image of the incident light on the imaging surface of the imaging element 3b. The lens 3a may have any configuration and may include, for example, a plurality of lens groups.

[0160] The imaging element 3b includes, for example, a complementary metal oxide semiconductor (CMOS) image sensor using a ToF method. The imaging element 3b images the objects 100 and 101 based on the control of the control unit 4 and supplies the obtained image signal as an imaging result to the signal processing circuit 3c.

[0161] Specifically, the imaging element 3b generates a pixel signal indicating the correlation between the reference signal provided from the control unit 4 and the received light (including the reflected light obtained by reflecting the emitted light emitted from the laser light source 2b by the objects 100 and 101, etc.) and provides the pixel signal to the signal processing circuit 3c.

[0162] Note that the reference signal includes a reference code indicating a pattern for detecting correlation with received light.

[0163] The signal processing circuit 3c processes the pixel signal supplied from the imaging element 3b based on the control of the control unit 4. For example, the signal processing circuit 3c generates a distance image based on the pixel signal supplied from the imaging element 3b.

[0164] The signal processing circuit 3 c supplies the generated distance image to the control unit 4 .

[0165] The control unit 4 includes, for example, a control circuit such as a field programmable gate array (FPGA) or a digital signal processor (DSP), a processor, etc. The control unit 4 controls the irradiation control unit 2a, the imaging element 3b, and the signal processing circuit 3c.

[0166] Furthermore, the control unit 4 supplies the distance image acquired from the imaging unit 3 to the display unit 5 and displays the distance image on the display unit 5 .

[0167] Furthermore, the control unit 4 causes the storage unit 6 to store the distance image acquired from the imaging unit 3 .

[0168] Furthermore, the control unit 4 outputs the distance image acquired from the imaging unit 3 to the outside.

[0169] The display unit 5 includes a panel-type display device such as a liquid crystal display device, an organic electroluminescence (EL) display device, or the like.

[0170] The storage unit 6 may include any storage device, storage medium, etc. and stores the distance image and the like.

[0171] Furthermore, in each unit included in the imaging device 1 , each unit for executing various types of processing is provided.

[0172] <1-2. Configuration of Imaging Element>

[0173] The configuration of the imaging element 3b is as follows Figure 2 shown.

[0174] The imaging element 3 b includes a pixel array unit 7 , a vertical drive unit 8 , a column processing unit 9 , a horizontal drive unit 10 , a system control unit 11 , pixel drive lines 12 , vertical signal lines 13 , a signal processing unit 14 , and a data storage unit 15 .

[0175] The pixel array unit 7 includes pixels including a photoelectric conversion element that generates and accumulates electric charge according to the amount of reflected light entering from an object. The pixels included in the pixel array unit 7 are Figure 2 The shown rows and columns are arranged in a two-dimensional array.

[0176] For example, in the pixel array unit 7 , the pixel drive lines 12 are arranged in the row direction for each pixel row including pixels arranged in the row direction, and the vertical signal lines 13 are arranged in the column direction for each pixel column including pixels arranged in the column direction.

[0177] The vertical driving unit 8 includes a shift register, an address decoder, etc. and supplies a signal, etc. to each pixel via a plurality of pixel driving lines 12. All pixels or pixels in a row of the pixel array unit 7 are simultaneously driven based on the supplied signal.

[0178] The column processing unit 9 reads a signal from each pixel through the vertical signal line 13 for each pixel column of the pixel array unit 7 , performs noise removal processing, correlated double sampling processing, analog-to-digital (A / D) conversion processing, etc., and generates a pixel signal.

[0179] The horizontal drive unit 10 includes a shift register, an address decoder, and the like, and sequentially selects unit circuits corresponding to pixel columns of the column processing unit 9. Through the selection scanning of the horizontal drive unit 10, pixel signals, which have been signal-processed by the column processing unit 9 for each unit circuit, are sequentially output to the signal processing unit 14.

[0180] The system control unit 11 includes a timing generator that generates various timing signals, and the like, and drives and controls the vertical drive unit 8, the column processing unit 9, and the horizontal drive unit 10 based on the timing signals generated by the timing generator. The signal processing unit 14 performs signal processing such as calculation processing on the pixel signal supplied from the column processing unit 9, while temporarily storing the data in the data storage unit 15 as needed, and outputs an image signal including each pixel signal.

[0181] <1-3. Configuration of Unit Pixels>

[0182] Will refer to Figure 3 Describes a configuration example of a unit pixel. Note that Figure 3 It is the equivalent circuit of a unit pixel.

[0183] The unit pixel 16 constituting the pixel array unit 7 of the imaging element 3 b includes a photoelectric conversion element 17 , transfer transistors 18 a and 18 b , floating diffusions (FD) 20 a and 20 b , reset transistors 22 and 23 , a selection transistor 24 , and amplification transistors 25 and 26 .

[0184] Note that in Figure 3 , an example is shown in which N-channel MOS transistors are used as the reset transistors 22 and 23, the selection transistor 24, and the amplification transistors 25 and 26. However, the combination of conductivity types of the reset transistors 22 and 23, the selection transistor 24, and the amplification transistors 25 and 26 is not limited to this example.

[0185] The photoelectric conversion element 17 includes, for example, an embedded photodiode. Figure 4 Provide explanation.

[0186] The photoelectric conversion element 17 is formed by forming a p-type layer 28 on the substrate surface side of a p-type semiconductor substrate 27 and embedding an n-type embedded layer 29 .

[0187] The transfer transistor 18a includes a gate electrode 19a. The gate electrode 19a is formed so as to cover a region between the photoelectric conversion element 17 and the FD 20a via an insulating film 30 formed on the surface of the semiconductor substrate 27.

[0188] A transfer signal TXa is supplied from the vertical drive unit 8 to the gate electrode 19a through the pixel drive line 12. When the voltage of the transfer signal TXa is set to a predetermined high level and the transfer transistor 18a is in an on state, charges generated by the photoelectric conversion element 17 are transferred to the FD 20a through the transfer transistor 18a.

[0189] The transfer transistor 18b includes a gate electrode 19b. The gate electrode 19b is formed so as to cover a region between the photoelectric conversion element 17 and the FD 20b via an insulating film 30 formed on the surface of the semiconductor substrate 27.

[0190] A transfer signal TXb is supplied from the vertical drive unit 8 to the gate electrode 19b through the pixel drive line 12. When the voltage of the transfer signal TXb is set to a predetermined high level and the transfer transistor 18b is in an on state, charges generated by the photoelectric conversion element 17 are transferred to the FD 20b via the transfer transistor 18b.

[0191] Note that it is assumed that the transmission signal TXa is the same signal as the reference signal supplied from the control unit 4 to the system control unit 11, and the transmission signal TXb is a signal obtained by inverting one bit of the reference signal. Therefore, the charge generated by the photoelectric conversion element 17 is distributed to the FDs 20a and 20b.

[0192] As the correlation between the received light entering the photoelectric conversion element 17 and the reference signal is higher, the amount of charge transferred to and accumulated in the FD 20a increases, the amount of charge transferred to and accumulated in the FD 20b decreases, and the difference between the charge amounts increases.

[0193] On the other hand, as the correlation between the received light entering the photoelectric conversion element 17 and the reference signal becomes lower, the amount of charge transferred to and accumulated in the FD 20a decreases, the amount of charge transferred to and accumulated in the FD 20b increases, and the difference between the charge amounts decreases.

[0194] The insulating film 30 is formed to cover the entire surface of the semiconductor substrate 27 except for a portion of the upper side of the FD 20 a and a portion of the upper side of the FD 20 b .

[0195] The light shielding curtain 31 is formed to cover the entire upper portion of the semiconductor substrate 27 except for the upper side of the photoelectric conversion element 17 , a portion of the upper side of the FD 20 a , and a portion of the upper side of the FD 20 b .

[0196] The FD 20 a accumulates the charges transferred from the photoelectric conversion element 17 via the transfer transistor 18 a and converts the accumulated charges into a voltage.

[0197] The FD 20 b accumulates the charge transferred from the photoelectric conversion element 17 via the transfer transistor 18 b and converts the accumulated charge into a voltage.

[0198] The reset transistor 22 has a drain electrode connected to the pixel power supply Vdd, and a source electrode connected to the FD 20 a and the gate electrode of the amplification transistor 25 .

[0199] A reset signal RSTa is supplied from the vertical drive unit 8 to the gate electrode of the reset transistor 22 via the pixel drive line 12. When the voltage of the reset signal RSTa is set to a predetermined high level and the reset transistor 22 is turned on, the FD 20a is reset and charge is discharged from the FD 20a.

[0200] The reset transistor 23 has a drain electrode connected to the pixel power supply Vdd, and a source electrode connected to the FD 20 b and the gate electrode of the amplifying transistor 26 .

[0201] A reset signal RSTb is supplied from the vertical drive unit 8 to the gate electrode of the reset transistor 23 via the pixel drive line 12. When the voltage of the reset signal RSTb is set to a predetermined high level and the reset transistor 23 is turned on, the FD 20b is reset and charge is discharged from the FD 20b.

[0202] The selection transistor 24 has a drain electrode connected to the pixel power supply Vdd, and a source electrode connected to the drain electrode of the amplifier transistor 25 .

[0203] The source of the amplifying transistor 25 is connected to the vertical signal line 13. This vertical signal line is provided as 13a.

[0204] The source of the amplifying transistor 26 is connected to the vertical signal line 13. This vertical signal line is provided as 13b.

[0205] A selection signal SEL is supplied from the vertical drive unit 8 to the gate electrode of the selection transistor 24 via the pixel drive line 12. When the voltage of the selection signal SEL is set to a predetermined high level and the selection transistor 24 is turned on, the unit pixel 16 from which the pixel signal is read is selected.

[0206] That is, when the selection transistor 24 is turned on, the amplification transistor 25 supplies the signal SPa indicating the voltage of the FD 20 a to the column processing unit 9 via the vertical signal line 13 a .

[0207] When the selection transistor 24 is turned on, the amplification transistor 26 supplies a signal SPb indicating the voltage of the FD 20 b to the column processing unit 9 via the vertical signal line 13 b .

[0208] The signal processing unit 14 of the imaging element 3b generates a difference signal of the signals SPa and SPb supplied from each unit pixel 16 via the column processing unit 9. Furthermore, the signal processing unit 14 supplies the difference signal as a pixel signal of each unit pixel 16 to the signal processing circuit 3c.

[0209] Therefore, the pixel signal output from the imaging element 3b is a signal indicating the correlation between the reference signal and the received light of each unit pixel 16. That is, the higher the correlation between the reference signal and the received light, the larger the value of the pixel signal. The lower the correlation between the reference signal and the received light, the smaller the value of the pixel signal.

[0210] <2. Range Image Generation Method>

[0211] Will refer to Figure 5 and Figure 6 An example of a distance image generation method performed by the imaging device 1 is described with reference to each of .

[0212] The imaging device 1 generates a range image by performing imaging a plurality of times using different combinations of a transmission signal and a reference signal.

[0213] Figure 5 : is a timing chart of the range image generation method, in which the horizontal axis is the time axis t and illustrates respective timings of the transmission signal, the first reference signal a, the second reference signal b, and the received light.

[0214] Figure 6 The following schematically shows the relationship between the distance from the imaging device 1 to the subject and the pixel signal output from the unit pixel 16. The horizontal axis represents the distance to the subject, and the vertical axis represents the value of the pixel signal.

[0215] In the illustrated example, a combination of the transmission signal and the first reference signal a is used in the first imaging. The transmission signal and the first reference signal a are signals including pulses having the same phase and the same pulse width T.

[0216] The emission light emitted from the laser light source 2b has a waveform substantially the same as the emission signal and has a rising waveform unique to the manufactured laser resonator. In the following, for the convenience of describing the basic concept of the distance image generation method, the case of omitting the basic rectangular wave of the rising waveform will be described.

[0217] Part of the emission light emitted from the laser light source 2 b is reflected by an object located in the emission direction, and part of the reflected light is incident on the photoelectric conversion element 17 of each unit pixel 16 of the imaging element 3 b .

[0218] Here, the received light entering the photoelectric conversion element 17 enters the photoelectric conversion element 17 with a delay time Δt relative to the transmitted signal (transmitted light) according to the distance between the imaging device 1 and the subject.

[0219] At this time, the value of the pixel signal Sa output from the unit pixel 16 is proportional to the time that the first reference signal a overlaps with the received light. Specifically, with the rising pulse of the first reference signal a as a reference (the time of the rising pulse of the first reference signal a is set to zero), the value of the pixel signal Sa is maximized when the delay time Δt is zero and is proportional to the time T-Δt. Then, when the delay time Δt is equal to or greater than the pulse width T, the value of the pixel signal Sa becomes zero.

[0220] During the second imaging, a combination of the transmission signal and the second reference signal b is used. The second reference signal b has a waveform whose phase is delayed by the same time as the pulse width T as compared with the first reference signal a.

[0221] At this time, the value of the pixel signal Sb output from the unit pixel 16 is proportional to the time when the second reference signal b and the received light overlap. That is, when the delay time Δt is from zero to T, the value of the pixel signal Sb is proportional to the delay time Δt, and when the delay time Δt is from T to 2T, the value of the pixel signal Sb is proportional to the time 2T-Δt. When the delay time Δt becomes equal to or longer than 2T, the value of the pixel signal Sb becomes zero.

[0222] Here, the received light includes the reflected light of the transmitted light emitted in the same pattern as the transmitted signal (transmitted code), and the waveform of the received light is similar to the waveform obtained by shifting the phase of the transmitted signal. Figure 6 The waveform of the pixel signal Sa is similar to the waveform of the correlation function of the first reference signal a and the emission signal, and Figure 6 The waveform of the pixel signal Sb in is similar to the waveform of the correlation function of the second reference signal b and the emission signal.

[0223] Note that the waveform of the pixel signal Sa here represents the Figure 6 The distance indicated by the middle dashed line includes the waveform in the negative area.

[0224] As shown in the following Formula 1, the distance to the object is proportional to the ratio of the pixel signal Sb to the sum of the pixel signal Sa and the pixel signal Sb.

[0225] [Expression 1]

[0226]

[0227] For example, the signal processing circuit 3c generates a distance pixel signal based on the distance to the object based on the ratio of the pixel signal Sb of each pixel to the sum of the pixel signal Sa and the pixel signal Sb, and generates a distance image including the distance pixel signal. Based on the distance image, for example, the distance to the object can be measured for each pixel or the difference between distances can be identified.

[0228] Here, as Figure 6 As shown in FIG, since the inclination of the pixel signals Sa and Sb with respect to the distance is constant, the distance measurement accuracy (distance measurement accuracy) is constant regardless of the distance. The basic rectangular wave with the rising waveform omitted has been described above. However, in practice, it is desirable to consider Figure 6 And the rising waveform in Formula 1. In this case, the formula taking the rising waveform into account can be used for distance calculation. The result of the distance calculation is calculated without taking the rising waveform into account, and then the calculation result can be corrected based on the rising waveform.

[0229] Furthermore, the range in which the distance can be measured (distance measurement range) is set to the range before the pixel signal Sa becomes zero, and specifically, the range from zero to time c (speed of light)×T / 2.

[0230] <3. Validity Determination Process>

[0231] <3-1. Laser Rising Waveform>

[0232] In this embodiment, light emitted from laser light source 2b included in imaging device 1 enters imaging element 3b as reflected light from the object, and distance measurement is performed based on the incident light. Therefore, there is a possibility that erroneous measurement may be made due to light from a light source other than laser light source 2b, natural light, or the like entering imaging element 3b.

[0233] The imaging device 1 according to this embodiment has a configuration that reduces the possibility of such erroneous measurements. Specifically, a validity determination process is performed to determine whether the light received by the imaging element 3b is the light emitted by the laser light source 2b. This will be described below with reference to each of the accompanying drawings.

[0234] First, refer to Figures 7 to 11 , each of which is used to describe the rising waveform of the laser light source 2b.

[0235] The laser resonator included in the laser light source 2b can be regarded as a light emitting diode (LED) that satisfies laser oscillation conditions.

[0236] Figure 7This figure shows the relationship between wavelength and light intensity. A laser resonator has a unimodal characteristic, where light intensity is maximized at a single wavelength (center frequency). The oscillation wavelength emitted from a laser resonator has a temperature dependency, increasing as temperature increases. Therefore, the relationship between time and light intensity depends on changes in the laser resonator's temperature.

[0237] Figure 8 and Figure 9 This example is an example of a case where the operating current (operating power) of the laser resonator is small, and Figure 8 The wavelength F1 of light emitted when the temperature of the laser resonator rises from the temperature F0 to the temperature F1 is shown. Figure 9 Shows the temporal variation of light intensity.

[0238] As shown in the figure, because the operating current is low, the wavelength of light emitted from the laser resonator is set to be shorter than the wavelength at which the light intensity reaches maximum. Therefore, the light intensity (i.e., the rising waveform of the laser light) gradually increases over time and then remains constant after reaching a certain value.

[0239] Another example is Figure 10 and Figure 11 This example is an example in which the operating current of the laser resonator is large, and Figure 10 The wavelength of light emitted when the temperature of the laser resonator is increased from temperature F0 to temperature F2 is shown. Figure 11 Shows the temporal variation of light intensity.

[0240] As shown in the figure, because the operating current is high, the wavelength of light emitted from the laser resonator is set to be longer than the wavelength that maximizes the light intensity. As a result, the rising waveform of the laser light rises sharply, reaches a peak, and then drops to a certain value.

[0241] In the following description, Figure 9 The emission mode of the laser resonator with the rising waveform shown is called the "first emission mode" and will be represented by Figure 11 The emission mode of the laser resonator with the rising waveform shown is referred to as the “second emission mode.” Furthermore, the laser light in the first emission mode is referred to as the “first laser light,” and the laser light in the second emission mode is referred to as the “second laser light.”

[0242] The unimodal characteristics of the laser resonator described above depend on the structure of the laser resonator. In the case of a semiconductor laser, the center frequency depends on the components of the semiconductor. In addition, the oscillation spectrum depends on the light output (that is, the spectral linewidth is inversely proportional to the light output). For these reasons, the relationship between wavelength and light intensity ( Figure 7 ) and the relationship between time and light intensity ( Figure 9 and Figure 11) varies from one laser resonator to another, making it difficult for anyone other than the manufacturer to replicate. Therefore, validity can be determined by observing the rising edge waveform of the laser.

[0243] <3-2. First Example of Validity Determination Processing>

[0244] Will refer to Figures 12 to 17 , a first example of the validity determination process is described with reference to each of .

[0245] In the first example, reference conditions corresponding to the time waveform of light intensity during emission are pre-stored in the storage unit 6, and measurement data corresponding to the time waveform of light intensity during reception are compared with the reference conditions to confirm the correctness of the light reception waveform.

[0246] Flowchart as Figure 12 shown.

[0247] The control unit 4 performs emission processing in step S101. As a result, the laser light source 2b emits laser light (first laser light or second laser light) in the first emission mode or the second emission mode. The emitted laser light becomes light reflected by the object and partially enters the imaging element 3b.

[0248] The control unit 4 performs light reception processing in step S102, and as a result, obtains measurement data of the light reception waveform.

[0249] In step S103, the control unit 4 performs branch processing for determining whether the light reception waveform is valid. Figure 13 ) is consistent with the light receiving waveform (for example, reference Figure 14 ) match (or feature match) to confirm validity. Specifically, this is done by confirming whether the ratio of the maximum light intensity of the received light waveform to the light intensity at the final stabilization time (i.e., the steady-state value) is substantially the same, or by comparing the time required for the waveform to rise and fall. Alternatively, the determination can be performed by comparing the time from rise to steady-state value.

[0250] If the light reception waveform is determined to be valid, the control unit 4 performs normal processing in step S104. In normal processing, the desired processing using or on the range image is performed. In other words, since the light reception waveform is valid, it can be determined that the acquired range image is correct. Therefore, various types of processing can be performed normally using the range image.

[0251] On the other hand, if it is determined in step S103 that the light reception waveform is invalid, the control unit 4 performs abnormal time processing in step S105. In the abnormal time processing, for example, the validity can be re-determined by executing each process in steps S101 to S103. This is effective in cases where the validity cannot be confirmed due to the accidental entry of light noise from the outside. In addition, a backup process for functional safety can be performed, and in the case of a fail-safe function, a process that prioritizes the fail-safe function can be performed, or a process that separates information based on self-luminescence from information based on light emitted by other objects can be performed. In addition, an abnormality notification process can be performed for the user of the imaging device 1, and the position and rotation angle of a crosstalker (including a disturber) or an attacker (including a disturber) can be specified by specifying the position and rotation angle of an invalid light source, or supplementary processing can be performed in the presence of other measurement methods, other distance measurement systems, and other distance measurement devices. However, these crosstalkers and attackers include not only humans, but also non-humans such as other devices or nature. For example, light emitted by other devices (including light sources (cameras, mobile phones, smartphones, tablets, personal computers, game consoles, televisions, monitors, electronic devices, mobile devices, cars, mobile objects, drones, flying objects, robots, mobile objects, etc.), the sun, etc.), or light emitted by its objects (such as natural light), is used to take measures to deal with erroneous measurements of the self-luminescence of the imaging device 1 (including misjudgment, failure, and unmeasurable conditions caused thereby).

[0252] Note that an example of a light reception waveform determined to be invalid in the determination processing in step S103 is shown in FIG. Figures 15 to 17 Shown in.

[0253] Because the length of time required from the rise to the fall of the light receiving waveform does not match the reference condition, Figure 15 Determined to be invalid.

[0254] Since there is no peak shape of the rising waveform, Figure 16 Alternatively, it may be determined to be invalid because the ratio of the maximum value to the steady-state value of the light intensity is different from the reference condition.

[0255] Since the peak exists outside the rising part, Figure 17 Determined to be invalid.

[0256] Note that as the reference conditions stored in the storage unit 6, it is desirable to store information using the laser light source 2b in a calibrated state. Furthermore, calibration may be performed periodically in consideration of temporal variations of the laser resonator, and the stored reference conditions may be updated based on the calibration.

[0257] <3-3. Second Example of Validity Determination Processing>

[0258] In the second example of the effectiveness determination process, the imaging device 1A includes the light reflection unit 32. The block diagram of the imaging device 1A in this example is shown in FIG. Figure 18 Shown in.

[0259] The imaging device 1A includes, in addition to the irradiation unit 2, the imaging unit 3, the control unit 4, the display unit 5, and the storage unit 6, a light reflecting unit 32 that reflects at least a portion of the light emitted from the irradiation unit 2. The light emitted from the irradiation unit 2 and reflected by the light reflecting unit 32 enters the imaging element 3b.

[0260] The flowchart of the processing example in this example is Figure 19 Note that Figure 12 Similar processing to the processing shown in the flowchart is denoted by the same reference numerals, and description thereof will be appropriately omitted.

[0261] The control unit 4 performs emission processing in step S101. As a result, the irradiation unit 2 emits light.

[0262] Light from the irradiation unit 2 and reflected by the light reflection unit 32 (hereinafter referred to as “internally reflected light”) reaches the imaging element 3 b earlier than light reflected by the objects 100 and 101 (hereinafter referred to as “externally reflected light”).

[0263] Therefore, the light reception process of the internally reflected light is performed first, and then the light reception process of the externally reflected light is performed.

[0264] That is, in step S111, the control unit 4 acquires the reference condition by executing the first light reception process as the light reception process of the internally reflected light. The reference condition is stored in the storage unit 6, for example.

[0265] Subsequently, in step S112 , the control unit 4 acquires measurement data by performing a second light reception process as a light reception process of external reflected light.

[0266] In step S103 , the control unit 4 performs branching processing for determining whether the light reception waveform is valid by comparing the reference condition with the measurement data.

[0267] In the case of determining that the light reception waveform is valid, the control unit 4 performs normal time processing in step S104 , and in the case of determining that the light reception waveform is invalid, the control unit 4 performs abnormal time processing in step S105 .

[0268] A modification of the second example of the validity determination process will be described.

[0269] In the modification, the transmit process is performed twice.

[0270] Specific reference Figure 20 The flowchart is described.

[0271] The control unit 4 performs a first transmission process in step S121 and performs a first light reception process in step S111. As a result, the reference condition is acquired and stored.

[0272] The control unit 4 performs the second transmission process in step S122 and performs the second light reception process in step S112. As a result, measurement data is acquired.

[0273] In step S103 , the control unit 4 performs branching processing for determining whether the light reception waveform is valid by comparing the reference condition with the measurement data.

[0274] In the case where it is determined that the light reception waveform is valid, the control unit 4 performs normal time processing in step S104 , and in the case where it is determined that the light reception waveform is invalid, the control unit 4 performs abnormal time processing in step S105 .

[0275] By performing the transmission process twice, the accuracy of the validity determination process can be improved. In addition, erroneous measurements by crosstalkers can be reduced and security against malicious attackers who attempt to invalidate the validity determination process can be enhanced.

[0276] Note that the control unit 4 may perform the first emission process under the first emission condition in step S121, and may perform the second emission process under the second emission condition that is at least partially different from the first emission condition in step S122. By changing the emission conditions in the first emission process and the second emission process, it becomes more difficult to imitate the irradiation unit 2.

[0277] The first emission condition and the second emission condition are different conditions such as the length of light emission time, light intensity, etc.

[0278] Furthermore, a light guide unit including a light guide path may be included instead of the light reflecting unit. Similar effects can be obtained using the light guide path.

[0279] Note that, in the modification, the processing may be performed by the imaging device 1 that does not include the light reflecting unit 32 or may be performed by the imaging device 1A that includes the light reflecting unit 32 .

[0280] Furthermore, the irradiation unit 2 may include a plurality of laser light sources, and the laser light source that emits in the first emission process may be different from the laser light source that emits in the second emission process.

[0281] <3-4. Third Example of Validity Determination Processing>

[0282] In the third example of the effectiveness determination process, effectiveness is determined by emitting laser light having different optical waveforms.

[0283] As the current flowing into laser light source 2b, a semiconductor laser, gradually increases, the light output of laser light source 2b gradually increases. However, until the light output exceeds the current value at the start of oscillation, LED light, rather than laser light, is emitted. When the current value at the start of laser oscillation exceeds the current value at the start of oscillation, the light output increases rapidly, and laser oscillation begins. The output changes rapidly with respect to the current. In other words, lower limits are set for the light output (light intensity) and the operating current.

[0284] On the other hand, an upper limit value is set for light output (light intensity) from the viewpoint of safety of laser products and the like according to various standards, regulations based on the operating environment, and the like.

[0285] When the laser light source 2b emits light using two or more current values within the range between the lower limit and the upper limit (i.e., in the first and second emission modes described above), the validity of the received light waveform can be confirmed based on rising waveforms having different shapes. This further reduces erroneous measurements caused by crosstalk and makes it more difficult for malicious attackers, etc., to replicate the laser light.

[0286] exist Figures 21 to 23 Examples are shown in each of .

[0287] Figure 21 An example of an emission waveform is shown when the laser light source 2b emits multiple times according to the first emission mode. The laser light source 2b emits at a constant repetition period T0. The length of time when the laser light source 2b emits light (emission period T1) is set to be constant.

[0288] Figure 22 ] is an example of an emission waveform in which the first laser light and the second laser light are repeatedly and alternately emitted.

[0289] The emission of the first light emitting pattern and the emission of the second light emitting pattern are repeated at a repetition period T0. Both the first light emitting pattern and the second light emitting pattern are included in the light emitting period T1.

[0290] Figure 23 is an emission waveform in the case where the first laser light and the second laser light are emitted in random order.

[0291] The emission of both modes is performed with a constant repetition period T0 and a light emission period T1.

[0292] Note that the lighting period T1 and the non-lighting period (T0-T1) can be modulated or set to random periods. Furthermore, in addition to the first and second lighting modes, a third lighting mode different from the first and second lighting modes can be used. In the third lighting mode, a current different from the current applied to laser light source 2b in the first lighting mode and the current applied to laser light source 2b in the second lighting mode is applied to laser light source 2b.

[0293] In addition, the laser light source 2b can emit by setting the current applied to the laser light source 2b to a random value within the above range, instead of setting the current applied to the laser light source 2b to a predetermined value as in the first light-emitting mode and the second light-emitting mode.

[0294] By having such a variety of patterns, the laser can be made more difficult to replicate.

[0295] Note that, regarding the switching between the first laser light and the second laser light, it suffices that the above-mentioned transmission signal includes a first transmission signal (first transmission code) and a second transmission signal (second transmission code).

[0296] Figure 24 Examples of first and second transmission signals and transmission waveforms of first and second laser beams emitted in response to the signals are shown.

[0297] For example, in the case where the light emitted from the laser light source 2b is switched between the first laser light and the second laser light, it is sufficient that the imaging device 1 includes a switching unit. In addition, the switching unit may be provided in the irradiation unit 2 or the irradiation control unit 2a.

[0298] <3-5. Fourth Example of Validity Determination Processing>

[0299] The fourth example of the effectiveness determination process is an example in which the unit pixel 16 of the imaging element 3b includes four or more charge accumulation units (FD). Specifically, reference will be made to Figure 25 Provide a description.

[0300] Figure 25 is a diagram showing some cells extracted from cells included in the unit pixel 16 .

[0301] As shown in the figure, each photoelectric conversion element 17 included in the unit pixel 16 is connected to a first transfer transistor 18 a , a second transfer transistor 18 b , . . . , and an n-th transfer transistor 18 n .

[0302] Each transfer transistor 18 is connected to the FD 20. Specifically, the first transfer transistor 18a is connected to the first FD 20a, and the second transfer transistor 18b is connected to the second FD 20b. Similarly, the n-th transfer transistor 18n is connected to the n-th FD 20n.

[0303] In each FD 20, the charge generated by the photoelectric conversion element 17 is accumulated in the corresponding charge accumulation period. The charge accumulation period corresponding to each FD 20 is set so as not to overlap.

[0304] Figure 26 is a diagram showing the amount of charge accumulated in each charge accumulation period in the case where a single unit pixel 16 includes eight FDs 20 to grasp the characteristic amount of a light reception waveform.

[0305] As shown in the figure, the repetition period T0 is divided into eight periods from a first charge accumulation period Ta to an eighth charge accumulation period Th. The charge accumulation periods are set to periods having equal lengths.

[0306] The first charge accumulation period Ta begins simultaneously with the emission of light from the laser light source 2b. In the illustrated example, during the first charge accumulation period Ta, the light emitted from the laser light source 2b has not yet reached the imaging element 3b, and the intensity of the received light is zero. Furthermore, the second to fifth charge accumulation periods Tb to Te are included in the period during which light emitted from the laser light source 2b is received (light reception period T2).

[0307] like Figure 25 and Figure 26 As shown, in the case where the photoelectric conversion element 17 is connected to a plurality of FDs 20 , it is possible to grasp the characteristic amount of the rising waveform of the light reception waveform.

[0308] Note that in order to measure the amount of charge of light received in each of the first and second halves of the light emission period T1 having a time length half that of the repetition period T0, it is necessary to divide the repetition period T0 into four periods as a charge accumulation period. That is, it is preferable that the photoelectric conversion element 17 is connected to four FDs 20 (a first FD 20a, a second FD 20b, a third FD 20c, and a fourth FD 20d).

[0309] Note that although the charge accumulation time periods are equal to each other, the charge accumulation time periods may not be equal.

[0310] By connecting more FDs 20 to the photoelectric conversion element 17 , the feature amount of the light reception waveform can be extracted in more detail.

[0311] Examples of light reception waveforms and the amount of charge accumulated in each charge accumulation period are shown in FIG. Figure 27 shown.

[0312] The emission waveform of the laser light source 2b may include overshoot, undershoot, ringing, etc. in addition to rising and falling shapes. In this case, by connecting more FDs 20 to the photoelectric conversion element 17 as shown in the figure, the characteristics of the waveform can be grasped.

[0313] A method for generating a distance image using the configuration in this example will be described. Figure 28 is a diagram showing the relationship among the emission signal, the reference signal, and the charge amount for each charge accumulation period. Figure 28 The illustrated example has a configuration in which the photoelectric conversion element 17 is connected to 16 FDs 20 .

[0314] As shown in the figure, the rising moment of the emission signal coincides with the rising moment of reference signal a1. Reference signal a2 rises simultaneously with the falling moment of reference signal a1. In this way, pulses of reference signals a1 through a8 are generated. By selecting a charge accumulation unit corresponding to each reference signal, the charge generated by photoelectric conversion element 17 is accumulated during the pulse output period.

[0315] At the same time as reference signal a8 falls, reference signal b1 rises. After reference signal b1, pulses of reference signals b2 to b8 are generated so as not to overlap.

[0316] The time period from the rise of the reference signal a1 to the fall of the reference signal a8 (first time period) is substantially the same as the pulse width T. In addition, the time period from the rise of the reference signal b1 to the fall of the reference signal b8 (second time period) is substantially the same as the pulse width T.

[0317] Since the light emitted from the laser light source 2b reaches the imaging element 3b as reflected light from the object, a delay time Δt is generated from the rise of the emission signal to the start of light reception. Therefore, the imaging element 3b can perform light reception operation in both the first period and the second period.

[0318] In the case where reflected light is received during the first time period and the second time period, the distance to the object can be measured.

[0319] The pixel signal Sa is proportional to the total amount Qa of the charge accumulated in each FD 20 during the first period. Furthermore, the pixel signal Sb is proportional to the total amount Qb of the charge accumulated in each FD 20 during the second period. That is, the distance to the object can be calculated using the value calculated by Qb / (Qa+Qb).

[0320] Note that the distance to the object can be calculated without using the sum Qa and the sum Qb. For example, based on the charge amounts in time periods other than the first and second time periods, the distance can be calculated using the charge amount obtained by correcting the charge amount in at least a portion of the first time period (e.g., the charge amount in the rising portion) and the charge amount in at least a portion of the second time period (e.g., the charge amount in the falling portion). This calculation can sometimes be used to correct errors in distance calculation.

[0321] <3-6. Fifth Example of Validity Determination Processing>

[0322] The fifth example of the effectiveness determination process is an example in which the light emission period is changed for each emission of the laser light source 2 b .

[0323] Specific examples include Figure 29 shown.

[0324] In this example, the transmit signal includes three types of pulses with different pulse widths. Specifically, the transmit signal includes a first pulse P1 with a large pulse width, a second pulse P2 with a medium pulse width, and a third pulse P3 with a small pulse width.

[0325] The light emitting period based on the first pulse P1 is set as the light emitting period T3. The light emitting period based on the second pulse P2 is set as the light emitting period T4. The light emitting period based on the third pulse P3 is set as the light emitting period T5.

[0326] The first reference signal group A obtained by adding the first reference signals a respectively corresponding to the pulses in the transmission signal is a signal substantially the same as the transmission signal.

[0327] Furthermore, the second reference signal group B obtained by adding the second reference signals b respectively corresponding to the pulses in the transmission signal includes a pulse temporally adjacent to each pulse in the first reference signal group A.

[0328] The time length between pulses in the emission signal, that is, the non-light-emitting period has the same time period length. As a result, the emission interval of the laser light source 2b, which is the time length from the rise of a pulse to the rise of the next pulse, is not constant.

[0329] By making the laser light source 2b's lighting period different for each shot, distance calculation needs to be based on the lighting period. However, even without comparing the emission waveform and the rising waveform of the light reception waveform, the effectiveness of the laser light source 2b can be determined using only the lighting period, thus simplifying the process.

[0330] Note that by performing comparison between the rising waveforms of the emission waveform and the light reception waveform and comparison of the lengths of the light emission periods, it is possible to make imitation of laser light more difficult.

[0331] Note that by repeating Figure 29 The reference signal shown in can complicate the simulation of laser light while suppressing the increase in the amount of information required to generate the reference signal.

[0332] <3-7. Sixth Example of Validity Determination Processing>

[0333] In the sixth example of the effectiveness determination process, the non-light emitting period, which is the period from the fall of the emission pulse of the laser light source 2 b to the rise of the next emission pulse, is randomly determined.

[0334] Specifically, reference will be made to Figure 30 Provide a description.

[0335] The pulse width of each pulse in the emission signal is the same, so each emission cycle has the same duration.

[0336] Furthermore, the time periods between the pulses are different from each other, and therefore the non-light emitting time periods are different.

[0337] Therefore, the transmission interval is not constant.

[0338] Since the calculation formula for distance measurement is set to be the same by setting a constant lighting period, the process can be simplified.

[0339] In addition, by making the non-luminous time period of the laser light source 2b different each time it is emitted, the effectiveness of the laser light source 2b can be judged only by the non-luminous period even without comparing the emission waveform and the rising waveform of the light receiving waveform, thereby simplifying the processing.

[0340] Note that by making a comparison between the rising waveforms of the emission waveform and the light reception waveform and comparing the length of the non-light emitting period, it is possible to make imitation of laser light more difficult.

[0341] As a modification of the sixth example of the validity determination process, reference will be made to Figure 31 An example is described in which both the light emission period and the non-light emission period are variable while the emission interval is set to be constant.

[0342] As shown in the figure, in this modification, the emission interval is constant. Therefore, the ratio of the light-emitting period and the non-light-emitting period represented in the emission interval is variable for each emission.

[0343] Even in this mode, it can be made difficult to imitate a laser.

[0344] Note that by detecting the time ratio between the luminous period and the non-luminous period, the effectiveness of the laser light source 2b can be determined without comparing the emission waveform with the rising waveform of the light receiving waveform. In addition, by comparing the emission waveform with the rising waveform of the light receiving waveform, it is possible to make the simulation of the laser more difficult.

[0345] <3-8. Seventh Example of Validity Determination Processing>

[0346] In the seventh example of the effectiveness determination process, the unit pixel 16 of the imaging element 3 b includes a first unit pixel 16A and a second unit pixel 16B. This will be described in detail.

[0347] like Figure 25 As shown, the first unit pixel 16A includes one first photoelectric conversion element 17A and four or more FDs 20 .

[0348] like Figure 3 and Figure 4 As shown, the second unit pixel 16B includes one second photoelectric conversion element 17B and two FDs 20 .

[0349] The number of first unit pixels 16A including the first photoelectric conversion elements 17A is set to be smaller than the number of second unit pixels 16B including the second photoelectric conversion elements 17B.

[0350] As a result, the number of FDs 20 included in the imaging element 3 b is reduced, which reduces the cost and size of the imaging element 3 b.

[0351] An example of arrangement of the first photoelectric conversion element 17A and the second photoelectric conversion element 17B is shown in FIG. Figure 32 and Figure 33 Shown in.

[0352] Figure 32 The example shown is an example in which the first photoelectric conversion element 17A is arranged in the outer edge portion 33 of the imaging element 3b and the second photoelectric conversion element 17B is arranged inside. That is, the first photoelectric conversion element 17A is arranged only at the outermost portion.

[0353] As a result, the number of first photoelectric conversion elements 17A can be reduced. Therefore, compared with the case where the first photoelectric conversion elements 17A are arranged in all the photoelectric conversion elements 17, the number of FDs 20 can be significantly reduced.

[0354] Figure 33 The illustrated example is an example in which the first photoelectric conversion elements 17A are arranged at the four corners of the imaging element 3 b and the second photoelectric conversion elements 17B are arranged in portions other than the four corners.

[0355] As a result, because the number of the first photoelectric conversion elements 17A can be further reduced, the number of the FDs 20 can be reduced more significantly.

[0356] Notice, Figure 32 and Figure 33 The examples shown in are merely examples. As long as at least one first photoelectric conversion element 17A is arranged, the various types of effectiveness determination processing described above can be performed.

[0357] Since the other photoelectric conversion elements 17 are used as the photoelectric conversion elements 17 for distance measurement, it is sufficient to arrange the second photoelectric conversion elements 17B to which a smaller number of FDs 20 are connected.

[0358] In addition, if Figure 32 and Figure 33 As shown, by arranging more second photoelectric conversion elements 17B than first photoelectric conversion elements 17A, it is possible to contribute to cost reduction.

[0359] Furthermore, by arranging the first photoelectric conversion element 17A in the outer edge portion 33 of the imaging element 3b, the second photoelectric conversion element 17B, which serves as the photoelectric conversion element 17 for distance measurement, can be densely arranged in the central portion. Therefore, the distance measurement function and the validity determination function can be realized without increasing the size of the imaging element 3b. Note that the configuration in which the second unit pixel 16B includes a single second photoelectric conversion element 17B and two FDs 20 has been described. However, the second unit pixel 16B may include a single second photoelectric conversion element 17B and three or more (e.g., four or eight) FDs 20.

[0360] <3-9. Eighth Example of Validity Determination Processing>

[0361] In the eighth example of the effectiveness determination process, effectiveness is determined and a distance measurement process is performed.

[0362] Specifically, the description will be made with reference to the accompanying drawings.

[0363] Figure 34 This is the first example of the eighth example of the validity determination process. In this example, the distance measurement process and the validity determination process are performed based on one light receiving process. Note that Figure 12 Similar processing to the processing shown in the flowchart is denoted by the same reference numerals, and description thereof will be appropriately omitted.

[0364] The control unit 4 performs emission processing in step S101. As a result, the irradiation unit 2 emits light.

[0365] The control unit 4 performs light reception processing in step S131. In the light reception processing, data for distance measurement is acquired based on the amount of charge converted by the second photoelectric conversion element 17B. In addition, data for validity determination is acquired based on the amount of charge converted by the first photoelectric conversion element 17A.

[0366] In step S132, the control unit 4 performs distance measurement processing and generates a distance image. That is, distance measurement using the pixel signals Sa and Sb is performed for each pixel. Note that, as described above, distance measurement using the first photoelectric conversion element 17A can also be performed. That is, distance measurement can be performed using the sum Qa of the amount of charge accumulated in each FD 20 in the first time period and the sum Qb of the amount of charge accumulated in each FD 20 in the second time period. As a result, for example, even in the case of Figure 32 In the case where the photoelectric conversion element 17 is arranged in the image sensor 3 b, distance measurement can also be performed on all pixels of the imaging element 3 b.

[0367] In step S103 , the control unit 4 determines whether the light reception waveform is valid.

[0368] In the case where it is determined that the light reception waveform is valid, the control unit 4 performs normal time processing in step S104 , and in the case where it is determined that the light reception waveform is invalid, the control unit 4 performs abnormal time processing in step S105 .

[0369] Figure 35 This is the second example of the eighth example of the validity determination process. In this example, the emission process and the light reception process are performed twice. Note that Figure 12 Similar processing to the processing shown in the flowchart is denoted by the same reference numerals, and description thereof will be appropriately omitted.

[0370] The control unit 4 performs transmission processing for distance measurement in step S141 to perform transmission for distance measurement, and performs light reception processing for distance measurement in step S142. As a result, data for distance measurement is acquired based on the amount of charge converted by the second photoelectric conversion element 17B.

[0371] In step S132 , the control unit 4 performs distance measurement processing and generates a distance image.

[0372] The control unit 4 performs a transmission process for determination in step S143 to acquire information for effectiveness determination, and performs a light reception process for determination in step S144. As a result, data for effectiveness determination is acquired.

[0373] In step S103 , the control unit 4 determines whether the light reception waveform is valid.

[0374] In the case of determining that the light reception waveform is valid, the control unit 4 performs normal time processing in step S104 , and in the case of determining that the light reception waveform is invalid, the control unit 4 performs abnormal time processing in step S105 .

[0375] In this example, the first photoelectric conversion element 17A may be configured to generate only data for effectiveness determination, or may be configured to generate data for effectiveness determination and data for distance measurement.

[0376] Figure 36 This is the third example of the eighth example of the validity determination process. This example is an example of performing distance measurement after confirming validity.

[0377] The control unit 4 performs transmission processing in step S101 and light reception processing in step S131. As a result, data for validity determination is acquired based on the charge amount converted by the first photoelectric conversion element 17A, and data for distance measurement is acquired based on the charge amount converted by the second photoelectric conversion element 17B.

[0378] In step S103, the control unit 4 determines whether the light reception waveform is valid. In the case where it is determined that the light reception waveform is valid, the control unit 4 generates a distance image by performing a distance measurement process in step S132.

[0379] On the other hand, in the case where it is determined that the light reception waveform is invalid, the control unit 4 ends Figure 36 A series of processing shown.

[0380] Note that if the light reception waveform is determined to be valid, the control unit 4 performs distance measurement processing in step S132 and normal time processing in step S104. In addition, if the light reception waveform is determined to be invalid, the control unit may perform abnormal time processing in step S105.

[0381] According to this example, validity is determined before distance measurement processing. Therefore, if validity cannot be confirmed, distance measurement processing is not performed, thereby reducing the amount of computational processing. In other words, because the processing can be performed efficiently, it can help reduce power consumption.

[0382] Figure 37 This is the fourth example of the eighth example of the validity determination process. This example is an example of performing distance measurement after confirming validity.

[0383] The control unit 4 performs a transmission process for determination in step S143 and performs a light reception process for determination in step S144. Subsequently, the control unit 4 determines whether the light reception waveform is valid in step S103.

[0384] If the light reception waveform is determined to be valid, the control unit 4 performs transmission processing for distance measurement in step S141 to perform transmission for distance measurement, and performs light reception processing for distance measurement in step S142. As a result, data for distance measurement is obtained based on the amount of charge converted by the second photoelectric conversion element 17B.

[0385] Subsequently, the control unit 4 performs distance measurement processing to generate a distance image in step S132 .

[0386] On the other hand, in the case where it is determined that the light reception waveform is invalid, the control unit 4 ends Figure 37 A series of processing shown.

[0387] In this example, as in the third example, validity is determined before distance measurement. Therefore, if validity cannot be confirmed, distance measurement is not performed, reducing the amount of computational processing. This allows for efficient processing, contributing to reduced power consumption.

[0388] In each of the eighth examples of the above-described validity determination processing, validity determination may be periodically confirmed by repeatedly executing a series of processing at certain time intervals, and the series of processing may be repeatedly executed irregularly.

[0389] Furthermore, the series of processing may be executed each time an execution instruction is received.

[0390] Furthermore, the distance measurement process and the validity determination process may be performed simultaneously or may be performed in a time-division manner.

[0391] When the processes are executed simultaneously, the number of processes that can be processed within a predetermined time increases, thereby improving the accuracy and speed of distance measurement results.

[0392] Furthermore, when processing is performed in a time-division manner, the processing can be performed so that the execution times partially overlap, or the processing can be performed so that the execution times do not overlap. By performing the processing so that the execution times do not overlap, the concentration of the computational processing can be reduced. Consequently, the maximum power consumption can be reduced.

[0393] <3-10. Ninth Example of Validity Determination Processing>

[0394] The ninth example of the effectiveness determination process is an example of the effectiveness determination process performed by the imaging device 1B including the pattern generation unit.

[0395] Figure 38, an example of the configuration of an imaging device 1B is shown in FIG. The imaging device 1B is, for example, an optical three-dimensional camera. The imaging device 1B includes an irradiation unit 2B, an imaging unit 3B, a control unit 4, a display unit 5, a storage unit 6, an oscillator 34, and a distributor 35.

[0396] In addition to the irradiation control unit 2a and the laser light source 2b, the irradiation unit 2B also includes a pattern generation unit 36. Note that the irradiation unit 2B may also include a lens system through which the light emitted from the laser light source 2b passes. Note that in the above-described irradiation unit 2, a lens system may be provided through which the light emitted from the pattern generation unit 36 and the laser light source 2b passes.

[0397] The imaging unit 3B includes a phase-sensitive detector 37 and a multiplexer 38 in addition to the lens 3 a , the imaging element 3 b , and the signal processing circuit 3Bc.

[0398] The oscillator 34 outputs a predetermined modulation signal (eg, a sine wave signal, a pulse wave signal, a rectangular wave signal, a sawtooth wave signal, a triangle wave signal, etc.) The modulation signal output from the oscillator 34 is input to the distributor 35 .

[0399] The distributor 35 splits the input modulated signal into two paths and outputs the respective signals to the laser light source unit 2 b and the phase sensitive detector 37 .

[0400] The laser light source 2b emits laser light whose intensity is modulated according to an input modulation signal.

[0401] For example, the pattern generating unit 36 diffuses the light emitted from the laser light source 2 b to generate a predetermined emission intensity pattern.

[0402] The generated predetermined intensity pattern is used as emission light with a predetermined spread angle to illuminate the objects 100 and 101. The predetermined spread angle here is, for example, the angle of the emission area within the instantaneous field of view of the imaging element 3b.

[0403] The imaging element 3b receives incident light entering via the lens 3a and performs photoelectric conversion for each pixel based on control of the control unit 4. For each pixel, a signal that has been photoelectrically converted is a reception signal and is output to the phase-sensitive detector 37.

[0404] The phase-sensitive detector 37 has a structure in which elements are arranged in a two-dimensional array in the row and column directions, and each element corresponds to each pixel of the imaging element 3 b. In other words, the reception signal output from a certain pixel of the imaging element 3 b is input to one element of the phase-sensitive detector 37.

[0405] The phase-sensitive detector 37 performs phase detection using the modulated signal input from the distributor 35. Each element of the phase-sensitive detector 37 outputs a signal (for example, a complex amplitude signal) as a result of the phase detection to the multiplexer 38.

[0406] The multiplexer 38 multiplexes the signals received as a result of phase detection from the respective elements to form one output in which the signals are sequentially arranged. The multiplexed signal formed into one signal is output to the signal processing circuit 3Bc.

[0407] The configuration example of the signal processing circuit 3Bc is Figure 39 Shown in.

[0408] The signal processing circuit 3Bc includes an intensity detection unit 39 , a first distance measurement unit 40 , a second distance measurement unit 41 , a calculation unit 42 , and an image output unit 43 .

[0409] The signal processing circuit 3Bc processes the multiplexed signal from the multiplexer 38 and calculates and outputs a three-dimensional shape.

[0410] Intensity detection unit 39 detects the signal intensity of each element of phase-sensitive detector 37 from the multiplexed signal received from multiplexer 38. For elements (pixels) that receive more emitted light from laser light source unit 2b, the signal intensity of each element detected by intensity detection unit 39 increases, and its intensity is modulated based on a predetermined modulation signal. The signal intensity information for each element obtained by intensity detection unit 39 is input to first distance measurement unit 40 and second distance measurement unit 41 in the next stage.

[0411] The first distance measurement unit 40 uses the received signal strength information to perform distance measurement using triangulation. Distance measurement is performed for each element of the phase-sensitive detector 37, and also for each pixel of the imaging element 3b. Specifically, by measuring the change in received intensity for each element (or pixel), the distance information to the object is calculated.

[0412] The first distance measurement unit 40 outputs three-dimensional shape information (3D information) to the calculation unit 42 based on the calculated distance information.

[0413] The second distance measuring section 41 performs distance measurement by the ToF method using the received signal strength information. Distance measurement is performed for each element of the phase-sensitive detector 37, and further, distance measurement is performed for each pixel of the imaging element 3b.

[0414] The second distance measurement unit 41 outputs three-dimensional shape information (3D information) to the calculation unit 42 based on the calculated distance information.

[0415] The calculation unit 42 selects one of the 3D information by triangulation or the 3D information by ToF method for each element of the phase-sensitive detector 37 (ie, for each pixel of the imaging element 3 b ), and outputs the selected 3D information to the image output unit 43 .

[0416] The image output unit 43 calculates a final 3-dimensional shape of the object based on the received 3D information of each element (each pixel) and outputs the result.

[0417] Notice, Figure 39 The configuration of the signal processing circuit 3Bc shown includes two distance measurement units (a first distance measurement unit 40 and a second distance measurement unit 41). However, if the signal processing circuit 3Bc performs distance measurement using the ToF method, it is not necessary to include the second distance measurement unit 41. In that case, the calculation unit 42 does not need to perform the selection process. In addition, it is sufficient for the image output unit 43 to calculate the three-dimensional shape based on the distance measurement results of the triangulation method.

[0418] For example, the imaging device 1B has Figure 38 and Figure 39 The illustrated configuration enables the irradiation unit 2B to output laser light having a dot pattern in which small dot-shaped light (spot light) beams are arranged in a predetermined pattern.

[0419] Specifically, Figure 40 Shown are examples of the shapes (light reception spot shapes) of the multiple spot light beams emitted from the irradiation unit 2B and received by the imaging element 3b when the multiple spot light beams directed toward the subject are imaged by the imaging unit 3B.

[0420] Figure 40 The optical paths of five point light beams emitted from irradiation unit 2B until they reach object 100 are shown. Furthermore, the point shapes formed by the point light beams on imaging element 3b when the point light beams projected onto object 100 are imaged by imaging unit 3B, i.e., the light receiving point shapes, are shown. The light receiving point shapes emitted to a position roughly corresponding to the front surface of irradiation unit 2B are perfectly circular. Furthermore, since the emission position is separated from the roughly front surface of irradiation unit 2B, the light receiving point shapes deviate from the perfect circle and become elliptical. However, with these shapes, there are cases where blurring, spots, etc. should be considered.

[0421] That is, the shape of the light receiving point of the light having a dot pattern emitted from the irradiation unit 2B becomes closer to a perfect circle as it gets closer to the center of the imaging range. This is because the positions of the irradiation unit 2B and the imaging unit 3B in space are substantially the same.

[0422] Figure 41The light receiving spot shape when light of a dot pattern emitted from a light source 200 other than the irradiation unit 2B toward the subject 100 is imaged by the imaging unit 3B is shown.

[0423] As shown in the figure, all the light receiving point shapes of the light having a dot pattern emitted from another light source are elliptical. In addition, the flattening of the ellipse is set to be smaller than Figure 40 Each light receiving point shown in the shape is higher (closer to 1).

[0424] This is because the other light source 200 and the imaging unit 3B are located apart in space. That is, even if the other light source 200 at a location separate from the imaging device 1B emits a dot pattern toward the subject, it can be determined that the other light source 200 is not an effective light source by detecting the light receiving point shape.

[0425] It is effective to match or substantially match the emitting range of the irradiation unit 2B and the imageable range of the imaging unit 3B.

[0426] That is, at least a portion of the irradiation unit 2B (for example, any one of the optical systems such as the laser light source 2b, the pattern generation unit 36, and the lens in front thereof) and at least a portion of the imaging unit 3B (for example, one of the imaging element 3b and the lens 3a) are arranged at the same or substantially the same angle relative to the object 100. As a result, at normal times, a light receiving spot shape having a substantially symmetrical shape (for example, a perfect circle) is detected by the imaging unit 3B.

[0427] Furthermore, the shape of the light-receiving point can be configured to be closer to a symmetrical shape as the position is closer to the center of the imageable range.

[0428] On the other hand, for example, in an abnormal situation, such as when another light source 200 emits light for the purpose of imitation, at least a portion of the imaging unit 3B and the other light source 200 generally have different arrangement angles relative to the object 100. Therefore, an asymmetrical light receiving point shape such as an elliptical shape is detected by the imaging unit 3B.

[0429] As a result, effectiveness determination processing based on the light-receiving spot shape of laser light can be performed.

[0430] Note that in Figure 40In the example shown, an example has been described in which the number of spot beams emitted from the irradiation unit 2B is five. However, this number may be more than five and, for example, may be as small as one. However, in the case where the number of spot beams is multiple, the validity determination process can be performed based on the changing trends of the shapes of the multiple light receiving points. On the other hand, in the case where the number of spot beams is one, by emitting at least a portion of the imageable range with a laser beam so that the laser beam draws a specific trajectory, the validity determination process can be performed based on the changing trends of the light receiving point shapes. For example, in some measurement environments where the spot beams are diffusely reflected, it is desirable to perform the validity determination process based on the changing trends of the light receiving point shapes. Of course, the basis for the validity determination process can be selected or switched.

[0431] Note that in this example, the shape of the spot beam emitted from the irradiation unit 2B is a perfect circle, but other shapes are conceivable. Figure 42 is a table showing various examples of spot beam shapes (spot shapes).

[0432] Dot shape A is the above-mentioned perfect circle. Dot shape B is a donut-like shape. In addition, various shapes are also considered, such as an ellipse (dot shape C), an ellipse (dot shape D), a shape in which multiple figures overlap (dot shape E), a shape including multiple non-overlapping figures (dot shape F), a triangle (dot shape G), a square (dot shape H), a rectangle (dot shape I), a trapezoid (dot shape J), a rhombus (dot shape K), a pentagon (dot shape L), a polygon with a hexagon or larger shape (dot shape M), a star (dot shape N), etc. However, at least a portion of these shapes may be circular, blurred, or have a shape that takes into account spots, etc.

[0433] A mirror-symmetrical shape or a point-symmetrical shape is desirable for easy manufacture of the laser light source 2b and the pattern generating unit 36. In particular, a perfect circle (spot shape A) and a ring shape (spot shape B) are more desirable.

[0434] Furthermore, considering that it is difficult to replicate laser light, a spot shape other than a perfect circle is desirable, and in particular, a non-mirror-symmetric spot shape or a non-point-symmetric spot shape is desirable.

[0435] For example, a beam shaping element can be used to shape the spot shape of the laser light.

[0436] Note that in order to confirm the effectiveness of the laser light source 2b based on the light receiving point shape of the laser, for example, considering the detection of two or more shape changes of the point shape, the area of the emission range is preferably at least equal to or less than 25% of the area of the imageable range.

[0437] <3-11. Tenth Example of Validity Determination Processing>

[0438] The tenth example of the effectiveness determination process is an example in which the irradiation unit 2B emits laser light having a dot pattern in which a plurality of spot beams are arranged at predetermined positions.

[0439] Figures 43 to 45 Each of φ and φ shows an example of a dot pattern. In addition, the dot pattern shown in each figure is a part of the dot pattern (emission dot pattern) emitted from the irradiation unit 2B.

[0440] Figure 43 A dot pattern for distance measurement using triangulation is shown. Specifically, a dot pattern in which spot beams are irregularly arranged is shown.

[0441] Figure 44 A regularly arranged dot pattern of spot beams. This dot pattern can be used to measure distances using triangulation.

[0442] Figure 45 An example is shown in which a dot pattern is formed by periodically repeating a set of spot beam groups including a plurality of spot beams.

[0443] For example, the irradiation unit 2B of the imaging device 1B includes a diffractive optical element so that the point beam group can be replicated.

[0444] In this case, a laser beam with a dot pattern consisting of a large number of spot beams can be generated with a simple structure. Furthermore, since the dot pattern is a repeated pattern of groups of spot beams, some distance measurement calculations can be performed collectively, shortening the time required for calculations and reducing the burden on the processing unit.

[0445] The dot pattern shown in each figure can be applied to distance measurement using the ToF method.

[0446] When the irradiation unit 2B is used Figure 44 When the dot pattern shown in FIG. 1 illuminates the object and the imaging unit 3 b images the object, the dot pattern (light receiving dot pattern) formed on the imaging element 3 b is as shown in FIG. Figure 46 shown.

[0447] The shown state represents a state in which the light-receiving spot shape is set within the area of one pixel.

[0448] also, Figure 47 An example of the light-receiving spot shape in the case where the ineffective another light source 200 emits a dot pattern is shown in FIG.

[0449] Since the light receiving point shape is an ellipse as shown in the figure, the light receiving point shape receives light not within the area of one pixel but over the area of two pixels.

[0450] like Figure 46 and Figure 47As shown, when the condition "total number of spot beams of laser light*4≤total number of pixels in the imaging element 3 b" is satisfied, the effectiveness of the laser light source can be confirmed based on the light receiving spot shape of the laser light.

[0451] Figure 48 An example of the processing executed by the control unit 4 in this example is shown in . Note that processing similar to that in each of the above-described flowcharts is denoted by the same reference numerals, and description thereof will be appropriately omitted.

[0452] The control unit 4 performs a transmission process in step S101 and a light receiving process in step S102. As a result, for example, the light source based on the light source is obtained. Figure 46 The light receiving dot pattern signal is shown.

[0453] In step S151 , the control unit 4 performs branching processing for determining whether the light-receiving point shape is valid.

[0454] In the case of determining that the light-receiving point shape is valid, the control unit 4 performs normal time processing in step S104 , and in the case of determining that the light-receiving point shape is invalid, the control unit 4 performs abnormal time processing in step S105 .

[0455] <3-12. Eleventh Example of Validity Determination Processing>

[0456] The eleventh example of the effectiveness determination process is an example in which determination is performed based on the number of light-receiving-point light beams.

[0457] Specifically, the effectiveness of the laser light source is confirmed by comparing the number of spot beams included in the emission spot pattern (the number of emission points) and the number of light-receiving spot beams included in the light-receiving spot pattern (the number of light-receiving points).

[0458] For example, Figure 49 An example of a light-receiving dot pattern in which the validity determination result is “invalid” is illustrated.

[0459] As shown in the figure, the imaging element 3b receives light of an invalid spot beam in addition to the effective spot beam.

[0460] In addition, similarly, Figure 50 The diagram illustrates an example of a light-receiving dot pattern in which the validity determination result is "invalid." As shown in the figure, if the number of light-receiving point beams included in the light-receiving dot pattern is at least one larger than the number of point beams included in the emission dot pattern, the validity determination result is "invalid."

[0461] Note that if the number of light-receiving points is smaller than the number of light-emitting points, it is possible that the reflected light from the object is too weak to be detected. Therefore, the validity determination result may be "valid." Note that in this case, the validity determination result can be determined based on the distance and arrangement of the light beams between the light-receiving points.

[0462] <3-13. Twelfth Example of Validity Determination Processing>

[0463] The twelfth example of the effectiveness determination process is an example of a case where the shape of the light receiving point is larger than the size of the unit pixel 16 of the imaging element 3b. Specifically, reference will be made to Figure 51 and Figure 52 Provide a description.

[0464] Figure 51 is an example in which a light beam at one light-receiving point is located within a range including 25 unit pixels 16. In this example, since the amount of charge accumulated in the unit pixel 16 located at the center is zero, the ring-shaped spot shape can be appropriately determined.

[0465] also, Figure 52 1 is an example in which a light beam at one light-receiving point is located within a range including 25 unit pixels 16. The light at the light-receiving point is based on light emitted from another ineffective light source 200.

[0466] Although the shape of the light receiving point is the same in all figures, Figure 52 The amount of charge accumulated in the unit pixel 16 located at the center is not zero. Therefore, it can be understood that the light at the light receiving point is different from the light at the light receiving point of the effective laser light source 2b.

[0467] Note that to appropriately determine the dot shape having a ring shape, it is sufficient to arrange other unit pixels 16 around a certain unit pixel 16. If the shape of unit pixel 16 is triangular or substantially triangular, then "the total number of laser spot beams * 4 ≤ the total number of pixels in imaging element 3b"; if the shape of unit pixel 16 is quadrilateral or substantially quadrilateral, then "the total number of laser spot beams * 5 ≤ the total number of pixels in imaging element 3b"; if the shape of unit pixel 16 is pentagonal or substantially pentagonal, then "the total number of laser spot beams * 6 ≤ the total number of pixels in imaging element 3b"; and if the shape of unit pixel 16 is hexagonal or substantially hexagonal, then "the total number of laser spot beams * 7 ≤ the total number of pixels in imaging element 3b" It is sufficient for at least a portion of the imaging range to satisfy these conditions.

[0468] In this way, by complicating the emission point shape, it is difficult to imitate the laser light, and the accuracy of the effectiveness determination process can be improved.

[0469] <3-14. Thirteenth Example of Validity Determination Processing>

[0470] The thirteenth example of the effectiveness determination process is an example in which effectiveness is determined based on a light-receiving dot pattern.

[0471] first, Figure 53 A light-receiving dot pattern is shown when an emission dot pattern emitted from the effective laser light source 2 b and emitted to the subject is imaged.

[0472] Next, Figure 54 1 shows a light receiving dot pattern when an emission dot pattern emitted from an invalid other light source 200 is imaged. This may include a case where reflected light is received when light emitted from an invalid other light source 200 is reflected by an object or a case where light directly entering from another light source 200 is received.

[0473] Can be compared Figure 53 and Figure 54 The effectiveness of the laser light source is determined by the light receiving dot pattern in the image.

[0474] Note that this validity determination can be performed when the spot beams are randomly arranged in a dot pattern and imaging unit 3B is aware of the random arrangement. In this example and other examples below, imaging unit 3B and irradiation unit 2B, which emits laser light, are provided in the same imaging device 1B, and imaging unit 3B is aware of the random dot pattern emitted by irradiation unit 2B. Therefore, the validity determination described above can be performed.

[0475] Note that Figure 55 As shown, in the case where the shapes of the light receiving points are different even though the light receiving points are correctly arranged, it is determined that invalid light is received.

[0476] <3-15. Fourteenth Example of Validity Determination Processing>

[0477] The fourteenth example of the effectiveness determination process is an example in the case where the laser light source 2 b is used as the other light source 200 .

[0478] Figure 53 A light reception dot pattern is shown when a subject is irradiated with light emitted from the laser light source 2 b included in the imaging device 1B via the pattern generating unit 36 and reflected light thereof is imaged by the imaging unit 3B.

[0479] also, Figure 56 The diagram shows a light receiving dot pattern in the case where an emission dot pattern emitted from another light source 200 is received by the imaging unit 3B of the imaging device 1B, in which a laser light source as another light source 200 is manufactured and positioned at a position substantially facing the imaging device 1B similarly to the laser light source 2b included in the imaging device 1B.

[0480] exist Figures 53 to 56 In any of the two cases, the dot pattern emitted is the same. However, Figure 53 and Figure 56 The light receiving point pattern in is mirror symmetrical.

[0481] In this case, you can Figure 56 The light receiving dot pattern shown confirms that the laser light source included in the other light source 200 is invalid, and the position of the other light source 200 can be roughly understood. That is, if a crosstalker or a malicious attacker uses the other light source 200, the position of the other light source 200 can be specified, and erroneous measurements and attacks can be dealt with.

[0482] Note that in the twelfth example of the above-mentioned validity determination process, when obtaining Figure 55 In the case of the light-receiving point pattern shown, the arrangement of the light-receiving points is correct, and therefore, it can be understood that the other light source 200 and the imaging device 1B are positioned in the same direction relative to the object. Since the light-receiving points are in the shape of a vertically elongated ellipse, it can be inferred that the imaging device 1B and the other light source 200 are positioned differently in the vertical direction.

[0483] Furthermore, even when the shape of the emission spot beam is asymmetrical, the emission position of another light source 200 can be understood from the light receiving point shape. Note that information on the emission position of another light source 200 can be applied to gaming applications or various applications.

[0484] <3-16. Fifteenth Example of Validity Determination Processing>

[0485] The fifteenth example of the effectiveness determination process is another example of a case where the laser light source 2 b is used as the other light source 200 .

[0486] When the object is irradiated with light emitted from the effective laser light source 2b included in the imaging device 1B via the pattern generating unit 36 and the reflected light is imaged by the imaging unit 3B, the light receiving dot pattern determined to be effective is Figure 53 Shown in.

[0487] on the other hand, Figure 57 It is a light receiving point pattern when using the laser light source 2b included in another imaging device AC as another light source 200, the object is irradiated with light emitted by the other laser light source 2b via the pattern generating unit 36 included in the other imaging device AC, and the reflected light is received by the imaging unit 3B of the imaging device 1B.

[0488] Compare Figure 53 and Figure 57 , Figure 57 The light receiving dot pattern shown is obtained by Figure 53 The dot pattern shown is a pattern obtained by rotating it by 180 degrees. Note that whether a pattern is a pattern rotated by 180 degrees can be determined by the asymmetry of the emitted dot pattern.

[0489] Upon detection Figure 57 In the case of the light receiving dot pattern shown, the laser light source is determined to be an invalid laser light source, and the position and posture of another imaging device AC including the laser light source 2b as another light source 200 can be estimated.

[0490] For example, Figure 58 As shown, the imaging device 1B is in a posture with its upper surface portion facing upward (sky) and its lower surface portion facing downward (ground), while the other imaging device AC is in a posture with its upper surface portion facing downward (ground) and its lower surface portion facing upward (sky). In this way, when the two imaging devices emit the same emission dot pattern in a state where the two imaging devices are rotated 180 degrees relative to each other, Figure 53 and Figure 57 The difference between the light receiving dot patterns is shown.

[0491] That is, in Figure 57 When imaging the light receiving dot pattern shown, the laser is found to be an invalid laser by another imaging device AC, and it is estimated that the other imaging device AC is in a posture rotated 180 degrees relative to the imaging device 1B, and it can be estimated that the other imaging device AC is located in the same direction as the imaging device 1B relative to the object 100.

[0492] <3-17. Example 16 of Validity Determination Processing>

[0493] The sixteenth example of the effectiveness determination process is yet another example of a case where the laser light source 2 b is used as another light source 200 .

[0494] When the object is irradiated with light emitted from the effective laser light source 2b included in the imaging device 1B via the pattern generating unit 36 and the reflected light is imaged by the imaging unit 3B, the light receiving dot pattern determined to be effective is Figure 53 Shown in.

[0495] Figure 59 It is a light receiving point pattern when using the laser light source 2b included in another imaging device AC as another light source 200, the object is irradiated with light emitted by the other laser light source 2b via the pattern generating unit 36 included in the other imaging device AC, and the reflected light is received by the imaging unit 3B of the imaging device 1B.

[0496] Compare Figure 53 and Figure 59 , Figure 59The light receiving dot pattern shown is obtained by moving Figure 53 The dot pattern shown is rotated approximately 15 degrees.

[0497] Upon detection Figure 59 In the case of the light reception dot pattern shown, it can be determined that the light reception dot pattern is not a dot pattern emitted from the effective laser light source 2 b included in the imaging device 1B.

[0498] Note that since the irradiation unit 2B and the imaging unit 3B are fixed to the outside or inside of the imaging device 1B, the irradiation unit 2B and the imaging unit 3B rotate according to the rotation of the imaging device 1B. Similarly, the irradiation unit (the other light source 200) included in the other imaging device AC rotates according to the rotation of the other imaging device AC.

[0499] Specifically, validity determination processing can be performed by extracting measurement data regarding the rotation angle of the light source performing emission based on the measurement data of the light receiving dot pattern acquired by imaging unit 3B and comparing the measurement data with reference conditions. Furthermore, by determining validity only when the measurement data matches or substantially matches the reference conditions, erroneous measurements by crosstalkers are reduced, and fraud by malicious attackers can be prevented. Furthermore, information regarding the rotation angle of another light source 200 can be applied to gaming applications and various other applications.

[0500] <3-18. Seventeenth Example of Validity Determination Processing>

[0501] The seventeenth example of the effectiveness determination process is an example of an imaging device 1C including an imaging unit 3C including an imaging element 3Cb using a single photon avalanche diode (SPAD).

[0502] Figure 60 An example of the configuration of the imaging element 3Cb is shown in FIG.

[0503] The imaging element 3Cb includes a pixel array unit 7C and a bias voltage applying unit 45 .

[0504] The pixel array unit 7C is a light receiving surface that receives light collected by the lens 3 a and in which a plurality of SPAD pixels 46 are arranged in a two-dimensional array in the row direction and the column direction.

[0505] like Figure 60 As enlarged and shown in FIG, the SPAD pixel 46 includes a SPAD element 47, a p-type metal oxide semiconductor field effect transistor (MOSFET) 48, and a CMOS inverter 49.

[0506] The SPAD element 47 forms an avalanche multiplication region by applying a large negative voltage VBD to the cathode and avalanche multiplies electrons generated by the incidence of one photon.

[0507] When the voltage caused by the electrons avalanche-multiplied by the SPAD element 47 reaches the negative voltage VBD, the p-type MOSFET 48 performs quenching to release the electrons multiplied by the SPAD element 47 and returns to the initial voltage.

[0508] The CMOS inverter 49 shapes the voltage generated by the electrons multiplied by the SPAD element 47 to output a light-receiving signal (APD OUT) having a pulse waveform starting from the arrival time of one photon.

[0509] The bias voltage applying unit 45 applies a bias voltage to each of the plurality of SPAD pixels 46 arranged in the pixel array unit 7C.

[0510] The imaging element 3Cb having such a configuration outputs the light reception signal of each SPAD pixel 46 to the signal processing circuit 3 c at the subsequent stage.

[0511] For example, the signal processing circuit 3 c performs calculation processing for obtaining the distance to the object based on the timing of generating a pulse indicating the arrival time of one photon in each light reception signal, and acquires distance measurement data for each SPAD pixel 46. Based on this distance measurement data, the signal processing circuit 3 c generates a distance image based on the distance to the object detected by the plurality of SPAD pixels 46.

[0512] Will refer to Figure 61 and Figure 62 A configuration example of a single SPAD pixel 46 and a peripheral portion is described.

[0513] Figure 61 is a cross-sectional view of a SPAD pixel 46 . Figure 62 is a plan view of the SPAD pixel 46.

[0514] The imaging element 3Cb has a stacked structure in which a sensor substrate 50, a sensor side wiring layer 51 and a logic side wiring layer 52 are stacked, and a logic circuit substrate (not shown) is stacked to the logic side wiring layer 52.

[0515] On the logic circuit substrate, for example, a bias voltage applying unit 45 , a p-type MOSFET 48 , a CMOS inverter 49 , and the like are formed.

[0516] For example, the sensor side wiring layer 51 can be formed on the sensor substrate 50, the logic side wiring layer 52 can be formed on the logic circuit substrate, and then the sensor side wiring layer 51 and the logic side wiring layer 52 can be bonded together. Figure 61 The imaging element 3Cb is manufactured by bonding the two surfaces (shown by the dotted line) together.

[0517] The sensor substrate 50 is a semiconductor substrate of, for example, thin-sliced single-crystal silicon, the p-type or n-type impurity concentration is controlled, and a SPAD element 47 is formed for each SPAD pixel 46 .

[0518] In addition, Figure 61 Among them, the surface opposite to the bonding surface with the sensor substrate 50 is a light receiving surface that receives light.

[0519] The sensor-side wiring layer 51 and the logic-side wiring layer 52 include wiring for supplying a voltage applied to the SPAD element 47 , wiring for extracting charges (electrons) generated by the SPAD element 47 from the sensor substrate 50 , and the like.

[0520] The SPAD element 47 includes an N well 53 , a P type diffusion layer 54 , an N type diffusion layer 55 , a hole accumulation layer 56 , a pinning layer 57 , and a high concentration P-type diffusion layer 58 , which are formed on a sensor substrate 50 .

[0521] In the SPAD element 47 , the avalanche multiplication region 59 is formed with a depletion layer formed in a region where the P-type diffusion layer 54 is connected to the N-type diffusion layer 55 .

[0522] The N-well 53 is formed by controlling the impurity concentration of the sensor substrate 50 to be n-type, and forms an electric field that transfers electrons generated by photoelectric conversion of the SPAD element 47 to the avalanche multiplication region 59 .

[0523] Note that, instead of the N-well 53 , a P-well may be formed by controlling the impurity concentration of the sensor substrate 50 to be p-type.

[0524] The P-type diffusion layer 54 is a dense P-type diffusion layer formed near the surface of the sensor substrate 50 and on the light-receiving side relative to the N-type diffusion layer 55 , and is formed across substantially the front surface of the SPAD element 47 .

[0525] The N-type diffusion layer 55 is a dense N-type diffusion layer formed near the surface of the sensor substrate 50 and formed on the bonding surface side relative to the P-type diffusion layer 54 , and is formed across substantially the front surface of the SPAD element 47 .

[0526] In order to be connected to a first contact electrode 63 (described later) to supply a negative voltage for forming the avalanche multiplication region 59 , the N-type diffusion layer 55 has a convex shape with a portion thereof formed to the surface of the sensor substrate 50 .

[0527] Hole accumulation layer 56 is a P-type diffusion layer formed to surround the side and bottom surfaces of N-well 53 and to accumulate holes. Hole accumulation layer 56 is electrically connected to the anode of SPAD element 47, enabling bias adjustment. Thus, the hole concentration in hole accumulation layer 56 is enhanced, and pinning by pinning layer 57 is strengthened, thereby preventing the generation of dark current, for example.

[0528] The pinning layer 57 is a dense P-type diffusion layer formed on the surface of the hole accumulation layer 56 on the light receiving face side and the adjacent SPAD element 47 side, and, for example, similarly to the hole accumulation layer 56 , prevents generation of dark current.

[0529] The high-concentration P-type diffusion layer 58 is a dense P-type diffusion layer formed near the surface of the sensor substrate 50 to surround the periphery of the N-well 53 and is used to connect to the second contact electrode 64 (described later) (which electrically connects the hole accumulation layer 56 to the anode of the SPAD element 47).

[0530] The avalanche multiplication region 59 is a high electric field region formed at the interface between the P-type diffusion layer 54 and the N-type diffusion layer 55 by applying a large negative charge to the N-type diffusion layer 55, and multiplies electrons generated by one photon entering the SPAD element 47. In addition, in the imaging element 3Cb, each SPAD element 47 is insulated and separated by an inter-pixel separation unit 62 (having a dual structure including a metal film 60 and an insulating film 61 formed between adjacent SPAD elements 47).

[0531] For example, the inter-pixel separation unit 62 is formed to penetrate from the back surface of the sensor substrate 50 to the front surface of the sensor substrate 50 .

[0532] The metal film 60 is a film containing a metal such as tungsten that reflects light, and the insulating film 61 is a film having insulating properties such as SiO 2 .

[0533] The inter-pixel separation unit 62 is formed by embedding the surface of the metal film 60 in the sensor substrate 50 to be covered with the insulating film 61 , for example, and electrically and optically separates the adjacent SPAD elements 47 .

[0534] In the sensor side wiring layer 51, a first contact electrode 63, a second contact electrode 64, a third contact electrode 65, a first metal wiring 66, a second metal wiring 67, a third metal wiring 68, a fourth contact electrode 69, a fifth contact electrode 70, a sixth contact electrode 71, a first metal pad 72, a second metal pad 73 and a third metal pad 74 are formed.

[0535] First contact electrode 63 connects N-type diffusion layer 55 and first metal wiring 66 , second contact electrode 64 connects high-concentration P-type diffusion layer 58 and second metal wiring 67 , and third contact electrode 65 connects metal film 60 and third metal wiring 68 .

[0536] For example, Figure 62 As shown, the first metal wiring 66 is formed to be wider than the avalanche multiplication region 59 so as to cover at least the avalanche multiplication region 59 .

[0537] The first metal wiring 66 reflects the light that has passed through the SPAD element 47 toward the SPAD element 47 .

[0538] For example, Figure 62 As shown, second metal wiring 67 is formed to overlap with high-concentration P-type diffusion layer 58 to cover the outer periphery of first metal wiring 66 .

[0539] For example, third metal wiring 68 is formed at four corners of SPAD pixel 46 to be connected to metal film 60 .

[0540] Fourth contact electrode 69 connects first metal wiring 66 and first metal pad 72 , fifth contact electrode 70 connects second metal wiring 67 and second metal pad 73 , and sixth contact electrode 71 connects third metal wiring 68 and second metal pad 74 .

[0541] The first metal pad 72, the second metal pad 73 and the third metal pad 74 are respectively used to electrically and mechanically connect to the fourth metal pad 82, the fifth metal pad 83 and the sixth metal pad 84 (all of which are described later) in the logic side wiring layer 52, each of which is formed with metal (copper).

[0542] In the logic side wiring layer 52, a first electrode pad 75, a second electrode pad 76, a third electrode pad 77, an insulating layer 78, a seventh contact electrode 79, an eighth contact electrode 80, a ninth contact electrode 81, a fourth metal pad 82, a fifth metal pad 83 and a sixth metal pad 84 are formed.

[0543] Each of the first, second, and third electrode pads 75 , 76 , and 77 is for connection to a logic circuit substrate, and the insulating layer 78 is a layer that insulates the first, second, and third electrode pads 75 , 77 from each other.

[0544] The seventh contact electrode 79 connects the first electrode pad 75 and the fourth metal pad 82 , the eighth contact electrode 80 connects the second electrode pad 76 and the fifth metal pad 83 , and the ninth contact electrode 81 connects the third electrode pad 77 and the sixth metal pad 84 .

[0545] The fourth metal pad 82 is bonded to the first metal pad 72 , the fifth metal pad 83 is bonded to the second metal pad 73 , and the sixth metal pad 84 is bonded to the third metal pad 74 .

[0546] Through such a wiring structure, for example, the first electrode pad 75 is connected to the N-type diffusion layer 55 through the seventh contact electrode 79 , the fourth metal pad 82 , the first metal pad 72 , the fourth contact electrode 69 , the first metal wiring 66 , and the first contact electrode 63 .

[0547] Therefore, in the SPAD pixel 46 , a large negative voltage applied to the N-type diffusion layer 55 can be supplied from the logic circuit substrate to the first electrode pad 75 .

[0548] Furthermore, the second electrode pad 76 is connected to the high-concentration P-type diffusion layer 58 through the eighth contact electrode 80 , the fifth metal pad 83 , the second metal pad 73 , the fifth contact electrode 70 , the second metal wiring 67 , and the second contact electrode 64 .

[0549] Therefore, in the SPAD pixel 46 , the anode of the SPAD element 47 electrically connected to the hole accumulation layer 56 is connected to the second electrode pad 76 , so that the bias with respect to the hole accumulation layer 56 can be adjusted through the second electrode pad 76 .

[0550] Furthermore, third electrode pad 77 is connected to metal film 60 through ninth contact electrode 81 , sixth metal pad 84 , third metal pad 74 , sixth contact electrode 71 , third metal wiring 68 , and third contact electrode 65 .

[0551] Therefore, in the SPAD pixel 46 , the bias voltage supplied from the logic circuit substrate to the third electrode pad 77 can be applied to the metal film 60 .

[0552] Furthermore, as described above, the SPAD pixel 46 is formed to be wider than the avalanche multiplication region 59 so that the first metal wiring 66 covers at least the avalanche multiplication region 59 , and the metal film 60 is formed to pass through the sensor substrate 50 .

[0553] That is, the SPAD pixel 46 is formed to have a reflective structure in which all surfaces of the SPAD element 47 except the light incident surface are surrounded by the first metal wiring 66 and the metal film 60 .

[0554] As a result, SPAD pixel 46 can prevent the occurrence of optical crosstalk and improve the sensitivity of SPAD element 47 based on the effect of reflecting light by first metal wiring 66 and metal film 60 .

[0555] Furthermore, the SPAD pixel 46 can adjust the bias voltage using a configuration in which the side and bottom surfaces of the N well 53 are surrounded by the hole accumulation layer 56 and the hole accumulation layer 56 is electrically connected to the anode of the SPAD element 47 .

[0556] Furthermore, the SPAD pixel 46 can form an electric field that assists carriers to the avalanche multiplication region 59 by applying a bias voltage to the metal film 60 of the inter-pixel separation unit 62 .

[0557] As described above, the SPAD pixel 46 prevents the occurrence of crosstalk and improves the sensitivity of the SPAD element 47 , and can improve characteristics.

[0558] In the distance measurement of the ToF method using the SPAD pixel 46 having the above-mentioned structure, by directly measuring Δt, the distance measurement is not performed based on the sum Qa (pixel signal Sa) of the charge amounts obtained by using the first reference signal a and the sum Qb (pixel signal Sb) of the charge amounts obtained by using the second reference signal b, and the distance measurement can be performed using the relational expression of "distance = c (speed of light) * Δt / 2".

[0559] Furthermore, as with the above-described ToF method, in this ToF method, by using at least a portion of the configuration of each of the above-described examples, it is possible to confirm the effectiveness of the laser light source and recognize the positional relationship with another imaging device AC.

[0560] In distance measurement by the ToF method using the SPAD pixel 46 , distance measurement can be performed with at least one spot beam by irradiating (scanning) at least a partial area of the imageable range with laser light to draw a specific trajectory.

[0561] In this case, instead of irradiating the laser simultaneously over a large area, the laser is irradiated continuously at one point. Therefore, while meeting the safety standards and prescribed rules for laser products, a distance farther than the distance measurement by the above-mentioned ToF method and triangulation method can be measured. However, the distance measurement using the SPAD pixel 46 can be configured as in the above-mentioned ToF method (configured so that the irradiation unit 2B can output a laser with a dot pattern, etc.). In addition, the two ToF methods can be combined. That is, by irradiating at least a portion of the imageable range with multiple laser beams to draw a specific trajectory, distance measurement can be performed with multiple point beams. Note that the specific trajectories of the multiple corresponding laser beams may be at least partially the same or substantially the same, or may be at least partially different from each other.

[0562] Figure 63 An example of a specific trajectory (scanning trajectory) traced by the laser light emitted from the effective laser light source 2 b is shown.

[0563] As shown in the figure, the laser is irradiated in a clockwise direction from the approximate center of the irradiation range (which may be the entire irradiable range or a portion of the irradiable range). Of course, the laser can also be irradiated in a counterclockwise direction from the approximate center of the irradiation range.

[0564] also, Figure 64 An example of a trajectory drawn by laser light emitted from the ineffective another light source 200 is shown in FIG.

[0565] As shown in the figure, the laser is irradiated from left to right for each pixel row from the top to the bottom within the irradiation range. Of course, the laser can be irradiated from the top to the bottom, from right to left for each pixel row, from the bottom to the top, from left to right for each pixel row, from the bottom to the top, from right to left for each pixel row within the irradiation range, or along a trajectory obtained by rotating the above trajectory.

[0566] When detecting Figure 64 In the case of the laser trajectory shown, the ineffectiveness of the other light source 200 can be determined by comparison with the specific trajectory of the effective laser light source 2b.

[0567] Furthermore, the position, posture, etc. of the invalid another light source 200 can be estimated based on the received laser trajectory.

[0568] In order to realize such a function, it is sufficient that the irradiation unit 2 , the irradiation control unit 2 a , or the imaging device 1C includes a scanning unit that performs irradiation control to draw a specific trajectory.

[0569] It is sufficient to configure the scanning unit so that the laser draws a specific trajectory. For example, the scanning unit can be implemented by a mechanical moving unit, can be implemented by changing the direction of a micromirror using a microelectromechanical system (MEMS), can be implemented by controlling the refractive properties of a liquid crystal material, and further, can be implemented by applying a phased array.

[0570] Other examples of specific trajectories of the effective laser light source 2b are given in Figure 65 and Figure 66 Shown in.

[0571] Figure 65 The specific trajectory shown is a trajectory that irradiates in a zigzag pattern from the upper left to the lower right of the irradiation range. Of course, irradiation can also be performed in a zigzag pattern from the lower left corner to the upper right corner, or along a trajectory obtained by rotating these trajectories.

[0572] exist Figure 66In the specific trajectory shown, pixels within the irradiation range are divided into left and right pixels, and only the left pixels are irradiated in a zigzag pattern, followed by the right pixels. That is, irradiation can be performed along a trajectory obtained by combining two or more types of trajectories with different irradiation orders and irradiation range areas.

[0573] Notice, Figure 64 The trajectory shown may be a specific trajectory of the effective laser light source 2b.

[0574] The structure that allows various modes to be selected for specific trajectories enables high-precision confirmation of the effectiveness of the laser light source.

[0575] Note, about Figure 66 The specific trajectory shown can be made to mimic the laser trajectory more complexly by changing the ratio between the left and right pixels. In addition, by making it possible to select the irradiation start pixel from each of the left and right pixels, the laser trajectory can be made more difficult to mimic.

[0576] <4. Summary>

[0577] As described in each of the above examples and modified examples, the semiconductor device (imaging devices 1, 1A, 1B and 1C) includes an imaging unit 3 (3B and 3C) (including a photoelectric conversion element 17 (17A and 17B) that receives reflected light of the light emitted from the specific laser light source 2b reflected by the object 100 (101) and performs photoelectric conversion) and a control unit 4 (which performs validity determination processing for determining whether the light received by the photoelectric conversion element 17 (17A and 17B) is light emitted from the specific laser light source 2b).

[0578] For example, in the case of performing distance measurement using the specific laser light source 2 b , the control unit 4 determines that the received light is reflected light of the light emitted from the specific laser light source 2 b .

[0579] This can reduce the possibility of erroneous measurement caused by the entry of another laser light source (another light source 200 ).

[0580] As described in the fourth example of the effectiveness determination process, the photoelectric conversion element 17 includes the first photoelectric conversion element 17A, and the first photoelectric conversion element 17A can be connected to at least four charge accumulation units (FD 20), and charges accumulated in the first photoelectric conversion element 17A at different times are transferred to the FD20.

[0581] As a result, a light reception waveform based on the waveform of the light emitted from the laser light source 2b can be obtained.

[0582] Therefore, the specific waveform of each laser light source 2b can be used to determine whether the light received by the first photoelectric conversion element 17A is emitted by a specific laser light source 2b. Specifically, the waveform of the rising edge, overshoot, and undershoot of the laser light source can be used to determine whether the light is emitted by the specific laser light source 2b.

[0583] As described in the seventh example of the validity determination process, etc., the photoelectric conversion element 17 includes the second photoelectric conversion element 17B, and the number of charge accumulation units (FD 20) connected to the second photoelectric conversion element 17B can be smaller than the number of charge accumulation units (FD 20) connected to the first photoelectric conversion element 17A.

[0584] For example, the second photoelectric conversion element 17B is used for distance measurement.

[0585] As a result, unlike the first photoelectric conversion element 17A for confirming the validity of the laser light source or authenticating the laser light source, the number of charge accumulation units (FD 20) connected to the second photoelectric conversion element 17B for distance measurement is small, so the number of components can be reduced and the cost can be reduced.

[0586] As described in the seventh example of the effectiveness determination process and the like, the number of the second photoelectric conversion elements 17B may be larger than the number of the first photoelectric conversion elements 17A.

[0587] As a result, the number of charge accumulation units (FDs 20) connected to the photoelectric conversion elements 17 (17A and 17B) is further reduced.

[0588] Therefore, the number of components can be further reduced, and this can contribute to cost reduction.

[0589] As described in the seventh example of the effectiveness determination process and the like, the first photoelectric conversion elements 17A may be arranged outside the group of the second photoelectric conversion elements 17B.

[0590] As a result, the second photoelectric conversion elements 17B can be densely arranged.

[0591] Therefore, the imaging element 3 b (3Cb) can have a function for confirming the effectiveness of the laser light source, or can have both a function for authenticating the laser light source and a function for measuring distance, without increasing the size of the imaging element 3 b. Furthermore, because the first photoelectric conversion element 17A that controls the authentication function is located at the edge of the imaging element 3 b (3Cb), the effect on the captured image can be minimized.

[0592] As described in the eighth example of the effectiveness determination process and the like, the first photoelectric conversion element 17A can be used for the effectiveness determination process and can also be used for distance measurement.

[0593] As a result, the light reception data of the light reception data received by the first photoelectric conversion element 17A can be effectively used.

[0594] Therefore, light reception data used for distance measurement can be increased, and the accuracy of the distance measurement result can be improved.

[0595] As described in the eighth example of the effectiveness determination process and the like, the first photoelectric conversion element 17A may be used for the effectiveness determination process, and the second photoelectric conversion element 17B may be used for distance measurement.

[0596] That is, some photoelectric conversion elements are used for effectiveness determination processing.

[0597] This simplifies the validity determination process.

[0598] As described in the first example of the effectiveness determination process and the like, in the effectiveness determination process, determination based on the rising waveform of the laser light source 2 b may be performed.

[0599] The rising waveform of light emitted from the laser light source 2b is unique to each laser resonator that generates laser light, and it is difficult for anyone other than the manufacturer of the laser light source 2b to replicate the rising waveform.

[0600] Therefore, by comparing the rising waveform unique to the laser resonator with the reception waveform, the effectiveness of the laser light source 2 b can be confirmed with high accuracy.

[0601] As described in the ninth example of the effectiveness determination process and the like, in the effectiveness determination process, determination based on the light receiving spot shape of light emitted from the laser light source 2 b may be performed.

[0602] The light receiving spot shape is determined based on the spatial positional relationship among the laser light source 2 b , the object 100 ( 101 ), and the imaging unit 3 ( 3B and 3C).

[0603] By performing the effectiveness determination process based on the light receiving point shape, it can be determined whether light emitted from another light source 200 located at a position different from the laser light source 2 b is received, and the effectiveness determination process can be implemented.

[0604] As described in each of the above examples, the imaging apparatus 1 ( 1A, 1B, and 1C) may include the irradiation unit 2 ( 2B) (including a specific laser light source).

[0605] By integrating the specific laser light source 2 b and the imaging unit 3 ( 3B and 3C), the positional relationship between the specific laser light source 2 b and the imaging unit 3 ( 3B and 3C) is constant.

[0606] This makes it easier to determine the effectiveness of the light emitted from a particular laser light source 2b.

[0607] As described in the third example of the effectiveness determination process and the like, the irradiation unit 2 ( 2B) may be capable of emitting laser light having a first waveform and laser light having a second waveform different from the first waveform.

[0608] As a result, effectiveness determination processing can be performed using both the first waveform and the second waveform.

[0609] Therefore, the accuracy of the determination result of the validity determination process can be improved. In addition, the possibility of accidentally determining that the light source is valid by receiving light from another light source (another light source 200) can be reduced. In particular, by imitating (copying) the emission of the irradiation unit 2 (2B), it is difficult to inappropriately pass the validity determination process.

[0610] As described in the third example of the validity determination process and the like, the light intensity of the first waveform and the light intensity of the second waveform may be different at the time of emission.

[0611] As a result, effectiveness determination processing taking light intensity into consideration can be performed.

[0612] Therefore, the accuracy of the determination result of the effectiveness determination process can be further improved.

[0613] As described in the third example of the effectiveness determination process and the like, the rising shapes of the first waveform and the second waveform may be different from each other.

[0614] As a result, the validity determination process can be performed using the rising shapes of both the first waveform and the second waveform.

[0615] Therefore, the accuracy of the determination result of the effectiveness determination process can be improved.

[0616] As described in the third example of the effectiveness determination process and the like, the irradiation unit 2 ( 2B) may emit the first waveform and the second waveform in a random order.

[0617] As a result, the validity determination process can be performed in consideration of the order of appearance of the first waveform and the second waveform.

[0618] Therefore, the accuracy of the determination result of the effectiveness determination process can be improved.

[0619] As described in the fifth example of the effectiveness determination process and the like, the light emission time lengths of the first waveform and the second waveform may be different from each other.

[0620] As a result, effectiveness determination processing can be performed in consideration of the light emission time lengths of the first waveform and the second waveform.

[0621] Therefore, the accuracy of the determination result of the effectiveness determination process can be improved.

[0622] As described in the sixth example of the effectiveness determination process and the like, the non-light emission time lengths of the first waveform and the second waveform may be different from each other.

[0623] As a result, effectiveness determination processing can be performed in consideration of the non-light emission time lengths of the first waveform and the second waveform.

[0624] Therefore, the accuracy of the determination result of the effectiveness determination process can be improved.

[0625] As described in the ninth example of the effectiveness determination process and the like, the irradiation unit 2 ( 2B) may emit spot-shaped laser light having an emission range equal to or smaller than one-fourth of the imaging range of the imaging unit 3 ( 3B and 3C).

[0626] As a result, even if the spot shape of the light irradiated on the object 100 (101) is modified and doubled, the spot shape can be within the imaging range.

[0627] Therefore, the validity determination process can be appropriately performed.

[0628] As described in the ninth example of the effectiveness determination process and the like, the irradiation unit 2 ( 2B) is capable of emitting laser light having a mirror-symmetrical or point-symmetrical dot-like shape.

[0629] This makes it easy to realize point-shaped lasers.

[0630] Therefore, costs can be reduced.

[0631] As described in the fourteenth example of the effectiveness determination process and the like, the irradiation unit 2 ( 2B) may be capable of emitting laser light having a non-mirror-symmetric or non-point-symmetric spot shape.

[0632] This makes it difficult to mimic point-like lasers.

[0633] Therefore, the accuracy of the determination result of the effectiveness determination process can be improved.

[0634] As described in the tenth example of the effectiveness determination process and the like, the irradiation unit 2 ( 2B) is capable of emitting a dot pattern in which a plurality of dot laser beams are irregularly arranged.

[0635] As a result, a plurality of point-like laser beams can be detected by one imaging by the imaging unit 3 ( 3B and 3C).

[0636] Therefore, it is more difficult to imitate the laser light, and the accuracy of the determination result of the effectiveness determination process can be improved. In addition, the transmission time for the effectiveness determination process can be shortened.

[0637] As described in the tenth example of the effectiveness determination process and the like, the irradiation unit 2 ( 2B) may be capable of emitting a dot pattern in which a plurality of dot laser beams are regularly arranged.

[0638] This makes it easy to generate dot patterns.

[0639] Therefore, while improving the accuracy of the determination result of the effectiveness determination process, it is also possible to ensure ease of implementation. In addition, the transmission time for the effectiveness determination process can be shortened.

[0640] As described in the tenth example of the effectiveness determination process and the like, the irradiation unit 2 ( 2B) may emit a dot pattern in which dot laser beams whose number is equal to or less than one quarter of the number of the photoelectric conversion elements 17 are provided.

[0641] As a result, effectiveness determination processing taking into account the shape of the light-receiving spot can be performed based on each of the spot light beams forming the dot pattern.

[0642] That is, by performing the effectiveness determination process that takes into account not only the arrangement of the light receiving spots but also the dot shape, the difficulty of imitating the laser light can be further increased, and the accuracy of the judgment result of the effectiveness determination process can be improved. In addition, the transmission time for the effectiveness determination process can be shortened.

[0643] As described in the seventeenth example of the effectiveness determination processing, etc., the irradiation unit 2 (2B) can emit the laser light source 2b so that the laser having a spot shape smaller than the imaging range of the imaging unit 3 (3B and 3C) draws a specific trajectory within the imaging range.

[0644] As a result, the validity determination process using a specific trajectory, ie, the scanning trajectory, can be performed.

[0645] Therefore, the difficulty of imitating the laser light can be increased, and the accuracy of the determination result of the effectiveness determination process can be improved.

[0646] In each of the above examples, the description has been made with reference to a flowchart. However, the processing order of each process shown in the flowchart can be arbitrarily changed. For example, the distance measurement process can be performed after the validity determination process is performed, or vice versa. In addition, in each of the above examples, the description has been made with reference to the accompanying drawings of the system configuration, component configuration, and circuit configuration. However, the direction of the arrows in the drawings is an example, and at least some of the arrows may be reversed or bidirectional. Note that at least a portion of the functions, controls, and processes described as being performed by the control unit 4 can be configured to be performed by the signal processing unit 14 instead of the control unit 4. Furthermore, at least a portion of the functions, controls, and processes described as being performed by the signal processing unit 14 can be configured to be performed by the control unit 4 instead of the signal processing unit 14. Furthermore, at least a portion of the data stored in the storage unit 6 can be data stored in the data storage unit 15 instead of the storage unit 6, and can be configured to be read from or written to by the signal processing unit 14. Furthermore, at least a portion of the data stored in the data storage unit 15 can be data stored in the storage unit 6 instead of the data storage unit 15, and can be configured to be read from and written to by the control unit 4. Note that the present technology can be applied to cameras, mobile phones, smartphones, tablets, personal computers, game consoles, televisions, monitors, electronic devices, mobile devices, cars, mobile objects, drones, flying objects, robots, movable objects, and the like.

[0647] The embodiments described in each of the examples above have various modifications. That is, some components in each of the examples above may be omitted, and some or all of these components may be changed or modified. In addition, some components may be replaced with other components, and other components may be added to some or all of the components.

[0648] Furthermore, some or all of the components may be divided into a plurality of parts, some or all of the components may be divided into a plurality of parts, and at least some of the plurality of divided or separated components may have different functions or characteristics.

[0649] Additionally, at least some components may be moved and different embodiments may be formed.

[0650] Furthermore, various embodiments may be formed by adding coupling elements or relay elements to the combination of at least some components.

[0651] Furthermore, various embodiments may be formed by adding a switching function to a combination of at least some components.

[0652] The present embodiment is not limited to the configuration indicated in each of the above examples, and various changes can be made without departing from the scope of the present technology.

[0653] Note that the effects described in this specification are merely exemplary and are not limited to these. In addition, there may be additional effects.

[0654] <5. This technology>

[0655] Note that the present technology can have the following configurations.

[0656] (1) A semiconductor device comprising:

[0657] an imaging unit including a photoelectric conversion element that receives reflected light of light emitted from a specific laser light source reflected from an object and performs photoelectric conversion; and

[0658] A control unit is configured to execute validity determination processing for determining whether light received by the photoelectric conversion element is light emitted from a specific laser light source.

[0659] (2) The semiconductor device according to (1), wherein

[0660] The photoelectric conversion element includes a first photoelectric conversion element, and

[0661] The first photoelectric conversion element is connected to at least four charge accumulation units, and charges accumulated in the first photoelectric conversion element for different time periods are transferred to the charge accumulation units.

[0662] (3) The semiconductor device according to (2), wherein

[0663] The photoelectric conversion element includes a second photoelectric conversion element, and

[0664] The number of charge accumulation units connected to the second photoelectric conversion element is smaller than the number of charge accumulation units connected to the first photoelectric conversion element.

[0665] (4) The semiconductor device according to (3), wherein

[0666] The number of the second photoelectric conversion elements is greater than the number of the first photoelectric conversion elements.

[0667] (5) The semiconductor device according to (3) or (4), wherein

[0668] The first photoelectric conversion elements are arranged outside a group of second photoelectric conversion elements.

[0669] (6) The semiconductor device according to any one of (2) to (5), wherein

[0670] The first photoelectric conversion element is used for effectiveness determination processing and is also used for distance measurement.

[0671] (7) The semiconductor device according to any one of (3) to (5), wherein

[0672] The first photoelectric conversion element is used for effectiveness determination processing, and

[0673] The second photoelectric conversion element is used for distance measurement.

[0674] (8) The semiconductor device according to any one of (1) to (7), wherein

[0675] In the effectiveness determination process, determination is performed based on the rising waveform of the laser light source.

[0676] (9) The semiconductor device according to any one of (1) to (8), wherein

[0677] In the effectiveness determination process, determination based on the shape of a light-receiving spot of light emitted from the laser light source is performed.

[0678] (10) The semiconductor device according to any one of (1) to (9), further comprising:

[0679] The irradiation unit includes a specific laser light source.

[0680] (11) The semiconductor device according to (10), wherein

[0681] The irradiation unit is capable of emitting laser light having a first waveform and laser light having a second waveform different from the first waveform.

[0682] (12) The semiconductor device according to (11), wherein

[0683] The light intensities at the time of emission of the first waveform and the second waveform are different from each other.

[0684] (13) The semiconductor device according to (11) or (12), wherein

[0685] The rising shapes of the first waveform and the second waveform are different from each other.

[0686] (14) The semiconductor device according to any one of (11) to (13), wherein

[0687] The irradiation unit emits the first waveform and the second waveform in a random order.

[0688] (15) The semiconductor device according to any one of (11) to (14), wherein

[0689] The light emission time lengths of the first waveform and the second waveform are different from each other.

[0690] (16) The semiconductor device according to any one of (11) to (15), wherein

[0691] The first waveform and the second waveform have different non-lighting time lengths.

[0692] (17) The semiconductor device according to any one of (10) to (16), wherein

[0693] The irradiation unit is capable of emitting a point-shaped laser light, the emission range of which is equal to or smaller than one quarter of the imaging range of the imaging unit.

[0694] (18) The semiconductor device according to any one of (10) to (17), wherein

[0695] The irradiation unit can emit laser light having a mirror-symmetrical or point-symmetrical point shape.

[0696] (19) The semiconductor device according to any one of (10) to (17), wherein

[0697] The irradiation unit can emit laser light having a non-mirror-symmetrical or non-point-symmetrical spot shape.

[0698] (20) The semiconductor device according to any one of (10) to (19), wherein

[0699] The irradiation unit is capable of emitting a plurality of point-shaped laser beams in an irregularly arranged point pattern.

[0700] (21) The semiconductor device according to any one of (10) to (19), wherein the irradiation unit can emit a dot pattern in which the plurality of dot laser beams are regularly arranged.

[0701] (22) The semiconductor device according to any one of (10) to (21), wherein

[0702] The irradiation unit is capable of emitting a dot pattern in which the dot laser beams are arranged in a number equal to or less than one fourth the number of the photoelectric conversion elements.

[0703] (23) The semiconductor device according to any one of (10) to (19), wherein

[0704] The irradiation unit emits a specific laser light source so that the laser light having a spot shape smaller than the imaging range of the imaging unit draws a specific trajectory within the imaging range.

[0705] Reference designator list

[0706] 1,1A,1B,1C Imaging Device

[0707] 2 Irradiation Unit

[0708] 2B Irradiation Unit

[0709] 2b Laser light source

[0710] 4 Control unit

[0711] 17 Photoelectric conversion element

[0712] 17A First photoelectric conversion element

[0713] 17B Second photoelectric conversion element

[0714] 20, 20a, 20b, 20c, 20d FD

[0715] 100,101 objects

[0716] T1, T3, T4, T5 are the luminous time periods.

Claims

1. A semiconductor device comprising: an imaging unit including a photoelectric conversion element that receives reflected light of light emitted from a specific laser light source reflected from an object and performs photoelectric conversion; as well as a control unit configured to perform validity determination processing for determining whether the light received by the photoelectric conversion element is light emitted from the specific laser light source, The photoelectric conversion element includes a first photoelectric conversion element and a second photoelectric conversion element. the first photoelectric conversion element is connected to at least four charge accumulation units, and charges accumulated in the first photoelectric conversion element at different time periods are transferred to the charge accumulation units, the number of charge accumulation units connected to the second photoelectric conversion element is smaller than the number of charge accumulation units connected to the first photoelectric conversion element, The number of the second photoelectric conversion elements is greater than the number of the first photoelectric conversion elements. The first photoelectric conversion element is arranged outside a group of the second photoelectric conversion elements, the first photoelectric conversion element is used for the validity determination process, and The second photoelectric conversion element is used for distance measurement.

2. The semiconductor device according to claim 1, wherein In the effectiveness determination process, determination is performed based on a rising waveform of the laser light source.

3. The semiconductor device according to claim 1, wherein In the effectiveness determination process, determination based on a light-receiving spot shape of light emitted from the laser light source is performed.

4. The semiconductor device according to claim 1 , further comprising: The irradiation unit includes a specific laser light source.

5. The semiconductor device according to claim 4, wherein The irradiation unit is capable of emitting laser light having a first waveform and laser light having a second waveform different from the first waveform. The semiconductor device according to claim 5 , wherein The light intensities of the first waveform and the second waveform at the time of emission are different from each other.

7. The semiconductor device according to claim 5, wherein Rising shapes of the first waveform and the second waveform are different from each other. The semiconductor device according to claim 5 , wherein The irradiation unit emits the first waveform and the second waveform in a random order.

9. The semiconductor device according to claim 5, wherein The light emitting time lengths of the first waveform and the second waveform are different from each other.

10. The semiconductor device according to claim 5, wherein The first waveform and the second waveform have different non-light-emission time lengths.

11. The semiconductor device according to claim 4, wherein The irradiation unit is capable of emitting a point-shaped laser beam, and an emission range of the point-shaped laser beam is equal to or smaller than a quarter of an imaging range of the imaging unit.

12. The semiconductor device according to claim 4, wherein The irradiation unit can emit laser light having a mirror-symmetrical or point-symmetrical point shape.

13. The semiconductor device according to claim 4, wherein The irradiation unit can emit laser light having a non-mirror-symmetrical or non-point-symmetrical point shape.

14. The semiconductor device according to claim 4, wherein The irradiation unit is capable of emitting a plurality of point-shaped laser beams in an irregularly arranged point pattern.

15. The semiconductor device according to claim 4, wherein The irradiation unit is capable of emitting a plurality of regularly arranged dot patterns of dot laser beams.

16. The semiconductor device according to claim 4, wherein The irradiation unit is capable of emitting a dot pattern in which a number of dot laser beams equal to or less than one fourth the number of the photoelectric conversion elements is arranged.

17. The semiconductor device according to claim 4, wherein The irradiation unit emits the specific laser light source so that laser light having a spot shape smaller than an imaging range of the imaging unit describes a specific trajectory within the imaging range.

Images