Light detection device and distance measuring system

Abstract

Provided is an optical detection device comprising: a light-receiving unit (502) which includes a pixel array (510) comprising a plurality of light-receiving elements (512) arranged in a matrix; a control signal generation unit (576) which generates a control signal; a control unit (570) which is electrically connected to the light-receiving unit via a control line and controls the light-receiving unit on the basis of the control signal; a processing unit (530) which is electrically connected to the light-receiving unit via a signal line and processes an output signal from the light-receiving unit; and a fault determination unit (534) which detects faults. The fault determination unit detects a fault in the control line by means of the output signal from the light-receiving unit controlled on the basis of a fault detection control signal having a predetermined pattern.

Description

[Technical Field]

[0001] This disclosure relates to an optical detection device and a distance measuring system. [Background technology]

[0002] In recent years, distance measuring devices (distance measuring systems) that measure distance using the Time of Flight (ToF) method have attracted attention. Some distance measuring systems use a Single Photon Avalanche Diode (SPAD) as the light-receiving element. In this SPAD, a single photon of light is incident, and the electrons (charge) generated by photoelectric conversion are multiplied in the PN junction region (avalanche amplification), allowing for high-precision detection of the light. Then, in this distance measuring system, the timing of the current flow caused by the multiplied electrons is detected, enabling high-precision distance measurement. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-143996 [Patent Document 2] Japanese Patent Publication No. 2020-112528 [Patent Document 3] Japanese Patent Publication No. 2020-112501 [Overview of the project] [Problems that the invention aims to solve]

[0004] In the aforementioned distance measuring device, if a fault occurs in the control line that electrically connects the device to the control unit that controls each light-receiving element, the distance measuring device may not be properly controlled, and errors may be included in the pixel signals from the light-receiving elements. Furthermore, in such a case, it may lead to malfunctions in systems that utilize the distance measurement information from the distance measuring device (such as autonomous driving systems). Therefore, it is strongly required that distance measuring devices be able to detect such control line faults.

[0005] Therefore, this disclosure proposes an optical detection device and a distance measuring system capable of detecting control line failures. [Means for solving the problem]

[0006] According to this disclosure, an optical detection device is provided, comprising: a light-receiving unit including a pixel array unit consisting of a plurality of light-receiving elements arranged in a matrix; a control signal generation unit that generates a control signal; a control unit electrically connected to the light-receiving unit via a control line and controlling the light-receiving unit based on the control signal; a processing unit electrically connected to the light-receiving unit via a signal line and processing an output signal from the light-receiving unit; and a fault determination unit that detects a fault, wherein the fault determination unit detects a fault in the control line based on the output signal from the light-receiving unit controlled based on a fault detection control signal having a predetermined pattern.

[0007] Furthermore, according to this disclosure, a distance measuring system is provided which includes an illumination device that emits illumination light and a light detection device that receives reflected light reflected by a subject from the illumination light, wherein the light detection device includes a light receiving unit that includes a pixel array unit consisting of a plurality of light receiving elements arranged in a matrix, a control signal generation unit that generates a control signal, a control unit that is electrically connected to the light receiving unit via a control line and controls the light receiving unit based on the control signal, a processing unit that is electrically connected to the light receiving unit via a signal line and processes an output signal from the light receiving unit, and a fault determination unit that detects a fault, wherein the fault determination unit detects a fault in the control line based on the output signal from the light receiving unit which is controlled based on a fault detection control signal having a predetermined pattern. [Brief explanation of the drawing]

[0008] [Figure 1] This figure schematically illustrates distance measurement using a direct ToF method applicable to embodiments of the present disclosure. [Figure 2] This figure shows an example histogram based on the time of light reception by the light receiving unit, applicable to the embodiments of this disclosure. [Figure 3]It is a block diagram showing an example of the configuration of a distance measurement system to which an embodiment of the present disclosure is applicable. [Figure 4] It is a block diagram showing an example of the configuration of a distance measurement device to which an embodiment of the present disclosure is applicable. [Figure 5] It is a circuit diagram showing an example of the configuration of a pixel circuit to which an embodiment of the present disclosure is applicable. [Figure 6] It is an explanatory diagram showing an example of the connection between a plurality of pixel circuits to which an embodiment of the present disclosure is applicable. [Figure 7] It is a schematic diagram showing an example of the stacked structure of a distance measurement device to which an embodiment of the present disclosure is applicable. [Figure 8] It is a block diagram showing a part of the configuration of a distance measurement device according to an embodiment of the present disclosure. [Figure 9] It is an explanatory diagram for explaining the outline of an embodiment of the present disclosure. [Figure 10] It is a flowchart of a processing procedure according to an embodiment of the present disclosure. [Figure 11] It is a flowchart for explaining the outline of the flow of SPAD access check according to an embodiment of the present disclosure. [Figure 12] It is a table showing a control signal pattern for checking the failure of the I_SPAD line. [Figure 13] It is an explanatory diagram for explaining the details of the control signal pattern for checking the failure of the I_SPAD line. [Figure 14] It is a table showing a control signal pattern for checking the failure of the EN_VLINE line and the EN_AREA line. [Figure 15] It is an explanatory diagram for explaining the details of the control signal pattern for checking the failure of the EN_VLINE line and the EN_AREA line. [Figure 16] It is a table showing a control signal pattern for checking the failure of the ACT_SPAD_V line and the ACT_SPAD_H line. [Figure 17] It is an explanatory diagram for explaining the details of the control signal pattern for checking the failure of the ACT_SPAD_V line and the ACT_SPAD_H line. [Figure 18]It is a table showing a control signal pattern for failure checking of a downsampling circuit. [Figure 19] It is an explanatory diagram for explaining details of a control signal pattern for failure checking of a downsampling circuit. [Figure 20] It is a diagram showing an example of a timing chart at the time of failure detection. [Figure 21] It is a block diagram showing a configuration example of a vehicle control system. [Figure 22] It is a diagram showing an example of a sensing area.

Embodiments for Carrying Out the Invention

[0009] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted. In addition, in this specification and the drawings, a plurality of components having substantially the same or similar functional configurations may be distinguished by attaching different alphabets after the same reference numeral. However, when it is not necessary to particularly distinguish each of a plurality of components having substantially the same or similar functional configurations, only the same reference numeral is attached.

[0010] In addition, in the following description of circuits (electrical connections), unless otherwise specified, "electrically connected" means connecting such that electricity (signals) can conduct between a plurality of elements. In addition, "electrically connected" in the following description includes not only cases where a plurality of elements are directly and electrically connected, but also cases where they are indirectly and electrically connected via other elements.

[0011] The description will be made in the following order. 1. Background for the inventor to create the embodiments of the present disclosure 1.1 Distance measurement method 1.2 Distance measurement system 1.3 Distance measurement device 1.4 Pixel circuit 1.5 Laminated structure 1.6 Background 2. Embodiments 2.1 Ranging device 2.2 Processing Procedure 3. Summary 4. Application Examples 5. Supplementary Information

[0012] <<1. Background leading to the inventor's creation of the embodiments of this disclosure>> <1.1 Distance measurement method> First, before describing the embodiments of this disclosure, the present inventor will explain the background leading to the creation of these embodiments. First, an overview of the distance measurement methods to which each embodiment of this disclosure may be applied will be provided. This disclosure relates to a technology for measuring distance using light. In the embodiments of this disclosure, the direct ToF (Time of Flight) method is applied as the distance measurement method. The direct ToF method is a method in which light emitted from a light source is reflected by an object to be measured, and the reflected light is received by a photodetector (specifically, a SPAD), and the distance is measured based on the time difference between the light emission timing and the light reception timing.

[0013] The general outline of distance measurement using the direct ToF method will be explained with reference to Figures 1 and 2. Figure 1 is a schematic diagram showing distance measurement using the direct ToF method applicable to embodiments of the present disclosure, and Figure 2 is a diagram showing an example histogram based on the time of light reception by the light receiving unit 302, applicable to embodiments of the present disclosure.

[0014] As shown in Figure 1, the distance measuring device 300 includes a light source unit (illumination device) 301 and a light receiving unit 302. The light source unit 301 is, for example, a laser diode, which is driven to emit laser light in a pulsed manner. The light emitted from the light source unit 301 is reflected by the object to be measured (subject) 303 and received as reflected light by the light receiving unit 302. The light receiving unit 302 includes a plurality of light receiving elements that convert light into electrical signals by photoelectric conversion and can output a pixel signal corresponding to the received light.

[0015] Here, let the time when the light source unit 301 emits light (emission timing) be time t0, and let the time when the light receiving unit 302 receives the reflected light reflected by the measurement object 303 from the light emitted from the light source unit 301 (light reception timing) be time t1. Assuming a constant c as the speed of light (2.9979×10 8 [m / sec]), the distance D between the distance measuring device 300 and the measurement object 303 is shown in the following equation (1).

Equation

[0016] The distance measuring device 300 can repeatedly execute the above-described process a plurality of times. The light receiving unit 302 may include a plurality of light receiving elements, and the distance D may be calculated based on each light reception timing at which the reflected light is received by each light receiving element. The distance measuring device 300 measures the time t from the emission timing time t0 of the light until the light reception timing at which the light is received by the light receiving unit 302 m (referred to as the light reception time t m ), classifies it based on classes (bins), and generates a histogram.

[0017] Note that the light received by the light receiving unit 302 at the light reception time t m is not limited to the reflected light obtained by reflecting the light emitted from the light source unit 301 by the measurement object 303. For example, ambient light around the distance measuring device 300 (light receiving unit 302) is also received by the light receiving unit 302.

[0018] In the histogram shown in FIG. 2, the horizontal axis represents bins, and the vertical axis represents the frequency for each bin. The bins are obtained by classifying the light reception time t m in units of a predetermined unit time d. Specifically, bin #0 is 0 ≦ t m < d, bin #1 is d ≦ t m < 2×d, bin #2 is 2×d ≦ t m < 3×d,..., bin #(N - 2) is (N - 2)×d ≦ t m < (N - 1)×d. When the exposure time of the light receiving unit 302 is the time t ep , t ep = N×d.

[0019] The distance measuring device 300 has a light reception time t m The number of times each bin is acquired is counted based on the bins to determine the frequency 310 for each bin, and a histogram is generated. Here, the light receiving unit 302 also receives light other than the reflected light that is reflected from the light emitted from the light source unit 301. An example of such light other than the target reflected light is the ambient light mentioned above. The portion shown in range 311 in the histogram includes the ambient light component due to ambient light. Ambient light is light that is randomly incident on the light receiving unit 302 and becomes noise for the target reflected light.

[0020] On the other hand, the reflected light in question is light received at a specific distance and appears as the active light component 312 in the histogram. The bin corresponding to the frequency of the peaks within this active light component 312 becomes the bin corresponding to the distance D to the object 303 being measured. The distance measuring device 300 can calculate the distance D to the object 303 being measured according to equation (1) above by acquiring the representative time of that bin (for example, the time in the middle of the bin) as the time t1 described above. In this way, by using multiple received light results, it becomes possible to perform appropriate distance measurement even if random noise occurs.

[0021] <1.2 Distance Measurement System> Next, with reference to Figure 3, an example of the configuration of a distance measuring system 90 to which embodiments of the present disclosure may be applied will be described. Figure 3 is a block diagram showing an example of the configuration of a distance measuring system 90 to which embodiments of the present disclosure may be applied. As shown in Figure 3, the distance measuring system 90 may mainly include a light source unit (illumination device) 400, a distance measuring device 500, a storage device 600, a host 700, and an optical system 800. Hereinafter, each block included in the distance measuring system 90 will be described sequentially.

[0022] (Light source section 400) The light source unit 400 corresponds to the light source unit 301 in Figure 1 described above, and consists of a laser diode or the like, and is driven to emit laser light in a pulsed manner, for example. The light source unit 400 can be a VCSEL (Vertical Cavity Surface Emitting Laser) that emits laser light as a surface light source. Alternatively, the light source unit 400 may use an array of laser diodes arranged in a line, and the laser light emitted from the laser diode array may be scanned in a direction perpendicular to the line. Furthermore, the light source unit 400 may use a laser diode as a single light source, and the laser light emitted from the laser diode may be scanned in both horizontal and vertical directions.

[0023] (Risting device 500) The distance measuring device 500 includes the light-receiving unit 302 shown in Figure 1 above. Furthermore, the light-receiving unit 302 has a pixel array (not shown) consisting of multiple light-receiving elements arranged in a two-dimensional grid (matrix) (for example, 189 × 600 elements). Details of the distance measuring device 500 and the light-receiving unit 302 will be described later. Furthermore, the optical system 800 can guide light incident from the outside to the pixel array of the light-receiving unit 302 of the distance measuring device 500.

[0024] Furthermore, the distance measuring device 500 provides time information (reception time t) indicating the timing at which light was received in the pixel array. m The number of times the data is acquired is counted within a predetermined time range, and the frequency for each bin is determined to generate the histogram described above. Then, the distance measuring device 500 calculates the distance D to the object to be measured 303 based on the generated histogram. The information showing the calculated distance D is stored, for example, in the storage device 600.

[0025] (Host 700) The host 700 can control the overall operation of the distance measuring system 90. For example, the host 700 supplies a light emission trigger to the distance measuring device 500, which is a trigger for illuminating the light source unit 400. The distance measuring device 500 illuminates the light source unit 400 at a timing based on this light emission trigger and stores the time t0 indicating the timing of the illumination. The host 700 may also set a pattern for distance measurement for the distance measuring device 500, for example, in response to an external instruction.

[0026] <1.3 Distance measuring device> Next, with reference to Figure 4, an example of the configuration of a distance measuring device 500 to which embodiments of the present disclosure may be applied will be described. Figure 4 is a block diagram showing an example of the configuration of a distance measuring device 500 to which embodiments of the present disclosure may be applied. As shown in Figure 4, the distance measuring device 500 mainly includes a light receiving unit 502 including a pixel array unit 510, a processing unit 530, a control unit 570, a light emission timing control unit 580, and an interface (I / F) 590. Hereinafter, each block included in the distance measuring device 500 will be described sequentially.

[0027] (Light receiving section 502) As shown in Figure 4, the light-receiving unit 502 includes a pixel array unit 510. The pixel array unit 510 has a plurality of SPADs (photodetectors) 512 arranged in a matrix (for example, arranged in a 189 × 600 grid). Each SPAD 512 is controlled by a control unit 570, which will be described later. For example, the control unit 570 can control the reading of pixel signals from each SPAD 512 for each block containing (p × q) SPADs 512, with p SPADs in the row direction and q SPADs in the column direction. The control unit 570 can also scan each SPAD 512 in the row direction and then in the column direction, using the above block as a unit, to read out pixel signals from each SPAD 512. Details of the light-receiving unit 502 will be described later.

[0028] (Processing unit 530) The processing unit 530 can process the pixel signals read out from each SPAD 512 via the signal lines. As shown in Figure 4, the processing unit 530 includes a conversion unit 540, a generation unit 550, and a signal processing unit 560.

[0029] First, the pixel signals read from each SPAD 512 and output from the pixel array unit 510 are supplied to the conversion unit 540. The conversion unit 540 converts the pixel signals supplied from the pixel array unit 510 into digital information. More specifically, the conversion unit 540 converts the pixel signals supplied from the pixel array unit 510 into time information indicating the timing at which light was received by the SPAD 512 corresponding to the pixel signal.

[0030] The generation unit 550 generates a histogram based on the time information obtained by the conversion unit 540 from the converted pixel signal.

[0031] The signal processing unit 560 performs predetermined calculations based on the histogram data generated by the generation unit 550, for example, to calculate distance information. The signal processing unit 560 creates an approximate curve of the histogram based on the histogram data generated by the generation unit 550. The signal processing unit 560 can detect the peak of this approximated histogram curve and determine the distance D (an example of distance measurement information) based on the detected peak. The signal processing unit 560 may also apply filtering to the approximated histogram curve when performing histogram curve approximation. For example, the signal processing unit 560 can suppress noise components by applying a low-pass filter to the approximated histogram curve.

[0032] The distance information obtained by the signal processing unit 560 is then supplied to the interface 590. The interface 590 functions as an output unit that outputs the distance information supplied from the signal processing unit 560 to the outside as output data. For example, MIPI (Mobile Industry Processor Interface) can be applied as the interface 590. In the above example, the distance information is output to the outside via the interface 590, but this is not the only example. That is, the histogram data, which is the histogram data generated by the generation unit 550, may also be output to the outside from the interface 590. In this case, the histogram data output from the interface 590 is supplied to an external information processing device, for example, and processed as appropriate.

[0033] (Control unit 570) The control unit 570 can, for example, execute control of the light receiving unit 502, etc., based on control signals or a reference clock signal supplied from an external source, according to a pre-programmed program. Furthermore, as described above, the control unit 570 can designate a predetermined area of ​​the pixel array unit 510 as a target area and control the SPAD 512 included in the target area to be the target for reading pixel signals. The control unit 570 can also scan multiple rows (multiple lines) together, and then scan them further in the column direction to read pixel signals from each SPAD 512.

[0034] (Light emission timing control unit 580) The light emission timing control unit 580 generates a light emission control signal indicating the light emission timing according to a light emission trigger signal supplied from an external source. The light emission control signal is supplied to the light source unit 400 and also to the processing unit 530.

[0035] <1.4 Pixel Circuit> More specifically, the light-receiving unit 502 described above is composed of multiple pixel circuits 900, each containing a multiple SPAD 512. Therefore, an example of the configuration of a pixel circuit 900 to which the embodiments of this disclosure can be applied will be described with reference to Figures 5 and 6. Figure 5 is a circuit diagram showing an example of the configuration of a pixel circuit 900 to which the embodiments of this disclosure can be applied, and Figure 6 is an explanatory diagram showing an example of the connections between multiple pixel circuits 900.

[0036] As shown in Figure 5, the pixel circuit 900 includes a SPAD (photodetector) 512, transistors 902 and 904, a constant current source 910, switch circuits 920 and 940, an inverter circuit 930, and an OR circuit (theoretical sum circuit) 950.

[0037] SPAD512 is a single-photon avalanche diode that converts incident light into an electrical signal via photoelectric conversion and outputs it. In the embodiments of this disclosure, SPAD512 converts incident photons into electrical signals via photoelectric conversion and outputs a pulse corresponding to the incident photon. SPAD512 has the characteristic that when a large negative voltage that causes avalanche multiplication is applied to the cathode, electrons generated in response to the incident single photon undergo avalanche multiplication, causing a large current to flow. By utilizing this characteristic of SPAD512, the incident of a single photon can be detected with high sensitivity.

[0038] The cathode of the SPAD512 is connected to the coupling point of two transistors 902 and 904 and to the input of the inverter circuit 930. The anode of the SPAD512 is electrically connected to a voltage source of voltage (-Vbd). Voltage (-Vbd) is a large negative voltage required to generate avalanche multiplication for the SPAD512.

[0039] Transistor 902 is a P-channel MOSFET (Metal Oxide Semiconductor Field Effect Transistor), and transistor 904 is an N-channel MOSFET, both electrically connected to each other at their source and drain. The gates of transistors 902 and 904 are also electrically connected to the switch circuit 920. Furthermore, the source of transistor 902 is electrically connected to the power supply voltage Vdd via the constant current source 910.

[0040] The switch circuit (drive switch) 920 is composed of a NAND gate and functions as a switch that drives the SPAD512 based on the control signals from the control unit 570 described above. Specifically, the switch circuit 920 controls transistors 902 and 904 based on the control signals from the ACT_SPAD_V line and ACT_SPAD_H line (first and third control lines) to apply a reverse bias to the SPAD512. When the reverse bias is applied, the SPAD512 becomes active, and when a photon is incident on the SPAD512 in this state, avalanche multiplication is initiated, and current flows from the cathode to the anode of the SPAD512.

[0041] In detail, the ACT_SPAD_V line and the ACT_SPAD_H line can transmit control signals from the pixel array unit 510 to the switch circuit 920 of the pixel circuit 900, respectively. These control signals allow the on / off state of the switch circuit 920 to be controlled for each pixel circuit 900. The on state of the switch circuit 920 activates the SPAD 912, while the off state deactivates the SPAD 912. Therefore, the power consumption of the SPAD 512 in the pixel array unit 510 can be reduced.

[0042] Furthermore, the pixel signal from the SPAD512 is input to the inverter circuit 930. The inverter circuit 930 performs, for example, a threshold check on the input pixel signal, and each time the pixel signal exceeds the threshold in the positive or negative direction, it inverts the signal and outputs a pulsed signal.

[0043] The pulse signal output from the inverter circuit 930 is input to the switch circuit (output switch) 940. The switch circuit 940 is an AND circuit and functions as a switch that controls the output of the pixel signal from the SPAD 512 based on the control signal from the control unit 570 described above. Specifically, the switch circuit 940 outputs the pulse signal to the OR circuit 950 based on the control signals from the EN_VLINE line and the EN_AREA line (second and fourth control lines).

[0044] In detail, the EN_VLINE line and the EN_AREA line can transmit control signals from the pixel array unit 510 to the switch circuit 940 of the pixel circuit 900, respectively. These control signals control the on / off state of the switch circuit 940 for each pixel circuit 900, and as a result, the output of each SPAD 512 can be enabled / disabled.

[0045] The OR circuit 950 receives the output of the switch circuit 940 as one input, and the output of the switch circuit 940 of another pixel circuit 900, or a horizontal control signal from the control unit 570 described above, as input to the other input via the I_SPAD line (fifth control line). The OR circuit 950 then outputs according to the input signal. In detail, in multiple SPADs 512 located in the same column, adjacent SPADs 512 separated by a predetermined number of rows (for example, 20 rows) are connected in a so-called daisy-chain. Furthermore, one end of the daisy-chain is electrically connected to the control unit 570, and the other end is connected to the processing unit 530 (in detail, a downsampling circuit (not shown)). In this way, the circuit size of the processing unit 530 can be reduced (in detail, the number of downsampling circuits can be reduced).

[0046] Furthermore, multiple pixel circuits 900 can be connected as shown in Figure 6, for example, when a 4SPAD period is used.

[0047] <1.5 Laminated structure> Furthermore, the distance measuring device 500 to which the embodiments of this disclosure may be applied may have a stacked structure in which multiple semiconductor substrates are stacked. Therefore, an example of a stacked structure of the distance measuring device 500 to which the embodiments of this disclosure may be applied will be described with reference to Figure 7. Figure 7 is a schematic diagram showing an example of a stacked structure of the distance measuring device 500 to which the embodiments of this disclosure may be applied.

[0048] As shown in Figure 7, the distance measuring device 500 has a stacked structure in which two semiconductor substrates (first and second substrates) 200 and 250 are stacked. These semiconductor substrates 200 and 250 may also be referred to as semiconductor chips.

[0049] In detail, the aforementioned pixel array section 510 is provided on the semiconductor substrate (first substrate) 200 shown in the upper part of Figure 7, and within the region of the pixel array section 510, the SPADs 512 included in each of the multiple pixel circuits 900 are arranged in a matrix. Furthermore, the transistors 902, 904, constant current source 910, switch circuits 920, 940, inverter circuit 930, and OR circuit 950 of the pixel circuit 900 are provided on the semiconductor substrate 250 shown in the lower part of Figure 7. The SPADs 512 can be electrically connected to each element on the semiconductor substrate 250, for example, by a Copper-Copper Connection (CCC).

[0050] The semiconductor substrate 250 is provided with a logic array section 252, which includes transistors 902 and 904, a constant current source 910, switch circuits 920 and 940, an inverter circuit 930, and an OR circuit 950 in the pixel circuit 900. Furthermore, the semiconductor substrate 250 can be provided in close proximity to the logic array section 252 with a processing unit 530 that processes the pixel signals acquired by the SPAD 512, and a control unit 570 that controls the operation as a distance measuring device 500. In addition, the control lines such as the ACT_SPAD_V line, ACT_SPAD_H line, EN_VLINE line, EN_AREA line, and I_SPAD line are provided on the semiconductor substrate 250.

[0051] Furthermore, the layout on the semiconductor substrates 200 and 250 is not limited to the configuration example shown in Figure 7, and various layouts can be selected. In addition, the distance measuring device 500 is not limited to being composed of a stack of multiple semiconductor substrates, but may be composed of a single semiconductor substrate.

[0052] <1.6 Background> Next, based on the above-described example of the configuration of the distance measuring device 500, the present inventors will now explain in detail the background that led them to create the embodiments of the present disclosure, with reference to Figures 8 and 9. Figure 8 is a block diagram showing a part of the configuration of the distance measuring device 500 according to the embodiment of the present disclosure, and Figure 9 is an explanatory diagram for explaining the outline of the embodiments of the present disclosure.

[0053] As shown in Figure 8, the control unit 570 can control the pixel circuit 900 via multiple control lines (specifically, the ACT_SPAD_V line and ACT_SPAD_H line, EN_VLINE line and EN_AREA line, I_SPAD line, etc. as described above). However, if a failure occurs in such a control line, as explained earlier, proper control will not be achieved, which may result in errors in the pixel signals from the pixel circuit 900. Furthermore, in such a case, it will lead to malfunctions in the system that utilizes the distance measurement information from the distance measuring device 500. Therefore, the distance measuring device 500 is strongly required to detect such control line failures.

[0054] However, in most conventional distance measuring devices 500, detailed consideration has not been given to the detection of control line failures. Furthermore, even when a means for detecting failures was provided, it was not possible to distinguish between a failure in the pixel circuit 900 and a failure in the control line. In addition, detecting failures required a lot of time, and measurements had to be taken under predetermined conditions (such as when no light was irradiated or when a predetermined amount of light was irradiated) in order to obtain reference values ​​for failure detection in advance, making frequent failure detection difficult.

[0055] Therefore, in view of these circumstances, the inventors have created the embodiments of the present disclosure described below. In the embodiments of the present disclosure, control line failures can be detected. Specifically, as shown in Figure 9, in the embodiments created by the inventors, in order to detect a control line failure, a fault detection control signal having a predetermined pattern is generated, and based on the generated fault detection control signal, the control unit 570 controls the pixel circuit 900. Next, the fault determination unit 534 provided in the processing unit 530 acquires the pixel signal output from the pixel circuit 900 controlled based on the fault detection control signal, and if the acquired pixel signal differs from the expected output value, it determines that there is a control line failure and notifies the host 700, etc.

[0056] According to the embodiments of this disclosure created by the present inventors, control line failures can be detected. Furthermore, according to these embodiments, the failure can be detected by the distance measuring device 500 alone, without receiving instructions from the host 700 while the distance measuring device 500 is in operation. In addition, in these embodiments, there is no need to irradiate the light receiving unit 502 (pixel array unit 510) with light, and there is no need to acquire a reference signal for determination in advance, making it easy to detect failures. The details of each embodiment of this disclosure will be described in order below.

[0057] <<2. Embodiments>> <2.1 Distance measuring device> First, with reference to Figure 8, the main components of the distance measuring device 500 according to the embodiment of this disclosure will be described. As shown in Figure 8, the distance measuring device 500 mainly includes a pixel circuit 900, a control unit 570, and a processing unit 530. Hereinafter, each block included in the distance measuring device 500 will be described in order, but the explanation of parts that overlap with the distance measuring device 500 described so far will be omitted.

[0058] (Pixel circuit 900) As shown in Figure 8, the pixel circuit 900 includes a SPAD (photodetector) 512, switch circuits 920 and 940, and an OR circuit (theoretical sum circuit) 950. The switch circuit (drive switch) 920 functions as a switch that drives the SPAD 512 based on control signals from the vertical control unit 572 and horizontal control unit 574 of the control unit 570 via the ACT_SPAD_V line and ACT_SPAD_H line (first and third control lines). The switch circuit (output switch) 940 functions as a switch that controls the output of the pixel signal from the SPAD 512 based on control signals from the vertical control unit 572 and horizontal control unit 574 of the control unit 570 via the EN_VLINE line and EN_AREA line (second and fourth control lines). Furthermore, the OR circuit (theoretical sum circuit) 950 can output the pixel signal from the SPAD 512 to the downsampling circuit 532 of the processing unit 530 based on a control signal from the vertical control unit 572 of the control unit 570 via the I_SPAD line (fifth control line). The downsampling circuit 532 may also include a column shift circuit (not shown) for removing noise.

[0059] (Control unit 570) As shown in Figure 8, the control unit 570 includes a vertical control unit (row-direction readout control unit) 572, a horizontal control unit (column-direction readout control unit) 574, and a control signal generation unit 576. The vertical control unit 572 can control the pixel circuit 900 (specifically, the operation and output of the SPAD 512) in the vertical direction, i.e., on a row-by-row basis. Furthermore, the vertical control unit 572 can also control the OR circuit 950 described above. In addition, the horizontal control unit 574 can control the pixel circuit 900 (specifically, the operation and output of the SPAD 512) in the horizontal direction, i.e., on a column-by-column basis.

[0060] Furthermore, the control signal generation unit 576 can generate control signals and output them to the vertical control unit 572 and the horizontal control unit 574 described above. Specifically, the control signal generation unit 576 can, for example, generate fault detection control signals having a predetermined pattern to detect faults in various control lines and output them to the vertical control unit 572 and the horizontal control unit 574 described above.

[0061] (Processing unit 530) As shown in Figure 8, the processing unit 530 mainly includes a downsampling circuit 532 and a fault detection unit 534. The downsampling circuit 532 converts the pixel signal, which is read from the pixel circuit 900 via a signal line and has undergone signal processing such as noise reduction in a column shift circuit (not shown), into a digital signal. Furthermore, the downsampling circuit 532 outputs the pixel signal converted into a digital signal to the fault detection unit 534, which will be described later.

[0062] Furthermore, the fault detection unit 534 acquires a pixel signal (output signal) from the pixel circuit 900 controlled by a fault detection control signal, determines whether the acquired pixel signal differs from the expected output value, and if it differs, determines that the control line is faulty and notifies the host 700, etc.

[0063] In this embodiment, the configuration of the distance measuring device 500 is not limited to that shown in Figure 8, which illustrates the main components of the device; other elements may be added.

[0064] <2.2 Processing Procedure> (Summary of processing procedure) Next, with reference to Figure 10, an overview of the processing procedure of this embodiment will be described. Figure 10 is a flowchart of the processing procedure according to this embodiment. As shown in Figure 10, the processing procedure according to this embodiment mainly includes a plurality of steps from step S101 to step S107. The details of each of these steps according to this embodiment will be described below.

[0065] The fault detection process according to this embodiment is performed while the distance measuring device 500 is operating and the frame synchronization signal is inactive. Furthermore, this process is repeated at a suitable frequency while the distance measuring device 500 is operating.

[0066] First, the distance measuring device 500 determines the content of the fault detection (step S101). For example, in this embodiment, in addition to detecting a fault in the control line (SPAD access check), it may also detect a fault in the processing unit 530 (data path check), or a fault in the pixel circuit 900 (light receiving unit) (SPAD check). In this embodiment, if the processing unit 530 fails, it may become impossible to detect faults in the control line or the pixel circuit 900, so it is preferable to perform the data path check first. That is, it is preferable to perform the SPAD access check and then the SPAD check in the order of the data path check. However, in this embodiment, the order of the SPAD access check and SPAD check is not limited, and they may be in reverse order, or only one of the fault detections may be performed. In this embodiment, it is possible to select whether to perform each fault detection using the enable register corresponding to each fault detection.

[0067] Thus, according to this embodiment, the location of the failure can be identified by sequentially detecting failures in different locations.

[0068] Next, the distance measuring device 500 generates a fault detection control signal having a predetermined pattern for performing fault detection in a predetermined order based on the decision made in step S101 (step S102). For example, if the checks are performed in the order of data path check, SPAD access check, and SPAD check, the fault detection control signal will contain a control signal for the data path check, a control signal for the SPAD access check, and a control signal for the SPAD check.

[0069] Then, the distance measuring device 500 performs control for fault detection using the fault detection control signal generated in step S102 described above (step S103).

[0070] Next, the distance measuring device 500 acquires the pixel signal from the SPAD 512 under the control described in step S103 (step S104).

[0071] The distance measuring device 500 then determines whether a malfunction has been detected (step S105). Specifically, the distance measuring device 500 determines that a malfunction has been detected, for example, if the pixel signal acquired in step S104 described above differs from the expected output value. If the distance measuring device 500 determines that a malfunction has been detected (step S105: Yes), it proceeds to the process in step S106; if it determines that no malfunction has been detected (step S105: No), it proceeds to the process in step S107.

[0072] The distance measuring device 500 notifies the host 700, etc., of the result that a fault has been detected (step S106). For example, when a fault is detected, the distance measuring device 500 may output a low-level signal from a predetermined output terminal (error output terminal). Alternatively, when a fault is detected, the distance measuring device 500 may write an ON signal (high-level signal) to the area indicating the error status among the status signals indicating the state of the distance measuring device 500 that are transmitted from the distance measuring device 500 to the host 700, or information indicating the error may be written on the signal (MIPI data) output from the interface 590 described above. Note that such information may be stored in memory, etc. (not shown), in addition to being notified to the host 700.

[0073] In the above description, it was explained that the host 700 is notified of the detection of a fault, but this embodiment is not limited to this. For example, the distance measuring device 500 may notify detailed information, such as whether the fault was detected in the data path check, SPAD access check, or SPAD check.

[0074] The distance measuring device 500 outputs the processing result of the pixel signal acquired in step S104 described above, and terminates the series of processes (step S107). In this embodiment, as explained above, the series of processes are repeated at a suitable frequency while the distance measuring device 500 is operating.

[0075] In the above description, it was explained that a series of checks—data path check, SPAD access check, and SPAD check—are performed sequentially. However, this embodiment is not limited to this, and the checks may be stopped when a fault is detected.

[0076] (SPAD access check) Next, with reference to Figure 11, the details of the SPAD access check for detecting control line faults according to this embodiment will be described. Figure 11 is a flowchart illustrating the overview of the SPAD access check flow according to this embodiment. As explained earlier, the SPAD access check primarily detects control line faults.

[0077] In detail, as shown in Figure 11, first, a check is performed to detect a fault in the I_SPAD line (the fifth control line) connecting the OR circuit (theoretical sum circuit) 950 and the vertical control unit 572, etc. Next, a check is performed to detect a fault in the EN_VLINE line and EN_AREA line (the second and fourth control lines) connecting the switch circuit 940 and the vertical control unit 572 and horizontal control unit 574 of the control unit 570. Then, a check is performed to detect a fault in the ACT_SPAD_V line and ACT_SPAD_H line (the first and third control lines) connecting the switch circuit 920 and the vertical control unit 572 and horizontal control unit 574 of the control unit 570. Furthermore, a check is performed to detect a fault in the downsampling circuit 532.

[0078] In this embodiment, the steps are not limited to those shown in Figure 11, and the order may be changed. In this embodiment, the check may be stopped when a fault is detected, but in such cases, it is preferable to check for faults sequentially, starting from the processing unit 530. This makes it possible to identify the fault location while shortening the check time. The details of each stage of the SPAD access check will be described below.

[0079] (I_SPAD line fault check) Refer to Figures 12 and 13 to describe the details of the I_SPAD line fault check. Figure 12 is a table showing the control signal pattern (first signal) for I_SPAD line fault check, and Figure 13 is an explanatory diagram illustrating the details of the control signal pattern for I_SPAD line fault check.

[0080] In Test #1 in Figure 12, all target areas (all pixel circuits 900, including the I_SPAD line) are set to a state where they are short-circuited with the power supply voltage (fixed at a high level). Specifically, the control signals from the vertical control unit 572 and horizontal control unit 574 to the switch circuits 920 and 940 are set to a low level via the ACT_SPAD_V line, ACT_SPAD_H line, EN_VLINE line, and EN_AREA line. Furthermore, the control signal from the vertical control unit 572 to the OR circuit 950 via the I_SPAD line is set to a low level. At this point, if a pixel signal corresponding to the short circuit with the power supply voltage is output, it can be confirmed that the electrical connection of the I_SPAD line is secured, meaning there is no fault.

[0081] In Test #2 in Figure 12, all target areas (all pixel circuits 900, including the I_SPAD line) are set to a state where they are shorted to GND (fixed at a low level). Specifically, the control signals from the vertical control unit 572 and horizontal control unit 574 to the switch circuits 920 and 940 are set to a low level via the ACT_SPAD_V line, ACT_SPAD_H line, EN_VLINE line, and EN_AREA line. Furthermore, one pulse (high level) is input as a control signal via the I_SPAD line from the vertical control unit 572 to the OR circuit 950. At this time, if a pixel signal corresponding to the short to GND is output, it can be confirmed that the electrical connection of the I_SPAD line is secured, that is, there is no fault.

[0082] Tests #3 and #4 in Figure 12 verify the electrical connection of I_SPAD lines between adjacent pixel circuits 900. For example, suppose that the I_SPAD lines between multiple pixel circuits 900 are located on two different layers so as not to electrically influence each other. In such a case, multiple signal patterns are prepared according to the layer of the I_SPAD lines to check for faults in the I_SPAD lines connecting up, down, left, and right, and each is executed. For example, as shown in Figure 13, when multiple pixel circuits 900 are located on two different layers (two-layer wiring), the control signals of the two patterns A and B shown in the figure are used as control signals from the vertical control unit 572 to the OR circuit 950 via the I_SPAD lines. In detail, as shown in the upper part of Figure 13, for pattern A, for multiple I_SPAD lines located on the same layer, two off-state I_SPAD lines are placed between on-state I_SPAD lines, and a pattern is prepared such that the states do not coincide with adjacent I_SPAD lines located on different layers. Furthermore, as shown in the lower part of Figure 13, pattern B is a pattern obtained by shifting pattern A horizontally by one position. The detectable locations for each pattern are indicated by the solid lines in Figure 13. At this time, as with previous tests, the control signals from the vertical control unit 572 and horizontal control unit 574 to the switch circuits 920 and 940 via the ACT_SPAD_V line, ACT_SPAD_H line, EN_VLINE line, and EN_AREA line are set to a low level. By using the control signals of these two patterns, it is possible to check for faults in the I_SPAD lines that connect the top, bottom, left, and right. At this time, if the pixel signals corresponding to the electrical connections of patterns A and B shown in Figure 13 are output, it can be confirmed that the electrical connection of the I_SPAD lines is secured, that is, there are no faults.

[0083] Furthermore, if multiple pixel circuits 900 are provided on a single layer (single-layer wiring), it is only necessary to check for a fault in the I_SPAD line connecting the left and right sides, so there is no need to prepare multiple signal patterns.

[0084] (Fault check for EN_VLINE and EN_AREA lines) Refer to Figures 14 and 15 to describe the details of fault checking for the EN_VLINE and EN_AREA lines. Figure 14 is a table showing the control signal patterns (second signals) for fault checking for the EN_VLINE and EN_AREA lines, and Figure 15 is an explanatory diagram illustrating the details of the control signal patterns for fault checking for the EN_VLINE and EN_AREA lines.

[0085] In Test #1 in Figure 14, the EN_VLINE line connecting to the switch circuit 940 of all target areas (all pixel circuits 900) is short-circuited with the power supply voltage (fixed at a high level), and the electrical connection between the power supply voltage and the EN_VLINE line is checked. Specifically, the control signals from the vertical control unit 572 and horizontal control unit 574 to the switch circuit 920 of all target areas (all pixel circuits 900) via the ACT_SPAD_V line and ACT_SPAD_H line are set to a low level. Then, the control signal from the horizontal control unit 574 to the switch circuit 940 of all target areas (all pixel circuits 900) via the EN_AREA line is set to a low level, and the control signal from the vertical control unit 572 to the switch circuit 940 of all target areas (all pixel circuits 900) via the EN_VLINE line is set to a high level. At this time, if a pixel signal corresponding to the short circuit with the power supply voltage is output, it can be confirmed that the electrical connection of the EN_VLINE line is secured, that is, there is no fault.

[0086] In Test #2 of Figure 14, the EN_AREA line connecting to the switch circuit 940 of all target areas (all pixel circuits 900) is short-circuited with the power supply voltage (fixed at a high level), and the electrical connection between the power supply voltage and the EN_AREA line is checked. Specifically, the control signals from the vertical control unit 572 and horizontal control unit 574 to the switch circuit 920 of all target areas (all pixel circuits 900) via the ACT_SPAD_V line and ACT_SPAD_H line are set to a low level. Then, the control signal from the vertical control unit 572 to the switch circuit 940 of all target areas (all pixel circuits 900) via the EN_VLINE line is set to a low level, and the control signal from the horizontal control unit 574 to the switch circuit 940 of all target areas (all pixel circuits 900) via the EN_AREA line is set to a high level. At this time, if a pixel signal corresponding to the short circuit with the power supply voltage is output, it can be confirmed that the electrical connection of the EN_AREA line is secured, that is, there is no fault.

[0087] Next, in tests #3 to #11 in Figure 14, each of the 21 target areas (21 rows of pixel circuits 900) is set to a state where it is shorted to GND (fixed at a low level). Specifically, the control signals from the vertical control unit 572 and horizontal control unit 574 to the switch circuit 920 of all target areas (all pixel circuits 900) via the ACT_SPAD_V line and ACT_SPAD_H line are set to a low level. Then, the control signal from the vertical control unit 572 to the switch circuit 940 of each of the above target areas (a predetermined number of rows of pixel circuits 900) via the EN_VLINE line is set to a high level, and the control signal from the horizontal control unit 574 to the switch circuit 940 of all target areas (all pixel circuits 900) via the EN_AREA line is set to a high level. At this time, if a pixel signal corresponding to the short to GND is output, it can be confirmed that the electrical connection of the EN_VLINE line and EN_AREA line of the above target areas (a predetermined number of rows of pixel circuits 900) is secured, that is, there is no fault.

[0088] Next, in test #12 in Figure 14, the electrical connections in the column direction of the target area from row 0 to row 20 (pixel circuits 900 from row 0 to row 20) are checked. Specifically, the control signals from the vertical control unit 572 and horizontal control unit 574 to the switch circuit 920 of all target areas (all pixel circuits 900) via the ACT_SPAD_V line and ACT_SPAD_H line are set to Low level. Then, the control signal from the vertical control unit 572 to the switch circuit 940 of the target area from row 0 to row 20 (pixel circuits 900) via the EN_VLINE line is set to High level, and the control signal from the horizontal control unit 574 to the switch circuit 940 of the odd-numbered columns (2x+1 (x=0~20)) of all target areas (all pixel circuits 900) via the EN_AREA line is set to High level. In other words, the signal pattern is set so that different pixel signals are output at adjacent locations (up / down adjacent, left / right adjacent) where there is a risk of short circuits, as shown in Figure 15. In this case, if no pixel signals corresponding to short circuits between adjacent pixels (up / down or left / right) are output, as shown in Figure 15, it can be confirmed that the EN_AREA line is not short-circuited, meaning there is no malfunction.

[0089] Next, in test #13 in Figure 14, similar to #12, the electrical connections in the column direction of the target area from row 21 to row 41 (pixel circuits 900 from row 0 to row 20) are checked. Specifically, the control signals from the vertical control unit 572 and horizontal control unit 574 to the switch circuit 920 of all target areas (all pixel circuits 900) via the ACT_SPAD_V line and ACT_SPAD_H line are set to Low level. Then, the control signal from the vertical control unit 572 to the switch circuit 940 of the target area from row 21 to row 41 (pixel circuits 900 from row 21 to row 41) via the EN_VLINE line is set to High level, and the control signal from the horizontal control unit 574 to the switch circuit 940 of the odd-numbered columns (2x+1 (Xx=0~20)) of all target areas (all pixel circuits 900) via the EN_AREA line is set to High level. In other words, the signal pattern is designed to output different pixel signals at adjacent locations (up / down adjacent, left / right adjacent) where a short circuit is likely to occur, as shown in Figure 15. In this case, if no pixel signals corresponding to the short circuits at the up / down adjacent and left / right adjacent locations shown in Figure 15 are output, it can be confirmed that the EN_AREA line is not short-circuited, meaning there is no fault.

[0090] (Fault check for ACT_SPAD_V line and ACT_SPAD_H line) Refer to Figures 16 and 17 to describe the details of the ACT_SPAD_V and ACT_SPAD_H lines. Figure 16 is a table showing the control signal patterns (third signals) for fault checking the ACT_SPAD_V and ACT_SPAD_H lines, and Figure 17 is an explanatory diagram illustrating the details of the control signal patterns for fault checking the ACT_SPAD_V and ACT_SPAD_H lines.

[0091] Here, the power supply voltage and electrical connection to GND for the ACT_SPAD_V line and ACT_SPAD_H line are checked. If a fault is detected in even one connection line, it is determined to be a fault. As shown on the left side of Figure 17, detection is performed when all target areas (all pixel circuits 900) are shorted to GND (Low level fixation). Then, as shown on the right side of Figure 17, detection is performed for each target area every 21 rows when it is shorted to the power supply voltage in the row and column directions (High level fixation). For example, in the case of H level fixation, a fault can be detected if a Low level signal is output.

[0092] In Test #1 shown in Figure 16, as shown on the left side of Figure 17, the control signals from the vertical control unit 572 and the horizontal control unit 574 to the switch circuits 920 and 940 are set to High level via the ACT_SPAD_V line, ACT_SPAD_H line, EN_VLINE line, and EN_AREA line in order to short-circuit all target areas (all pixel circuits 900) to GND (fixed at Low level). In this case, if a High level signal is output, a fault in the ACT_SPAD_H line and ACT_SPAD_V line can be detected.

[0093] Next, in tests #2 to #10 in Figure 16, the electrical connection with the power supply voltage in the row direction of each target area (21 rows of pixel circuits 900) is checked (fixed at high level). Specifically, the control signal from the vertical control unit 572 to the switch circuit 920 of all target areas (all pixel circuits 900) via the ACT_SPAD_V line is set to a low level, and the control signal from the horizontal control unit 574 to the switch circuit 920 of all target areas (all pixel circuits 900) via the ACT_SPAD_H line is set to a high level. Then, the control signal from the vertical control unit 572 to the switch circuit 940 of each target area (21 rows of pixel circuits 900) via the EN_VLINE line is set to a high level, and the control signal from the horizontal control unit 574 to the switch circuit 940 of all target areas (all pixel circuits 900) via the EN_AREA line is set to a high level. At this time, if a low-level signal is output, a fault in the ACT_SPAD_V line of the target area can be detected.

[0094] Next, in tests #11 to #19 of Figure 16, the electrical connection with the power supply voltage in the column direction of each target area (21 rows of pixel circuits 900) is checked (fixed at high level). Specifically, the control signal from the vertical control unit 572 to the switch circuit 920 of all target areas (all pixel circuits 900) via the ACT_SPAD_V line is set to high level, and the control signal from the horizontal control unit 574 to the switch circuit 920 of all target areas (all pixel circuits 900) via the ACT_SPAD_H line is set to low level. Then, the control signal from the vertical control unit 572 to the switch circuit 940 of each target area (21 rows of pixel circuits 900) via the EN_VLINE line is set to high level, and the control signal from the horizontal control unit 574 to the switch circuit 940 of all target areas (all pixel circuits 900) via the EN_AREA line is set to high level. At this time, if a low-level signal is output, a fault in the ACT_SPAD_H line of the target area can be detected.

[0095] (Fault check for downsampling circuit 532) The fault check for the downsampling circuit 532 will be explained with reference to Figures 18 and 19. Figure 18 is a table showing the control signal pattern (fourth signal) for fault checking of the downsampling circuit 532, and Figure 19 is an explanatory diagram illustrating the details of the control signal pattern for fault checking of the downsampling circuit 532.

[0096] Here, failures in the downsampling circuit 532 corresponding to sequentially selected target areas are sequentially detected. Specifically, in tests #1 to #11 in Figure 18, the presence or absence of column-direction failures in the target area from row 0 to row 20 (pixel circuits 900 from row 0 to row 20) is sequentially checked. Specifically, in tests #1 to #11 in Figure 18, the control signals from the vertical control unit 572 and horizontal control unit 574 to the switch circuit 920 of all target areas (all pixel circuits 900) via the ACT_SPAD_V line and ACT_SPAD_H line are set to a Low level. Then, the control signal from the vertical control unit 572 to the switch circuit 940 of the target area from row 0 to row 20 (pixel circuits 900 from row 0 to row 20) via the EN_VLINE line is set to a High level. Furthermore, for each of the multiple columns, the control signal from the horizontal control unit 574 to the switch circuit 940 via the EN_AREA line is set to a High level. Furthermore, for each test, the control signals are sequentially increased to a high level while shifting the rows horizontally, as shown in Figure 19. If the pixel signals corresponding to the control signal patterns shown in Figure 18 are output during this process, it indicates that the downsampling circuit 532 is functioning correctly.

[0097] (Regarding fault detection) Furthermore, in this embodiment, when a fault is detected, as described above, after performing a series of detections, a signal indicating that a fault has been detected is output only if a fault is detected. For example, as shown in the lower region R100 of Figure 20, which shows an example of a timing chart for fault detection, a low-level signal may be output from a predetermined output terminal (error output terminal) of the distance measuring device 500 when a fault is detected.

[0098] Furthermore, as shown in the middle region R101 of Figure 20, when a fault is detected, an ON signal (high-level signal) may be written to the region indicating the error status among the status signals indicating the status of the distance measuring device 500 transmitted from the distance measuring device 500 to the host 700.

[0099] <<3. Summary>> As described above, according to the embodiments of this disclosure, control line failures can be detected. In particular, according to this embodiment, the failure can be detected by the distance measuring device 500 alone while the distance measuring device 500 is in operation, without receiving instructions from the host 700. Furthermore, in this embodiment, there is no need to irradiate the light receiving unit 502 (pixel array unit 510) with light, and there is no need to acquire a reference signal for determination in advance, making it easy to detect failures.

[0100] <<4. Application Examples>> Referring to Figure 21, an example of a mobile device control system to which the technology proposed in this disclosure may be applied will be described. Figure 21 is a block diagram showing an example configuration of a vehicle control system 11, which is an example of a mobile device control system to which this technology may be applied.

[0101] The vehicle control system 11 is installed in the vehicle 1 and performs processing related to driving assistance and autonomous driving of the vehicle 1.

[0102] The vehicle control system 11 includes a vehicle control ECU (Electronic Control Unit) 21, a communication unit 22, a map information storage unit 23, a location information acquisition unit 24, an external recognition sensor 25, an in-vehicle sensor 26, a vehicle sensor 27, a memory unit 28, a driving support / automatic driving control unit 29, a DMS (Driver Monitoring System) 30, an HMI (Human Machine Interface) 31, and a vehicle control unit 32.

[0103] The vehicle control ECU 21, communication unit 22, map information storage unit 23, location information acquisition unit 24, external recognition sensor 25, in-vehicle sensor 26, vehicle sensor 27, memory unit 28, driving support / autonomous driving control unit 29, driver monitoring system (DMS) 30, human-machine interface (HMI) 31, and vehicle control unit 32 are interconnected and can communicate with each other via a communication network 41. The communication network 41 consists of an in-vehicle communication network or bus that conforms to digital bidirectional communication standards such as CAN (Controller Area Network), LIN (Local Interconnect Network), LAN (Local Area Network), FlexRay (registered trademark), and Ethernet (registered trademark). The communication network 41 may be used differently depending on the type of data being transmitted. For example, CAN may be applied to data related to vehicle control, and Ethernet may be applied to large-capacity data. Furthermore, each part of the vehicle control system 11 may be directly connected using wireless communication intended for relatively short-range communication, such as Near Field Communication (NFC) or Bluetooth®, without going through the communication network 41.

[0104] In the following, when each part of the vehicle control system 11 communicates via the communication network 41, the description of the communication network 41 will be omitted. For example, when the vehicle control ECU 21 and the communication unit 22 communicate via the communication network 41, it will simply be described as the vehicle control ECU 21 and the communication unit 22 communicating.

[0105] The vehicle control ECU 21 is composed of various processors, such as a CPU (Central Processing Unit) and an MPU (Micro Processing Unit). The vehicle control ECU 21 can control the functions of the entire vehicle control system 11 or a part of it.

[0106] The communication unit 22 can communicate with various devices inside and outside the vehicle, other vehicles, servers, base stations, etc., and send and receive various types of data. In this case, the communication unit 22 may use multiple communication methods for communication.

[0107] Here, we will briefly explain the external communication capabilities of the communication unit 22. The communication unit 22 can communicate with servers (hereinafter referred to as "external servers") located on an external network via a base station or access point using wireless communication methods such as 5G (fifth-generation mobile communication system), LTE (Long Term Evolution), and DSRC (Dedicated Short Range Communications). The external network with which the communication unit 22 communicates is, for example, the internet, a cloud network, or a network specific to a carrier. The communication method used by the communication unit 22 to the external network is not particularly limited, as long as it is a wireless communication method that enables digital two-way communication at a predetermined communication speed and over a predetermined distance.

[0108] Furthermore, for example, the communication unit 22 can communicate with terminals located near the vehicle using P2P (Peer To Peer) technology. Examples of terminals located near the vehicle include terminals worn by mobile objects moving at relatively low speeds, such as pedestrians and cyclists, terminals installed in fixed locations such as stores, or MTC (Machine Type Communication) terminals. In addition, the communication unit 22 can also perform V2X communication. V2X communication refers to communication between the vehicle and other entities, such as vehicle-to-vehicle communication with other vehicles, vehicle-to-infrastructure communication with roadside devices, etc., vehicle-to-home communication with homes, and vehicle-to-pedestrian communication with terminals carried by pedestrians, etc.

[0109] The communication unit 22 can, for example, receive programs from an external source (over the air) to update the software that controls the operation of the vehicle control system 11. Furthermore, the communication unit 22 can receive map information, traffic information, information about the vehicle 1's surroundings, etc., from an external source. In addition, the communication unit 22 can transmit information about the vehicle 1 and information about the vehicle 1's surroundings, etc., to an external source. Examples of information about the vehicle 1 that the communication unit 22 transmits to an external source include data indicating the status of the vehicle 1 and recognition results from the recognition unit 73. Furthermore, the communication unit 22 can also perform communications corresponding to vehicle emergency notification systems such as e-Call.

[0110] For example, the communication unit 22 can also receive electromagnetic waves transmitted by road traffic information communication systems (VICS (Vehicle Information and Communication System) (registered trademark)) such as radio beacons, optical beacons, and FM multiplex broadcasting.

[0111] Furthermore, the communication capabilities of the communication unit 22 with the vehicle will be briefly explained. The communication unit 22 can communicate with various devices in the vehicle, for example, using wireless communication. The communication unit 22 can communicate wirelessly with devices in the vehicle using communication methods that enable digital bidirectional communication at a predetermined or higher communication speed via wireless communication, such as Wi-Fi, Bluetooth, NFC, and WUSB (Wireless USB). Not limited to these, the communication unit 22 can also communicate with various devices in the vehicle using wired communication. For example, the communication unit 22 can communicate with various devices in the vehicle via wired communication through a cable connected to a connection terminal (not shown). The communication unit 22 can communicate with various devices in the vehicle using communication methods that enable digital bidirectional communication at a predetermined or higher communication speed via wired communication, such as USB (Universal Serial Bus), HDMI (High-Definition Multimedia Interface) (registered trademark), and MHL (Mobile High-definition Link).

[0112] Here, "devices inside the vehicle" refers to, for example, devices inside the vehicle that are not connected to the communication network 41. Examples of devices inside the vehicle include mobile devices and wearable devices carried by passengers such as the driver, and information devices that are brought into the vehicle and temporarily installed.

[0113] The map information storage unit 23 can store either or both maps acquired from external sources and maps created by the vehicle 1. For example, the map information storage unit 23 can store three-dimensional high-precision maps, global maps with lower precision than high-precision maps but covering a wide area, and so on.

[0114] High-precision maps include, for example, dynamic maps, point cloud maps, and vector maps. A dynamic map is, for example, a map consisting of four layers: dynamic information, semi-dynamic information, semi-static information, and static information, and is provided to vehicle 1 from an external server. A point cloud map is a map composed of point clouds (point cloud data). A vector map is, for example, a map that maps traffic information such as the location of lanes and traffic lights to a point cloud map, making it suitable for ADAS (Advanced Driver Assistance System) and AD (Autonomous Driving).

[0115] The point cloud map and vector map may be provided from, for example, an external server, or they may be created in the vehicle 1 as maps for matching with the local map described later, based on sensing results from the camera 51, radar 52, LiDAR 53, etc., and stored in the map information storage unit 23. In addition, if high-precision maps are provided from an external server, in order to reduce communication capacity, map data of, for example, several hundred meters square, relating to the planned route that the vehicle 1 will travel will be acquired from the external server.

[0116] The location information acquisition unit 24 can receive GNSS (Global Navigation Satellite System) signals from GNSS satellites and acquire location information of the vehicle 1. The acquired location information is supplied to the driving support / automatic driving control unit 29. The location information acquisition unit 24 is not limited to using GNSS signals; for example, it may acquire location information using beacons.

[0117] The external recognition sensor 25 has various sensors used to recognize the external conditions of the vehicle 1, and can supply sensor data from each sensor to various parts of the vehicle control system 11. The types and number of sensors that the external recognition sensor 25 has are not particularly limited.

[0118] For example, the external recognition sensor 25 includes a camera 51, a radar 52, a LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) 53, and an ultrasonic sensor 54. However, the external recognition sensor 25 may have one or more of the following sensors: camera 51, radar 52, LiDAR 53, and ultrasonic sensor 54. The number of cameras 51, radar 52, LiDAR 53, and ultrasonic sensors 54 is not particularly limited as long as it is a number that can be realistically installed in the vehicle 1. Furthermore, the types of sensors included in the external recognition sensor 25 are not limited to this example, and the external recognition sensor 25 may have other types of sensors. Examples of the sensing areas of each sensor included in the external recognition sensor 25 will be described later.

[0119] The shooting method of camera 51 is not particularly limited. For example, various types of cameras capable of distance measurement, such as ToF (Time of Flight) cameras, stereo cameras, monocular cameras, and infrared cameras, can be applied to camera 51 as needed. However, camera 51 may also be used simply for acquiring images, regardless of distance measurement.

[0120] Furthermore, for example, the external recognition sensor 25 may have an environmental sensor for detecting the environment relative to the vehicle 1. The environmental sensor is a sensor for detecting the environment such as weather, climate, and brightness, and may include various sensors such as a raindrop sensor, fog sensor, sunshine sensor, snow sensor, and illuminance sensor.

[0121] Furthermore, for example, the external recognition sensor 25 has a microphone used for detecting sounds around the vehicle 1 and the location of sound sources.

[0122] The in-vehicle sensor 26 has various sensors for detecting information inside the vehicle and can supply sensor data from each sensor to various parts of the vehicle control system 11. The types and number of sensors provided by the in-vehicle sensor 26 are not particularly limited as long as they are types and numbers that can realistically be installed in the vehicle 1.

[0123] For example, the in-vehicle sensor 26 may have one or more sensors from among a camera, radar, seat sensor, steering wheel sensor, microphone, and biosensor. The camera included in the in-vehicle sensor 26 may be a camera of various imaging types capable of distance measurement, such as a ToF camera, stereo camera, monocular camera, or infrared camera. However, it is not limited to these, and the camera included in the in-vehicle sensor 26 may simply be for acquiring images, regardless of distance measurement. The biosensor included in the in-vehicle sensor 26 may be installed, for example, on the seat or steering wheel, to detect various biometric information of the driver or other passengers.

[0124] The vehicle sensor 27 has various sensors for detecting the state of the vehicle 1 and can supply sensor data from each sensor to various parts of the vehicle control system 11. The types and number of sensors provided by the vehicle sensor 27 are not particularly limited as long as they are of a type and number that can be realistically installed on the vehicle 1.

[0125] For example, the vehicle sensor 27 may include a speed sensor, an acceleration sensor, an angular velocity sensor (gyro sensor), and an inertial measurement unit (IMU) that integrates them. For example, the vehicle sensor 27 may include a steering angle sensor for detecting the steering angle of the steering wheel, a yaw rate sensor, an accelerator sensor for detecting the amount of operation of the accelerator pedal, and a brake sensor for detecting the amount of operation of the brake pedal. For example, the vehicle sensor 27 may include a rotation sensor for detecting the rotation speed of the engine or motor, an air pressure sensor for detecting the air pressure of the tires, a slip ratio sensor for detecting the slip ratio of the tires, and a wheel speed sensor for detecting the rotation speed of the wheels. For example, the vehicle sensor 27 may include a battery sensor for detecting the remaining charge and temperature of the battery, and an impact sensor for detecting external impacts.

[0126] The storage unit 28 includes at least one of a non-volatile storage medium and a volatile storage medium, and can store data and programs. The storage unit 28 can be used as, for example, an EEPROM (Electrically Erasable Programmable Read Only Memory) and a RAM (Random Access Memory), and the storage medium can be a magnetic storage device such as an HDD (Hard Disk Drive), a semiconductor storage device, an optical storage device, or a magneto-optical storage device. The storage unit 28 stores various programs and data used by each part of the vehicle control system 11. For example, the storage unit 28 has an EDR (Event Data Recorder) and a DSSAD (Data Storage System for Automated Driving), and stores information about the vehicle 1 before and after an event such as an accident, and information acquired by the in-vehicle sensors 26.

[0127] The driving assistance / automatic driving control unit 29 can provide driving assistance and control the automatic driving of the vehicle 1. For example, the driving assistance / automatic driving control unit 29 includes an analysis unit 61, an action planning unit 62, and an operation control unit 63.

[0128] The analysis unit 61 can perform analysis processing on the vehicle 1 and its surroundings. The analysis unit 61 includes a self-position estimation unit 71, a sensor fusion unit 72, and a recognition unit 73.

[0129] The self-position estimation unit 71 can estimate the vehicle 1's own position based on sensor data from the external recognition sensor 25 and a high-precision map stored in the map information storage unit 23. For example, the self-position estimation unit 71 generates a local map based on sensor data from the external recognition sensor 25 and estimates the vehicle 1's own position by matching the local map with the high-precision map. The position of the vehicle 1 can be based on, for example, the center of the rear wheel relative to the axle.

[0130] Local maps are, for example, three-dimensional high-precision maps created using technologies such as SLAM (Simultaneous Localization and Mapping), or occupancy grid maps. Three-dimensional high-precision maps are, for example, the point cloud maps mentioned above. Occupancy grid maps divide the three-dimensional or two-dimensional space around vehicle 1 into grids of a predetermined size and show the occupancy status of objects on a grid-by-grid basis. The occupancy status of objects is indicated, for example, by the presence or absence of an object or the probability of its existence. Local maps are also used, for example, in the detection and recognition processing of the external conditions of vehicle 1 by the recognition unit 73.

[0131] The self-position estimation unit 71 may estimate the vehicle 1's own position based on the position information acquired by the position information acquisition unit 24 and the sensor data from the vehicle sensor 27.

[0132] The sensor fusion unit 72 can perform sensor fusion processing to obtain new information by combining multiple different types of sensor data (for example, image data supplied from the camera 51 and sensor data supplied from the radar 52). Methods for combining different types of sensor data include integration, fusion, and union.

[0133] The recognition unit 73 can perform detection processing to detect the external conditions of the vehicle 1, and recognition processing to recognize the external conditions of the vehicle 1.

[0134] For example, the recognition unit 73 performs detection and recognition processing of the external conditions of the vehicle 1 based on information from the external recognition sensor 25, information from the self-position estimation unit 71, information from the sensor fusion unit 72, etc.

[0135] Specifically, for example, the recognition unit 73 performs detection and recognition processing of objects around the vehicle 1. Object detection processing includes, for example, detecting the presence, size, shape, position, and movement of objects. Object recognition processing includes, for example, recognizing attributes such as the type of object or identifying a specific object. However, detection processing and recognition processing are not necessarily clearly separated and may overlap.

[0136] For example, the recognition unit 73 detects objects around the vehicle 1 by performing clustering, which classifies the point cloud based on sensor data from the radar 52 or LiDAR 53 into clusters of points. This allows the presence, size, shape, and position of objects around the vehicle 1 to be detected.

[0137] For example, the recognition unit 73 detects the movement of objects around the vehicle 1 by performing tracking that follows the movement of clusters of points classified by clustering. This allows the velocity and direction of travel (movement vector) of objects around the vehicle 1 to be detected.

[0138] For example, the recognition unit 73 detects or recognizes vehicles, people, bicycles, obstacles, structures, roads, traffic lights, traffic signs, road markings, etc., based on image data supplied from the camera 51. The recognition unit 73 may also recognize the types of objects around the vehicle 1 by performing recognition processing such as semantic segmentation.

[0139] For example, the recognition unit 73 can perform recognition processing of traffic rules around the vehicle 1 based on the map stored in the map information storage unit 23, the self-position estimation result by the self-position estimation unit 71, and the recognition result of objects around the vehicle 1 by the recognition unit 73. Through this processing, the recognition unit 73 can recognize the location and status of traffic lights, the content of traffic signs and road markings, the content of traffic regulations, and the lanes that can be driven on.

[0140] For example, the recognition unit 73 can perform recognition processing of the environment surrounding the vehicle 1. The surrounding environment that the recognition unit 73 is intended to recognize may include weather, temperature, humidity, brightness, and road surface conditions.

[0141] The action planning unit 62 creates an action plan for vehicle 1. For example, the action planning unit 62 can create an action plan by performing route planning and route following processes.

[0142] Global path planning is the process of planning the general route from the start to the finish line. This path planning also includes a process called local path planning, which involves generating a track that allows vehicle 1 to move safely and smoothly in its vicinity, taking into account the vehicle's motion characteristics along the planned route.

[0143] Route following is the process of planning actions to safely and accurately travel along the route planned by the route planner within the planned time. The action planning unit 62 can, for example, calculate the target speed and target angular velocity of vehicle 1 based on the results of this route following process.

[0144] The motion control unit 63 can control the operation of the vehicle 1 in order to realize the action plan created by the action planning unit 62.

[0145] For example, the motion control unit 63 controls the steering control unit 81, brake control unit 82, and drive control unit 83, which are included in the vehicle control unit 32 described later, to perform acceleration / deceleration control and direction control so that the vehicle 1 moves along the trajectory calculated by the trajectory plan. For example, the motion control unit 63 performs coordinated control for the purpose of realizing ADAS functions such as collision avoidance or impact mitigation, follow driving, vehicle speed maintenance driving, collision warning of the vehicle, and lane departure warning of the vehicle. For example, the motion control unit 63 performs coordinated control for the purpose of autonomous driving, such as driving autonomously without driver operation.

[0146] The DMS30 can perform driver authentication and driver status recognition based on sensor data from the in-vehicle sensors 26 and input data input to the HMI31, which will be described later. Examples of driver status to be recognized include physical condition, alertness level, concentration level, fatigue level, gaze direction, intoxication level, driving operation, and posture.

[0147] Furthermore, the DMS30 may perform authentication processing for passengers other than the driver and recognition processing for the status of said passengers. Also, for example, the DMS30 may perform recognition processing of the conditions inside the vehicle based on sensor data from the in-vehicle sensor 26. Examples of conditions inside the vehicle to be recognized include temperature, humidity, brightness, and odor.

[0148] The HMI31 can input various types of data and instructions, and display various types of data to the driver or other personnel.

[0149] A brief explanation of data input by HMI31 is provided. HMI31 has an input device for human data input. HMI31 generates input signals based on data and instructions input by the input device and supplies them to various parts of the vehicle control system 11. HMI31 has operators such as touch panels, buttons, switches, and levers as input devices. However, HMI31 may also have input devices that allow information to be input by methods other than manual operation, such as voice or gestures. Furthermore, HMI31 may use external connected devices such as a remote control device using infrared or radio waves, or a mobile device or wearable device that corresponds to the operation of the vehicle control system 11, as input devices.

[0150] This section provides a brief overview of how HMI31 presents data. HMI31 generates visual, auditory, and tactile information for the occupant or those outside the vehicle. HMI31 also performs output control, managing the output, content, timing, and method of each generated piece of information. As visual information, HMI31 generates and outputs information indicated by images and light, such as operation screens, vehicle status displays, warning displays, and monitor images showing the surroundings of vehicle 1. As auditory information, HMI31 generates and outputs information indicated by sound, such as voice guidance, warning sounds, and warning messages. Furthermore, as tactile information, HMI31 generates and outputs information that is perceived by the occupant's sense of touch through force, vibration, movement, etc.

[0151] As output devices for visual information output by HMI31, for example, a display device that presents visual information by displaying images itself, or a projector device that presents visual information by projecting images, can be applied. In addition to display devices with ordinary displays, the display device may also be a device that displays visual information within the passenger's field of view, such as a head-up display, a transparent display, or a wearable device with AR (Augmented Reality) functionality. Furthermore, HMI31 can also use display devices such as navigation devices, instrument panels, CMS (Camera Monitoring System), electronic mirrors, and lamps installed in the vehicle 1 as output devices for visual information output.

[0152] For HMI31, output devices that output auditory information can include, for example, audio speakers, headphones, and earphones.

[0153] As an output device for HMI31 to output tactile information, for example, a haptic element using haptic technology can be applied. The haptic element is installed in parts of the vehicle 1 that are in contact with by the occupant, such as the steering wheel and the seat.

[0154] The vehicle control unit 32 can control various parts of the vehicle 1. The vehicle control unit 32 includes a steering control unit 81, a brake control unit 82, a drive control unit 83, a body system control unit 84, a light control unit 85, and a horn control unit 86.

[0155] The steering control unit 81 can detect and control the state of the steering system of the vehicle 1. The steering system includes, for example, a steering mechanism including a steering wheel, electric power steering, etc. The steering control unit 81 includes, for example, a steering ECU that controls the steering system, an actuator that drives the steering system, etc.

[0156] The brake control unit 82 can detect and control the state of the brake system of the vehicle 1. The brake system includes, for example, a brake mechanism including a brake pedal, an ABS (Antilock Brake System), a regenerative braking mechanism, etc. The brake control unit 82 includes, for example, a brake ECU that controls the brake system, an actuator that drives the brake system, etc.

[0157] The drive control unit 83 can detect and control the state of the vehicle 1's drive system. The drive system includes, for example, an accelerator pedal, a drive force generating device for generating driving force such as an internal combustion engine or a drive motor, and a drive force transmission mechanism for transmitting driving force to the wheels. The drive control unit 83 includes, for example, a drive ECU for controlling the drive system and an actuator for driving the drive system.

[0158] The body system control unit 84 can detect and control the state of the body system of the vehicle 1. The body system includes, for example, a keyless entry system, a smart key system, power window devices, power seats, air conditioning devices, airbags, seat belts, a shift lever, etc. The body system control unit 84 includes, for example, a body system ECU that controls the body system, and actuators that drive the body system.

[0159] The light control unit 85 can detect and control the state of various lights on the vehicle 1. Examples of lights to be controlled include headlights, taillights, fog lights, turn signals, brake lights, projection lights, bumper displays, etc. The light control unit 85 includes a light ECU that controls the lights, actuators that drive the lights, etc.

[0160] The horn control unit 86 can detect and control the state of the car horn of the vehicle 1. The horn control unit 86 includes, for example, a horn ECU that controls the car horn, an actuator that drives the car horn, and so on.

[0161] Figure 22 shows an example of the sensing area of ​​the external recognition sensor 25 in Figure 21, including the camera 51, radar 52, LiDAR 53, and ultrasonic sensor 54. In Figure 22, the vehicle 1 is schematically shown as viewed from above, with the left end being the front end of the vehicle 1 and the right end being the rear end of the vehicle 1.

[0162] Sensing regions 101F and 101B show examples of sensing regions of the ultrasonic sensor 54. Sensing region 101F covers the area around the front end of the vehicle 1 by multiple ultrasonic sensors 54. Sensing region 101B covers the area around the rear end of the vehicle 1 by multiple ultrasonic sensors 54.

[0163] The sensing results in sensing area 101F and sensing area 101B are used, for example, for parking assistance of vehicle 1.

[0164] Sensing areas 102F to 102B show examples of sensing areas for short-range or medium-range radar 52. Sensing area 102F covers a position further in front of vehicle 1 than sensing area 101F. Sensing area 102B covers a position further in rear of vehicle 1 than sensing area 101B. Sensing area 102L covers the rear periphery of the left side of vehicle 1. Sensing area 102R covers the rear periphery of the right side of vehicle 1.

[0165] The sensing results in sensing region 102F are used, for example, to detect vehicles or pedestrians in front of vehicle 1. The sensing results in sensing region 102B are used, for example, to prevent collisions behind vehicle 1. The sensing results in sensing regions 102L and 102R are used, for example, to detect objects in blind spots to the sides of vehicle 1.

[0166] Sensing areas 103F to 103B show examples of sensing areas by camera 51. Sensing area 103F covers a position further in front of vehicle 1 than sensing area 102F. Sensing area 103B covers a position further in rear of vehicle 1 than sensing area 102B. Sensing area 103L covers the periphery of the left side of vehicle 1. Sensing area 103R covers the periphery of the right side of vehicle 1.

[0167] The sensing results in sensing region 103F can be used, for example, for recognition of traffic lights and traffic signs, lane departure prevention support systems, and automatic headlight control systems. The sensing results in sensing region 103B can be used, for example, for parking assistance and surround view systems. The sensing results in sensing regions 103L and 103R can be used, for example, for surround view systems.

[0168] Sensing area 104 shows an example of the sensing area of ​​LiDAR 53. Sensing area 104 covers a position further in front of vehicle 1 than sensing area 103F. On the other hand, sensing area 104 has a narrower range in the left-right direction than sensing area 103F.

[0169] The sensing results in the sensing region 104 can be used, for example, to detect objects such as surrounding vehicles.

[0170] Sensing area 105 shows an example of the sensing area of ​​the long-range radar 52. Sensing area 105 covers a position further in front of vehicle 1 than sensing area 104. On the other hand, sensing area 105 has a narrower range in the left-right direction than sensing area 104.

[0171] The sensing results in sensing area 105 are used, for example, for ACC (Adaptive Cruise Control), emergency braking, collision avoidance, etc.

[0172] Furthermore, the sensing areas of the camera 51, radar 52, LiDAR 53, and ultrasonic sensor 54 included in the external recognition sensor 25 may take various configurations other than those shown in Figure 22. Specifically, the ultrasonic sensor 54 may also sense the sides of the vehicle 1, or the LiDAR 53 may be configured to sense the rear of the vehicle 1. In addition, the installation positions of each sensor are not limited to the examples described above. Also, there may be one or more sensors.

[0173] The technology disclosed herein can be applied to, for example, a LiDAR 53. For example, by applying the technology disclosed herein to the LiDAR 53 of the vehicle control system 11, it becomes possible to easily detect malfunctions in the LiDAR 53, thereby preventing malfunctions and false detections of the LiDAR 53. Consequently, it becomes possible to properly detect surrounding vehicles and the like by the LiDAR 53, thereby ensuring the safety of the vehicle 1's operation.

[0174] <<5. Supplement>> While preferred embodiments of the present disclosure have been described in detail above with reference to the attached drawings, the technical scope of the present disclosure is not limited to such examples. It is clear to any person with ordinary skill in the art of the present disclosure that various modifications or alterations may be conceived within the scope of the technical ideas described in the claims, and these will naturally also fall within the technical scope of the present disclosure.

[0175] Furthermore, the effects described herein are merely descriptive or illustrative and not limiting. In other words, the technology relating to this disclosure may produce other effects that will be apparent to those skilled in the art from the description herein, in addition to or in lieu of the effects described herein.

[0176] Furthermore, this technology can also be configured as follows. (1) A light-receiving unit including a pixel array unit consisting of multiple light-receiving elements arranged in a matrix, A control signal generation unit that generates control signals, A control unit is electrically connected to the light receiving unit via a control line and controls the light receiving unit based on the control signal, A processing unit is electrically connected to the light receiving unit via a signal line and processes the output signal from the light receiving unit, A fault detection unit that detects faults, Equipped with, The fault determination unit detects a fault in the control line based on the output signal from the light receiving unit, which is controlled based on a fault detection control signal having a predetermined pattern. Light detection device. (2) The photodetector described in (1) above is an avalanche photodiode. (3) The light receiving unit is A drive switch for driving the light-receiving element is provided for each of the light-receiving elements, An output switch is provided for each of the aforementioned light-receiving elements to control the output of the output signal from the light-receiving element, A theoretical sum circuit that outputs according to the output signals from each of the aforementioned photodetectors and the output signals from other photodetectors, The photodetector described in (1) or (2) above, further comprising: (4) The control unit, The control unit, A horizontal control unit that controls the light-receiving elements in rows, A vertical control unit that controls the light-receiving elements on a row-by-row basis, The light detection device described in (3) above, including the above. (5) The aforementioned control line is A first control line electrically connects the vertical control unit and the drive switch, A second control line electrically connects the vertical control unit and the output switch, A third control line electrically connects the horizontal control unit and the drive switch, A fourth control line electrically connects the horizontal control unit and the output switch, A fifth control line electrically connects the vertical control unit and the theoretical sum circuit, The light detection device described in (4) above, including the above. (6) The processing unit is the photodetector according to (5) above, which includes a downsampling circuit. (7) The optical detection device according to (6) above, wherein the control signal generation unit outputs the fault detection control signal while the frame synchronization signal is inactive. (8) The optical detection device according to (7) above, wherein the fault detection control signal includes a processing unit fault detection signal for detecting a fault in the processing unit, followed by a control line fault detection signal for detecting a fault in the control line. (9) The light detection device according to (8) above, wherein the fault detection control signal includes a light receiving unit fault detection signal for detecting a fault in the light receiving unit before or after the control line fault detection signal. (10) The optical detection device according to (9) above, wherein the control line fault detection signal includes a first signal for detecting a fault in the fifth control line. (11) The optical detection device according to (10) above, wherein the control line fault detection signal includes a second signal for detecting faults in the second and fourth control lines. (12) The optical detection device according to (11) above, wherein the control line fault detection signal includes a third signal for detecting faults in the first and third control lines. (13) The optical detection device according to (12) above, wherein the control line fault detection signal includes a fourth signal for detecting a fault in the downsampling circuit. (14) The optical detection device according to (13) above, wherein the control line fault detection signal includes the first signal, the second signal, the third signal, and the fourth signal in that order. (15) The optical detection device according to any one of (1) to (14) above, wherein the fault determination unit outputs a predetermined signal to an output terminal or a host computer when a fault is detected. (16) A first substrate on which the pixel array portion is provided, A second substrate is laminated on the first substrate and is provided with the control unit, the processing unit, and the fault determination unit, A light detection device according to any one of (1) to (15) above, formed from the above. (17) The control line is provided on the second substrate, and is the light detection device described in (16) above. (18) A lighting device that emits light, A light detection device that receives reflected light reflected by the subject, Includes, The aforementioned light detection device is A light-receiving unit including a pixel array unit consisting of multiple light-receiving elements arranged in a matrix, A control signal generation unit that generates control signals, A control unit is electrically connected to the light receiving unit via a control line and controls the light receiving unit based on the control signal, A processing unit is electrically connected to the light receiving unit via a signal line and processes the output signal from the light receiving unit, A fault detection unit that detects faults, It has, The fault determination unit detects a fault in the control line based on the output signal from the light receiving unit, which is controlled based on a fault detection control signal having a predetermined pattern. Distancing system. [Explanation of Symbols]

[0177] 1 vehicle 11. Vehicle control system 21. Vehicle Control ECU (Electronic Control Unit) 22 Communications Department 23 Map Information Storage Unit 24 Location information acquisition section 25 External recognition sensors 26 In-car sensors 27 Vehicle Sensors 28 Memory section 29. Driving Assistance / Automated Driving Control Unit 30. Driver Monitoring System (DMS) 31. Human-Machine Interface (HMI) 32 Vehicle Control Unit 41 Communication Networks 51 Camera 52 Radar 53 LiDAR 54 Ultrasonic Sensors 61 Analysis Department 62 Action Planning Department 63 Operation Control Unit 71 Self-position estimation part 72 Sensor Fusion Unit 73 Recognition part 81 Steering Control Unit 82 Brake Control Unit 83 Drive control unit 84 Body System Control Unit 85 Light Control Unit 86 Horn Control Unit 90 Distancing System 200, 250 semiconductor substrates 252 Logic Array Section 300, 500 range finder 301, 400 Light source section 302, 502 Light receiving section 303 Object to be measured 310 Frequency 311 Range 312 Active Light Components 510-pixel array section 512 SPAD 530 Processing Unit 532 Downsampling Circuit 534 Failure determination section 540 Conversion Unit 550 Generation part 560 Signal Processing Unit 570 Control Unit 572 Vertical Control Unit 574 Horizontal Control Unit 576 Control signal generation unit 580 Light emission timing control unit 590 Interface 600 storage device 700 hosts 800 Optical system 900-pixel circuit 902, 904 transistors 910 constant current source 920, 940 switch circuits 930 Inverter Circuit 950 OR circuit R100, R101 area

Claims

1. A light-receiving unit including a pixel array unit consisting of multiple light-receiving elements arranged in a matrix, A control signal generation unit that generates control signals, A control unit is electrically connected to the light receiving unit via a control line and controls the light receiving unit based on the control signal, A processing unit is electrically connected to the light receiving unit via a signal line and processes the output signal from the light receiving unit, A fault detection unit that detects faults, Equipped with, The fault determination unit detects a fault in the control line based on the output signal from the light receiving unit, which is controlled based on a fault detection control signal having a predetermined pattern. The light receiving unit is A drive switch for driving the light-receiving element is provided for each of the light-receiving elements, An output switch is provided for each of the aforementioned light-receiving elements to control the output of the output signal from the light-receiving element, A theoretical sum circuit that outputs according to the output signals from each of the aforementioned photodetectors and the output signals from other photodetectors, Includes, The control unit, A horizontal control unit that controls the light-receiving elements in rows, A vertical control unit that controls the light-receiving elements on a row-by-row basis, Includes, The aforementioned control line is A first control line electrically connects the vertical control unit and the drive switch, A second control line electrically connects the vertical control unit and the output switch, A third control line electrically connects the horizontal control unit and the drive switch, A fourth control line electrically connects the horizontal control unit and the output switch, A fifth control line electrically connects the vertical control unit and the theoretical sum circuit, including, Light detection device.

2. The light-receiving element is an avalanche photodiode, as described in claim 1.

3. The photodetector according to claim 1 or 2, wherein the processing unit includes a downsampling circuit.

4. The optical detection device according to claim 3, wherein the control signal generation unit outputs the fault detection control signal while the frame synchronization signal is inactive.

5. The optical detection device according to claim 4, wherein the fault detection control signal includes a processing unit fault detection signal for detecting a fault in the processing unit, followed by a control line fault detection signal for detecting a fault in the control line.

6. The light detection device according to claim 5, wherein the fault detection control signal includes a light receiving unit fault detection signal for detecting a fault in the light receiving unit, either before or after the control line fault detection signal.

7. The optical detection device according to claim 6, wherein the control line fault detection signal includes a first signal for detecting a fault in the fifth control line.

8. The optical detection device according to claim 7, wherein the control line fault detection signal includes a second signal for detecting faults in the second and fourth control lines.

9. The optical detection device according to claim 8, wherein the control line fault detection signal includes a third signal for detecting faults in the first and third control lines.

10. The optical detection device according to claim 9, wherein the control line fault detection signal includes a fourth signal for detecting a fault in the downsampling circuit.

11. The optical detection device according to claim 10, wherein the control line fault detection signal includes the first signal, the second signal, the third signal, and the fourth signal in that order.

12. The light detection device according to any one of claims 1 to 11, wherein the fault determination unit outputs a predetermined signal to an output terminal or a host computer when it detects a fault.

13. A first substrate on which the pixel array portion is provided, A second substrate is laminated on the first substrate and is provided with the control unit, the processing unit, and the fault determination unit, A light detection device according to any one of claims 1 to 12, formed from the above.

14. The light detection device according to claim 13, wherein the control line is provided on the second substrate.

15. A lighting device that emits light, A light detection device that receives reflected light reflected by the subject, Includes, The aforementioned light detection device is A light-receiving unit including a pixel array unit consisting of multiple light-receiving elements arranged in a matrix, A control signal generation unit that generates control signals, A control unit is electrically connected to the light receiving unit via a control line and controls the light receiving unit based on the control signal, A processing unit is electrically connected to the light receiving unit via a signal line and processes the output signal from the light receiving unit, A fault detection unit that detects faults, It has, The fault determination unit detects a fault in the control line based on the output signal from the light receiving unit, which is controlled based on a fault detection control signal having a predetermined pattern. The light receiving unit is A drive switch for driving the light-receiving element is provided for each of the light-receiving elements, An output switch is provided for each of the aforementioned light-receiving elements to control the output of the output signal from the light-receiving element, A theoretical sum circuit that outputs according to the output signals from each of the aforementioned photodetectors and the output signals from other photodetectors, Includes, The control unit, A horizontal control unit that controls the light-receiving elements in rows, A vertical control unit that controls the light-receiving elements on a row-by-row basis, Includes, The aforementioned control line is A first control line electrically connects the vertical control unit and the drive switch, A second control line electrically connects the vertical control unit and the output switch, A third control line electrically connects the horizontal control unit and the drive switch, A fourth control line electrically connects the horizontal control unit and the output switch, A fifth control line electrically connects the vertical control unit and the theoretical sum circuit, including, Distancing system.

Images