Control device, distance measurement device, and control method

Abstract

In the present invention, a control unit is a control device that detects echoes of a pulse laser beam on the basis of physical quantities of photons that are respectively incident on a plurality of SPAD pixels that are arranged. The control unit comprises an invalid interval determination unit and an SPAD control unit. When a saturation echo, which is an echo saturated for a predetermined time or more, is detected, the invalidation interval determination unit determines an SPAD invalidation interval that invalidates the SPAD pixel in which the saturation echo is detected, such that a start time of the SPAD invalidation interval is set at a start time of the saturation echo, and the SPAD invalidation interval gradually decreases each time the laser light is emitted. The SPAD control unit invalidates the SPAD pixel in accordance with the determined SPAD invalidation interval, and irradiates the SPAD pixel with a laser light.

Description

Technical Field

[0001] This disclosure relates to control devices, distance measuring devices, and control methods. Background Technology

[0002] In recent years, distance image sensors (hereinafter also referred to as ToF sensors) that measure distance using the time-of-flight (ToF) method have attracted attention. For example, there are ToF sensors that are fabricated using complementary metal-oxide-semiconductor (CMOS) semiconductor integrated circuit technology and that measure the distance to an object using multiple planar arranged single-photon avalanche diodes (SPADs).

[0003] In a Time-of-Flight (ToF) sensor using a SPAD, the time from when light is emitted by the light source to when the reflected light (hereinafter referred to as the echo) is incident on the SPAD (hereinafter referred to as the time of flight) is measured multiple times as a physical quantity. The distance to the object is then specified based on a histogram of the physical quantities generated from the measurement results.

[0004] Citation List

[0005] Patent documents

[0006] Patent Document 1: JP2016-533140A Summary of the Invention

[0007] Technical issues

[0008] However, in ToF sensors using SPADs, while the first wave can be observed when consecutive echoes are incident, subsequent waves may not be observable. This is because the SPAD saturates due to the first wave and may exhibit a spurious response independent of incident light for a period of time. This dead-time period is called the "dead time." Therefore, ToF sensors using SPADs have the problem of not being able to observe the echoes incident during the dead time.

[0009] Therefore, this disclosure provides a control device, a distance measuring device, and a control method that enable the observation of echoes that cannot be observed during the dead time.

[0010] Solution to the problem

[0011] To address the aforementioned problems, one aspect of the control device according to this disclosure detects the echo relative to pulsed emitted light based on the physical quantity of photons incident on each of a plurality of arranged pixels. The control device includes: a determining unit that, upon detecting a saturated echo in which the echo has saturated for a predetermined time or longer, determines an invalid interval such that the start time point of the invalid interval that invalidates pixels where saturated echoes have been detected is set as the start time point of the saturated echo, and the invalid interval gradually decreases with each emission of emitted light; and a pixel control unit that invalidates pixels based on the determined invalid interval and causes the emitted light to be emitted. Attached Figure Description

[0012] Figure 1 This is a block diagram illustrating a schematic configuration example of a ToF sensor as a distance measuring device according to an embodiment of the present disclosure.

[0013] Figure 2 This is an illustration of an optical system including a ToF sensor according to an embodiment of the present disclosure.

[0014] Figure 3 This is a block diagram illustrating a schematic configuration example of an optical receiving unit according to an embodiment of the present disclosure.

[0015] Figure 4 This is an illustrative diagram illustrating an example of observation data in a ToF sensor according to an embodiment of this disclosure.

[0016] Figure 5 This is an explanatory diagram of the data structure for observation.

[0017] Figure 6 This is a diagram illustrating dead time.

[0018] Figure 7 This is an explanatory diagram of a control method according to an embodiment of the present disclosure.

[0019] Figure 8 This is a block diagram illustrating an example configuration of a control unit according to an embodiment of the present disclosure.

[0020] Figure 9 It is a flowchart indicating the processing procedure performed by the control unit according to an embodiment of the present disclosure.

[0021] Figure 10 This is a block diagram illustrating a configuration example of the control unit according to the modified example.

[0022] Figure 11 This is an illustrative diagram illustrating the combination with glare removal treatment.

[0023] Figure 12 This is a hardware configuration diagram illustrating an example of a computer that implements the functions of the control unit.

[0024] Figure 13 This is a block diagram illustrating an example of a schematic configuration of a vehicle control system.

[0025] Figure 14 This is an explanatory diagram showing an example of the installation location of the vehicle external information detection unit and the imaging unit. Detailed Implementation

[0026] In the following, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Note that in the following embodiments, the same parts are indicated by the same reference numerals, and redundant descriptions will be omitted.

[0027] Furthermore, in the following description, it is assumed that the distance measuring device according to the embodiment of this disclosure (hereinafter appropriately referred to as "this embodiment") is... Figure 1 And the ToF sensor 1 shown in the following figures. Furthermore, it is assumed that the control device according to this embodiment is a control unit 11 included in the ToF sensor 1. Furthermore, in the following description, it is assumed that the control method according to this embodiment is a control method executed by the control unit 11.

[0028] Furthermore, this disclosure will be described in the following order of items.

[0029] 1. Summary

[0030] 1-1. Distance measuring device (ToF sensor)

[0031] 1-2. Optical System

[0032] 1-3. Optical receiving unit

[0033] 1-4. Examples of Observational Data

[0034] 1-5. Dead Time

[0035] 1-6. Overview of the control method according to this embodiment

[0036] 2. Configuration example of the control unit

[0037] 3. Processing procedure

[0038] 4. Modified Example

[0039] 4-1. Combination with glare removal treatment and its effects

[0040] 4-2. Determining the lengths of the SPAD invalid and valid intervals

[0041] 4-3. Ratio between SPAD invalid intervals and SPAD valid intervals

[0042] 4-4. Other

[0043] 5. Hardware Configuration

[0044] 6. Application Examples

[0045] 7. Conclusion

[0046] <<1. Summary>>

[0047] First, refer to Figures 1 to 7 A summary of this implementation method is provided.

[0048] <1-1. Distance Measurement Device (ToF Sensor)>

[0049] Figure 1 This is a block diagram illustrating a schematic configuration example of the ToF sensor 1, which is a distance measuring device according to this embodiment. Figure 1 As shown, the ToF sensor 1 includes a control unit 11, a storage unit 12, a light projection unit 13, a light receiving unit 14, and an external interface (I / F) 19.

[0050] The control unit 11 controls each unit of the ToF sensor 1. For example, the control unit 11 can be implemented by executing a program stored in the storage unit 12 according to this embodiment using random access memory (RAM) as the working area, such as a central processing unit (CPU), microprocessor unit (MPU), etc. Furthermore, the control unit 11 can be implemented by an integrated circuit, such as an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA).

[0051] Storage unit 12 is implemented by a storage device such as RAM, read-only memory (ROM) or flash memory.

[0052] The light projection unit 13 includes, for example, one or more semiconductor laser diodes as a light source. Under the control of the control unit 11, the light projection unit 13 emits pulsed laser light (also called emitted light) L1 with a predetermined time width at a predetermined period (also called emission period). Furthermore, the light projection unit 13 emits laser light L1 with a time width of 1 ns (nanosecond) at a period of 1 MHz (megahertz). For example, if an object 90 is present within the distance measurement range, the laser light L1 emitted from the light projection unit 13 is reflected by the object 90 and incident on the light receiving unit 14 as reflected light L2.

[0053] Although details will be described later, the light receiving unit 14 includes, for example, a plurality of SPAD pixels arranged in a two-dimensional lattice pattern. The light receiving unit 14 outputs information to the control unit 11 regarding the number of SPAD pixels that detected incident photons after emission by the light projection unit 13 (hereinafter referred to as the "detected number") (e.g., such a number corresponds to the number of detection signals to be described later). For example, the light receiving unit 14 detects incident photons for a single emission from the light projection unit 13 at a predetermined sampling period and outputs the detected number.

[0054] The control unit 11 aggregates the number of detections output from the light receiving unit 14 for each of a plurality of SPAD pixels (e.g., corresponding to one or more macro pixels to be described later), and generates a histogram based on the pixel values ​​obtained by aggregation, where the horizontal axis represents the time of flight and the vertical axis represents the cumulative pixel value. For example, the control unit 11 obtains pixel values ​​by aggregating the number detected at a predetermined sampling frequency for a single emission from the light projection unit 13, and repeats this operation for multiple emission from the light projection unit 13 to generate a histogram, where the horizontal axis (the bars of the histogram) represents the sampling period corresponding to the time of flight and the vertical axis represents the cumulative pixel value obtained by accumulating the pixel values ​​obtained in each sampling period.

[0055] Furthermore, after performing a predetermined filtering process on the generated histogram, the control unit 11 specifies the time of flight at which the accumulated pixel value reaches its peak based on the filtered histogram. Then, based on the specified time of flight, the control unit 11 calculates the distance from the ToF sensor 1 or a device equipped with the ToF sensor 1 to the object 90 present within the distance measurement range. Note that information regarding the distance calculated by the control unit 11 is output to the host 80, for example, via an external I / F 19.

[0056] External I / F 19 is, for example, a communication adapter for establishing communication with external host 80 via a communication network that conforms to any standard such as Controller Area Network (CAN), Local Area Network (LIN), or FlexRay (registered trademark), in addition to wireless local area network (LAN) and wired LAN.

[0057] For example, when the ToF sensor 1 is installed on a car or similar device, the host unit 80 is an electronic control unit (ECU) or similar device installed on the car or similar device. Furthermore, when the ToF sensor 1 is installed on an autonomous mobile robot such as a home pet robot, or on an autonomous mobile body such as a robot cleaner, unmanned aerial vehicle, or a follow-up delivery robot, the host unit 80 is, for example, a control device that controls the autonomous mobile body.

[0058] <1-2. Optical System>

[0059] Next, Figure 2 This is an explanatory diagram of the optical system including the ToF sensor 1 according to this embodiment. Note that in Figure 2 The example illustrates a so-called scanning optical system that scans the viewing angle of the light receiving unit 14 in the horizontal direction; however, this embodiment is not limited to this, and for example, a so-called flash optical system that fixes the viewing angle of the light receiving unit 14 can be used.

[0060] like Figure 2 As shown, the ToF sensor 1 includes a light source 31, a collimating lens 32, a semi-reflecting mirror 33, a current mirror 35, a light receiving sensor 36, and a light receiving lens 38 as an optical system. The light source 31, collimating lens 32, semi-reflecting mirror 33, and current mirror 35 are, for example, included in... Figure 1 The light projection unit 13 is included. Furthermore, the light receiving sensor 36 and the light receiving lens 38 are, for example, included in... Figure 1 In the optical receiving unit 14.

[0061] exist Figure 2 In the configuration shown, the laser L1 emitted from the light source 31 is converted by the collimating lens 32 into rectangular parallel light with a cross-sectional intensity spectrum that is long in the vertical direction, and then incident on the semi-reflecting mirror 33. The semi-reflecting mirror 33 reflects a portion of the incident laser L1. The laser L1 reflected by the semi-reflecting mirror 33 is then incident on the current mirror 35. For example, the current mirror 35 is vibrated in the horizontal direction about a predetermined rotation axis by a drive unit 34, which operates based on control from the control unit 11. Thus, the laser L1 is horizontally scanned, causing the viewing angle SR of the laser L1 reflected by the current mirror 35 to reciprocate in the horizontal direction within the distance measurement range AR. Note that a microelectromechanical system (MEMS), a micromotor, or the like can be used as the drive unit 34.

[0062] The laser L1 reflected by the current mirror 35 is reflected by an object 90 present within the distance measurement range AR, and is incident on the current mirror 35 as reflected light L2. A portion of the reflected light L2 incident on the current mirror 35 is transmitted through the semi-reflective mirror 33 and incident on the light receiving lens 38, thereby forming an image on the SPAD array 37 in the light receiving sensor 36. Note that the SPAD array 37 may be the entire light receiving sensor 36 or a part of the light receiving sensor 36.

[0063] <1-3. Optical Receiver Unit>

[0064] Next, Figure 3 This is a block diagram illustrating a schematic configuration example of the optical receiving unit 14 according to this embodiment. Figure 3As shown, the optical receiving unit 14 includes a SPAD array 37, a timing control circuit 43, a driving circuit 44, and an output circuit 45.

[0065] SPAD array 37 includes a plurality of SPAD pixels 20 arranged in a two-dimensional lattice pattern. Among the plurality of SPAD pixels 20, a pixel driving line LD (vertical direction in the figure) is connected for each column, and an output signal line LS (horizontal direction in the figure) is connected for each row. One end of the pixel driving line LD is connected to an output terminal corresponding to each column of the driving circuit 44, and one end of the output signal line LS is connected to an input terminal corresponding to each row of the output circuit 45.

[0066] In this embodiment, all or a portion of the SPAD array 37 is used to detect the reflected light L2. The area used in the SPAD array 37 can be a rectangle of the same vertical length as the image of the reflected light L2 formed on the light receiving sensor 36 when the entire laser L1 is reflected as reflected light L2. However, the area is not limited to this, and various modifications such as a larger or smaller area than the image of the reflected light L2 formed on the SPAD array 37 can be used.

[0067] The timing control circuit 43 includes a timing generator that generates various timing signals, and controls the drive circuit 44 and the output circuit 45 based on the various timing signals generated by the timing generator.

[0068] The driving circuit 44 includes a shift register, an address decoder, etc., and drives each of the SPAD pixels 20 in the SPAD array 37, for example, in units of columns including one or more SPAD pixels 20 or in units of macro pixels, or simultaneously drives all pixels. In this embodiment, the driving circuit 44 selects each of the SPAD pixels 20 to be driven based on the control of the control unit 11.

[0069] Note that in the following description, the control unit 11 causing the drive circuit 44 to select each of the SPAD pixels 20 to be driven can be expressed as the control unit 11 making the SPAD pixel 20 "enabled". On the other hand, the control unit 11 causing the drive circuit 44 to select each of the SPAD pixels 20 that are not driven can be expressed as the control unit 11 making the SPAD pixel 20 "disabled".

[0070] The control unit 11 enables any SPAD pixel 20 by turning on the enable signal from the SPAD pixel 20 to the drive circuit 44. Conversely, the control unit 11 disables any SPAD pixel 20 by turning off the enable signal from the SPAD pixel 20 to the drive circuit 44.

[0071] The detection signal output from each SPAD pixel 20 of the column selectively scanned by the drive circuit 44 is input to the output circuit 45 through each output signal line in the output signal line LS. The output circuit 45 outputs the detection signal input from each SPAD pixel 20 to the control unit 11.

[0072] <1-4. Examples of Observational Data>

[0073] Next, Figure 4 This is an explanatory diagram illustrating an example of observed data in a ToF sensor 1 according to an embodiment of this disclosure. Furthermore, Figure 5 This is an explanatory diagram of the data structure for observation.

[0074] like Figure 4 As shown, the detection signal input from the light receiving unit 14 to the control unit 11 is observed as a laser waveform in an orthogonal coordinate system with the X-axis as the horizontal direction, the Y-axis as the vertical direction, and the Z-axis as the time direction (also referred to as the "depth direction"). Based on this observation data, the control unit 11 sets the peak position of the brightness (number of detected photons) of the reflected light L2 of the laser L1 emitted from the light projection unit 13 and reflected by the object 90 present in the viewpoint of each SPAD pixel 20 to ToF, and performs distance conversion.

[0075] like Figure 5 As shown, the control unit 11 uses the SPAD array 37 to treat the data structure of the observed data as a brightness image (corresponding to...). Figure 4 It is processed by successive aggregation of mapping information on the XY plane. Note that... Figure 5 The number of points and the distance (m) values ​​indicated in the text are just examples.

[0076] <1-5. Dead Zone Time>

[0077] Incidentally, in existing ToF sensors, when continuous echoes are incident, the first wave can be observed, but there may be cases where the subsequent second wave cannot be observed. Figure 6 This is a diagram illustrating dead time.

[0078] like Figure 6 As shown, in a ToF sensor according to the prior art, when a first echo e-1 with a strong intensity exceeding the saturation exposure S of the SPAD pixel 20 is incident on the SPAD, the SPAD pixel 20 becomes saturated, and a dead time may occur, which is a pseudo-response independent of the incident light. The dead time increases with the intensity of the echo e-1.

[0079] Therefore, existing technologies have the problem of being unable to observe the echo e-2, which is the second wave incident during the dead time. That is, ToF sensors according to existing technologies cannot determine whether there is a potentially incident echo e-2 during the dead time, or whether only a strong echo e-1 is incident first. Note that in the following description, the echo potentially incident during the dead time will be appropriately referred to as a "potential echo".

[0080] <1-6. Overview of the control method according to this embodiment>

[0081] Therefore, in the control method according to this embodiment, the control unit 11 detects the echo relative to the laser L1 based on the physical quantity of photons incident on each of the plurality of SPAD pixels 20 arranged in the array. Furthermore, if a saturated echo is detected where the echo saturates for a predetermined time or longer, the control unit 11 determines that the SPAD invalidation interval for invalidating the SPAD pixel 20 that has detected the saturated echo is gradually reduced using the end time point of the saturated echo as a reference. More specifically, the start time point of the SPAD invalidation interval is set to the start time point of the saturated echo, and the SPAD invalidation interval gradually decreases each time the laser L1 is emitted. Furthermore, the control unit 11 invalidates the SPAD pixel 20 according to the determined SPAD invalidation interval and causes the laser L1 to be emitted. Figure 7 This is an explanatory diagram of a control method according to an embodiment of the present disclosure.

[0082] Specifically, such as Figure 7 As shown, in the control method according to this embodiment, the control unit 11 of the ToF sensor 1 first causes the light projection unit 13 to perform the first emission of laser L1, and performs normal reflected light measurement processing based on the reflected light L2. At this time, the control unit 11 controls the enable signal for all SPAD pixels 20 to be turned on, and enables all SPAD pixels 20.

[0083] Assuming an echo e-1 exceeding the saturation exposure S is incident on the first emission of laser L1, and as shown in the "SPAD response" diagram, the light receiving unit 14 outputs a detection signal to the control unit 11, indicating the laser waveform including the dead time. Therefore, the control unit 11 detects the echo e-1 saturated for a predetermined time or longer using standard reflected light measurement processing.

[0084] When an echo e-1 that has been saturated for a predetermined time or longer is detected, the control unit 11 performs a potential echo search process at the time points T1 and T2 when the echo e-1 is observed to search for the presence or absence of potential echoes.

[0085] In the potential echo search process, the control unit 11 determines the SPAD invalid interval starting from time point T1 at the time point T1 and time point T2 when the echo e-1 is observed. At this time, in the potential echo search process, the control unit 11 determines the SPAD invalid interval by changing the SPAD valid interval so that the SPAD invalid interval gradually decreases from the time point T2 side when laser L1 is emitted each time.

[0086] Then, the control unit 11 detects potential echoes in the SPAD effective range that have changed in this way. Note that in the potential echo search process, the control unit 11 dynamically controls the activation or deactivation of the SPAD, for example, on a per-SPAD pixel that has detected echo e-1 or on a per-macro-pixel basis that includes the corresponding SPAD pixel 20.

[0087] In latent echo search processing, for example, such as Figure 7 As shown, in the first potential echo search process, the control unit 11 disables the SPAD from time point T1 to the halfway point between time point T1 and time point T2, and enables the SPAD from the halfway point to time point T2. Then, the control unit 11 causes the light projection unit 13 to emit laser L1 once, and performs a reflected light measurement process on the reflected light L2.

[0088] Therefore, when no potential echo is detected, in the second potential echo search process, the control unit 11 disables the SPAD from time point T1 to the 1 / 4 time point between time point T1 and time point T2, and enables the SPAD from the 1 / 4 time point to time point T2. Afterwards, the control unit 11 causes the light projection unit 13 to emit laser L1 once, and performs reflected light measurement processing on the reflected light L2.

[0089] Then, as Figure 7 As shown, when the rise of echo e-2 can be detected as a potential echo in the second potential echo search process, the control unit 11 records echo e-2 separately from echo e-1. Furthermore, when no potential echo can be detected in the second potential echo search process, the control unit 11 repeats the potential echo search process in a manner similar to the first and second potential echo search processes. Note that the control unit 11 terminates the repetition of the potential echo search process when predetermined termination conditions are met. Predetermined termination conditions include situations such as the number of executions of the potential echo search process exceeding a specified value (first specified value) or the length of the invalid interval of the SPAD becoming equal to or less than a specified value (second specified value).

[0090] As described above, in the control method according to this embodiment, the control unit 11 detects the echo relative to the laser L1 based on the physical quantity of photons incident on each of the plurality of SPAD pixels 20 arranged in the array. Furthermore, if a saturated echo is detected where the echo saturates for a predetermined time or longer, the control unit 11 determines a SPAD invalidation interval, such that the start time point of the SPAD invalidation interval that invalidates the SPAD pixels 20 where saturated echoes have been detected is set as the start time point of the saturated echo, and the SPAD invalidation interval gradually decreases each time the laser L1 is emitted. Furthermore, the control unit 11 invalidates the SPAD pixels 20 according to the determined SPAD invalidation interval and causes the laser L1 to be emitted.

[0091] Therefore, according to the control method of this embodiment, echoes that cannot be observed during the dead time can be observed.

[0092] Furthermore, in the control method of this embodiment that targets only saturated echoes, the control unit 11 controls the activation / deactivation of SPAD pixels 20 that have detected the saturated echo. Therefore, according to the control method of this embodiment, the sensitivity to unsaturated echoes with weak intensity is not affected. As a result, the decrease in distance measurement accuracy of the ToF sensor 1 can be suppressed.

[0093] Furthermore, the control unit 11 can also perform glare removal processing, where highly reflective scattered light saturates the surrounding SPAD pixels 20 and is detected as false echoes. In this case, the control method of this embodiment can also prevent true echoes that are not observable due to glare from being removed by the glare removal processing. (See below for further details.) Figure 10 and Figure 11 This modified example will be described in more detail below. The following description will provide a more specific example of the configuration of the control unit 11 according to this embodiment.

[0094] <<2. Configuration Example of Control Unit>>

[0095] Figure 8 This is a block diagram illustrating an example configuration of the control unit 11 according to an embodiment of the present disclosure. Note that in the description below... Figure 8 and Figure 10 In this document, only the components necessary for describing the features of embodiments of the present disclosure are shown, and descriptions of general components are omitted.

[0096] In other words, Figure 8 and Figure 10Each of the components shown is conceptual in function and is not necessarily physically configured as illustrated in the figures. For example, the specific form of distribution and integration of each block is not limited to the form shown, and all or part of it may be functionally or physically distributed and integrated in arbitrary units according to various loads, usage conditions, etc.

[0097] In use Figure 8 and Figure 10 In the description, the description of the already described parts can be simplified or omitted.

[0098] like Figure 8 As shown, the control unit 11 includes a light projection control unit 11a, a histogram generation unit 11b, an echo detection unit 11c, an invalid interval determination unit 11d, and a SPAD control unit 11e, and implements or performs the information processing functions and actions described later.

[0099] Storage unit 12 includes an accumulation memory 12a and a lookup table (LUT) 12b. The accumulation memory 12a stores the aforementioned accumulated pixel values.

[0100] LUT 12b is a lookup table in which, during latent echo search processing, a fixed value is preset for the length of the effective SPAD interval, which extends retrospectively from the end time of observation of the saturated echo, each time a laser L1 is emitted. In LUT 12b, a fixed value can also be preset for the length of the ineffective SPAD interval, which gradually decreases from the start time of echo observation based on changes in the effective SPAD interval.

[0101] The light projection control unit 11a controls the light projection unit 13. The light projection control unit 11a causes the light projection unit 13 to emit pulsed lasers L1 with a predetermined time width at a predetermined period. Furthermore, in the latent echo search process, the light projection control unit 11a causes the light projection unit 13 to emit one laser L1 each time the SPAD invalid region is changed by the SPAD control unit 11e described later.

[0102] The histogram generation unit 11b aggregates the number of detections output from the light receiving unit 14 for each SPAD pixel 20 (e.g., one or more macro pixel units) and stores the accumulated pixel value obtained by aggregation in the accumulation memory 12a. Furthermore, the histogram generation unit 11b generates the aforementioned histogram based on the accumulated pixel value stored in the accumulation memory 12a.

[0103] The echo detection unit 11c detects echoes based on the histogram generated by the histogram generation unit 11b. Note that if there are saturated echoes among the detected echoes that have been saturated for a predetermined time or longer, the echo detection unit 11c extracts the corresponding saturated echoes as targets for potential echo search processing.

[0104] For example, if the data structure of the above-mentioned observation data contains a predetermined number or more brightness images with the same peak value exceeding the saturation exposure S, the echo detection unit 11c determines that the echo corresponding to the peak value is a saturated echo that has been saturated for a predetermined time or longer.

[0105] The invalid interval determination unit 11d determines the SPAD invalid interval from the start time point of each of the saturated echoes extracted by the echo detection unit 11c based on the LUT 12b. Furthermore, the invalid interval determination unit 11d notifies the SPAD control unit 11e of the determined SPAD invalid interval and the corresponding SPAD valid interval.

[0106] The SPAD control unit 11e dynamically controls the activation or deactivation of SPADs based on the content determined by the invalid interval determination unit 11d. The SPAD control unit 11e controls the enable signal of the SPAD pixel 20 that has detected saturated echoes and is the target of potential echo search processing to be turned on / off, according to the content determined by the invalid interval determination unit 11d.

[0107] Furthermore, each time the SPAD invalid and SPAD valid regions are changed, the SPAD control unit 11e causes the light projection control unit 11a to emit a laser L1 from the light projection unit 13.

[0108] The echo detection unit 11c detects echoes by taking into account the histogram generated by the histogram generation unit 11b during the latent echo search process. Note that when the latent echo search process is not required, the echo detection unit 11c detects echoes based on the histogram generated by the histogram generation unit 11b during the normal reflected light measurement process.

[0109] The echo detection unit 11c specifies the time of flight when the accumulated pixel value reaches its peak based on the detected echoes. Furthermore, based on the specified time of flight, the echo detection unit 11c calculates the distance from the ToF sensor 1 or a device equipped with the ToF sensor 1 to the object 90 located within the distance measurement range. Additionally, the echo detection unit 11c outputs information related to the calculated distance to the host computer 80, for example, via an external I / F 19.

[0110] <<3. Processing Procedure>>

[0111] Next, we will refer to Figure 9 The processing procedure executed by the control unit 11 according to this embodiment is described. Figure 9 It is a flowchart indicating the processing procedures performed by the control unit 11 according to an embodiment of the present disclosure.

[0112] First, the control unit 11 performs the normal reflected light measurement process (step S101). In the normal reflected light measurement process, the control unit 11 causes the light projection unit 13 to emit laser L1, and causes the light receiving unit 14 to output a detection signal based on the reflected light L2. Furthermore, the control unit 11 generates the aforementioned histogram based on the detection signal, and detects the echo based on the histogram.

[0113] Then, the control unit 11 determines whether there is a saturated echo in the detected echo that has been saturated for a predetermined time or longer (step S102). When there is no saturated echo (no in step S102), the control unit 11 outputs distance-related information to the host 80 based on the normal reflected light measurement process in step S101, and repeats the process from step S101.

[0114] When a saturated echo exists (yes in step S102), the control unit 11 extracts the saturated echo as the target for potential echo search processing (step S103).

[0115] Then, the control unit 11 determines the SPAD invalid interval from the start time point of each of the extracted saturated echoes (step S104). The control unit 11 determines the SPAD invalid interval such that, during the latent echo search process, at each emission of laser L1, the SPAD invalid interval based on the observation start time point of the saturated echo is gradually reduced by the SPAD valid interval extending retrospectively from the observation end time point of the saturated echo. Furthermore, the control unit 11 controls the enable signal for the SPAD pixel 20 to be turned on / off according to the determined content.

[0116] In addition, the control unit 11 determines whether the above-mentioned termination condition for the potential echo search process is met (step S105).

[0117] When the termination condition is met (yes in step S105), the control unit 11 repeats the process from step S101. When the termination condition is not met (no in step S105), the control unit 11 causes the laser L1 to emit once and measures the reflected light L2 (step S106).

[0118] Then, the control unit 11 generates the above histogram based on the measurement results (step S107) and determines whether a potential echo rise has been detected according to the histogram (step S108).

[0119] When a potential echo rise cannot be detected (No in step S108), the control unit 11 repeats the process from step S101. Furthermore, when a potential echo rise is detected (Yes in step S108), the control unit 11 separates and records the potential echo from the saturated echo (step S109), and repeats the process from step S104.

[0120] <<4. Modified Example>>

[0121] Incidentally, the implementation of the above-described disclosure may include several modifications.

[0122] <4-1. Combination with glare removal treatment and its effects>

[0123] Figure 10 This is a block diagram illustrating a configuration example of the control unit 11A according to the modified example. Furthermore, Figure 11 This is an illustrative diagram showing the case when combined with glare removal treatment. Note that, due to... Figure 10 and Figure 8 Therefore, this section will primarily describe the corresponding... Figure 8 The differences are...

[0124] According to the modified example, control unit 11A and Figure 8 The control unit 11 differs in that it also includes a glare removal unit 11f. The glare removal unit 11f performs glare removal processing, in which scattered light from highly reflective areas saturates the surrounding SPAD pixels 20 and is detected as false echoes. In the glare removal processing, data in the glare region, including the glare, is removed by performing a predetermined filtering process.

[0125] In the case of combined glare removal processing, the control method according to this embodiment can prevent true echoes that cannot be observed due to glare from being removed by glare removal processing in the prior art.

[0126] This will refer to Figure 11 Provide a detailed description. For example... Figure 11 As shown, the case of different objects 90-1 and 90-2 existing in a two-dimensional space including the horizontal and depth directions is considered as a "deep arrangement". It is assumed that object 90-1 is a strong reflector.

[0127] Here, in the typical reflected light measurement process, as shown in the "Observation Echo" section of the figure, the existing ToF sensor observes the echo e-1 from object 90-1, which has glare f diffused in the glare region R, but the echo from object 90-2 cannot be observed. This is because the echo from object 90-2 overlaps with the dead time of glare f.

[0128] Then, when performing glare removal processing as shown in "glare removal", the ToF sensor according to the prior art removes the glare f included in the glare region R, but also removes the data of hidden (unidentified) objects 90-2 and the like included in the glare region R.

[0129] Therefore, as shown in the "Detection Results", the existing ToF sensor ultimately failed to detect the object 90-2, which was not observable due to glare.

[0130] Meanwhile, in the same "deep arrangement" as in the prior art, as shown in "Echo Observation," the ToF sensor 1 according to this embodiment observes the echo e-1 from an object 90-1 having glare f diffused in the glare region R. Simultaneously, the ToF sensor 1 according to this embodiment observes the echo e-2 of the object 90-2 separately from the glare f through a potential echo search process targeting the glare f.

[0131] Then, as shown in "Glare Removal", even though the glare removal unit 11f performs glare removal processing on the glare region R, the echo e-2 is separated and recorded (identified), and therefore is not removed from the glare region R.

[0132] Therefore, as shown in the "Detection Results", the ToF sensor 1 according to this embodiment can detect the true echo of the object 90-2 that was initially not observable due to glare f without removing the true echo through glare removal processing.

[0133] <4-2. Determining the Length of the SPAD Invalid Interval and the SPAD Valid Interval>

[0134] Furthermore, in this embodiment, a fixed value for the length of the SPAD invalid interval and / or the fixed value for the length of the SPAD valid interval are preset in the LUT 12b based on the observation end time of the saturated echo.

[0135] Meanwhile, in LUT 12b, the aforementioned fixed value can be preset so that the fixed value can be changed, for example, according to the dead time characteristics of SPAD based on the observation start time of saturated echo.

[0136] <4-3. Ratio between SPAD invalid interval and SPAD valid interval>

[0137] Furthermore, in this embodiment, such as Figure 7 The diagram illustrates an example where the ratio between the SPAD invalid region and the SPAD effective region changes to 1:1, 1:3, etc., for each emission of laser L1, but this is only an example and the ratio is not limited to this.

[0138] For example, for each emission of laser L1, the ratio between the SPAD invalid region and the SPAD effective region can be changed to 3:1, 1:1, 1:3, etc. Furthermore, this ratio is not limited to dividing the observation start time to the observation end time of the saturated echo into four segments; it can be divided into fewer than four segments, or even five or more segments. Moreover, this ratio is not necessarily a ratio of natural numbers to natural numbers.

[0139] <4-4. Others>

[0140] Furthermore, in the various processes described in the above embodiments, all or part of the processes described as automatically executed can be executed manually, or all or part of the processes described as manually executed can be executed automatically by known methods. Moreover, unless otherwise stated, the process procedures, specific names, and information including the various data and parameters shown in the above documents and figures can be arbitrarily changed. For example, the various types of information shown in each figure are not limited to the information shown.

[0141] Furthermore, each component of each device shown in the accompanying drawings is functionally conceptual and not necessarily physically configured as shown in the drawings. That is, the specific form of distribution and integration of each device is not limited to the form shown, and all or part of it can be functionally or physically distributed and integrated in any unit according to various loads, usage conditions, etc.

[0142] Furthermore, the embodiments of this disclosure described above can be appropriately combined in areas where the processing content does not contradict each other. Additionally, the order of each step shown in the sequence diagrams or flowcharts of this embodiment can be appropriately changed.

[0143] <<5. Hardware Configuration>>

[0144] Furthermore, the ToF sensor 1 according to the embodiments of the present disclosure described above is, for example, composed of a sensor having such as Figure 12 The computer 1000 with the configuration shown is used to implement this. Figure 12 This is a hardware configuration diagram illustrating an example of a computer 1000 that implements the functions of the ToF sensor 1. The computer 1000 includes a CPU 1100, RAM 1200, ROM 1300, secondary storage device 1400, communication interface 1500, and input / output interface 1600. Each unit of the computer 1000 is connected via a bus 1050.

[0145] The CPU 1100 operates based on programs stored in ROM 1300 or secondary storage device 1400 and controls each unit. For example, the CPU 1100 executes programs stored in ROM 1300 or secondary storage device 1400 in RAM 1200 and performs processing corresponding to various programs.

[0146] ROM 1300 stores boot programs such as the Basic Input / Output System (BIOS) and programs that depend on the hardware of computer 1000, which are executed by CPU 1100 when computer 1000 is started.

[0147] Secondary storage device 1400 is a computer-readable recording medium that non-transitorily records programs executed by CPU 1100, data used by the programs, etc. Specifically, secondary storage device 1400 is a recording medium that records programs according to this embodiment or programs according to a modified example.

[0148] Communication interface 1500 is an interface for computer 1000 to connect to external network 1550. For example, CPU 1100 receives data from other devices or transmits data generated by CPU 1100 to other devices via communication interface 1500.

[0149] Input / output interface 1600 is an interface for connecting input / output device 1650 and computer 1000. For example, CPU 1100 receives data from input devices such as keyboards or mice via input / output interface 1600. Furthermore, CPU 1100 sends data to output devices such as displays, speakers, or printers via input / output interface 1600. Additionally, input / output interface 1600 can be used as a media interface for reading programs recorded on a predetermined recording medium (medium). This medium is, for example, an optical recording medium such as a digital multifunction optical disc (DVD) or phase-change rewritable disc (PD), or a magneto-optical disc (MO), magnetic tape, magnetic recording medium, or magneto-optical recording medium of semiconductor memory.

[0150] For example, when computer 1000 is used as ToF sensor 1, the CPU 1100 of computer 1000 implements the function of control unit 11 by executing a program loaded on RAM 1200. Furthermore, secondary storage device 1400 stores programs according to this disclosure, programs according to modified examples, and data in storage unit 12. Note that CPU 1100 reads program data 1450 from secondary storage device 1400 and executes the program data; however, as another example, these programs can be obtained from other devices via external network 1550.

[0151] <<6. Application Examples>>

[0152] The technology according to this disclosure can be applied to a variety of products. For example, the technology according to this disclosure can be implemented as a device mounted on any type of mobile body, such as automobiles, electric vehicles, hybrid electric vehicles, motorcycles, bicycles, personal mobile devices, airplanes, drones, ships, robots, construction machinery, agricultural machinery (tractors), etc.

[0153] Figure 13 This is a block diagram illustrating an example configuration of a vehicle control system 7000, which is an example of a mobile body control system to which the technology of embodiments according to this disclosure can be applied. The vehicle control system 7000 includes multiple electronic control units interconnected via a communication network 7010. Figure 13 In the example depicted, the vehicle control system 7000 includes a drive system control unit 7100, a body system control unit 7300, a battery control unit 7300, an external information detection unit 7400, an internal information detection unit 7500, and an integrated control unit 7600. The communication network 7010 connecting the multiple control units to each other can be, for example, an in-vehicle communication network conforming to any standard such as CAN, LIN, LAN, or FlexRay (registered trademark).

[0154] Each of the control units includes: a microcomputer that performs arithmetic processing according to various programs; a storage unit that stores programs executed by the microcomputer, parameters for various operations, etc.; and a drive circuit that drives various control target devices. Each of the control units also includes: a network interface (I / F) for communicating with other control units via the communication network 7010; and a communication I / F for communicating with devices, sensors, etc., inside and outside the vehicle via wired or wireless communication. Figure 13 The integrated control unit 7600 shown is configured with the following functionalities: a microcomputer 7610, a general communication I / F 7620, a dedicated communication I / F 7630, a positioning unit 7640, a beacon receiver 7650, an in-vehicle device I / F 7660, a voice / image output unit 7670, an in-vehicle network I / F 7680, and a storage unit 7690. Other control units similarly include microcomputers, communication I / Fs, and storage units.

[0155] The drive system control unit 7100 controls the operation of devices related to the vehicle's drive system according to various types of programs. For example, the drive system control unit 7100 functions as a control device for: a drive force generating device for generating the vehicle's driving force, such as an internal combustion engine or drive motor; a drive force transmission mechanism for transmitting the driving force to the wheels; a steering mechanism for adjusting the vehicle's steering angle; and a braking device for generating the vehicle's braking force. The drive system control unit 7100 may also function as a control device for anti-lock braking systems (ABS), electronic stability control (ESC), etc.

[0156] The drive system control unit 7100 is connected to the vehicle condition detection unit 7110. The vehicle condition detection unit 7110 includes, for example, at least one of the following: a gyroscope sensor for detecting the angular velocity of the axial rotational movement of the vehicle body; an acceleration sensor for detecting the acceleration of the vehicle; and sensors for detecting the amount of operation of the accelerator pedal, the amount of operation of the brake pedal, the steering angle of the steering wheel, engine speed, or wheel rotation speed. The drive system control unit 7100 performs computational processing using signals input from the vehicle condition detection unit 7110 and controls the internal combustion engine, drive motor, electric power steering system, braking system, etc.

[0157] The vehicle body system control unit 7200 controls the operation of various types of devices installed on the vehicle body according to various types of programs. For example, the vehicle body system control unit 7200 is used as a control device for: keyless entry systems, smart key systems, power windows, or various types of lights such as headlights, reversing lights, brake lights, turn signals, fog lights, etc. In this case, radio waves or signals from various types of switches, which are sent from a moving device that serves as a substitute for a key, can be input to the vehicle body system control unit 7200. The vehicle body system control unit 7200 receives these input radio waves or signals and controls the vehicle's door locking devices, power windows, lights, etc.

[0158] The battery control unit 7300 controls the secondary battery 7310, which serves as a power supply source for driving the motor, according to various types of programs. For example, information regarding battery temperature, battery output voltage, and remaining charge in the battery is supplied to the battery control unit 7300 from the battery device including the secondary battery 7310. The battery control unit 7300 uses these signals to perform arithmetic processing and executes controls to regulate the temperature of the secondary battery 7310 or to control the cooling devices of the battery device, etc.

[0159] The exterior information detection unit 7400 detects information about the exterior of the vehicle, including the vehicle control system 7000. For example, the exterior information detection unit 7400 is connected to at least one of the imaging unit 7410 and the exterior information detection unit 7420. The imaging unit 7410 includes at least one of a time-of-flight (ToF) camera, a stereo camera, a monocular camera, an infrared camera, and other camera devices. The exterior information detection unit 7420 includes, for example, at least one of: an environmental sensor for detecting current atmospheric or weather conditions, and a peripheral information detection sensor for detecting other vehicles, obstacles, pedestrians, etc., surrounding the vehicle, including the vehicle control system 7000.

[0160] For example, the environmental sensor can be at least one of the following: a raindrop sensor for detecting rain, a fog sensor for detecting fog, a sunlight sensor for detecting sunlight intensity, and a snow sensor for detecting snowfall. The peripheral information detection sensor can be at least one of the following: an ultrasonic sensor, a radar device, and a LIDAR device (light detection and ranging device or laser imaging detection and ranging device). Each of the imaging unit 7410 and the external information detection unit 7420 can be configured as an independent sensor or device, or can be configured as a device integrating multiple sensors or devices.

[0161] Figure 14 Examples of mounting positions for the imaging unit 7410 and the exterior information detection unit 7420 are depicted. For example, imaging units 7910, 7912, 7914, 7916, and 7918 are located at at least one of the following positions: the upper portion of the windshield inside the vehicle interior, and the front nose, side mirrors, rear bumper, and rear door of the vehicle 7900. The imaging unit 7910 located at the front nose and the imaging unit 7918 located at the upper portion of the windshield inside the vehicle interior primarily obtain images of the front of the vehicle 7900. The imaging units 7912 and 7914 located at the side mirrors primarily obtain images of the sides of the vehicle 7900. The imaging unit 7916 located at the rear bumper or rear door primarily obtains images of the rear of the vehicle 7900. The imaging unit 7918 located at the upper portion of the windshield inside the vehicle interior is primarily used to detect vehicles, pedestrians, obstacles, signals, traffic signs, lanes, etc., ahead.

[0162] Incidentally, Figure 14Examples of the imaging ranges of the corresponding imaging units 7910, 7912, 7914, and 7916 are depicted. Imaging range a represents the imaging range of the imaging unit 7910 located at the front nose. Imaging ranges b and c represent the imaging ranges of the imaging units 7912 and 7914 located at the side mirrors, respectively. Imaging range d represents the imaging range of the imaging unit 7916 located at the rear bumper or rear door. For example, a bird's-eye view of the vehicle 7900 viewed from above can be obtained by overlaying the image data captured by the imaging units 7910, 7912, 7914, and 7916.

[0163] Exterior information detection units 7920, 7922, 7924, 7926, 7928, and 7930, located at the front, rear, sides, corners, and upper portion of the windshield inside the vehicle 7900, can be, for example, ultrasonic sensors or radar devices. For instance, exterior information detection units 7920, 7926, and 7930 located at the front nose, rear bumper, rear door, and upper portion of the windshield inside the vehicle 7900 can be LIDAR devices. These exterior information detection units 7920 to 7930 are primarily used to detect vehicles, pedestrians, obstacles, etc., ahead.

[0164] Return to Figure 13 The description will continue. The exterior information detection unit 7400 causes the imaging unit 7410 to image the exterior of the vehicle and receives the image data. Furthermore, the exterior information detection unit 7400 receives detection information from the exterior information detection section 7420 connected to it. If the exterior information detection section 7420 is an ultrasonic sensor, radar device, or LIDAR device, the exterior information detection unit 7400 transmits ultrasonic waves, electromagnetic waves, etc., and receives information about the received reflected waves. Based on the received information, the exterior information detection unit 7400 can perform processing for detecting objects (e.g., people, vehicles, obstacles, signs, symbols, etc. on the road surface) or processing for the distance to the detected objects. The exterior information detection unit 7400 can perform environmental recognition processing based on the received information, such as identifying rain, fog, road conditions, etc. The exterior information detection unit 7400 can calculate the distance to objects outside the vehicle based on the received information.

[0165] Furthermore, based on the received image data, the exterior information detection unit 7400 can perform image recognition processing to identify people, vehicles, obstacles, signs, symbols, etc. on the road surface, or to detect their distance. The exterior information detection unit 7400 can subject the received image data to processing such as distortion correction and alignment, and combine image data captured by multiple different imaging units 7410 to generate a bird's-eye view or panoramic image. The exterior information detection unit 7400 can use image data captured by imaging units 7410 including different imaging components to perform viewpoint switching processing.

[0166] The in-vehicle information detection unit 7500 detects information about the interior of the vehicle. The in-vehicle information detection unit 7500 is connected, for example, to a driver state detection unit 7510 that detects the driver's state. The driver state detection unit 7510 may include a camera device for imaging the driver, a biosensor for detecting the driver's biological information, a microphone for collecting sounds inside the vehicle, etc. The biosensor is disposed, for example, in the seat surface, steering wheel, etc., and detects the biological information of occupants sitting in the seat or the driver holding the steering wheel. Based on the detection information input from the driver state detection unit 7510, the in-vehicle information detection unit 7500 can calculate the driver's fatigue level or concentration level, or determine whether the driver is dozing off. The in-vehicle information detection unit 7500 can process the audio signals obtained through sound collection, such as through noise cancellation processing.

[0167] The integrated control unit 7600 controls the general operation within the vehicle control system 7000 according to various types of programs. The integrated control unit 7600 is connected to the input unit 7800. The input unit 7800 is implemented by a device capable of input operation by an occupant, such as a touch panel, button, microphone, switch, joystick, etc. Data obtained from voice recognition of voice input via a microphone can be supplied to the integrated control unit 7600. The input unit 7800 can be, for example, a remote control device using infrared or other radio waves, or an external connection device supporting the operation of the vehicle control system 7000, such as a mobile phone, personal digital assistant (PDA), etc. The input unit 7800 can be, for example, a camera device. In this case, the occupant can input information via gestures. Alternatively, data obtained by detecting the movement of a wearable device worn by the occupant can be input. Furthermore, the input unit 7800 may include, for example, an input control circuit that generates an input signal based on information input by the occupant using the aforementioned input unit 7800, and outputs the generated input signal to the integrated control unit 7600. Passengers can input various types of data or give instructions for processing operations to the vehicle control system 7000 through the operation input unit 7800.

[0168] The storage unit 7690 may include a ROM for storing various types of programs executed by a microcomputer and a RAM for storing various types of parameters, operation results, sensor values, etc. Furthermore, the storage unit 7690 may be implemented using magnetic storage devices such as hard disk drives (HDDs), semiconductor storage devices, optical storage devices, magneto-optical storage devices, etc.

[0169] The Universal Communication I / F 7620 is a widely used communication I / F that mediates communication with various devices present in the external environment 7750. The Universal Communication I / F 7620 can implement cellular communication protocols such as GSM, WiMAX, LTE, and LTE-A Advanced, or other wireless communication protocols such as Wi-Fi (also known as Wireless LAN) and Bluetooth. For example, the Universal Communication I / F 7620 can connect to devices (e.g., application servers or control servers) present on external networks (e.g., the Internet, cloud networks, or corporate private networks) via base stations or access points. Furthermore, the Universal Communication I / F 7620 can use, for example, peer-to-peer (P2P) technology to connect to terminals present near the vehicle (e.g., terminals belonging to drivers, pedestrians, or shopkeepers, or machine-type communication (MTC) terminals).

[0170] The Dedicated Communications I / F 7630 is a communications I / F that supports the development of communication protocols for use in vehicles. The Dedicated Communications I / F 7630 can implement standard protocols such as Wireless Access in a Vehicle Environment (WAVE) (a combination of IEEE 802.11p as the lower layer and IEEE 1609 as the upper layer), Dedicated Short Range Communications (DSRC), or cellular communication protocols. The Dedicated Communications I / F 7630 typically performs V2X communication, which is one or more concepts including vehicle-to-vehicle (V2V) communication, road-to-vehicle (V2V) communication, vehicle-to-home (V2V) communication, and pedestrian-to-vehicle (V2P) communication.

[0171] The positioning unit 7640 performs positioning, for example, by receiving GNSS signals from Global Navigation Satellite System (GNSS) satellites (e.g., GPS signals from Global Positioning System (GPS) satellites) and generates location information including the vehicle's latitude, longitude, and altitude. Incidentally, the positioning unit 7640 can identify its current location by exchanging signals with a wireless access point, or by obtaining location information from a terminal such as a mobile phone with positioning capabilities, a Personal Handheld Phone System (PHS), or a smartphone.

[0172] The beacon receiver 7650 receives, for example, radio waves or electromagnetic waves transmitted from a radio station installed on a road, thereby obtaining information about current location, congestion, road closures, and estimated time. Incidentally, the functionality of the beacon receiver 7650 can be included in the aforementioned dedicated communication I / F 7630.

[0173] The in-vehicle device I / F 7660 is a communication interface that mediates the connection between the microcomputer 7610 and various in-vehicle devices 7760 present within the vehicle. The in-vehicle device I / F 7660 can establish a wireless connection using wireless communication protocols such as Wireless LAN, Bluetooth (registered trademark), Near Field Communication (NFC), or Wireless Universal Serial Bus (WUSB). Furthermore, the in-vehicle device I / F 7660 can establish a wired connection via connection terminals not shown in the figure (and cables as required) through Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI (registered trademark)), Mobile High Definition Link (MHL), etc. The in-vehicle device 7760 may include, for example, at least one of mobile devices and wearable devices owned by the occupants, and information devices carried into or attached to the vehicle. The in-vehicle device 7760 may also include a navigation device for searching paths to any destination. The in-vehicle device I / F 7660 exchanges control signals or data signals with these in-vehicle devices 7760.

[0174] The vehicle network I / F 7680 is an interface that mediates communication between the microcomputer 7610 and the communication network 7010. The vehicle network I / F 7680 sends and receives signals according to a predetermined protocol supported by the communication network 7010.

[0175] The microcomputer 7610 of the integrated control unit 7600 controls the vehicle control system 7000 according to various types of programs based on information obtained via at least one of the following: general communication I / F 7620, dedicated communication I / F 7630, positioning unit 7640, beacon receiving unit 7650, in-vehicle device I / F 7660, and vehicle network I / F 7680. For example, the microcomputer 7610 can calculate control target values ​​for the drive force generation device, steering mechanism, or braking device based on the obtained information about the vehicle's interior and exterior, and output control commands to the drive system control unit 7100. For example, the microcomputer 7610 can perform cooperative control aimed at realizing functions of advanced driver assistance systems (ADAS), including: collision avoidance or impact buffering for the vehicle, following driving based on following distance, vehicle speed-maintaining driving, vehicle collision warning, vehicle lane departure warning, etc. Furthermore, the microcomputer 7610 can perform cooperative control of autonomous driving, which aims to enable the vehicle to drive automatically without relying on the driver's operation, by controlling the drive force generation device, steering mechanism, braking device, etc., based on the information obtained about the vehicle's surroundings.

[0176] The microcomputer 7610 can generate three-dimensional distance information between the vehicle and objects such as surrounding structures and people based on information obtained via at least one of the following: a general communication I / F 7620, a dedicated communication I / F 7630, a positioning unit 7640, a beacon receiving unit 7650, an in-vehicle device I / F 7660, and an in-vehicle network I / F 7680. It also generates local map information including information about the vehicle's current location and its surroundings. Furthermore, the microcomputer 7610 can predict hazards such as vehicle collisions, pedestrian approach, and entry into closed roads based on the obtained information, and generate warning signals. These warning signals may, for example, be signals used to generate warning sounds or illuminate warning lights.

[0177] The sound / image output unit 7670 sends an output signal of at least one of sound and image to an output device capable of visually or audibly notifying vehicle occupants or the outside of the vehicle of information. Figure 13In the example, the audio speaker 7710, display unit 7720, and instrument panel 7730 are shown as output devices. The display unit 7720 may include, for example, at least one of a vehicle-mounted display and a head-up display. The display unit 7720 may have augmented reality (AR) display functionality. The output device may be a device other than these, and may be other devices such as headphones, wearable devices such as glasses-type displays worn by occupants, projectors, lamps, etc. When the output device is a display device, the display device visually displays, in various forms such as text, images, tables, and graphics, the results obtained from various types of processing performed by the microcomputer 7610 or information received from other control units. Furthermore, when the output device is an audio output device, the audio output device converts audio signals composed of reproduced audio data or sound data into analog signals and audibly outputs the analog signals.

[0178] Incidentally, in Figure 13 In the example depicted, at least two control units connected to each other via communication network 7010 can be integrated into a single control unit. Alternatively, each individual control unit may include multiple control units. Furthermore, the vehicle control system 7000 may include other control units not depicted in the figures. Additionally, some or all of the functions performed by one of the control units described above can be distributed to other control units. That is, any control unit can perform predetermined computational processing as long as information is sent and received via communication network 7010. Similarly, sensors or devices connected to one of the control units can be connected to other control units, and multiple control units can send and receive detection information to each other via communication network 7010.

[0179] Note that this is used to implement the reference. Figure 1 The program for each function of the control unit 11 according to this embodiment can be installed on any control unit or the like. Furthermore, a computer-readable recording medium for storing such a program can also be provided. This recording medium is, for example, a magnetic disk, optical disk, magneto-optical disk, flash memory, etc. A recording medium is also a mode of this disclosure. Furthermore, the aforementioned program can be distributed via, for example, a network without using a recording medium.

[0180] In the aforementioned vehicle control system 7000, the distance measuring device (ToF sensor) according to the above embodiment or modification can be applied to... Figure 13 The application example shown includes the vehicle exterior information detection unit 7420 and / or the driver state detection unit 7510. This allows for more accurate distance measurement, thereby enabling safer driving support and autonomous driving.

[0181] <<7. Conclusion>>

[0182] As described above, according to embodiments of this disclosure, the control unit 11 (corresponding to an example of a "control device") is a control device that detects the echo relative to the pulsed laser L1 (corresponding to an example of an "emitted light") based on the physical quantity of photons incident on each of the plurality of arranged SPAD pixels 20 (corresponding to an example of a "pixel"). The control unit 11 includes an invalid interval determination unit 11d (corresponding to an example of a "determination unit") and a SPAD control unit 11e (corresponding to an example of a "pixel control unit"). In the case of detecting a saturated echo where the echo saturates for a predetermined time or longer, the invalid interval determination unit 11d determines a SPAD invalid interval (corresponding to an example of an "invalid interval") such that the start time point of the SPAD invalid interval that invalidates the SPAD pixel 20 that has detected a saturated echo is set as the start time point of the saturated echo, and the SPAD invalid interval is gradually reduced each time the laser L1 is emitted. The SPAD control unit 11e invalidates the SPAD pixel 20 according to the determined SPAD invalid interval and causes the laser L1 to be emitted. This makes it possible to observe echoes that cannot be observed during the dead time.

[0183] While embodiments of the present disclosure have been described above, the technical scope of the present disclosure is not limited to the embodiments described as is, and various modifications can be made without departing from the spirit of the present disclosure. Furthermore, components from different embodiments and modifications can be appropriately combined.

[0184] Furthermore, the effects of the embodiments described in this specification are merely illustrative and not limiting, and may provide other effects.

[0185] Note that this technology can also have the following configurations. (1)

[0187] A control device that detects an echo relative to pulsed emitted light based on a physical quantity of photons incident on each of a plurality of arranged pixels, the control device comprising:

[0188] The determining unit, upon detecting a saturated echo that has reached saturation for a predetermined time or longer, determines an invalid interval, such that the start time point of the invalid interval that invalidates the pixels that have detected the saturated echo is set as the start time point of the saturated echo, and the invalid interval gradually decreases each time the emitted light is emitted; and

[0189] A pixel control unit that invalidates the pixel according to a determined invalid region and causes the emitted light to be emitted. (2)

[0191] According to the control device described in (1), wherein,

[0192] The determining unit determines the invalid interval such that the invalid interval and the valid interval relative to the invalid interval are set at an arbitrary ratio between the start time point and the end time point of the saturated echo. (3)

[0194] According to the control device described in (2), wherein,

[0195] The determining unit determines the invalid interval such that the time period between the start time point and the end time point is divided at a predetermined interval, and the invalid interval gradually decreases according to the interval each time the emitted light is emitted. (4)

[0197] According to the control device described in (3), wherein,

[0198] The time period between the start time and the end time is divided into four parts, and

[0199] The determining unit determines the invalid interval such that, each time the emitted light is emitted, the invalid interval gradually decreases according to the interval of the four parts. (5)

[0201] According to the control device described in (2), (3) or (4), wherein,

[0202] The determining unit determines the invalid interval based on a lookup table, in which at least one of a fixed value for the valid interval or a fixed value for the invalid interval is preset based on the end time point. (6)

[0204] According to the control device described in (2), (3) or (4), wherein,

[0205] The determining unit determines the invalid interval based on a lookup table, in which at least one of a fixed value for the valid interval or a fixed value for the invalid interval is preset based on the start time point. (7)

[0207] According to the control device described in (6), wherein,

[0208] The determining unit determines the invalid interval based on the following lookup table: in the lookup table, the fixed value is preset according to the dead time characteristics of the pixel. (8)

[0210] According to any one of (1) to (7), the control device, wherein,

[0211] If a predetermined termination condition is met, the potential echo search process performed by the determination unit and the pixel control unit when the saturation echo is detected ends. (9)

[0213] According to the control device described in (8), wherein,

[0214] The termination conditions include:

[0215] The case where the number of executions of the potential echo search process exceeds the first specified value or the length of the invalid interval becomes equal to or less than the second specified value. (10)

[0217] The control device according to any one of (1) to (9) further includes:

[0218] A glare removal unit removes glare caused by scattered light from highly reflective light through a predetermined filtering process. (11)

[0220] A distance measuring device, comprising:

[0221] A light projection unit, wherein the light projection unit emits pulsed light;

[0222] An optical receiving unit, wherein multiple pixels are arranged to detect the incidence of photons;

[0223] A detection unit that detects the echo relative to the emitted light based on a physical quantity of photons emitted from each of the pixels.

[0224] The determining unit, when the detection unit detects a saturated echo that has reached saturation for a predetermined time or longer, determines an invalid interval, such that the start time point of the invalid interval that invalidates the pixels that have detected the saturated echo is set as the start time point of the saturated echo, and the invalid interval gradually decreases each time the emitted light is emitted; and

[0225] A pixel control unit, which invalidates the pixel according to a determined invalid region, and causes the light projection unit to emit the emitted light. (12)

[0227] A control method executed by a control device that detects an echo relative to pulsed emitted light based on a physical quantity of photons incident on each of a plurality of arranged pixels, the control method comprising:

[0228] If a saturated echo is detected that has reached saturation for a predetermined time or longer, an invalid region is determined such that the start time point of the invalid region, which invalidates the pixel that has detected the saturated echo, is set as the start time point of the saturated echo, and the invalid region gradually decreases each time the emitted light is emitted; and

[0229] The pixel is invalidated according to the determined invalid region, and the emitted light is emitted. (13)

[0231] A computer-readable recording medium having a program recorded thereon that causes a computer to perform processing including the following operations:

[0232] The echo relative to the pulsed emitted light is detected based on the physical quantity of the photons incident on each of the multiple arranged pixels.

[0233] If a saturated echo is detected that has reached saturation for a predetermined time or longer, an invalid region is determined such that the start time point of the invalid region, which invalidates the pixel that has detected the saturated echo, is set as the start time point of the saturated echo, and the invalid region gradually decreases each time the emitted light is emitted; and

[0234] The pixel is invalidated according to the determined invalid region, and the emitted light is emitted.

[0235] List of reference numerals

[0236] 1ToF sensor

[0237] 11, 11A Control Unit

[0238] 11a Light Projection Control Unit

[0239] 11b Histogram Generation Unit

[0240] 11c echo detection unit

[0241] 11d invalid interval determination unit

[0242] 11e SPAD control unit

[0243] 11F Glare Reduction Unit

[0244] 12 storage units

[0245] 12a Accumulator Memory

[0246] 12b LUT

[0247] 13 light projection units

[0248] 14 optical receiving units

[0249] 19External I / F

[0250] 20 SPAD pixels

[0251] 31 light sources

[0252] 32 collimating lens

[0253] 33. Semi-reflective mirror

[0254] 34 drive units

[0255] 35 Current Mirror

[0256] 36. Optical receiving sensor

[0257] 37SPAD array

[0258] 38. Light receiving lens

[0259] 43 Timing control circuit

[0260] 44 Drive Circuit

[0261] 45 Output Circuit

[0262] 80 host

[0263] 90 objects

Claims

1. A control device that detects an echo relative to pulsed emitted light based on a physical quantity of photons incident on each of a plurality of arranged pixels, the control device comprising: The determining unit, when detecting a saturated echo that has reached saturation for a predetermined time or longer, determines an invalid interval, such that the start time point of the invalid interval that invalidates the pixel that has detected the saturated echo is set as the start time point of the saturated echo, and the invalid interval gradually decreases each time the emitted light is emitted; as well as A pixel control unit that invalidates the pixel according to a determined invalid region and causes the emitted light to be emitted.

2. The control device according to claim 1, wherein, The determining unit determines the invalid interval such that the invalid interval and the valid interval relative to the invalid interval are set at an arbitrary ratio between the start time point and the end time point of the saturated echo.

3. The control device according to claim 2, wherein, The determining unit determines the invalid interval such that the time period between the start time point and the end time point is divided at a predetermined interval, and the invalid interval gradually decreases according to the interval each time the emitted light is emitted.

4. The control device according to claim 3, wherein, The time period between the start time and the end time is divided into four parts, and The determining unit determines the invalid interval such that, each time the emitted light is emitted, the invalid interval gradually decreases according to the interval of the four parts.

5. The control device according to claim 2, wherein, The determining unit determines the invalid interval based on a lookup table, in which at least one of a fixed value for the valid interval or a fixed value for the invalid interval is preset based on the end time point.

6. The control device according to claim 2, wherein, The determining unit determines the invalid interval based on a lookup table, in which at least one of a fixed value for the valid interval or a fixed value for the invalid interval is preset based on the start time point.

7. The control device according to claim 6, wherein, The determining unit determines the invalid interval based on the following lookup table: in the lookup table, the fixed value is preset according to the dead time characteristics of the pixel.

8. The control device according to claim 1, wherein, If a predetermined termination condition is met, the potential echo search process performed by the determination unit and the pixel control unit when the saturation echo is detected ends.

9. The control device according to claim 8, wherein, The termination conditions include: The case where the number of executions of the potential echo search process exceeds the first specified value or the length of the invalid interval becomes equal to or less than the second specified value.

10. The control device according to claim 1, further comprising: A glare removal unit removes glare caused by scattered light from highly reflective light through a predetermined filtering process.

11. A distance measuring device, comprising: A light projection unit, wherein the light projection unit emits pulsed light; An optical receiving unit, wherein multiple pixels are arranged to detect the incidence of photons; A detection unit that detects the echo relative to the emitted light based on a physical quantity of photons emitted from each of the pixels. The determining unit determines an invalid interval when the detection unit detects that the echo saturates for a predetermined time or longer, such that the start time point of the invalid interval that invalidates the pixel that has detected the saturated echo is set as the start time point of the saturated echo, and the invalid interval gradually decreases each time the emitted light is emitted. as well as A pixel control unit, which invalidates the pixel according to a determined invalid region, and causes the light projection unit to emit the emitted light.

12. A control method executed by a control device, the control device detecting an echo relative to pulsed emitted light based on a physical quantity of photons incident on each of a plurality of arranged pixels, the control method comprising: If a saturated echo is detected that has reached saturation for a predetermined time or longer, an invalid interval is determined such that the start time point of the invalid interval that invalidates the pixel that has detected the saturated echo is set as the start time point of the saturated echo, and the invalid interval gradually decreases each time the emitted light is emitted. as well as The pixel is invalidated according to the determined invalid region, and the emitted light is emitted.

Images