Control device, control method, control program

Abstract

To provide a control device that ensures measurement accuracy of an optical sensor.SOLUTION: A processor of a control device for controlling an optical sensor, in which a plurality of SPADs are arrayed for each light receiving pixel and which receives light from a sensing area, is configured to execute: acquiring a light receiving signal waveform WR including a reflection light output component WRr from a SPAD that responds by receiving reflection light to irradiation light from the optical sensor to the sensing area and an external light output component WRo from a SPAD that responds by receiving external light from the sensing area for each light receiving pixel; and estimating reflection intensity IR of the reflection light on the basis of a correlation between a reflection light response number NRr that is the number of SPADs which output the reflection light output component WRr and an external light response number NRo that is the number of SPADs which output the external light output component WRo for each light receiving pixel.SELECTED DRAWING: Figure 11

Description

Technical Field

[0001] The present disclosure relates to control technology for controlling an optical sensor.

Background Art

[0002] An optical sensor in which a plurality of single photon avalanche diodes (SPADs) are arranged for each light receiving pixel to receive light from a sensing area is widely known. For example, in the optical sensor disclosed in Patent Document 1, the distance to an object is measured based on the timing at which the number of responses of the SPAD reaches its maximum value for each detection area corresponding to the light receiving pixel.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the disclosed technology of Patent Document 1, the number of responses of the SPAD represents the reflection intensity from the object by its maximum value. Therefore, it is possible to measure the reflection intensity according to the number of responses of the SPAD. However, since the number of responses of the SPAD is affected by external light from the sensing area, there is a risk of a decrease in measurement accuracy.

[0005] An object of the present disclosure is to provide a control device for ensuring the measurement accuracy of an optical sensor. Another object of the present disclosure is to provide a control method for ensuring the measurement accuracy of an optical sensor. Still another object of the present disclosure is to provide a control program for ensuring the measurement accuracy of an optical sensor.

Means for Solving the Problems

[0006] The following describes the technical means of solving the problem described in this disclosure. Note that the claims and the reference numerals in parentheses in this section indicate the correspondence with the specific means described in the embodiments detailed later, and do not limit the technical scope of this disclosure.

[0007] The first aspect of this disclosure is, A control device for controlling an optical sensor (10) having a processor (1b) and in which multiple SPADs (460), which are single-photon avalanche diodes, are arranged for each light-receiving pixel (46) to receive light from a sensing area (AS), The processor is The system acquires a received signal waveform (WR) for each receiving pixel, which includes the reflected light output component (WRr) from the SPAD in response to the reception of reflected light from the optical sensor to the sensing area, and the ambient light output component (WRo) from the SPAD in response to the reception of ambient light from the sensing area. The system is configured to estimate the reflected light intensity (IR) for each light-receiving pixel based on the correlation between the number of reflected light responses (NRr), which is the number of SPADs that output the reflected light output component, and the number of ambient light responses (NRo), which is the number of SPADs that output the ambient light output component.

[0008] A second aspect of this disclosure is, A control method executed by a processor (1b) to control an optical sensor (10) in which multiple SPADs (460), which are single-photon avalanche diodes, are arranged for each light-receiving pixel (46) to receive light from a sensing area (AS), wherein The system acquires a received signal waveform (WR) for each receiving pixel, which includes the reflected light output component (WRr) from the SPAD in response to the reception of reflected light from the optical sensor to the sensing area, and the ambient light output component (WRo) from the SPAD in response to the reception of ambient light from the sensing area. This includes estimating the reflected light intensity (IR) for each light-receiving pixel based on the correlation between the number of reflected light responses (NRr), which is the number of SPADs that output the reflected light output component, and the number of ambient light responses (NRo), which is the number of SPADs that output the ambient light output component.

[0009] A third aspect of this disclosure is: A control program, which includes instructions to be executed by a processor (1b) and stored in a storage medium (1a) for controlling an optical sensor (10) in which multiple SPADs (460), which are single-photon avalanche diodes, are arranged for each light-receiving pixel (46) to receive light from a sensing area (AS), is provided. The order is, The system acquires a received signal waveform (WR) for each receiving pixel, which includes the reflected light output component (WRr) from the SPAD in response to the reception of reflected light from the optical sensor to the sensing area, and the ambient light output component (WRo) from the SPAD in response to the reception of ambient light from the sensing area. This includes estimating the reflected light intensity (IR) for each light-receiving pixel based on the correlation between the number of reflected light responses (NRr), which is the number of SPADs that output the reflected light output component, and the number of ambient light responses (NRo), which is the number of SPADs that output the ambient light output component.

[0010] In these first to third embodiments, the received signal waveform is acquired for each receiving pixel so as to include the reflected light output component from the SPAD that responds to the reception of reflected light from the optical sensor to the sensing area, and the ambient light output component from the SPAD that responds to the reception of ambient light from the sensing area. Therefore, according to the first to third embodiments, the reflected light intensity can be accurately estimated, reflecting the influence of ambient light reception for each receiving pixel, based on the correlation between the number of reflected light responses, which is the number of SPADs that output the reflected light output component, and the number of ambient light responses, which is the number of SPADs that output the ambient light output component. Thus, it becomes possible to ensure the measurement accuracy of the reflected light intensity through such estimation. [Brief explanation of the drawing]

[0011] [Figure 1]It is a schematic diagram showing the overall configuration of the sensing system according to the first embodiment. [Figure 2] It is a schematic diagram showing the detailed configuration of the optical sensor according to the first embodiment. [Figure 3] It is a block diagram showing the functional configuration of the optical sensor and the control device according to the first embodiment. [Figure 4] It is a schematic diagram showing the light projector according to the first embodiment. [Figure 5] It is a schematic diagram showing the light receiver according to the first embodiment. [Figure 6] It is a schematic diagram showing the light-receiving pixel according to the first embodiment. [Figure 7] It is a circuit diagram showing the light-receiving circuit according to the first embodiment. [Figure 8] It is a graph showing the characteristics of the light-receiving circuit according to the first embodiment. [Figure 9] It is a graph showing the characteristics of the light-receiving circuit according to the first embodiment. [Figure 10] It is a graph showing the light-receiving signal waveform according to the first embodiment. [Figure 11] It is a flowchart showing the control flow according to the first embodiment. [Figure 12] It is a table showing the correlation table according to the first embodiment. [Figure 13] It is a table showing the correlation map according to the first embodiment. [Figure 14] It is a block diagram showing the functional configuration of the optical sensor and the control device according to the second embodiment. [Figure 15] It is a flowchart showing the control flow according to the second embodiment. [Figure 16] It is a schematic diagram showing the response range in the light-receiving pixel according to the second embodiment. [Figure 17] It is a schematic diagram showing the response range in the light-receiving pixel according to the second embodiment. [Figure 18] It is a graph showing the response range in the light-receiving pixel according to the second embodiment. [Figure 19] It is a graph showing the response range in the light-receiving pixel according to the second embodiment. [Figure 20] This flowchart shows the control flow according to the third embodiment. [Figure 21] This is a schematic diagram illustrating the setting of the response range in the light-receiving pixel according to the third embodiment. [Figure 22] This flowchart shows the control flow according to a modified example of Figure 20. [Figure 23] This is a flowchart showing the control flow according to the fourth embodiment. [Figure 24] This flowchart shows the control flow according to the fifth embodiment. [Figure 25] This table shows the correlation table according to the fifth embodiment. [Figure 26] This is a schematic diagram showing the functional configuration of an optical sensor and control device according to the sixth embodiment. [Figure 27] This flowchart shows the correction flow according to the sixth embodiment. [Figure 28] This flowchart shows the control flow according to the sixth embodiment. [Figure 29] This is a schematic diagram showing a photodetector based on a modified example of Figure 5. [Modes for carrying out the invention]

[0012] Hereinafter, several embodiments of this disclosure will be described with reference to the drawings. In each embodiment, the same reference numerals will be used for corresponding components, and redundant explanations may be omitted. Furthermore, if only a part of the configuration is described in each embodiment, the configuration of other embodiments described earlier may be applied to the other parts of that configuration. Moreover, not only the combinations of configurations explicitly stated in the description of each embodiment, but also the configurations of multiple embodiments can be partially combined even if not explicitly stated, as long as there are no particular problems with the combination.

[0013] (First Embodiment) As shown in Figure 1, a first embodiment of the present disclosure relates to a sensing system 2 comprising an optical sensor 10 and a control device 1. The sensing system 2 is mounted on a vehicle 5, which is a mobile vehicle. The vehicle 5 is, for example, an automobile, which is capable of traveling on a road with an occupant on board.

[0014] Vehicle 5 is capable of autonomous driving, either on a steady or temporary basis, in an autonomous driving control mode. Here, the autonomous driving control mode may be implemented by autonomous driving control in which the system performs all driving tasks during operation, such as conditional driving automation, advanced driving automation, or full driving automation. The autonomous driving control mode may also be implemented by advanced driving assistance control in which the occupant performs some or all of the driving tasks, such as driving assistance or partial driving automation. The autonomous driving control mode may be implemented by either one of these autonomous driving controls or advanced driving assistance controls, or by a combination of them, or by switching between them.

[0015] In the following explanation, unless otherwise specified, the forward, backward, up, down, left, and right directions are defined relative to the vehicle 5 on the horizontal plane. The horizontal direction refers to the direction parallel to the horizontal plane that serves as the direction reference for the vehicle 5. Furthermore, the vertical direction refers to the direction perpendicular to the horizontal plane that serves as the direction reference for the vehicle 5, which is also the up and down direction.

[0016] The optical sensor 10 is a so-called LiDAR (Light Detection and Ranging / Laser Imaging Detection and Ranging) for acquiring image data that can be used for driving control of the vehicle 5, including automatic control driving modes. The optical sensor 10 is positioned at least one location on the vehicle 5, for example, on the front, left and right sides, rear, and roof.

[0017] As shown in Figure 2, the optical sensor 10 defines a three-dimensional Cartesian coordinate system using the X, Y, and Z axes, which are mutually orthogonal to each other. In particular, in the first embodiment, the X and Z axes are set along different horizontal directions of the vehicle 5, and the Y axis is set along the vertical direction of the vehicle 5. Note that in Figure 2, the portion to the left of the dashed line along the Y axis (the optical window 12 side described later) actually shows a cross-section perpendicular to the portion to the right of the dashed line (the unit 21, 41 side described later).

[0018] As shown in Figure 3, the optical sensor 10 emits light towards a sensing area AS in the external space of the vehicle 5, corresponding to its placement location and field of view. The optical sensor 10 receives reflected light that enters the sensing area AS after the emitted light is reflected back from it. The optical sensor 10 is also capable of receiving ambient light that enters the sensing area AS when the emission of light to the sensing area AS is stopped. Hereafter, these reflected light and ambient light will be collectively referred to as area light.

[0019] The optical sensor 10 senses targets Tr that are present within the sensing area AS and reflect light, in response to such light reception. In particular, sensing in the first embodiment means measuring the reflected light intensity IR, which is the intensity of the reflected light reflected from the target Tr. Typical targets to be observed in the optical sensor 10 applied to the vehicle 5 may be at least one of the following moving objects, such as pedestrians, cyclists, animals other than humans, and other vehicles. Typical targets to be observed in the optical sensor 10 applied to the vehicle 5 may be at least one of the following stationary objects, such as guardrails, road signs, roadside structures, and fallen objects on the road.

[0020] As shown in Figure 2, the optical sensor 10 comprises a housing 11, a light-emitting unit 21, a scanning unit 31, and a light-receiving unit 41. The housing 11 is box-shaped and has light-shielding properties. The housing 11 houses the light-emitting unit 21, the scanning unit 31, and the light-receiving unit 41 inside. The housing 11 has a light-transmitting optical window 12.

[0021] The light projection unit 21 includes a light projector 22 and a light projection lens 26. As shown in Figure 4, the light projector 22 is formed by arranging a plurality of laser diodes 24 in an array on a substrate. Each laser diode 24 is arranged in a single row along the Y axis. Each laser diode 24 has a resonator structure capable of resonating with light oscillated in a PN junction layer, and a mirror layer structure capable of repeatedly reflecting light across the PN junction layer. Each laser diode 24 emits pulsed light in the near-infrared region, which is difficult for humans to see in the external space of the vehicle 5, including the sensing area AS outside the optical sensor 10, according to a control signal from the control device 1.

[0022] The light projector 22 has a light projection window 25 formed on one side of the substrate, which is pseudo-defined by a rectangular contour along the Y-axis on its longer side. The light projection window 25 is configured as an aggregate of projection apertures from each laser diode 24. The light emitted from the projection aperture of each laser diode 24 is projected from the light projection window 25 as a longitudinal line of illumination along the Y-axis in the sensing area AS. The illumination light may include non-emitting portions in the Y-axis direction corresponding to the spacing between the laser diodes 24. Even in this case, it is preferable that a line of illumination is formed in the sensing area AS, with the non-emitting portions macroscopically eliminated by diffraction.

[0023] As shown in Figure 2, the light projection lens 26 guides the light emitted from the light projector 22 toward the scanning mirror 32 of the scanning unit 31. One or more light projection lenses 26 are provided to perform at least one optical function, such as focusing, collimating, and shaping.

[0024] The scanning unit 31 comprises a scanning mirror 32 and a scanning motor 35. The scanning mirror 32 is formed in the shape of a plate with a reflective film deposited on a reflective surface 33, which is one side of a substrate. The scanning mirror 32 is supported by the housing 11 so as to be rotatable around a rotation centerline along the Y axis. The scanning mirror 32 oscillates within a finite drive range determined by a mechanical or electrical stopper. The scanning motor 35 rotates (i.e., oscillates) the scanning mirror 32 within a finite drive range according to a control signal from the control device 1. At this time, the rotation angle of the scanning mirror 32 changes sequentially in accordance with the irradiation period of the irradiation light, which substantially coincides with the pulse emission period of each laser diode 24.

[0025] The scanning mirror 32 reflects the incident light from the light-emitting lens 26 of the light-emitting unit 21 using the reflective surface 33 and projects it onto the sensing area AS through the optical window 12, thereby scanning the sensing area AS according to the rotation angle of the scanning motor 35. In particular, in the first embodiment, the mechanical scanning of the sensing area AS by the incident light is substantially limited in the horizontal direction.

[0026] The scanning mirror 32 reflects the area light (i.e., reflected light and ambient light) incident from the sensing area AS through the optical window 12 towards the light receiving unit 41 using its reflective surface 33, according to the rotation angle of the scanning motor 35. Here, the speed of the illuminated light and the reflected light are sufficiently large compared to the rotational speed of the scanning mirror 32. As a result, the reflected light relative to the illuminated light is further reflected to the light receiving lens 42 so that it travels in the opposite direction to the illuminated light on the scanning mirror 32, which is rotating at approximately the same angle as the illuminated light.

[0027] The light receiving unit 41 includes a light receiving lens 42 and a light receiver 45. The light receiving lens 42 is positioned below the light emitting lens 26 in the Y-axis direction. The light receiving lens 42 guides the area light incident from the scanning mirror 32 toward the light receiver 45. One or more light receiving lenses 42 are provided to image the area light toward the light receiver 45.

[0028] The light receiver 45 is positioned below the light emitter 22 in the Y-axis direction. The light receiver 45 receives area light from the sensing area AS, which is imaged by the light receiving lens 42, and outputs a light receiving signal corresponding to the received light. As shown in Figure 5, the light receiver 45 has a rectangular-shaped light receiving surface 47 formed on one side of the substrate. The light receiving surface 47 is composed of an aggregate of the incident surfaces of each light receiving pixel 46, which will be described later. Each light receiving pixel 46 receives area light incident from the light receiving lens 42 onto the light receiving surface 47. Here, the longer side of the rectangular-shaped light receiving surface 47 is defined along the Y-axis. As a result, the reflected light from the incident light that forms a line in the sensing area AS is received as a beam that spreads out in a line shape.

[0029] As shown in Figure 5, the photodetector 45 has multiple light-receiving pixels 46 arranged in a single row in the X-axis direction and a one-dimensional array in the Y-axis direction. As shown in Figure 6, each light-receiving pixel 46 is composed of multiple single-photon avalanche diodes (SPADs) 460. For each light-receiving pixel 46, the SPADs 460 are arranged in a two-dimensional array in the X-axis and Y-axis directions that is finer than each of the light-receiving pixels 46. For the sake of visual clarity, only some of the light-receiving pixels 46 are labeled in Figure 5, and only some of the SPADs 460 are labeled in Figure 6.

[0030] Each light-receiving pixel 46 has a SPAD 460 that outputs a pulse signal in response to the area light it receives. As shown in Figure 7, each light-receiving pixel 46 has a light-receiving circuit 461 for outputting a pulse signal for its own SPAD 460. Specifically, in the light-receiving circuit 461, a reverse bias voltage VB is applied to the cathode of the SPAD 460 via a quench resistor 462. In addition, an inverter 463 that outputs a pulse signal with a voltage amplitude VI is connected to the midpoint of the connection between the SPAD 460 and the quench resistor 462 in the light-receiving circuit 461.

[0031] In this light receiving circuit 461, when area light is input to the SPAD460, as shown in Figure 8, the SPAD460 responds at the timing ts shown in Figure 9, according to the response probability PDE which depends on the voltage VS across the SPAD460, and a pulse signal with voltage amplitude VI is output. At this time, the voltage VS across the SPAD460 drops instantaneously in response to the SPAD460's response and then gradually recovers toward the reverse bias voltage VB, thus being recharged. Therefore, when the SPAD460 responds to ambient light from the area light, independently of reflected light, the average value of the voltage VS across the SPAD460 within a unit time decreases due to the recovery process of the voltage VS, and the response probability PDE also decreases. In particular, if the ambient light intensity is strong, the recharge time per unit time increases, and the response probability PDE of the SPAD460, i.e., the response sensitivity, decreases. For these reasons, in an ambient light receiving environment, the correlation between the number of responses of the SPAD460 to reflected light and the reflection intensity IR of the reflected light fluctuates according to the intensity of the ambient light. Therefore, simply correcting the IR reflection intensity using the simple ratio of the SPAD460's response number to reflected light to its response number to ambient light is unlikely to yield sufficient accuracy.

[0032] As shown in Figures 2 and 3, the photodetector 45 has an output circuit 48. The output circuit 48 performs sampling processing at sampling periods according to a control signal from the control device 1 in the measurement frame FM shown in Figure 10, which is associated with the rotation angle of the scanning mirror 32 according to the irradiation period of the irradiated light. At this time, the output circuit 48 generates a photodetector signal for each photodetector pixel 46 by synthesizing pulse signals from the SPAD 460 that respond to the unit time which is the sampling period, so that the number of responses of the SPAD 460 per unit time is expressed by amplitude. The photodetector signals for each photodetector pixel 46 thus generated are output from the output circuit 48 to the control device 1 for each scanning line.

[0033] As shown in Figure 10, at the timing when only ambient light is received from the area light, only the ambient light output component WRo, which represents the response number of the SPAD460 to the ambient light, appears in the amplitude waveform WR of the received signal. On the other hand, at the timing when reflected light from the irradiated light is received together with the ambient light, the reflected light output component WRr, which represents the response number of the SPAD460 to the reflected light, is superimposed on the ambient light output component WRo and appears in the amplitude waveform WR of the received signal. Here, the amplitude waveform WR of the received signal refers to the time transition of the voltage amplitude per unit time related to the received signal in the measurement frame FM, and is specifically defined as the received signal waveform WR in the first embodiment.

[0034] The control device 1 shown in Figure 1 is connected to the optical sensor 10 via at least one of the following: a LAN (Local Area Network), a wire harness, and an internal bus. The control device 1 is configured to include at least one dedicated computer. The dedicated computer constituting the control device 1 may be a sensor ECU (Electronic Control Unit) specialized in controlling the optical sensor 10, in which case the sensor ECU may be housed in the housing 11. The dedicated computer constituting the control device 1 may be a driving control ECU that controls the operation of the vehicle 5. The dedicated computer constituting the control device 1 may be a navigation ECU that navigates the driving path of the vehicle 5. The dedicated computer constituting the control device 1 may be a locator ECU that estimates the self-state quantities of the vehicle 5.

[0035] The dedicated computer constituting the control device 1 has at least one memory 1a and at least one processor 1b. Memory 1a is at least one type of non-transitory tangible storage medium, such as semiconductor memory, magnetic media, and optical media, which non-temporarily stores programs and data that can be read by the computer. Processor 1b includes at least one type as a core, such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), RISC (Reduced Instruction Set Computer)-CPU, DFP (Data Flow Processor), and GSP (Graph Streaming Processor).

[0036] The processor 1b executes multiple instructions contained in the control program stored in memory 1a. This allows the control device 1 to construct multiple functional blocks for controlling the optical sensor 10. In this way, the control device 1 constructs multiple functional blocks by having the processor 1b execute multiple instructions from the control program stored in memory 1a for controlling the optical sensor 10. The multiple functional blocks constructed by the control device 1 include a signal acquisition block 100 and an intensity measurement block 110, as shown in Figure 3.

[0037] Through the combined action of these blocks 100 and 110, the control method by which the control device 1 controls the optical sensor 10 is executed according to the control flow shown in Figure 11. This control flow is repeatedly executed for each measurement frame FM for each scanning line while the vehicle 5 is starting up. In the control flow, each "S" represents multiple steps executed by multiple instructions included in the control program.

[0038] In control flow S10, the signal acquisition block 100 provides a control signal to the photodetector 45 in the current measurement frame FM, thereby acquiring the received signal waveform WR for each photodetector 46 from the output circuit 48 of the photodetector 45 (see Figure 3). At this time, the received signal waveform WR acquired for each photodetector 46 will include the reflected light output component WRr from the SPAD460, which is in response to the reception of reflected light from the irradiated light, and the ambient light output component WRo from the SPAD460, which is in response to the reception of ambient light, as shown in Figure 10.

[0039] As shown in Figure 11, the control flow transitions to S20 after the completion of S10. In S20, the intensity measurement block 110 extracts the number of reflected light responses NRr, which is the number of SPAD460s that output the reflected light output component WRr, and the number of ambient light responses NRo, which is the number of SPAD460s that output the ambient light output component WRo, from the received signal waveform WR for each of the 46 light-receiving pixels (see Figure 3). At this time, the intensity measurement block 110 separates the ambient light output component WRo from the reflected light output component WRr using a waveform processing algorithm, for example, to separate these output components WRr and WRo as shown in the solid line graph and dashed line graph in Figure 10. Furthermore, the intensity measurement block 110 extracts the number of reflected light responses NRr corresponding to the voltage amplitude at the peak timing tp in Figure 10 from the separated reflected light output component WRr. At the same time, the intensity measurement block 110 extracts the ambient light response number NRo, which corresponds to the average voltage amplitude of the delimited ambient light output component WRo.

[0040] As shown in Figure 11, the control flow transitions to S30 after the completion of S20. In S30, the intensity measurement block 110 estimates the reflection intensity IR of the received and reflected light in the measurement frame FM for each of the 46 receiving pixels based on the correlation between the extracted reflected light response number NRr and the ambient light response number NRo (see Figure 3). At this time, the intensity measurement block 110 may estimate the reflection intensity IR corresponding to the correlation pair between the reflected light response number NRr and the ambient light response number NRo based on a matrix correlation table assumed and pre-stored in memory 1a, as shown in Figure 12. The intensity measurement block 110 may also estimate the reflection intensity IR corresponding to the reflected light response number NRr and the ambient light response number NRo based on a three-dimensional correlation map or correlation function equation assumed and pre-stored in memory 1a, as shown in Figure 13 and Equation 1. In Equation 1, a1, a2, and a3 are coefficient parameters that are set in advance based on, for example, regression analysis or empirical rules.

number

[0041] As shown in Figure 11, the control flow completes the measurement of the reflected intensity IR related to the target Tr by estimation in S30 for each light-receiving pixel 46 that constitutes the scanning line corresponding to the measurement frame FM, and then the execution of this step is completed. The reflected intensity IR for each light-receiving pixel 46, which will be measured for each measurement frame FM of each scanning line, is combined into intensity image data for all scanning lines and then output, which is used for driving control, including the automatic driving control mode of the vehicle 5.

[0042] (Effects and Benefits) Now, I will explain the effects and benefits of the first embodiment described above.

[0043] In the first embodiment, the received signal waveform WR is acquired for each receiving pixel 46 so as to include the reflected light output component WRr from the SPAD 460, which responds to the reception of reflected light from the optical sensor 10 to the sensing area AS, and the ambient light output component WRo from the SPAD, which responds to the reception of ambient light from the sensing area AS. Therefore, according to the first embodiment, the reflected light intensity IR can be accurately estimated by reflecting the influence of ambient light reception for each receiving pixel 46 based on the correlation between the number of reflected light responses NRr, which is the number of SPAD 460s that output the reflected light output component WRr, and the number of ambient light responses NRo, which is the number of SPAD 460s that output the ambient light output component WRo. Thus, it is possible to ensure the measurement accuracy of the reflected light intensity IR by such estimation.

[0044] In the first embodiment, the reflected light response number NRr and the ambient light response number NRo are extracted from the reflected light output component WRr and the ambient light output component WRo, respectively, which are separated from the received light signal waveform WR. Based on this, the reflected light response number NRr and the ambient light response number NRo, which can be extracted individually by separation, can be accurately estimated to reflect the influence of ambient light reception, IR. Therefore, it is possible to improve the measurement accuracy of the reflected light intensity IR by such estimation.

[0045] (Second embodiment) The second embodiment is a modification of the first embodiment.

[0046] In the second embodiment shown in Figure 14, sensing means measuring not only the reflection intensity IR of the reflected light reflected from the target Tr, but also the reflection point distance DR, which is the distance from the optical sensor 10 to the target Tr, which is the point of reflection of the reflected light. Therefore, in addition to the signal acquisition block 100 and intensity measurement block 110, a response setting block 120 and a distance measurement block 130 are added to the multiple functional blocks constructed in the second embodiment. Accordingly, as shown in Figure 15, the control flow of the second embodiment executes S200 before executing S10 to S30 and S240.

[0047] In S200, the response setting block 120 sets the response range RR, which is the array range of SPAD460 that allows responses, for each of the 46 light-receiving pixels according to the intensity of ambient light (see Figure 14). At this time, the response setting block 120 reads the ambient light response number NRo extracted in S20 of the past measurement frame FM from the memory 1a as the past response number NRop, corresponding to the ambient light output component WRo included in the previously acquired light-receiving signal waveform WR, for the same scan line as the current measurement frame FM from the current control flow. The response setting block 120 further compares the read past response number NRop with the judgment criterion NRb.

[0048] In particular, in the second embodiment, the time lag between the current measurement frame FM corresponding to the scan line of the current control flow and the past measurement frame FM corresponding to the scan line of the past control flow is very short. Therefore, assuming that the change in ambient light intensity between the current measurement frame FM and the past measurement frame FM is minute, S200 determines the ambient light intensity in the current measurement frame FM by comparing the past response number NRop with the judgment criterion NRb. Here, the judgment criterion NRb is pre-set to define the boundary between ambient light intensity for which measurement of reflectance IR should be prioritized and ambient light intensity for which measurement of reflect point distance DR should be prioritized, and is stored in the characteristic memory area 1as, and is read out together with the past response number NRop in S200.

[0049] As a result of the comparison, if the past response count NRop exceeds the judgment criterion NRb, the response setting block 120 of S200 sets the response range RR of the SPAD460 to be narrower than when the past response count NRop is less than or equal to the judgment criterion NRb, as shown in Figures 15-19. In the second embodiment, where scanning by the irradiated light is substantially limited to mechanical scanning in the horizontal direction, similar to the first embodiment, the response range RR is adjusted in the X-axis direction corresponding to the horizontal direction. This adjustment of the response range RR in the X-axis direction relies on the phenomenon that, due to horizontal scanning by a long, vertical line of irradiated light, the reflection intensity IR of the received reflected light decreases from the center position Cx in the X-axis direction toward both ends Ex, as shown in Figure 18, and is distributed for each row of SPAD460 along the Y-axis direction. At the same time, in the Y-axis direction corresponding to the vertical direction in the second embodiment, the response range RR is fixed as shown in Figures 16 and 17, regardless of the relationship between the past response count NRop and the judgment criterion NRb. This fixing of the response range RR in the Y-axis direction relies on the phenomenon where the reflection intensity IR of the received reflected light is suppressed in each row of SPAD460 along the Y-axis direction, due to horizontal scanning by a long, vertically oriented line of irradiating light.

[0050] In the second embodiment in particular, two different ranges, RRw and RRn, are assumed as the response range RR in the X-axis direction, as shown in Figures 16 to 19. Specifically, the wide-side response range RRw, which is the response range RR when the past response number NRop is less than or equal to the judgment criterion NRb, is fixed in each light-receiving pixel 46 as shown in Figures 16 and 18, within the widest possible range, not extending from the center position Cx in the X-axis direction to both ends Ex, regardless of the change in the past response number NRop, as shown in Figure 19.

[0051] On the other hand, the narrow response range RRn, which is the response range RR when the past response count NRop exceeds the judgment criterion NRb, is variably set in each light-receiving pixel 46 to a range narrower than the wide response range RRw, from the center position Cx in the X-axis direction to both ends Ex, as shown in Figures 17 and 18, as shown in Figure 19. At this time, the variable setting of the narrow response range RRn is performed so that the response range RR becomes narrower as the past response count NRop increases. Here, in the X-axis direction, the center position Cx of each light-receiving pixel 46 that determines the origin position of each response range RRw and RRn ideally corresponds to the peak reception position where the reflection intensity IR of the reflected light received by each light-receiving pixel 46 is at its maximum intensity, as shown in Figure 18.

[0052] In S10 of Figure 15, which follows S200, the received signal waveform WR acquired in the measurement frame FM will include the reflected light output component WRr and the ambient light output component WRo output from the SPAD460 within the response range RR. Accordingly, in S20 following S200, the reflected light response number NRr and the ambient light response number NRo within the response range RR are extracted, and in S30 following S200, the reflected intensity IR is measured by estimation based on these response numbers NRr and NRo within the response range RR. In particular, in S20 following S200, at least the ambient light response number NRo from the extracted results is stored in the characteristic memory area 1as.

[0053] Furthermore, as a step following S200, the control flow of the second embodiment executes S240, which transitions from S30. In S240, the distance measurement block 130 measures the reflection point distance DR of the reflected light based on the received signal waveform WR acquired in S10 (see Figure 14). In the second embodiment, the reflection point distance DR is acquired by dTOF (direct Time Of Flight), which is based on the time difference between the peak timing tp of the reflected light output component WRr recognized from the received signal waveform WR in S20 (see Figure 10 of the first embodiment) and the irradiation start timing when the irradiation cycle of the irradiated light started in the measurement frame FM.

[0054] As shown in Figure 15, the control flow completes the measurement of the reflection point distance DR for each light-receiving pixel 46 that constitutes the scanning line corresponding to the measurement frame FM in the current measurement, and the execution ends in S240. The reflection point distance DR for each light-receiving pixel 46, which will be measured for each measurement frame FM of each scanning line, is combined into distance image data for all scanning lines and then output, which is used for driving control, including the automatic driving control mode of the vehicle 5. Similarly to the first embodiment, in the second embodiment as well, the reflection intensity IR for each light-receiving pixel 46, which will be measured for each measurement frame FM of each scanning line, is combined into intensity image data for all scanning lines and then output, which is used for driving control, including the automatic driving control mode of the vehicle 5. Furthermore, the result of setting the response range RR may also be output along with the output of these distance image data and intensity image data.

[0055] (Effects and Benefits) Now, I will explain the effects and advantages specific to the second embodiment described above.

[0056] According to the second embodiment, prior to acquiring the received signal waveform WR, the response range RR, which is the array range of SPAD460 that allows response, is set for each of the light-receiving pixels 46 according to the ambient light intensity. This allows the response range RR for each of the light-receiving pixels 46 to be adapted to either the estimation of the reflection intensity IR based on the correlation of each output component WRr,WRo included in the received signal waveform WR, or the measurement of the reflection point distance DR based on the received signal waveform WR, depending on the ambient light intensity. Therefore, it becomes possible to appropriately select the measurement that prioritizes accuracy by reflecting the ambient light intensity, between the measurement of the estimated reflection intensity IR and the measurement of the reflection point distance DR.

[0057] According to the second embodiment, prior to the acquisition of the received signal waveform WR, if the past response number NRop, which is the ambient light response number NRo corresponding to the ambient light output component WRo included in the previously acquired received signal waveform WR, exceeds the judgment criterion NRb, the response range RR(RRn) is set to be narrower than the response range RR(RRw) when the past response number NRop is less than or equal to the judgment criterion NRb. This makes it possible to prioritize the measurement accuracy of the reflection point distance DR by using a narrow response range RR(RRn) when the past response number NRop is increasing due to high ambient light intensity, while prioritizing the measurement accuracy of the reflection intensity IR by using a wide response range RR(RRw) when the past response number NRop is decreasing due to low ambient light intensity. Therefore, along with the output of the distance image data and intensity image data described above, the type of data for which accuracy is prioritized may also be output instead of or in addition to the setting result of the response range RR.

[0058] According to the second embodiment, the response range RR(RRn) is set to be narrower as the number of past responses NRop increases when the judgment criterion NRb is exceeded. As a result, under conditions where the number of past responses NRop is increasing, the narrower the response range RR(RRn) for higher ambient light intensity, the more the influence of measurement errors caused by ambient light reception noise on the reflection point distance DR can be suppressed. Therefore, it becomes possible to improve the measurement accuracy of the reflection point distance DR.

[0059] (Third embodiment) The third embodiment is a modification of the second embodiment.

[0060] As shown in Figure 20, the control flow of the third embodiment executes S300 instead of S200. In S300, the response setting block 120 sets one of two different ranges, RRw and RRn, as the response range RR of the SPAD 460 that is permitted to respond, for each scan line corresponding to the measurement frame FM, as shown in Figure 21. Note that, for the sake of visual clarity in the illustration, only some of the light-receiving pixels 46 that make up each scan line are labeled in Figure 21.

[0061] In this case, as shown in Figures 20 and 21, in the second embodiment, the narrow-side response range RRn is set for the measurement frame FM of the odd-numbered scanning line from the scanning origin OS, while the wide-side response range RRw is set for the measurement frame FM of the even-numbered scanning line from the scanning origin OS. Alternatively, as shown in Figure 22, the narrow-side response range RRn may be set for the measurement frame FM of the even-numbered scanning line from the scanning origin OS, while the wide-side response range RRw may be set for the measurement frame FM of the odd-numbered scanning line from the scanning origin OS. In either of these cases, the intensity image data and distance image data for all scanning lines will be data (see Figure 21) in which the measurement of the reflected intensity IR and reflected point distance DR is performed with the narrow-side response range RRn and the measurement is performed with the wide-side response range RRw, respectively, alternating between the two.

[0062] (Effects and Benefits) Now, I will explain the effects and advantages specific to the third embodiment described above.

[0063] According to the third embodiment, prior to acquiring the received signal waveform WR, two different response ranges RRw and RRn are set for each scan line of the receiving pixel 46 as the response range RR of the SPAD460 that allows response. This allows the scan line of the wide response range RRw to be fitted for estimating the reflection intensity IR based on the correlation of each output component WRr and WRo included in the received signal waveform WR, while the scan line of the narrow response range RRn can be fitted for measuring the reflection point distance DR based on the received signal waveform WR. Therefore, it is possible to ensure a good balance between the measurement accuracy of the estimated reflection intensity IR and the measurement accuracy of the reflection point distance DR for each scan line.

[0064] (Fourth embodiment) The fourth embodiment is a modification of the second embodiment.

[0065] As shown in Figure 23, the control flow of the fourth embodiment executes S400 instead of S200. In S400, the response setting block 120 sets one of two different wide and narrow ranges RRw and RRn as the response range RR of the SPAD460 that allows a response, depending on the comparison between the estimated distance DE to the target Tr to be sensed and the reference distance DEb, for each light-receiving pixel 46 of the scanning line corresponding to the measurement frame FM. Here, the reference distance DEb is set in advance to determine the boundary between the distance to the target Tr for which measurement of the reflection intensity IR needs to be prioritized and the distance to the target Tr for which measurement of the reflection point distance DR needs to be prioritized, and is stored in the characteristic storage area 1as (see Figure 14 described in the second embodiment) and read out in S400.

[0066] In this case, the estimated distance DE to the target Tr to be sensed may be estimated based on the reflection point distance DR measured by the same or adjacent light-receiving pixel 46 as the current measurement frame FM, on the same scanning line as the current measurement frame FM, among the scanning lines corresponding to the past measurement frame FM. This estimation assumes that the change in distance to the target Tr is small between the same scanning lines of the current measurement frame FM and the past measurement frame FM. The estimated distance DE may also be estimated based on the reflection point distance DR measured by the same or adjacent light-receiving pixel 46 as the current measurement frame FM, on the scanning line immediately preceding the current measurement frame FM, which is adjacent to the scanning origin OS side, among the scanning lines corresponding to the past measurement frame FM. This estimation assumes that the probability of detecting the same target Tr by sensing is high between consecutive scanning lines of the current measurement frame FM and the past measurement frame FM.

[0067] For photodetecting pixels 46 where the estimated distance DE to the target Tr is short (below the reference distance DEb), the response range RR is set to the wide-side response range RRw, prioritizing accuracy in measuring the reflection intensity IR over the reflection point distance DR. On the other hand, for photodetecting pixels 46 where the estimated distance DE to the target Tr is long (beyond the reference distance DEb), the response range RR is set to the narrow-side response range RRn, prioritizing accuracy in measuring the reflection point distance DR over the reflection intensity IR. Although not shown in the diagram, for photodetecting pixels 46 where the target Tr is not detected by sensing, the response range RR is set to the narrow-side response range RRn, similar to the latter case of long distances, prioritizing accuracy in measuring the reflection point distance DR over the reflection intensity IR.

[0068] (Effects and Benefits) Now, I will explain the effects and advantages specific to the fourth embodiment described above.

[0069] According to the fourth embodiment, prior to acquiring the received signal waveform WR, the response range RR, which is the array range of SPAD460 that allows response, is set for each of the light-receiving pixels 46 according to the estimated distance DE to the target Tr being sensed. This allows the response range RR for each of the light-receiving pixels 46 to be adapted to either the estimation of the reflection intensity IR based on the correlation of each output component WRr,WRo included in the received signal waveform WR, or the measurement of the reflection point distance DR based on the received signal waveform WR, depending on the distance to the target Tr. Therefore, it becomes possible to appropriately select the measurement that prioritizes accuracy between the measurement of the estimated reflection intensity IR and the measurement of the reflection point distance DR, according to the distance to the target Tr.

[0070] According to the fourth embodiment, prior to acquiring the received signal waveform WR, the response range RR(RRn) when the estimated distance DE to the sensed target Tr exceeds the reference distance DEb is set to be narrower than the response range RR(RRw) when the estimated distance DE is less than or equal to the reference distance DEb. This makes it possible to prioritize the measurement accuracy of the reflected intensity IR with a wide response range RR(RRw) when the target Tr is at a short distance, while prioritizing the measurement accuracy of the reflected point distance DR with a narrow response range RR(RRn) when the target Tr is at a long distance. Therefore, in the fourth embodiment, along with the output of distance image data and intensity image data similar to the second embodiment, at least one of the setting result of the response range RR and the type of data for which accuracy is prioritized may also be output.

[0071] (Fifth embodiment) The fifth embodiment is a modification of the first embodiment.

[0072] As shown in Figure 24, the control flow of the fifth embodiment executes S530 instead of S30. In S530, the intensity measurement block 110 estimates the reflection intensity IR of the received and reflected light in the measurement frame FM for each light-receiving pixel 46 based on the correlation between the reflected light response number NRr and the ambient light response number NRo for each ambient temperature τ. At this time, the current ambient temperature τ is recognized, for example, using the detection result of a temperature sensor provided in the optical sensor 10 or the vehicle 5.

[0073] Here, as shown in Figure 25, the intensity measurement block 110 may estimate the reflected intensity IR corresponding to the correlation pair between the reflected light response number NRr and the ambient light response number NRo at the current ambient temperature τ, based on a matrix correlation table pre-stored in memory 1a, which is assumed for each ambient temperature τ. The intensity measurement block 110 may also estimate the reflected intensity IR corresponding to the reflected light response number NRr and the ambient light response number NRo at the current ambient temperature τ, based on a three-dimensional correlation map or the correlation function equation of Equation 2, which is assumed for each ambient temperature τ and is pre-stored in memory 1a. In Equation 2, a1, a2, a3, and a4 are coefficient parameters that are set in advance based on, for example, regression analysis or empirical rules.

number

[0074] (Effects and Benefits) Now, I will explain the effects and advantages specific to the fifth embodiment described above.

[0075] According to the fifth embodiment, the reflected light intensity IR for each light-receiving pixel 46 is estimated based on the correlation between the reflected light response number NRr and the ambient light response number NRo at different ambient temperatures τ. This allows for the estimation of an accurate reflected light intensity IR for each light-receiving pixel 46, reflecting the influence of temperature characteristics on, for example, the emission intensity of the light emitter 22 that provides the illumination light, and / or the influence of temperature characteristics on, for example, the light-receiving sensitivity and dead time of the SPAD 460 that receives the reflected light from the illumination light. Therefore, this estimation makes it possible to improve the measurement accuracy of the reflected light intensity IR. The fifth embodiment may be combined with the second to fourth embodiments as a modification thereof.

[0076] (Sixth Embodiment) The sixth embodiment is a modification of the first embodiment.

[0077] As shown in Figure 26, in the sixth embodiment, a corrective reflective target Tc is provided in the sensing area AS, which is also set up inside the optical sensor 10. The corrective reflective target Tc is formed so that it can reflect the illumination light emitted from the scanning mirror 32 and re-incidentate it back into the mirror 32. Furthermore, in order to achieve these optical characteristics, the corrective reflective target Tc is positioned either around the optical window 12 in the housing 11, or inside the housing 11, between the said surrounding area and the scanning mirror 32.

[0078] In this sixth embodiment, the control method for controlling the optical sensor 10 is executed according to the correction flow shown in Figure 27 and the control flow shown in Figure 28. First, the correction flow in Figure 27 will be explained. The correction flow of the sixth embodiment is repeatedly executed at each irradiation cycle of the irradiated light, or at set time intervals longer than the irradiation cycle, in the measurement frame FM of the scanning line where the scanning target by the irradiated light is the correction reflection target Tc, while the vehicle 5 is starting up. In the correction flow, each "S" represents multiple steps executed by multiple instructions included in the control program.

[0079] In steps S610, S620, and S630 of the correction flow, processing equivalent to steps S10, S20, and S30 of the control flow is executed in the measurement frame FM of the scanning line for the correction reflection target Tc. The correction flow then proceeds to S650 after the completion of S630. In S650, the intensity measurement block 110 obtains the ratio IRb / IR, which is the ratio of the reflection intensity IR estimated for the correction reflection target Tc in S630 to the reference intensity IRb, as the correction coefficient γ. Here, the reference intensity IRb is set in advance based on, for example, a common design value for the product (i.e., an ideal value) or an initial value for each product, and is stored in the characteristic memory area 1as as shown in Figure 26, and is read out when S650 is executed. In S650, the intensity measurement block 110 further stores the latest acquired correction coefficient γ in the characteristic memory area 1as of memory 1a.

[0080] Next, the control flow shown in Figure 28 will be explained. In the sixth embodiment, where the object scanned by the irradiated light is the target Tr, similar to the first embodiment, S660 is executed instead of S30. In S660, the intensity measurement block 110 corrects the reflected intensity IR estimated with respect to the target Tr in accordance with S30 by multiplying it by the latest correction coefficient γ read from the characteristic memory area 1as as shown in Figure 26. Note that in S650, the ratio IR / IRb is obtained as the correction coefficient γ, so in the correction in S660, the reflected intensity IR may be divided by the latest correction coefficient γ.

[0081] (Effects and Benefits) Now, I will explain the effects and advantages specific to the sixth embodiment described so far.

[0082] In the sixth embodiment, the ratio of the estimated reflection intensity IR with respect to the correction reflection target Tc provided in the sensing area AS inside the optical sensor 10 to the reference intensity IRb is obtained as a correction coefficient γ. Thus, according to the sixth embodiment, the estimated reflection intensity IR with respect to the target Tr is corrected by the correction coefficient γ. As a result, even if characteristic changes such as aging or temperature changes occur in the light emitter 22 that provides the illumination light and / or the SPAD 460 that receives the reflected light from the illumination light, an accurate reflection intensity IR that reflects these characteristic changes can be estimated with respect to the target Tr. Therefore, it is possible to improve the measurement accuracy of the reflection intensity IR by such estimation. The sixth embodiment may be combined with the second to fifth embodiments as a modification thereof.

[0083] (Other embodiments) Although several embodiments have been described above, this disclosure is not intended to be limited to those embodiments, and can be applied to various embodiments and combinations without departing from the spirit of this disclosure.

[0084] In the modified example, the dedicated computer constituting the control device 1 may have at least one of the digital circuit and analog circuit as a processor. Here, the digital circuit is at least one of the following, for example, ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), SOC (System on a Chip), PGA (Programmable Gate Array), and CPLD (Complex Programmable Logic Device). Furthermore, such a digital circuit may have a memory that stores a program.

[0085] In S10 by the modified signal acquisition block 100, an additional measurement frame FM may be performed to acquire a received signal waveform WR containing only the ambient light output component WRo by stopping the irradiation of the illumination light to the same scanning line as the current measurement frame FM. In this case, in S20 by the intensity measurement block 110, the ambient light response number NRo may be extracted from the average voltage amplitude of the ambient light output component WRo in the additional measurement frame FM, or from the average voltage amplitude of the ambient light output component WRo in the additional measurement frame FM and the current measurement frame FM.

[0086] In S20 using the modified intensity measurement block 110, the reflected light output component WRr and the ambient light output component WRo may not be separated from the received light signal waveform WR, and the corresponding response numbers NRr and NRo may be directly extracted from these output components WRr and WRo, respectively. In S200 using the modified response setting block 120, the response range RR(RRw,RRn) may be set according to the ambient light intensity detected in the measurement frame FM by a sensor different from the optical sensor 10, such as an ambient light sensor.

[0087] In S200 using the modified response setting block 120, the wide-side response range RRw may be variably set such that it narrows as the number of past responses NRop increases, as long as it is wider than the narrow-side response range RRn. In S200, S300, and S400 using the modified response setting block 120, the wide-side response range RRw may be set to cover the entire area of ​​each light-receiving pixel 46. In S200 using the modified response setting block 120, the narrow-side response range RRn may be fixed regardless of the change in the number of past responses NRop, as long as it is narrower than the wide-side response range RRw. In S200, S300, and S400 using the modified response setting block 120, the narrow-side response range RRn may be set to avoid the center position of each light-receiving pixel 46, as long as it is narrower than the wide-side response range RRw, thereby suppressing the occurrence of measurement errors due to multiple reflections between the near-range target Tr and the optical sensor 10. In S200 using the modified response setting block 120, if the intensity distribution is suppressed for each column of SPAD460, a process to set the response ranges RRw and RRn to the same range, such as setting a wide response range RRn that matches the response range RRw, may be executed. In other words, a process to set one type of response range RR may be executed.

[0088] The modified optical sensor 10 may employ various scanning methods, including a mechanically oscillating method limited to the vertical direction, or a mechanically oscillating method in both the horizontal and vertical directions, in addition to the mechanically oscillating method limited to the horizontal direction as described. The modified optical sensor 10 may employ various two-dimensional or three-dimensional scanning methods, such as a rotary type, a MEMS (Micro Electro Mechanical Systems) type, or a Lissajous type. Depending on the scanning method of the optical sensor 10, in S200, S300, and S400 of the modified response setting block 120, the response range RR may be adjusted in the Y-axis direction corresponding to the vertical direction, in addition to or instead of the X-axis direction corresponding to the horizontal direction. Here, for example, if the scanning method of the optical sensor 10 is a mechanically oscillating method limited to the vertical direction, it is preferable to adjust the response range RR in the Y-axis direction. In the photodetector 45 of the modified optical sensor 10, as shown in Figure 29, a plurality of light-receiving pixels 46 may be arranged in a two-dimensional array in the X-axis direction and the Y-axis direction.

[0089] In the modified version, S200 by the response setting block 120 is executed between S20 and S30, and in S20, the ambient light response number NRo extracted to the measurement frame FM may be used for comparison with the judgment criterion NRb instead of the past response number NRop. In this case, the received signal waveform WR acquired in S10 by the signal acquisition block 100 will include the reflected light output component WRr and the ambient light output component WRo output from the SPAD 460 within the entire range of each light receiving pixel 46. Therefore, in S30 by the intensity measurement block 110 in this case, the response numbers NRr and NRo within the response range RR (RRw, RRn) set in S200 may be extracted from the response numbers NRr and NRo extracted to the measurement frame FM in S20, and these extracted numbers may be used to estimate the reflected intensity IR. In addition, in S240 by the distance measurement block 130 in this case, the reflected light output component WRr corresponding to the number of reflected light responses NRr within the response range RR(RRw,RRn) is extracted from the reflected light output component WRr separated by S20, and the peak timing tp of the extracted component may be used to obtain the reflection point distance DR.

[0090] In S200 using the modified response setting block 120, three or more ranges of different widths and widths may be set as the response range RR according to the ambient light intensity (past response count NRop). In S200 using the modified response setting block 120, the response range RR may be set to change continuously according to the ambient light intensity (past response count NRop). In S300 using the modified response setting block 120, three or more ranges of different widths and widths may be set as the response range RR for each scanning line of the light-receiving pixel 46. In S400 using the modified response setting block 120, three or more ranges of different widths and widths may be set as the response range RR according to the estimated distance DE to the sensed target Tr. In S400 using the modified response setting block 120, the response range RR may be set to change continuously according to the estimated distance DE to the sensed target Tr.

[0091] In the modified example, the mobile body to which the control device 1 is applied may be, for example, an autonomous vehicle whose movement on a road can be remotely controlled. The control device 1 in the modified example may be applied to environments other than mobile bodies. In addition to the embodiments described so far, the above embodiments and modifications may be implemented as a semiconductor device (for example, a semiconductor chip) having at least one processor 1b and one memory 1a.

[0092] (Additional note) This specification discloses several technical concepts and several combinations thereof, as listed below.

[0093] (Technical thought 1) A control device for controlling an optical sensor (10) having a processor (1b) and in which multiple SPADs (460), which are single-photon avalanche diodes, are arranged for each light-receiving pixel (46) to receive light from a sensing area (AS), The aforementioned processor, A received signal waveform (WR) is acquired for each receiving pixel, which includes a reflected light output component (WRr) from the SPAD in response to the reception of reflected light from the optical sensor to the sensing area, and an ambient light output component (WRo) from the SPAD in response to the reception of ambient light from the sensing area. A control device configured to estimate the reflected light intensity (IR) for each light-receiving pixel based on the correlation between the number of reflected light response pixels (NRr), which is the number of SPADs that output the reflected light output component, and the number of ambient light response pixels (NRo), which is the number of SPADs that output the ambient light output component.

[0094] Furthermore, this technical concept 1 and the technical concepts 2 to 10 described below may be implemented in the form of methods and programs.

[0095] (Technical thought 2) Estimating the aforementioned reflectance intensity is A control device according to technical concept 1, which includes extracting the reflected light response number and the ambient light response number, respectively, from the reflected light output component and the ambient light output component separated from the received light signal waveform.

[0096] (Technical Thought 3) The aforementioned processor, Prior to acquiring the received light signal waveform, the response range (RR), which is the array range of the SPAD that allows a response, is set for each light-receiving pixel according to the intensity of the ambient light. A control device according to technical idea 1 or 2, further configured to measure the reflection point distance (DR) of the reflected light based on the received light signal waveform.

[0097] (Technical Thought 4) Setting the aforementioned response range means Prior to acquiring the received light signal waveform in the present, the control device according to technical concept 3, wherein the response range is set narrower when the number of past responses (NRop), which is the number of ambient light responses corresponding to the ambient light output component included in the previously acquired received light signal waveform, exceeds a judgment criterion (NRb), than when the number of past responses is less than or equal to the judgment criterion.

[0098] (Technical Thought 5) Setting the aforementioned response range means The control device according to technical concept 4, wherein the response range is narrowed as the number of past responses increases when the judgment criterion is exceeded.

[0099] (Technical Thought 6) The aforementioned processor, Prior to acquiring the received light signal waveform, different wide and narrow ranges (RRw, RRn) are set for each scan line of the light-receiving pixel as the response range (RR), which is the array range of the SPAD that allows response. A control device according to technical idea 1 or 2, further configured to measure the reflection point distance (DR) of the reflected light based on the received light signal waveform.

[0100] (Technical Thought 7) The aforementioned processor, Prior to acquiring the received light signal waveform, the response range (RR), which is the array range of the SPAD that allows a response, is set for each light-receiving pixel according to the estimated distance (DE) to the target (Tr) being sensed. A control device according to technical idea 1 or 2, further configured to measure the reflection point distance (DR) of the reflected light based on the received light signal waveform.

[0101] (Technical Thought 8) Setting the aforementioned response range means Prior to acquiring the received light signal waveform, the control device according to technical concept 7, wherein the response range when the estimated distance exceeds the reference distance (DEb) is set to be narrower than the response range when the estimated distance is less than or equal to the reference distance.

[0102] (Technical Thought 9) Estimating the aforementioned reflectance intensity is A control device according to any one of the technical ideas 1 to 8, which includes estimating the reflection intensity for each light-receiving pixel based on the correlation between the number of reflected light responses and the number of ambient light responses at different ambient temperatures (τ).

[0103] (Technical Thought 10) The aforementioned processor, The system is configured to perform estimation of the reflection intensity with respect to a target (Tr) located in the sensing area outside the optical sensor and a correction reflection target (Tc) provided in the sensing area inside the optical sensor. The estimation of the reflection intensity with respect to the correction reflection target is as follows: This includes obtaining the ratio of the estimated reflection intensity with respect to the correction reflection target to the reference intensity (IRb) as a correction coefficient (γ), The estimation of the reflectance intensity with respect to the aforementioned target is as follows: A control device according to any one of the technical ideas 1 to 9, which includes correcting the estimated reflection intensity with respect to the target by the correction coefficient. [Explanation of Symbols]

[0104] 1: Control device, 1a: Memory, 1b: Processor, 10: Optical sensor, 46: Photodetector pixel, 460: SPAD, AS: Sensing area, DE: Estimated distance, DEb: Reference distance, DR: Reflection point distance, IR: Reflection intensity, IRb: Reference intensity, NRb: Judgment criterion, NRo: Ambient light response count, NRop: Past response count, NRr: Reflected light response count, RR: Response range, RRn: Narrow side response range, RRw: Wide side response range, Tc: Correction reflection target, Tr: Target, WR: Received signal waveform, WRo: Ambient light output component, WRr: Reflected light output component, γ: Correction coefficient, τ: Ambient temperature

Claims

1. A control device for controlling an optical sensor (10) having a processor (1b) and in which multiple SPADs (460), which are single-photon avalanche diodes, are arranged for each light-receiving pixel (46) to receive light from a sensing area (AS), The aforementioned processor, The process involves acquiring a received signal waveform (WR) for each receiving pixel, which includes a reflected light output component (WRr) from the SPAD in response to the reception of reflected light from the optical sensor to the sensing area, and an ambient light output component (WRo) from the SPAD in response to the reception of ambient light from the sensing area. A control device configured to estimate the reflected light intensity (IR) for each light-receiving pixel based on the correlation between the number of reflected light responses (NRr), which is the number of SPADs that output the reflected light output component, and the number of ambient light responses (NRo), which is the number of SPADs that output the ambient light output component.

2. Estimating the aforementioned reflectance intensity is The control device according to claim 1, further comprising extracting the reflected light response number and the ambient light response number from the reflected light output component and the ambient light output component separated from the received light signal waveform, respectively.

3. The aforementioned processor, Prior to acquiring the received light signal waveform, the response range (RR), which is the array range of the SPAD that allows a response, is set for each light-receiving pixel according to the intensity of the ambient light. The control device according to claim 1 or 2, further configured to measure the reflection point distance (DR) of the reflected light based on the received light signal waveform.

4. Setting the aforementioned response range means Prior to acquiring the received light signal waveform in the present, the control device according to claim 3, wherein the response range when the past response number (NRop), which is the number of ambient light responses corresponding to the ambient light output component included in the previously acquired received light signal waveform, exceeds a judgment criterion (NRb) is set to be narrower than the response range when the past response number is less than or equal to the judgment criterion.

5. Setting the aforementioned response range means The control device according to claim 4, wherein the response range is set to be narrower as the number of past responses that exceed the judgment criterion increases.

6. The aforementioned processor, Prior to acquiring the aforementioned light-receiving signal waveform, different wide and narrow ranges (RRw, RRn) are set for each scan line of the light-receiving pixel as the response range (RR), which is the array range of the SPAD that allows response. The control device according to claim 1 or 2, further configured to measure the reflection point distance (DR) of the reflected light based on the received light signal waveform.

7. The aforementioned processor, Prior to acquiring the received light signal waveform, the response range (RR), which is the array range of the SPAD that allows a response, is set for each light-receiving pixel according to the estimated distance (DE) to the target (Tr) being sensed. The control device according to claim 1 or 2, further configured to measure the reflection point distance (DR) of the reflected light based on the received light signal waveform.

8. Setting the aforementioned response range means Prior to acquiring the received light signal waveform, the control device according to claim 7, wherein the response range when the estimated distance exceeds the reference distance (DEb) is set to be narrower than the response range when the estimated distance is less than or equal to the reference distance.

9. Estimating the aforementioned reflectance intensity is The control device according to claim 1 or 2, further comprising estimating the reflection intensity for each light-receiving pixel based on the correlation between the number of reflected light responses and the number of ambient light responses at different ambient temperatures (τ).

10. The aforementioned processor, The system is configured to perform estimation of the reflection intensity with respect to a target (Tr) located in the sensing area outside the optical sensor and a correction reflection target (Tc) provided in the sensing area inside the optical sensor. The estimation of the reflection intensity with respect to the correction reflection target is as follows: This includes obtaining the ratio of the estimated reflection intensity with respect to the correction reflection target to the reference intensity (IRb) as a correction coefficient (γ), The estimation of the reflectance intensity with respect to the aforementioned target is as follows: The control device according to claim 1 or 2, further comprising correcting the estimated reflectance intensity with respect to the target using the correction coefficient.

11. A control method executed by a processor (1b) to control an optical sensor (10) in which multiple SPADs (460), which are single-photon avalanche diodes, are arranged for each light-receiving pixel (46) to receive light from a sensing area (AS), wherein The process involves acquiring a received signal waveform (WR) for each receiving pixel, which includes a reflected light output component (WRr) from the SPAD in response to the reception of reflected light from the optical sensor to the sensing area, and an ambient light output component (WRo) from the SPAD in response to the reception of ambient light from the sensing area. A control method comprising estimating the reflected light intensity (IR) for each light-receiving pixel based on the correlation between the number of reflected light responses (NRr), which is the number of SPADs that output the reflected light output component, and the number of ambient light responses (NRo), which is the number of SPADs that output the ambient light output component.

12. A control program, which includes instructions, is stored in a storage medium (1a) and executed by a processor (1b) in order to control an optical sensor (10) in which multiple SPADs (460), which are single-photon avalanche diodes, are arranged for each light-receiving pixel (46) to receive light from a sensing area (AS), The aforementioned instruction is, The method involves acquiring a received signal waveform (WR) for each receiving pixel, which includes a reflected light output component (WRr) from the SPAD in response to the reception of reflected light from the optical sensor to the sensing area, and an ambient light output component (WRo) from the SPAD in response to the reception of ambient light from the sensing area. A control program that includes estimating the reflected light intensity (IR) for each light-receiving pixel based on the correlation between the number of reflected light responses (NRr), which is the number of SPADs that output the reflected light output component, and the number of ambient light responses (NRo), which is the number of SPADs that output the ambient light output component.

Images