Control device, control method, and control program

Abstract

To provide a control device which suppresses erroneous detection of an optical sensor.SOLUTION: A processor of a control device controlling an optical sensor is configured to: acquire distance image data Dd expressing a distance value to a reflection target reflecting light in a detection area on the basis of an echo detected by an optical sensor with respect to irradiation light with the first intensity I1; acquire intensity image data Di expressing an intensity value of the echo reflected from the reflection target in the detection area on the basis of the echo detected by the optical sensor with respect to the irradiation light with the second intensity I2 lower than the first intensity I1; estimate a flare pixel region Rf where imaging of a flare is predicted in the periphery of a target pixel region Rt where the reflection target is imaged in the distance image data Dd on the basis of the intensity image data Di; and remove a distance value of the flare pixel region Rf where the detection timing of the echo is overlapped on the target pixel region Rt in the distance image data Dd.SELECTED DRAWING: Figure 16

Description

[Technical Field]

[0001] The present disclosure relates to a technique for controlling an optical sensor that detects echoes in response to light irradiated onto a detection area of ​​a vehicle. [Background technology]

[0002] In the technology disclosed in Patent Document 1, two types of pixels with different sensitivities are mixed together in order to detect incident light that becomes a reflected echo of irradiated light in a distance measuring device, which is an optical sensor. This makes it possible to detect incident light without causing false detection due to saturation in the low-sensitivity pixels, even under conditions where the amount of incident light is high. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-190892 Summary of the Invention [Problem to be solved by the invention]

[0004] However, when the echo reflection intensity is high, flare may occur due to factors such as unwanted reflection inside the lens, and the subject's physique may be inaccurately captured by the optical sensor. However, the technology disclosed in Patent Document 1 has difficulty in resolving false detections caused by such flare.

[0005] An object of the present disclosure is to provide a control device that suppresses erroneous detection by an optical sensor. Another object of the present disclosure is to provide a control method that suppresses erroneous detection by an optical sensor. Yet another object of the present disclosure is to provide a control program that suppresses erroneous detection by an optical sensor. [Means for solving the problem]

[0006] The technical means of the present disclosure for solving the problems will be described below. Note that the claims and the reference characters in parentheses in this section indicate the correspondence with the specific means described in the embodiments described later in detail, and do not limit the technical scope of the present disclosure.

[0007] A first aspect of the present disclosure is A control device (1) having a processor (1b) and controlling an optical sensor (10) that detects an echo of an irradiated light irradiated onto a detection area (Ad) of a vehicle (5), The processor acquiring distance image data (Dd) representing a distance value to a reflective target (Tr) that reflects light in a detection area based on an echo detected by an optical sensor in response to irradiation light of a first intensity (I1); acquiring intensity image data (Di) representing an intensity value of an echo reflected from a reflective target in a detection area based on the echo detected by the optical sensor in response to illumination light having a second intensity (I2) lower than the first intensity; Estimating a flare pixel region (Rf) in which a flare is expected to be captured around a target pixel region (Rt) in which a reflective target is captured in the range image data based on the intensity image data; and removing the distance values ​​of the flare pixel regions where the echo detection timing overlaps with the target pixel region in the range image data.

[0008] A second aspect of the present disclosure is A control method executed by a processor (1b) for controlling an optical sensor (10) that detects an echo of an irradiated light onto a detection area (Ad) of a vehicle (5), the method comprising: acquiring distance image data (Dd) representing a distance value to a reflective target (Tr) that reflects light in a detection area based on an echo detected by an optical sensor in response to irradiation light of a first intensity (I1); acquiring intensity image data (Di) representing an intensity value of an echo reflected from a reflective target in a detection area based on the echo detected by the optical sensor in response to illumination light having a second intensity (I2) lower than the first intensity; Estimating a flare pixel region (Rf) in which a flare is expected to be captured around a target pixel region (Rt) in which a reflective target is captured in the range image data based on the intensity image data; The method includes removing distance values ​​of flare pixel regions where the detection timing of the echo overlaps with the target pixel region in the range image data.

[0009] A third aspect of the present disclosure is A control program is stored in a storage medium (1a) for controlling an optical sensor (10) that detects an echo of an irradiated light onto a detection area (Ad) of a vehicle (5), the control program including instructions to be executed by a processor (1b), The command is, acquiring distance image data (Dd) representing a distance value to a reflective target (Tr) that reflects light in a detection area based on an echo detected by an optical sensor in response to irradiation light of a first intensity (I1); acquiring intensity image data (Di) representing an intensity value of an echo reflected from a reflective target in a detection area based on the echo detected by the optical sensor in response to irradiation light having a second intensity (I2) lower than the first intensity; Estimating a flare pixel region (Rf) in which an image of a flare is predicted to be captured around a target pixel region (Rt) in which an image of a reflective target is captured in the range image data based on the intensity image data; The method includes removing the distance values ​​of the flare pixel regions where the detection timing of the echo overlaps with the target pixel region in the distance image data.

[0010] According to these first to third aspects, distance image data representing distance values ​​to reflecting targets in the detection area are acquired based on echoes detected by an optical sensor in response to illumination light of a first intensity. Therefore, in the first to third aspects, intensity image data representing intensity values ​​of echoes reflected from reflecting targets in the detection area are acquired based on echoes detected by an optical sensor in response to illumination light of a second intensity lower than the first intensity. This allows flare pixel regions in the distance image data, where flare is expected to be captured around a target pixel region in which the reflecting target is captured, to be properly estimated based on intensity image data in which the captured flare is suppressed in response to low-intensity illumination light. Therefore, distance values ​​in flare pixel regions in the distance image data where the echo detection timing overlaps with the target pixel region can be eliminated as pseudo values ​​resulting from the capture of flare, thereby preventing erroneous detection of distance values ​​by the optical sensor.

[0011] A fourth aspect of the present disclosure is A control device (1) having a processor (1b) and controlling an optical sensor (10) that detects an echo of an irradiated light irradiated onto a detection area (Ad) of a vehicle (5), The processor acquiring distance image data (Dd) representing a distance value to a reflective target (Tr) that reflects light in a detection area based on an echo detected by an optical sensor in response to the irradiated light; Acquiring intensity image data (Db) representing the intensity value of the echo reflected from the reflective target in the detection area based on the echo detected by the optical sensor against background light having a lower intensity than the irradiated light in the detection area; Estimating a flare pixel region (Rf) in which a flare is expected to be captured around a target pixel region (Rt) in which a reflective target is captured in the range image data based on the intensity image data; and removing the distance values ​​of the flare pixel regions where the echo detection timing overlaps with the target pixel region in the range image data.

[0012] A fifth aspect of the present disclosure is A control method executed by a processor (1b) for controlling an optical sensor (10) that detects an echo of an irradiated light onto a detection area (Ad) of a vehicle (5), the method comprising: acquiring distance image data (Dd) representing a distance value to a reflective target (Tr) that reflects light in a detection area based on an echo detected by an optical sensor in response to the irradiated light; Acquiring intensity image data (Db) representing the intensity value of the echo reflected from the reflective target in the detection area based on the echo detected by the optical sensor against background light having a lower intensity than the irradiated light in the detection area; Estimating a flare pixel region (Rf) in which a flare is expected to be captured around a target pixel region (Rt) in which a reflective target is captured in the range image data based on the intensity image data; The method includes removing distance values ​​of flare pixel regions where the detection timing of the echo overlaps with the target pixel region in the range image data.

[0013] A sixth aspect of the present disclosure is A control program is stored in a storage medium (1a) for controlling an optical sensor (10) that detects an echo of an irradiated light onto a detection area (Ad) of a vehicle (5), the control program including instructions to be executed by a processor (1b), The command is, Acquiring distance image data (Dd) representing a distance value to a reflective target (Tr) that reflects light in a detection area based on an echo detected by an optical sensor in response to the irradiated light; Acquiring intensity image data (Db) representing the intensity value of an echo reflected from a reflective target in a detection area based on the echo detected by the optical sensor against background light in the detection area that has a lower intensity than the irradiated light; Estimating a flare pixel region (Rf) in which an image of a flare is predicted to be captured around a target pixel region (Rt) in which an image of a reflective target is captured in the range image data based on the intensity image data; The method includes removing the distance values ​​of the flare pixel regions where the detection timing of the echo overlaps with the target pixel region in the distance image data.

[0014] According to the fourth to sixth aspects, distance image data representing distance values ​​to reflective targets reflecting light in the detection area is acquired based on echoes detected by an optical sensor in response to irradiated light. Therefore, in the fourth to sixth aspects, intensity image data representing intensity values ​​of echoes reflected from reflective targets in the detection area is acquired based on echoes detected by an optical sensor in response to background light in the detection area that has a lower intensity than the irradiated light. This allows flare pixel regions in the distance image data, where flare is expected to be captured around a target pixel region capturing an image of the reflective target, to be properly estimated based on intensity image data in which the captured flare is suppressed in response to low-intensity background light. Therefore, distance values ​​in flare pixel regions in the distance image data where the echo detection timing overlaps with the target pixel region can be eliminated as pseudo values ​​resulting from the capture of flare, thereby preventing erroneous detection of distance values ​​by the optical sensor. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a schematic diagram showing the overall configuration of a detection system according to a first embodiment. [Figure 2] FIG. 2 is a schematic diagram showing a detailed configuration of the optical sensor according to the first embodiment. [Figure 3] FIG. 2 is a block diagram showing the functional configuration of a control device according to the first embodiment. [Figure 4] FIG. 1 is a schematic diagram showing a floodlight according to a first embodiment. [Figure 5] FIG. 2 is a schematic diagram showing characteristics of the laser diode according to the first embodiment. [Figure 6] 4 is a time chart showing detection frames according to the first embodiment. [Figure 7] FIG. 2 is a schematic diagram showing a light receiver according to the first embodiment. [Figure 8] FIG. 3 is a schematic diagram for explaining distance image data according to the first embodiment. [Figure 9] FIG. 3 is a schematic diagram for explaining intensity image data according to the first embodiment. [Figure 10] FIG. 2 is a schematic diagram for explaining a reflective target in the first embodiment. [Figure 11] 4 is a graph for explaining optical characteristics according to the first embodiment. [Figure 12] FIG. 4 is a schematic diagram for explaining a flare pixel region according to the first embodiment. [Figure 13] FIG. 4 is a schematic diagram for explaining a flare pixel region according to the first embodiment. [Figure 14] 10 is a graph for explaining removal of distance values ​​according to the first embodiment. [Figure 15] 10 is a graph for explaining removal of distance values ​​according to the first embodiment. [Figure 16] 4 is a flowchart showing a control flow according to the first embodiment. [Figure 17] FIG. 10 is a block diagram showing the functional configuration of a control device according to a second embodiment. [Figure 18] FIG. 10 is a schematic diagram for explaining removal of distance values ​​according to the second embodiment. [Figure 19] 10 is a flowchart showing a control flow according to a second embodiment. [Figure 20] FIG. 10 is a block diagram showing a functional configuration of a control device according to a third embodiment. [Figure 21] 10 is a time chart showing detection frames according to a third embodiment. [Figure 22] 10 is a flowchart showing a control flow according to a third embodiment. [Figure 23] FIG. 10 is a block diagram showing a functional configuration of a control device according to a fourth embodiment. [Figure 24] 10 is a time chart showing detection frames according to the fourth embodiment. [Figure 25] 10 is a flowchart showing a control flow according to the fourth embodiment. [Figure 26] 10 is a flowchart showing a control flow according to the fourth embodiment. [Figure 27] FIG. 10 is a block diagram showing a functional configuration of a control device according to a modified example. [Figure 28] 10 is a flowchart showing a control flow according to a modified example. [Figure 29] FIG. 10 is a block diagram showing a functional configuration of a control device according to a modified example. [Figure 30] 10 is a flowchart showing a control flow according to a modified example. [Figure 31] FIG. 10 is a block diagram showing a functional configuration of a control device according to a modified example. [Figure 32] 10 is a flowchart showing a control flow according to a modified example. [Figure 33] FIG. 10 is a block diagram showing a functional configuration of a control device according to a modified example. [Figure 34] 10 is a flowchart showing a control flow according to a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, multiple embodiments of the present disclosure will be described with reference to the drawings. Note that corresponding components in each embodiment are designated by the same reference numerals, and redundant description may be omitted. Furthermore, when only a portion of the configuration is described in each embodiment, the configuration of another previously described embodiment may be applied to the remaining portions of the configuration. Furthermore, in addition to the combinations of configurations explicitly stated in the description of each embodiment, configurations of multiple embodiments may be partially combined together even if not explicitly stated, provided that there is no particular problem with the combination.

[0017] (First embodiment) 1, a first embodiment of the present disclosure relates to a detection system 2 including an optical sensor 10 and a control device 1. The detection system 2 is mounted on a vehicle 5. The vehicle 5 is a mobile object, such as an automobile, that can travel on a road with an occupant on board.

[0018] In an autonomous driving control mode, the vehicle 5 is capable of steady or temporary autonomous driving. Here, the autonomous driving control mode may be realized by autonomous driving control, such as conditional driving automation, highly automated driving, or fully automated driving, in which the system performs all driving tasks when activated. The autonomous driving control mode may also be realized by advanced driving assistance control, such as driving assistance or partial driving automation, in which the occupant performs some or all driving tasks. The autonomous driving control mode may be realized by either autonomous driving control or advanced driving assistance control, or by a combination of these, or by switching between them.

[0019] In the following description, unless otherwise specified, the directions of front, rear, up, down, left, and right are defined with respect to the vehicle 5 on a horizontal plane. The horizontal direction refers to a direction parallel to and also lateral to the horizontal plane that serves as the directional reference for the vehicle 5. The vertical direction refers to a direction perpendicular to and also up and down to the horizontal plane that serves as the directional reference for the vehicle 5.

[0020] The optical sensor 10 is a so-called LiDAR (Light Detection and Ranging / Laser Imaging Detection and Ranging) sensor for acquiring image data that can be used for driving control of the vehicle 5, including an automatic driving mode. The optical sensor 10 is disposed in at least one location of the vehicle 5, such as the front, left and right side portions, rear portion, or upper roof. As shown in FIG. 2 , a three-dimensional Cartesian coordinate system is defined in the optical sensor 10 by three mutually orthogonal axes: an X-axis, a Y-axis, and a Z-axis. In this embodiment, the X-axis and the Z-axis are set along different horizontal directions of the vehicle 5, and the Y-axis is set along the vertical direction of the vehicle 5. Note that in FIG. 2 , the left side of the dashed-dotted line along the Y-axis (the side of the translucent cover 12 described below) actually illustrates a cross section perpendicular to the right side of the dashed-dotted line (the side of units 21 and 41 described below).

[0021] 3, the optical sensor 10 emits light toward a detection area Ad corresponding to the installation location in the external space of the vehicle 5. The optical sensor 10 detects the reflected light that enters the detection area Ad when the emitted light is reflected from the detection area Ad as an echo from the detection area Ad. The optical sensor 10 can also detect light that enters the detection area Ad when background light (i.e., external light) is reflected from the detection area Ad when the emitted light is not being emitted as an echo from the detection area Ad.

[0022] The optical sensor 10 detects these echoes to observe a reflecting target Tr that reflects light within the detection area Ad. Here, particularly in this embodiment, observation means sensing the distance value from the optical sensor 10 to the reflecting target Tr and the intensity value of the echo reflected from the reflecting target Tr. A typical observed target in the optical sensor 10 applied to the vehicle 5 may be at least one of moving objects such as a pedestrian, a cyclist, a non-human animal, and another vehicle. A typical observed target in the optical sensor 10 applied to the vehicle 5 may be at least one of stationary objects such as a guardrail, a road sign, a roadside structure, and an object fallen on the road.

[0023] As shown in FIG. 2, the optical sensor 10 includes a housing 11, a light-emitting unit 21, a scanning unit 31, and a light-receiving unit 41. The housing 11 forms the exterior of the optical sensor 10. The housing 11 is formed in a box shape and has light-blocking properties. The housing 11 houses the light-emitting unit 21, the scanning unit 31, and the light-receiving unit 41 inside. A light-transmitting cover 12 is provided on an open optical window in the housing 11. The light-transmitting cover 12 is formed in a plate shape and is transmissive to the above-mentioned irradiated light and echoes. The light-transmitting cover 12 closes the optical window of the housing 11 while allowing both the irradiated light and echoes to pass through.

[0024] The light-projecting unit 21 includes a light-projecting device 22 and a light-projecting lens system 26. The light-projecting device 22 is disposed within the housing 11. As shown in FIG. 4, the light-projecting device 22 is formed by arranging a plurality of laser diodes 24 in an array on a substrate. The laser diodes 24 are arranged in a single row along the Y axis. Each laser diode 24 has a resonator structure that can resonate light oscillated in a PN junction layer, and a mirror layer structure that can repeatedly reflect light across the PN junction layer.

[0025] Each laser diode 24 emits light in response to application of a current in accordance with a control signal from the control device 1. In particular, each laser diode 24 in this embodiment emits light in the near-infrared region that is difficult for humans in the external space of the vehicle 5 to see. As shown in FIG. 5 , when a current higher than the switching current value Cth is applied to each laser diode 24, the laser diode 24 enters an oscillating LD (Laser Diode) mode and emits pulsed light. The light emitted from each laser diode 24 in this LD mode constitutes irradiation light having a first intensity I1 in the near-infrared region, as shown in FIG. 6 . On the other hand, when a current lower than the switching current value Cth is applied to each laser diode 24, the laser diode 24 enters a non-oscillating LED (Light Emitting Diode) mode and emits DC (Direct Current) light. The light emitted from each laser diode 24 in this LED mode constitutes irradiation light having a second intensity I2 in the near-infrared region that is lower than the first intensity I1, as shown in FIG. 6 .

[0026] As shown in FIG. 4, the light projector 22 has a light projection window 25 formed on one side of the substrate. The light projection window 25 is configured as a collection of projection apertures of the laser diodes 24. Light emitted from the projection apertures of the laser diodes 24 is projected from the light projection window 25 as a longitudinal linear illumination light along the Y axis in the detection area Ad. The illumination light may include non-light-emitting portions corresponding to the arrangement intervals of the laser diodes 24 in the Y axis direction. Even in this case, it is preferable that a linear illumination light in which the non-light-emitting portions are macroscopically eliminated by diffraction is formed in the detection area Ad.

[0027] As shown in FIG. 2 , the projection lens system 26 projects the illumination light from the projector 22 toward the scanning mirror 32 of the scanning unit 31. The projection lens system 26 is disposed between the projector 22 and the scanning mirror 32 within the housing 11. The projection lens system 26 exerts at least one optical function, such as focusing, collimating, and shaping. The projection lens system 26 forms a projection optical axis along the Z axis. The projection lens system 26 has at least one projection lens 27 on the projection optical axis, the lens shape of which corresponds to the optical function to be exerted. The projector 22 is positioned on the projection optical axis of the projection lens system 26. The illumination light emitted from the center of the projection window 25 of the projector 22 is guided along the projection optical axis of the projection lens system 26.

[0028] The scanning unit 31 includes a scanning mirror 32 and a scanning motor 35. The scanning mirror 32 scans the illumination light emitted from the light-projecting lens system 26 of the light-projecting unit 21 toward the detection area Ad, and reflects echoes from the detection area Ad toward the light-receiving lens system 42 of the light-receiving unit 41. The scanning mirror 32 is disposed between the light-transmitting cover 12 and the light-projecting lens system 26 on the optical path of the illumination light, and between the light-transmitting cover 12 and the light-receiving lens system 42 on the optical path of the echo.

[0029] The scanning mirror 32 is formed into a plate shape by depositing a reflective film on a reflective surface 33, which is one surface of a base material. The scanning mirror 32 is supported by the housing 11 so as to be rotatable about a rotation center line along the Y axis. The normal direction of the reflective surface 33 of the scanning mirror 32 can be adjusted by rotating about the rotation center line. The scanning mirror 32 oscillates within a driving range that is limited by mechanical or electrical stoppers.

[0030] The scanning mirror 32 is provided in common to the light-projecting unit 21 and the light-receiving unit 41. That is, the scanning mirror 32 is provided in common to both the irradiated light and the echo. As a result, the scanning mirror 32 has a light-projecting reflecting surface portion used to reflect the irradiated light and a light-receiving reflecting surface portion used to reflect the echo, which are offset in the Y-axis direction on the reflecting surface 33.

[0031] The illumination light is reflected by the light-projecting / reflecting surface of the reflecting surface 33, which faces the normal direction according to the rotation of the scanning mirror 32, and passes through the light-transmitting cover 12 to scan the detection area Ad in time and space. Scanning of the detection area Ad by the illumination light is essentially limited to horizontal scanning. The illumination light and background light are reflected by reflective targets Tr present in the detection area Ad and enter the optical sensor 10 as echoes. These echoes pass through the light-transmitting cover 12 and are reflected by the light-receiving / reflecting surface of the reflecting surface 33, which faces the normal direction according to the rotation of the scanning mirror 32, and are guided to the light-receiving lens system 42 of the light-receiving unit 41. The speeds of the illumination light and the echo are sufficiently high compared to the rotational speed of the scanning mirror 32. As a result, the echo corresponding to the illumination light is guided to the light-receiving lens system 42 in a manner that travels in the opposite direction to the illumination light on the scanning mirror 32, which has approximately the same rotation angle as the illumination light.

[0032] The scanning motor 35 is disposed around the scanning mirror 32 inside the housing 11. The scanning motor 35 is, for example, a voice coil motor, a brushed DC motor, or a stepping motor. The scanning motor 35 drives the scanning mirror 32 to rotate (i.e., swing) within a limited driving range in accordance with a control signal from the control device 1.

[0033] The light-receiving unit 41 includes a light-receiving lens system 42 and a light-receiver 45. The light-receiving lens system 42 guides the echo reflected by the scanning mirror 32 toward the light-receiver 45. The light-receiving lens system 42 is disposed within the housing 11 between the scanning mirror 32 and the light-receiver 45. The light-receiving lens system 42 is positioned below the light-projecting lens system 26 in the Y-axis direction. The light-receiving lens system 42 exerts an optical function to focus the echo on the light-receiver 45. The light-receiving lens system 42 forms a light-receiving optical axis along the Z-axis. The light-receiving lens system 42 has at least one light-receiving lens 43 on the light-receiving optical axis, the lens shape of which corresponds to the optical function to be exerted. The echo from the detection area Ad, reflected from the light-receiving reflecting surface portion of the reflecting surface 33 of the scanning mirror 32, is guided along the light-receiving optical axis of the light-receiving lens system 42 within the driving range of the scanning mirror 32.

[0034] The photoreceiver 45 receives the echo from the detection area Ad that is imaged by the photoreceptor lens system 42, and outputs a detection signal corresponding to the received light. The photoreceiver 45 is disposed inside the housing 11 on the opposite side of the photoreceptor lens system 42 from the scanning mirror 32. The photoreceiver 45 is positioned below the projector 22 in the Y-axis direction and on the photoreceptor optical axis of the photoreceptor lens system 42.

[0035] As shown in FIG. 7, the photodetector 45 is formed by arranging photodetector elements 46 in a two-dimensional array in the X-axis and Y-axis directions on a substrate. Each photodetector element 46 is composed of a plurality of photodetectors. That is, since each photodetector element 46 corresponds to a plurality of photodetectors, the output value varies depending on the number of responses of the photodetectors. The photodetector elements of each photodetector element 46 are mainly constructed using photodiodes such as single photon avalanche diodes (SPADs). The photodetector elements of each photodetector element 46 may be integrally constructed by stacking a microlens array in front of the photodiode array.

[0036] The light receiver 45 has a rectangular light receiving surface 47 formed on one side of the substrate. The light receiving surface 47 is configured as a collection of incident surfaces of each light receiving element 46. The geometric center of the light receiving surface 47 with respect to the rectangular contour is aligned with the light receiving optical axis of the light receiving lens system 42 or is slightly offset from the light receiving optical axis of the light receiving lens system 42. Each light receiving element 46 receives echoes incident on the light receiving surface 47 from the light receiving lens system 42 using its respective light receiving element. Here, the long side of the rectangular light receiving surface 47 is defined along the Y axis. As a result, in response to the linear illumination light in the detection area Ad, the echoes corresponding to the illumination light are received by the light receiving elements of each light receiving element 46 as a linear beam.

[0037] As shown in FIG. 2, the photoreceiver 45 is integrally provided with a decoder 48. The decoder 48 sequentially reads, by sampling, electrical pulses generated by each photoreceptor element 46 in response to reception of an echo on the photoreceptor surface 47. The decoder 48 outputs the sequentially read electrical pulses to the control device 1 as a detection signal in a detection frame (i.e., a detection cycle) Fd shown in FIG. 6. At this time, the detection frame Fd is repeated at predetermined time intervals while the vehicle 5 is running. The control device 1 receives the detection signal from the decoder 48 and acquires image data Dd and Di representing the target observation results within the detection area Ad based on physical quantities related to the echoes detected by each photoreceptor element 46 as the scanning mirror 32 rotates, as shown in FIGS. 8 and 9. In the acquired image data Dd and Di, the vertical direction corresponds to the Y-axis direction of the vehicle 5, and the horizontal direction corresponds to the X-axis direction of the vehicle 5.

[0038] The control device 1 shown in FIG. 1 is connected to the optical sensor 10 via at least one of, for example, a local area network (LAN), a wire harness, and an internal bus. The control device 1 includes at least one dedicated computer. The dedicated computer constituting the control device 1 may be a sensor ECU (Electronic Control Unit) specialized for controlling the optical sensor 10, and in this case, the sensor ECU may be housed in a housing 11. The dedicated computer constituting the control device 1 may be a driving control ECU (Electronic Control Unit) that controls driving of the vehicle 5. The dedicated computer constituting the control device 1 may be a navigation ECU that navigates the driving route of the vehicle 5. The dedicated computer constituting the control device 1 may be a locator ECU that estimates the self-state quantity of the vehicle 5.

[0039] The dedicated computer constituting the control device 1 has at least one memory 1a and one processor 1b. The memory 1a is at least one type of non-transitory tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium, that non-temporarily stores computer-readable programs and data. The processor 1b includes at least one type of core, such as a central processing unit (CPU), a graphics processing unit (GPU), a reduced instruction set computer (RISC)-CPU, a data flow processor (DFP), or a graph streaming processor (GSP).

[0040] The processor 1b executes a plurality of instructions included in the control program stored in the memory 1a. This causes the control device 1 to construct a plurality of functional blocks for controlling the optical sensor 10. In this manner, the control device 1 constructs a plurality of functional blocks by having the processor 1b execute a plurality of instructions from the control program stored in the memory 1a to control the optical sensor 10. The functional blocks constructed by the control device 1 include a distance acquisition block 100, an intensity acquisition block 110, an estimation block 120, and a removal block 130, as shown in FIG. 3 .

[0041] The distance acquisition block 100 controls the optical sensor 10 to an LD mode in which each laser diode 24 is in an oscillating state during a distance acquisition period Pd set in a detection frame Fd shown in FIG. 6. The LD mode control causes the optical sensor 10 to irradiate the detection area Ad with intermittent pulses of irradiation light at a first intensity I1. Based on the echo detected by the optical sensor 10 in response to the irradiation light at the first intensity I1, the distance acquisition block 100 acquires three-dimensional point cloud distance image data Dd shown in FIG. 8 to represent the distance to the reflecting target Tr in the detection area Ad. The distance values ​​of each pixel constituting the distance image data Dd are acquired using dTOF (direct time of flight), which is based on the time of flight of light from the pulse irradiation to the detection of the echo.

[0042] In conjunction with the acquisition of distance image data Dd, distance acquisition block 100 also controls the rotational drive of scanning mirror 32 by scanning motor 35 in synchronization with the pulsed irradiation of illumination light. Distance acquisition block 100 generates distance image data Dd for each of multiple scanning lines Ls according to the rotation angle of scanning mirror 32, thereby enabling the distance image data Dd for each scanning line Ls to be synthesized over distance acquisition period Pd. Here, the scanning lines Ls related to distance image data Dd are set as vertical pixel rows corresponding to the Y-axis direction, and as multiple horizontal rows corresponding to the X-axis direction.

[0043] The intensity acquisition block 110 shown in Fig. 3 controls the optical sensor 10 to an LED mode in which each laser diode 24 is in a non-oscillating state during an intensity acquisition period Pi that is set before the distance acquisition period Pd in ​​the detection frame Fd shown in Fig. 6. By controlling the optical sensor 10 to the LED mode, the optical sensor 10 continuously irradiates the detection area Ad with irradiation light of a second intensity I2 that is lower than the first intensity I1. Therefore, the intensity acquisition block 110 acquires two-dimensional intensity image data Di shown in Fig. 9 that represents the intensity values ​​reflected from the reflecting target Tr in the detection area Ad based on the echo detected by the optical sensor 10 in response to the irradiation light of the second intensity I2.

[0044] In conjunction with the acquisition of the intensity image data Di, the intensity acquisition block 110 also controls the rotational drive of the scanning mirror 32 by the scanning motor 35 in parallel with the continuous irradiation of the irradiation light. The intensity acquisition block 110 generates intensity image data Di for each of multiple scanning lines Ls corresponding to the rotation angle of the scanning mirror 32, thereby enabling the intensity image data Di for each scanning line Ls to be synthesized over the intensity acquisition period Pi. As shown in FIG. 9, the scanning lines Ls of the intensity image data Di are set to have a similar 1:1 correspondence with the scanning lines Ls of the distance image data Dd shown in FIG. 8. Note that in FIGS. 8 and 9, only the first, middle, and last scanning lines Ls in the image data Dd and Di are shown in bold frames, and the other scanning lines Ls are omitted.

[0045] The estimation block 120 shown in Fig. 3 searches for a target pixel region Rt in which an image of a reflecting target Tr is captured, as shown in Fig. 9, from intensity image data Di obtained by combining scan lines Ls worth of data over an intensity acquisition period Pi. Here, the target pixel region Rt is defined as a pixel region representing intensity values ​​equal to or greater than a predicted threshold value, in which flare is predicted to occur due to strong reflection from a reflecting target Tr, such as a sign, for illumination light of a first intensity I1 corresponding to distance image data Dd, as shown in Fig. 10. The first embodiment to which this definition is applied is premised on the premise that capturing of flare is suppressed as much as possible for illumination light of a second intensity I2 corresponding to the intensity image data Di.

[0046] 8 and 9, the estimation block 120 estimates a flare pixel region Rf in which a flare is predicted to be captured around a target pixel region Rt in which a reflecting target Tr is captured in the distance image data Dd, based on the intensity value of the target pixel region Rt in the intensity image data Di. In Figures 8 and 9, the reflecting target Tr in which a flare is predicted to occur is virtually shown by a two-dot chain line, thereby schematically associating the reflecting target Tr with each of the regions Rt and Rf.

[0047] Specifically, the estimation block 120 estimates a flare pixel region Rf correlated with the optical characteristic Os of the light receiving lens system 42 in the optical sensor 10 and the intensity value of the target pixel region Rt in the intensity image data Di. Here, the optical characteristic Os provides a range Ef within which the probability of flare occurrence around the reflecting target Tr is equal to or exceeds a set value, as shown in FIG. 11 . The optical characteristic Os is stored in the memory 1a as, for example, a function or table, so as to provide a range Ef corresponding to the incident intensity Ii of the echo onto the light receiving lens system 42, as shown by cross-hatching in FIGS. 12 and 13 . Here, the incident intensity Ii of the echo onto the light receiving lens system 42 corresponding to the first intensity I1 can be estimated from, for example, a representative value or average value of the intensity values ​​of the target pixel region Rt corresponding to the second intensity I2.

[0048] From these facts, estimation block 120 extracts, as flare pixel region Rf, a pixel region in distance image data Dd that is estimated to correspond to range Ef that correlates with optical characteristic Os of light receiving lens system 42 and intensity values ​​of target pixel region Rt in intensity image data Di. This estimation of flare pixel region Rf can be thought of as an estimation for distance image data Dd acquired for each scan line Ls, or as an estimation for distance image data Dd obtained by combining data for each scan line Ls over distance acquisition period Pd.

[0049] In the first embodiment, the removal block 130 shown in FIG. 3 extracts distance image data Dd for the estimated scan line Ls of the flare pixel region Rf, as shown in FIG. 14, from the distance image data Dd for each scan line Ls. The removal block 130 removes, as pseudo values, distance values ​​for the flare pixel region Rf where the echo detection timing overlaps with the target pixel region Rt, as shown by the bold ellipse in FIG. 15. "Removal" here means deleting the point cloud representing the distance value for the corresponding flare pixel region Rf from the distance image data Dd. "Overlap" here means that the echo peak points are detected within a predetermined error range, resulting in overlapping of the echo intensity waveforms above the baseline. In the dTOF applied to the optical sensor 10, the echo detection timing and the distance value correspond one-to-one. Therefore, removing the distance value according to the detection timing is essentially equivalent to removing the distance value of the flare pixel region Rf when the difference between the distance value of the flare pixel region Rf and the distance value of the target pixel region Rt falls within a predetermined error range.

[0050] The removal block 130 synthesizes, over the distance acquisition period Pd, the distance image data Dd from which the distance values ​​of the flare pixel region Rf have been removed for each scan line Ls. The removal block 130 may then store the distance image data Dd from which the distance values ​​of the flare pixel region Rf have been removed in memory 1a, in association with at least one of, for example, a timestamp and information about the driving environment of the vehicle 5. The removal block 130 may then transmit the distance image data Dd from which the distance values ​​of the flare pixel region Rf have been removed to an external center, in association with, for example, a timestamp and information about the driving environment of the vehicle 5, so that the data is stored in a storage medium of the external center.

[0051] The control method in which the control device 1 controls the optical sensor 10 of the vehicle 5 through cooperation of the blocks 100, 110, 120, and 130 described so far is executed according to the control flow shown in Fig. 16. This control flow is repeatedly executed for each detection frame Fd while the vehicle 5 is running. Note that each "S" in the control flow represents a plurality of steps executed by a plurality of commands included in the control program.

[0052] In S101, the intensity acquisition block 110 acquires intensity image data Di representing the intensity value of the echo from the reflecting target Tr for the illumination light of the second intensity I2 during the intensity acquisition period Pi of the current detection frame Fd. At this time, the intensity image data Di for each scanning line Ls is synthesized over the intensity acquisition period Pi.

[0053] In the next step S102, the estimation block 120 determines whether a target pixel region Rt that represents an intensity value equal to or greater than the prediction threshold value exists in the intensity image data Di. If the determination is affirmative, the control flow proceeds to step S103.

[0054] In S103, the estimation block 120 estimates the flare pixel region Rf in the distance image data Dd based on the intensity values ​​of the target pixel region Rt in the intensity image data Di. At this time, the flare pixel region Rf is estimated to be in a range Ef that correlates with the optical characteristic Os of the light receiving lens system 42 and the intensity values ​​of the target pixel region Rt in the intensity image data Di.

[0055] In the next S104, the distance acquisition block 100 acquires distance image data Dd representing the distance to the reflection target Tr for each scan line Ls using the illumination light of the first intensity I1 during the distance acquisition period Pd of the current detection frame Fd. Then, in S105, to which the control flow advances each time distance image data Dd for each scan line Ls is acquired, the removal block 130 determines whether the scan line Ls for which distance image data Dd was acquired is the scan line Ls for which the flare pixel region Rf was estimated in S103. If the determination is affirmative, the control flow advances to S106.

[0056] In S106, the removal block 130 determines whether or not the distance image data Dd of the estimated scan line Ls of the flare pixel region Rf contains distance values ​​of the flare pixel region Rf whose echo detection timing overlaps with the target pixel region Rt. If the result is a positive determination, the control flow proceeds to S107. In S107, the removal block 130 removes, as pseudo values, the distance values ​​of the flare pixel region Rf whose echo detection timing overlaps with the target pixel region Rt from the distance image data Dd of the estimated scan line Ls of the flare pixel region Rf.

[0057] If S107 is completed or if a negative determination is made in each of S105 and S106, the control flow proceeds to S108. In S108, the distance acquisition block 100 determines whether the distance acquisition period Pd has ended. If a negative determination is made, the control flow returns to S104 by the distance acquisition block 100, whereby distance image data Dd is acquired for the next incomplete scan line Ls. On the other hand, if a positive determination is made, the control flow proceeds to S109 by the removal block 130, whereby distance image data Dd for each scan line Ls, including the distance image data Dd for the scan line Ls from which the pseudo values ​​have been removed, is synthesized over the distance acquisition period Pd. When S109 is completed, the current execution of the control flow ends.

[0058] If a negative determination is made in S102, the control flow proceeds to S110. In S110, the distance acquisition block 100 acquires distance image data Dd during the intensity acquisition period Pi in the detection frame Fd for each scanning line Ls, as in S104, and then combines the data as in S109. When S110 is completed, the current execution of the control flow ends.

[0059] (Action and effect) The effects of the first embodiment described above will be explained below.

[0060] According to the first embodiment, distance image data Dd representing the distance to a reflecting target Tr reflecting light in the detection area Ad is acquired based on echoes detected by the optical sensor 10 in response to illumination light having a first intensity I1. Therefore, in the first embodiment, intensity image data Di representing the intensity of the echo reflected from the reflecting target Tr in the detection area Ad is acquired based on echoes detected by the optical sensor 10 in response to illumination light having a second intensity I2 lower than the first intensity I1. This allows flare pixel regions Rf in the distance image data Dd, where flare is predicted to be captured around the target pixel region Rt where the reflecting target Tr is captured, to be accurately estimated based on intensity image data Di in which the captured flare is suppressed in response to low-intensity illumination light. Therefore, distance values ​​in the flare pixel regions Rf in the distance image data Dd where the echo detection timing overlaps with the target pixel region Rt can be eliminated as pseudo values ​​resulting from the captured flare, thereby preventing erroneous detection of distance values ​​by the optical sensor 10.

[0061] According to the first embodiment, distance image data Dd is acquired for light irradiated at a first intensity I1 by laser diode 24 controlled to an oscillating state in optical sensor 10. Therefore, in the first embodiment, intensity image data Di is acquired for light irradiated at a second intensity I2 by laser diode 24 controlled to a non-oscillating state in optical sensor 10. This makes it possible to acquire not only distance image data Dd, for which erroneous detection is to be suppressed, but also intensity image data Di for estimating flare pixel region Rf, in accordance with changes in the intensity of light irradiated by the common laser diode 24. Therefore, it is possible to suppress erroneous detection of distance values ​​using optical sensor 10, which is relatively small.

[0062] According to the first embodiment, the flare pixel region Rf is estimated within a range Ef that correlates with the optical characteristic Os of the light receiving lens system 42 that images the echo in the optical sensor 10 and the intensity value of the target pixel region Rt in the intensity image data Di. This makes it possible to properly estimate the flare pixel region Rf, where an image of a flare is predicted to occur, within the range Ef according to the optical characteristic Os, around the target pixel region Rt that can be identified from the intensity values ​​in the intensity image data Di. Therefore, it is possible to accurately remove the distance values ​​of the flare pixel region Rf whose echo detection timing overlaps with the target pixel region Rt in the distance image data Dd, thereby suppressing erroneous detection of distance values ​​by the optical sensor 10.

[0063] According to the first embodiment, distance image data Dd is acquired for each of multiple scan lines Ls during the distance acquisition period Pd, while intensity image data Di is acquired for each of those scan lines Ls during the intensity acquisition period Pi that precedes the distance acquisition period Pd. This makes it possible to remove spurious distance values ​​from the distance image data Dd that is limited to the scan line Ls for which the flare pixel region Rf is estimated, based on the intensity image data Di synthesized over the intensity acquisition period Pi. This makes it possible to prevent erroneous detection of distance values ​​by the optical sensor 10.

[0064] Second Embodiment The second embodiment is a modification of the first embodiment.

[0065] In the second embodiment shown in Fig. 17, the removal block 2130 removes the distance values ​​of the flare pixel region Rf estimated by the estimation block 120 as shown in Fig. 18 in the distance image data Dd obtained by combining the scan lines Ls worth of the distance acquisition block 2100 over the distance acquisition period Pd. Therefore, the removal block 2130 compares the distance values ​​of the flare pixel region Rf with the distance values ​​of the target pixel region Rt in the combined distance image data Dd. In this case, the distance values ​​of the flare pixel region Rf are compared with, for example, a representative value, an average value, or a value for each pixel. Note that in Fig. 18, only the scan lines Ls in the intensity image data Di that include the removal target are indicated by a bold frame, and the other scan lines Ls are omitted from the illustration.

[0066] If the comparison result shows that the difference between the distance value of the flare pixel region Rf and the distance value of the target pixel region Rt falls within a predetermined error range, the removal block 2130 removes the distance value of the flare pixel region Rf. Here, as in the first embodiment, in the dTOF applied to the optical sensor 10, the distance value and the detection timing of the echo correspond one-to-one. Therefore, removing the distance value by comparing the regions Rf and Rt is essentially synonymous with removing the distance value of the flare pixel region Rf whose detection timing of the echo overlaps with that of the target pixel region Rt.

[0067] 19, the control flow of the second embodiment executes steps S2104, S2106, and S2107 instead of steps S104 to S109 of the first embodiment. In step S2104, the distance acquisition block 2100 acquires distance image data Dd in the detection frame Fd during the distance acquisition period Pd for each scanning line Ls using irradiation light of the first intensity I1, and then combines the data over the distance acquisition period Pd.

[0068] In the following step S2106, the removal block 2130 determines whether the difference between the estimated distance value of the flare pixel region Rf and the distance value of the target pixel region Rt in the combined distance image data Dd falls within the error range. If a positive determination is made as a result, i.e., if there is a distance value in the flare pixel region Rf where the detection timing of the echo overlaps with the target pixel region Rt, the control flow proceeds to step S2107. In step S2107, the removal block 2130 removes, from the combined distance image data Dd, the distance value that is a pseudo value in the flare pixel region Rf, i.e., the distance value in the flare pixel region Rf where the detection timing of the echo overlaps with the target pixel region Rt. If the execution of step S2107 is completed or if a negative determination is made in step S2106, the current execution of the control flow ends.

[0069] As described above, according to the second embodiment, distance image data Dd is acquired for each of multiple scan lines Ls during the distance acquisition period Pd, while intensity image data Di is acquired for each of those scan lines Ls during the intensity acquisition period Pi that precedes the distance acquisition period Pd. This allows pseudo distance values ​​to be removed all at once from the flare pixel region Rf estimated based on the intensity image data Di obtained by combining the scan lines Ls over the intensity acquisition period Pi in the distance image data Dd obtained by combining the scan lines Ls over the distance acquisition period Pd. This makes it possible to prevent erroneous detection of distance values ​​by optical sensor 10.

[0070] (Third embodiment) The third embodiment is a modification of the first embodiment.

[0071] The intensity acquisition block 3110 of the third embodiment shown in Fig. 20 sets a background light acquisition period Pb before or after (Fig. 21 shows an example of the before) the intensity acquisition period Pi for acquiring intensity image data Di in the detection frame Fd as shown in Fig. 21. Therefore, during the background light acquisition period Pb, the intensity acquisition block 3110 controls the optical sensor 10 to enter a stop mode in which application of current is stopped and each laser diode 24 is put into a non-light-emitting state.

[0072] 20, the intensity acquisition block 3110 acquires two-dimensional background light image data Db representing the intensity values ​​reflected from the reflective target Tr in the detection area Ad based on the echo detected by the optical sensor 10 against the background light when not illuminated according to the stop mode. At this time, the intensity of the background light is lower in the near-infrared range than the first intensity I1 of the illumination light. In other words, the first intensity I1 described in the first embodiment is set to be higher than the intensity of the background light in the near-infrared range.

[0073] In conjunction with the acquisition of the background light image data Db, the intensity acquisition block 3110 also controls the rotational driving of the scanning mirror 32 by the scanning motor 35. Therefore, the intensity acquisition block 3110 generates background light image data Db for each of a plurality of scanning lines Ls according to the rotation angle of the scanning mirror 32, thereby making it possible to synthesize the background light image data Db for each scanning line Ls over the background light acquisition period Pb.

[0074] In each of the scan lines Ls thus acquired and in the combined background light image data Db, the vertical direction corresponds to the Y-axis direction of the vehicle 5, and the horizontal direction corresponds to the X-axis direction of the vehicle 5. Therefore, the scan lines Ls of the background light image data Db are set to have the same 1:1 correspondence with the scan lines Ls for the distance image data Dd and the intensity image data Di.

[0075] The estimation block 3120 of the third embodiment extracts intensity values ​​in the target pixel region Rt searched for in the intensity image data Di from the intensity values ​​represented by the background light image data Db. The estimation block 3120 then corrects the intensity values ​​in the target pixel region Rt of the intensity image data Di by subtracting the intensity values ​​in the target pixel region Rt of the background light image data Db, and then uses the corrected intensity values ​​in the target pixel region Rt to estimate the flare pixel region Rf.

[0076] As shown in Fig. 22, the control flow of the third embodiment executes S3100 before or after S101 by the intensity acquisition block 3110 (Fig. 22 shows the former example). In S3100, the intensity acquisition block 3110 acquires background light image data Db representing the intensity value of the echo from the reflecting target Tr for the background light when no irradiation light is applied, during the background light acquisition period Pb of the current detection frame Fd. At this time, the background light image data Db for each scanning line Ls is synthesized over the background light acquisition period Pb.

[0077] In the control flow of the third embodiment, S3103 is executed instead of S103. In S3103, the estimation block 3120 estimates the flare pixel region Rf in the distance image data Dd based on the intensity values ​​of the target pixel region Rt in the intensity image data Di corrected by the intensity values ​​of the target pixel region Rt in the background light image data Db.

[0078] As described above, according to the third embodiment, background light image data Db representing the intensity values ​​of echoes reflected from reflecting targets Tr in the detection area Ad is acquired based on the echoes detected by the optical sensor 10 against the background light in the detection area Ad. Therefore, in the third embodiment, the flare pixel region Rf can be estimated with high accuracy based on the intensity values ​​of the target pixel region Rt in the intensity image data Di corrected by the intensity values ​​of the target pixel region Rt in the background light image data Db. This makes it possible to accurately remove the distance values ​​of the flare pixel region Rf in the distance image data Dd where the echo detection timing overlaps with the target pixel region Rt, thereby preventing erroneous detection of distance values ​​by the optical sensor 10.

[0079] (Fourth embodiment) The fourth embodiment is a modification of the third embodiment.

[0080] The intensity acquisition block 4110 of the fourth embodiment shown in Fig. 23 switches between the intensity acquisition period Pi and the background light acquisition period Pb, which period is set for each detection frame Fd, depending on the brightness of the detection area Ad, as shown in Fig. 24. Specifically, in a dark environment such as nighttime, where the average or representative value of the background light intensity is less than or equal to the switching threshold, the intensity acquisition block 4110 selects and executes the intensity acquisition period Pi so as to control the optical sensor 10 to the LED mode and acquire the intensity image data Di.

[0081] On the other hand, in a bright environment such as daytime when the average or representative value of the background light intensity is equal to or exceeds the switching threshold, the intensity acquisition block 4110 controls the optical sensor 10 to stop mode and selects and executes the background light acquisition period Pb to acquire background light image data Db. In this case, the background light acquisition period Pb is set before the distance acquisition period Pd in ​​the detection frame Fd. For these reasons, the background light acquisition period Pb and the background light image data Db can also be considered to be the intensity acquisition period and intensity image data for the background light.

[0082] Here, the background light intensity, which is the basis for switching between the intensity acquisition period Pi and the background light acquisition period Pb, is recognized based on the intensity image data Di and distance image data Dd acquired in the previous detection frame Fd, or based on the background light image data Db acquired in the previous detection frame Fd. At this time, the image data of the previous detection frame Fd is stored in a data storage unit 1ad of the memory 1a shown in FIG. 23, and is read out in conjunction with recognition of the background light intensity. Note that in addition to or instead of this recognition, the background light intensity may be recognized based on sensor information of the vehicle 5.

[0083] The estimation block 4120 of the fourth embodiment performs the same processing as the estimation block 120 of the first embodiment in a detection frame Fd in which an intensity acquisition period Pi is set. On the other hand, in a detection frame Fd in which a background light acquisition period Pb is set, the estimation block 4120 searches for a target pixel region Rt in which an image of a reflected target Tr is captured from background light image data Db in which each scan line Ls is synthesized over the background light acquisition period Pb. Here, the fourth embodiment is premised on the premise that capturing of flare is suppressed as much as possible not only for illumination light of the second intensity I2 corresponding to the intensity image data Di, but also for background light corresponding to the background light image data Db.

[0084] Therefore, the estimation block 4120 estimates the flare pixel region Rf predicted to surround the target pixel region Rt in the distance image data Dd based on the intensity value of the target pixel region Rt in the background light image data Db, in accordance with the estimation block 120 of the first embodiment. At this time, the incident intensity Ii of the echo to the light receiving lens system 42 corresponding to the first intensity I1 can be estimated from, for example, a representative value or average value of the intensity values ​​of the target pixel region Rt corresponding to the intensity of the background light.

[0085] In the control flow of the fourth embodiment, as shown in Fig. 25, S4100 is executed before S101 by the intensity acquisition block 4110. In S4100, the intensity acquisition block 4110 switches the period to be set for the current detection frame Fd to either the intensity acquisition period Pi or the background light acquisition period Pb, depending on the background light intensity in the detection area Ad. As a result, if the intensity acquisition period Pi is selected as a result of the switching, the control flow proceeds to S101, whereby S101 by the intensity acquisition block 4110 and S102 and S103 by the estimation block 4120 are executed. On the other hand, if the background light acquisition period Pb is selected as a result of the switching, the control flow proceeds to S4101 shown in Fig. 26.

[0086] In S4101, the intensity acquisition block 4110 acquires background light image data Db representing the intensity value of the echo from the reflecting target Tr for the background light during the background light acquisition period Pb of the current detection frame Fd. At this time, the background light image data Db for each scan line Ls is synthesized over the background light acquisition period Pb.

[0087] In the next step S4102, the estimation block 4120 determines whether a target pixel region Rt exhibiting an intensity value equal to or greater than the prediction threshold value is present in the background light image data Db. If a negative determination is made, the control flow proceeds to step S110 shown in Fig. 25. On the other hand, if a positive determination is made as shown in Fig. 26, the control flow proceeds to step S4103.

[0088] In S4103, the estimation block 4120 estimates the flare pixel region Rf in the distance image data Dd based on the intensity values ​​in the target pixel region Rt in the background light image data Db. At this time, the flare pixel region Rf is estimated to be in a range Ef that correlates with the optical characteristic Os of the light receiving lens system 42 and the intensity values ​​in the target pixel region Rt in the background light image data Db. When execution of S4103 is completed, the control flow proceeds to S104 shown in FIG. 25, as in the case when S103 is completed.

[0089] According to the fourth embodiment, the distance image data Dd representing the distance to the reflecting target Tr reflecting light in the detection area Ad is also acquired based on the echo detected by the optical sensor 10 in response to the irradiated light. Therefore, in the fourth embodiment, the background light image data Db is acquired as intensity image data representing the intensity of the echo reflected from the reflecting target Tr in the detection area Ad based on the echo detected by the optical sensor 10 in response to background light in the detection area Ad that has a lower intensity than the irradiated light. This allows the flare pixel region Rf in the distance image data Dd, where flare is predicted to be captured around the target pixel region Rt where the reflecting target Tr is captured, to be accurately estimated based on the background light image data Db, in which the capturing of the flare is suppressed in response to the low-intensity background light. Therefore, the distance value in the flare pixel region Rf where the echo detection timing overlaps with the target pixel region Rt in the distance image data Dd can be eliminated as a pseudo value resulting from the capturing of the flare, thereby preventing erroneous detection of the distance value by the optical sensor 10.

[0090] According to the fourth embodiment, the flare pixel region Rf is estimated within a range Ef that correlates with the optical characteristic Os of the light receiving lens system 42 that forms an image of the echo in the optical sensor 10 and the intensity value of the target pixel region Rt in the background light image data Db as intensity image data. This allows the flare pixel region Rf, where an image of a flare is predicted to occur, to be appropriately estimated within the range Ef according to the optical characteristic Os around the target pixel region Rt that can be identified from the intensity value of the background light image data Db. Therefore, it is possible to accurately remove the distance value of the flare pixel region Rf where the echo detection timing overlaps with the target pixel region Rt in the distance image data Dd, thereby suppressing erroneous detection of the distance value by the optical sensor 10.

[0091] According to the fourth embodiment, distance image data Dd is acquired for each of multiple scan lines Ls during the distance acquisition period Pd, while background light image data Db is acquired for each of those scan lines Ls during the background light acquisition period Pb, which serves as an intensity acquisition period prior to the distance acquisition period Pd. This allows for the background light image data Db, which is synthesized over the background light acquisition period Pb for each scan line Ls, to be based on background light image data Db, which is synthesized over the background light acquisition period Pb, thereby eliminating pseudo distance values ​​from the distance image data Dd limited to the scan line Ls for which the flare pixel region Rf is estimated. This makes it possible to prevent erroneous detection of distance values ​​by the optical sensor 10.

[0092] (Other embodiments) Although multiple embodiments have been described above, the present disclosure should not be construed as being limited to those embodiments, and can be applied to various embodiments and combinations within the scope that does not deviate from the gist of the present disclosure.

[0093] In a modified example, the dedicated computer constituting the control device 1 may be a computer other than the vehicle 5 that constitutes an external center or a mobile terminal capable of communicating with the vehicle 5. In a modified example, the dedicated computer constituting the control device 1 may have at least one of a digital circuit and an analog circuit as a processor. Here, the digital circuit is, for example, at least one of an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), an SOC (System on a Chip), a PGA (Programmable Gate Array), and a CPLD (Complex Programmable Logic Device). Furthermore, such a digital circuit may have a memory that stores a program.

[0094] In the optical sensor 10 of the modified example, scanning of the detection area Ad with the irradiated light may be substantially limited to scanning in the vertical direction. In this case, the long sides of the rectangular contours of the light-projecting window 25 and the light-receiving surface 47 may be defined along the X-axis. Furthermore, in this case, the scanning lines Ls for each image data Dd, Di may be set as a horizontal pixel row corresponding to the X-axis direction and multiple vertical rows corresponding to the Y-axis direction. In the optical sensor 10 of the modified example, the light-projector 22 that irradiates the irradiated light of the first intensity I1 and the light-projector 22 that irradiates the irradiated light of the second intensity I2 may be provided separately. In this case, the light-projector 22 that irradiates the irradiated light of the second intensity I2 may be a light-emitting diode (LED) instead of the laser diode 24.

[0095] 27 and 28, in a modification, blocks 2100, 2130, S2104, S2106, and S2107 of the second embodiment may be implemented in place of blocks 100, 130, and S104 to S109 in the third embodiment. As shown in Figures 29 and 30, in a modification, blocks 2100, 2130, S2104, S2106, and S2107 of the second embodiment may be implemented in place of blocks 100, 130, and S104 to S109 in the fourth embodiment.

[0096] 31 and 32 , in a modified example, the estimation block 3120 and S3103 of the third embodiment may be implemented in the second embodiment instead of the block 120 and S103. In this case, however, in S3103 by the estimation block 3120, the flare pixel region Rf of the distance image data Dd may be estimated based on the intensity values ​​of the intensity image data Di, from which the background light intensity has been subtracted using an algorithm for comparing the intensity values ​​of the distance image data Dd with the target pixel region Rt. To this end, S3103 by the estimation block 3120 may be executed following S2104 by the distance acquisition block 2100, thereby acquiring the distance image data Dd so as to include the intensity values ​​of the echo from the reflecting target Tr in response to the illumination light of the first intensity I1.

[0097] 33 and 34, in the modified example, S101 to S103 by blocks 4110 and 4120 do not have to be executed. However, in this case, when the intensity acquisition period Pi is selected by switching, the current execution of the control flow ends. In S103, S3013, and S4103 by estimation blocks 120, 3120, and 4120 of the modified example, the flare pixel region Rf may be estimated to be a fixed range Ef around the target pixel region Rt. In this case, it is possible to remove pseudo values ​​from the flare pixel region Rf in the fixed range Ef where the probability of flare occurrence is high.

[0098] In a modified example, the vehicle to which the control device 1 is applied may be, for example, a drone whose running on a road can be remotely controlled from an external center. In addition to the forms described so far, the above-mentioned embodiment and modified example may be implemented as a semiconductor device (for example, a semiconductor chip) having at least one processor 1b and one memory 1a. [Explanation of symbols]

[0099] 1: control device, 1a: memory, 1b: processor, 5: vehicle, 10: optical sensor, 24: laser diode, 42: light receiving lens system, Ad: detection area, Dd: distance image data, Di: intensity image data, Db: background light image data, I1: first intensity, I2: second intensity, Ls: scanning line, Os: optical characteristics, Rf: flare pixel area, Rt: target pixel area, Pd: distance acquisition period, Pi: intensity acquisition period, Pb: background light acquisition period, Tr: reflective target

Claims

1. A control device (1) having a processor (1b) and controlling an optical sensor (10) that detects an echo of an irradiated light to a detection area (Ad) of a vehicle (5), The processor, acquiring distance image data (Dd) representing a distance value to a reflective target (Tr) reflecting light in the detection area based on the echo detected by the optical sensor in response to the irradiation light of a first intensity (I1); acquiring intensity image data (Di) representing an intensity value of the echo reflected from the reflecting target in the detection area based on the echo detected by the optical sensor in response to the irradiation light having a second intensity (I2) lower than the first intensity; estimating a flare pixel region (Rf) in which a flare is predicted to be captured around a target pixel region (Rt) in which the reflecting target is captured in the range image data, based on the intensity image data; and removing the distance value of the flare pixel region where the detection timing of the echo overlaps with the target pixel region in the range image data.

2. The acquiring of the range image data includes: acquiring the distance image data for the irradiation light of the first intensity by a laser diode (24) controlled to an oscillating state in the optical sensor; acquiring the intensity image data The control device according to claim 1 , further comprising acquiring the intensity image data for the irradiation light of the second intensity by the laser diode controlled to a non-oscillating state in the optical sensor.

3. The processor, The method is further configured to obtain background light image data (Db) representing intensity values ​​of the echoes reflected from the reflecting targets in the detection area based on the echoes detected by the optical sensor against background light in the detection area; estimating the flare pixel region includes: The control device according to claim 1 or 2, further comprising estimating the flare pixel region based on an intensity value of the target pixel region in the intensity image data corrected by an intensity value of the target pixel region in the background light image data.

4. A control device (1) having a processor (1b) and controlling an optical sensor (10) that detects an echo of an irradiated light to a detection area (Ad) of a vehicle (5), The processor, acquiring distance image data (Dd) representing a distance value to a reflective target (Tr) that reflects light in the detection area based on the echo detected by the optical sensor in response to the irradiated light; acquiring intensity image data (Db) representing an intensity value of the echo reflected from the reflecting target in the detection area based on the echo detected by the optical sensor against background light having an intensity lower than that of the irradiated light in the detection area; estimating a flare pixel region (Rf) in which a flare is predicted to be captured around a target pixel region (Rt) in which the reflecting target is captured in the range image data, based on the intensity image data; and removing the distance value of the flare pixel region where the detection timing of the echo overlaps with the target pixel region in the range image data.

5. estimating the flare pixel region includes: The control device according to any one of claims 1 to 4, further comprising estimating the flare pixel region in a range correlated with an optical characteristic (Os) of a lens system (42) that forms an image of the echo in the optical sensor and the intensity value of the target pixel region in the intensity image data.

6. The acquiring of the range image data includes: acquiring the distance image data for each of a plurality of scanning lines (Ls) during a distance acquisition period (Pd); acquiring the intensity image data acquiring the intensity image data for each of the scanning lines (Ls) in an intensity acquisition period (Pi, Pb) prior to the distance acquisition period; estimating the flare pixel region includes: estimating the flare pixel region based on the intensity image data synthesized over the intensity acquisition period for each of the scan lines; Removing the distance values ​​comprises: The control device according to claim 1 , further comprising removing the distance values ​​of the flare pixel regions in the distance image data of the scan lines in which the flare pixel regions are estimated.

7. The acquiring of the range image data includes: acquiring the distance image data for each of a plurality of scanning lines (Ls) during a distance acquisition period (Pd); acquiring the intensity image data acquiring the intensity image data for each of the scanning lines (Ls) in an intensity acquisition period (Pi, Pb) prior to the distance acquisition period; estimating the flare pixel region includes: estimating the flare pixel region based on the intensity image data synthesized over the intensity acquisition period for each of the scan lines; Removing the distance values ​​comprises: The control device according to any one of claims 1 to 5, further comprising removing the distance values ​​in the flare pixel region in the distance image data synthesized over the distance acquisition period for each of the scanning lines.

8. A storage medium (1a), The processor, The control device according to any one of claims 1 to 7, further configured to store, in the storage medium, the distance image data from which the distance values ​​in the flare pixel regions have been removed.

9. A control method executed by a processor (1b) for controlling an optical sensor (10) for detecting an echo of an irradiated light onto a detection area (Ad) of a vehicle (5), the method comprising: acquiring distance image data (Dd) representing a distance value to a reflective target (Tr) reflecting light in the detection area based on the echo detected by the optical sensor in response to the irradiation light of a first intensity (I1); acquiring intensity image data (Di) representing an intensity value of the echo reflected from the reflecting target in the detection area based on the echo detected by the optical sensor in response to the irradiation light having a second intensity (I2) lower than the first intensity; estimating a flare pixel region (Rf) in which a flare is predicted to be captured around a target pixel region (Rt) in which the reflecting target is captured in the range image data, based on the intensity image data; and removing the distance value of the flare pixel region where the detection timing of the echo overlaps with the target pixel region in the distance image data.

10. A control method executed by a processor (1b) for controlling an optical sensor (10) for detecting an echo of an irradiated light onto a detection area (Ad) of a vehicle (5), the method comprising: acquiring distance image data (Dd) representing a distance value to a reflective target (Tr) that reflects light in the detection area based on the echo detected by the optical sensor in response to the irradiated light; acquiring intensity image data (Db) representing an intensity value of the echo reflected from the reflecting target in the detection area based on the echo detected by the optical sensor against background light having an intensity lower than that of the irradiated light in the detection area; estimating a flare pixel region (Rf) in which a flare is predicted to be captured around a target pixel region (Rt) in which the reflecting target is captured in the range image data, based on the intensity image data; and removing the distance value of the flare pixel region where the detection timing of the echo overlaps with the target pixel region in the distance image data.

11. A control program is stored in a storage medium (1a) for controlling an optical sensor (10) that detects an echo of light irradiated onto a detection area (Ad) of a vehicle (5), the control program including instructions to be executed by a processor (1b), The instruction: acquiring distance image data (Dd) representing a distance value to a reflective target (Tr) reflecting light in the detection area based on the echo detected by the optical sensor in response to the irradiation light of a first intensity (I1); acquiring intensity image data (Di) representing an intensity value of the echo reflected from the reflecting target in the detection area based on the echo detected by the optical sensor in response to the irradiation light having a second intensity (I2) lower than the first intensity; estimating a flare pixel region (Rf) in which a flare is predicted to be captured around a target pixel region (Rt) in which the reflecting target is captured in the range image data, based on the intensity image data; and removing the distance value of the flare pixel region where the detection timing of the echo overlaps with the target pixel region in the distance image data.

12. A control program is stored in a storage medium (1a) for controlling an optical sensor (10) that detects an echo of light irradiated onto a detection area (Ad) of a vehicle (5), the control program including instructions to be executed by a processor (1b), The instruction: acquiring distance image data (Dd) representing a distance value to a reflective target (Tr) reflecting light in the detection area based on the echo detected by the optical sensor in response to the irradiated light; acquiring intensity image data (Db) representing an intensity value of the echo reflected from the reflecting target in the detection area based on the echo detected by the optical sensor against background light having an intensity lower than that of the irradiated light in the detection area; estimating a flare pixel region (Rf) in which a flare is predicted to be captured around a target pixel region (Rt) in which the reflecting target is captured in the range image data, based on the intensity image data; and removing the distance value of the flare pixel region where the detection timing of the echo overlaps with the target pixel region in the distance image data.

Images