Control device, control method, and control program

Abstract

The processor of the control device that controls the optical sensor is configured to perform the following processing: acquire distance image data (Dd) representing distance values to a reflection target of light reflection in a detection region, based on echoes detected by the optical sensor for irradiation light of a first intensity (I1); acquire intensity image data (Di) representing intensity values of echoes reflected from a reflection target in the detection region, based on echoes detected by the optical sensor for irradiation light of a second intensity (I2) lower than the first intensity (I1); estimate, based on the intensity image data (Di), a glare pixel region (Rf) in which glare imaging is predicted around a target pixel region (Rt) in which a target image of the reflection target is captured, in the distance image data (Dd); and remove, in the distance image data (Dd), distance values of the glare pixel region (Rf) in which the detection timing of echoes overlaps with the target pixel region (Rt).

Description

[0001] Related applications

[0002] This application is based on Japanese Patent Application No. 2021-149661, filed on September 14, 2021, the contents of which are incorporated herein by reference in their entirety. Technical Field

[0003] This disclosure relates to techniques for controlling optical sensors, wherein the optical sensors are used to detect the echo of illumination light illuminating a detection area of ​​a vehicle. Background Technology

[0004] In the technology disclosed in Patent Document 1, in order to detect the reflected echo of the irradiated light, i.e., the incident light, in a ranging device that serves as an optical sensor, two types of pixels with different sensitivities are mixed together. Therefore, even under conditions of high incident light intensity, the incident light can be detected without false detections due to saturation in the pixels on the low-sensitivity side.

[0005] Prior art literature

[0006] Patent documents

[0007] Patent Document 1: Japanese Patent Application Publication No. 2019-190892 Summary of the Invention

[0008] However, when the reflection intensity of the echo is high, there is a risk that glare, caused by factors such as unnecessary reflections inside the lens, may lead to inaccurate imaging of the subject by the optical sensor. However, the technology disclosed in Patent Document 1 is insufficient to solve this problem of false detection caused by glare.

[0009] The present disclosure addresses the problem of providing a control device for suppressing false detections of optical sensors. Another problem is providing a control method for suppressing false detections of optical sensors. Yet another problem is providing a control program for suppressing false detections of optical sensors.

[0010] The technical means of this disclosure used to solve the problem will be described below.

[0011] The first aspect of this disclosure relates to a control device having a processor and controlling an optical sensor, wherein the optical sensor detects the echo of illumination light incident on a detection area of ​​the vehicle, and in the control device,

[0012] The processor is configured to perform the following processes:

[0013] Based on the echo detected by the optical sensor for illumination of a first intensity, distance image data representing the distance value up to the reflective target in the detection area that reflects the light is obtained;

[0014] Intensity image data representing the intensity value of the echo reflected from a reflective target in the detection area is acquired based on the echo detected by the optical sensor for a second intensity of illumination light that is lower than the first intensity.

[0015] Based on intensity image data, glare pixel regions are estimated in distance image data to predict the area around the target pixel region where the reflective target is captured; and

[0016] The distance value of the glare pixel region that overlaps with the target pixel region is used to remove the detection timing of echoes from the distance image data.

[0017] The second aspect of this disclosure relates to a control method executed by a processor for controlling an optical sensor, wherein the optical sensor detects the echo of illumination light incident on a detection area of ​​the vehicle, and the control method includes:

[0018] Based on the echo detected by the optical sensor for illumination of a first intensity, distance image data representing the distance value up to the reflective target in the detection area that reflects the light is obtained;

[0019] Intensity image data representing the intensity value of the echo reflected from a reflective target in the detection area is acquired based on the echo detected by the optical sensor for a second intensity of illumination light that is lower than the first intensity.

[0020] Based on intensity image data, glare pixel regions are estimated in distance image data to predict the area around the target pixel region where the reflective target is captured; and

[0021] The distance value of the glare pixel region that overlaps with the target pixel region is used to remove the detection timing of echoes from the distance image data.

[0022] The third aspect of this disclosure relates to a control program, which is stored in a storage medium for controlling an optical sensor and contains commands for execution by a processor, wherein the optical sensor detects the echo of illumination light illuminating a detection area of ​​the vehicle, and in the control program,

[0023] The commands include:

[0024] Based on the echo detected by the optical sensor for illumination of a first intensity, distance image data representing the distance value up to the reflective target in the detection area that reflects the light is obtained;

[0025] Intensity image data representing the intensity value of the echo reflected from a reflective target in the detection area is acquired based on the echo detected by the optical sensor for a second intensity of illumination light that is lower than the first intensity.

[0026] Based on intensity image data, glare pixel regions are estimated in distance image data to predict the area around the target pixel region where the reflective target is captured; and

[0027] The distance value of the glare pixel region that overlaps with the target pixel region is used to remove the detection timing of echoes from the distance image data.

[0028] According to the first to third schemes described above, distance image data representing the distance value up to the reflective target reflecting light in the detection area is acquired based on the echo detected by the optical sensor for illumination light of a first intensity. Therefore, in the first to third schemes, intensity image data representing the intensity value of the echo reflected from the reflective target in the detection area is acquired based on the echo detected by the optical sensor for illumination light of a second intensity lower than the first intensity. Accordingly, based on the intensity image data that suppresses glare imaging according to low-intensity illumination light, glare pixel regions that predict the glare imaging around the target pixel region where the reflective target is captured can be appropriately estimated in the distance image data. Therefore, the distance value of the glare pixel region where the echo detection timing overlaps with the target pixel region in the distance image data can be removed as a spurious value caused by glare imaging, thus suppressing false detection of distance values ​​in the optical sensor.

[0029] The fourth aspect of this disclosure relates to a control device having a processor and controlling an optical sensor, wherein the optical sensor detects the echo of illumination light incident on a detection area of ​​the vehicle, and in the control device,

[0030] The processor is configured to perform the following processes:

[0031] Based on the echo detected by the optical sensor for the illumination light, distance image data representing the distance value up to the reflective target that reflects the light in the detection area is obtained;

[0032] Intensity image data representing the intensity value of the echo reflected from the reflective target in the detection area is obtained based on the echo detected by the optical sensor for the background light with a lower intensity than the illumination light in the detection area.

[0033] Based on intensity image data, glare pixel regions are estimated in distance image data to predict the area around the target pixel region where the reflective target is captured; and

[0034] The distance value of the glare pixel region that overlaps with the target pixel region is used to remove the detection timing of echoes from the distance image data.

[0035] The fifth aspect of this disclosure relates to a control method executed by a processor for controlling an optical sensor, wherein the optical sensor detects the echo of illumination light incident on a detection area of ​​the vehicle, and the control method includes:

[0036] Based on the echo detected by the optical sensor for the illumination light, distance image data representing the distance value up to the reflective target that reflects the light in the detection area is obtained;

[0037] Intensity image data representing the intensity value of the echo reflected from the reflective target in the detection area is obtained based on the echo detected by the optical sensor for the background light with a lower intensity than the illumination light in the detection area.

[0038] Based on intensity image data, glare pixel regions are estimated in distance image data to predict the area around the target pixel region where the reflective target is captured; and

[0039] The distance value of the glare pixel region that overlaps with the target pixel region is used to remove the detection timing of echoes from the distance image data.

[0040] The sixth aspect of this disclosure relates to a control program, which is stored in a storage medium for controlling an optical sensor and contains commands for execution by a processor, wherein the optical sensor detects the echo of illumination light illuminating a detection area of ​​the vehicle, and in the control program,

[0041] Commands include:

[0042] Based on the echo detected by the optical sensor for the illumination light, distance image data representing the distance value up to the reflective target that reflects the light in the detection area is obtained;

[0043] Intensity image data representing the intensity value of the echo reflected from the reflective target in the detection area is obtained based on the echo detected by the optical sensor for the background light with a lower intensity than the illumination light in the detection area.

[0044] Based on intensity image data, glare pixel regions are estimated in distance image data to predict the area around the target pixel region where the reflective target is captured; and

[0045] The distance value of the glare pixel region that overlaps with the target pixel region is used to remove the detection timing of echoes from the distance image data.

[0046] According to the fourth to sixth schemes described above, distance image data representing the distance value up to the reflective target reflecting light in the detection area is acquired based on the echo detected by the optical sensor for the illumination light. Therefore, in the fourth to sixth schemes, intensity image data representing the intensity value of the echo reflected from the reflective target in the detection area is acquired based on the echo detected by the optical sensor for background light with an intensity lower than the illumination light in the detection area. Accordingly, based on the intensity image data that suppresses glare due to low-intensity background light, glare pixel areas predicting the glare image around the target pixel area of ​​the reflected target can be appropriately estimated in the distance image data. Therefore, the distance value of the glare pixel area where the echo detection timing overlaps with the target pixel area in the distance image data can be removed as a spurious value caused by glare, thus suppressing false detection of distance values ​​in the optical sensor. Attached Figure Description

[0047] Figure 1 This is a schematic diagram showing the overall configuration of the detection system according to the first embodiment.

[0048] Figure 2 This is a schematic diagram showing the detailed configuration of the optical sensor according to the first embodiment.

[0049] Figure 3 This is a block diagram illustrating the functional configuration of the control device in the first embodiment.

[0050] Figure 4 This is a schematic diagram showing the projector of the first embodiment.

[0051] Figure 5 This is a schematic diagram illustrating the characteristics of the laser diode according to the first embodiment.

[0052] Figure 6 This is a timing diagram representing the detection frame of the first embodiment.

[0053] Figure 7 This is a schematic diagram showing the light receiver of the first embodiment.

[0054] Figure 8 This is a schematic diagram illustrating the distance image data of the first embodiment.

[0055] Figure 9 This is a schematic diagram used to illustrate the intensity image data of the first embodiment.

[0056] Figure 10 This is a schematic diagram illustrating the reflective target in the first embodiment.

[0057] Figure 11 This is a coordinate graph used to illustrate the optical characteristics of the first embodiment.

[0058] Figure 12 This is a schematic diagram illustrating the glare pixel region of the first embodiment.

[0059] Figure 13 This is a schematic diagram illustrating the glare pixel region of the first embodiment.

[0060] Figure 14 This is a coordinate graph used to illustrate the removal of distance values ​​in the first embodiment.

[0061] Figure 15 This is a coordinate graph used to illustrate the removal of distance values ​​in the first embodiment.

[0062] Figure 16 This is a flowchart illustrating the control flow of the first embodiment.

[0063] Figure 17 This is a block diagram illustrating the functional configuration of the control device in the second embodiment.

[0064] Figure 18 This is a schematic diagram illustrating the removal of distance values ​​in the second embodiment.

[0065] Figure 19 This is a flowchart illustrating the control flow of the second embodiment.

[0066] Figure 20 This is a block diagram illustrating the functional configuration of the control device in the third embodiment.

[0067] Figure 21 This is a timing diagram representing the detection frame of the third embodiment.

[0068] Figure 22 This is a flowchart illustrating the control flow of the third embodiment.

[0069] Figure 23 This is a block diagram illustrating the functional configuration of the control device in the fourth embodiment.

[0070] Figure 24 This is a timing diagram representing the detection frame of the fourth embodiment.

[0071] Figure 25 This is a flowchart illustrating the control flow of the fourth embodiment.

[0072] Figure 26 This is a flowchart illustrating the control flow of the fourth embodiment.

[0073] Figure 27 This is a block diagram illustrating the functional configuration of the control device in the modified example.

[0074] Figure 28 It is a flowchart representing the control flow of a variant example.

[0075] Figure 29 This is a block diagram illustrating the functional configuration of the control device in the modified example.

[0076] Figure 30 It is a flowchart representing the control flow of a variant example.

[0077] Figure 31 This is a block diagram illustrating the functional configuration of the control device in the modified example.

[0078] Figure 32 It is a flowchart representing the control flow of a variant example.

[0079] Figure 33 This is a block diagram illustrating the functional configuration of the control device in the modified example.

[0080] Figure 34 It is a flowchart representing the control flow of a variant example. Detailed Implementation

[0081] Hereinafter, several embodiments of the present disclosure will be described based on the accompanying drawings. Furthermore, in each embodiment, the same reference numerals are used to denote corresponding constituent elements, and repeated descriptions may be omitted. Also, when only a portion of the configuration is described in each embodiment, the configurations of other previously described embodiments can be applied to the other parts of that configuration. Furthermore, not only combinations of configurations explicitly shown in the descriptions of each embodiment, but also combinations of configurations from multiple embodiments can be partially combined with each other, even without explicit description, as long as such combinations do not present particular obstacles.

[0082] (First Implementation)

[0083] like Figure 1 As shown, the first embodiment of this disclosure relates to a detection system 2 comprising an optical sensor 10 and a control device 1. The detection system 2 is mounted on a vehicle 5. The vehicle 5 is a mobile body, such as a car, capable of traveling on a road with occupants in a passenger position.

[0084] Vehicle 5 is capable of stable or temporary autonomous driving in automatic driving control mode. Here, automatic driving control mode can be achieved through autonomous driving control where the system performs all driving tasks during operation, such as conditional driving automation, high driving automation, or full driving automation. Automatic driving control mode can also be achieved through high driving support control where the occupant performs some or all of the driving tasks, such as driving assistance or partial driving automation. Automatic driving control mode can also be achieved through any one, combination, or switching between the aforementioned autonomous driving control and high driving support control.

[0085] Furthermore, in the following explanation, unless otherwise stated, the directions of front, rear, up, down, left, and right are defined with reference to vehicle 5 on the horizontal plane. Additionally, the horizontal direction refers to a direction parallel to the horizontal plane that serves as the directional reference for vehicle 5. Furthermore, the vertical direction refers to a direction perpendicular to the horizontal plane that serves as the directional reference for vehicle 5.

[0086] The optical sensor 10 is a so-called LiDAR (Light Detection and Ranging / Laser Imaging Detection and Ranging) system used to acquire image data that can be effectively utilized in the driving control of the vehicle 5, including automatic control driving modes. The optical sensor 10 is disposed at at least one location in the vehicle 5, such as in the front, left and right sides, rear, and top of the vehicle. Figure 2 As shown, in the optical sensor 10, a three-dimensional orthogonal coordinate system is defined using the X-axis, Y-axis, and Z-axis as three mutually orthogonal axes. Here, particularly in this embodiment, the X-axis and Z-axis are set along different horizontal directions of the vehicle 5, while the Y-axis is set along the vertical direction of the vehicle 5. Furthermore, in Figure 2 In the diagram, the portion to the left of the dashed line along the Y-axis (the side of the light-transmitting cover 12 described later) is actually a cross-section perpendicular to the portion to the right of the dashed line (the side of units 21 and 41 described later).

[0087] like Figure 3 As shown, the optical sensor 10 illuminates light toward the detection area Ad in the external space of the vehicle 5, corresponding to the placement location. The optical sensor 10 detects the reflected light from the detection area Ad as an echo from the detection area Ad. The optical sensor 10 can also detect the background light (i.e., external light) reflected from the detection area Ad when the illuminating light is not illuminating it as an echo from the detection area Ad.

[0088] The optical sensor 10 observes the reflecting target Tr of the reflected light within the detection area Ad by detecting such echoes. Here, specifically in this embodiment, observation means sensing the distance from the optical sensor 10 to the reflecting target Tr and the intensity of the echo reflected back from the reflecting target Tr. Representative observation targets in the optical sensor 10 suitable for the vehicle 5 can be at least one of moving objects, such as pedestrians, cyclists, animals other than humans, and other vehicles. Representative observation targets in the optical sensor 10 suitable for the vehicle 5 can also be at least one of stationary objects, such as guardrails, road signs, roadside structures, and debris on the road.

[0089] like Figure 2 As shown, the optical sensor 10 is configured to include a frame 11, a light-emitting unit 21, a scanning unit 31, and a light-receiving unit 41. The frame 11 constitutes the outer packaging of the optical sensor 10. The frame 11 is box-shaped and has light-shielding properties. The frame 11 houses the light-emitting unit 21, the scanning unit 31, and the light-receiving unit 41 inside. A light-transmitting cover 12 is provided on the open optical window in the frame 11. The light-transmitting cover 12 is plate-shaped and has light transmittance relative to the aforementioned illumination light and echo. The light-transmitting cover 12 closes the optical window of the frame 11 in a manner that allows both illumination light and echo to pass through.

[0090] The projection unit 21 includes a projector 22 and a projection lens system 26. The projector 22 is disposed within the frame 11. Figure 4 As shown, the projector 22 is formed by arranging a plurality of laser diodes 24 in an array on a substrate. Each laser diode 24 is arranged in a single column along the Y-axis. Each laser diode 24 has a resonator structure that enables light oscillating in the PN junction layer to resonate and a mirror layer structure that enables light to be repeatedly reflected through the PN junction layer.

[0091] Each laser diode 24 emits light corresponding to the application of current according to the control signal from the control device 1. In particular, each laser diode 24 in this embodiment emits light in the near-infrared region, which is difficult for humans in the external space of the vehicle 5 to visually perceive. Figure 5 As shown, each laser diode 24 enters an oscillating LD (Laser Diode) mode when a current higher than the switching current value Cth is applied, thereby emitting pulsed light. The light emitted from each laser diode 24 in this LD mode is as follows... Figure 6 The illumination light shown constitutes the near-infrared region with an intensity of the first intensity I1. On the other hand, as... Figure 5As shown, each laser diode 24 enters a non-oscillating LED (Light Emitting Diode) mode when a current lower than the switching current value Cth is applied, thereby emitting DC (Direct Current) light. The light emitted from each laser diode 24 in this LED mode is as follows... Figure 6 The light shown is an illumination light with a second intensity I2 that is lower than the first intensity I1, constituting the near-infrared region.

[0092] like Figure 4 As shown, the projector 22 has a virtually defined projection window 25 on one side of the substrate, having a rectangular outline with its long side along the Y-axis. The projection window 25 is configured as an assembly of projection openings in each laser diode 24. Light emitted from the projection openings of each laser diode 24 is projected from the projection window 25 as a linear illumination light simulated along the Y-axis in the detection region Ad. The illumination light may also include non-light-emitting portions in the Y-axis direction corresponding to the arrangement spacing of each laser diode 24. In this case, it is preferable to form a linear illumination light in the detection region Ad where the non-light-emitting portions are macroscopically eliminated due to diffraction.

[0093] like Figure 2 As shown, the projection lens system 26 projects the illumination light from the projector 22 toward the scanning mirror 32 of the scanning unit 31. The projection lens system 26 is disposed within the housing 11 between the projector 22 and the scanning mirror 32. The projection lens system 26 performs at least one optical function, such as focusing, collimation, and shaping. The projection lens system 26 forms a projection optical axis along the Z-axis. The projection lens system 26 has at least one projection lens 27 on the projection optical axis, with a lens shape corresponding to the optical function to be performed. The projector 22 is positioned on the projection optical axis of the projection lens system 26. Illumination light emitted from the center of the projection window 25 in the projector 22 is guided along the projection optical axis of the projection lens system 26.

[0094] The scanning unit 31 includes a scanning mirror 32 and a scanning motor 35. The scanning mirror 32 scans the illumination light emitted from the projection lens system 26 of the projection unit 21 toward the detection area Ad, and reflects the echo from the detection area Ad toward the light-receiving lens system 42 of the light-receiving unit 41. The scanning mirror 32 is positioned between the light-transmitting cover 12 and the projection lens system 26 in the optical path of the illumination light, and between the light-transmitting cover 12 and the light-receiving lens system 42 in the optical path of the echo.

[0095] The scanning mirror 32 is formed into a plate shape by depositing a reflective film on one side of the substrate, namely the reflective surface 33. The scanning mirror 32 is supported by the frame 11 and can rotate about a rotation center line along the Y-axis. The normal direction of the reflective surface 33 can be adjusted by rotating the scanning mirror 32 about the rotation center line. The scanning mirror 32 can oscillate within a limited driving range by mechanical or electric limiting members.

[0096] The scanning mirror 32 is provided in common with respect to the light-projecting unit 21 and the light-receiving unit 41. That is, the scanning mirror 32 is provided in common with respect to the illumination light and the echo. Thus, the scanning mirror 32 has the light-projecting reflective surface for reflecting the illumination light and the light-receiving reflective surface for reflecting the echo formed on the reflective surface 33 in a manner offset along the Y-axis direction.

[0097] The illumination light is reflected from the projection reflector surface on the reflective surface 33, which is oriented in the normal direction corresponding to the rotation of the scanning mirror 32, and thus passes through the light-transmitting cover 12 to scan the detection area Ad in time and space. At this time, the scanning of the detection area Ad based on the illumination light is essentially limited to a horizontal scan. The illumination light and background light are reflected as echoes from the reflective target Tr present in the detection area Ad and incident on the optical sensor 10. Such echoes pass through the light-transmitting cover 12 and are reflected from the light-receiving reflector surface on the reflective surface 33, which is oriented in the normal direction corresponding to the rotation of the scanning mirror 32, and are thus guided to the light-receiving lens system 42 of the light-receiving unit 41. Here, the speed of the illumination light and the echo is sufficiently high relative to the rotational speed of the scanning mirror 32. As a result, the echo relative to the illumination light is guided to the light-receiving lens system 42 in a manner that runs counter to the illumination light on the scanning mirror 32, which is rotated at approximately the same angle as the illumination light.

[0098] A scanning motor 35 is disposed within the housing 11 around the scanning mirror 32. The scanning motor 35 is, for example, a voice coil motor, a brushed DC motor, or a stepper motor. The scanning motor 35 drives the scanning mirror 32 to rotate (i.e., oscillate) within a limited driving range according to the control signal from the control device 1.

[0099] The light-receiving unit 41 includes a light-receiving lens system 42 and a light receiver 45. The light-receiving lens system 42 guides the echo reflected by the scanning mirror 32 toward the light receiver 45. The light-receiving lens system 42 is disposed within the housing 11 between the scanning mirror 32 and the light receiver 45. The light-receiving lens system 42 is positioned lower than the projection lens system 26 in the Y-axis direction. The light-receiving lens system 42 performs an optical function in a manner that images the echo onto the light receiver 45. The light-receiving lens system 42 forms a light-receiving optical axis along the Z-axis. The light-receiving lens system 42 has at least one light-receiving lens 43 on the light-receiving optical axis, with a lens shape corresponding to the optical function to be performed. The echo from the detection area Ad, reflected from the light-receiving reflective surface in the reflective surface 33 of the scanning mirror 32, is guided along the light-receiving optical axis of the light-receiving lens system 42 within the driving range of the scanning mirror 32.

[0100] The light receiver 45 receives the echo from the detection area Ad, imaged by the light-receiving lens system 42, and thereby outputs a detection signal corresponding to the received light. The light receiver 45 is disposed within the housing 11 on the side opposite to the scanning mirror 32, separated by the light-receiving lens system 42. The light receiver 45 is positioned below the projector 22 in the Y-axis direction and is located on the light-receiving optical axis of the light-receiving lens system 42.

[0101] like Figure 7 As shown, the light receiver 45 is formed by arranging light-receiving elements 46 in a two-dimensional array along the X and Y axes on a substrate. Each light-receiving element 46 is composed of multiple light-receiving components. That is, each light-receiving element 46 corresponds to multiple light-receiving components, so the output value varies depending on the response number of these light-receiving components. The light-receiving components of each light-receiving element 46 can be constructed, for example, using a photodiode such as a single-photon avalanche diode (SPAD) as the main body. Alternatively, the light-receiving components of each light-receiving element 46 can be integrally constructed by stacking a microlens array in front of a photodiode array.

[0102] The light receiver 45 has a rectangular light-receiving surface 47 formed on one side of the substrate. The light-receiving surface 47 is constructed as an assembly of the incident surfaces of each light-receiving element 46. The geometric center of the light-receiving surface 47 relative to the rectangular outline is aligned with or slightly offset from the light-receiving optical axis of the light-receiving lens system 42. Each light-receiving element 46 uses its own light-receiving element to receive the echo incident from the light-receiving lens system 42 onto the light-receiving surface 47. Here, the long side of the rectangular light-receiving surface 47 is defined along the Y-axis. Thus, corresponding to the linear illumination light in the detection region Ad, the echo relative to the illumination light is received by the light-receiving element of each light-receiving element 46 as a linearly extended beam of light.

[0103] like Figure 2 As shown, the photodetector 45 integrally incorporates a decoder 48. The decoder 48 sequentially reads out the electrical pulses generated by each photodetector element 46 based on the reception of the echo on the photodetector surface 47 through sampling processing. The decoder 48 uses the sequentially read electrical pulses as... Figure 6 The detection signal in the detection frame (i.e., detection cycle) Fd shown is output to the control device 1. The detection frame Fd is repeated at predetermined time intervals during the start-up of the vehicle 5. In the control device 1, which receives the detection signal from the decoder 48, based on physical quantities related to the echoes detected by each light-receiving element 46 accompanying the rotation of the scanning mirror 32, it is possible to... Figure 8 , Figure 9 Image data Dd and Di representing the target observation results within the detection area Ad are obtained as shown. In the image data Dd and Di obtained in this way, the vertical direction corresponds to the Y-axis direction of vehicle 5, and the horizontal direction corresponds to the X-axis direction of vehicle 5.

[0104] Figure 1 The control device 1 shown is connected to the optical sensor 10 via at least one of, for example, a LAN (Local Area Network), a wiring harness, and an internal bus. The control device 1 is configured to include at least one dedicated computer. The dedicated computer constituting the control device 1 can also be a sensor ECU (Electronic Control Unit) dedicated to controlling the optical sensor 10; in this case, the sensor ECU can also be housed within the housing 11. The dedicated computer constituting the control device 1 can also be a driving control ECU that controls the driving of the vehicle 5. The dedicated computer constituting the control device 1 can also be a navigation ECU that navigates the driving path of the vehicle 5. The dedicated computer constituting the control device 1 can also be a locator ECU that estimates the self-state of the vehicle 5.

[0105] The dedicated computer constituting the control device 1 has at least one memory 1a and one processor 1b. The memory 1a is a non-transitory tangible storage medium, such as semiconductor memory, magnetic media, and optical media, that non-transitors programs and data that can be read by the computer. The processor 1b includes at least one of the following as its core: CPU (Central Processing Unit), GPU (Graphics Processing Unit), RISC (Reduced Instruction Set Computer) CPU, DFP (Data Flow Processor), and GSP (Graphics Streaming Processor).

[0106] The processor 1b executes multiple commands contained in the control program stored in the memory 1a. Thus, the control device 1 constructs multiple functional modules for controlling the optical sensor 10. In this way, within the control device 1, the control program stored in the memory 1a for controlling the optical sensor 10 causes the processor 1b to execute multiple commands to construct multiple functional modules. The multiple functional modules constructed by the control device 1 are as follows: Figure 3 The diagram includes a distance acquisition module 100, an intensity acquisition module 110, an estimation module 120, and a removal module 130.

[0107] Distance acquisition module 100 in Figure 6 During the distance acquisition period Pd set in the detection frame Fd shown, the optical sensor 10 is controlled in LD mode, where each laser diode 24 is in an oscillating state. By controlling it to LD mode, a first intensity I1 of illumination light is intermittently pulsed from the optical sensor 10 onto the detection area Ad. Therefore, the distance acquisition module 100 acquires distance based on the echo detected by the optical sensor 10 of the first intensity I1 of illumination light. Figure 8 The distance image data Dd of the three-dimensional point group shown represents the distance value up to the reflective target Tr in the detection region Ad. At this time, the distance value of each pixel that constitutes the distance image data Dd is obtained by dTOF (direct Time Of Flight) based on the time of flight of light from pulse illumination to the detection of the echo.

[0108] Along with acquiring distance image data Dd, the distance acquisition module 100 also synchronously controls the scanning motor 35 to rotate the scanning mirror 32 in conjunction with the pulsed illumination of the illumination light. Therefore, the distance acquisition module 100 generates distance image data Dd by using multiple scan lines Ls corresponding to the rotation angle of the scanning mirror 32, and can synthesize the distance image data Dd generated by each scan line Ls throughout the distance acquisition period Pd. Here, the scan lines Ls associated with the distance image data Dd are set as vertical pixel columns corresponding to the Y-axis direction, and multiple columns are set horizontally corresponding to the X-axis direction.

[0109] Figure 3 The strength acquisition module 110 shown is in Figure 6 In the detection frame Fd shown, the intensity acquisition period Pi, which is set earlier than the distance acquisition period Pd, controls the optical sensor 10 to operate in LED mode, where each laser diode 24 is in a non-oscillating state. By controlling it to LED mode, a second intensity I2, which is lower than the first intensity I1, is continuously irradiated from the optical sensor 10 into the detection area Ad. Therefore, the intensity acquisition module 110 acquires the intensity based on the echo detected by the optical sensor 10 of the second intensity I2 irradiated light. Figure 9 The two-dimensional intensity image data Di shown represents the intensity value reflected from the reflective target Tr in the detection region Ad.

[0110] Along with the acquisition of intensity image data Di, the intensity acquisition module 110 also controls the rotation drive of the scanning mirror 32 by the scanning motor 35 in parallel with the continuous illumination of the illumination light. Therefore, the intensity acquisition module 110 can synthesize the intensity image data Di generated by each scanning line Ls according to a plurality of scan lines Ls corresponding to the rotation angle of the scanning mirror 32, and can synthesize the intensity image data Di generated by each scan line Ls throughout the intensity acquisition period Pi. Here, as Figure 9 As shown, the scan line Ls of the intensity image data Di is set to be consistent with... Figure 8 The scan lines Ls of the distance image data Dd shown are also in a 1:1 correspondence. Additionally, in Figure 8 , 9 In the image data Dd and Di, only the initial, central, and final scan lines Ls are shown with thick outlines; the scan lines Ls other than these are omitted.

[0111] Figure 3 The estimation module 120 shown uses the intensity image data Di obtained by synthesizing the intensity data Pi obtained by spreading the intensity of each scan line Ls throughout the intensity acquisition period, such as... Figure 9 The target pixel region Rt is explored and captured as shown. Here, the target pixel region Rt is defined as the pixel region that is predicted to be affected by the illumination light of the first intensity I1 corresponding to the distance image data Dd due to the reflection from... Figure 10 The area shown, for example, is a region where strong reflection from a reflective target Tr, such as an identifier, produces glare, and represents an intensity value above or exceeding the prediction threshold. The first embodiment applying this definition assumes that glare imaging is suppressed as much as possible for illumination light of a second intensity I2 corresponding to the intensity image data Di.

[0112] Therefore, as Figure 8 , 9 As shown, the estimation module 120 estimates, based on the intensity value of the target pixel region Rt in the intensity image data Di, a glare pixel region Rf predicted around the target pixel region Rt in which the reflective target Tr is captured, based on the intensity value of the target pixel region Rt in the distance image data Dd. Additionally, in Figure 8 , Figure 9 In this diagram, the reflective target Tr that is predicted to produce glare is hypothetically represented by a double-dotted line, thereby schematically corresponding the reflective target Tr with each region Rt and Rf.

[0113] Specifically, the estimation module 120 estimates the glare pixel region Rf related to the optical characteristics Os of the light-receiving lens system 42 in the optical sensor 10 and the intensity value of the target pixel region Rt in the intensity image data Di. Here, the optical characteristics Os are as follows: Figure 11 The probability of glare generation around the reflective target Tr is given by a range Ef that is above or exceeds a set value. Therefore, the optical characteristics Os are as follows: Figure 12 , Figure 13 The range Ef corresponding to the incident intensity Ii of the echo-receiving lens system 42 is stored in memory 1a as a function or table, as shown by the cross-shading. Here, the incident intensity Ii of the echo-receiving lens system 42 corresponding to the first intensity I1 can be estimated based on, for example, a representative value or an average value related to the intensity value of the target pixel region Rt corresponding to the second intensity I2.

[0114] Based on this, the estimation module 120 extracts the glare pixel region Rf from the distance image data Dd, which is estimated to be related to the range Ef of the optical characteristics Os of the light-receiving lens system 42 and the intensity value of the target pixel region Rt in the intensity image data Di. This estimation of the glare pixel region Rf can be considered as an estimation for the distance image data Dd acquired separately for each scan line Ls, and also as an estimation for the distance image data Dd synthesized by applying the values ​​of each scan line Ls across the distance acquisition period Pd.

[0115] Figure 3 The removal module 130 shown in the first embodiment extracts the image data Dd according to the distance of each scan line Ls, such as... Figure 14The distance image data Dd of the scan line Ls with glare pixel region Rf is estimated as shown. The removal module 130 extracts the distance image data Dd of the scan line Ls as shown... Figure 15 The distance value of the glare pixel region Rf, which overlaps with the target pixel region Rt, as shown by the thick-lined ellipse enclosing the echo detection timing, is removed as a pseudo-value. Here, removal means deleting the group of points representing the distance value of the corresponding glare pixel region Rf from the distance image data Dd. Furthermore, overlap here refers to the overlap of echo intensity waveforms above the baseline due to the peak points of the echo being detected within a specified error range. Here, in the dTOF applicable to the optical sensor 10, the echo detection timing corresponds to the distance value in a 1:1 ratio. Therefore, removing the distance value corresponding to the detection timing is essentially the same as removing the distance value of the glare pixel region Rf when the difference between the distance value of the glare pixel region Rf and the distance value of the target pixel region Rt falls within a specified error range.

[0116] The removal module 130 synthesizes distance image data Dd, in which the distance values ​​of the glare pixel regions Rf are removed according to each scan line Ls, throughout the distance acquisition period Pd. Therefore, the removal module 130 can also associate the distance image data Dd with the distance values ​​of the glare pixel regions Rf removed with at least one of, for example, a timestamp and vehicle 5 driving environment information, and store it in memory 1a. The removal module 130 can also associate the distance image data Dd with the distance values ​​of the glare pixel regions Rf removed with at least one of, for example, a timestamp and vehicle 5 driving environment information, and send it to an external center, thereby accumulating it in the storage medium of that external center.

[0117] according to Figure 16 The control flow shown is a control method for controlling the optical sensor 10 of the vehicle 5 by the cooperation of the modules 100, 110, 120, and 130 described so far. This control flow is repeatedly executed according to each detection frame Fd during the start-up of the vehicle 5. In addition, each "S" in the control flow means multiple steps executed by multiple commands included in the control program.

[0118] In S101, during the intensity acquisition period Pi of the current detection frame Fd, the intensity acquisition module 110 acquires intensity image data Di representing the intensity value of the echo from the reflecting target Tr for the illumination light of the second intensity I2. At this time, the intensity image data Di of each scan line Ls is synthesized throughout the intensity acquisition period Pi.

[0119] In the following S102, the estimation module 120 determines whether a target pixel region Rt, representing an intensity value above or exceeding the prediction threshold, exists in the intensity image data Di. If a positive determination is made, the control flow proceeds to S103.

[0120] In S103, the estimation module 120 estimates the glare pixel region Rf at a distance from the image data Dd based on the intensity value of the target pixel region Rt in the intensity image data Di. At this time, the glare pixel region Rf is estimated to be a range Ef related to the optical characteristics Os of the light-receiving lens system 42 and the intensity value of the target pixel region Rt in the intensity image data Di.

[0121] In the following S104, during the distance acquisition period Pd in ​​the current detection frame Fd, the distance acquisition module 100 acquires distance image data Dd representing the distance value up to the reflecting target Tr for each scan line Ls under the illumination light of the first intensity I1. Therefore, in S105, when the distance image data Dd of each scan line Ls is acquired each time, the control flow moves to S105, where module 130 determines whether the scan line Ls for which the distance image data Dd has been acquired is the scan line Ls that was presumed to have the glare pixel region Rf in S103. As a result, if an affirmative determination is made, the control flow moves to S106.

[0122] In S106, the removal module 130 determines whether there is a distance value in the distance image data Dd of the scan line Ls where the estimated glare pixel region Rf overlaps with the target pixel region Rt, indicating whether the echo detection timing exists. If a positive determination is made, the control flow moves to S107. In S107, the removal module 130 removes the distance value of the glare pixel region Rf overlapping with the target pixel region Rt from the distance image data Dd of the scan line Ls where the estimated glare pixel region Rf overlaps with the target pixel region Rt, treating it as a false value.

[0123] If execution of S107 is completed, or if a negative determination is made in each of S105 and S106, the control flow moves to S108. In S108, the distance acquisition module 100 determines whether the distance acquisition period Pd has ended. If a negative determination is made, the control flow returns to S104 based on the distance acquisition module 100, thereby acquiring the distance image data Dd for the next scan line Ls where the scan is not yet complete. On the other hand, if a positive determination is made, the control flow moves to S109 based on the removal module 130, thereby synthesizing the distance image data Dd of each scan line Ls, including the distance image data Dd of the scan line Ls with spurious values ​​removed, over the distance acquisition period Pd. When execution of S109 is completed, the current execution of the control flow ends.

[0124] Furthermore, if a negative decision is made in S102, the control flow moves to S110. In S110, the distance acquisition module 100 acquires the distance image data Dd of the intensity acquisition period Pi in the detection frame Fd according to each scan line Ls, just as in S104, and then performs synthesis in the same way as in S109. When the execution of S110 is completed, the current execution of the control flow ends.

[0125] (Effects)

[0126] The effects of the first embodiment described above will now be explained.

[0127] According to the first embodiment, distance image data Dd, representing the distance value up to the reflecting target Tr in the detection area Ad, is acquired based on the echo detected by the optical sensor 10 for illumination light of a first intensity I1. Therefore, in the first embodiment, intensity image data Di, representing the intensity value of the echo reflected from the reflecting target Tr in the detection area Ad, is acquired based on the echo detected by the optical sensor 10 for illumination light of a second intensity I2, which is lower than the first intensity I1. Accordingly, based on the intensity image data Di, which suppresses glare imaging according to low-intensity illumination light, a glare pixel region Rf predicting the glare imaging around the target pixel region Rt where the reflecting target Tr is captured, can be appropriately estimated in the distance image data Dd. Therefore, the distance value of the glare pixel region Rf, where the echo detection timing overlaps with the target pixel region Rt, can be removed as a spurious value caused by glare imaging, thus suppressing false detection of distance values ​​in the optical sensor 10.

[0128] According to the first embodiment, distance image data Dd is acquired for illumination light of a first intensity I1 generated by a laser diode 24 controlled to oscillate in the optical sensor 10. Therefore, in the first embodiment, intensity image data Di is acquired for illumination light of a second intensity I2 generated by a laser diode 24 controlled to be in a non-oscillating state in the optical sensor 10. Accordingly, based on the intensity variation of the illumination light generated by the common laser diode 24, not only the distance image data Dd, which is the object of false detection suppression, can be acquired, but also the intensity image data Di used to estimate the glare pixel region Rf can be acquired. Therefore, false detection of distance values ​​can be suppressed using a relatively small optical sensor 10.

[0129] According to the first embodiment, the glare pixel region Rf is estimated to be within a range Ef related to the optical characteristics Os of the light-receiving lens system 42 that images the echo in the optical sensor 10 and the intensity value of the target pixel region Rt in the intensity image data Di. Accordingly, the glare pixel region Rf predicted to cause glare imaging can be appropriately estimated within a range Ef corresponding to the optical characteristics Os around the target pixel region Rt, which can be determined based on the intensity value of the intensity image data Di. Therefore, the distance values ​​of the glare pixel regions Rf that overlap with the target pixel region Rt in the distance image data Dd can be accurately removed, thereby suppressing false detections of distance values ​​in the optical sensor 10.

[0130] According to the first embodiment, during the distance acquisition period Pd, distance image data Dd is acquired along multiple scan lines Ls respectively. On the other hand, during the intensity acquisition period Pi, which precedes the distance acquisition period Pd, intensity image data Di is acquired along each of the aforementioned scan lines Ls respectively. Accordingly, based on the intensity image data Di obtained by synthesizing the amounts of each scan line Ls throughout the intensity acquisition period Pi, distance values ​​that are spurious can be removed from the distance image data Dd defined on the scan lines Ls where glare pixel regions Rf are presumed. Therefore, false detection of distance values ​​in the optical sensor 10 can be suppressed.

[0131] (Second Implementation)

[0132] The second embodiment is a variation of the first embodiment.

[0133] exist Figure 17 In the second embodiment shown, the distance image data Dd obtained by the removal module 2130 synthesizing the amount of each scan line Ls during the distance acquisition module 2100's cross-sectional area Pd during the distance acquisition period is as follows: Figure 18 As shown, the distance value of the glare pixel region Rf estimated by the estimation module 120 is set as the removal target. Therefore, the removal module 2130 compares the distance value of the glare pixel region Rf with the distance value of the target pixel region Rt in the synthesized distance image data Dd. At this time, the distance value of the target pixel region Rt, which is set as, for example, a representative value or an average value, is used as the distance value of the glare pixel region Rf, for example, by comparing the representative value, the average value, or the value of each pixel individually. In addition, in Figure 18 In the image, only the scan lines Ls containing the object, excluding the object, are shown with thick outlines in the intensity image data Di; the scan lines Ls other than these are omitted from the illustration.

[0134] Based on the comparison, if the difference between the distance value of the glare pixel region Rf and the distance value of the target pixel region Rt falls within a specified error range, the removal module 2130 removes the distance value of the glare pixel region Rf. Here, similar to the first embodiment, in the dTOF applicable to the optical sensor 10, the distance value corresponds 1:1 to the echo detection timing. Therefore, the removal of the distance value based on the comparison between regions Rf and Rt, and the removal of the echo detection timing, are essentially the same as the removal of the distance value of the glare pixel region Rf overlapping with the target pixel region Rt.

[0135] In the control flow of such a second embodiment, such as Figure 19 As shown, S2104, S2106, and S2107, which replace S104 to S109 in the first embodiment, are executed. In S2104, the distance acquisition module 2100 acquires distance image data Dd during the distance acquisition period Pd in ​​the detection frame Fd according to each scan line Ls for the illumination light of the first intensity I1, and then synthesizes it throughout the distance acquisition period Pd.

[0136] In the following S2106, the removal module 2130 determines whether the difference between the estimated distance value of the glare pixel region Rf and the distance value of the target pixel region Rt in the synthesized distance image data Dd falls within the error range. As a result, if a positive determination is made, that is, if there is a distance value of the glare pixel region Rf where the echo detection timing overlaps with the target pixel region Rt, the control flow moves to S2107. In S2107, the removal module 2130 removes the spurious distance value from the glare pixel region Rf in the synthesized distance image data Dd, namely the distance value of the glare pixel region Rf where the echo detection timing overlaps with the target pixel region Rt. If the execution of S2107 is completed, or if a negative determination is made in S2106, the current execution of the control flow ends.

[0137] Thus, according to the second embodiment, during the distance acquisition period Pd, distance image data Dd is acquired according to multiple scan lines Ls respectively. On the other hand, during the intensity acquisition period Pi, which precedes the distance acquisition period Pd, intensity image data Di is acquired according to the aforementioned scan lines Ls respectively. Accordingly, in the distance image data Dd synthesized by applying the amount of each scan line Ls throughout the distance acquisition period Pd, distance values ​​that are spurious can be uniformly removed from the glare pixel region Rf estimated based on the intensity image data Di synthesized by applying the amount of each scan line Ls throughout the intensity acquisition period Pi. Therefore, false detection of distance values ​​in the optical sensor 10 can be suppressed.

[0138] (Third Implementation)

[0139] The third embodiment is a variation of the first embodiment.

[0140] Figure 20 The intensity acquisition module 3110 of the third embodiment shown in the detection frame Fd is as follows: Figure 21 As shown, the intensity acquisition period Pi is earlier or later than the intensity acquisition period Di of the intensity image data. Figure 21 (For the example above) A background light acquisition period Pb is set. Therefore, during the background light acquisition period Pb, the intensity acquisition module 3110 controls the optical sensor 10 to stop the application of current and sets each laser diode 24 to a stop mode of non-emitting state.

[0141] like Figure 20 As shown, the intensity acquisition module 3110 acquires two-dimensional background light image data Db, representing the intensity value reflected from the reflective target Tr in the detection region Ad, based on the echo detected by the optical sensor 10 for the background light during non-illumination corresponding to the stop mode. At this time, the intensity of the background light is reduced in the near-infrared region compared to the first intensity I1 of the illumination light. In other words, the first intensity I1 described in the first embodiment is set to be higher than the intensity of the background light in the near-infrared region.

[0142] Along with acquiring the background light image data Db, the intensity acquisition module 3110 also controls the scanning motor 35 to drive the rotation of the scanning mirror 32. Therefore, the intensity acquisition module 3110 generates background light image data Db according to multiple scan lines Ls corresponding to the rotation angle of the scanning mirror 32, and can synthesize the background light image data Db of each scan line Ls throughout the background light acquisition period Pb.

[0143] In the background light image data Db of each scan line Ls obtained in this way, and in the synthesized background light image data Db, the vertical direction corresponds to the Y-axis direction of vehicle 5, and the horizontal direction corresponds to the X-axis direction of vehicle 5. Therefore, the scan line Ls of the background light image data Db is set to be the same as the scan line Ls related to the distance image data Dd and the intensity image data Di, and corresponds to them in a 1:1 ratio.

[0144] In the third embodiment, the estimation module 3120 extracts the intensity value in the target pixel region Rt explored based on the intensity image data Di from the intensity value represented by the background light image data Db. Therefore, the estimation module 3120 corrects the intensity value in the target pixel region Rt of the intensity image data Di by subtracting the intensity value in the target pixel region Rt of the background light image data Db, and then uses it for estimating the glare pixel region Rf.

[0145] The control flow of this third implementation method is as follows: Figure 22The above is shown before or after S101 performed based on the strength acquisition module 3110. Figure 22 (For the previous example) S3100 is executed. In S3100, during the background light acquisition period Pb in the current detection frame Fd, the intensity acquisition module 3110 acquires background light image data Db representing the intensity value of the echo from the reflecting target Tr for the background light when the illumination light is not illuminating. At this time, the background light image data Db of each scan line Ls is synthesized throughout the background light acquisition period Pb.

[0146] In the control flow of the third embodiment, S3103, which replaces S103, is executed. In S3013, the estimation module 3120 estimates the glare pixel region Rf of the distance image data Dd based on the intensity value of the target pixel region Rt in the intensity image data Di after correction using the intensity value of the target pixel region Rt in the background light image data Db.

[0147] Thus, according to the third embodiment, the background light image data Db, representing the intensity value of the echo reflected from the reflective target Tr in the detection area Ad, is obtained based on the echo detected by the optical sensor 10 for the background light in the detection area Ad. Therefore, in the third embodiment, based on the intensity value of the target pixel region Rt in the intensity image data Di, which is corrected using the intensity value of the target pixel region Rt in the background light image data Db, the glare pixel region Rf can be estimated with high accuracy. Accordingly, the distance value of the glare pixel region Rf that overlaps with the detection timing of the echo in the distance image data Dd with the target pixel region Rt can be accurately removed, thereby suppressing false detection of distance values ​​in the optical sensor 10.

[0148] (Fourth Implementation)

[0149] The fourth embodiment is a variation of the third embodiment.

[0150] Figure 23 The strength acquisition module 4110 of the fourth embodiment shown is as follows: Figure 24 As shown, the intensity acquisition period Pi and the background light acquisition period Pb, which are set for each detection frame Fd respectively, are switched according to the brightness of the detection area Ad. Specifically, in dim environments such as nighttime when the average or representative value of the background light intensity is less than or below the switching threshold, the intensity acquisition module 4110 selects to execute the intensity acquisition period Pi by controlling the optical sensor 10 to LED mode to acquire intensity image data Di.

[0151] On the other hand, in bright environments such as daytime when the average or representative value of the background light intensity is above or exceeds the switching threshold, the intensity acquisition module 4110 selects to execute the background light acquisition period Pb by controlling the optical sensor 10 to a stop mode to acquire background light image data Db. At this time, the background light acquisition period Pb is set earlier in the detection frame Fd than the distance acquisition period Pd. Based on this, it can be considered that the background light acquisition period Pb and the background light image data Db are still the intensity acquisition period and intensity image data for the background light.

[0152] Here, the background light intensity, which serves as the switching reference between intensity acquisition period Pi and background light acquisition period Pb, is identified based on the intensity image data Di and distance image data Dd acquired in the previous detection frame Fd, or based on the background light image data Db acquired in the previous detection frame Fd. At this time, the image data of the previous detection frame Fd is stored in memory 1a. Figure 23 The data storage unit 1ad shown is read out along with the identification of the background light intensity. Alternatively, the background light intensity can be identified based on sensor information from vehicle 5, either instead of this identification.

[0153] The estimation module 4120 of the fourth embodiment performs the same processing as the estimation module 120 of the first embodiment in the detection frame Fd during which the intensity acquisition period Pi is set. On the other hand, in the detection frame Fd during which the background light acquisition period Pb is set, the estimation module 4120 explores the target pixel region Rt of the reflective target Tr based on the background light image data Db obtained by synthesizing the amounts of each scan line Ls over the background light acquisition period Pb. Here, the fourth embodiment is based on the premise of suppressing glare imaging not only as much as possible for the illumination light of the second intensity I2 corresponding to the intensity image data Di, but also as much as possible for the background light corresponding to the background light image data Db.

[0154] Therefore, the estimation module 4120, based on the intensity value of the target pixel region Rt in the background light image data Db, estimates the predicted glare pixel region Rf around the target pixel region Rt in the distance image data Dd, as in the estimation module 120 of the first embodiment. At this time, the incident intensity Ii of the echo-directing light-receiving lens system 42 corresponding to the first intensity I1 can be estimated based on, for example, a representative value or an average value related to the intensity value of the target pixel region Rt corresponding to the intensity of the background light.

[0155] The control flow of this fourth implementation method is as follows: Figure 25S4100 is executed before S101, which is based on the intensity acquisition module 4110. In S4100, the intensity acquisition module 4110 switches the period set for the current detection frame Fd to either the intensity acquisition period Pi or the background light acquisition period Pb, corresponding to the background light intensity in the detection region Ad. As a result, if the intensity acquisition period Pi is selected through switching, the control flow moves to S101, thereby executing S101 based on the intensity acquisition module 4110 and S102 and S103 based on the estimation module 4120. On the other hand, if the background light acquisition period Pb is selected through switching, the control flow moves to... Figure 26 S4101 is shown.

[0156] In S4101, during the background light acquisition period Pb in the current detection frame Fd, the intensity acquisition module 4110 acquires background light image data Db representing the intensity value of the echo from the reflecting target Tr. At this time, the background light image data Db of each scan line Ls is synthesized throughout the background light acquisition period Pb.

[0157] In the following S4102, the estimation module 4120 determines whether a target pixel region Rt, representing an intensity value above or exceeding the prediction threshold, exists in the background light image data Db. As a result, if a negative determination is made, the control flow moves to... Figure 25 S110 is shown. On the other hand, as shown... Figure 26 As shown, after a positive determination is made, the control flow moves to S4103.

[0158] In S4103, the estimation module 4120 estimates the glare pixel region Rf at a distance from the image data Dd based on the intensity value in the target pixel region Rt of the background light image data Db. At this time, the glare pixel region Rf is estimated to be a range Ef related to the optical characteristics Os of the light-receiving lens system 42 and the intensity value of the target pixel region Rt in the background light image data Db. Upon completion of S4103, similar to the completion of S103, the control flow moves to... Figure 25 S104 is shown.

[0159] According to this fourth embodiment, distance image data Dd, representing the distance value up to the reflecting target Tr of the reflected light in the detection area Ad, is also obtained based on the echo detected by the optical sensor 10 for the illumination light. Therefore, in the fourth embodiment, background light image data Db is obtained as intensity image data representing the intensity value of the echo reflected from the reflecting target Tr in the detection area Ad, based on the echo detected by the optical sensor 10 for background light with an intensity lower than the illumination light in the detection area Ad. Accordingly, based on the background light image data Db, which suppresses glare imaging according to the low-intensity background light, the glare pixel region Rf that predicts the glare imaging around the target pixel region Rt where the reflecting target Tr is captured can be appropriately estimated in the distance image data Dd. Therefore, the distance value of the glare pixel region Rf, where the detection timing of the echo in the distance image data Dd overlaps with the target pixel region Rt, can be removed as a spurious value caused by glare imaging, thus suppressing false detection of distance values ​​in the optical sensor 10.

[0160] According to the fourth embodiment, the glare pixel region Rf is estimated to be a range Ef related to the optical characteristics Os of the light-receiving lens system 42 that images the echo in the optical sensor 10 and the intensity value of the target pixel region Rt in the intensity image data, i.e., the background light image data Db. Accordingly, the glare pixel region Rf predicted to capture glare can be appropriately estimated to be a range Ef corresponding to the optical characteristics Os around the target pixel region Rt, which can be determined based on the intensity value of the background light image data Db. Therefore, the distance value of the glare pixel region Rf that overlaps with the target pixel region Rt in the distance image data Dd can be accurately removed, thereby suppressing false detection of distance values ​​in the optical sensor 10.

[0161] According to the fourth embodiment, during the distance acquisition period Pd, distance image data Dd is acquired along multiple scan lines Ls respectively. On the other hand, during the background light acquisition period Pb, which is the intensity acquisition period preceding the distance acquisition period Pd, background light image data Db is acquired along each of the aforementioned scan lines Ls respectively. Accordingly, based on the background light image data Db obtained by synthesizing the amounts of each scan line Ls throughout the background light acquisition period Pb, distance values ​​that are spurious can be removed from the distance image data Dd defined on the scan lines Ls where the glare pixel region Rf is presumed. Therefore, false detection of distance values ​​in the optical sensor 10 can be suppressed.

[0162] (Other implementation methods)

[0163] The above describes several embodiments, but this disclosure is not to be construed as being limited to the above embodiments and can be applied to various embodiments and combinations without departing from the spirit of this disclosure.

[0164] In a variation, the dedicated computer constituting the control device 1 may be a computer other than the vehicle 5, which is configured as an external center or mobile terminal capable of communicating with the vehicle 5. In another variation, the dedicated computer constituting the control device 1 may also have at least one of digital circuitry and analog circuitry as a processor. Here, the digital circuitry may be at least one of ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), SOC (System on a Chip), PGA (Programmable Gate Array), and CPLD (Complex Programmable Logic Device). Furthermore, such digital circuitry may have a memory storing programs.

[0165] In the modified optical sensor 10, the scanning of the detection area Ad by the illumination light can be substantially limited to a vertical scan. In this case, the long side of the rectangular outline of the projection window 25 and the light-receiving surface 47 can be defined along the X-axis. Furthermore, in this case, the scan lines Ls associated with each image data Dd, Di can be arranged in multiple columns in the vertical direction corresponding to the Y-axis to serve as horizontal pixel columns corresponding to the X-axis direction. In the modified optical sensor 10, a light emitter 22 for illuminating the first intensity I1 and a light emitter 22 for illuminating the second intensity I2 can also be provided separately. In this case, a light-emitting diode (LED) can be used instead of a laser diode 24 as the light emitter 22 for illuminating the second intensity I2.

[0166] like Figure 27 , 28 As shown, in a modified example, modules 2100, 2130, S2104, S2106, and S2107 of the second embodiment can also be implemented in the third embodiment, replacing modules 100, 130 and S104-S109. For example... Figure 29 , 30 As shown, in a modified example, modules 2100, 2130, S2104, S2106, and S2107 of the second embodiment can also be implemented in the fourth embodiment instead of modules 100, 130 and S104 to S109.

[0167] like Figure 31 , 32As shown, in a modified example, the estimation module 3120 and S3103 of the third embodiment can be implemented instead of module 120 and S103 in the second embodiment. However, in S3103 based on the estimation module 3120 in this case, the glare pixel region Rf of the distance image data Dd can be estimated based on the intensity value of the intensity image data Di obtained by subtracting the amount of background light intensity from the intensity value of the distance image data Dd using a comparison algorithm with the intensity value of the distance image data Dd for the target pixel region Rt. In addition, for this purpose, S3103 based on the estimation module 3120 can be executed after S2104 performed by the distance acquisition module 2100, thereby acquiring the distance image data Dd in a manner that also includes the intensity value of the echo from the reflecting target Tr of the illumination light for the first intensity I1.

[0168] like Figure 33 , 34 As shown, in the modified example, S101 to S103 based on modules 4110 and 4120 may not be executed. However, in this case, if the intensity acquisition period Pi is selected by switching, the execution of this control flow ends. In S103, S3013, and S4103 based on estimation modules 120, 3120, and 4120 in the modified example, the glare pixel region Rf can be estimated as a fixed range Ef around the target pixel region Rt. In this case, pseudo-values ​​can be removed from the glare pixel region Rf within the fixed range Ef where the probability of glare generation is high.

[0169] In a variation, the vehicle to which control device 1 is applied may be, for example, an autonomous vehicle capable of remotely controlling its movement on the road from an external center. Besides the descriptions above, the embodiments and variations described can also be implemented as a semiconductor device (e.g., a semiconductor chip) having at least one processor 1b and one memory 1a.

Claims

1. A control device having a processor and controlling an optical sensor, wherein, The optical sensor detects the echo of the illumination light illuminating the detection area of ​​the vehicle, and the control device is characterized in that... The processor is configured to perform the following processes: Based on the echo detected by the optical sensor for the illumination light of the first intensity, distance image data representing the distance value up to the reflective target that reflects the light in the detection area is acquired; Based on the echo detected by the optical sensor for the illumination light of a second intensity lower than the first intensity, intensity image data representing the intensity value of the echo reflected from the reflecting target in the detection area is acquired; Based on the intensity image data, a glare pixel region for glare imaging is predicted around the target pixel region of the reflected target in the distance image data. as well as The distance value of the glare pixel region that overlaps with the target pixel region is removed from the distance image data.

2. The control device according to claim 1, characterized in that, The process of acquiring the distance image data includes: acquiring the distance image data based on the illumination light of the first intensity generated by the laser diode controlled to oscillate in the optical sensor. The process of acquiring the intensity image data includes: acquiring the intensity image data for the illumination light of the second intensity generated by the laser diode controlled to a non-oscillating state in the optical sensor.

3. The control device according to claim 1 or 2, characterized in that, The processor is configured to further perform the following processing: based on the echo detected by the optical sensor for background light in the detection area, acquire background light image data representing the intensity value of the echo reflected from the reflective target in the detection area. The process of estimating the glare pixel region includes: estimating the glare pixel region based on the intensity value of the target pixel region in the intensity image data after correction using the intensity value of the target pixel region in the background light image data.

4. A control device having a processor and controlling an optical sensor, wherein, The optical sensor detects the echo of the illumination light illuminating the detection area of ​​the vehicle, and the control device is characterized in that... The processor is configured to perform the following processes: Based on the echo detected by the optical sensor for the illumination light, distance image data representing the distance value up to the reflective target that reflects the light in the detection area is obtained; Based on the echo detected by the optical sensor for background light with an intensity lower than that of the illumination light in the detection area, intensity image data representing the intensity value of the echo reflected from the reflective target in the detection area is acquired; Based on the intensity image data, a glare pixel region for glare imaging is predicted around the target pixel region of the reflected target in the distance image data. as well as The distance value of the glare pixel region that overlaps with the target pixel region is removed from the distance image data.

5. The control device according to claim 4, characterized in that, The process of estimating the glare pixel region includes estimating the glare pixel region as a range related to the optical characteristics of the lens system in the optical sensor that images the echo and the intensity value of the target pixel region in the intensity image data.

6. The control device according to claim 4, characterized in that, The process of acquiring the distance image data includes: acquiring the distance image data separately along multiple scan lines during the distance acquisition period. The process of acquiring the intensity image data includes: acquiring the intensity image data according to each of the scan lines during an intensity acquisition period earlier than the distance acquisition period. The process of estimating the glare pixel region includes: estimating the glare pixel region based on the intensity image data synthesized during the intensity acquisition, where the amount of each scan line pervades the intensity. The process of removing the distance value includes: removing the distance value of the glare pixel region from the distance image data of the scan line where the glare pixel region is estimated.

7. The control device according to claim 4, characterized in that, The process of acquiring the distance image data includes: acquiring the distance image data separately for each plurality of scan lines during the distance acquisition period. The process of acquiring the intensity image data includes: acquiring the intensity image data according to each of the scan lines during an intensity acquisition period earlier than the distance acquisition period. The process of estimating the glare pixel region includes: estimating the glare pixel region based on the intensity image data synthesized during the intensity acquisition, where the amount of each scan line pervades the intensity. The process of removing the distance values ​​includes: removing the distance values ​​of the glare pixel regions from the distance image data synthesized during the distance acquisition, where the amount of each of the scan lines pervades the distance acquisition.

8. The control device according to any one of claims 4 to 7, characterized in that, The control device has a storage medium. The processor is configured to further perform the following process: storing the distance image data, after removing the distance values ​​of the glare pixel regions, in the storage medium.

9. A control method, which is executed by a processor for controlling an optical sensor, wherein, The optical sensor detects the echo of the illumination light illuminating the detection area of ​​the vehicle, and the control method is characterized by comprising: Based on the echo detected by the optical sensor for the illumination light of the first intensity, distance image data representing the distance value up to the reflective target that reflects the light in the detection area is acquired; Based on the echo detected by the optical sensor for the illumination light of a second intensity lower than the first intensity, intensity image data representing the intensity value of the echo reflected from the reflecting target in the detection area is acquired; Based on the intensity image data, a glare pixel region is estimated in the distance image data around the target pixel region where the reflective target is captured, and a glare pixel region for glare imaging is predicted; and The distance value of the glare pixel region that overlaps with the target pixel region is removed from the distance image data.

10. A control method, which is executed by a processor for controlling an optical sensor, wherein, The optical sensor detects the echo of the illumination light illuminating the detection area of ​​the vehicle, and the control method is characterized by comprising: Based on the echo detected by the optical sensor for the illumination light, distance image data representing the distance value up to the reflective target that reflects the light in the detection area is obtained; Based on the echo detected by the optical sensor for background light with an intensity lower than that of the illumination light in the detection area, intensity image data representing the intensity value of the echo reflected from the reflective target in the detection area is acquired; Based on the intensity image data, a glare pixel region is estimated in the distance image data around the target pixel region where the reflective target is captured, and a glare pixel region for glare imaging is predicted; and The distance value of the glare pixel region that overlaps with the target pixel region is removed from the distance image data.

11. A storage medium storing a control program containing commands executed by a processor for controlling an optical sensor, wherein, The optical sensor detects the echo of the illumination light illuminating the detection area of ​​the vehicle, and the storage medium is characterized in that... The commands include: Based on the echo detected by the optical sensor for the illumination light of the first intensity, distance image data representing the distance value up to the reflective target that reflects the light in the detection area is acquired; Based on the echo detected by the optical sensor for the illumination light of a second intensity lower than the first intensity, intensity image data representing the intensity value of the echo reflected from the reflecting target in the detection area is acquired; Based on the intensity image data, a glare pixel region is estimated in the distance image data around the target pixel region where the reflective target is captured, and a glare pixel region for glare imaging is predicted; and The distance value of the glare pixel region that overlaps with the target pixel region is removed from the distance image data.

12. A storage medium storing a control program containing commands executed by a processor for controlling an optical sensor, wherein, The optical sensor detects the echo of the illumination light illuminating the detection area of ​​the vehicle, and the storage medium is characterized in that... The commands include: Based on the echo detected by the optical sensor for the illumination light, distance image data representing the distance value up to the reflective target that reflects the light in the detection area is obtained; Based on the echo detected by the optical sensor for background light with an intensity lower than that of the illumination light in the detection area, intensity image data representing the intensity value of the echo reflected from the reflective target in the detection area is acquired; Based on the intensity image data, a glare pixel region is estimated in the distance image data around the target pixel region where the reflective target is captured, and a glare pixel region for glare imaging is predicted; and The distance value of the glare pixel region that overlaps with the target pixel region is removed from the distance image data.

Images