Vehicle control device, vehicle control method, and vehicle control program

Abstract

To provide a vehicle control device for improving adequateness of vehicle control.SOLUTION: A vehicle control device 1 controls a vehicle which mounts: an optical sensor 40 for acquiring a reflection light image Ior by sensing of reflection light with respect to light irradiation; and a sensing camera 41 for acquiring a camera image Ic corresponding to external light intensity in a sensing area overlapped with that of the optical sensor 40. The vehicle control device includes: an extraction section 100 for extracting a mismatching pixel group Po by comparing the reflection light image Ior with the camera image Ic; and a control section 120 for giving a command of control corresponding to a water-related substance Mw to be estimated to correspond to the mismatching pixel group Po to the vehicle.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to a control technique that utilizes a sensor system in a vehicle. [Background technology]

[0002] It is known that the presence of water-related substances, such as rain, fog, or water vapor, in the sensing area of ​​a vehicle's sensor system reduces the sensing accuracy. Therefore, the presence of water-related substances affects the control of the vehicle. In particular, in recent years, the presence of water-related substances in the vehicle's autonomous driving mode has become an important control factor because it determines the continuity of autonomous driving.

[0003] Here, Patent Document 1 discloses a technique for determining whether or not there is dirt, such as water droplets, adhering to an incident surface where light is incident from a sensing area in a sensor system. In this disclosed technique, dirt on the incident surface is determined based on the change in intensity of reflected light in response to light irradiation. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2005-10094 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, the technology disclosed in Patent Document 1 cannot determine the presence of water-related substances unless the substances adhere to the incident surface. Therefore, there is a concern that once water-related substances adhere to the incident surface, it will be difficult to prevent a situation in which the appropriateness of vehicle control deteriorates due to a decrease in the sensing accuracy of the sensor system.

[0006] An object of the present disclosure is to provide a vehicle control device that improves the appropriateness of vehicle control. Another object of the present disclosure is to provide a vehicle control method that improves the appropriateness of vehicle control. Yet another object of the present disclosure is to provide a vehicle control program that improves the appropriateness of vehicle control. [Means for solving the problem]

[0007] The technical means of the present disclosure for solving the problems will be described below. Note that the claims and the reference characters in parentheses in this section indicate the corresponding relationship with the specific means described in the embodiments described later in detail, and do not limit the technical scope of the present disclosure.

[0008] A first aspect of the present disclosure is A vehicle control device (1) for controlling a vehicle (2) equipped with an optical sensor (40) for acquiring a reflected light image (Ior) by sensing reflected light in response to light irradiation, and a sensing camera (41) for acquiring a camera image (Ic) corresponding to an external light intensity in a sensing area (Ac) overlapping with the optical sensor, An extraction unit (100, 5100) that extracts unmatched pixel groups (Po) by comparing a reflected light image with a camera image; The vehicle is provided with a control unit (120, 2120, 5120) that commands control according to a water-related substance (Mw) estimated to correspond to an unmatched pixel group.

[0009] A second aspect of the present disclosure is A vehicle control method for controlling a vehicle (2) equipped with an optical sensor (40) that obtains a reflected light image (Ior) by sensing reflected light in response to light irradiation, and a sensing camera (41) that obtains a camera image (Ic) corresponding to an external light intensity in a sensing area (Ac) that overlaps with the optical sensor, comprising: An extraction step (S101, S102, S103, S500, S501, S502, S503, S506) of extracting unmatched pixel groups (Po) by comparing the reflected light image with the camera image; The method includes control steps (S104, S105, S106, S205, S206, S504, S505, S507, S508) of instructing the vehicle to perform control according to the water-related substance (Mw) estimated to correspond to the unmatched pixel group.

[0010] A third aspect of the present disclosure is A vehicle control program including instructions to cause a processor (12) to execute a control program for controlling a vehicle (2) equipped with an optical sensor (40) that obtains a reflected light image (Ior) by sensing reflected light in response to light irradiation, and a sensing camera (41) that obtains a camera image (Ic) corresponding to an external light intensity in a sensing area (Ac) that overlaps with the optical sensor, the control program including instructions to cause a processor (12) to execute the control program for controlling a vehicle (2) equipped with an optical sensor (40) that obtains a reflected light image (Ior) by sensing reflected light in response to light irradiation ... the control program including instructions to cause a processor (12) to execute the control program for controlling a vehicle (2) equipped with an optical sensor (40) that obtains a reflected light image (Ior) by sensing reflected light in response to light irradiation, the control program including instructions to cause a processor (12) to execute the control program for controlling a vehicle (2) equipped with an optical sensor (40) that obtains a reflected light image (Ior) by sensing reflected light in response to light irradiation, the control program including instructions to cause a processor (12) to execute the control program for controlling a vehicle (2) equipped with an optical sensor (40) that obtains a reflected light image (Ior) The command is, An extraction step (S101, S102, S103, S500, S501, S502, S503, S506) of extracting unmatched pixel groups by comparing the reflected light image with the camera image; The method includes control steps (S104, S105, S106, S205, S206, S504, S505, S507, S508) of instructing the vehicle to perform control according to the water-related substance (Mw) estimated to correspond to the unmatched pixel group.

[0011] According to the first to third aspects, a reflected light image acquired by an optical sensor by sensing reflected light from light irradiation is compared with a camera image acquired by a sensing camera according to the external light intensity in a sensing area overlapping with the optical sensor. The unmatched pixel group extracted by such a comparison can be accurately estimated to correspond to a water-related substance that is easily sensed by reflected light, i.e., a water-related substance that is easily captured in a reflected light image. Therefore, if a control corresponding to the water-related substance is instructed to the vehicle as a result of the estimation, it is possible to improve the suitability of the vehicle control.

[0012] A fourth aspect of the present disclosure is A vehicle control device (1) controls a vehicle (2) equipped with an optical sensor (3040) that acquires a reflected light image (Ior) by sensing reflected light in response to light irradiation and acquires an external light image (Ioo) according to an external light intensity while light irradiation is stopped, An extraction unit (3100) that extracts unmatched pixels (Po) by comparing a reflected light image with an external light image; The vehicle is provided with a control unit (120, 4120) that commands control according to a water-related substance (Mw) estimated to correspond to an unmatched pixel group.

[0013] A fifth aspect of the present disclosure is a method for manufacturing a semiconductor device comprising: A vehicle control method for controlling a vehicle (2) equipped with an optical sensor (3040) that acquires a reflected light image (Ior) by sensing reflected light in response to light irradiation and acquires an external light image (Ioo) according to an external light intensity while light irradiation is stopped, comprising: An extraction step (S302, S303) of extracting an unmatched pixel group (Po) by comparing the reflected light image and the external light image; The method includes control steps (S104, S105, S106, S405, S406) of instructing the vehicle to perform control in accordance with the water-related substance (Mw) estimated to correspond to the unmatched pixel group.

[0014] A sixth aspect of the present disclosure is A vehicle control program including instructions to cause a processor (12) to execute an optical sensor (3040) for controlling a vehicle (2) equipped with the optical sensor (3040) that acquires a reflected light image (Ior) by sensing reflected light in response to light irradiation and acquires an external light image (Ioo) according to an external light intensity while light irradiation is stopped, the vehicle control program including instructions to cause a processor (12) to execute the optical sensor (3040) The command is, An extraction step (S302, S303) of extracting unmatched pixel groups by comparing the reflected light image and the external light image; The method includes control steps (S104, S105, S106, S405, S406) of instructing the vehicle to perform control according to the water-related substance (Mw) estimated to correspond to the unmatched pixel group.

[0015] According to the fourth to sixth aspects, a reflected light image acquired by an optical sensor by sensing reflected light from light irradiation is compared with an external light image acquired by the same sensor according to the external light intensity while the light irradiation is stopped. The unmatched pixel group extracted by such a comparison can be accurately estimated to correspond to water-related substances that are easily sensed by reflected light, i.e., water-related substances that are easily captured in a reflected light image. Therefore, if a control corresponding to the water-related substances is instructed to the vehicle as a result of the estimation, it is possible to improve the suitability of the vehicle control. [Brief description of the drawings]

[0016] [Figure 1] 1 is a side view showing the state in which the automatic driving unit according to the first embodiment is mounted on a vehicle. FIG. [Diagram 2] 1 is a schematic cross-sectional view showing the overall configuration of an automatic driving unit according to a first embodiment. FIG. [Diagram 3] 1 is a block diagram showing a functional configuration of a vehicle control device according to a first embodiment; [Figure 4] 3 is a schematic cross-sectional view showing a sensing range of an optical sensor and a sensing camera according to the first embodiment; FIG. [Diagram 5] 3A to 3C are schematic diagrams for explaining an image acquired by an optical sensor according to the first embodiment. [Figure 6] 3A to 3C are schematic diagrams for explaining an image acquired by a sensing camera according to the first embodiment. [Figure 7] 4A to 4C are schematic diagrams for explaining an interpolated image of the sensing camera according to the first embodiment. [Figure 8] 3 is a flowchart showing a vehicle control method according to the first embodiment. [Figure 9] FIG. 11 is a block diagram showing a functional configuration of a vehicle control device according to a second embodiment. [Figure 10] 10 is a flowchart showing a vehicle control method according to a second embodiment. [Figure 11] FIG. 11 is a block diagram showing a functional configuration of a vehicle control device according to a third embodiment. [Figure 12]13A and 13B are schematic diagrams for explaining an image acquired by an optical sensor according to a third embodiment. [Figure 13] 10 is a flowchart showing a vehicle control method according to a third embodiment. [Figure 14] FIG. 13 is a block diagram showing a functional configuration of a vehicle control device according to a fourth embodiment. [Figure 15] 13 is a flowchart showing a vehicle control method according to a fourth embodiment. [Figure 16] FIG. 13 is a schematic diagram showing a detailed configuration of a sensing camera according to a fifth embodiment. [Figure 17] FIG. 13 is a block diagram showing a functional configuration of a vehicle control device according to a fifth embodiment. [Figure 18] 13 is a flowchart showing a vehicle control method according to a fifth embodiment. [Figure 19] 10 is a flowchart showing a vehicle control method according to a modified example. [Figure 20] 10 is a flowchart showing a vehicle control method according to a modified example. [Figure 21] 10 is a flowchart showing a vehicle control method according to a modified example. [Figure 22] 10 is a flowchart showing a vehicle control method according to a modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Hereinafter, a number of embodiments will be described with reference to the drawings. In addition, by assigning the same reference numerals to corresponding components in each embodiment, duplicated descriptions may be omitted. In addition, when only a part of the configuration is described in each embodiment, the configuration of the other embodiment described above can be applied to the other parts of the configuration. Furthermore, in addition to the combination of configurations explicitly stated in the description of each embodiment, configurations of multiple embodiments can be partially combined together even if not explicitly stated, as long as there is no particular problem with the combination.

[0018] First embodiment As shown in Fig. 1, an autonomous driving unit ADU including a vehicle control device 1 of the first embodiment is mounted on a vehicle 2. The vehicle 2 is capable of steady or temporary autonomous driving in an autonomous driving mode by autonomous driving control or advanced driving assistance control. The autonomous driving unit ADU includes a housing 3, a sensor system 4, and a cleaning system 5 together with the vehicle control device 1. In the following description, the terms front, rear, left, right, up, and down are defined with reference to the vehicle 2 on a horizontal plane.

[0019] 1 and 2, the housing 3 is formed into a hollow flat box shape made of, for example, metal. The housing 3 is installed on the roof 20 of the vehicle 2. A plurality of sensor windows 31 are opened in a wall 30 of the housing 3. Each sensor window 31 is covered by a plate-shaped translucent cover 32. The outer surface of each translucent cover 32 constitutes an entrance surface 33 into which light enters from the outside of the vehicle 2.

[0020] 2 to 4, the sensor system 4 includes an optical sensor 40. The optical sensor 40 is a so-called LiDAR (Light Detection and Ranging / Laser Imaging Detection and Ranging) that acquires optical information that can be used in an autonomous driving mode in the vehicle 2. The optical sensor 40 includes a light emitting element 400, an image sensor 401, and an image sensor circuit 402.

[0021] The light emitting element 400 is a semiconductor element that emits directional laser light, such as a laser diode. The light emitting element 400 irradiates laser light toward the outside of the vehicle 2 in the form of an intermittent pulse beam. The imaging element 401 is a semiconductor element that is highly sensitive to light, such as a SPAD (Single Photon Avalanche Diode). An incident surface 33o dedicated to the optical sensor 40 is arranged on the outside side (front side in this embodiment) of the imaging element 401. The imaging element 401 is exposed to light that is incident on the incident surface 33o from a sensing area Ao determined by the angle of view of the imaging element 401 out of the outside world. The imaging circuit 402 is an integrated circuit that controls exposure and scanning of multiple pixels in the imaging element 401 and processes signals from the imaging element 401 to convert them into data.

[0022] In a reflected light mode in which the imaging circuit 402 exposes the imaging element 401 to light irradiation from the light emitting element 400, an object point within the sensing area Ao becomes a reflection point of the laser light. As a result, the laser light reflected at the reflection point (hereinafter referred to as reflected light) is incident on the imaging element 401 through the incident surface 33o. At this time, the imaging circuit 402 senses the reflected light by scanning multiple pixels of the imaging element 401. In particular, the imaging circuit 402 converts the distance values ​​acquired for multiple pixels according to the reflection point distance dr of the sensed reflected light as shown in FIG. 4 into three-dimensional data as each pixel value, thereby acquiring a distance image Iord as a reflected light image Ior as shown in FIG.

[0023] 2 to 4, the sensor system 4 includes a sensing camera 41 in addition to the optical sensor 40. The sensing camera 41 is a so-called external camera that acquires optical information that can be used in the autonomous driving mode of the vehicle 2. The sensing camera 41 includes an imaging element 411 and an imaging circuit 402.

[0024] The imaging element 411 is a semiconductor element such as a CMOS. An incident surface 33c dedicated to the sensing camera 41 is disposed on the external side (front side in this embodiment) of the imaging element 411. The imaging element 411 is exposed to light incident on the incident surface 33c from a sensing area Ac determined by the angle of view of the imaging element 411 in the external world. Here, as shown in FIG. 4, the sensing area Ac of the sensing camera 41 partially overlaps with the sensing area Ao of the optical sensor 40. The overlapping rate of the sensing areas Ac and Ao, that is, the proportion of the overlapping area Aoc in each of the areas Ac and Ao, is, for example, 50% or more, preferably 70% or more, and more preferably 90% or more. The imaging circuit 412 is an integrated circuit that controls exposure and scanning of a plurality of pixels in the imaging element 411 and processes signals from the imaging element 411 to convert them into data.

[0025] In an exposure mode in which the imaging circuit 412 exposes the imaging element 411 to light, an object point in the sensing area Ac becomes a reflection point of external light. As a result, the external light reflected at the reflection point is incident on the imaging element 411 through the incident surface 33c. At this time, the imaging circuit 412 senses the reflected external light by scanning a plurality of pixels of the imaging element 411. Here, in particular, the imaging circuit 412 converts the luminance value acquired for each of the plurality of pixels according to the intensity of the sensed external light into two-dimensional data as each pixel value, thereby acquiring a camera image Ic as shown in FIG. 6.

[0026] The cleaning system 5 shown in FIGS. 1 to 3 cleans a plurality of incident surfaces 33 including the incident surfaces 33o and 33c on which light is incident from the sensing areas Ao and Ac in the optical sensor 40 and the sensing camera 41. The cleaning system 5 includes a cleaning module 50 for each incident surface 33. Each cleaning module 50 may have a cleaning nozzle that sprays a cleaning gas such as air as a cleaning fluid for cleaning the incident surface 33 onto the incident surface 33. Each cleaning module 50 may have a cleaning nozzle that sprays a cleaning liquid onto the incident surface 33 as a cleaning fluid for cleaning the incident surface 33. Each cleaning module 50 may have a cleaning wiper that cleans the incident surface 33 by wiping it.

[0027] The vehicle control device 1 shown in FIGS. 2 and 3 is connected to electrical components 4 and 5 of the autonomous driving unit ADU via at least one of a LAN (Local Area Network), a wire harness, and an internal bus. The vehicle control device 1 includes at least one dedicated computer. The dedicated computer constituting the vehicle control device 1 may be a driving control ECU that controls the autonomous driving mode in cooperation with an ECU (Electronic Control Unit) in the vehicle 2. The dedicated computer constituting the vehicle control device 1 may be an actuator ECU that individually controls the driving actuators of the vehicle 2. The dedicated computer constituting the vehicle control device 1 may be a locator ECU that estimates the state quantity of the vehicle 2 including its own position. The dedicated computer constituting the vehicle control device 1 may be a navigation ECU that navigates the driving route of the vehicle 2. The dedicated computer constituting the vehicle control device 1 may be an HCU (Human Machine Interface (HMI) Control Unit) that controls the information presentation of an information presentation system of the vehicle 2.

[0028] The vehicle control device 1 is configured to include such a dedicated computer, and has at least one memory 10 and one processor 12. The memory 10 is at least one type of non-transitory tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium, that non-temporarily stores computer-readable programs and data. The processor 12 includes at least one type of core, such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a RISC (Reduced Instruction Set Computer)-CPU.

[0029] The processor 12 executes a plurality of instructions included in the vehicle control program stored in the memory 10. In this way, the vehicle control device 1 constructs a plurality of functional units (i.e., functional blocks) for controlling the vehicle 2. In this manner, in the vehicle control device 1, the vehicle control program stored in the memory 10 causes the processor 12 to execute a plurality of instructions to control the vehicle 2, thereby constructing a plurality of functional units. The plurality of functional units constructed by the vehicle control device 1 include an extraction unit 100 and a control unit 120, as shown in FIG. 3.

[0030] For each control cycle, the extraction unit 100 receives a distance image Iord as a reflected light image Ior from the optical sensor 40, and also receives a camera image Ic from the sensing camera 41. The extraction unit 100 compares the distance image Iord with the camera image Ic to extract unmatched pixel groups Po. To this end, the extraction unit 100 has a pixel interpolation unit 102, an edge detection unit 104, and a matching determination unit 106 as sub-functional units.

[0031] The pixel interpolation unit 102 performs pixel interpolation between the distance image Iord as the reflected light image Ior and the camera image Ic, adjusting the higher resolution side to the lower resolution side. In this embodiment, the number of pixel groups capturing the overlapping area Aoc of the sensing areas Ao, Ac is set to be greater in the camera image Ic than in the distance image Iord. That is, the camera image Ic is on the higher resolution side, while the distance image Iord is on the lower resolution side. Therefore, the pixel interpolation unit 102 performs internal division interpolation of the pixel values ​​of each pixel coordinate in the camera image Ic to match each pixel coordinate in the distance image Iord. As a result of the interpolation, the pixel interpolation unit 102 generates a camera image Ic with substantially the same resolution as the distance image Iord for the overlapping area Aoc, as shown in FIG. 7.

[0032] The edge detection unit 104 detects edges from the pixel-interpolated camera image Ic by applying at least one type of edge filter processing to the image Ic. The edge detection unit 104 also detects edges from the image Iord, which is a reflected light image Ior of the overlapping area Aoc that has been given substantially the same resolution as the camera image Ic by applying at least one type of edge filter processing to the image Iord.

[0033] The matching determination unit 106 compares the edges detected in the distance image Iord as the reflected light image Ior and the camera image Ic to determine the matching state of the images Iord and Ic. At this time, the matching determination unit 106 determines the unmatching state when, for example, a pixel group in which the difference is outside the matching range is confirmed by normalizing the pixel values ​​constituting the edge to each other and calculating the difference. Here, in particular, in the first embodiment, the determination of unmatching due to a water-related substance Mw (described in detail later) that exists in the distance image Iord but does not exist in the camera image Ic is made based on at least one of, for example, a comparison between the past and present of the images Iord and Ic, and object recognition. This is to avoid a determination of unmatching due to a specular reflection object, such as a black car body, other than the water-related substance Mw that exists in the camera image Ic but does not exist in the distance image Iord. Through such a determination, the matching determination unit 106 extracts pixel groups Po in the distance image Iord that correspond to edges that are unmatched with the camera image Ic, as shown in Fig. 5. Note that a pixel group refers to a group of multiple pixels, and in particular, when a pixel group Po is determined to be unmatched, it is preferable that the pixel group Po is a group of multiple adjacent, continuous pixels. For ease of explanation, the unmatched pixel group Po is shown in Fig. 5 as a thick solid circle with the symbol Po attached.

[0034] As shown in Fig. 3, the control unit 120 receives a pixel group Po extracted by the unmatching determination in the matching determination unit 106 in the distance image Iord serving as the reflected light image Ior. As shown in Fig. 5, the control unit 120 estimates that such an unmatched pixel group Po corresponds to the presence of a water-related substance Mw that is easy to sense in the reflected light of an intense laser beam. Here, the water-related substance Mw is at least one of, for example, rain (including water droplets), fog, water vapor (including components in exhaust gas), and water-containing sludge. Note that the example in Fig. 5 shows the presence of fog as the water-related substance Mw in an easy-to-understand schematic manner for the sake of convenience.

[0035] The control unit 120 also receives the value of the reflection point distance dr, which is the pixel value in the unmatched pixel group Po of the distance image Iord, from the matching determination unit 106. The control unit 120 then determines whether the reflection point distance dr in the unmatched pixel group Po is long or short by comparing it with a predetermined set range. Here, the set range is defined as a range less than a threshold value for the reflection point distance dr or equal to or less than the threshold value. The threshold value defining the set range is set to an appropriate value, such as 1 to 3 m when stopped and 3 to 10 m when moving. Of course, the threshold value defining the set range may be set to an appropriate value such as an example value. Under such definition and setting, the control unit 120 predicts that the water-related substance Mw, which is estimated to correspond to the unmatched pixel group Po, is likely to adhere to the incident surface 33o of the optical sensor 40 when the reflection point distance dr is within the set range. The water-related substance Mw predicted to adhere to the incident surface 33o in this way is assumed to be rain, fog, water vapor, sludge, etc., as the above-mentioned example substances.

[0036] For this reason, when it is determined that the water-related substance Mw estimated to correspond to the unmatched pixel group Po is predicted to adhere to the incident surface 33o of the optical sensor 40, the control unit 120 commands the cleaning system 5 of the vehicle 2 in the autonomous driving mode to perform cleaning control so as to clean the incident surface 33o of the optical sensor 40. Upon receiving the command, the cleaning system 5 performs cleaning processing of the incident surface 33o by controlling the driving of the cleaning module 50o corresponding to the incident surface 33o as shown in FIG. 2. At this time, the command for cleaning control regarding the incident surface 33c of the sensing camera 41 is not essential, and therefore the cleaning module 50c is not an essential element. However, when the adhesion prediction of the water-related substance Mw is established for the incident surface 33o of the optical sensor 40, the adhesion prediction is also established for the incident surface 33c of the sensing camera 41 arranged as close as possible to the optical sensor 40, and cleaning control regarding the same surface 33c may be commanded.

[0037] The flow of the vehicle control method in which the vehicle control device 1 controls the vehicle 2 through cooperation of the functional units 100 and 120 will be described below with reference to Fig. 8. Note that each "S" in this flow represents a plurality of steps executed by a plurality of commands included in the vehicle control program.

[0038] First, in S101, pixel interpolation unit 102 of extraction unit 100 performs pixel interpolation on high-resolution camera image Ic to match it with distance image Iord serving as reflected light image Ior on the low-resolution side. Next, in S102, edge detection unit 104 of extraction unit 100 detects edges from each of distance image Iord and camera image Ic. Then, in S103, matching determination unit 106 of extraction unit 100 determines the matching state of images Iord, Ic by comparing the edges detected in distance image Iord and camera image Ic.

[0039] As a result of S103, if a pixel group Po determined to be unmatched from the camera image Ic is extracted in the distance image Iord as the reflected light image Ior, the flow proceeds to S104. In S104, the control unit 120 determines whether or not the water-related substance Mw estimated to correspond to the unmatched pixel group Po is predicted to adhere to the incident surface 33o of the optical sensor 40, based on the reflection point distance dr of the unmatched pixel group Po.

[0040] As a result of S104, if the reflection point distance dr of the unmatched pixel group Po is within the set range and adhesion of the water-related substance Mw to the incident surface 33o is predicted, the flow proceeds to S105. In S105, the control unit 120 commands the cleaning system 5 to execute cleaning control on the incident surface 33o of the optical sensor 40 as control of the vehicle 2 in the autonomous driving mode.

[0041] Now, in the case of perfect matching in which the result of S103 is that no pixel group Po determined to be unmatched is extracted, or in the case of not predicting adhesion of the water-related substance Mw to the incident surface 33o in the result of S104, the flow proceeds to S106. In S106, the control unit 120 commands the cleaning system 5 to stop cleaning control. Thus, in the first embodiment, S101, S102, and S103 correspond to the extraction step, and S104, S105, and S106 correspond to the control step.

[0042] (Action and effect) The effects of the first embodiment described above will be described below.

[0043] According to the first embodiment, a reflected light image Ior acquired by the optical sensor 40 by sensing reflected light in response to light irradiation is compared with a camera image Ic acquired by the sensing camera 41 according to the external light intensity in a sensing area Ac overlapping with the optical sensor 40. The unmatched pixel group Po extracted by such a comparison can be accurately estimated to correspond to a water-related substance Mw whose reflected light is easily sensed, that is, a water-related substance Mw that is easily captured in the reflected light image Ior. Therefore, by commanding the vehicle 2 to control according to the water-related substance Mw as a result of the estimation, it is possible to improve the suitability of the vehicle control.

[0044] According to the first embodiment, the cleaning system 5 is mounted on the vehicle 2 so as to clean the incident surface 33o on which light is incident from the sensing area Ao of the optical sensor 40. With this, the unmatched pixel group Po extracted from the comparison of the reflected light image Ior and the camera image Ic acquired by the optical sensor 40 and the sensing camera 41, respectively, can be accurately estimated to correspond to the water-related substance Mw. Therefore, for the incident surface 33o predicted to have a high possibility of adhesion of the water-related substance Mw as a result of the estimation, cleaning control is commanded to the cleaning system 5 of the vehicle 2 in the autonomous driving mode, so that the appropriateness of the cleaning control can be improved.

[0045] According to the first embodiment, the distance image Iord is compared with the camera image Ic as a reflected light image Ior corresponding to the reflection point distance dr of the reflected light relative to the light irradiation. The unmatched pixel group Po extracted by such a comparison can be accurately estimated to correspond to a water-related substance Mw whose reflection point distance dr is easily sensed, that is, a water-related substance Mw that is easily captured in the distance image Iord. Therefore, it is possible to increase the appropriateness of the cleaning control by issuing a command for cleaning control corresponding to the water-related substance Mw as a result of the estimation.

[0046] According to the first embodiment, among the unmatched pixel groups Po, in particular, the pixel group Po whose reflection point distance dr in the distance image Iord as the reflected light image Ior is within a set range can be accurately estimated to correspond to the water-related substance Mw located within the set range and predicted to be likely to adhere to the incident surface 33o of the optical sensor 40. Therefore, it becomes possible to appropriately command and control the cleaning of the incident surface 33o in a situation where there is a high possibility of the water-related substance Mw adhering.

[0047] By comparing edges between the reflected light image Ior and the camera image Ic as in the first embodiment, it becomes easy to accurately capture and extract the unmatched pixel group Po from those edges, and therefore it becomes possible to appropriately command and execute control according to the water-related substance Mw that is estimated to correspond to the unmatched pixel group Po.

[0048] According to the first embodiment, the reflected light image Ior and the camera image Ic are compared after the higher resolution side of the image is interpolated to match the lower resolution side. This comparison makes it possible to prevent a situation in which a pixel group that should be matched is erroneously extracted due to a difference in resolution. Therefore, it becomes possible to appropriately command and execute control according to the water-related substance Mw that is estimated to correspond to the unmatched pixel group Po.

[0049] Second Embodiment The second embodiment is a modification of the first embodiment.

[0050] As shown in Fig. 9, a driving control device 6 is installed in the vehicle 2 of the second embodiment. The driving control device 6 cooperates with the automatic driving unit ADU to execute driving control of the vehicle 2, including the automatic mode, based on various sensing information by the sensor system 4. For this purpose, the driving control device 6 is configured to include at least one dedicated computer, such as the ECU in the vehicle 2 described in the first embodiment. Note that the cleaning system 5 of the second embodiment is not essential, and the illustration of the system 5 is omitted in Fig. 9.

[0051] In the control unit 2120 of the second embodiment, the reflection point distance dr in the unmatched pixel group Po is not input, and the determination of whether the distance dr is long or short is omitted. Therefore, the control unit 2120 commands the driving control device 6 of the vehicle 2 to perform driving control in the autonomous driving mode so as to exclude the unmatched pixel group Po estimated to correspond to the water-related substance Mw from being recognized as an obstacle.

[0052] In the flow of the vehicle control method according to the second embodiment, as shown in Fig. 10, S101, S102, and S103 are executed according to the first embodiment, while S205 and S206 are executed instead of S104, S105, and S106. Specifically, in S205, the control unit 2120 commands the driving control device 6 of the vehicle 2 to perform driving control in the autonomous driving mode so as to remove the water-related substance Mw from the recognition as an obstacle. In response to this, in S206, the control unit 2120 commands the driving control device 6 to stop the driving control that removes the water-related substance Mw from the obstacle recognition. Thus, in the second embodiment, S101, S102, and S103 correspond to the extraction process, and S205 and S206 correspond to the control process.

[0053] (Action and effect) The effects of the second embodiment described above will be described below.

[0054] According to the second embodiment, the vehicle 2 is commanded to perform driving control in the autonomous driving mode so as to exclude the water-related substance Mw estimated to correspond to the unmatched pixel group Po extracted from the comparison between the reflected light image Ior and the camera image Ic from being recognized as an obstacle. This makes it possible to ensure the continuity of the autonomous driving mode by appropriate driving control that prevents the water-related substance Mw from being erroneously recognized as an obstacle.

[0055] Third embodiment The third embodiment is a modification of the first embodiment.

[0056] As shown in Fig. 11, the optical sensor 3040 of the third embodiment has a function of acquiring a reflected light image Ior during light irradiation according to the first embodiment, as well as a function of acquiring an image while the light irradiation is stopped. Specifically, in an external light mode in which the imaging circuit 3402 exposes the imaging element 401 while the intermittent light irradiation from the light emitting element 400 is stopped, an object point in the sensing area Ao becomes a reflection point of the external light. As a result, the external light reflected at the reflection point is incident on the imaging element 401 through the incident surface 33o. At this time, the imaging circuit 3402 senses the reflected external light by scanning a plurality of pixels of the imaging element 401.

[0057] Here, the imaging circuit 3402 converts the luminance values ​​acquired for each of a plurality of pixels according to the intensity of the sensed external light into two-dimensional data as pixel values ​​to acquire an external light image Ioo as shown in Fig. 12. At this time, the sensing of external light in the function of acquiring the external light image Ioo is realized by the same imaging element 401 as the sensing of reflected light in the function of acquiring the reflected light image Ior. Note that the sensing camera 41 in the third embodiment is not essential, and is not shown in Fig. 11.

[0058] In the third embodiment, an extraction unit 3100 sequentially receives a distance image Iord as a reflected light image Ior and an external light image Ioo from the optical sensor 3040 for each control cycle. In the extraction unit 3100, the pixel interpolation unit 102 as a sub-functional unit is omitted due to sensing by the same element 401 described above. Note that the extraction unit 3100 realizes the respective functions of the edge detection unit 104 and the matching determination unit 106 between the external light image Ioo, which replaces the camera image Ic, and the reflected light image Ior in the same manner as in the first embodiment.

[0059] 13, in the flow of the vehicle control method according to the third embodiment, first, in S302, the edge detection unit 104 of the extraction unit 3100 detects edges from each of the distance image Iord as the reflected light image Ior and the external light image Ioo. Next, in S303, the matching determination unit 106 of the extraction unit 3100 determines the matching state between the images Iord and Ioo by comparing the edges detected in the distance image Iord and the external light image Ioo.

[0060] Incidentally, S104, S105, and S106 after S303 are executed in accordance with the first embodiment. As described above, in the third embodiment, S302 and S303 correspond to the extraction step, and S104, S105, and S106 correspond to the control step.

[0061] (Action and effect) The effects of the third embodiment described above will be described below.

[0062] According to the third embodiment, a reflected light image Ior acquired by the optical sensor 3040 by sensing reflected light from light irradiation is compared with an external light image Ioo acquired by the optical sensor 3040 according to the external light intensity while the light irradiation is stopped. The unmatched pixel group Po extracted by such a comparison can be accurately estimated to correspond to a water-related substance Mw for which reflected light is easily sensed, that is, a water-related substance Mw that is easily captured in the reflected light image Ior. Therefore, by instructing the vehicle 2 to control according to the water-related substance Mw as a result of the estimation, it is possible to improve the suitability of the vehicle control.

[0063] According to the third embodiment, the cleaning system 5 is mounted on the vehicle 2 so as to clean the incident surface 33o on which light is incident from the sensing area Ao of the optical sensor 3040. With this, the unmatched pixel group Po extracted from the comparison between the reflected light image Ior and the external light image Ioo acquired by the optical sensor 3040 can be accurately estimated to correspond to the water-related substance Mw. Therefore, for the incident surface 33o on which the possibility of the water-related substance Mw adhering is predicted as a result of the estimation, cleaning control is commanded to the cleaning system 5 of the vehicle 2 in the autonomous driving mode, so that the suitability of the cleaning control can be improved.

[0064] According to the third embodiment, the distance image Iord is compared with the external light image Ioo as a reflected light image Ior corresponding to the reflection point distance dr of the reflected light relative to the light irradiation. The unmatched pixel group Po extracted by such a comparison can be accurately estimated to correspond to a water-related substance Mw whose reflection point distance dr is easily sensed, that is, a water-related substance Mw that is easily captured in the distance image Iord. Therefore, it is possible to increase the appropriateness of the cleaning control by issuing a command for cleaning control corresponding to the water-related substance Mw as a result of the estimation.

[0065] According to the third embodiment, among the unmatched pixel groups Po, in particular, the pixel group Po whose reflection point distance dr in the distance image Iord as the reflected light image Ior is within a set range can be accurately estimated to correspond to the water-related substance Mw that is located within the set range and is predicted to be likely to adhere to the incident surface 33o of the optical sensor 3040. Therefore, it becomes possible to appropriately command and control the cleaning of the incident surface 33c in a situation where there is a high possibility of the water-related substance Mw adhering.

[0066] According to the third embodiment, it is possible to prevent an error in determining whether the reflection point distance dr sensed by the same element 401 as the external light intensity in the optical sensor 3040 is inside or outside the set range due to an axial misalignment between the sensing. Therefore, it is possible to properly command and control the cleaning of the entrance surface 33o to which the water-related substance Mw is expected to adhere in correspondence with the pixel group Po in which the reflection point distance dr falls within the set range.

[0067] (Fourth embodiment) The fourth embodiment is a modification of the third embodiment.

[0068] As shown in Fig. 14, a driving control device 6 similar to that of the second embodiment is mounted in the vehicle 2 of the fourth embodiment. In addition, a control unit 4120 of the fourth embodiment has a driving control command function similar to that of the second embodiment. The washing system 5 of the fourth embodiment is not essential, and is not shown in Fig. 14.

[0069] In the flow of the vehicle control method according to the fourth embodiment, as shown in Fig. 15, S302 and S303 are executed in accordance with the third embodiment, while S405 and S406 corresponding to S205 and S206 of the second embodiment are executed instead of S104, S105, and S106. That is, in S405, the control unit 4120 commands the driving control device 6 of the vehicle 2 to perform driving control in the automatic driving mode so as to remove the water-related substance Mw from the recognition as an obstacle. In response to this, in S406, the control unit 4120 commands the driving control device 6 to stop the driving control that removes the water-related substance Mw from the obstacle recognition. Thus, in the fourth embodiment, S302 and S303 correspond to the extraction process, and S405 and S406 correspond to the control process.

[0070] (Action and effect) The effects of the fourth embodiment described above will be described below.

[0071] According to the fourth embodiment, the vehicle 2 is commanded to perform driving control in the autonomous driving mode so as to exclude the water-related substance Mw estimated to correspond to the unmatched pixel group Po extracted from the comparison between the reflected light image Ior and the external light image Ioo from being recognized as an obstacle. This makes it possible to ensure the continuity of the autonomous driving mode by appropriate driving control that prevents the water-related substance Mw from being erroneously recognized as an obstacle.

[0072] Fifth embodiment The fifth embodiment is a modified example of the first embodiment. In the following description of the fifth embodiment, the incident surface 33c of the sensing camera 41 is referred to as a camera incident surface 33c, and the incident surface 33o of the optical sensor 40 is referred to as an optical incident surface 33o.

[0073] 16, in the sensing camera 41 of the first embodiment applied to the fifth embodiment, the camera incident surface 33c into which light is incident from the sensing area Ac overlapping with the optical sensor 40 is disposed on the inner side on the optical axis L relative to the outer-side focal position pf of the imaging lens system 413 that forms an image of the incident light on the image sensor 411. That is, the camera incident surface 33c is located within a focal length df on the sensing area Ac side of the imaging lens system 413. Here, within the focal length df, there is a high possibility that the edge of the water-related substance Mw will not be detected in the camera image Ic captured by the sensing camera 41 even by edge filter processing.

[0074] 17, the extraction unit 5100 of the fifth embodiment has a search unit 5108 in addition to the pixel interpolation unit 102, edge detection unit 104, and matching determination unit 106 as sub-functional units. The search unit 5108 searches for a target whose reflection point distance dr is within a set range Δd in the distance image Iord captured by the optical sensor 40 (hereinafter, also referred to as a target within a set range). This means searching for an object point near the optical incidence surface 33o whose reflection point distance dr is within the set range Δd. Here, the set range Δd in the fifth embodiment is set to a range in which the edge of the water-related substance Mw can be detected by edge filter processing in the distance image Iord captured by the optical sensor 40 according to the focal length df on the sensing area Ac side in the imaging lens system 413.

[0075] In the fifth embodiment, a distance image Iord including a target within a set range searched by the search unit 5108 and a camera image Ic pixel-interpolated by the pixel interpolation unit 102 are subjected to edge detection by the edge detection unit 104, and then their edges are compared by the matching determination unit 106. This enables the matching determination unit 106, which determines the matching state, to extract a pixel group Po representing the edge of a target within the set range in the distance image Iord as a pixel group Po unmatched with the camera image Ic.

[0076] Therefore, when an unmatched pixel group Po representing an edge of a target within a set range is extracted, the control unit 5120 of the fifth embodiment determines that the water-related substance Mw, which is estimated to correspond to the unmatched pixel group Po, is predicted to adhere to the optical incidence surface 33o. In other words, when the reflection point distance dr in the unmatched pixel group Po is within the set range Δd according to the focal length df, the control unit 5120 predicts that the water-related substance Mw will adhere to the optical incidence surface 33o.

[0077] In this way, when the reflection point distance dr in the unmatched pixel group Po falls within the set range Δd, the control unit 5120, which has determined that adhesion of the water-related substance Mw to the optical incidence surface 33o is expected, commands the cleaning system 5 to perform cleaning control of the optical incidence surface 33o using cleaning gas. After issuing this cleaning control command, if the pixel group Po having the reflection point distance dr within the set range Δd is extracted from the distance image Iord by the extraction unit 5100, the control unit 5120 further commands the cleaning system 5 to perform cleaning control of the optical incidence surface 33o using cleaning liquid. At this time, the extraction of the pixel group Po is realized by the edge detection unit 104 detecting the edge of the unmatched pixel group Po in the distance image Iord before the command for cleaning control using cleaning gas in the latest distance image Iord after the command. Note that, if the pixel group Po having the reflection point distance dr within the set range Δd no longer exists in the distance image Iord after issuing the command for cleaning control using cleaning gas, the control unit 5120 commands the cleaning system 5 to stop cleaning control.

[0078] In the flow of the vehicle control method according to the fifth embodiment, as shown in Fig. 18, in S500, the search unit 5108 of the extraction unit 5100 searches for a target whose reflection point distance dr is within a set range Δd in the distance image Iord, that is, a target within the set range. As a result, when a target within the set range is found, the flow proceeds to S501.

[0079] In S501, the pixel interpolation unit 102 of the extraction unit 5100 performs pixel interpolation on the high-resolution camera image Ic to match it with the distance image Iord serving as the reflected light image Ior on the low-resolution side. Next, in S502, the edge detection unit 104 of the extraction unit 5100 detects edges from each of the distance image Iord and the camera image Ic.

[0080] Next, in S503, the matching determination unit 106 of the extraction unit 5100 determines the matching state of the distance image Iord and the camera image Ic by comparing the edges detected in these images Iord and Ic. If, as a result, a pixel group Po representing an edge of a target within a set range in the distance image Iord is extracted as a pixel group Po unmatched with the camera image Ic, the flow proceeds to S504.

[0081] In S504, the control unit 5120 estimates that the unmatched pixel group Po representing the edge of the target within the set range corresponds to the water-related substance Mw predicted to adhere to the optical incidence surface 33o. In other words, the water-related substance Mw predicted to correspond to the unmatched pixel group Po is predicted to adhere to the optical incidence surface 33o. Then, in S505, the control unit 5120 commands the cleaning system 5 to execute cleaning control of the optical incidence surface 33o with cleaning gas as control of the vehicle 2 in the autonomous driving mode.

[0082] In S506, after issuing a cleaning control command to the optical incidence surface 33o using cleaning gas, the edge detection unit 104 of the extraction unit 5100 determines whether or not a pixel group Po having a reflection point distance dr within the set range Δd has been extracted from the distance image Iord. If the edge detection unit 104 has extracted a pixel group Po having a reflection point distance dr within the set range Δd, the flow proceeds to S507. In S507, the control unit 5120 commands the cleaning system 5 to execute cleaning control to the optical incidence surface 33o using cleaning liquid as control of the vehicle 2 in the autonomous driving mode.

[0083] On the other hand, if the edge detection unit 104 cannot extract the pixel group Po having the reflection point distance dr within the set range Δd as a result of S506, the flow proceeds to S508, and the control unit 5120 commands the cleaning system 5 to stop cleaning control. Also, if the target within the set range is not found as a result of S500, or if the pixel group Po representing the edge of the target within the set range is not extracted as a result of S503, which is perfect matching, the flow proceeds to S508, and the control unit 5120 commands the cleaning system 5 to stop cleaning control. As described above, in the fifth embodiment, S500, S501, S502, S503, and S506 correspond to the extraction process, and S504, S505, S507, and S508 correspond to the control process.

[0084] (Action and effect) The effects of the fifth embodiment described above will be described below.

[0085] According to the fifth embodiment, the camera incident surface 33c on which light is incident from the sensing area Ac overlapping with the optical sensor 40 in the sensing camera 41 is located within the focal length df on the sensing area Ac side of the imaging lens system 413 that forms an image of the incident light on the image sensor 411. According to this, when the reflection point distance dr in the unmatched pixel group Po is within the set range Δd according to the focal length df, it can be accurately determined that the adhesion of the water-related substance Mw is predicted. Therefore, it becomes possible to appropriately command and control the cleaning of the incident surface 33o in a situation where there is a high possibility of the water-related substance Mw adhering.

[0086] According to the fifth embodiment, when the reflection point distance dr is within the set range Δd, cleaning control using cleaning gas is commanded to the cleaning system 5, and then when a pixel group Po with a reflection point distance dr within the set range Δd is extracted from the distance image Iord, cleaning control using cleaning liquid is commanded to the cleaning system 5. This makes it possible to save on cleaning liquid by using different cleaning gases and cleaning liquids according to the type or adhesion state of the water-related substance Mw. For example, if the water-related substance Mw is rain, mist, water vapor, or the like, cleaning can be easily completed even with cleaning gas, and therefore the cleaning liquid can be effectively used to clean sludge or the like as the water-related substance Mw.

[0087] (Other embodiments)

[0088] Although several embodiments have been described above, the present disclosure should not be construed as being limited to those embodiments, and can be applied to various embodiments and combinations within the scope not departing from the gist of the present disclosure.

[0089] In a modified example, the dedicated computer constituting the vehicle control device 1 may be at least one external center computer capable of communicating with the vehicle 2. In a modified example, the dedicated computer constituting the vehicle control device 1 may include at least one of a digital circuit and an analog circuit as a processor. Here, the digital circuit is at least one of, for example, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), an SOC (System on a Chip), a PGA (Programmable Gate Array), and a CPLD (Complex Programmable Logic Device). In addition, such a digital circuit may have a memory that stores a program.

[0090] In a modified example, the light-transmitting cover 32 forming the incident surface 33o may be provided in the optical sensor 40, 3040. In a modified example, the incident surface 33o may be formed by an optical member such as a lens in the optical sensor 40, 3040.

[0091] In the modified example, the optical sensor 40, 3040 may acquire an intensity image as the reflected light image Ior, in which the luminance values ​​acquired for each of a plurality of pixels according to the reflected light intensity are converted into two-dimensional data as pixel values. In this case, the functions of the sub-functional units constituting the extraction unit 100, 3100 are realized between the camera image Ic or the external light image Ioo and the intensity image as the reflected light image Ior. However, in this case, the function of determining the reflection point distance dr by the control unit 120 in the first and third embodiments may be performed based on the distance image Iord as the reflected light image Ior, or may be omitted.

[0092] In the modified example, S104 in the first and third embodiments may be omitted as shown in Figures 19 and 20. That is, in the flow of the vehicle control method, the possibility of adhesion of the water-related substance Mw to the incident surface 33o is predicted only based on the unmatching determination in S103 or S303, and S105 may be executed.

[0093] In the modified example, the first embodiment and the second embodiment may be realized in combination as shown in Fig. 21. That is, if adhesion of the water-related substance Mw to the incident surface 33o is not predicted as a result of S104, the flow of the vehicle control method may proceed to S205, and if perfect matching is indicated as a result of S103, the flow may proceed to S106, S206 in that order.

[0094] In a modified example, the third and fourth embodiments may be realized in combination as shown in Fig. 22. That is, if adhesion of the water-related substance Mw to the incident surface 33o is not predicted as a result of S104, the flow of the vehicle control method may proceed to S405, and if perfect matching is indicated as a result of S303, the flow may proceed sequentially to S106 and S406.

[0095] In the modified example, travel control equivalent to the second embodiment may be executed instead of or in addition to the cleaning control according to the fifth embodiment. In the modified example, instead of executing S505 to S507 of the fifth embodiment, a step in which the control unit 5120 commands the cleaning system 5 to perform cleaning control using at least one of a cleaning gas and a cleaning liquid may be executed. In the modified example, in S506 of the fifth embodiment, processing equivalent to S500 to S503 may be executed, so that extraction determination of the pixel group Po may be realized by the edge detection unit 104 of the extraction unit 5100. In the modified example, in S105 of the first and third embodiments, steps equivalent to S506 to S508 of the fifth embodiment (however, the set range is the range defined in the first embodiment) may be executed. [Explanation of symbols]

[0096] 1: vehicle control device, 2: vehicle, 5: cleaning system, 6: driving control device, 12: processor, 33c: incident surface / camera incident surface, 33o: incident surface / optical incident surface, 40, 3040: optical sensor, 41: sensing camera, 100, 3100, 5100: extraction unit, 120, 2120, 4120, 5120: control unit, 411: imaging element, 413: lens system, Ac, Ao: sensing area, dr: reflection point distance, df: focal length, Ic: camera image, Ioo: external light image, Ior: reflected light image, Iord: distance image, Po: pixel group, Mw: water-related substance

Claims

1. A vehicle control device (1) for controlling a vehicle (2) equipped with an optical sensor (40) for acquiring a reflected light image (Ior) by sensing reflected light in response to light irradiation, and a sensing camera (41) for acquiring a camera image (Ic) corresponding to an external light intensity in a sensing area (Ac) overlapping with the optical sensor, an extraction unit (100, 5100) that extracts an unmatched pixel group (Po) by comparing the reflected light image with the camera image; A vehicle control device comprising: a control unit (120, 2120, 5120) that commands the vehicle to perform control according to a water-related substance (Mw) that is estimated to correspond to the unmatched pixel group.

2. A cleaning system (5) is mounted on the vehicle so as to clean an incident surface (33o) on which light is incident from a sensing area (Ao) of the optical sensor, 2. The vehicle control device according to claim 1, wherein the control unit (120) instructs the cleaning system of the vehicle in an autonomous driving mode to perform cleaning control for the incident surface when the water-related substance corresponding to the unmatched pixel group is predicted to adhere to the incident surface.

3. The extraction unit compares a distance image (Iord) as the reflected light image corresponding to a reflection point distance (dr) of the reflected light with the camera image, The vehicle control device according to claim 2 , wherein the control unit determines that adhesion of the water-related substance is predicted when the reflection point distance in the unmatched pixel group is within a set range.

4. In the sensing camera, an incident surface (33c) on which light is incident from a sensing area (Ac) overlapping with the optical sensor is located within a focal length (df) on the sensing area (Ac) side of an imaging lens system (413) that forms an image of the incident light on an image sensor (411); The vehicle control device according to claim 3, wherein the control unit (5120) determines that adhesion of the water-related substance is predicted when the reflection point distance in the unmatched pixel group is within the set range corresponding to the focal length.

5. A vehicle control device as described in claim 3 or 4, wherein the control unit (5120) instructs the cleaning system to perform cleaning control using cleaning gas when the reflection point distance falls within the set range, and then instructs the cleaning system to perform cleaning control using cleaning liquid when a group of pixels whose reflection point distance falls within the set range is extracted from the distance image by the extraction unit (5100).

6. The vehicle control device according to any one of claims 1 to 5, wherein the control unit instructs driving control of the vehicle in an autonomous driving mode so as to exclude the water-related substances estimated to correspond to the unmatched pixel group from being recognized as obstacles.

7. A vehicle control method for controlling a vehicle (2) equipped with an optical sensor (40) that obtains a reflected light image (Ior) by sensing reflected light in response to light irradiation, and a sensing camera (41) that obtains a camera image (Ic) corresponding to an external light intensity in a sensing area (Ac) that overlaps with the optical sensor, comprising: an extraction step (S101, S102, S103, S500, S501, S502, S503, S506) of extracting an unmatched pixel group (Po) by comparing the reflected light image with the camera image; A vehicle control method including a control step (S104, S105, S106, S205, S206, S504, S505, S507, S508) of instructing the vehicle to perform control according to a water-related substance (Mw) estimated to correspond to the unmatched pixel group.

8. A cleaning system (5) is mounted on the vehicle so as to clean an incident surface (33o) on which light is incident from a sensing area (Ao) of the optical sensor, 8. The vehicle control method according to claim 7, wherein the control steps (S104, S105, S106, S504, S505, S507, S508) command the cleaning system of the vehicle in an autonomous driving mode to perform cleaning control on the incident surface when adhesion of the water-related substance corresponding to the unmatched pixel group to the incident surface is predicted.

9. The extraction step compares a distance image (Iord) as the reflected light image corresponding to a reflection point distance (dr) of the reflected light with the camera image, The vehicle control method according to claim 8 , wherein the control step determines that adhesion of the water-related substance is predicted when the reflection point distance in the unmatched pixel group is within a set range.

10. In the sensing camera, an incident surface (33c) on which light is incident from a sensing area (Ac) overlapping with the optical sensor is located within a focal length (df) on the sensing area (Ac) side of an imaging lens system (413) that forms an image of the incident light on an image sensor (411); 10. The vehicle control method according to claim 9, wherein the control steps (S504, S505, S507, S508) determine that adhesion of the water-related substance is predicted when the reflection point distance in the unmatched pixel group is within the set range corresponding to the focal length.

11. The vehicle control method of claim 9 or 10, wherein the control steps (S504, S505, S507, S508) instruct the cleaning system to perform cleaning control using cleaning gas when the reflection point distance falls within the set range, and then instruct the cleaning system to perform cleaning control using cleaning liquid when a pixel group whose reflection point distance falls within the set range is extracted from the distance image by the extraction step (S506).

12. The vehicle control method according to any one of claims 7 to 11, wherein the control step instructs driving control of the vehicle in an autonomous driving mode so as to exclude the water-related substance corresponding to the unmatched pixel group from being recognized as an obstacle.

13. A vehicle control program including instructions to cause a processor (12) to execute a vehicle (2) equipped with an optical sensor (40) that obtains a reflected light image (Ior) by sensing reflected light in response to light irradiation, and a sensing camera (41) that obtains a camera image (Ic) corresponding to an external light intensity in a sensing area (Ac) that overlaps with the optical sensor, the vehicle control program including instructions to cause a processor (12) to execute the vehicle control program, the vehicle control program including instructions to cause a processor (12) to execute the vehicle control program, The instruction: an extraction step (S101, S102, S103, S500, S501, S502, S503, S506) of extracting unmatched pixel groups by comparing the reflected light image with the camera image; A vehicle control program including a control process (S104, S105, S106, S205, S206, S504, S505, S507, S508) for instructing the vehicle to perform control according to a water-related substance (Mw) estimated to correspond to the unmatched pixel group.

14. A cleaning system (5) is mounted on the vehicle so as to clean an incident surface (33o) on which light is incident from a sensing area (Ao) of the optical sensor, The vehicle control program according to claim 13, wherein the control steps (S104, S105, S106, S504, S505, S507, S508) command the cleaning system of the vehicle in an autonomous driving mode to perform cleaning control on the incident surface when adhesion of the water-related substance corresponding to the unmatched pixel group to the incident surface is predicted.

15. The extraction step compares a distance image (Iord) as the reflected light image corresponding to a reflection point distance (dr) of the reflected light with the camera image, The vehicle control program according to claim 14 , wherein the control step determines that adhesion of the water-related substance is predicted when the reflection point distance in the unmatched pixel group is within a set range.

16. In the sensing camera, an incident surface (33c) on which light is incident from a sensing area (Ac) overlapping with the optical sensor is located within a focal length (df) on the sensing area (Ac) side of an imaging lens system (413) that forms an image of the incident light on an image sensor (411); 16. The vehicle control program according to claim 15, wherein the control steps (S504, S505, S507, S508) determine that adhesion of the water-related substance is predicted when the reflection point distance in the unmatched pixel group is within the set range corresponding to the focal length.

17. The vehicle control program according to claim 15 or 16, wherein the control steps (S504, S505, S507, S508) instruct the cleaning system to perform cleaning control using cleaning gas when the reflection point distance falls within the set range, and then instruct the cleaning system to perform cleaning control using cleaning liquid when a pixel group whose reflection point distance falls within the set range is extracted from the distance image by the extraction step (S506).

18. The vehicle control program according to any one of claims 13 to 17, wherein the control step instructs driving control of the vehicle in an autonomous driving mode so as to exclude the water-related substance corresponding to the unmatched pixel group from being recognized as an obstacle.

Images