Cleaning control device, cleaning control method, cleaning control program

Abstract

The cleaning control device (1) of the cleaning system (5) of the vehicle (2) that mounts an optical sensor (40) that acquires an external light image (Ioo) corresponding to an external light intensity in the stop of light irradiation that senses reflected light, a sensing camera (41) that acquires a camera image (Ic) corresponding to an external light intensity, and a cleaning system (5) that cleans the optical sensor (40) and an incidence surface (33o, 33c) on which light from the overlapping sensing regions (Ao, Ac) is incident, includes: an extraction section (100) that extracts a group of pixels that do not match by comparing the external light image (Ioo) and the camera image (Ic); and a control section (120) that issues an instruction for cleaning control of dirt (Do, Dc) estimated to correspond to the group of pixels (Po, Pc) that do not match on the incidence surface (33o, 33c) to the cleaning system (5).

Description

[0001] Cross-reference to related applications

[0002] This application claims priority to Japanese Patent Application No. 2020-153237, filed on September 11, 2020, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to control technology for cleaning systems of sensor systems in vehicles. Background Technology

[0004] It is known that in vehicle sensor systems, if dirt adheres to the light-receiving surface where light enters the sensing area, it leads to a decrease in sensing accuracy. Therefore, when dirt adheres to the light-receiving surface, it needs to be cleaned. Especially in recent years, the cleaning of the light-receiving surface in autonomous driving modes has become crucial to the continuity of autonomous driving.

[0005] Patent Document 1 discloses a technique for determining dirt on the incident surface. In this disclosed technique, dirt on the incident surface is determined based on the change in intensity of reflected light under illumination.

[0006] Patent Document 1: Japanese Patent Application Publication No. 2005-10094

[0007] However, in the technology disclosed in Patent Document 1, there is a concern that it is difficult to accurately distinguish the intensity of reflected light from an object approaching the injection surface and the intensity of reflected light from dirt adhering to the injection surface. Therefore, if the cleaning of the injection surface is controlled based on the determination results of the technology disclosed in Patent Document 1, wasteful cleaning is anticipated. Summary of the Invention

[0008] The present disclosure addresses the issue of providing a cleaning control device that improves the appropriateness of cleaning control. Other issues of the present disclosure include providing a cleaning control method that improves the appropriateness of cleaning control. Still other issues of the present disclosure include providing a cleaning control procedure that improves the appropriateness of cleaning control.

[0009] The technical solution of this disclosure used to solve the problem will be described below.

[0010] The first aspect of this disclosure is a cleaning control device for controlling a vehicle equipped with an optical sensor that acquires an external light image corresponding to the external light intensity during the cessation of light irradiation by the sensed reflected light, a sensing camera that acquires a camera image corresponding to the external light intensity, and a cleaning system that cleans the incident surface of the optical sensor and the sensing camera from which light enters from the overlapping sensing areas, comprising:

[0011] The extraction unit extracts mismatched pixel groups by comparing the external light image with the camera image; and

[0012] The control section issues an instruction of cleaning control of dirt estimated on the incident surface to correspond to the unmatched pixel group to the cleaning system.

[0013] The second aspect of the present disclosure is a cleaning control method of a cleaning system of a vehicle equipped with an optical sensor that acquires an ambient light image corresponding to an ambient light intensity in a stop of light irradiation for sensing reflected light, a sensing camera that acquires a camera image corresponding to the ambient light intensity, and a cleaning system that cleans an incident surface on which light from a sensing region overlapping with the optical sensor and the sensing camera is incident, including:

[0014] a extraction step of extracting an unmatched pixel group by comparing the ambient light image and the camera image; and

[0015] a control step of causing the processor to issue an instruction of cleaning control of dirt estimated on the incident surface to correspond to the unmatched pixel group to the cleaning system.

[0016] The third aspect of the present disclosure is a cleaning control program including commands for causing a processor to execute in order to control a cleaning system of a vehicle equipped with an optical sensor that acquires an ambient light image corresponding to an ambient light intensity in a stop of light irradiation for sensing reflected light, a sensing camera that acquires a camera image corresponding to the ambient light intensity, and a cleaning system that cleans an incident surface on which light from a sensing region overlapping with the optical sensor and the sensing camera is incident,

[0017] The commands include:

[0018] a extraction step of causing the processor to compare the ambient light image and the camera image, and extract an unmatched pixel group; and

[0019] a control step of causing the processor to issue an instruction of cleaning control of dirt estimated on the incident surface to correspond to the unmatched pixel group to the cleaning system.

[0020] According to the first to third aspects, the ambient light image acquired by the optical sensor according to the ambient light intensity in the stop of light irradiation for sensing reflected light, and the camera image acquired by the sensing camera according to the ambient light intensity with the sensing region overlapping with the sensor are compared. It can be accurately estimated that the unmatched pixel group extracted by such comparison corresponds to dirt attached on any one of the incident surfaces of the optical sensor and the sensing camera. Therefore, according to the instruction of the cleaning control of the dirt as a result of the estimation issued to the cleaning system, the appropriateness of the cleaning control can be improved.

[0021] A fourth aspect of the present disclosure is a cleaning control device of a cleaning system of a vehicle that controls a cleaning system that mounts an optical sensor that acquires a reflected light intensity for light irradiation, a sensing camera that acquires a camera image corresponding to an external light intensity in a sensing region overlapping the optical sensor, and a cleaning system that cleans an entry surface into which light from the sensing region enters in the optical sensor, including:

[0022] an extraction section that extracts an attention object in the camera image that is an attention target; and

[0023] a control section that issues an instruction for cleaning control of dirt estimated on the entry surface to correspond to the amount of change outside the allowable range to the cleaning system in a case where the amount of change in the reflected light intensity from the attention object is outside the allowable range.

[0024] A fifth aspect of the present disclosure is a cleaning control method of a cleaning system of a vehicle that controls a cleaning system that mounts an optical sensor that acquires a reflected light intensity for light irradiation, a sensing camera that acquires a camera image corresponding to an external light intensity in a sensing region overlapping the optical sensor, and a cleaning system that cleans an entry surface into which light from the sensing region enters in the optical sensor, including:

[0025] an extraction step of extracting an attention object in the camera image that is an attention target; and

[0026] a control step of issuing an instruction for cleaning control of dirt estimated on the entry surface to correspond to the amount of change outside the allowable range to the cleaning system in a case where the amount of change in the reflected light intensity from the attention object is outside the allowable range.

[0027] A sixth aspect of the present disclosure is a cleaning control program that causes a processor to execute commands for controlling a cleaning system of a vehicle that mounts an optical sensor that acquires a reflected light intensity for light irradiation, a sensing camera that acquires a camera image corresponding to an external light intensity in a sensing region overlapping the optical sensor, and a cleaning system that cleans an entry surface into which light from the sensing region enters in the optical sensor,

[0028] the commands including:

[0029] an extraction step of causing the processor to extract an attention object in the camera image that is an attention target; and

[0030] a control step of causing the processor to issue an instruction for cleaning control of dirt estimated on the entry surface to correspond to the amount of change outside the allowable range to the cleaning system in a case where the amount of change in the reflected light intensity from the attention object is outside the allowable range.

[0031] According to these fourth to sixth modes, in a camera image acquired by a sensing camera from an external light intensity in a sensing region overlapping with an optical sensor, an attention object of interest is extracted. In a case where a change amount of a reflection light intensity acquired by the optical sensor from the attention object of interest thus extracted by light irradiation is outside an allowable range, it is accurately estimated as corresponding to dirt attached on an entrance surface of the optical sensor. Therefore, according to an instruction of a dirt cleaning control issued to a cleaning system as a result of the estimation, the appropriateness of the cleaning control can be improved.

[0032] The seventh mode of the present disclosure is a cleaning control device of a cleaning system of a vehicle that controls a cleaning system of a vehicle that mounts an optical sensor that acquires a reflection light intensity for light irradiation and acquires an external light image corresponding to an external light intensity in a stop of the light irradiation, and a cleaning system of an entrance surface on which light from a sensing region of the cleaning optical sensor enters,

[0033] an extraction section that extracts an attention object of interest in the external light image; and

[0034] a control section that, in a case where a change amount of the reflection light intensity from the attention object of interest is outside an allowable range, issues an instruction of a cleaning control of dirt estimated on the entrance surface as corresponding to the change amount outside the allowable range to the cleaning system.

[0035] The eighth mode of the present disclosure is a cleaning control method of a cleaning system of a vehicle that controls a cleaning system of a vehicle that mounts an optical sensor that acquires a reflection light intensity for light irradiation and acquires an external light image corresponding to an external light intensity in a stop of the light irradiation, and a cleaning system of an entrance surface on which light from a sensing region of the cleaning optical sensor enters,

[0036] an extraction step of extracting an attention object of interest in the external light image; and

[0037] a control step of, in a case where a change amount of the reflection light intensity from the attention object of interest is outside an allowable range, issuing an instruction of a cleaning control of dirt estimated on the entrance surface as corresponding to the change amount outside the allowable range to the cleaning system.

[0038] The ninth mode of the present disclosure is a cleaning control program including commands for causing a processor to execute in order to control a cleaning system of a vehicle that mounts an optical sensor that acquires a reflection light intensity for light irradiation and acquires an external light image corresponding to an external light intensity in a stop of the light irradiation, and a cleaning system of an entrance surface on which light from a sensing region of the cleaning optical sensor enters,

[0039] the commands include:

[0040] an extraction step of causing the processor to extract an attention object of interest in the external light image; and

[0041] The control process causes the processor to issue an instruction for cleaning control of dirt estimated to correspond to the amount of change outside the allowable range on the irradiation surface to the cleaning system when the amount of change in the reflected light intensity from the object of interest is outside the allowable range.

[0042] According to these seventh to ninth modes, the object of interest is extracted from the external light image acquired by the optical sensor according to the external light intensity in the stop of the light irradiation for acquiring the reflected light intensity. When the amount of change in the reflected light intensity acquired by the optical sensor from the object of interest thus extracted by the light irradiation is outside the allowable range, it can be accurately estimated to correspond to the dirt attached on the irradiation surface of the optical sensor. Therefore, according to the instruction for the cleaning control of the dirt issued to the cleaning system as a result of the estimation, the appropriateness of the cleaning control can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 is a side view showing a mounting state of the automatic driving unit of the first embodiment to a vehicle.

[0044] Figure 2 is a cross-sectional schematic view showing the overall configuration of the automatic driving unit of the first embodiment.

[0045] Figure 3 is a block diagram showing a detailed configuration of the cleaning control device of the first embodiment.

[0046] Figure 4 is a cross-sectional schematic view showing the sensing range of the optical sensor and the sensing camera of the first embodiment.

[0047] Figure 5 is a schematic view for explaining the acquired image of the optical sensor of the first embodiment.

[0048] Figure 6 is a schematic view for explaining the acquired image of the sensing camera of the first embodiment.

[0049] Figure 7 is a schematic view for explaining the interpolated image of the sensing camera of the first embodiment.

[0050] Figure 8 is a flowchart showing the cleaning control method of the first embodiment.

[0051] Figure 9 is a block diagram showing a detailed configuration of the cleaning control device of the second embodiment.

[0052] Figure 10 is a graph for explaining the acquired intensity of the optical sensor of the second embodiment.

[0053] Figure 11is a schematic view for explaining an acquired image of the optical sensor of the second embodiment.

[0054] Figure 12 is a schematic view for explaining an acquired image of the sensing camera of the second embodiment.

[0055] Figure 13 is a flowchart showing a cleaning control method of the second embodiment.

[0056] Figure 14 is a block diagram showing a detailed configuration of a cleaning control device of the third embodiment.

[0057] Figure 15 is a schematic view for explaining an acquired image of the optical sensor of the third embodiment.

[0058] Figure 16 is a flowchart showing a cleaning control method of the third embodiment. DETAILED DESCRIPTION

[0059] Hereinafter, a plurality of embodiments will be described based on the drawings. In addition, sometimes, by adding the same reference numerals to the corresponding constituent elements in each embodiment, the repeated description is omitted. In addition, in a case where only a part of the configuration is described in each embodiment, the other part of the configuration can apply the configuration of the other embodiment described in advance. Also, not only the combination of the configuration explicitly shown in the description of each embodiment, but also the configuration of each of the embodiments can be partially combined with each other even if not explicitly shown, as long as the combination does not particularly cause hindrance.

[0060] (First Embodiment)

[0061] As shown in Figure 1 , an autonomous driving unit ADU provided with a cleaning control device 1 of the first embodiment is mounted on a vehicle 2. The vehicle 2 is capable of always or temporarily automatically traveling in an autonomous driving mode based on autonomous driving control or high-level driving assistance control. The autonomous driving unit ADU is configured to include the cleaning control device 1, a housing 3, a sensor system 4, and a cleaning system 5. In addition, in the following description, the front, rear, left, right, upper, and lower are defined with the vehicle 2 on the horizontal plane as a reference.

[0062] As shown in Figure 1 , 2 , the housing 3 is formed in a hollow flat box shape, for example, by metal or the like. The housing 3 is provided on a roof 20 of the vehicle 2. A plurality of sensor windows 31 are opened in a wall 30 of the housing 3. Each of the sensor windows 31 is covered by a plate-shaped light-transmissive cover 32. An outer surface of each of the light-transmissive covers 32 constitutes an incident surface 33 through which light is incident from the outside of the vehicle 2.

[0063] As shown in Figures 2-4The illustrated sensor system 4 is provided with an optical sensor 40. The optical sensor 40 is a so-called LiDAR (Light Detection and Ranging / Laser Imaging Detection and Ranging) that acquires optical information usable in the automatic driving mode in the vehicle 2. The optical sensor 40 has an emission element 400, a photographing element 401, and a photographing circuit 402.

[0064] The emission element 400 is, for example, a semiconductor element such as a laser diode that generates directional laser light. The emission element 400 causes the laser light emission toward the outside of the vehicle 2 to be intermittent pulsed light beams. The photographing element 401 is, for example, a semiconductor element such as a SPAD (Single Photon Avalanche Diode) that is highly sensitive to light. An emission surface 33o specific to the optical sensor 40 is disposed on the outside (front side in the present embodiment) of the photographing element 401. The photographing element 401 is exposed to light that has entered the emission surface 33o from a sensing region Ao determined by the angle of view of the element 401 from the outside. The photographing circuit 402 is an integrated circuit that controls the exposure and scanning of a plurality of pixels in the photographing element 401 and processes signals from the element 401 to be digitized.

[0065] In a reflection light mode in which the photographing circuit 402 exposes the photographing element 401 to light emission from the emission element 400, an object point within the sensing region Ao becomes a reflection point of the laser light. As a result, the laser light (hereinafter referred to as reflection light) that has been reflected at the reflection point enters the photographing element 401 through the emission surface 33o. At this time, the photographing circuit 402 senses the reflection light by scanning a plurality of pixels of the photographing element 401.

[0066] On the other hand, in an external light mode in which the photographing circuit 402 exposes the photographing element 401 to light emission from the emission element 400 that is intermittent, an object point within the sensing region Ao becomes a reflection point of external light. As a result, the external light that has been reflected at the reflection point enters the photographing element 401 through the emission surface 33o. At this time, the photographing circuit 402 senses the reflected external light by scanning a plurality of pixels of the photographing element 401. Here, in particular, the photographing circuit 402 acquires an external light image loo by two-dimensionally digitizing luminance values acquired for each of the plurality of pixels according to the intensity of the sensed external light as pixel values. Figure 5

[0067] As Figures 2-4 ​As shown, the sensor system 4 is provided with a sensing camera 41 in addition to such an optical sensor 40. The sensing camera 41 is a so-called outside camera that acquires optical information usable in the automatic driving mode in the vehicle 2. The sensing camera 41 has a photographing element 411 and a photographing circuit 402.

[0068] The photographing element 411 is, for example, a semiconductor element such as a CMOS. A light-receiving surface 33c specific to the sensing camera 41 is arranged on the outside (front side in the present embodiment) of the photographing element 411. Light that has entered the light-receiving surface 33c from a sensing region Ac determined by the angle of view of the photographing element 411 in the outside exposes the element 411. Here, as shown, the sensing region Ac of the sensing camera 41 partially overlaps the sensing region Ao of the optical sensor 40. The overlap ratio of the sensing regions Ac, Ao, i.e., the proportion of an overlapping region Aoc in these respective regions Ac, Ao, is, for example, 50% or more, preferably 70% or more, and more preferably 90% or more. The photographing circuit 412 is an integrated circuit that controls exposure and scanning of a plurality of pixels in the photographing element 411 and processes signals from the element 411 to digitize data. Figure 4

[0069] In an exposure mode in which the photographing circuit 412 exposes the photographing element 411, an object point in the sensing region Ac becomes a reflection point of outside light. As a result, outside light that has been reflected at the reflection point enters the photographing element 411 through the light-receiving surface 33c. At this time, the photographing circuit 412 senses the reflected outside light by scanning a plurality of pixels of the photographing element 411. Here, in particular, the photographing circuit 412 acquires a camera image Ic as shown by two-dimensionally digitizing luminance values acquired for each of the plurality of pixels in accordance with the intensity of the sensed outside light as pixel values. Figure 6

[0070] Figures 1-3 The cleaning system 5 shown cleans a plurality of light-receiving surfaces 33 including the light-receiving surfaces 33o, 33c through which light from the sensing regions Ao, Ac enters in the optical sensor 40 and the sensing camera 41. The cleaning system 5 is provided with a cleaning module 50 for each light-receiving surface 33. Each cleaning module 50 can have a cleaning nozzle that sprays a cleaning gas as a cleaning fluid for cleaning the light-receiving surface 33 toward the light-receiving surface 33. Each cleaning module 50 can also have a cleaning nozzle that sprays a cleaning liquid as a cleaning fluid for cleaning the light-receiving surface 33 toward the light-receiving surface 33. Each cleaning module 50 can also have a cleaning wiper that cleans the light-receiving surface 33 by wiping.

[0071] Figure 2 、 3 ​​The illustrated cleaning control device 1 is connected with the electric components 4, 5 of the autonomous driving unit ADU via at least one of a LAN (Local Area Network), a wire harness, an internal bus, and the like. The cleaning control device 1 is configured to include at least one dedicated computer. The dedicated computer constituting the cleaning control device 1 can be a drive control ECU (Electronic Control Unit) that cooperates with an ECU in the vehicle 2 to control the autonomous driving mode. The dedicated computer constituting the cleaning control device 1 can also be an actuator ECU that independently controls a travel actuator of the vehicle 2. The dedicated computer constituting the cleaning control device 1 can also be a localizer ECU that estimates a state quantity of the vehicle 2 including its own position. The dedicated computer constituting the cleaning control device 1 can also be a navigation ECU that navigates a travel route of the vehicle 2. The dedicated computer constituting the cleaning control device 1 can also be an HCU (HMI (Human Machine Interface) Control Unit) that controls information presentation of an information presentation system of the vehicle 2.

[0072] The cleaning control device 1 has at least a memory 10 and a processor 12 by being configured to include such a dedicated computer. The memory 10 is a non-transitory tangible storage medium such as at least one of a semiconductor memory, a magnetic medium, and an optical medium that non-transitorily stores programs and data that can be read by a computer. The processor 12 includes at least one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a RISC (Reduced Instruction Set Computer)-CPU as a core, for example.

[0073] The processor 12 executes a plurality of commands included in the cleaning control program stored in the memory 10. The cleaning control device 1 thereby constructs a plurality of functional sections (i.e., functional modules) for controlling the cleaning system 5. In the cleaning control device 1, the cleaning control program stored in the memory 10 causes the processor 12 to execute a plurality of commands in order to control the cleaning system 5, thereby constructing a plurality of functional sections. As described above, the cleaning control device 1 is configured to include a plurality of functional sections for controlling the cleaning system 5. Figure 3 The plurality of functional sections constructed by the cleaning control device 1 include the extraction section 100 and the control section 120.

[0074] The outside light image Ioo is input from the optical sensor 40 and the camera image Ic is input from the sensing camera 41 at the extraction section 100 in each control cycle. The extraction section 100 extracts a non-matching pixel group by comparing the outside light image Ioo and the camera image Ic. To this end, the extraction section 100 has a pixel interpolation section 102, an edge detection section 104, and a matching determination section 106 as sub-function sections.

[0075] The pixel interpolation section 102 interpolates the pixels in the outside light image Ioo and the camera image Ic corresponding to the high resolution side and the low resolution side. Here, in the present embodiment, the number of pixel groups in the camera image Ic that capture the overlapping region Aoc in the sensing regions Ao, Ac is set to be larger than that in the outside light image Ioo. That is, the camera image Ic becomes the high resolution side and the outside light image Ioo becomes the low resolution side. Therefore, the pixel interpolation section 102 interpolates the pixel values of the camera image Ic for each pixel coordinate from the outside light image Ioo. As a result of the interpolation, the pixel interpolation section 102 generates the camera image Ic having substantially the same resolution as the outside light image Ioo with respect to the overlapping region Aoc. Figure 7

[0076] The edge detection section 104 detects edges from the camera image Ic on which the pixel interpolation has been performed by at least one edge filter process. The edge detection section 104 also detects edges from the outside light image Ioo of the overlapping region Aoc that has substantially the same resolution as the camera image Ic by performing at least one edge filter process on the image Ioo.

[0077] The matching determination section 106 determines the matching state of the outside light image Ioo and the camera image Ic by comparing the edges detected in each of the images Ioo, Ic with each other. At this time, the matching determination section 106 determines non-matching, for example, in the case where it is confirmed that the difference between the pixel values constituting the edges is outside the matching range by normalizing the pixel values with respect to each other and calculating the difference. By such determination, the matching determination section 106 extracts the pixel groups Po, Pc corresponding to the edges that do not match in the outside light image Ioo and the camera image Ic as shown in Figure 5 7 Figure 5 7 The pixel groups are groups of a plurality of pixels, and in particular, the pixel groups Po, Pc that are determined to be non-matching can be groups of a plurality of pixels that are adjacent and continuous. For convenience of explanation,

[0078] As shown in Figure 3 ​​​​As shown, the control unit 120 receives pixel groups Po and Pc extracted from each image Ioo and Ic by the mismatch determination unit 106. In the control unit 120, these mismatched pixel groups Po and Pc are estimated to be on either of the incident surfaces 33o and 33c of the optical sensor 40 and the sensing camera 41, respectively. Figures 5-7 The dirt Do and Dc that are attached correspond to these. Therefore, the control unit 120 issues a cleaning control command to the cleaning system 5 in the vehicle 2 in autonomous driving mode to clean the dirt Do and Dc that are estimated to correspond to the mismatched pixel groups Po and Pc. The cleaning system 5 receives the command and cleans the dirt Do and Dc that are attached to the mismatched pixel groups Po and Pc. Figure 2 The cleaning modules 50o and 50c corresponding to the injection surfaces 33o and 33c are driven and controlled to perform cleaning processes on these surfaces 33o and 33c.

[0079] The following is based on Figure 8 The process of the cleaning control method by which the cleaning control device 1 controls the cleaning system 5 through the cooperation of such functional units 100 and 120 will be described. Furthermore, each "S" in this process refers to a multiple step executed according to multiple commands included in the cleaning control program.

[0080] First, in S101, the pixel interpolation unit 102 of the extraction unit 100 performs pixel interpolation on the high-resolution camera image Ic based on the low-resolution external light image Ioo. Next, in S102, the edge detection unit 104 of the extraction unit 100 detects edges from both the external light image Ioo and the camera image Ic. Then, in S103, the matching determination unit 106 of the extraction unit 100 determines the matching state of the images Ioo and Ic by comparing the edges detected in the external light image Ioo and the camera image Ic with each other.

[0081] If, in step S103, a mismatched pixel group Po or Pc is identified from the external light image Ioo and the camera image Ic, the process proceeds to step S104. In step S104, the control unit 120, in the vehicle 2 in autonomous driving mode, issues an instruction to the cleaning system 5 to execute cleaning control for dirt Do or Dc that is estimated to correspond to the mismatched pixel group Po or Pc.

[0082] On the other hand, if the result of S103 is that no complete match of the pixel groups Po and Pc determined to be mismatched is extracted, the process moves to S105. In S105, the control unit 120 issues a stop command for cleaning control to the cleaning system 5. Based on the above, in this process, S101, S102, and S103 are equivalent to extraction steps, and S104 and S105 are equivalent to control steps.

[0083] (Effects)

[0084] The following describes the effects of the first embodiment described above.

[0085] According to the first embodiment, the outside light image loo acquired by the optical sensor 40 in the stop of the light irradiation that senses the reflected light, and the camera image Ic acquired by the sensing camera 41 that overlaps the sensing region Ao, Ac with the sensor 40, are compared according to the outside light intensity. It is possible to accurately estimate that the mismatched pixel groups Po, Pc extracted by such comparison correspond to the dirt Do, Dc attached to either of the entrance surfaces 330, 33c of the optical sensor 40 and the sensing camera 41. Therefore, according to the instruction to the cleaning system 5 that issues the cleaning control of the dirt as a result of the estimation, it is possible to improve the appropriateness of the cleaning control. This is effective as the cleaning control that contributes to the continuation of the automatic driving mode in the vehicle 2.

[0086] According to the comparison of the edges of the outside light image loo and the camera image Ic as in the first embodiment, it is easy to accurately capture and extract the mismatched pixel groups Po, Pc according to these edges. Therefore, it is possible to appropriately issue the instruction and control the cleaning of the dirt Do, Dc estimated to correspond to the mismatched pixel groups Po, Pc.

[0087] According to the first embodiment, the pixel interpolation is performed from the low resolution side to the high resolution side in the outside light image loo and the camera image Ic, and then these images loo, Ic are compared. It is possible to suppress the case where the pixel groups that should match are mistakenly extracted due to the difference in resolution. Therefore, it is possible to appropriately issue the instruction and control the cleaning of the dirt Do, Dc estimated to correspond to the mismatched pixel groups Po, Pc.

[0088] (Second Embodiment)

[0089] Figure 9 The second embodiment shown is a modification of the first embodiment.

[0090] In the optical sensor 2040 of the second embodiment, the reflected light pattern photographing circuit 2402 acquires the intensity of the reflected light sensed in the light irradiation as the reflected light intensity ir as shown in Figure 10 At this time, the photographing circuit 2402 can output the reflected light intensity ir of the specified pixel as described later. Or the photographing circuit 2402 can acquire the reflected light image Ior as shown in Figure 11 by two-dimensionally dataizing the luminance values acquired according to the reflected light intensity ir for each of a plurality of pixels as pixel values. Note that the function of acquiring the outside light image loo is not necessary in the optical sensor 2040 of the second embodiment, and in Figure 9 the illustration of this function is omitted.

[0091] The extraction section 2100 of the second embodiment inputs the camera image Ic from the sensing camera 41 at every control cycle. The extraction section 2100 extracts at least one object of interest Oa in the camera image Ic as shown. Figure 12 At this time, the extraction of the object of interest Oa is performed, for example, by pattern recognition using an image filter or a machine learning model, etc. As a result, one object of interest Oa whose feature amount is clear can be extracted, or a plurality of objects of interest Oa scattered in the camera image Ic can be extracted. The extraction section 2100 generates pixel coordinate information Ca constituting a pixel corresponding to each object of interest Oa thus extracted in the camera image Ic.

[0092] As shown in FIG. 12, the control section 2120 of the second embodiment inputs the pixel coordinate information Ca of the object of interest Oa extracted by the extraction section 2100. The control section 2122 has an intensity determination section 2122 and a cleaning instruction section 2124 as sub-function sections. Figure 9 The intensity determination section 2122 determines the variation δi of the reflected light intensity ir from the object of interest Oa as shown in FIG. 13. At this time, the intensity determination section 2122 can acquire the reflected light intensity ir on the specified pixel as the reflected light intensity ir from the object of interest Oa by specifying the pixel corresponding to the pixel coordinate information Ca of the object of interest Oa on the optical sensor 2040 at every control cycle. Alternatively, the intensity determination section 2122 can acquire the reflected light intensity ir indicated by the luminance value of the pixel corresponding to the pixel coordinate information Ca of the object of interest Oa in the reflected light image Ior input from the optical sensor 2040 as the reflected light intensity ir from the object of interest Oa.

[0093] Figure 10 The intensity determination section 2122 determines the variation δi of the reflected light intensity ir from the object of interest Oa as shown in FIG. 13. At this time, the intensity determination section 2122 can acquire the reflected light intensity ir on the specified pixel as the reflected light intensity ir from the object of interest Oa by specifying the pixel corresponding to the pixel coordinate information Ca of the object of interest Oa on the optical sensor 2040 at every control cycle. Alternatively, the intensity determination section 2122 can acquire the reflected light intensity ir indicated by the luminance value of the pixel corresponding to the pixel coordinate information Ca of the object of interest Oa in the reflected light image Ior input from the optical sensor 2040 as the reflected light intensity ir from the object of interest Oa.

[0094] The intensity determination section 2122 determines the variation δi of the reflected light intensity ir from the object of interest Oa as shown in FIG. 13. At this time, the intensity determination section 2122 can acquire the reflected light intensity ir on the specified pixel as the reflected light intensity ir from the object of interest Oa by specifying the pixel corresponding to the pixel coordinate information Ca of the object of interest Oa on the optical sensor 2040 at every control cycle. Alternatively, the intensity determination section 2122 can acquire the reflected light intensity ir indicated by the luminance value of the pixel corresponding to the pixel coordinate information Ca of the object of interest Oa in the reflected light image Ior input from the optical sensor 2040 as the reflected light intensity ir from the object of interest Oa. Figure 10 Figure 11 The intensity determination section 2122 determines the variation δi of the reflected light intensity ir from the object of interest Oa as shown in FIG. 13. At this time, the intensity determination section 2122 can acquire the reflected light intensity ir on the specified pixel as the reflected light intensity ir from the object of interest Oa by specifying the pixel corresponding to the pixel coordinate information Ca of the object of interest Oa on the optical sensor 2040 at every control cycle. Alternatively, the intensity determination section 2122 can acquire the reflected light intensity ir indicated by the luminance value of the pixel corresponding to the pixel coordinate information Ca of the object of interest Oa in the reflected light image Ior input from the optical sensor 2040 as the reflected light intensity ir from the object of interest Oa.

[0095] ​​In such an intensity determination section 2122, the pixel in the reflected light image Io that becomes the acquisition target of the reflected light intensity ir and the determination target of the variation amount δi corresponds to the pixel coordinate information Ca of the object of interest Oa given from the extraction section 2100 as Figure 11 Thus, for the same object of interest Oa, it is possible to determine the variation amount δi of the reflected light intensity ir in a single pixel, and it is also possible to determine the average variation amount δi or the total variation amount δi of the reflected light intensity ir in a pixel group that becomes a plurality of pixels.

[0096] As Figure 9 indicated, the cleaning instruction section 2124 issues an instruction of cleaning control to the cleaning system 5 in the vehicle 2 in the autonomous driving mode to clean the dirt Do estimated to correspond to the variation amount δi outside the allowable range, in the case where the variation amount δi of the reflected light intensity ir from the object of interest Oa is outside the allowable range. The cleaning system 5 accepts the instruction and performs the cleaning process of the surface 33o corresponding to the light sensor 2040 by driving control of the cleaning module 50o. At this time, the instruction of the cleaning control related to the surface 33c of the sensing camera 41 is not necessarily required, and thus the cleaning module 50c is not necessarily required either. However, in the case where the dirt estimation is established on the surface 33o of the light sensor 2040, it can also be considered that the dirt estimation is established on the surface 33c of the sensing camera 41 disposed as close as possible to the sensor 2040, and an instruction of the cleaning control related to the surface 33c is issued.

[0097] In the flow of the cleaning control method of such a second embodiment, as Figure 13 indicated, the extraction section 2100 extracts the object of interest Oa from the camera image Ic in S201. Next, the intensity determination section 2122 of the control section 2120 determines which of the variation amounts δi of the reflected light intensity ir from the object of interest Oa is outside the allowable range in S202.

[0098] In the case where the result of S202 is that the variation amount δi of the reflected light intensity ir from the object of interest Oa is outside the allowable range, the flow moves to S203. In S203, the cleaning instruction section 2124 of the control section 2120 issues an instruction of the cleaning control of the dirt Do estimated to correspond to the variation amount δi outside the allowable range to the cleaning system 5 in the vehicle 2 in the autonomous driving mode.

[0099] On the other hand, in the case where the result of S202 is that the variation amount δi is within the allowable range, the flow moves to S204. In S204, the cleaning instruction section 2124 of the control section 2120 issues an instruction of the stop of the cleaning control to the cleaning system 5. According to the above, in the flow, S201 corresponds to the extraction process, and S202, S203, and S204 correspond to the control process.

[0100] (Effects)

[0101] The effects of the second embodiment described above will be explained below.

[0102] According to the second embodiment, an object of interest Oa is extracted from a camera image Ic acquired by the sensing camera 41 based on the external light intensity in the sensing area Ac overlapping with the optical sensor 2040. Even when the change δi of the reflected light intensity ir obtained by the optical sensor 2040 from the extracted object of interest Oa by light illumination is outside the permissible range, it can be accurately estimated as the amount of dirt Do adhering to the incident surface 33o of the optical sensor 2040 (see reference). Figure 11 Therefore, by issuing a cleaning control command for dirt Do to the cleaning system 5 based on the estimated result, the appropriateness of the cleaning control can be improved.

[0103] As in the second embodiment, in the reflected light image Ior acquired by the optical sensor 2040, the pixel corresponding to the object of interest Oa extracted from the camera image Ic acquired by the sensing camera 41 can be easily and accurately determined. Accordingly, if the change δi of the reflected light intensity ir in the pixel corresponding to the object of interest Oa in the reflected light image Ior is outside the allowable range, it is possible to appropriately issue instructions and control the cleaning of dirt Do estimated to correspond to the change δi outside the allowable range.

[0104] (Third Implementation)

[0105] like Figure 14 The third embodiment shown is a variation of the second embodiment.

[0106] In the optical sensor 3040 of the third embodiment, in addition to the function of acquiring the intensity ir of reflected light during illumination according to the second embodiment, the function of acquiring the external light image Ioo during the cessation of illumination according to the first embodiment is also necessary. Here, the sensing of reflected light in the function of acquiring the intensity ir of reflected light and the sensing of external light in the function of acquiring the external light image Ioo are both realized by the same imaging element 401. In addition, the sensing camera 41 of the third embodiment is not necessary. Figure 14 The illustration of camera 41 is omitted in the text.

[0107] In the extraction unit 3100 of the third embodiment, an external light image Ioo is input from the optical sensor 3040 in each control cycle. The extraction unit 3100 is as follows... Figure 15 As shown, at least one object of interest Oa in the external light image Ioo is extracted. Furthermore, the extraction unit 3100 performs the specific extraction function of the object of interest Oa according to the second embodiment.

[0108] In the flow of the cleaning control method of such a third embodiment, as shown in FIG. 31, the extraction section 3100 extracts the object of interest Oa from the external light image Ioo in S301. In addition, S202, S203, and S204 are executed after S301 according to the second embodiment. According to the above in the present flow, S301 corresponds to the extraction process, and S202, S203, and S204 correspond to the control process. Figure 16

[0109] (Action Effects)

[0110] The action effects of the above-described third embodiment will be described below.

[0111] According to the third embodiment, the object of interest Oa is extracted from the external light image Ioo acquired by the optical sensor 3040 according to the external light intensity in the stop of the light irradiation in which the reflected light intensity ir is acquired. In the case where the amount of change δi of the reflected light intensity ir acquired by the optical sensor 3040 from the thus extracted object of interest Oa through the light irradiation is outside the allowable range, it is possible to accurately estimate that it corresponds to the dirt Do (refer to FIG. 30) attached to the entrance surface 33o of the optical sensor 3040. Therefore, according to the instruction to the cleaning system 5 to perform the cleaning control of the dirt Do as a result of the estimation, it is possible to improve the appropriateness of the cleaning control. Figure 15

[0112] In the reflected light image Ior acquired by the optical sensor 3040 as in the third embodiment, it is easy to accurately determine the pixel corresponding to the object of interest Oa extracted from the external light image Ioo acquired by the sensor 3040. Thereby, in the case where the amount of change δi of the reflected light intensity ir in the pixel corresponding to the object of interest Oa in the reflected light image Ior is outside the allowable range, it is possible to appropriately issue an instruction and control the cleaning of the dirt Do estimated to correspond to the amount of change δi outside the allowable range.

[0113] According to the third embodiment, it is possible to suppress the case where the amount of change δi of the reflected light intensity ir from the object of interest Oa sensed by the same element 401 according to the external light intensity in the optical sensor 3040 becomes outside the allowable range due to the axis shift between these intensity sensing. Therefore, it is possible to appropriately issue an instruction and control the cleaning of the dirt estimated to correspond to the amount of change δi outside the allowable range.

[0114] (Other Embodiments)

[0115] The above describes a plurality of embodiments, but the present disclosure is not limited to the explanation of these embodiments, and can be applied to various embodiments and combinations within the scope of the gist of the present disclosure. ​​

[0116] The dedicated computer that constitutes the washing control device 1 in the modification example can also be at least one external center computer capable of communicating with the vehicle 2. The dedicated computer that constitutes the washing control device 1 in the modification example can also include at least one of a digital circuit and an analog circuit as the processor. Here, the digital circuit refers to at least one of, for example, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), a SOC (System on a Chip), a PGA (Programmable Gate Array), and a CPLD (Complex Programmable Logic Device), and the like. Such a digital circuit can also have a memory in which a program is stored.

[0117] The light-transmitting cover 32 that forms the light-incident surface 33o in the modification example can also be provided to the optical sensor 40, 2040, 3040. The light-incident surface 33o in the modification example can also be formed by an optical member such as a lens or the like in the optical sensor 40, 2040, 3040.

Claims

1. A cleaning control device that controls a cleaning system of a vehicle, the vehicle mounting an optical sensor that acquires an outside light image corresponding to an outside light intensity in a stop of light irradiation of reflected light, a sensing camera that acquires a camera image corresponding to the outside light intensity, and the cleaning system that cleans an irradiation surface of the optical sensor and the sensing camera on which light from a sensing region overlapping with the optical sensor is irradiated, the cleaning control device comprising: an extraction section that extracts a pixel group that does not match by comparing the outside light image and the camera image; and a control section that issues an instruction of cleaning control of dirt estimated to correspond to the pixel group that does not match on the irradiation surface to the cleaning system.

2. The cleaning control device according to claim 1, wherein the extraction section compares edges of the outside light image and the camera image with each other.

3. The cleaning control device according to claim 1 or 2, wherein the extraction section performs pixel interpolation from a low resolution side to a high resolution side in the outside light image and the camera image, and then compares the images.

4. The cleaning control device according to claim 1 or 2, wherein the control section issues the instruction of the cleaning control to the cleaning system in the vehicle in an automatic driving mode.

5. A cleaning control device that controls a cleaning system of a vehicle, the vehicle mounting an optical sensor that acquires a reflected light intensity for light irradiation, a sensing camera that acquires a camera image corresponding to an outside light intensity in a sensing region overlapping with the optical sensor, and the cleaning system that cleans an irradiation surface of the optical sensor on which light from the sensing region is irradiated, the cleaning control device comprising: an extraction section that extracts an object of interest in the camera image by using an image filter or pattern recognition of a machine learning model, and generates pixel coordinate information constituting a pixel with respect to a pixel corresponding to the object of interest; and a control section that issues an instruction of cleaning control of dirt estimated to correspond to a change amount of the reflected light intensity from the object of interest on the irradiation surface to the cleaning system in a case where the change amount is outside an allowable range.

6. The cleaning control device according to claim 5, wherein the control section issues the instruction of the cleaning control to the cleaning system in a case where the change amount of the reflected light intensity from the object of interest is outside the allowable range in a reflected light image acquired by the optical sensor from the reflected light intensity.

7. The cleaning control device according to claim 5, wherein the control section issues the instruction of the cleaning control to the cleaning system in the vehicle in an automatic driving mode. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ 8. A cleaning control device that controls a cleaning system of a vehicle, the vehicle being equipped with an optical sensor that acquires a reflected light intensity for light irradiation and acquires an outside light image corresponding to an outside light intensity in a stop of the light irradiation, and the cleaning system that cleans an irradiation surface on which light from a sensing region of the optical sensor is irradiated, the cleaning control device comprising: an extraction section that extracts a focus object of interest in the outside light image by using a pattern recognition of an image filter or a machine learning model, and generates pixel coordinate information that constitutes a pixel with respect to a pixel corresponding to the focus object of interest; and a control section that issues an instruction of a cleaning control of dirt estimated on the irradiation surface to correspond to a variation amount of the reflected light intensity from the focus object of interest outside an allowable range, in a case where the variation amount of the reflected light intensity from the focus object of interest is outside the allowable range.

9. The cleaning control device according to claim 8, wherein the control section issues the instruction of the cleaning control to the cleaning system in a case where the variation amount of the reflected light intensity from the focus object of interest, which is sensed by the same element as the outside light intensity in the optical sensor, is outside the allowable range.

10. The cleaning control device according to claim 8 or 9, wherein the control section issues the instruction of the cleaning control to the cleaning system in a case where the variation amount of the reflected light intensity from the focus object of interest is outside the allowable range in a reflected light image acquired by the optical sensor on the basis of the reflected light intensity.

11. The cleaning control device according to claim 8 or 9, wherein the control section issues the instruction of the cleaning control to the cleaning system in the vehicle in an autonomous driving mode.

12. A cleaning control method that controls a cleaning system of a vehicle, the vehicle being equipped with an optical sensor that acquires an outside light image corresponding to an outside light intensity in a stop of light irradiation in which reflected light is sensed, a sensing camera that acquires a camera image corresponding to an outside light intensity, and the cleaning system that cleans an irradiation surface on which light from overlapping sensing regions of the optical sensor and the sensing camera is irradiated, the cleaning control method comprising: an extraction process of extracting a pixel group that does not match by comparing the outside light image with the camera image; and a control process of issuing an instruction of a cleaning control of dirt estimated on the irradiation surface to correspond to the pixel group that does not match to the cleaning system, the optical sensor is an optical sensor that acquires optical information that can be used in an autonomous driving mode of the vehicle, and the outside light image corresponding to the outside light intensity is acquired using a photographing element of the optical sensor in the stop of the light irradiation in which the reflected light is sensed.

13. The cleaning control method according to claim 12, wherein the extraction process compares edges of the outside light image and the camera image with each other.

14. The cleaning control method according to claim 12 or 13, wherein ​ ​ ​ ​ ​ ​ ​ ​ The extraction process interpolates pixels from the low-resolution side to the high-resolution side in the above-described outside light image and the above-described camera image, and then compares these images.

15. The cleaning control method according to claim 12 or 13, wherein The control process issues an instruction of the cleaning control to the cleaning system in the vehicle in the autonomous driving mode.

16. A cleaning control method of controlling a cleaning system of a vehicle, the vehicle being equipped with an optical sensor that acquires a reflected light intensity for light irradiation, a sensing camera that acquires a camera image corresponding to an outside light intensity in a sensing region overlapping with the optical sensor, and the cleaning system that cleans an incidence surface in the optical sensor through which light from the sensing region is incident, the cleaning control method comprising: an extraction process of extracting, in the camera image, an object of interest by using pattern recognition of an image filter or a machine learning model, and generating pixel coordinate information constituting a pixel with respect to a pixel corresponding to the object of interest; and a control process of issuing an instruction of the cleaning control of dirt estimated to correspond to a variation amount of the reflected light intensity outside an allowable range to the cleaning system on the incidence surface in a case where the variation amount of the reflected light intensity from the object of interest is outside the allowable range.

17. The cleaning control method according to claim 16, wherein The control process issues an instruction of the cleaning control to the cleaning system in a case where the variation amount of the reflected light intensity from the object of interest is outside the allowable range in a reflected light image acquired by the optical sensor from the reflected light intensity.

18. The cleaning control method according to claim 16, wherein The control process issues an instruction of the cleaning control to the cleaning system in the vehicle in the autonomous driving mode.

19. A cleaning control method of controlling a cleaning system of a vehicle, the vehicle being equipped with an optical sensor that acquires a reflected light intensity for light irradiation and acquires an outside light image corresponding to an outside light intensity in a stop of the light irradiation, and the cleaning system that cleans an incidence surface in the optical sensor through which light from a sensing region is incident, the cleaning control method comprising: an extraction process of extracting, in the outside light image, an object of interest by using pattern recognition of an image filter or a machine learning model, and generating pixel coordinate information constituting a pixel with respect to a pixel corresponding to the object of interest; and a control process of issuing an instruction of the cleaning control of dirt estimated to correspond to a variation amount of the reflected light intensity outside an allowable range to the cleaning system on the incidence surface in a case where the variation amount of the reflected light intensity from the object of interest is outside the allowable range.

20. The cleaning control method according to claim 19, wherein The control process issues an instruction of the cleaning control to the cleaning system in a case where the variation amount of the reflected light intensity from the object of interest is outside the allowable range in the optical sensor in which the outside light intensity and the reflected light intensity from the object of interest are sensed by the same element.

21. The cleaning control method according to claim 19 or 20, wherein The control procedure instructs the cleaning system to perform the cleaning control when the amount of change in the intensity of the reflected light from the object of interest in the reflected light image obtained from the optical sensor based on the intensity of the reflected light is outside the allowable range.

22. The cleaning control method according to claim 19 or 20, wherein The control procedure instructs the cleaning system to perform the cleaning control in the vehicle in the autonomous driving mode.

23. A storage medium storing a cleaning control program including commands for causing a processor to control a cleaning system of a vehicle, the vehicle being equipped with an optical sensor that obtains an outside light image corresponding to an outside light intensity in a stop of light irradiation for sensing reflected light, a sensing camera that obtains a camera image corresponding to the outside light intensity, and the cleaning system that cleans a surface on which light from a sensing region overlapping with the optical sensor is incident, The commands include: an extraction procedure that causes the processor to extract a group of pixels that do not match by comparing the outside light image and the camera image; and a control procedure that causes the processor to instruct the cleaning system to perform cleaning control of dirt estimated to correspond to the group of pixels that do not match on the surface on which light is incident, The optical sensor is an optical sensor that obtains optical information usable in an autonomous driving mode of the vehicle, and an outside light image corresponding to an outside light intensity is obtained using a photographing element of the optical sensor in a stop of light irradiation for sensing reflected light.

24. The storage medium according to claim 23, wherein The extraction procedure causes the processor to compare edges of the outside light image and the camera image with each other.

25. The storage medium according to claim 23 or 24, wherein The control procedure causes the processor to instruct the cleaning system to perform the cleaning control in the vehicle in the autonomous driving mode.

26. The storage medium according to claim 23 or 24, wherein The extraction procedure causes the processor to perform pixel interpolation from a low-resolution side to a high-resolution side with respect to the outside light image and the camera image, and then compare the images.

27. A storage medium storing a cleaning control program including commands for causing a processor to control a cleaning system of a vehicle, the vehicle being equipped with an optical sensor that obtains an intensity of reflected light for light irradiation, a sensing camera that obtains a camera image corresponding to an outside light intensity in a sensing region overlapping with the optical sensor, and the cleaning system that cleans a surface on which light from the sensing region is incident, The commands include: an extraction procedure that causes the processor to extract an object of interest in the camera image by using a pattern recognition using an image filter or a machine learning model with respect to the object of interest, and generate pixel coordinate information constituting pixels with respect to pixels corresponding to the object of interest; and The control process causes the processor to issue an instruction for the cleaning control of the dirt estimated on the incident surface to correspond to the variation amount outside the allowable range to the cleaning system in a case where the variation amount of the reflected light intensity from the object of interest is outside the allowable range.

28. The storage medium according to claim 27, wherein The control process causes the processor to issue an instruction for the cleaning control of the dirt estimated on the incident surface to correspond to the variation amount outside the allowable range to the cleaning system in a case where the variation amount of the reflected light intensity from the object of interest is outside the allowable range in the reflected light image acquired from the reflected light intensity by the optical sensor.

29. The storage medium according to claim 27, wherein The control process causes the processor to issue an instruction for the cleaning control to the cleaning system in the vehicle in the autonomous driving mode.

30. A storage medium storing a cleaning control program that is a cleaning control program containing commands for causing a processor to control a cleaning system of a vehicle, in which the vehicle is equipped with an optical sensor that acquires a reflected light intensity for light irradiation and acquires an outdoor light image corresponding to an outdoor light intensity in a stop of the light irradiation, and the cleaning system that cleans an incident surface on which light from a sensing region of the optical sensor is incident, The commands contain: an extraction process that causes the processor to extract an object of interest in the outdoor light image by using a pattern recognition of an image filter or a machine learning model, and generate pixel coordinate information constituting a pixel with respect to a pixel corresponding to the object of interest; and a control process that causes the processor to issue an instruction for the cleaning control of the dirt estimated on the incident surface to correspond to the variation amount outside the allowable range to the cleaning system in a case where the variation amount of the reflected light intensity from the object of interest is outside the allowable range.

31. The storage medium according to claim 30, wherein The control process causes the processor to issue an instruction for the cleaning control to the cleaning system in a case where the variation amount of the reflected light intensity from the object of interest that is sensed by the same element as the outdoor light intensity is outside the allowable range in the vehicle.

32. The storage medium according to claim 30 or 31, wherein The control process causes the processor to issue an instruction for the cleaning control of the dirt estimated on the incident surface to correspond to the variation amount outside the allowable range to the cleaning system in a case where the variation amount of the reflected light intensity from the object of interest is outside the allowable range in the reflected light image acquired from the reflected light intensity by the optical sensor.

33. The storage medium according to claim 30 or 31, wherein The control process causes the processor to issue an instruction for the cleaning control to the cleaning system in the vehicle in the autonomous driving mode.

Images