Sensor cleaning device, sensor cleaning method, and sensor cleaning program

Abstract

To provide a sensor cleaning device which can properly use air cleaning and liquid cleaning selectively, and has reduced influence of liquid-cleaning on automatic operation, and a sensor cleaning method and a sensor cleaning program.SOLUTION: An automatic operation control unit 20 controls automatic operation of a vehicle by using information on an object detected by an object detection unit 16. When a contamination detection unit 18 detects soil on sensor faces of a camera 12 as a sensor or a LiDAR 14 and also determines that cleaning with cleaning liquid, the automatic operation control unit outputs a command to clean the camera 12 or the LiDAR 14 with cleaning liquid to a cleaning control unit 22. The cleaning control unit 22 controls a cleaning actuator 24 so as to clean the sensor faces of the camera 12 or the LiDAR 14 by injecting cleaning liquid thereto following the command, and notifies the automatic operation control unit 20 that the sensor faces of the camera 12 or the LiDAR 14 are being cleaned with cleaning liquid.SELECTED DRAWING: Figure 2

Description

Technical Field

[0004] , , , , ,

[0006] , , , , , , , , , ,

[0005] ,

[0001] The present invention relates to a sensor cleaning device, a sensor cleaning method, and a sensor cleaning program.

Background Art

[0002] In the automatic driving of a vehicle, based on the image information around the vehicle acquired by an imaging device such as a camera and the information of the outside of the vehicle obtained by scanning the surroundings of the vehicle with laser light such as LiDAR, control such as acceleration, deceleration, turning, and braking of the vehicle is executed.

[0003] Therefore, in sensors such as imaging devices and LiDAR, if the so-called sensor surface related to the acquisition of information about the outside of the vehicle is dirty, it becomes difficult to normally acquire the information about the outside of the vehicle. The sensor surface is, in the case of an imaging device, the imaging lens or the transparent glass that protects the imaging lens, and in the case of LiDAR, the light emitting part that irradiates laser light to the outside and the light receiving part that receives the laser light reflected outside.

[0004] In Patent Document 1, an invention of cleaning a recognition sensor for recognizing the situation around a vehicle with high-pressure air or a cleaning liquid has been proposed.

Prior Art Document

Patent Document

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] While a recognition sensor is being cleaned, it becomes difficult to obtain information about the outside world from the recognition sensor being cleaned. However, the invention disclosed in Patent Document 1 cleans the recognition sensor to be cleaned if, among the multiple recognition sensors provided on the vehicle, other recognition sensors recognize the same object as the recognition sensor to be cleaned. However, in the case of air cleaning using high-pressure air, there is no decrease in the recognition performance of the recognition sensor, so there is no particular need to decide whether to clean the recognition sensor to be cleaned if other recognition sensors recognize the same object as the recognition sensor to be cleaned.

[0007] Furthermore, in the invention disclosed in Patent Document 1, when performing liquid cleaning to clean the sensor surface with a cleaning solution, it is necessary to notify the control device that detects dirt on the sensor surface that "liquid cleaning in progress" in order to avoid misidentifying the cleaning solution as dirt. However, since such notification is not made, there was a risk of misidentifying the cleaning solution adhering to the sensor surface as dirt.

[0008] This invention was created in view of the above problems, and aims to provide a sensor cleaning device, a sensor cleaning method, and a sensor cleaning program that accurately differentiate between air cleaning and liquid cleaning, while suppressing the impact of liquid cleaning on automatic operation. [Means for solving the problem]

[0009] To achieve the above objective, the sensor cleaning device according to the present invention includes an object detection unit (16) that detects objects around the vehicle (300) from information acquired by a plurality of sensors (12A~12D, 14A~14D) that monitor the area around the vehicle (300), a dirt detection unit (18) that detects dirt on the sensors (12A~12D, 14A~14D), a cleaning control unit (22) that continues air cleaning by injecting high-pressure air onto the sensors (12A~12D, 14A~14D) for a predetermined air cleaning time, and controls the automatic operation of the vehicle using the information of the objects detected by the object detection unit (16), and the dirt detection unit detects the sensor The system includes an automatic operation control unit (20) that detects dirt on sensors (12A~12D, 14A~14D) and determines that cleaning with a cleaning solution is possible, and outputs a command to the cleaning control unit (22) to clean the sensors (12A~12D, 14A~14D) with a cleaning solution. The cleaning control unit (22) controls the liquid cleaning actuators (24A~24H) to spray the cleaning solution onto the sensors (12A~12D, 14A~14D) in accordance with the command, and also notifies the automatic operation control unit (20) that the sensors (12A~12D, 14A~14D) are being cleaned with the cleaning solution.

[0010] By configuring it in this way, it is possible to obtain a sensor cleaning device, sensor cleaning method, and sensor cleaning program that accurately differentiate between air cleaning and liquid cleaning while suppressing the impact of liquid cleaning on automatic operation. [Brief explanation of the drawing]

[0011] [Figure 1] This schematic diagram shows an example of the arrangement of an autonomous driving ECU that controls the autonomous driving of a vehicle, a camera which is a sensor that acquires information about the outside world of the vehicle necessary for autonomous driving, a LiDAR which is also a sensor that acquires information about the outside world of the vehicle necessary for autonomous driving, and a cleaning actuator that cleans the camera and LiDAR, respectively, in the first embodiment of the present invention. [Figure 2] This is a block diagram showing an example of the configuration of a sensor cleaning device according to the first embodiment of the present invention. [Figure 3]This is a flowchart showing an example of the processing of a sensor cleaning device according to the first embodiment of the present invention. [Figure 4] This is an example of a magnified view of image data acquired by a camera when foreign matter such as dirt adheres to the sensor surface. [Figure 5] Figure 4 shows an example of a histogram of the image data. [Figure 6] This is a block diagram showing an example of the configuration of a sensor cleaning device according to a second embodiment of the present invention. [Figure 7] This is a block diagram showing an example of the configuration of a sensor cleaning device according to the third embodiment of the present invention. [Modes for carrying out the invention]

[0012] [First Embodiment] Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. Figure 1 is a schematic diagram showing an example of the arrangement of an automatic driving ECU (Electronic Control Unit) 10 that controls the automatic driving of a vehicle 300, cameras 12A, 12B, 12C, and 12D which are sensors that acquire information about the outside world of the vehicle 300 necessary for automatic driving, LiDARs 14A, 14B, 14C, and 14D which are sensors that acquire information about the outside world of the vehicle 300 necessary for automatic driving, cleaning actuators 24A, 24B, 24C, and 24D that clean the cameras 12A, 12B, 12C, and 12D respectively, and cleaning actuators 24E, 24F, 24G, and 24H that clean the LiDARs 14A, 14B, 14C, and 14D respectively.

[0013] As shown in Figure 1, camera 12A is mounted on the upper part of the front windscreen of vehicle 300 to acquire an image of the front of vehicle 300. Camera 12B is mounted on the upper part of the rear windscreen of vehicle 300 to acquire an image of the rear of vehicle 300. Camera 12C is mounted on the left side of vehicle 300 to acquire an image of the left side of vehicle 300. And camera 12D is mounted on the right side of vehicle 300 to acquire an image of the right side of vehicle 300.

[0014] Each of the LiDARs 14A, 14B, 14C, and 14D is configured to measure scattered light from a pulsed laser beam emitted from the vehicle 300 into the outside world, enabling analysis of the distance to a distant object and the properties of that object. As shown in Figure 1, LiDAR 14A is installed on the vehicle 300, for example, the front bumper, to acquire information in front of the vehicle 300. LiDAR 14B is installed on the vehicle 300, for example, the rear bumper, to acquire information behind the vehicle 300. LiDAR 14C is installed on the left side of the vehicle 300 to acquire information on the left side of the vehicle 300. And LiDAR 14D is installed on the right side of the vehicle 300 to acquire information on the right side of the vehicle 300.

[0015] The cleaning actuators 24A, 24B, 24C, and 24D perform air cleaning by spraying high-pressure air or liquid cleaning by spraying cleaning fluid onto each of the cameras 12A, 12B, 12C, and 12D, respectively, under the control of the automatic operation ECU 10. Additionally, the cleaning actuators 24E, 24F, 24G, and 24H perform air cleaning or liquid cleaning onto each of the LiDARs 14A, 14B, 14C, and 14D, respectively, under the control of the automatic operation ECU 10. Each of the cleaning actuators 24A to 24H may also perform gas-liquid mixed-phase cleaning by spraying high-pressure air and cleaning fluid onto each of the cameras 12A to 12D and LiDARs 14A to 14D, respectively, in addition to air cleaning and liquid cleaning.

[0016] Figure 2 is a block diagram showing an example of the configuration of the sensor cleaning device 100 according to this embodiment. As shown in Figure 1, the automatic driving ECU 10 is connected to a plurality of cameras 12 (12A to 12D), LiDAR 14 (14A to 14D), and cleaning actuators 24 (24A to 24H) which are devices that clean the camera 12 and LiDAR 14 sensors using one of the cleaning means of "gas," "liquid," or "gas-liquid mixed phase."

[0017] As shown in FIG. 2, the automatic driving ECU 10 includes an object detection unit 16 that detects objects around the vehicle 300 based on the information acquired by the camera 12 and the LiDAR 14, and an automatic driving control unit 20 that controls the automatic driving of the vehicle 300 based on the detection result of the object detection unit 16.

[0018] In addition, the automatic driving ECU 10 includes a dirt detection unit 18 that detects dirt adhering to each sensor surface of the camera 12 and the LiDAR 14, and a cleaning control unit 22 that controls the cleaning actuator 24 based on the detection result of the dirt detection unit 18. When the cleaning control unit 22 operates the cleaning actuator 24 to clean the sensor surface of the camera 12 or the LiDAR 14, it notifies the automatic driving control unit 20 of which sensor to be cleaned is one of the cameras 12A to 12D or the LiDARs 14A to 14D. The automatic driving control unit 20 does not recognize the cleaning liquid adhering to the sensor surface of the sensor to be cleaned as dirt. As described above, the sensor surface is a photographing lens or a transparent glass that protects the photographing lens in the case of the camera 12, and is a light emitting unit that irradiates laser light to the outside world and a light receiving unit that receives the laser light reflected in the outside world in the case of the LiDAR 14.

[0019] FIG. 3 is a flowchart showing an example of the processing of the sensor cleaning device 100 according to the present embodiment. The processing shown in FIG. 3 starts together with the automatic driving of the vehicle 300 as shown in step S100, and the automatic driving and the detection of sensor dirt are started simultaneously.

[0020] In step S102, the dirt detection unit 18 determines whether or not it has detected dirt on the sensor surface. For the dirt on the sensor surface, when the sensor is the camera 12, the presence or absence of dirt on the sensor surface is determined from the image data acquired by the camera 12. Also, when the sensor is the LiDAR 14, it is determined that dirt is adhering to the sensor surface when the amount of the laser light (received light) reflected by the outside of the vehicle 300 becomes equal to or less than a predetermined first light amount threshold value.

[0021] Figure 4 is an example of a magnified view of image data acquired by camera 12 when foreign matter such as dirt is attached to the sensor surface. Each square in Figure 4 is a pixel that makes up the image data. When there is no foreign matter such as dirt on the sensor surface of camera 12, it shows a brightness (pixel value) close to white, as shown by pixel PXL, but when dirt is present on the sensor surface, it shows a pixel value close to black, as shown by pixel PXB.

[0022] Figure 5 is an example of a histogram of the image data shown in Figure 4. In Figure 5, the horizontal axis represents the pixel value and the vertical axis represents the number of pixels. In this embodiment, if the brightness difference ΔB, which is the difference between the maximum and minimum pixel values ​​of pixels that exist for a predetermined number of pixels N or more, is greater than or equal to a predetermined first brightness difference threshold, it is determined that foreign matter such as dirt is present on the sensor surface. Pixels with fewer than the predetermined number of pixels N may contain noise in the image data, so such pixels are not used to determine the maximum and minimum pixel values. The predetermined number of pixels N varies depending on the specifications of the camera 12, but one example is N=2m (m=2~4).

[0023] If step S102 determines that foreign matter such as dirt is present on the sensor surface, the procedure proceeds to step S104. If step S102 determines that no foreign matter such as dirt is present on the sensor surface, dirt detection by the dirt detection unit 18 continues.

[0024] In step S104, the dirt detection unit 18 determines whether the foreign matter adhering to the sensor surface is a water droplet or not. If the sensor is the camera 12, a water droplet is determined to be adhering to the sensor surface if the brightness difference ΔB shown in Figure 5 is less than a second brightness difference that is greater than a predetermined first brightness difference threshold. In other words, if the brightness difference ΔB is greater than or equal to a predetermined second brightness difference threshold, it can be determined that dirt other than a water droplet is adhering to the sensor surface of the camera 12. The predetermined first brightness difference threshold and the predetermined second brightness difference threshold are specifically determined, for example, based on image information acquired with dirt or water droplets adhering to the sensor surface of the camera 12.

[0025] If the sensor is a LiDAR 14, it is determined that water droplets are attached to the sensor surface if the amount of laser light reflected from the outside of the vehicle is greater than a predetermined second light intensity threshold that is less than a predetermined first light intensity threshold. In other words, if the amount of laser light reflected from the outside of the vehicle is less than or equal to the predetermined second threshold, it can be determined that dirt other than water droplets is attached to the sensor surface of the LiDAR 14. The predetermined first light intensity threshold and the predetermined second light intensity threshold are specifically determined, for example, based on the change in the amount of light received when dirt or water droplets are attached to the sensor surface of the LiDAR 14.

[0026] If water droplets are detected on the sensor surface in step S104, the procedure proceeds to step S106. If no water droplets are detected on the sensor surface, i.e., if it is suspected that dirt other than water droplets is attached to the sensor surface, the procedure proceeds to step S112.

[0027] In step S106, the cleaning control unit 22 controls the cleaning actuator 24 to perform air cleaning for a predetermined time by spraying high-pressure air onto the sensor surface of the sensor on which water droplets have been determined to be attached. In this embodiment, dirt on the sensor surface is detected even during the air cleaning in step S106. The spray pressure, spray volume, and predetermined time of the high-pressure air used in the air cleaning in step S106 are determined specifically, for example, through experiments to remove water droplets that have actually adhered to the sensor surface. Alternatively, the air cleaning may be continued until the determination of whether or not the sensor performance has improved in step S108, and if no improvement in sensor performance is observed in step S108, the air cleaning may be stopped.

[0028] In step S108, the object detection unit 16 and the automatic driving control unit 20 determine whether the sensor performance has improved. Specifically, if the sensor is a camera 12, the sensor performance is determined to have improved if the brightness difference ΔB shown in Figure 5 is less than a predetermined first brightness difference threshold. If the sensor is a LiDAR 14, the sensor performance is determined to have improved if the amount of laser light reflected from the outside of the vehicle is greater than a predetermined first light intensity threshold.

[0029] If the sensor performance improves in step S108, proceed to step S102; otherwise, proceed to step S110.

[0030] In step S110, the automatic driving control unit 20 reduces the vehicle speed of the vehicle 300. The degree of speed reduction is influenced by factors such as the speed limit of the driving lane, but one example is 10-15 km / h.

[0031] In step S112, the automatic driving control unit 20 determines whether liquid cleaning control is possible. Specifically, it determines whether object detection for automatic driving will not be hindered even if the sensor to be cleaned is covered with cleaning liquid, and whether liquid cleaning control is possible if there are no predetermined objects around the vehicle 300 that should not be covered with cleaning liquid, such as pedestrians or motorcycles.

[0032] More specifically, this applies when the object detection unit 16 does not detect pedestrians or motorcycles (pre-set objects that should not be splashed with cleaning fluid) within a predetermined range around the vehicle 300 immediately before cleaning with the cleaning fluid. The predetermined range is determined specifically according to the spray pressure and spray volume of the cleaning fluid from the cleaning actuator 24. Furthermore, even if the sensor to be cleaned is covered with cleaning fluid, object detection for automatic driving will not be hindered. This applies, for example, when the dirt detected by the dirt detection unit 18 is minor and can be easily removed by spraying cleaning fluid, or when only some of the multiple sensors, such as cameras 12A to 12D and LiDARs 14A to 14D, are to be cleaned, and the sensors other than those being cleaned are functioning normally.

[0033] If the dirt detected by the dirt detection unit 18 is minor and can be easily removed by spraying cleaning solution, for example, if the sensor is a camera 12, the dirt detection unit 18 will detect the dirt when the brightness difference ΔB shown in Figure 5 is approximately equal to a predetermined first brightness difference threshold, or if the sensor is a LiDAR 14, the amount of laser light reflected from the outside of the vehicle 300 is approximately equal to a predetermined first light intensity threshold.

[0034] In step S112, if liquid cleaning control is possible, a command to clean the sensor with cleaning fluid is output to the cleaning control unit 22 and the procedure proceeds to step S114. If liquid cleaning control is not possible, the procedure proceeds to step S102.

[0035] In step S114, the cleaning control unit 22 determines whether liquid cleaning is possible. Specifically, it determines "No" if liquid cleaning cannot be performed due to constraints on the cleaning actuator 24 side, and "Yes" if it is possible. Constraints on the cleaning actuator 24 side in step S114 include, for example, when multiple sensors are to be cleaned, the limitations of the pump that pumps the cleaning liquid prevent multiple sensors from being cleaned simultaneously, resulting in some sensors being forced to wait for cleaning, or when there is insufficient cleaning liquid and cleaning itself cannot be performed. If liquid cleaning is possible in step S114, the procedure proceeds to step S116; if liquid cleaning is not possible, the procedure proceeds to step S122.

[0036] In step S116, the cleaning control unit 22 controls the cleaning actuator 24 to perform liquid cleaning by spraying cleaning liquid onto the sensor surface of the sensor to be cleaned for a predetermined time. In this embodiment, dirt detection on the sensor surface is also performed during the liquid cleaning in step S116. Furthermore, the spray pressure, spray volume, and predetermined time of the cleaning liquid in the liquid cleaning in step S116 are specifically determined, for example, through experiments to remove dirt that has actually adhered to the sensor surface.

[0037] In step S118, the cleaning control unit 22 notifies that the sensor to be cleaned is being cleaned with liquid. This notification is made because the cleaning liquid may cover the sensor surface and affect sensor detection, and also because the cleaning liquid will be scattered around the vehicle 300. On the other hand, since air cleaning does not directly affect sensor detection or objects present around the vehicle 300, the cleaning control unit 22 does not need to notify the automatic driving control unit 20 that cleaning is in progress when air cleaning is performed.

[0038] In step S120, the object detection unit 16 and the automatic driving control unit 20 determine whether the sensor performance has improved. Specifically, if the sensor is a camera 12, the sensor performance is determined to have improved if the brightness difference ΔB shown in Figure 5 is less than a predetermined first brightness difference threshold. If the sensor is a LiDAR 14, the sensor performance is determined to have improved if the amount of laser light reflected from the outside of the vehicle is greater than a predetermined first light intensity threshold.

[0039] If the sensor performance improves in step S120, the procedure proceeds to step S102 to resume detecting dirt on the sensor surface. If the sensor performance does not improve, the procedure proceeds to step S122.

[0040] In step S122, the automatic driving control unit 20 switches from automatic driving to driver assistance. Specifically, this may involve displaying a warning for an obstacle detected by the object detection unit 16, or a lane departure warning. Alternatively, measures such as further reducing the vehicle speed, prohibiting automatic lane changes, or stopping automatic driving may be taken. Whether to reduce the vehicle speed, prohibit lane changes, or stop automatic driving depends on the location of the sensor from which the dirt could not be removed. For example, if the dirt on the sensor surface of the camera 12A or LiDAR 14A that acquires information in front of the vehicle 300 could not be removed, the impact on automatic driving would be significant, so measures to stop automatic driving would be taken. Also, if the dirt on the sensor surface of the camera 12B or LiDAR 14B that acquires information behind the vehicle 300 could not be removed, measures such as reducing the level of automatic driving, such as prohibiting lane changes, would be taken. Furthermore, if dirt cannot be removed from the sensor surfaces of cameras 12C, 12D, or LiDAR 14C, 14D that acquire information from the side of the vehicle 300, measures such as reducing the vehicle speed will be taken.

[0041] In step S114, the automatic driving control unit 20 determines whether the driver has stopped the automatic driving. In step S124, if the automatic driving is stopped, the automatic driving control unit 20 outputs a command to the cleaning control unit 22 to stop cleaning with cleaning fluid, and terminates the series of processes including liquid cleaning shown in Figure 3. Even if liquid cleaning is terminated, air cleaning may continue. In step S124, if the automatic driving is not stopped, the procedure proceeds to step S102 to resume detecting dirt on the sensor surface.

[0042] As described above, in this embodiment, by appropriately using air cleaning and liquid cleaning, dirt on the sensor surface can be effectively removed. Furthermore, in this embodiment, by performing liquid cleaning when the object detection unit 16 does not detect pedestrians or motorcycles within a predetermined range around the vehicle 300 immediately before cleaning, sensor cleaning can be performed with consideration for the surroundings.

[0043] Furthermore, in this embodiment, if the dirt detected by the dirt detection unit 18 is minor and can be easily removed by spraying cleaning fluid, or if only some of the multiple sensors, namely cameras 12A to 12D and LiDARs 14A to 14D, are to be cleaned and the other sensors are functioning normally, liquid cleaning is performed. In addition, the cleaning control unit 22 notifies the automatic operation control unit 20 that the sensors are being cleaned, thereby suppressing the impact on object detection for automatic operation even if the sensors to be cleaned are covered with cleaning fluid.

[0044] [Second Embodiment] Next, a second embodiment of the present invention will be described. Figure 6 is a block diagram showing an example of the configuration of the sensor cleaning device 200 according to this embodiment. In this embodiment, the cleaning control unit 62 is a circuit independent of the automatic operation ECU 50, and the dirt detection unit 18 outputs information on the detected dirt on the sensor surface not only to the cleaning control unit 62 but also to the automatic operation control unit 20, which is different from the first embodiment. However, the other configurations are the same as in the first embodiment, so the same reference numerals as in the first embodiment are used for the same components, and detailed explanations are omitted.

[0045] In this embodiment, as shown in Figure 6, the automatic driving control unit 20 receives information about the dirt on the sensor surface from the dirt detection unit 18, and can quickly determine which sensor is to be cleaned and to what extent liquid cleaning of the sensor will affect object detection.

[0046] Furthermore, if the sensor surface cannot be removed by liquid cleaning, it is possible to quickly transition to control measures such as reducing the level of automatic operation or stopping automatic operation.

[0047] [Third Embodiment] Next, a third embodiment of the present invention will be described. Figure 7 is a block diagram showing an example of the configuration of the sensor cleaning device 210 according to this embodiment. In this embodiment, the dirt detection unit 18 in the automatic driving ECU 70 differs from the second embodiment in that it does not output information on the dirt on the detected sensor surface to the cleaning control unit 82, but outputs it to the automatic driving control unit 20. However, the other configurations are the same as in the second embodiment, so the same reference numerals as in the second embodiment are used for the same configurations as in the second embodiment, and detailed explanations are omitted.

[0048] In this embodiment, by centrally outputting object detection information from the object detection unit 16 and dirt detection information from the dirt surface detection unit 18 to the automatic driving control unit 20, the automatic driving control unit 20 comprehensively executes the control of the automatic driving of the vehicle 300 and the cleaning of the sensor surface, thereby contributing to suppressing the impact of liquid cleaning on the automatic driving. [Explanation of symbols]

[0049] 10...Automatic driving ECU, 12, 12A, 12B, 12C, 12D...Camera, 14, 14A, 14B, 14C, 14D...LiDAR, 16...Object detection unit, 18...Detection unit, 20...Automatic driving control unit, 22...Cleaning control unit, 24, 24A, 24B, 24C, 24D, 24E, 24F, 24G, 24H...Cleaning actuator, 50...Automatic driving ECU, 62...Cleaning control unit, 70...Automatic driving ECU, 100, 200, 210...Sensor cleaning device, 300...Vehicle

Claims

1. A sensor cleaning device provided on a vehicle (300), An object detection unit (16) detects objects around the vehicle (300) from information acquired by multiple sensors (12A to 12D, 14A to 14D) that monitor the area around the vehicle (300), A dirt detection unit (18) detects dirt on the sensors (12A to 12D, 14A to 14D) and determines whether the dirt is water droplets, When the dirt detection unit (18) determines that water droplets are adhering to the sensor surface of the sensors (12A-12D, 14A-14D), the cleaning control unit (22) continues air cleaning by injecting high-pressure air onto the sensors (12A-12D, 14A-14D) for a predetermined air cleaning time, and controls the liquid cleaning actuators (24A-24H) to inject cleaning liquid onto the sensors (12A-12D, 14A-14D) in accordance with a command from the automatic operation control unit (20), and notifies the automatic operation control unit (20) that the sensors (12A-12D, 14A-14D) are being cleaned with the cleaning liquid. The automatic driving control unit (20) controls the automatic driving of the vehicle (300) using the information of the object detected by the object detection unit (16), and when the dirt detection unit detects dirt on the sensors (12A to 12D, 14A to 14D), and determines that even if the sensors (12A to 12D, 14A to 14D) are covered with cleaning fluid, object detection for automatic driving will not be hindered, and there are no predetermined objects around the vehicle (300) that should not be exposed to cleaning fluid, outputs a command to the cleaning control unit (22) to clean the sensors (12A to 12D, 14A to 14D) with cleaning fluid. Includes, For camera sensors (12A to 12D), the dirt detection unit (18) determines that water droplets are attached to the sensor surface if the brightness difference, which is the difference between the maximum and minimum pixel values ​​of pixels with a predetermined number of pixels or more, is greater than or equal to a predetermined first brightness difference threshold defined as a threshold for determining the presence of foreign matter on the sensor surface, and less than a second brightness difference threshold defined as a threshold for determining that dirt other than water droplets is attached to the sensor surface, and is greater than a second brightness difference threshold that is greater than the first brightness difference threshold. For LiDAR sensors (14A to 14D), the dirt detection unit (18) determines that water droplets are attached to the sensor surface if the amount of laser light reflected from the outside of the vehicle (300) is less than or equal to a predetermined first light intensity threshold defined as a threshold for determining the presence of dirt on the sensor surface, and is greater than a predetermined second light intensity threshold defined as a threshold for determining that dirt other than water droplets is attached to the sensor surface, and is greater than a second light intensity threshold that is less than the first light intensity threshold. When the cleaning control unit (22) is air-cleaning the sensors (12A to 12D, 14A to 14D), it does not notify the automatic operation control unit (20) that the sensors (12A to 12D, 14A to 14D) are being cleaned. Sensor cleaning device.

2. The automatic driving control unit (20) outputs a command to the cleaning control unit (22) to clean the sensors to be cleaned with cleaning solution if some of the multiple sensors (12A to 12D, 14A to 14D) are to be cleaned and the dirt detection unit (18) has not detected any dirt on the sensors other than those to be cleaned. The sensor cleaning device according to claim 1.

3. When the automatic driving control unit (20) is cleaning the sensors (12A to 12D, 14A to 14D) with the cleaning solution, if the object detection unit (16) detects a predetermined object that should not be exposed to the cleaning solution within a certain range from the sensors (12A to 12D, 14A to 14D), the automatic driving control unit (20) interrupts the cleaning of the sensors (12A to 12D, 14A to 14D) with the cleaning solution. The sensor cleaning device according to claim 1.

4. The cleaning control unit (22) continues to spray the cleaning liquid onto the sensors (12A to 12D, 14A to 14D) for a predetermined liquid cleaning time. If the dirt detection unit (18) detects dirt on the sensors (12A-12D, 14A-14D) after continuing to spray the cleaning liquid for the predetermined cleaning time, the automatic driving control unit (20) will perform one of the following actions in automatic driving: limit the vehicle speed, reduce the level of automatic driving, or stop automatic driving. A sensor cleaning device according to any one of claims 1 to 3.

5. The automatic driving control unit (20) reduces the level of automatic driving by switching to driving assistance if the dirt detection unit (18) still detects dirt on the sensors (12A to 12D, 14A to 14D) after continuing to spray the cleaning liquid for the predetermined liquid cleaning time. The sensor cleaning device according to claim 4.

6. When the automatic driving control unit (20) decides to stop automatic driving, it outputs a command to the cleaning control unit (22) to stop cleaning the sensors (12A to 12D, 14A to 14D) with the cleaning solution. The sensor cleaning device according to claim 4.

7. The automatic operation control unit (20) does not recognize the cleaning liquid adhering to the sensors (12A-12D, 14A-14D) as dirt during the predetermined liquid cleaning time. The sensor cleaning device according to claim 6.

8. The automatic operation control unit (20) continues the air cleaning for a predetermined air cleaning time, and if the detection function of the sensors (12A to 12D, 14A to 14D) does not improve, it outputs a command to the cleaning control unit (22) to stop the air cleaning and to clean with cleaning solution. The sensor cleaning device according to claim 1.

9. A sensor cleaning method performed by a sensor cleaning device provided on a vehicle (300), A step of detecting objects around the vehicle (300) from information acquired by multiple sensors (12A to 12D, 14A to 14D) that monitor the area around the vehicle (300), A step of controlling the automatic driving of the vehicle (300) using information on the detected object, The process involves detecting contamination of the sensors (12A to 12D, 14A to 14D) and determining whether the contamination is water droplets or not. In the process of detecting the dirt, if it is determined that water droplets are attached to the sensor surface of the sensor (12A-12D, 14A-14D), air cleaning by injecting high-pressure air onto the sensor (12A-12D, 14A-14D) is continued for a predetermined air cleaning time, and the liquid cleaning actuators (24A-24H) are controlled to inject cleaning liquid onto the sensor (12A-12D, 14A-14D) in accordance with a command from the automatic operation control unit (20), and the automatic operation control unit (20) is notified that the sensor (12A-12D, 14A-14D) is being cleaned with the cleaning liquid, A step of detecting contamination in the sensors (12A to 12D, 14A to 14D) and controlling the liquid cleaning actuators (24A to 24H) to spray the cleaning solution onto the sensors (12A to 12D, 14A to 14D) to clean them, when cleaning with the cleaning solution is possible. Includes, The aforementioned determination process is as follows: For camera sensors (12A to 12D), if the brightness difference, which is the difference between the maximum and minimum pixel values ​​of pixels with a predetermined number of pixels or more, is greater than or equal to a predetermined first brightness difference threshold defined as a threshold for determining whether foreign matter is present on the sensor surface, and less than a second brightness difference threshold defined as a threshold for determining whether dirt other than water droplets is attached to the sensor surface, and is greater than a second brightness difference threshold that is greater than the first brightness difference threshold, then it is determined that water droplets are attached to the sensor surface. For LiDAR sensors (14A to 14D), if the amount of laser light reflected from the outside of the vehicle (300) is less than or equal to a predetermined first light intensity threshold defined as a threshold for determining whether dirt is attached to the sensor surface, and is greater than a predetermined second light intensity threshold defined as a threshold for determining whether dirt other than water droplets is attached to the sensor surface, and is greater than a second light intensity threshold that is less than the first light intensity threshold, then it is determined that water droplets are attached to the sensor surface. In the aforementioned notification step, if the sensors (12A-12D, 14A-14D) are being air-cleaned, the automatic operation control unit (20) is notified that the sensors (12A-12D, 14A-14D) are being cleaned. Sensor cleaning method.

10. A sensor cleaning program executed by a sensor cleaning device provided on a vehicle (300), Computers, An object detection unit detects objects around the vehicle (300) from information acquired by multiple sensors (12A-12D, 14A-14D) that monitor the area around the vehicle (300). A dirt detection unit detects dirt on the sensors (12A-12D, 14A-14D) and determines whether the dirt is water droplets. When the dirt detection unit (18) determines that water droplets are attached to the sensor surface of the sensors (12A-12D, 14A-14D), it controls the air cleaning actuator to spray high-pressure air onto the sensors (12A-12D, 14A-14D) and continues air cleaning for a predetermined air cleaning time, and continues air cleaning according to commands from the automatic driving control unit (20). The cleaning control unit controls the liquid cleaning actuators (24A-24H) to spray cleaning fluid onto sensors (12A-12D, 14A-14D) for cleaning, and notifies the automatic driving control unit (20) that the sensors (12A-12D, 14A-14D) are being cleaned with the cleaning fluid. The cleaning control unit also controls the automatic driving of the vehicle (300) using the object information detected by the object detection unit (16), and when the dirt detection unit (18) detects dirt on the sensors (12A-12D, 14A-14D) and determines that cleaning with the cleaning fluid is possible, it outputs a command to the cleaning control unit (22) to clean the sensors (12A-12D, 14A-14D) with the cleaning fluid. For camera sensors (12A to 12D), the dirt detection unit (18) determines that water droplets are attached to the sensor surface if the brightness difference, which is the difference between the maximum and minimum pixel values ​​of pixels with a predetermined number of pixels or more, is greater than or equal to a predetermined first brightness difference threshold defined as a threshold for determining the presence of foreign matter on the sensor surface, and less than a second brightness difference threshold defined as a threshold for determining that dirt other than water droplets is attached to the sensor surface, and is greater than a second brightness difference threshold that is greater than the first brightness difference threshold. For LiDAR sensors (14A to 14D), the dirt detection unit (18) determines that water droplets are attached to the sensor surface if the amount of laser light reflected from the outside of the vehicle (300) is less than or equal to a predetermined first light intensity threshold defined as a threshold for determining the presence of dirt on the sensor surface, and is greater than a predetermined second light intensity threshold defined as a threshold for determining that dirt other than water droplets is attached to the sensor surface, and is greater than a second light intensity threshold that is less than the first light intensity threshold. When the cleaning control unit (22) is air-cleaning the sensors (12A to 12D, 14A to 14D), it does not notify the automatic operation control unit (20) that the sensors (12A to 12D, 14A to 14D) are being cleaned. Sensor cleaning program.

Images