Poor visibility determination device

Abstract

To provide a poor visibility determination device capable of reducing time for determining whether the visibility of a camera loaded on a vehicle is poor or not.SOLUTION: A poor visibility determination device comprises: a determination unit 31 which on the bases of a plurality of images generated within a predetermined determination period by a camera 2 loaded on a vehicle 10 to image the periphery of the vehicle 10, determines whether the visibility of the camera 2 in an imaging range is poor or not; and a period setting unit 32 which sets a determination period when at least one of visibility securing apparatuses (6, 7) for securing the visibility of the camera 2 is in operation to be shorter than a determination period when the visibility securing apparatuses (6, 7) are not in operation.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a low visibility determination device that determines the visibility state within the shooting range of an in-vehicle camera. [Background technology]

[0002] A technique has been proposed for determining whether the visibility of a vehicle driver is in a poor visibility state (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-86815 Summary of the Invention [Problem to be solved by the invention]

[0004] For the purpose of autonomous driving control of a vehicle or for assisting a driver of the vehicle, images of the vehicle's surroundings generated by a camera mounted on the vehicle are sometimes used. However, if the camera's field of view becomes poor for some reason, the situation around the vehicle will not be displayed in the image, making it difficult to properly perform autonomous driving control or driving assistance. Therefore, it is desirable to be able to determine whether the camera's field of view is poor as quickly as possible.

[0005] Therefore, an object of the present invention is to provide a poor visibility determination device that can shorten the time required to determine whether the visibility of a camera mounted on a vehicle is poor. [Means for solving the problem]

[0006] According to one embodiment, there is provided a poor visibility determination device that includes a determination unit that is mounted on a vehicle and determines whether visibility within a camera's image capture range is poor based on a plurality of images generated within a predetermined determination period by the camera that captures the surroundings of the vehicle, and a period setting unit that sets the determination period when at least one visibility ensuring device for ensuring the camera's visibility is operating to be shorter than the determination period when the visibility ensuring device is not operating.

[0007] In one embodiment, the at least one visibility securing device includes a wiper, and the period setting unit shortens the determination period as the operation cycle of the wiper becomes shorter.

[0008] In one embodiment, the at least one visibility ensuring device includes a headlight, and the period setting unit sets the determination period when the headlight is turned on and the light distribution direction of the headlight is high to be shorter than the determination period when the light distribution direction is low.

[0009] In one embodiment, the at least one visibility ensuring device includes a wiper and a headlight, and the period setting unit sets the determination period when the headlight is on and the wiper is operating to be shorter than the determination period when the headlight is off or the wiper is not operating.

[0010] In one embodiment, at least one visibility assurance device includes a headlight, and when the headlight is on, the judgment unit judges whether the camera's visibility is poor or not based on the values of individual pixels included in an area corresponding to the headlight illumination range in each of a plurality of images generated within the judgment period. [Effects of the Invention]

[0011] The poor visibility determination device according to the present disclosure has the effect of reducing the time required to determine whether the visibility of a camera mounted on a vehicle is poor. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a schematic configuration diagram of a vehicle control system including a low visibility determination device. [Figure 2] FIG. 2 is a hardware configuration diagram of an ECU that is an example of a poor visibility determination device. [Figure 3] FIG. 2 is a functional block diagram of a processor of an ECU related to vehicle control processing including poor visibility determination processing. [Figure 4] FIG. 10 is a diagram showing an example of a table showing the relationship between the operation modes of headlights and windshield lights and the length of a period for determining poor visibility. [Figure 5] 10 is an operational flowchart of a vehicle control process including a process for determining poor visibility. DETAILED DESCRIPTION OF THE INVENTION

[0013] Below, with reference to the drawings, a description will be given of a poor visibility determination device, a poor visibility determination method executed by the poor visibility determination device, and a computer program for poor visibility determination. This poor visibility determination device determines whether the visibility in the camera's imaging range is poor based on multiple images generated by a camera mounted on a vehicle that captures the vehicle's surroundings within a predetermined determination period. In this case, the poor visibility determination device adjusts the length of the determination period depending on the operating status of at least one visibility assurance device that ensures the camera's visibility. Note that, hereinafter, the field of view in the camera's imaging range may be simply referred to as the camera's field of view.

[0014] FIG. 1 is a schematic diagram of a vehicle control system including a low visibility determination device. The vehicle control system 1 mounted on a vehicle 10 has a camera 2, a rainfall sensor 3, a notification device 4, and an electronic control unit (ECU) 5, which is an example of a low visibility determination device. The camera 2, the rainfall sensor 3, the notification device 4, and the ECU 5 are communicably connected to each other via an in-vehicle network. Furthermore, the ECU 5 is connected to an electronic control unit (BODY-ECU) 8 for controlling headlights 6 and wipers 7 via the in-vehicle network. The headlights 6 and wipers 7 are examples of visibility ensuring devices for ensuring the visibility of the camera 2.

[0015] Camera 2 is mounted inside the cabin of vehicle 10 so as to capture a predetermined imaging range around vehicle 10. Camera 2 captures an image of the predetermined imaging range at predetermined imaging intervals (for example, 1 / 30 to 1 / 10 seconds) and generates an image capturing the imaging range. In this embodiment, the imaging range is the area in front of vehicle 10, and camera 2 is mounted inside the cabin so as to face the front of vehicle 10. Therefore, if the windshield fogs up, the view in the imaging range of camera 2 is obstructed by the fogging, resulting in poor visibility for camera 2. Furthermore, if dirt adheres to the windshield at a position corresponding to the front of camera 2, the dirt will obstruct part or all of the view in the imaging range of camera 2, resulting in poor visibility for camera 2's imaging range. Note that vehicle 10 may be equipped with two or more cameras with different imaging ranges.

[0016] Every time the camera 2 generates an image, it outputs the generated image to the ECU 5 via the in-vehicle network.

[0017] The rainfall sensor 3 is provided, for example, on the windshield of the vehicle 10, and measures the amount of rainfall around the vehicle 10. The rainfall sensor 3 outputs a rainfall signal representing the measured amount of rainfall to the ECU 5 via the in-vehicle network at predetermined intervals.

[0018] The rainfall sensor 3 may be connected to the BODY-ECU 8. In this case, the ECU 5 may acquire the rainfall signal via the BODY-ECU 8.

[0019] The notification device 4 is provided in the passenger compartment of the vehicle 10 and is a device that provides predetermined notifications to the driver by light, sound, text display, or image display. To achieve this, the notification device 4 has, for example, at least one of a speaker, a light source, or a display device. When the notification device 4 receives a notification signal indicating a predetermined notification item to the driver from the ECU 5, it notifies the driver of the notification item by sound from the speaker, by emitting or flashing the light source, or by displaying a message or icon on the display device.

[0020] The ECU 5 is capable of controlling the automatic driving of the vehicle 10 based on the image generated by the camera 2, or assisting the driver of the vehicle 10 in driving. However, if the visibility of the camera 2 is poor, it becomes difficult to perform the automatic driving control and driving assistance. Therefore, the ECU 5 analyzes the image generated by the camera 2 to determine whether the visibility of the camera 2 is poor.

[0021] Fig. 2 is a hardware configuration diagram of the ECU 5. As shown in Fig. 2, the ECU 5 has a communication interface 21, a memory 22, and a processor 23. The communication interface 21, the memory 22, and the processor 23 may be configured as separate circuits, or may be configured integrally as a single integrated circuit.

[0022] The communication interface 21 has an interface circuit for connecting the ECU 5 to the in-vehicle network. The communication interface 21 passes images received from the camera 2 to the processor 23. Similarly, the communication interface 21 passes a rainfall signal received from the rainfall sensor 3 to the processor 23. When the communication interface 21 receives a signal indicating the operation mode of the headlights 6 or a signal indicating the operation mode of the wipers 7 from the BODY-ECU 8, it passes the signal to the processor 23. Furthermore, the communication interface 21 outputs a notification signal for the driver received from the processor 23 to the notification device 4.

[0023] The memory 22 is an example of a storage unit and includes, for example, a volatile semiconductor memory and a non-volatile semiconductor memory. The memory 22 stores various programs and various data used in the poor visibility determination process executed by the processor 23 of the ECU 5. For example, the memory 22 stores various parameters used in the poor visibility determination process of the camera 2 and messages to notify the driver. The memory 22 also stores images received within a recent fixed period, a rainfall signal, the operating mode currently applied to the headlights 6, and the operating mode currently applied to the wipers 7. The memory 22 also temporarily stores various data generated during the poor visibility determination process. The memory 22 also stores various data used for automatic driving control or driving assistance of the vehicle 10.

[0024] The processor 23 includes one or more central processing units (CPUs) and their peripheral circuits. The processor 23 may further include other arithmetic circuits such as a logic unit, a numerical calculation unit, or a graphics processing unit. The processor 23 executes the poor visibility determination process.

[0025] 3 is a functional block diagram of processor 23 relating to vehicle control processing including poor visibility processing. Processor 23 has a determination unit 31, a period setting unit 32, a notification processing unit 33, and a vehicle control unit 34. Each of these units in processor 23 is a functional module realized by, for example, a computer program running on processor 23. Alternatively, each of these units may be a dedicated arithmetic circuit provided separately from each other in processor 23. Of these units, determination unit 31 and period setting unit 32 are particularly related to the poor visibility determination processing.

[0026] The determination unit 31 determines whether the visibility of the camera 2 is in a poor state. In this embodiment, a poor visibility state of the camera 2 refers to a state in which the situation around the vehicle 10 shown in the image generated by the camera 2 is so unclear that it becomes difficult to execute automatic driving control or driving assistance for the vehicle 10. As described above with respect to the camera 2, a poor visibility state of the camera 2 occurs when, for example, the glass in the passenger compartment located in front of the camera 2 (the windshield in this embodiment) becomes fogged, or the glass becomes dirty or scratched.

[0027] The determination unit 31 determines whether the visibility of the camera 2 is poor based on a plurality of images generated by the camera 2 within a determination period. As will be described in detail later, the length of this determination period is set by the period setting unit 32.

[0028] When the window in front of the camera 2 is fogged, it becomes difficult to distinguish between light and dark, reducing the contrast of the image generated by the camera 2. Furthermore, road markings or objects on the road surface around the vehicle 10 appear blurred in the image, reducing the edge strength in the image. Therefore, the determination unit 31 calculates the edge strength of each pixel by performing edge detection filtering, such as a Sobel filter or a Laplacian filter, on each pixel within a predetermined area of the image for each image. The determination unit 31 then calculates a statistical representative value of the edge strength calculated for each pixel for each image. The statistical representative value of the edge strength may be, for example, the average value, median value, or the upper quartile of the edge strength distribution. If the statistical representative value of the edge strength remains below the fog determination threshold for a continuous period of time, the determination unit 31 determines that the visibility of the camera 2 is poor. Alternatively, the judgment unit 31 may judge that the visibility of the camera 2 is poor if the ratio of the number of images in which the statistical representative value of edge intensity is equal to or less than the cloudiness judgment threshold to the total number of images obtained during the judgment period is equal to or greater than a predetermined ratio (e.g., 0.8 to 0.9).

[0029] Alternatively, the determination unit 31 may obtain the luminance distribution of each pixel in a predetermined region for each image, and calculate the difference between the luminance of a predetermined upper ratio (e.g., the top 5%) and the luminance of a predetermined lower ratio (e.g., the bottom 5%) in the luminance distribution as the contrast.The determination unit 31 may then determine that the visibility of the camera 2 is poor if the contrast remains below a predetermined cloudiness determination threshold for a determination period.

[0030] Furthermore, if the glass in front of camera 2 is dirty or scratched, the dirt or scratch will partially or entirely block the field of view of camera 2. As a result, areas without movement or brightness change will appear in the multiple images acquired in time series. Therefore, the determination unit 31 divides each of the multiple images acquired in time series into multiple blocks and performs block matching between the previous and next images in time series for each block. The determination unit 31 then detects, as a block without movement change, a block in which the change in position between the most matching blocks between the previous and next images is less than a predetermined motion determination threshold (e.g., 1 pixel). Furthermore, for the block without movement change, the determination unit 31 calculates the sum of absolute values of the brightness differences between corresponding pixels in the previous and next images. Then, the determination unit 31 detects, among the blocks without movement change, a block in which the sum of absolute values of the brightness differences between the previous and next images is less than a brightness change threshold (e.g., 5 to 10), as a block without time change. If the determination unit 31 continuously detects blocks without time change in the same position over the determination period, the determination unit 31 determines that the field of view of camera 2 is poor.

[0031] The predetermined area may be a fixed area pre-stored in the memory 22, such as the entire image or an area on the image where the road surface is expected to be displayed. Alternatively, the predetermined area may be set to a different area depending on the operation mode of the headlights 6. When the headlights 6 are turned on, the predetermined area may be an area on the image corresponding to the illumination range of the headlights 6. Therefore, the predetermined area when the light distribution direction of the headlights 6 is high is set to be wider than the predetermined area when the light distribution direction of the headlights 6 is low. On the other hand, when the headlights 6 are off, the predetermined area may be an area on the image where the road surface is expected to be displayed. In this case, the determination unit 31 may determine the predetermined area for which the statistical representative value of edge intensity is to be calculated by referring to the latest signal representing the operation mode of the headlights 6 received from the BODY-ECU 8 and information representing the predetermined area corresponding to the operation mode of the headlights 6 that is pre-stored in the memory 22.

[0032] Furthermore, when the wipers 7 are operating, it is preferable to use an image generated immediately after the wiper blades have wiped the windshield in front of the camera 2 for determining poor visibility. This is because if the image is generated immediately after the wiper blades have wiped the windshield in front of the camera 2, it is highly likely that water droplets have been removed from the windshield in front of the camera 2, minimizing the impact of water droplets on the windshield on the camera 2's visibility. Therefore, each time the determination unit 31 receives a signal from the BODY-ECU 8 indicating that the wiper blades are in the reference position, it identifies an image generated by the camera 2 immediately after the elapse of the time required for the wiper blades to pass from the reference position to a position in front of the camera 2, which time depends on the operation mode of the wipers 7. The determination unit 31 then uses the identified image for the poor visibility determination process. The time required for the wiper blades to pass from the reference position to a position in front of the camera 2 is pre-stored in the memory 22 for each operation mode of the wipers 7.

[0033] The determination unit 31 notifies the notification processing unit 33 and the vehicle control unit 34 of the determination result as to whether or not the visibility of the camera 2 is poor.

[0034] The period setting unit 32 sets the length of the determination period used to determine whether the visibility of the camera 2 is poor. In this embodiment, the period setting unit 32 references at least one of the operation mode of the headlights 6 and the operation mode of the wipers 7 to set the length of the determination period. That is, when the headlights 6 are turned on, the headlights 6 illuminate the shooting range of the camera 2, so turning on the headlights 6 is useful for ensuring the visibility of the camera 2. Similarly, when the wipers 7 are operating, raindrops on the windshield in front of the camera 2 are removed, so operation of the wipers 7 is useful for ensuring the visibility of the camera 2. That is, when the headlights 6 are turned on or the wipers 7 are operating, the visibility of the camera 2 is essentially ensured, so even if the determination period is relatively short, it is possible to appropriately determine whether the visibility of the camera 2 is poor. On the other hand, when the headlights 6 are not turned on and the wipers 7 are not operating, even if the visibility of the camera 2 is not poor, the brightness distribution or contrast of the image may temporarily decrease depending on the conditions around the vehicle 10, or an area with no movement or brightness change may temporarily appear in the image. Therefore, in order to properly determine whether the visibility of the camera 2 is poor or not, it is preferable to set the determination period to a relatively long period.

[0035] Therefore, the period setting unit 32 refers to the latest operation mode of the headlights 6 and the latest operation mode of the wipers 7 received from the BODY-ECU 8. Then, the period setting unit 32 sets the determination period so that the determination period when at least one visibility ensuring device is operating is shorter than the determination period when the visibility ensuring device is not operating. In other words, the determination period when the headlights 6 are turned on or the wipers 7 are operating is set shorter than the determination period when the headlights 6 are turned off and the wipers 7 are stopped.

[0036] Furthermore, the period setting unit 32 may set the determination period when the light distribution direction of the headlights 6 is high to be shorter than the determination period when the light distribution direction of the headlights 6 is low. This is because when the light distribution direction is high, the shooting range of the camera 2, i.e., the field of view, is brighter than when the light distribution direction is low, making it easier to determine whether visibility is poor.

[0037] Furthermore, the period setting unit 32 may set the determination period to be shorter as the operation cycle of the wiper 7 is shorter. This is because the shorter the operation cycle of the wiper 7, the higher the possibility that water droplets adhering to the glass in front of the camera 2 have been removed.

[0038] 4 is a diagram showing an example of a table showing the relationship between the length of the determination period and each operation mode of the headlights 6 and the wipers 7. Table 400 is stored in advance in memory 22, and period setting unit 32 can set the length of the determination period according to each operation mode of the headlights 6 and the wipers 7 by referring to table 400.

[0039] In table 400, each column indicates the length of the determination period. Furthermore, the rows, from top to bottom, indicate the lengths of the determination periods when the headlights 6 are off, when the headlights 6 are on and the light distribution direction is low, and when the headlights 6 are on and the light distribution direction is high. Furthermore, the columns, from left to right, indicate the lengths of the determination periods when the wipers 7 are stopped, when the wipers 7 are operating intermittently (INT), when the wipers 7 are operating at a relatively low speed (i.e., when the operating cycle is relatively long), and when the wipers 7 are operating at a relatively high speed (i.e., when the operating cycle is relatively short). As shown in table 400, the faster the operating speed of the wipers 7, the shorter the determination period. It is also shown that the determination period when the headlights 6 are on is set to a shorter period than the determination period when the headlights 6 are off. It can also be seen that when the headlights 6 are turned on, the determination period when the light distribution direction is high is set to a shorter period than the determination period when the light distribution direction is low.

[0040] In table 400, the length of the determination period when the wipers 7 are not operating is set to a constant length regardless of whether the headlights 6 are on or not, and is set to be longer than when the wipers 7 are operating, but this is not limited to this. Even when the wipers 7 are not operating, the determination period may be set to be shorter when the headlights 6 are on than when they are off. Furthermore, even when the wipers 7 are not operating, the determination period may be set to be shorter when the light distribution direction of the headlights 6 is high than when it is low.

[0041] If the amount of rain measured by the rain sensor 3 is less than the lower threshold for operating the wipers 7, it is estimated that the circumstances around the vehicle 10 do not require operating the wipers 7. In this case, the period setting unit 32 sets the length of the determination period in accordance with the operation mode of the headlights 6, regardless of whether the wipers 7 are operating. For example, if the amount of rain measured is less than the lower threshold, the period setting unit 32 sets the length of the determination period in accordance with the operation mode of the headlights 6, regardless of the operation mode of the wipers 7, by referring to the column in the table 400 that corresponds to when the wipers 7 are operating at a relatively high speed.

[0042] Similarly, the illuminance measured by an illuminance sensor (not shown) mounted on the vehicle 10 may indicate that the surroundings of the vehicle 10 are bright enough that it is not necessary to turn on the headlights 6. In this case, the period setting unit 32 may set the length of the determination period in accordance with the operation mode of the wipers 7, regardless of whether the headlights 6 are on or not. For example, if the measured illuminance is equal to or greater than a threshold for turning off the headlights 6, the period setting unit 32 may refer to a row in the table 400 corresponding to when the headlights 6 are on and the light distribution direction is high, regardless of the operation mode of the headlights 6. The period setting unit 32 may then refer to that row to set the length of the determination period in accordance with the operation mode of the wipers 7.

[0043] According to the above process, the period setting unit 32 sets the length of the determination period every time the ignition switch of the vehicle 10 is turned on or the operation mode of the headlights 6 or the operation mode of the wipers 7 is changed. Then, the period setting unit 32 notifies the determination unit 31 of the length of the determination period that has been set.

[0044] When the notification processing unit 33 receives the determination result that the visibility of the camera 2 is poor from the determination unit 31, it notifies the driver via the notification device 4 that the visibility of the camera 2 is poor (hereinafter, may be simply referred to as poor visibility). Alternatively, when the notification processing unit 33 receives the determination result that the visibility of the camera 2 is poor, it may notify the driver via the notification device 4 that the autonomous driving control or driving assistance cannot be continued (hereinafter, may be simply referred to as driving control being unable to be continued). That is, if the notification device 4 has a display device, the notification processing unit 33 causes the display device of the notification device 4 to display a message or icon indicating that the visibility is poor or that driving control is unable to be continued. Furthermore, if the notification device 4 has a speaker, the notification processing unit 33 causes the speaker of the notification device 4 to output an audio signal indicating that the visibility is poor or that driving control is unable to be continued.

[0045] In addition, after receiving a notification of poor visibility or inability to continue driving control, if the notification processing unit 33 receives a determination result from the determination unit 31 that the visibility of the camera 2 is not in a poor state, the notification processing unit 33 stops the notification of poor visibility or inability to continue driving control via the notification device 4.

[0046] The vehicle control unit 34 controls the vehicle 10 autonomously or assists the driver in driving in accordance with the driving control mode currently applied to the vehicle 10 until it receives a determination result from the determination unit 31 that the visibility of the camera 2 is poor. To this end, the vehicle control unit 34 inputs images received from the camera 2 into a classifier to detect objects that may affect the driving of the vehicle 10 (e.g., other vehicles traveling around the vehicle 10, road markings such as lane markings, or pedestrians). The classifier may be configured, for example, by a so-called deep neural network (DNN) having a convolutional neural network architecture or an attention mechanism. Alternatively, the classifier may be a classifier based on a machine learning method other than DNN, such as an adaBoost classifier or a support vector machine. Such a classifier is trained in advance using a predetermined learning method, such as backpropagation, using a large number of training images depicting the object to be detected.

[0047] The vehicle control unit 34 performs automatic driving control so that the vehicle 10 continues traveling along the lane in which it is traveling, based on the detected lane markings. At this time, the vehicle control unit 34 controls the steering of the vehicle 10 so that the vehicle 10 travels along the center of the two lane markings closest to the vehicle 10. Alternatively, when assisting the driver's driving, the vehicle control unit 34 controls the steering of the vehicle 10 so that the vehicle 10 moves away from one of the lane markings when the distance between the vehicle 10 and the lane marking is equal to or less than a predetermined threshold, or warns the driver via the notification device 4 that the vehicle 10 is deviating from the lane. Note that, because the parameters of the camera 2, such as the mounting position, shooting direction, and angle of view of the camera 2, are known, the vehicle control unit 34 can estimate the distance between the vehicle 10 and the lane markings based on the position of the lane markings at the bottom of the image.

[0048] The vehicle control unit 34 also controls the accelerator or brake so that the speed of the vehicle 10 approaches the set target speed. Furthermore, when another vehicle traveling ahead of the vehicle 10 is detected and the inter-vehicle distance between the other vehicle and the vehicle 10 becomes less than a predetermined distance threshold, the vehicle control unit 34 controls the accelerator or brake to decelerate the vehicle 10 so that the inter-vehicle distance becomes equal to or greater than the distance threshold. Note that the position of the bottom edge of the area in which the other vehicle is displayed on the image is assumed to represent the position where the other vehicle is in contact with the road surface, and therefore the vehicle control unit 34 can estimate the inter-vehicle distance between the vehicle 10 and the other vehicle based on the position of the bottom edge of that area on the image and parameters such as the mounting position, shooting direction, and angle of view of the camera 2. Furthermore, if the vehicle 10 is equipped with a ranging sensor such as a LiDAR or radar, the vehicle control unit 34 may estimate the inter-vehicle distance between the vehicle 10 and the other vehicle as the distance measured by the ranging sensor in a direction corresponding to the area in which the other vehicle is displayed on the image.

[0049] Furthermore, when the vehicle control unit 34 receives a determination result from the determination unit 31 that the visibility of the camera 2 is poor, the vehicle control unit 34 stops the automatic driving control and driving assistance after a predetermined period has elapsed since the notification of the determination result, and the control of the vehicle 10 becomes manual driving control by the driver. Note that if a signal indicating that the driver has gripped the steering wheel is not received from a touch sensor (not shown) provided on the steering wheel even after a predetermined period has elapsed since the notification of the determination result that the visibility of the camera 2 is poor, the vehicle control unit 34 may bring the vehicle 10 to an emergency stop.

[0050] FIG. 5 is an operational flowchart of the vehicle control process, including the poor visibility determination process, executed by the processor 23.

[0051] The period setting unit 32 refers to the operation modes currently applied to the headlights 6 and the wipers 7 and determines whether at least one of the headlights 6 and the wipers 7 is operating (step S101). If the headlights 6 are on or the wipers 7 are operating (step S101-Yes), the period setting unit 32 sets a relatively short determination period according to the operation modes of the headlights 6 and the wipers 7 (step S102). On the other hand, if the headlights 6 are off and the wipers 7 are stopped (step S101-No), the period setting unit 32 sets a relatively long determination period (step S103).

[0052] After step S102 or S103, the determination unit 31 determines whether the visibility of the camera 2 is poor based on multiple images generated by the camera 2 within the determination period (step S104). If the visibility of the camera 2 is poor (step S104—Yes), the notification processing unit 33 notifies the driver of poor visibility or the inability to continue driving control via the notification device 4 (step S105). Furthermore, the vehicle control unit 34 stops the automatic driving control of the vehicle 10 and driving assistance for the driver (step S106). On the other hand, if the visibility of the camera 2 is not poor (step S104—No), the vehicle control unit 34 continues the automatic driving control of the vehicle 10 or driving assistance for the driver (step S107). After step S106 or S107, the processor 23 ends the vehicle control process.

[0053] As explained above, this poor visibility determination device adjusts the length of the determination period used to determine poor visibility for the vehicle-mounted camera in accordance with the operating status of at least one visibility ensuring device for ensuring the camera's visibility, thereby reducing the time required to determine whether the camera's visibility is poor.

[0054] The visibility ensuring devices are not limited to headlights and wipers. For example, the visibility ensuring devices may be an air conditioner or a defogger. The period setting unit 32 may set the determination period when the air conditioner is performing a defogging operation to be shorter than the determination period when the defogger is not performing a defogger operation. The period setting unit 32 may also set the determination period when the defogger is operating to be shorter than the determination period when the defogger is not operating. In particular, when the camera to be used for determining poor visibility is a camera mounted inside the vehicle cabin and capturing images of the rear area of the vehicle 10, it is preferable to set the determination period according to the operation mode of the air conditioner or the operation mode of the defogger provided on the rear window rather than the operation mode of the headlights 6.

[0055] In addition, a computer program that realizes the functions of the processor 23 of the ECU 5 according to the above embodiment or variant may be provided in a form recorded on a computer-readable portable recording medium such as a semiconductor memory, a magnetic recording medium or an optical recording medium.

[0056] As described above, those skilled in the art can make various modifications to the embodiments within the scope of the present invention. [Explanation of symbols]

[0057] 1 vehicle control system, 2 camera, 3 rainfall sensor, 4 notification device, 5 electronic control unit (ECU, poor visibility determination device), 6 headlight, 7 wiper, 8 BODY-ECU, 10 vehicle, 21 communication interface, 22 memory, 23 processor, 31 determination unit, 32 period setting unit, 33 notification processing unit, 34 vehicle control unit

Claims

1. a determination unit that is mounted on a vehicle and determines whether or not visibility in a photographing range of a camera is poor based on a plurality of images generated within a predetermined determination period by the camera, the determination unit determining whether or not visibility in a photographing range of the camera is poor; a period setting unit that sets the determination period when at least one visibility ensuring device for ensuring the field of view of the camera is operating to be shorter than the determination period when the visibility ensuring device is not operating; A poor visibility determination device having the following.

2. The visibility deterioration determination device according to claim 1 , wherein the visibility assurance device includes a windshield wiper, and the period setting unit shortens the determination period as the operation cycle of the windshield wiper becomes shorter.

3. 2. The visibility deterioration determination device according to claim 1, wherein the visibility ensuring device includes a headlight, and the period setting unit sets the determination period when the headlight is turned on and the light distribution direction of the headlight is high to be shorter than the determination period when the light distribution direction is low.

4. 2. The visibility deterioration determination device according to claim 1, wherein the visibility ensuring device includes a wiper and a headlight, and the period setting unit sets the determination period when the headlight is on and the wiper is operating to be shorter than the determination period when the headlight is off or the wiper is not operating.

5. The visibility assurance device includes a headlight, and the determination unit determines whether the visibility of the camera is in a poor state based on the values of individual pixels included in an area corresponding to the illumination range of the headlight in each of the plurality of images generated within the determination period when the headlight is on. The visibility assurance device according to any one of claims 1 to 4.

Images