Vehicle control system

Abstract

The present invention provides a vehicle control device that can provide appropriate information to the driver when the operation of driver assistance functions is affected by poor visibility. [Solution] The vehicle control device 10 includes an imaging device 11 that captures image data of the area around the vehicle, a visibility impairment calculation unit 22 configured to calculate a visibility impairment degree A representing the degree of visibility impairment of the imaging device 11 based on the image data, a driving support unit 21 configured to execute a driving support function for the vehicle based on the image data, and configured to temporarily suspend the driving support function when the visibility impairment degree A calculated by the visibility impairment calculation unit 22 is equal to or greater than a first threshold A1, and a display control unit 23 configured to display the image data on a display device 40 based on the visibility impairment degree A calculated by the visibility impairment calculation unit 22, and configured to change the timing of displaying the image data on the display device 40 according to the operating state of the vehicle.

Description

Technical Field

[0001] The present invention relates to a control device for a vehicle.

Background Art

[0002] Conventionally, in order to detect the surrounding situation of a vehicle, an imaging device mounted on the vehicle and imaging the surroundings of the vehicle is known. Patent Document 1 discloses a device that captures the front and rear of a vehicle using an imaging device for a vehicle, so-called a drive recorder, and displays video data captured of the rear of the vehicle when the vehicle has stopped. In the device described in Patent Document 1, when it is detected that the entire windshield is clouded due to snow, rain, frost, etc. and the visibility is poor, and the vehicle is stopped, the video data is displayed to ensure the safety when the occupant gets out of the vehicle.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Incidentally, image data obtained from an imaging device mounted on a vehicle to capture images of the vehicle's surroundings is sometimes used in devices such as advanced driver-assistance systems (ADAS) that assist the driver's driving operations. Advanced driver-assistance systems are configured to determine the surrounding conditions of the vehicle based on the image data obtained from the imaging device and to execute driving assistance functions according to the surrounding conditions. For example, if the visibility of the imaging device is poor, the imaging device cannot fully perform its function. In this case, the system will have difficulty accurately grasping the situation around the vehicle based on the image data obtained from the imaging device, and the operation of the driving assistance functions will be affected. However, the driver may not understand the situation and may feel anxious or uneasy. Therefore, when the operation of driving assistance functions is affected by poor visibility, it is desirable to provide appropriate information to give the driver a sense of security.

[0005] This invention has been made in view of the above-described circumstances, and its purpose is to provide a vehicle control device that can provide appropriate information to the driver when the operation of the driving assistance function is affected by poor visibility. [Means for solving the problem]

[0006] According to one aspect of the present invention, a vehicle control device includes an imaging device that images the area around a vehicle; a visibility impairment calculation unit configured to calculate a visibility impairment degree representing the degree of visibility impairment of the imaging device based on image data of the area around the vehicle acquired from the imaging device; a driving support unit configured to execute a driving support function for the vehicle based on image data of the area around the vehicle acquired from the imaging device, and to temporarily suspend the driving support function when the visibility impairment degree calculated by the visibility impairment calculation unit is equal to or greater than a first threshold; and a display control unit configured to display the image data acquired by the imaging device on a display device based on the visibility impairment degree calculated by the visibility impairment calculation unit, and to change the timing of displaying the image data on the display device according to the operating state of the vehicle. [Effects of the Invention]

[0007] The vehicle control device according to the present invention can provide appropriate information to the driver when the operation of the driver assistance function is affected by poor visibility. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is a block diagram showing a schematic configuration of a vehicle control device in one embodiment of the present invention. [Figure 2] Figure 2 is a schematic diagram showing an example of the arrangement of an imaging device inside a vehicle's interior. [Figure 3] Figure 3 shows an example of how visibility changes over time. [Modes for carrying out the invention]

[0009] Hereinafter, a vehicle control device according to one embodiment of the present invention will be described in detail with reference to the drawings. Figure 1 is a block diagram showing the schematic configuration of the vehicle control device in this embodiment, and Figure 2 is a schematic diagram showing an example of the arrangement of an imaging device in the passenger compartment of a vehicle on which the control device in this embodiment is installed.

[0010] As shown in Figure 1, the vehicle control device 10 includes, for example, an imaging device 11 for imaging the area around the vehicle, an operation input device 12, a controller 20, a driving module 30, and a display device 40.

[0011] The imaging device 11 is configured to capture images of the area around the vehicle, particularly the area in front of the vehicle, and to detect the surrounding conditions of the vehicle. The imaging device 11 consists of a camera having an image sensor such as a CCD or CMOS, and is configured to output information about the area around the vehicle as still images and / or video. The imaging device 11 can detect the position of road markings that define the vehicle's own lane and adjacent lanes, as well as the presence and relative distance of obstacles such as other vehicles and people around the vehicle.

[0012] When the imaging device 11 is configured to image the area in front of the vehicle, as shown in Figure 2, the imaging device 11 is positioned inside the vehicle 1, in front of the driver's seat 2 and behind the windshield 3, and is installed to image the area outside the vehicle 1 from inside the vehicle 1 through the windshield 3. The imaging device 11 is located at the top of the windshield 3 near the center in the vehicle width direction and at least partially overlaps with the rearview mirror 4 in the vehicle front-rear direction. Image data of the area around the vehicle captured by the imaging device 11 is input to the controller 20.

[0013] The operation input device 12 is configured to receive various operation inputs to the controller 20 from the occupants of the vehicle 1, such as the driver. The operation input device 12 includes, for example, an on / off switch for switching the on / off status of driving assistance functions performed by the controller 20, and a selection switch for selecting the control content of the driving assistance functions. The operation input device 12 is configured, for example, as a steering switch installed on the steering wheel (not shown). The operation input device 12 may also be configured, for example, as a touch panel integrated with the display of a navigation system (not shown). Furthermore, it may be configured to assign multiple functions to a single switch.

[0014] The controller 20 is composed of a computer including, for example, a ROM that stores programs and data, a CPU that performs calculations, a RAM that stores dynamic data and calculation results, and an input / output interface. The controller 20 is configured to perform functions such as the driving support unit 21, the visibility impairment calculation unit 22, the display control unit 23, and the determination unit 24, and to control the entire vehicle control device 10. In Figure 1, the controller 20 is shown to include the driving support unit 21, the visibility impairment calculation unit 22, the display control unit 23, and the determination unit 24, but the driving support unit 21, the visibility impairment calculation unit 22, the display control unit 23, and the determination unit 24 may be implemented as an integrated unit of the controller 20, or they may be implemented as separate controllers. The functions of each part of the controller 20 will be described later.

[0015] The driving module 30 is a module for performing braking and / or steering control of the vehicle 1, and includes the vehicle 1's driving actuators and a controller for controlling the driving actuators. The driving module 30 includes a braking device 31, a drive device 32, and a steering device 33. Based on control commands from the driver assistance unit 21 of the controller 20, the driving module 30 controls the operation of the braking device 31, the drive device 32, and / or the steering device 33 in order to perform the driver assistance functions described later.

[0016] The display device 40 is configured to display information to the occupants of the vehicle in response to commands from the controller 20. The display device 40 includes, for example, an in-vehicle display device 41 mounted on the vehicle 1 and a portable terminal 42 carried by the occupants. The in-vehicle display device 41 has a liquid crystal display, indicator lights, etc., located on the instrument panel at the front of the vehicle 1, and provides information to the occupants visually. The in-vehicle display device 41 may be configured, for example, as the display of a navigation system (not shown). The portable terminal 42 is a small information terminal device having a display unit, and includes, for example, a smartphone or a tablet terminal. The display device 40 displays information to inform the occupants of the operating status of the driver assistance functions, for example.

[0017] The mobile terminal 42 has an application program installed that enables information communication between the vehicle 1, a remote server, and the mobile terminal 42. The mobile terminal 42 is configured to display information representing the status of the vehicle 1 by activating the application program, for example, in response to a command from the controller 20 or a request from the user. The mobile terminal 42 may also be configured to receive messages from the vehicle 1 or the server when the status of the vehicle 1 changes.

[0018] The mobile terminal 42 may include a mobile terminal for the driver and a mobile terminal for the passengers. In the present embodiment, for example, a mobile terminal that matches the phone number registered as the driver in the above application program or the phone number registered as the driver in the system of the vehicle 1 may be recognized as the mobile terminal for the driver, and other mobile terminals may be recognized as the mobile terminals for the passengers.

[0019] Next, the functions of each part of the controller 20 will be described. The driving support unit 21 is configured to execute the driving support function of the vehicle 1 based on the image data around the vehicle acquired from the imaging device 11. Here, the driving support function is a function that executes the control of the driving force and / or the steering control of the vehicle 1 based on the surrounding situation of the vehicle 1 imaged by the imaging device 11. The driving support function is, for example, a function related to an advanced driving support system (ADAS) that supports the driver's driving operation according to the situation around the vehicle 1, and includes, for example, AEB control, ACC control, LKA control, LDP control, etc. The vehicle control device 10 in the present embodiment is configured to be able to execute one or more driving support functions by the driving support unit 21. Since each driving support function is a known vehicle behavior control, the control content will be briefly described below.

[0020] AEB (Autonomous Emergency Braking) control is an automatic braking control that activates an automatic brake, so-called collision damage reduction brake, according to the possibility of collision between the vehicle 1 and an obstacle. The driving support unit 21 determines the possibility of collision between the vehicle 1 and an obstacle based on the information of the obstacle input from the imaging device 11 and sensors (not shown) and the vehicle speed information of the vehicle 1. When the driving support unit 21 determines that the possibility of collision with an obstacle is high, it sends a control command to the driving module 30 to activate the automatic brake.

[0021] ACC (Adaptive Cruise Control) control is a vehicle distance control that performs constant-speed driving according to the set vehicle speed (target vehicle speed) when there is no preceding vehicle in the lane where vehicle 1 is traveling, and performs following driving while maintaining the set vehicle distance when there is a preceding vehicle. The driving support unit 21 transmits a control command to the driving module 30 to perform constant-speed driving or following driving based on, for example, the set vehicle speed and the set vehicle distance input from the operation input device 12, based on the image data of the front area of vehicle 1 input from the imaging device 11.

[0022] LKA (Lane Keeping Assist) control is a lane keeping control that performs steering control so that vehicle 1 can travel in the center of the lane. The driving support unit 21 detects the lane dividing line and the position of the own vehicle of the lane in which vehicle 1 is traveling based on, for example, the image data of the front area of vehicle 1 input from the imaging device 11. The driving support unit 21 calculates a target steering angle for vehicle 1 to travel near the center of the lane based on the position of the own vehicle of vehicle 1 and the vehicle speed, etc., and transmits a control command for realizing the target steering angle to the driving module 30.

[0023] LDP (Lane Departure Prevention) control is a lane departure suppression control that suppresses vehicle 1 from deviating from the driving lane. The driving support unit 21 determines the possibility that vehicle 1 will deviate from the lane based on the lane dividing line and the position of the own vehicle of the lane in which vehicle 1 is traveling. When it is determined that there is a high possibility that vehicle 1 will deviate from the lane, the driving support unit 21 transmits a control command to the driving module 30 to return vehicle 1 to the inside of the lane. The driving support unit 21 may be configured to issue an alarm when it is determined that there is a high possibility that vehicle 1 will deviate from the lane as LDP control.

[0024] The visibility impairment calculation unit 22 is configured to calculate a visibility impairment degree A, which represents the degree of visibility impairment of the imaging device 11, based on image data of the vehicle's surroundings acquired from the imaging device 11. For example, the visibility impairment calculation unit 22 performs image processing on the image data and calculates the visibility impairment degree A based on the brightness, contrast, etc. of the image. If the imaging device 11 is configured as a stereo camera having two cameras, namely a right camera and a left camera, the visibility impairment calculation unit 22 may calculate the visibility impairment degree A based on the difference in brightness of the images from the right camera and the left camera, and the difference in the overall image, etc.

[0025] Visibility Degree A is a value used to evaluate the visibility of the imaging device 11. A smaller Visibility Degree A indicates better visibility for the imaging device 11, while a larger Visibility Degree A indicates poor visibility for the imaging device 11. When the visibility of the imaging device 11 is poor, the imaging device 11 is unable to perform to its full potential. In this state, the image contrast is generally reduced, the image is blurry, and the controller 20 has difficulty accurately recognizing the situation of objects and obstacles in the area in front of the vehicle 1 based on the image data from the imaging device 11.

[0026] The aforementioned driver assistance unit 21 temporarily suspends the driver assistance function when the performance of the imaging device 11 cannot be fully utilized due to poor visibility of the imaging device 11, that is, when the surrounding conditions of the vehicle 1 cannot be accurately recognized. For example, the driver assistance unit 21 is configured to temporarily suspend the driver assistance function when the degree of poor visibility A calculated by the degree of poor visibility calculation unit 22 is equal to or greater than a preset first threshold A1. The driver assistance unit 21 resumes the suspension of the driver assistance function when the degree of poor visibility A falls below the first threshold A1 again. While the driver assistance function is temporarily suspended, it will not be started, and if the driver assistance function is running, it will gradually terminate. The driver is notified that the driver assistance function is suspended by visual information (e.g., display by the display device 40) and / or auditory information (e.g., operation of a buzzer).

[0027] Here, poor visibility of the imaging device 11 can be caused by various factors, including weather conditions such as fog, rain, and snow, dirt on the windshield 3 such as bird droppings or mud splashes, and fogging of the windshield 3. The driver is notified by visual and / or auditory information that the vehicle 1's driver assistance function has been temporarily suspended, but it is difficult for the driver to intuitively understand the content of the notification. The driver may feel anxious or uneasy because they do not understand why the notification is being made or why the driver assistance function has been temporarily suspended.

[0028] Therefore, in this embodiment, the control device 10 of the vehicle 1 is configured to provide information to the driver via the display device 40 when the driver assistance function is temporarily suspended due to poor visibility of the imaging device 11.

[0029] Specifically, the display control unit 24 of the controller 20 is configured to display image data from the imaging device 11 on the display device 40 according to the degree of visibility impairment A calculated by the visibility impairment calculation unit 22. The image data displayed on the display device 40 is an image captured by the imaging device 11 at the time the image data is displayed, and may be a video or a still image. The display control unit 23 is further configured to change the timing of displaying the image data on the display device 40 according to the operating state of the vehicle 1. The display control unit 23 may further change the timing of displaying the image data on the display device 40 according to the cause of the visibility impairment.

[0030] In this embodiment, by displaying the image data currently being captured by the imaging device 11 when visibility is poor, it is expected that the driver will be able to understand the current situation and be prompted to take action to resolve the poor visibility. Furthermore, by displaying the image data while considering the cause of the poor visibility and the operating state of the vehicle 1, information can be provided to the driver at an appropriate time. The following describes the general timing of displaying the image data of the imaging device 11 when the visibility of the imaging device 11 is poor.

[0031] First, we will consider the operating states of vehicle 1. The operating states of vehicle 1 include the state in which the driver assistance function is ON, the state in which the driver assistance function is being executed, and the state in which vehicle 1 is stopped. The state in which the driver assistance function is ON is a state in which the driver assistance function is turned ON, for example, by the operation of the user's operation input device 12. When the driver assistance function is ON, the driver assistance function starts executing when the start conditions for the driver assistance function are met. While the driver assistance function is being executed, the braking and driving force control and / or steering control of vehicle 1 are performed, and the behavior of vehicle 1 is controlled by the driver assistance function. When vehicle 1 is stopped, the vehicle speed of vehicle 1 is 0. When vehicle 1 is stopped, this includes the state in which the ignition switch is ON and the vehicle speed is 0, and the state in which the ignition switch is OFF and the vehicle speed is 0.

[0032] The behavior of Vehicle 1 differs depending on whether the driver assistance function is on, running, or stationary. For example, when the driver assistance function is on, it is preferable to promptly notify the driver that the driver assistance function will be temporarily suspended due to poor visibility. On the other hand, when the driver assistance function is running, the behavior of Vehicle 1 may change suddenly, so it is preferable to provide information to the driver after the vehicle's behavior has stabilized in order to allow the driver to concentrate on driving. Also, when Vehicle 1 is stationary, it does not affect the driver's driving operations, so it is preferable to provide information at an early stage. When Vehicle 1 is stationary, information may be provided in response to a request from the driver.

[0033] In other words, the display control unit 23 changes the timing of the image data display according to the operating state of the vehicle 1. For example, when the driver assistance function is turned on, the display control unit 23 sets the timing of the image data display to be earlier than when the driver assistance function is running. Also, when the vehicle 1 is stopped, the display control unit 23 displays the image data at the same time as, or earlier than, the turn-on state of, the driver assistance function.

[0034] Next, we will consider the causes of poor visibility. The determination unit 24 of the controller 20 is configured to determine the cause of poor visibility in the imaging device 11. As mentioned above, poor visibility in the imaging device 11 can be caused by weather, dirt on the windshield 3, or fogging of the windshield 3. When poor visibility in the imaging device 11 occurs due to weather conditions such as rain, fog, or snow, the driver seated in the driver's seat 2 can see that the weather is bad through the windshield 3.

[0035] The fogging of the windshield 3 occurs due to condensation on the interior side of the windshield 3 caused by temperature differences between the interior and exterior of the vehicle 1. When the fogging of the windshield 3 causes poor visibility of the imaging device 11, it is difficult for the driver seated in the driver's seat 2 to determine the cause of the poor visibility of the imaging device 11. In particular, if only the windshield 3 directly in front of the imaging device 11 is fogged, that is, if only the projection area on the windshield 3 when the imaging device 11 is projected onto the front windshield 3 in the front-rear direction of the vehicle 1 is fogged, the fogging on the windshield 3 will be hidden by the imaging device 11 itself or the rearview mirror 4, making it difficult for the driver seated in the driver's seat 2 to see. Furthermore, the fogging on the windshield 3 tends to accumulate gradually, causing the degree of visibility impairment A to change gradually.

[0036] Unlike the weather conditions and fogging on the windshield 3 described above, dirt on the exterior surface of the windshield 3, such as bird droppings and mud splashes, generally tends to occur suddenly. Thus, when the visibility of the imaging device 11 is poor, there are multiple causes of poor visibility, and the process by which poor visibility occurs also varies. Therefore, it is preferable to provide information to the driver considering not only the degree of poor visibility A of the imaging device 11, but also the cause of the poor visibility.

[0037] The determination unit 24 determines whether the cause of the poor visibility of the imaging device 11 is fogging of the windshield 3, dirt on the windshield 3, or weather conditions. Here, fogging of the windshield 3 is something that gradually accumulates on the windshield 3 in front of the imaging device 11, and will hereafter be referred to as the first cause of poor visibility. Weather conditions are visible to the driver seated in the driver's seat 2 of the vehicle 1 through the windshield 3, and will hereafter be referred to as the second cause of poor visibility.

[0038] The determination unit 24 performs image processing on, for example, the image data of the area in front of the vehicle 1 captured by the imaging device 11 to determine the cause of the poor visibility in the image. Based on the location and extent of the blur in the image, the time change of the blur, the size of the raindrops, the brightness of the image, etc., the determination unit 24 determines whether the cause of the poor visibility captured by the imaging device 11 is fogging of the windshield 3, dirt on the windshield 3, or weather conditions. The determination unit 24 may also determine the cause of the poor visibility by referring to additional information such as the temperature difference between the inside and outside of the vehicle and weather information.

[0039] Since fogging of the windshield 3 is difficult for the driver seated in the driver's seat 2 to see, it is preferable to promptly notify the driver that the driver assistance function will be temporarily suspended due to poor visibility of the imaging device 11. On the other hand, if the cause of poor visibility is weather that is visible to the driver seated in the driver's seat 2, there is no high need to provide information to the driver at an early stage. Therefore, the display control unit 23 displays the image data earlier when the cause of poor visibility is fogging of the windshield 3 (first cause) than when the cause of poor visibility is weather (second cause).

[0040] Since dirt on the exterior side of the windshield 3 tends to occur suddenly, it is desirable to provide information to the driver at the same time that the driver assistance function is temporarily suspended due to poor visibility. Therefore, the display control unit 23 displays the image data at different times depending on whether the cause of poor visibility is fogging of the windshield 3 or weather or dirt on the windshield 3.

[0041] In this way, the display control unit 23 appropriately sets the display timing of image data when visibility is poor, taking into consideration the operating state of the vehicle 1 and / or the cause of poor visibility. The following describes in detail how the display control unit 23 controls the display timing of image data when visibility is poor, taking into consideration the operating state of the vehicle 1 and / or the cause of poor visibility.

[0042] 1. Change the display timing according to the operating status of vehicle 1 and the cause of poor visibility. (1) When the cause of poor visibility is fogging of the windshield 3 (first cause) The fogging of the windshield 3 tends to gradually accumulate in a location that is difficult for the driver seated in the driver's seat 2 to see. Figure 3 shows an example of the time change of the degree of visibility impairment A when the cause of poor visibility for the imaging device 11 is fogging of the windshield 3. As shown in Figure 3, the degree of visibility impairment A gradually increases due to the fogging of the windshield 3, and then gradually decreases. When the degree of visibility impairment A exceeds the first threshold A1 at time t2, the driver assistance function is temporarily suspended. When the degree of visibility impairment A falls below the first threshold A1 at time t3, the temporary suspension of the driver assistance function is released. When the cause of poor visibility is fogging of the windshield 3, the display control unit 23 basically displays the image data at an early timing, and further adjusts the display timing according to the operating state of the vehicle 1 as described below.

[0043] A: Driver assistance function turned on If the cause of poor visibility is fogging of the windshield 3, and the driver assistance function is turned on, i.e., before the driver assistance function is executed, the display control unit 23 displays the image data before the driver assistance function is temporarily suspended due to poor visibility of the imaging device 11.

[0044] For example, when the visibility impairment A exceeds a preset second threshold A2, the display control unit 23 displays the image data from the imaging device 11 on the in-vehicle display device 41. The second threshold A2 is smaller than the first threshold A1 used to determine the temporary suspension of the driver assistance function. As shown in Figure 3, when the visibility impairment A exceeds the second threshold A2 at time t1, the image data is displayed on the in-vehicle display device 41. The display control unit 23 may also display the image data on the display unit of a passenger's mobile terminal at the same time as displaying it on the in-vehicle display device 41. For safe driving operation, the display control unit 23 does not display the image data on the driver's mobile terminal.

[0045] By displaying image data on the in-vehicle display device 41 at a time t1 earlier than the time t2 when the driver assistance function is temporarily suspended, the driver can be notified that the visibility of the imaging device 11 is deteriorating. Upon seeing the image data displayed on the in-vehicle display device 41, the driver can understand that the driver assistance function may be temporarily suspended soon. Furthermore, it is expected that the driver will be prompted to take action to resolve the poor visibility of the imaging device 11, such as activating the wipers or stopping the vehicle 1 and wiping away the fog from the windshield 3. If the image data is displayed on a passenger's mobile device, the driver can be informed by the passenger that the visibility is poor due to fog on the windshield 3, enabling them to understand the current situation and take action to resolve the poor visibility of the imaging device 11.

[0046] The display control unit 23 terminates the display of image data on the in-vehicle display device 41 when the visibility impairment A decreases to less than the second threshold A2. Alternatively, the display control unit 23 may terminate the display of image data when the visibility impairment A falls below the first threshold A1.

[0047] B: Driver assistance function is running. While the driver assistance function is running, the behavior of vehicle 1 may change suddenly due to braking force control and / or steering control. Therefore, if the cause of poor visibility is fogging of the windshield 3, the display control unit 23 displays image data on the in-vehicle display device 41 if the degree of poor visibility A is equal to or greater than the second threshold A2 and the amount of control of vehicle 1 by the driver assistance function is less than or equal to a predetermined value. The amount of control of vehicle 1 by the driver assistance function is, for example, the acceleration or deceleration when accelerating or decelerating vehicle 1 to follow a preceding vehicle while ACC control is running.

[0048] The display control unit 23 displays image data on the in-vehicle display device 41 when, for example, ACC control is in operation and the degree of visibility impairment A is equal to or greater than the second threshold A2, and the acceleration / deceleration of the vehicle 1 due to ACC control is less than or equal to a predetermined value (for example, predetermined value = 0). When constant speed driving is being performed while ACC control is in operation, the acceleration / deceleration of the vehicle 1 due to ACC control is less than or equal to the predetermined value, so the display control unit 23 displays image data on the in-vehicle display device 41 when the degree of visibility impairment A is equal to or greater than the second threshold A2. The display control unit 23 may also display image data on the display unit of a passenger's mobile terminal at the same time as displaying it on the in-vehicle display device 41. For safe driving operation, the display control unit 23 does not display image data on the driver's mobile terminal.

[0049] Alternatively, the display control unit 23 may display image data on the in-vehicle display device 41 when the degree of visibility impairment A exceeds a first threshold A1, regardless of the amount of control of the vehicle 1 by the driver assistance function. In this case, as shown in Figure 3, the image data is displayed on the in-vehicle display device 41 during the time t2 when the driver assistance function is temporarily suspended.

[0050] The display control unit 23 terminates the display of image data on the in-vehicle display device 41 when the visibility impairment A decreases to less than the second threshold A2. Alternatively, the display control unit 23 may terminate the display of image data when the visibility impairment A falls below the first threshold A1.

[0051] C: Vehicle 1 is stopped. If the cause of poor visibility is fogging of the windshield 3, the image data is displayed at the same timing as when the above-mentioned driver assistance function is turned on, while the vehicle 1 is stopped. When the vehicle 1 is stopped, even if the driver stares at the image data displayed on the in-vehicle display device 41, it will not affect the driving operation. Therefore, by displaying the image data in advance before the driver assistance function is temporarily suspended due to the visibility impairment A exceeding the first threshold A1, the driver is made aware that the imaging device 11 has poor visibility. For example, when the visibility impairment A exceeds the above-mentioned second threshold A2, the display control unit 23 displays the image data on the in-vehicle display device 41.

[0052] The display control unit 23 displays image data on the display unit of the mobile terminal 42 when the application program of the mobile terminal 42 is started, that is, in response to a request from the user. In this case, the display control unit 23 displays the image data captured by the imaging device 11 at the time the image data is displayed on the in-vehicle display device 41. In addition to this image data, or instead, the display control unit 23 may display the current image data being captured by the imaging device 11 at the time the application program of the mobile terminal 42 is started on the display unit of the mobile terminal 42.

[0053] If vehicle 1 is stationary and the ignition switch is off, image data will be displayed on the display of the mobile terminal 42 in response to a request from the user. The user can view the image data displayed on the display of the mobile terminal 42 by, for example, voluntarily launching an application program on the mobile terminal 42 or by responding to a message sent from vehicle 1 or the server.

[0054] (2) When the cause of poor visibility is weather (second cause) or dirt on the windshield 3 In adverse weather conditions such as rain, fog, or snow, the driver seated in the driver's seat 2 can understand the cause of the poor visibility of the imaging device 11 through the windshield 3. Also, dirt on the windshield 3, such as bird droppings or mud splashes, does not accumulate gradually but occurs suddenly. When the degree of poor visibility A of the imaging device 11 exceeds the first threshold A1 due to weather or dirt on the windshield 3, the driver assistance function is temporarily suspended, and when the degree of poor visibility A falls below the first threshold A1, the suspension of the driver assistance function is resumed. In cases of poor visibility where the driver can infer the cause through the windshield 3, or in cases of poor visibility that occurs suddenly, the display control unit 23 displays the image data at the same time as the temporary suspension of the driver assistance function.

[0055] A: Driver assistance function turned on When the cause of poor visibility is weather (secondary cause) or dirt on the windshield 3, and the driver assistance function is turned on, i.e., before the driver assistance function is executed, the display control unit 23 displays the image data from the imaging device 11 on the in-vehicle display device 41 when the degree of poor visibility A exceeds the first threshold A1. If the cause of poor visibility is weather, the driver understands that it is bad weather, so displaying the image data at an early stage helps to prevent the driver's attention from becoming distracted. In the case of sudden dirt on the windshield 3, it is considered that there is almost no time difference in the rise of the degree of poor visibility A from the second threshold A2 to the first threshold A1, so the image data is displayed when the degree of poor visibility A exceeds the first threshold A1.

[0056] The display control unit 23 may display image data on the in-vehicle display device 41 and simultaneously on the display unit of a passenger's mobile terminal. For safe driving, the display control unit 23 does not display image data on the driver's mobile terminal. When the visibility impairment A decreases to below the first threshold A1, the display control unit 23 terminates the display of image data on the in-vehicle display device 41.

[0057] B: Driver assistance function is running. If the cause of poor visibility is weather (secondary cause) or dirt on the windshield 3, and the driver assistance function is running, the display control unit 23 displays the image data from the imaging device 11 on the in-vehicle display device 41 when the degree of poor visibility A becomes equal to or greater than the first threshold A1, similar to when the driver assistance function is turned on. The display control unit 23 may also display the image data on the display unit of a passenger's mobile terminal at the same time as displaying it on the in-vehicle display device 41. For safe driving, the display control unit 23 does not display the image data on the driver's mobile terminal. When the degree of poor visibility A decreases to less than the first threshold A1, the display control unit 23 terminates the display of the image data on the in-vehicle display device 41.

[0058] C: Vehicle 1 is stopped. If the cause of poor visibility is weather (second cause) or dirt on the windshield 3, while the vehicle 1 is stationary, if the degree of poor visibility A exceeds the first threshold A1, the image data from the imaging device 11 is displayed on the in-vehicle display device 41, similar to the ON state of the driving assistance function described above. The display control unit 23 displays image data on the display unit of the mobile terminal 42 when the application program of the mobile terminal 42 is started, that is, in response to a request from the user. In this case, the display control unit 23 displays the image data captured by the imaging device 11 at the time the image data is displayed on the in-vehicle display device 41. In addition to this image data, or instead, the display control unit 23 may display the current image data being captured by the imaging device 11 at the time the application program of the mobile terminal 42 is started on the display unit of the mobile terminal 42.

[0059] If vehicle 1 is stationary and the ignition switch is off, image data will be displayed on the display of the mobile terminal 42 in response to a request from the user. The user can view the image data displayed on the display of the mobile terminal 42 by, for example, voluntarily launching an application program on the mobile terminal 42 or by responding to a message sent from vehicle 1 or the server.

[0060] 2. Changing the display timing according to the operating status of vehicle 1. The display control unit 23 may change the display timing of image data according only to the operating state of vehicle 1, regardless of the cause of poor visibility. When the driver assistance function is turned on, the display control unit 23 is set to display image data earlier than when the driver assistance function is running. Also, when vehicle 1 is stopped, the display control unit 23 displays image data at an earlier timing than when the driver assistance function is turned on. The display timing of image data in the event of poor visibility according to the operating state of vehicle 1 is described below.

[0061] A: Driver assistance function turned on When the driver assistance function is turned on, the display control unit 23 displays the image data from the imaging device 11 on the in-vehicle display device 41 if the visibility impairment A exceeds the second threshold A2 and this condition continues for a predetermined time T1.

[0062] B: Driver assistance function is running. The display control unit 23 displays image data on the in-vehicle display device 41 when the visibility impairment A is greater than or equal to the second threshold A2 and the amount of control of the vehicle 1 by the driving assistance function is less than or equal to a predetermined value. Alternatively, the display control unit 23 displays image data on the in-vehicle display device 41 when the visibility impairment A is greater than or equal to the first threshold A1 when the driving assistance function is being executed.

[0063] C: Vehicle 1 is stopped. When the vehicle 1 is stopped, the display control unit 23 displays the image data from the imaging device 11 on the in-vehicle display device 41 if the visibility impairment A exceeds the second threshold A2. The display control unit 23 displays the image data whenever the visibility impairment A exceeds the second threshold, regardless of the duration of the state where the visibility impairment A exceeds the second threshold A2. As a result, when the vehicle 1 is stopped, the image data from the imaging device 11 is displayed on the in-vehicle display device 41 at an earlier timing than when the driver assistance function is turned on.

[0064] Alternatively, instead of determining the duration of the state in which the visibility impairment A is equal to or greater than the second threshold A2, the display control unit 23 may set a third threshold A3 for determining the display timing while the vehicle 1 is stopped. The third threshold A3 is smaller than the second threshold A2. When the vehicle 1 is stopped, if the visibility impairment A is equal to or greater than the third threshold A3, the display control unit 23 displays the image data from the imaging device 11 on the in-vehicle display device 41. When the driver assistance function is turned on, if the visibility impairment A is equal to or greater than the second threshold A2, the display control unit 23 displays the image data from the imaging device 11 on the in-vehicle display device 41. As a result, when the vehicle 1 is stopped, the image data from the imaging device 11 is displayed on the in-vehicle display device 41 at an earlier timing than when the driver assistance function is turned on.

[0065] The display control unit 23 may further differentiate the display timing of the image data from the imaging device 11 depending on whether the vehicle 1 is traveling on a road (including expressways, etc., so-called public roads) or on an area other than a road (including pathways in a parking lot, etc.). When the vehicle 1 is traveling on an area other than a road, the display control unit 23 may display the image data from the imaging device 11 on the in-vehicle display device 41 at an earlier timing than when the vehicle 1 is traveling on a road. For example, when the driver assistance function is turned on and the vehicle 1 is traveling on an area other than a road, the display control unit 23 displays the image data from the imaging device 11 on the in-vehicle display device 41 at the same timing as when the vehicle 1 is stopped. When the vehicle 1 is traveling on an area other than a road, such as a pathway in a parking lot, the vehicle speed of the vehicle 1 is generally lower than when the vehicle 1 is traveling on a road, and the driver can be expected to operate the vehicle safely. Therefore, by displaying the image data at an earlier timing, it is possible to make the driver understand that the visibility of the imaging device 11 is poor before the driver assistance function is temporarily suspended.

[0066] The control device 10 of the vehicle 1 according to the embodiment described above can achieve the following effects.

[0067] (1) The control device 10 of the vehicle 1 includes an imaging device 11 that images the area around the vehicle 1, a visibility impairment calculation unit 22 configured to calculate a visibility impairment degree A representing the degree of visibility impairment of the imaging device 11 based on the image data of the area around the vehicle acquired from the imaging device 11, a driving support unit 21 configured to execute driving support functions of the vehicle 1 based on the image data of the area around the vehicle acquired from the imaging device 11, and configured to temporarily suspend the driving support functions when the visibility impairment degree A calculated by the visibility impairment calculation unit 22 is equal to or greater than a first threshold A1, and a display control unit 23 configured to display the image data acquired by the imaging device 11 on a display device 40 based on the visibility impairment degree A calculated by the visibility impairment calculation unit 22, and configured to change the timing of displaying the image data on the display device 40 according to the operating state of the vehicle 1.

[0068] If the visibility of the imaging device 11 is poor and the operation of the driver assistance function is affected, that is, if the degree of poor visibility A of the imaging device 11 exceeds the first threshold A1 and the driver assistance function is temporarily suspended, the driver of vehicle 1 may not understand the situation and may feel anxious or uneasy. Therefore, in the control device 10 of vehicle 1 in this embodiment, by displaying the image data acquired by the imaging device 11 on the display device 40 based on the degree of poor visibility A of the imaging device 11, information can be provided to help the driver of vehicle 1 understand the current situation. By displaying the image data from the imaging device 11, it is expected that the driver will be prompted to take action to resolve the poor visibility. Furthermore, by changing the timing of image display according to the operating state of vehicle 1, image data can be displayed at an appropriate time.

[0069] (2) The driver assistance unit 21 is configured to perform braking force control and / or steering control of the vehicle 1 based on the surrounding conditions of the vehicle 1 captured by the imaging device 11 as a driver assistance function. The operating state of the vehicle 1 includes the state in which the driver assistance function is turned on, the state in which the driver assistance function is being executed, and the state in which the vehicle is stopped. The behavior of the vehicle 1 differs in the state in which the driver assistance function that performs braking force control and / or steering control of the vehicle 1 is turned on, is being executed, and the state in which the vehicle 1 is stopped. By changing the display timing of the image data in consideration of the operating state of the vehicle 1, the display timing can be determined in consideration of the balance between the importance of providing information to the driver and the importance of allowing the driver to concentrate on driving operations when the behavior of the vehicle 1 changes suddenly.

[0070] (3) The display control unit 23 sets the timing of image data display to be earlier than when the driver assistance function is turned on when the vehicle 1 is stopped, and earlier than when the driver assistance function is running when the driver assistance function is turned on. When the vehicle 1 is stopped, the driver can look at the image data displayed on the display device 40 without affecting driving operations, so by displaying the image data at an early timing, the driver can be quickly notified that the view of the imaging device 11 is poor. On the other hand, when the driver assistance function is running, the behavior of the vehicle 1 changes suddenly, so the timing of image data display is delayed in order to allow the driver to concentrate on driving operations.

[0071] (4) The control device 10 of the vehicle 1 further includes a determination unit 24 configured to determine the cause of poor visibility of the imaging device 11 based on image data. The display control unit 23 further changes the timing of displaying the image data on the display device 40 according to the cause of poor visibility determined by the determination unit 24. Poor visibility of the imaging device 11 can occur due to various causes, such as weather conditions like rain, fog, and snow, or fogging of the windshield 3. Therefore, by changing the timing of displaying the image data according to the cause of poor visibility, the image data can be displayed at an appropriate timing that takes into account the balance between providing information and safe driving operations.

[0072] (5) The determination unit 24 is configured to determine whether the poor visibility of the imaging device 11 is due to a first cause occurring only on the windshield 3 in front of the imaging device 11, or to a second cause that can be seen by the driver seated in the driver's seat 2 of the vehicle 1 through the windshield 3. The display control unit 23 displays the image data earlier if the cause of the poor visibility is the first cause, compared to when the cause of the poor visibility is the second cause. Poor visibility due to the first cause occurring only on the windshield 3 in front of the imaging device 11 is difficult for the driver seated in the driver's seat 2 to see, and it is difficult for the driver to intuitively understand the reason even if the driving assistance function is temporarily suspended. Therefore, when poor visibility occurs due to the first cause, displaying the image data at an earlier timing helps the driver understand the current situation and suppresses the feeling of anxiety in the driver. On the other hand, when poor visibility occurs due to the second cause that can be seen by the driver through the windshield 3, the display timing is delayed so as not to distract the driver's attention by displaying the image data.

[0073] (6) The determination unit 24 determines that the cause of poor visibility of the imaging device 11 is fogging of the windshield, dirt on the windshield 3, and weather. The display control unit 23 changes the timing of displaying the image data depending on whether the cause of poor visibility is fogging of the windshield 3 or weather or dirt on the windshield. When poor visibility of the imaging device 11 is caused by fogging of the windshield 3, it is difficult for the driver seated in the driver's seat 2 to understand the cause of the poor visibility of the imaging device 11. Also, fogging on the windshield 3 tends to accumulate gradually and cause poor visibility of the imaging device 11. When poor visibility of the imaging device 11 is caused by weather such as rain, fog, or snow, the driver seated in the driver's seat 2 can usually see that the weather is bad through the windshield 3. Dirt on the exterior surface of the windshield 3, such as bird droppings or mud splashes, unlike fogging on the windshield 3, generally tends to occur suddenly. Poor visibility in the imaging device 11 occurs through various processes due to different causes. By considering the cause of the poor visibility in addition to the operating state of the vehicle 1 when determining the display timing, image data can be displayed at the appropriate timing.

[0074] (7) The display control unit 23 is: (i) If the determination unit 24 determines that the cause of poor visibility is weather or dirt on the windshield 3, and the degree of poor visibility A is equal to or greater than the first threshold A1, the image data is displayed on the display device 40. (ii) If the determination unit 24 determines that the cause of poor visibility is fogging of the windshield 3, When the driver assistance function is turned on, if the degree of poor visibility A becomes greater than or equal to the second threshold A2, which is less than the first threshold A1, the image data is displayed on the display device 40. While the driver assistance function is running, if the visibility impairment A is equal to or greater than the second threshold A2, and the amount of vehicle control by the driver assistance function is less than or equal to a predetermined value, the image data is displayed on the display device 40. As a result, if the cause of poor visibility is weather visible to the driver through the windshield 3, or sudden dirt on the windshield 3, the image data is displayed at the same time that the driver assistance function is temporarily suspended. On the other hand, if the cause of poor visibility is fogging of the windshield 3, which is difficult for the driver seated in the driver's seat 2 to see, the image data is displayed earlier when the driver assistance function is on compared to when the driver assistance function is running. This allows the driver to be notified in advance of the possibility that the driver assistance function may be temporarily suspended, giving the driver a sense of security. By making the driver understand that the visibility of the imaging device 11 is poor, it is also possible to encourage the driver to take action to resolve the poor visibility. Furthermore, when the driver assistance function is running, the image data is displayed when the vehicle's control amount is below a predetermined value, so that the image data can be displayed after the behavior of the vehicle 1 has stabilized without interfering with the driver's safe driving operations.

[0075] (8) Alternatively, the display control unit 23 shall (i) If the determination unit 24 determines that the cause of poor visibility is weather or dirt on the windshield 3, and the degree of poor visibility A is equal to or greater than the first threshold A1, the image data is displayed on the display device 40. (ii) If the determination unit 24 determines that the cause of poor visibility is fogging of the windshield 3, When the driver assistance function is turned on, if the degree of poor visibility A becomes greater than or equal to the second threshold A2, which is less than the first threshold A1, the image data is displayed on the display device 40. While the driver assistance function is running, if the visibility impairment A exceeds the first threshold A1, the image data is displayed on the display device 40. As a result, if the cause of poor visibility is weather visible to the driver through the windshield 3, or sudden dirt on the windshield 3, the image data is displayed at the same time that the driver assistance function is temporarily suspended. On the other hand, if the cause of poor visibility is fogging of the windshield 3, which is difficult for the driver seated in the driver's seat 2 to see, the image data is displayed earlier when the driver assistance function is on compared to when the driver assistance function is running. This allows the driver to be notified in advance of the possibility that the driver assistance function may be temporarily suspended, giving the driver a sense of security. By making the driver understand that the visibility of the imaging device 11 is poor, it is also expected that the driver will be encouraged to take action to resolve the poor visibility. When the driver assistance function is running, the image display timing is delayed compared to when the driver assistance function is on, so that the driver can concentrate on driving operations.

[0076] -Variations- (1) In the embodiment described above, the determination unit 24 determined whether the cause of the poor visibility of the imaging device 11 was fogging of the windshield 3, dirt on the windshield 3, or weather. However, it is not limited to this, and the determination unit 24 may be configured to determine whether the cause of the poor visibility of the imaging device 11 is fogging of the windshield 3 (first cause) or weather (second cause).

[0077] (2) In the embodiments described above, an example was described in which the display device 40 includes an in-vehicle display device 41 and the display unit of a mobile terminal 42, but the display device 40 is not limited to this, and may include at least an in-vehicle display device 41.

[0078] (3) The image data of the imaging device 11 displayed on the display device 40 may be the image itself captured by the imaging device 11, or it may be an image that has been processed (for example, enlarged, reduced, cropped, etc.). In addition, text indicating that the imaging device 11 has poor visibility may be displayed along with the image data.

[0079] (4) In the embodiment described above, the imaging device 11 was configured to image the area in front of the vehicle 1 as shown in Figure 2, but it is not limited to this, and the imaging device 11 may be configured to image the area to the side and rear of the vehicle 1 in addition to the area in front of the vehicle 1. In this case, the driving support functions performed by the driving support unit 21 include, for example, a so-called blind spot monitor that notifies the driver of information about obstacles present to the rear and sides of the vehicle 1, and an around-view monitor that assists in parking operations of the vehicle 1.

[0080] Although several embodiments of the present invention have been described above, it should be noted that the present invention is not limited to the above embodiments, and various further modifications and changes are possible within the scope of the present invention. [Explanation of symbols]

[0081] 1 vehicle 2. Driver's seat 10. Vehicle control system 11 Imaging device 20 Controller, 21 Driving support unit, 22 Visibility impairment calculation unit, 23 Display control unit, 24 Judgment unit 40 Display devices, 41 In-vehicle display devices, 42 Mobile terminals

Claims

1. An imaging device that captures images of the area around the vehicle, A visibility impairment calculation unit is configured to calculate a visibility impairment degree representing the degree of visibility impairment of the imaging device based on image data of the vehicle's surroundings acquired from the imaging device, A driving support unit configured to perform a driving support function of the vehicle based on image data of the vehicle's surroundings acquired from the imaging device, and configured to temporarily suspend the driving support function when the degree of poor visibility calculated by the visibility poorness calculation unit is equal to or greater than a first threshold, A display control unit is configured to display image data acquired by the imaging device on a display device based on the degree of visibility loss calculated by the visibility loss calculation unit, and to change the timing of displaying the image data on the display device according to the operating state of the vehicle. A vehicle control device equipped with [a specific feature].

2. The aforementioned driving support unit is configured to perform braking and / or steering control of the vehicle based on the surrounding conditions of the vehicle captured by the imaging device, as a driving support function. The vehicle control device according to claim 1, wherein the operating state of the vehicle includes the state in which the driver assistance function is turned on, the state in which the driver assistance function is being executed, and the state in which the vehicle is stopped.

3. The vehicle control device according to claim 2, wherein the display control unit sets the timing of displaying the image data to be earlier than the ON state of the driver assistance function when the vehicle is stopped, and earlier than the time when the driver assistance function is running when the driver assistance function is ON.

4. The system further includes a determination unit configured to determine the cause of poor visibility in the imaging device based on the image data, The vehicle control device according to claim 1, wherein the display control unit further changes the timing of displaying the image data on the display device according to the cause of the poor visibility determined by the determination unit.

5. The determination unit is configured to determine whether the poor visibility of the imaging device is due to a first cause occurring only on the windshield in front of the imaging device, or to a second cause that is visible to the driver seated in the driver's seat of the vehicle through the windshield. The control device for a vehicle according to claim 4, wherein the display control unit displays the image data earlier when the cause of the poor visibility is the first cause than when the cause of the poor visibility is the second cause.

6. The determination unit determines that the cause of poor visibility in the imaging device is fogging of the windshield, dirt on the windshield, and weather conditions. The vehicle control device according to claim 4, wherein the display control unit causes the timing of displaying the image data to differ depending on whether the cause of the poor visibility is fogging of the windshield or whether the cause of the poor visibility is weather or dirt on the windshield.

7. The aforementioned driving support unit is configured to perform braking and / or steering control of the vehicle based on the surrounding conditions of the vehicle captured by the imaging device, as a driving support function. The operating state of the vehicle includes the state in which the driver assistance function is turned on, the state in which the driver assistance function is being executed, and the state in which the vehicle is stopped. The display control unit, (i) If the determination unit determines that the cause of the poor visibility is weather or dirt on the windshield, and the degree of poor visibility is equal to or greater than the first threshold, the image data is displayed on the display device. (ii) If the determination unit determines that the cause of the poor visibility is fogging of the windshield, When the aforementioned driving assistance function is turned on, if the degree of poor visibility becomes greater than or equal to a second threshold which is less than the first threshold, the image data is displayed on the display device. The vehicle control device according to claim 6, wherein, while the driving assistance function is being performed, the image data is displayed on the display device when the degree of poor visibility is equal to or greater than the second threshold and the amount of vehicle control by the driving assistance function is less than or equal to a predetermined value.

8. The aforementioned driving support unit is configured to perform braking and / or steering control of the vehicle based on the surrounding conditions of the vehicle captured by the imaging device, as a driving support function. The operating state of the vehicle includes the state in which the driver assistance function is turned on, the state in which the driver assistance function is being executed, and the state in which the vehicle is stopped. The display control unit, (i) If the determination unit determines that the cause of the poor visibility is weather or dirt on the windshield, and the degree of poor visibility is equal to or greater than the first threshold, the image data is displayed on the display device. (ii) If the determination unit determines that the cause of the poor visibility is fogging of the windshield, When the aforementioned driving assistance function is turned on, if the degree of poor visibility becomes greater than or equal to a second threshold which is less than the first threshold, the image data is displayed on the display device. The vehicle control device according to claim 6, wherein, while the driving assistance function is being performed, the image data is displayed on the display device when the degree of poor visibility exceeds the first threshold.

Images