Vehicle driving assistance system, vehicle driving assistance method, and vehicle driving assistance program

Abstract

To provide a vehicle drive assist device that can prevent a drive assist image and an information providing image from becoming difficult to be recognized by an operator of a self vehicle.SOLUTION: A vehicle drive assist device 10 comprises: an image acquisition device 20 that images a specific area around a self vehicle 100 and acquires a specific range image; an image projection device 30 that projects an information providing image for representing information about the self vehicle into the specific area; and a display device 40 that displays the specific range image and a drive assist image to an operator of the self vehicle. The vehicle drive assist device displays the specific range image using the display device and displays the drive assist image superimposed on the specific range image. When both a condition for displaying the specific range image and the drive assist image and a condition for projecting the drive assist image are satisfied, the vehicle drive assist device reduces a visibility of one of the drive assist image and the information providing image in the specific range image and displays the specific range image on the display device.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present invention relates to a vehicle driving support device, a vehicle driving support method, and a vehicle driving support program.

Background Art

[0002] There is known a vehicle driving support device that supports a driving operation by a driver of a host vehicle by imaging an area behind the host vehicle with a camera and displaying an image (camera image) obtained thereby on a display. Also, when a shift lever is set to a reverse range, a camera image showing an area behind the host vehicle is displayed on the display, and a guideline representing an area in which the host vehicle travels is superimposed on the camera image as a driving support image and displayed. Further, there is known a vehicle driving support device that projects a drawing for notifying a person outside the host vehicle of the direction in which the host vehicle travels as an information providing image onto a road surface outside the host vehicle (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

[0004] When a driving support image such as a guideline is displayed superimposed on a camera image, if an information providing image such as a drawing image appears in the camera image, the driving support image and the information providing image overlap, making it difficult for the driver of the host vehicle to recognize both the driving support image and the information providing image.

[0005] An object of the present invention is to provide a vehicle driving support device, a vehicle driving support method, and a vehicle driving support program that can avoid the situation where it becomes difficult for an operator of the host vehicle including the driver to recognize the driving support image and the information providing image.

[0006] The vehicle driving assistance system according to the present invention comprises an image acquisition device, an image projection device, a display device, and a control device. The image acquisition device is a device that captures a specific area around the vehicle and acquires a specific range image. The image projection device is a device that projects an information providing image representing information about the vehicle onto the specific area. The display device is a device that displays the specific range image and a driving assistance image to the operator of the vehicle to assist in driving operations performed on the vehicle. The control device is a device that controls the operation of the image acquisition device, the image projection device, and the display device. Furthermore, the control device is configured to display the specific range image using the display device and to overlay the driving assistance image on the specific range image.

[0007] Furthermore, the control device is configured such that, when both the display conditions for displaying the specific range image and the driving support image and the projection conditions for projecting the information provision image are met, the visibility of either the driving support image or the information provision image in the specific range image is reduced and the specific range image is displayed by the display device.

[0008] When an information-providing image is included in a specific range image and a driving assistance image is displayed superimposed on that specific range image, it becomes difficult for the vehicle operator to recognize both the driving assistance image and the information-providing image. According to the vehicle driving assistance device of the present invention, the visibility of the driving assistance image or the information-providing image in the specific range image is reduced, making it easier for the vehicle operator to recognize the driving assistance image or the information-providing image. Therefore, it is possible to avoid the situation where the driving assistance image and the information-providing image become difficult for the vehicle operator to recognize.

[0009] Furthermore, in the vehicle driving support device according to the present invention, the control device is configured such that, in particular, when both the display condition and the projection condition are met, the visibility of the information providing image in the specific range image is reduced and the specific range image and the driving support image are displayed by the display device.

[0010] According to the vehicle driving assistance device of the present invention, the visibility of the information-providing image in a specific range of image is reduced. Therefore, it is possible to make the driving assistance image easier for the operator of the vehicle to recognize.

[0011] Furthermore, in the vehicle driving support device according to the present invention, the control device is configured such that, in particular, when both the display condition and the projection condition are met, the image acquisition device acquires the specific range image and the image projection device projects the information providing image, and by applying image processing to the specific range image, the visibility of the information providing image in the specific range image is reduced, and the specific range image and the driving support image are displayed on the display device.

[0012] According to the vehicle driving assistance device of the present invention, image processing applied to a specific range of images reduces the visibility of the information-providing image within that specific range. Therefore, it becomes easier for the driver of the vehicle to recognize the driving assistance image.

[0013] Furthermore, in the vehicle driving support device according to the present invention, the control device is configured such that, in particular, when both the display condition and the projection condition are met, the image acquisition device acquires the specific range image and the image projection device projects the information providing image so that the information providing image is not reflected in the specific range image, and the display device displays the specific range image and the driving support image.

[0014] According to the vehicle driving assistance device of the present invention, the information provision image is not included in the specific range image displayed by the display device. Therefore, it is possible to make the driving assistance image easier for the operator of the vehicle to recognize.

[0015] Furthermore, in the vehicle driving support device according to the present invention, the control device is configured such that, in particular, when both the display condition and the projection condition are met, the image acquisition device acquires the specific range image and the image projection device projects the information providing image, and by applying image processing to the specific range image, the difference between the brightness of the information providing image reflected in the specific range image and the brightness of the surrounding portion of the specific range image is reduced, thereby reducing the visibility of the information providing image in the specific range image and displaying the specific range image and the driving support image on the display device.

[0016] According to the vehicle driving assistance device of the present invention, the visibility of the information-providing image in a specific range of image is reduced by decreasing the difference in brightness between the information-providing image and the surrounding image within that range. Therefore, the driving assistance image can be more easily recognized by the operator of the vehicle.

[0017] Furthermore, in the vehicle driving support device according to the present invention, the control device is configured to store the specific range image acquired by the image acquisition device when the information providing image is not being projected. In this case, the control device is configured to, in particular, when both the display condition and the projection condition are met, acquire the specific range image by the image acquisition device and project the information providing image by the image projection device, and replace the portion of the stored image which is the specific range image that corresponds to the information providing image in the currently acquired specific range image with the information providing image in the currently acquired specific range image, thereby reducing the visibility of the information providing image in the specific range image and displaying the specific range image and the driving support image on the display device.

[0018] According to the vehicle driving assistance device of the present invention, the information provision image is not included in the specific range image displayed by the display device. Therefore, it is possible to make the driving assistance image easier for the operator of the vehicle to recognize.

[0019] In the vehicle driving support device according to the present invention, the control device is configured such that, in particular, when both the display condition and the projection condition are met, the timing of acquiring the specific range image by the image acquisition device and the timing of projecting the information providing image by the image projection device are shifted, and the acquisition of the specific range image by the image acquisition device and the projection of the information providing image by the image projection device are performed, and the specific range image and the driving support image are displayed on the display device.

[0020] According to the vehicle driving assistance device of the present invention, the information provision image is not included in the specific range image displayed by the display device. Therefore, it is possible to make the driving assistance image easier for the operator of the vehicle to recognize.

[0021] Furthermore, in the vehicle driving support device according to the present invention, the control device is configured such that, in particular when both the display condition and the projection condition are met and the illuminance outside the vehicle is below a predetermined illuminance threshold, the image acquisition device acquires the specific range image and the image projection device projects the information provision image, and displays the specific range image on the display device without displaying the driving support image on the display device. In this case, the control device is configured such that, in particular when both the display condition and the projection condition are met and the illuminance outside the vehicle is greater than the predetermined illuminance threshold, the image acquisition device acquires the specific range image and displays the specific range image and the driving support image on the display device without projecting the information provision image on the image projection device.

[0022] According to the vehicle driving assistance system of the present invention, when it is dark outside the vehicle, the image projection device projects an information-providing image, so the information-providing image is visible in the image of a specific range, but the driving assistance image is not displayed by the display device. This makes it easier for the driver of the vehicle to recognize the information-providing image. On the other hand, when it is bright outside the vehicle, the image projection device does not project the information-providing image, but the display device does display the driving assistance image. This makes it easier for the driver of the vehicle to recognize the driving assistance image.

[0023] Furthermore, the vehicle driving assistance method according to the present invention includes the steps of: acquiring a specific range image by capturing an image of a specific range including a specific area around the vehicle using an image acquisition device; projecting an information providing image representing information about the vehicle onto the specific range using an image projection device; and displaying the specific range image using a display device, and also displaying a driving assistance image superimposed on the specific range image to assist the operator of the vehicle in driving operations on the vehicle.

[0024] Furthermore, the vehicle driving assistance method according to the present invention includes a step in which, when both the display conditions for displaying the specific range image and the driving assistance image and the projection conditions for projecting the information provision image are met, the visibility of either the driving assistance image or the information provision image in the specific range image is reduced and the specific range image is displayed by the display device.

[0025] According to the vehicle driving assistance method of the present invention, for the same reasons as described above, it is possible to avoid the situation in which the driving assistance image and the information provision image become difficult to recognize by the operator of the vehicle.

[0026] Further, the vehicle driving support program according to the present invention acquires a specific range image by imaging a specific range including a specific area around the host vehicle with an image acquisition device, projects an information-providing image representing information related to the host vehicle onto the specific area with an image projection device, and displays the specific range image with a display device, and is a program that overlays a driving support image for assisting a driving operation on the host vehicle by an operator of the host vehicle on the specific range image and displays it for the operator.

[0027] And, when both the display condition for displaying the specific range image and the driving support image and the projection condition for projecting the information-providing image are satisfied, the vehicle driving support program according to the present invention is configured to reduce the visibility of either the driving support image or the information-providing image in the specific range image and display the specific range image with the display device.

[0028] According to the vehicle driving support program according to the present invention, for the same reason as described above, it is possible to avoid the driving support image and the information-providing image from being difficult for the operator of the host vehicle to recognize.

[0029] The components of the present invention are not limited to the embodiments of the present invention described below with reference to the drawings. Other objects, other features, and attendant advantages of the present invention will be easily understood from the description of the embodiments of the present invention.

Brief Description of Drawings

[0030] [Figure 1] FIG. 1 is a diagram showing a vehicle driving support device according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram showing a scene where an information-providing image such as a drawing is projected. [Figure 3] FIG. 3 is a diagram showing a display in which a driving support image such as a guideline is overlaid on a specific range image and displayed. [Figure 4] FIG. 4 is a diagram showing a display in which a specific range image when an information-providing image such as a drawing is projected is displayed. [Figure 5] Figure 5 is a flowchart showing the routines executed by a vehicle driving assistance device according to an embodiment of the present invention. [Figure 6] Figure 6 is a flowchart showing the routines executed by a vehicle driving assistance device according to an embodiment of the present invention. [Figure 7] Figure 7 shows a display that overlays driving assistance images, such as guidelines, onto a specific area image while projecting informational images such as drawings. [Figure 8] Figure 8 shows a specific range of the image after brightness adjustment image processing has been applied. [Figure 9] Figure 9 is a flowchart showing the routine executed by a vehicle driving assistance device according to a first modified embodiment of the present invention. [Figure 10] Figure 10 shows a specific area image after image processing has been applied. [Figure 11] Figure 11 is a time chart showing the operation of the image acquisition device and the image projection device when the vehicle driving support device according to the first modified embodiment of the present invention performs composite image processing. [Figure 12] Figure 12 is a flowchart showing a routine executed by a vehicle driving assistance device according to a second modified embodiment of the present invention. [Figure 13] Figure 13 is a time chart showing the acquisition of a specific range image and projection of an information provision image performed by a vehicle driving assistance device according to a second modified embodiment of the present invention. [Figure 14] Figure 14 is a flowchart showing a routine executed by a vehicle driving assistance device according to a third modified embodiment of the present invention. [Modes for carrying out the invention]

[0031] Hereinafter, with reference to the drawings, a vehicle driving support device, a vehicle driving support method, and a vehicle driving support program according to embodiments of the present invention will be described. As shown in Figure 1, the vehicle driving support device 10 according to embodiments of the present invention is mounted on the vehicle 100.

[0032] The following explanation of the vehicle driving support device 10 will be based on the example of a case where the operator of the vehicle 100 is a person who is riding in the vehicle 100 and driving the vehicle 100 (i.e., the driver of the vehicle 100).

[0033] However, the operator of the vehicle 100 may be a person who operates the vehicle 100 remotely without being in the vehicle 100 (i.e., a remote operator of the vehicle 100). If the operator of the vehicle 100 is a remote operator, the vehicle driving support device 10 is installed in the vehicle 100 and in the remote control equipment installed outside the vehicle 100 for remote operation of the vehicle 100, and the functions of the vehicle driving support device 10 described below are shared between the vehicle driving support device 10 installed in the vehicle 100 and the vehicle driving support device 10 installed in the remote control equipment.

[0034] Furthermore, the present invention is also applicable to a vehicle driving support device 10 that performs automatic driving control to drive its own vehicle 100 automatically or autonomously.

[0035] The vehicle driving assistance system 10 includes an ECU (electronic control unit) 90 as a control device. The ECU 90 mainly consists of a microcomputer. The microcomputer includes a CPU, ROM, RAM, non-volatile memory and other storage media, as well as interfaces. The CPU is designed to realize various functions by executing instructions, programs, or routines stored in the storage media. In particular, in this example, the vehicle driving assistance system 10 stores a vehicle driving assistance program that performs driving assistance control, which will be described later, in the storage media.

[0036] Furthermore, the vehicle driving assistance system 10 may be configured to update its vehicle driving assistance program via wireless communication with an external device (for example, internet communication).

[0037] Furthermore, although the vehicle driving assistance system 10 is equipped with one ECU, as will be described later, it may be configured to be equipped with multiple ECUs, and the various processes described later may be divided among these ECUs and executed accordingly.

[0038] Furthermore, the vehicle driving assistance system 10 includes an image acquisition device 20, an image projection device 30, and a display device 40. The ECU 90 is a control device that controls the operation of these image acquisition device 20, image projection device 30, and display device 40.

[0039] The image acquisition device 20 is a device that captures a specific range including a specific area around the vehicle 100 and acquires an image of that specific range (specific range image IMG_R), and is, for example, a camera. In particular, in this example, the image acquisition device 20 is a device that captures a specific range including a specific area such as the road surface or ground behind the vehicle 100 and acquires an image of that specific range (specific range image IMG_R).

[0040] The image acquisition device 20 is electrically connected to the ECU 90, and the ECU 90 acquires a specific range image IMG_R from the image acquisition device 20.

[0041] The image projection device 30 is a device that projects images (information provision images IMG_P) such as drawings that represent information about the vehicle 100. In particular, as shown in Figure 2, the image projection device 30 is mounted on the vehicle 100 so that it can project the information provision images IMG_P onto a specific area AREA, such as the road surface or ground, behind the vehicle 100. In this example, the information provision images IMG_P are mainly intended to allow people outside the vehicle 100 to recognize the direction of travel of the vehicle 100.

[0042] The image projection device 30 is electrically connected to the ECU 90, which controls the projection of the information-providing image IMG_P from the image projection device 30.

[0043] Furthermore, the image projection device 30 may be configured to project the information provision image IMG_P using technologies such as lasers or projection mapping. In this case, the vehicle driving support device 10 may be configured to project the information provision image IMG_P with a curve according to the steering angle θ of the vehicle 100.

[0044] The display device 40 is a device that displays various images, and in this example, it is equipped with a display 41. The display 41 is installed in a location inside the vehicle 100 where it can be seen by the driver sitting in the driver's seat of the vehicle 100.

[0045] When predetermined conditions are met, the display device 40 displays a specific range image IMG_R and a driving support image IMG_G on the display 41, as shown in Figure 3. The driving support image IMG_G is an image that assists the driver of the vehicle 100 in driving operations on the vehicle 100. The driving support image IMG_G shown in Figure 3 is a line (guideline) that indicates the range in which the vehicle 100 moves when it is reversing. The guideline indicates the direction of travel and the width of the vehicle 100. In addition, the guideline is displayed curved according to the steering angle of the vehicle 100. Therefore, the guideline also indicates the steering angle of the vehicle 100. The driver can reverse the vehicle 100 while viewing the driving support image IMG_G on the display 41.

[0046] Furthermore, the vehicle driving support system 10 includes a steering angle acquisition device 51, a shift position detection device 61, an illumination acquisition device 71, and a vehicle speed acquisition device 81.

[0047] The steering angle acquisition device 51 is a device that acquires the steering angle of the vehicle 100, and is, for example, a steering angle sensor. In this example, the steering angle of the vehicle 100 is the rotation angle of the steering shaft 52 from the reference position, and the driver of the vehicle 100 can rotate the steering shaft 52 via the steering wheel 53.

[0048] The steering angle acquisition device 51 is electrically connected to the ECU 90, and the ECU 90 acquires the steering angle of the vehicle 100 as the steering angle θ using the steering angle acquisition device 51.

[0049] The shift position detection device 61 is a device that detects the set position of the shift lever 62 mounted on the vehicle 100, and is, for example, a shift position sensor. The shift position detection device 61 is electrically connected to the ECU 90, and the ECU 90 detects the set position of the shift lever 62 by the shift position detection device 61. In this example, the driver of the vehicle 100 can set the shift lever 62 to any of the following positions: forward (D range), reverse (R range), neutral (N range), and park (P range).

[0050] When the shift lever 62 is set to the forward position, driving force is applied to the vehicle 100 so that it moves forward. When the shift lever 62 is set to the reverse position, driving force is applied to the vehicle 100 so that it moves backward. When the shift lever 62 is set to the neutral position, no driving force is applied to the vehicle 100. When the shift lever 62 is set to the parking position, no driving force is applied to the vehicle 100, and the wheels of the vehicle 100 are locked.

[0051] The illuminance acquisition device 71 is a device that acquires the illuminance (brightness) outside the vehicle 100, and is, for example, an illuminance sensor. The illuminance acquisition device 71 is electrically connected to the ECU 90, and the ECU 90 acquires the illuminance outside the vehicle 100 as illuminance BR from the illuminance acquisition device 71.

[0052] The vehicle speed acquisition device 81 is a device that acquires the driving speed of the vehicle 100, and for example, it is equipped with a wheel speed sensor that detects the rotational speed of the wheels of the vehicle 100. The vehicle speed acquisition device 81 is electrically connected to the ECU 90, and the ECU 90 acquires the driving speed of the vehicle 100 as the vehicle speed V from the vehicle speed acquisition device 81.

[0053] <Operation of vehicle driver assistance system> Next, the operation of the vehicle driving support system 10 will be explained. The vehicle driving support system 10 performs driving support control by executing the routine shown in Figure 5 at predetermined calculation intervals. Therefore, at a predetermined timing, the vehicle driving support system 10 starts processing from step S500 of the routine shown in Figure 5 and proceeds to step S505. Driving support control is a control that assists the driving operations performed by the driver of the vehicle 100.

[0054] In this example, display condition CD and projection condition CP are set.

[0055] Display condition CD is the condition under which the display device 40 displays a specific range image IMG_R and a driving support image IMG_G on the display 41. In this example, display condition CD is met when either the first display condition CD1 or the second display condition CD2 is met.

[0056] The first display condition CD1 is the condition that the shift lever 62 is set to the reverse position when the vehicle 100 is stopped (vehicle speed V is zero). The second display condition CD2 is a predetermined condition that does not include the first display condition CD1. In other words, the second display condition CD2 is a condition that does not satisfy the first display condition CD1, and requires the display device 40 to display a specific range image IMG_R and a driving support image IMG_G on the display 41.

[0057] Furthermore, projection condition CP is the condition for projecting the information provision image IMG_P onto a specific area AREA using the image projection device 30. In this example, projection condition CP is met when either the first projection condition CP1 or the second projection condition CP2 is met.

[0058] The first projection condition CP1 is the condition that when the vehicle 100 is stopped (vehicle speed V is zero), the shift lever 62 is set to the reverse position and a person is present behind the vehicle 100 and near the vehicle 100 (within a predetermined distance from the vehicle 100). The second projection condition CP2 is a predetermined condition that does not include the first projection condition CP1. In other words, the second projection condition CP2 is a condition that does not satisfy the first projection condition CP1 and requires the image projection device 30 to project the information provision image IMG_P onto a specific area AREA. Note that the first projection condition CP1 does not necessarily include the condition that a person is present behind the vehicle 100 and within a predetermined distance from the vehicle 100.

[0059] In this example, the vehicle driving support system 10 determines whether or not a person is present behind the vehicle 100 and within a predetermined distance from the vehicle 100 by analyzing the data of the specific range image IMG_R. However, if a known electromagnetic wave sensor, such as a radar sensor, is mounted on the vehicle 100 to emit electromagnetic waves into a specific area AREA of the vehicle 100, the vehicle driving support system 10 may be configured to determine whether or not a person is present behind the vehicle 100 and within a predetermined distance from the vehicle 100 based on the data acquired by the electromagnetic wave sensor.

[0060] When the vehicle driving support device 10 proceeds to step S505, it determines whether both the display condition CD and the projection condition CP are met. At this time, if the vehicle 100 is stopped, the shift lever 62 is set to the reverse position, and there is a person behind the vehicle 100 within a predetermined distance from the vehicle 100, both the first display condition CD1 and the first projection condition CP1 are met, and therefore both the display condition CD and the projection condition CP are met.

[0061] If the vehicle driving assistance system 10 determines "Yes" in step S505, it proceeds to step S510 and executes the routine shown in Figure 6. This routine will be described later.

[0062] On the other hand, if the vehicle driving support device 10 determines "No" in step S505, it proceeds to step S515 to determine whether the display condition CD is met. At this time, since "No" was determined in step S505, the display condition CD is met only when the second display condition CD2 is met.

[0063] If the vehicle driving support system 10 determines "Yes" in step S515, it proceeds to step S520, and as shown in Figure 3, the display device 40 displays the specific range image IMG_R on the display 41, and simultaneously displays the driving support image IMG_G superimposed on the specific range image IMG_R on the display 41. At this time, the projection condition CP is not met, so the information provision image IMG_P is not projected by the image projection device 30. Therefore, the information provision image IMG_P is not reflected in the specific range image IMG_R displayed on the display 41.

[0064] In other words, if display condition CD is met but projection condition CP is not met, the vehicle driving support device 10 does not project the information provision image IMG_P, but instead displays the specific range image IMG_R and the driving support image IMG_G using the display device 40, as shown in Figure 3. At this time, the information provision image IMG_P is not included in the specific range image IMG_R displayed by the display device 40.

[0065] Subsequently, the vehicle driving support system 10 proceeds to step S595 and terminates the processing of this routine.

[0066] On the other hand, if the vehicle driving support device 10 determines "No" in step S515, it proceeds to step S525 to determine whether the projection condition CP is met. At this time, since "No" was determined in step S505, the projection condition CP is met if the second projection condition CP2 is met.

[0067] If the vehicle driving support system 10 determines "Yes" in step S525, it proceeds to step S530 and projects the information provision image IMG_P using the image projection device 30. That is, if the projection condition CP is met but the display condition CD is not met, the vehicle driving support system 10 projects the information provision image IMG_P using the image projection device 30. At this time, neither the display device 40 displays the specific range image IMG_R nor the driving support image IMG_G.

[0068] Subsequently, the vehicle driving support system 10 proceeds to step S595 and terminates the processing of this routine.

[0069] On the other hand, if the vehicle driving support device 10 determines "No" in step S525, it proceeds to step S535. At this time, if the display device 40 is displaying the specific range image IMG_R and the driving support image IMG_G, it terminates the display of these specific range image IMG_R and driving support image IMG_G. If the image projection device 30 is projecting the information provision image IMG_P, it terminates the projection of the information provision image IMG_P.

[0070] Subsequently, the vehicle driving support system 10 proceeds to step S595 and terminates the processing of this routine.

[0071] By the way, when the information provision image IMG_P is visible in the specific range image IMG_R, and the driving assistance image IMG_G is displayed on the display 41 superimposed on the specific range image IMG_R, as shown in Figure 7, the driving assistance image IMG_G and the information provision image IMG_P are displayed in close proximity or overlapping, making it difficult for the driver of the vehicle 100 to recognize the driving assistance image IMG_G.

[0072] Therefore, if both the display condition CD and the projection condition CP are met, that is, if "Yes" is determined in step S505, the vehicle driving support device 10 proceeds to step S510 and executes the routine shown in Figure 6.

[0073] When the vehicle driving support device 10 proceeds to step S510, it starts processing from step S600 of the routine shown in Figure 6, proceeds to step S605, and projects the information providing image IMG_P using the image projection device 30. As a result, the information providing image IMG_P is projected onto the road surface or ground (specific area AREA) behind the vehicle 100.

[0074] Next, the vehicle driving support system 10 proceeds to step S610, where it applies brightness adjustment image processing to the specific range image IMG_R. In this example, as shown in Figure 8, the brightness adjustment image processing is a process that reduces the brightness of the information providing image LUM_P so that the difference (brightness difference ΔLUM) between the brightness of the information providing image IMG_P captured in the specific range image IMG_R (information providing image brightness LUM_P) and the brightness of the surrounding portion of the specific range image IMG_R (ambient brightness LUM_S) becomes zero. As a result, the vehicle driving support system 10 reduces the visibility of the information providing image IMG_P in the specific range image IMG_R. In this example, visibility is the degree to which the driver of the vehicle 100 can recognize the image, and the higher the visibility, the easier it is for the driver to recognize the image.

[0075] Furthermore, the vehicle driving support device 10 may be configured to project the information provision image IMG_P onto a specific area AREA using modulated visible light, such as visible light of a specific frequency or deflected visible light. This makes it possible to easily determine the position of the information provision image IMG_P that is captured in the specific area image IMG_R by analyzing the data of the specific range image IMG_R.

[0076] Next, the vehicle driving support system 10 proceeds to step S615, and the display device 40 displays the specific range image IMG_R after brightness adjustment image processing on the display 41, and simultaneously displays the driving support image IMG_G at normal brightness superimposed on the specific range image IMG_R on the display 41. As a result, the specific range image IMG_R is displayed on the display 41, and the driving support image IMG_G is displayed on the display 41 superimposed on the specific range image IMG_R.

[0077] Furthermore, regarding the driver assistance image IMG_G, the normal brightness is the brightness of the driver assistance image IMG_G displayed by the display device 40 when the display condition CD is met and the projection condition CP is not met. In this example, it is the brightness of the driver assistance image IMG_G displayed by the display device 40 in step S520 of the routine shown in Figure 5.

[0078] Subsequently, the vehicle driving support system 10 proceeds to step S695 and terminates the processing of this routine.

[0079] Thus, in this embodiment of the present invention, the vehicle driving support device 10 is configured to reduce the visibility of the information provision image IMG_P in the specific range image IMG_R and to display the specific range image IMG_R and the driving support image IMG_G on the display device 40 when both the display condition CD and the projection condition CP are met.

[0080] According to this, when the driver assistance image IMG_G is displayed on the display 41, the visibility of the information provision image IMG_P in the specific range image IMG_R is reduced. Therefore, it is possible to avoid the driver difficulty in recognizing the driver assistance image IMG_G.

[0081] In the embodiments of the present invention, the brightness adjustment image processing is an image processing that adjusts the brightness LUM_P of the information-providing image so that the brightness difference ΔLUM becomes zero. However, in the present invention, the brightness adjustment image processing does not need to make the brightness difference ΔLUM zero, as long as it adjusts the brightness LUM_P of the information-providing image so that the brightness difference ΔLUM becomes smaller than the original brightness difference ΔLUM (the brightness difference ΔLUM before the brightness adjustment image processing is applied to the information-providing image IMG_P).

[0082] Accordingly, in the present invention, the vehicle driving support device 10 is configured such that, when both display condition CD and projection condition CP are met, the image acquisition device 20 acquires a specific range image IMG_R and the image projection device 30 projects an information providing image IMG_P, and then performs image processing on the specific range image IMG_R to reduce the difference (luminance difference ΔLUM) between the luminance of the information providing image IMG_P reflected in the specific range image IMG_R (information providing image luminance LUM_P) and the luminance of the surrounding portion of the specific range image IMG_R (ambient luminance LUM_S), thereby reducing the visibility of the information providing image IMG_P in the specific range image IMG_R and displaying the specific range image IMG_R and the driving support image IMG_G on the display device 40. In this case, reducing the luminance difference ΔLUM includes making the luminance difference ΔLUM zero.

[0083] Furthermore, the present invention is not limited to the embodiments described above, and various modifications can be adopted within the scope of the present invention.

[0084] <First variation> For example, as a first modification of the embodiment of the present invention, the vehicle driving support device 10 may be configured to execute the routine shown in Figure 9 instead of the routine shown in Figure 6 when it proceeds to step S510 of the routine shown in Figure 5.

[0085] In this case, the vehicle driving support device 10 proceeds to step S510 of the routine shown in Figure 5, then starts processing from step S900 of the routine shown in Figure 9, proceeds to step S905, and, as in step S605, performs the projection of the information providing image IMG_P by the image projection device 30.

[0086] Next, the vehicle driving support device 10 proceeds to step S910 and performs composite image processing. The composite image processing involves extracting image IMG_C from a specific range image IMG_R, which was acquired and stored immediately before the projection of the information providing image IMG_P in step S905, and then compositing (replacing) the extracted image IMG_C with the portion of the information providing image IMG_P that is visible in the newly acquired specific range image IMG_R, as shown in Figure 10.

[0087] In this case, the vehicle driving support system 10 can employ any of the first to third composite image processing methods as the composite image processing method.

[0088] The first composite image processing involves extracting image IMG_C from a specific range image IMG_R, which was acquired and stored immediately before the projection of the information provision image IMG_P in step S905, and the portion of the information provision image IMG_P and its surrounding area that is captured in the newly acquired specific range image IMG_R. The extracted image IMG_C is then composited with the portion of the information provision image IMG_P and its surrounding area that is captured in the newly acquired specific range image IMG_R. In other words, the first composite image processing involves replacing the portion of the information provision image IMG_P and its surrounding area that is captured in the newly acquired specific range image IMG_R with the image IMG_C extracted as described above.

[0089] The second composite image processing involves extracting only the image IMG_C corresponding to the information providing image IMG_P that is captured in the newly acquired specific range image IMG_R from the specific range image IMG_R that was acquired and stored immediately before the projection of the information providing image IMG_P in step S905, and then compositing the extracted image IMG_C onto the portion of the newly acquired specific range image IMG_R that is captured by the information providing image IMG_P. In other words, the second composite image processing involves replacing the portion of the newly acquired specific range image IMG_R that is captured by the information providing image IMG_P with the image IMG_C extracted as described above.

[0090] The third composite image processing is the process of compositing the entire specific range image IMG_R, which was acquired and stored immediately before the projection of the information provision image IMG_P in step S905, with the specific range image IMG_R acquired this time. In other words, the third composite image processing is the process of replacing the specific range image IMG_R acquired this time with the specific range image IMG_R that was acquired and stored immediately before the projection of the information provision image IMG_P in step S905.

[0091] Next, the vehicle driving support system 10 proceeds to step S915, and the display device 40 displays the specific range image IMG_R after composite image processing on the display 41, and also displays the driving support image IMG_G on the display 41 at normal brightness superimposed on the specific range image IMG_R.

[0092] Subsequently, the vehicle driving support system 10 proceeds to step S995 and terminates the processing of this routine.

[0093] Thus, in this first modified example, the vehicle driving support device 10 is configured to reduce the visibility of the information provision image IMG_P in the specific range image IMG_R and display the specific range image IMG_R and the driving support image IMG_G on the display device 40 when both the display condition CD and the projection condition CP are met. As a result, the information provision image IMG_P is not displayed on the display 41. Therefore, it is possible to avoid the driving support image IMG_G becoming difficult for the driver to recognize.

[0094] In the first modified example, the vehicle driving support device 10 uses a specific range image IMG_R acquired and stored immediately before the projection of the information provision image IMG_P for composite image processing. However, in the present invention, the specific range image IMG_R used for composite image processing is not limited to the specific range image IMG_R acquired and stored immediately before the projection of the information provision image IMG_P, but can be any specific range image IMG_R acquired and stored when the projection of the information provision image IMG_P is not being performed.

[0095] For example, as shown in Figure 11, the vehicle driving support device 10 may continuously acquire a specific range image IMG_R by continuously operating the image acquisition device 20, while simultaneously operating and deactivating the image projection device 30 at predetermined intervals to project and stop the information provision image IMG_P at predetermined intervals, and use the specific range image IMG_R acquired and stored while the projection of the information provision image IMG_P is stopped for composite image processing.

[0096] In this case, the vehicle driving support device 10 uses the specific range image IMG_R acquired and stored between time t111 and time t112 for composite image processing applied to the specific range image IMG_R acquired between time t112 and time t113, uses the specific range image IMG_R acquired and stored between time t113 and time t114 for composite image processing applied to the specific range image IMG_R acquired between time t114 and time t115, and uses the specific range image IMG_R acquired between time t115 and time t116 for composite image processing applied to the specific range image IMG_R acquired between time t116 and time t117.

[0097] Accordingly, in the present invention, the vehicle driving support device 10 stores a specific range image IMG_R acquired by the image acquisition device 20 when the information provision image IMG_P is not projected, and when both the display condition CD and the projection condition CP are met, the image acquisition device 20 acquires the specific range image IMG_R and the image projection device 30 projects the information provision image IMG_P, and replaces the portion of the stored image which is the stored specific range image IMG_R with the information provision image IMG_P in the currently acquired specific range image IMG_R, thereby reducing the visibility of the information provision image IMG_P in the specific range image IMG_R and displaying the specific range image IMG_R and the driving support image IMG_G on the display device 40.

[0098] Furthermore, in the embodiments and first modifications of the present invention, the vehicle driving support device 10 commonly reduces the visibility of the information provision image IMG_P in the specific range image IMG_R by applying image processing to the specific range image IMG_R. In the present invention, the image processing for reducing the visibility of the information provision image IMG_P in the specific range image IMG_R is not limited to the image processing according to the embodiments and first modifications of the present invention, but may be other image processing.

[0099] Accordingly, in the present invention, the vehicle driving support device 10 is configured such that, when both display condition CD and projection condition CP are met, the image acquisition device 20 acquires a specific range image IMG_R and the image projection device 30 projects an information provision image IMG_P, and by applying image processing to the specific range image IMG_R, the visibility of the information provision image IMG_P in the specific range image IMG_R is reduced, and the specific range image IMG_R and the driving support image IMG_G are displayed by the display device 40.

[0100] <Second variation> Furthermore, as a second modification of the embodiment of the present invention, the vehicle driving support device 10 may be configured to execute the routine shown in Figure 12 instead of the routine shown in Figure 6 when it proceeds to step S510 of the routine shown in Figure 5.

[0101] In this case, the vehicle driving support device 10 proceeds to step S510 of the routine shown in Figure 5, then starts processing from step S1200 of the routine shown in Figure 12, proceeds to step S1205, and performs the operation timing adjustment process. The operation timing adjustment process, as shown in Figure 13, is a process in which the timing of acquiring the specific range image IMG_R by the image acquisition device 20 and the timing of projecting the information provision image IMG_P by the image projection device 30 are shifted, and the acquisition of the specific range image IMG_R by the image acquisition device 20 and the projection of the information provision image IMG_P by the image projection device 30 are performed.

[0102] In the example shown in Figure 13, between time t130 and time t131, the image acquisition device 20 is deactivated while the image projection device 30 is activated and the information provision image IMG_P is projected. Subsequently, between time t131 and time t132, the image acquisition device 20 is activated to acquire the specific range image IMG_R while the image projection device 30 is deactivated. Further after that, between time t132 and time t133, the image acquisition device 20 is deactivated while the image projection device 30 is activated and the information provision image IMG_P is projected. Further after that, between time t133 and time t134, the image acquisition device 20 is activated to acquire the specific range image IMG_R while the image projection device 30 is deactivated. Further after that, between time t134 and time t135, the image acquisition device 20 is deactivated while the image projection device 30 is activated and the information provision image IMG_P is projected. Subsequently, between time t135 and time t136, the image acquisition device 20 is activated to acquire the specific range image IMG_R, while the image projection device 30 is deactivated. Further after that, between time t136 and time t137, the image acquisition device 20 is deactivated, while the image projection device 30 is activated to project the information provision image IMG_P.

[0103] Next, the vehicle driving support system 10 proceeds to step S1210, and displays the specific range image IMG_R on the display 41 using the display device 40, and also displays the driving support image IMG_G on the display 41 at normal brightness, superimposed on the specific range image IMG_R.

[0104] Subsequently, the vehicle driving support device 10 proceeds to step S1295 and terminates the processing of this routine.

[0105] Thus, in this second modified example, the vehicle driving support device 10 is configured to reduce the visibility of the information provision image IMG_P in the specific range image IMG_R and display the specific range image IMG_R and the driving support image IMG_G on the display device 40 when both the display condition CD and the projection condition CP are met.

[0106] In particular, in the second modified example, the vehicle driving support device 10 is configured such that, when both display condition CD and projection condition CP are met, the timing of acquiring the specific range image IMG_R by the image acquisition device 20 and the timing of projecting the information provision image IMG_P by the image projection device 30 are staggered, and the acquisition of the specific range image IMG_R by the image acquisition device 20 and the projection of the information provision image IMG_P by the image projection device 30 are performed, and the specific range image IMG_R and the driving support image IMG_G are displayed on the display device 40.

[0107] According to this, the information provision image IMG_P is not displayed on the display 41. Therefore, it is possible to avoid the driver having difficulty recognizing the driver assistance image IMG_G.

[0108] <Third variation> Furthermore, as a third modification of the embodiment of the present invention, the vehicle driving support device 10 may be configured to execute the routine shown in Figure 14 instead of the routine shown in Figure 6 when it proceeds to step S510 of the routine shown in Figure 5.

[0109] In this case, when the vehicle driving support device 10 proceeds to step S510 of the routine shown in Figure 5, it starts processing from step S1400 of the routine shown in Figure 14, proceeds to step S1405, and determines whether the illuminance BR is less than a predetermined value (predetermined illuminance threshold BRth).

[0110] If the vehicle driving support system 10 determines "Yes" in step S1405, it proceeds to step S1410 and projects the information provision image IMG_P. Next, the vehicle driving support system 10 proceeds to step S1415 and displays the specific range image IMG_R. After that, the vehicle driving support system 10 proceeds to step S1495 and terminates the processing of this routine.

[0111] On the other hand, if the vehicle driving support system 10 determines "No" in step S1405, it proceeds to step S1420 and displays the specific range image IMG_R and the driving support image IMG_G. After that, the vehicle driving support system 10 proceeds to step S1495 and terminates the processing of this routine.

[0112] Thus, in this third modified example, the vehicle driving support device 10 is configured to reduce the visibility of the information provision image IMG_P in the specific range image IMG_R and display the specific range image IMG_R and the driving support image IMG_G on the display device 40 when both the display condition CD and the projection condition CP are met.

[0113] In particular, in the third modified example, the vehicle driving support device 10 is configured such that when both display condition CD and projection condition CP are met, and the illuminance BR outside the vehicle 100 is less than or equal to a predetermined illuminance threshold BRth, the image acquisition device 20 acquires a specific range image IMG_R and the image projection device 30 projects an information provision image IMG_P, and displays the specific range image IMG_R on the display device 40 without displaying the driving support image IMG_G. On the other hand, when both display condition CD and projection condition CP are met, and the illuminance BR outside the vehicle 100 is greater than a predetermined illuminance threshold BRth, the image acquisition device 20 acquires a specific range image IMG_R without projecting the information provision image IMG_P on the image projection device 30, and displays the specific range image IMG_R and the driving support image IMG_G on the display device 40.

[0114] According to this, when the driver assistance image IMG_G is displayed on the display 41, the information provision image IMG_P is not displayed. Therefore, it is possible to avoid the driver having difficulty recognizing the driver assistance image IMG_G.

[0115] In addition, in the third modified example, the vehicle driving support device 10 uses illuminance BR to determine whether or not to project the information provision image IMG_P, but it may also be configured not to use illuminance BR.

[0116] In this case, if both display condition CD and projection condition CP are met, the vehicle driving support device 10 is configured to acquire a specific range image IMG_R using the image acquisition device 20 without projecting the information provision image IMG_P using the image projection device 30, regardless of whether the illuminance BR outside the vehicle 100 is greater than a predetermined illuminance threshold BRth, and to display the specific range image IMG_R and the driving support image IMG_G using the display device 40.

[0117] Furthermore, in the second and third modifications of the present invention, the vehicle driving support device 10 commonly ensures that the information provision image IMG_P is not reflected in the specific range image IMG_R. In the present invention, the means for ensuring that the information provision image IMG_P is not reflected in the specific range image IMG_R is not limited to the means relating to the second modification, but may be other means.

[0118] Therefore, in the present invention, the vehicle driving support device 10 should be configured such that, when both display condition CD and projection condition CP are met, the image acquisition device 20 acquires the specific range image IMG_R and the image projection device 30 projects the information providing image IMG_P so that the information providing image IMG_P is not reflected in the specific range image IMG_R, and the display device 40 displays the specific range image IMG_R and the driving support image IMG_G.

[0119] Furthermore, in the embodiments of the present invention described above, as well as the first to third modifications, the vehicle driving support device 10 reduces the visibility of the information provision image IMG_P in the specific range image IMG_R. However, it may also be configured to reduce the visibility of the driving support image IMG_G, which is displayed superimposed on the specific range image IMG_R.

[0120] Therefore, in the present invention, the vehicle driving support device 10 is configured to display the specific range image IMG_R by the display device 40 by reducing the visibility of either the driving support image IMG_G or the information provision image IMG_P in the specific range image IMG_R when both the display condition CD and the projection condition CP are met.

[0121] In this case, the vehicle driving support device 10 may be configured to, for example, project the information provision image IMG_P using the image projection device 30 when both display condition CD and projection condition CP are met, and to display the driving support image IMG_G at a brightness lower than normal using the display device 40 without performing brightness adjustment image processing.

[0122] In other words, the vehicle driving support device 10 can be configured to display the driving support image IMG_G on the display device 40 in such a way that when both display condition CD and projection condition CP are met, it projects the information provision image IMG_P using the image projection device 30, and displays the driving support image IMG_G on the display device 40 at a brightness lower than normal brightness without performing brightness adjustment image processing, so that the difference (brightness difference) between the brightness of the driving support image IMG_G (driving support image brightness) and the brightness of the portion of the specific range image IMG_R surrounding the driving support image IMG_G (ambient brightness) becomes smaller than the normal brightness difference (the difference between the driving support image brightness and ambient brightness when the driving support image IMG_G is displayed at normal brightness).

[0123] Thus, when both display condition CD and projection condition CP are met, the vehicle driving support device 10 may be configured to reduce the visibility of the driving support image IMG_G in the specific range image IMG_R and display the specific range image IMG_R and the driving support image IMG_G on the display device 40.

[0124] Alternatively, in an embodiment of the present invention, the vehicle driving support device 10 may be configured to, for example, project an information provision image IMG_P using the image projection device 30 when both display condition CD and projection condition CP are met, and to display a specific range image IMG_R using the display device 40 without performing brightness adjustment image processing or displaying the driving support image IMG_G using the display device 40.

[0125] Furthermore, the above example is an example of applying the present invention to the operation of the vehicle driving support device 10 when, in a scene in which the vehicle 100 is being reversed, both the conditions for displaying a specific range image IMG_R (an image of a specific range including a specific area behind the vehicle 100 acquired by the image acquisition device 20) and a driving support image IMG_G (an image to support the driver's operation to reverse the vehicle 100) on the display device 40 (display condition CD) and the conditions for projecting an information provision image IMG_P (information about the vehicle 100 onto the specific area behind the vehicle 100) on the image projection device 30 (projection condition CP) are met.

[0126] However, the present invention is also applicable to the operation of the vehicle driving support device 10 when both conditions are met: a display device 40 displays an image of a specific range including a specific area in front of the vehicle 100 acquired by the image acquisition device 20 (specific range image) and an image to support the driver's driving operation to move the vehicle 100 forward (driving support image); and an image projection device 30 projects an information provision image onto a specific area in front of the vehicle 100 (projection condition). [Explanation of Symbols]

[0127] 10...Vehicle driving assistance system, 20...Image acquisition device, 30...Image projection device, 40...Display device, 90...ECU, 100...Own vehicle, AREA...Specific area, IMG_G...Driving assistance image (guideline), IMG_P...Information provision image (drawing), IMG_R...Specific range image

Claims

1. An image acquisition device that captures images of a specific area including a specific region around the vehicle and acquires an image of that specific area. An image projection device that projects an information-providing image representing information about the vehicle onto the specified area. A display device that displays the aforementioned specific range image and a driving support image that assists the operator of the vehicle in driving operations on the vehicle, and The system includes a control device that controls the operation of the image acquisition device, the image projection device, and the display device, The control device is configured to display the specific range image using the display device and to overlay the driving support image onto the specific range image. In a vehicle driving assistance system, The control device is configured such that, when both the display conditions for displaying the specific range image and the driving support image, and the projection conditions for projecting the information provision image, the visibility of either the driving support image or the information provision image in the specific range image is reduced, and the specific range image is displayed by the display device. Vehicle driving assistance system.

2. In the vehicle driving support device according to claim 1, The control device is configured to reduce the visibility of the information-providing image in the specific range image and to display the specific range image and the driving support image on the display device when both the display condition and the projection condition are met. Vehicle driving assistance system.

3. In the vehicle driving support device according to claim 1, The control device is configured such that, when both the display condition and the projection condition are met, it acquires the specific range image using the image acquisition device and projects the information provision image using the image projection device, and then applies image processing to the specific range image to reduce the visibility of the information provision image in the specific range image, thereby displaying the specific range image and the driving support image on the display device. Vehicle driving assistance system.

4. In the vehicle driving support device according to claim 1, The control device is configured such that, when both the display condition and the projection condition are met, the image acquisition device acquires the specific range image and the image projection device projects the information providing image so that the information providing image is not reflected in the specific range image, and the display device displays the specific range image and the driving support image. Vehicle driving assistance system.

5. In the vehicle driving support device according to claim 1, The control device is configured to perform the acquisition of the specific range image by the image acquisition device and the projection of the information providing image by the image projection device when both the display condition and the projection condition are met, and to reduce the visibility of the information providing image in the specific range image by performing image processing on the specific range image to reduce the difference between the brightness of the information providing image captured in the specific range image and the brightness of the surrounding portion of the specific range image, thereby displaying the specific range image and the driving support image on the display device. Vehicle driving assistance system.

6. In the vehicle driving support device according to claim 1, The control device is When the aforementioned information provision image is not projected, the image of the specific range acquired by the image acquisition device is stored. When both the display conditions and the projection conditions are met, the system is configured to acquire the specific range image using the image acquisition device and project the information providing image using the image projection device, and to replace the portion of the stored image, which is the specific range image, that corresponds to the information providing image in the currently acquired specific range image with the information providing image in the currently acquired specific range image, thereby reducing the visibility of the information providing image in the specific range image and displaying the specific range image and the driving support image on the display device. Vehicle driving assistance system.

7. In the vehicle driving support device according to claim 1, The control device is configured such that, when both the display condition and the projection condition are met, it performs the acquisition of the specific range image by the image acquisition device and the projection of the information provision image by the image projection device with a time delay between the timing of the acquisition of the specific range image by the image acquisition device and the projection of the information provision image by the image projection device, and displays the specific range image and the driving support image on the display device. Vehicle driving assistance system.

8. In the vehicle driving support device according to claim 1, The control device is When both the display conditions and the projection conditions are met, and the illuminance outside the vehicle is below a predetermined illuminance threshold, the image acquisition device acquires the specific range image and the image projection device projects the information provision image, and the display device displays the specific range image without displaying the driving support image. When both the display conditions and the projection conditions are met, and the illuminance outside the vehicle is greater than the predetermined illuminance threshold, the system is configured to acquire the specific range image by the image acquisition device without projecting the information provision image by the image projection device, and to display the specific range image and the driving support image on the display device. Vehicle driving assistance system.

9. The process involves using an image acquisition device to capture images of a specific area including a specific region around the vehicle and acquire an image of that specific area. The process involves projecting an information-providing image representing information about the vehicle onto the specified area using an image projection device, A step of displaying the specified range image on a display device, and displaying to the operator the driver of the vehicle an image of driver assistance that assists the driver of the vehicle in operating the vehicle, superimposed on the specified range image. A vehicle driving assistance method having, A vehicle driving assistance method comprising the step of displaying the specific range image by the display device when both the display conditions for displaying the specific range image and the driving assistance image and the projection conditions for projecting the information provision image are met, thereby reducing the visibility of either the driving assistance image or the information provision image in the specific range image.

10. The image acquisition device captures images of a specific area including a specific region around the vehicle and acquires an image of that specific area. The image projection device projects an information-providing image representing information about the vehicle onto the specified area. The display device displays the specified range image and also displays a driving support image superimposed on the specified range image to the operator of the vehicle, which assists in the operator's driving operations of the vehicle. It is a vehicle driving assistance program, When both the display conditions for displaying the specified range image and the driving assistance image, and the projection conditions for projecting the information provision image are met, the system is configured to display the specified range image by the display device while reducing the visibility of either the driving assistance image or the information provision image within the specified range image. Vehicle driving assistance program.

Images