Vehicle driving support image display device

Abstract

To provide an image display device for a vehicle driving support, which is capable of displaying a brighter image on a display as an image for allowing a driver to discriminate a place.SOLUTION: An image display device 10 for a vehicle driving support saves a camera image when a registered point is registered as a registered point image indicating the registered point as a registered camera image, and displays the registered camera image as the registered point image when displaying the registered point image on a display 61. Also, if an image update condition is satisfied under that a current illuminance around the own vehicle is higher than an illuminance around the own vehicle when the registered camera image is saved by the illuminance around the own vehicle at the current point, in the case where the own vehicle is entering an image update target area, which is an area around the registered point, the image display device 10 for a vehicle driving support updates the registered camera image with a camera image captured by a camera 71 at the current point.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a vehicle driving assistance image display device. [Background technology]

[0002] A vehicle driving assistance image display device is known that stores images captured by a camera mounted on the vehicle in advance so that the surroundings of the vehicle can be captured, and then displays the images on a display when a predetermined condition is met, allowing the driver of the vehicle who sees the images to identify the location indicated by the images (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-64104 Summary of the Invention

[0004] For example, if an image captured by a camera is saved at night, the surroundings of the vehicle are dark, and the saved image is also dark. When such a dark image is displayed on the display, even if the driver looks at the displayed image, it may be difficult for the driver to determine the location indicated by the image.

[0005] An object of the present invention is to provide a vehicle driving assistance image display device that can display a brighter image on a display as an image to allow a driver to distinguish a location.

[0006] The vehicle driving assistance image display device of the present invention comprises a camera mounted on the vehicle to capture images of the surroundings of the vehicle, a display mounted on the vehicle so that the driver of the vehicle can see the image, and a control device that displays a driving assistance image on the display, which is an image representing the situation around the vehicle, using the camera image captured by the camera.

[0007] The control device is configured to automatically display the driving assistance image on the display when a driving assistance image display condition is met that the vehicle has entered an automatic image display area, which is an area around a pre-registered registration point.The control device is also configured to save the camera image taken when the registration point is registered as a registration point image indicating the registration point as a registered camera image, and when displaying the registration point image on the display, to display the registered camera image on the display as the registration point image.The control device is also configured to update the registered camera image with the camera image currently captured by the camera when an image update condition is met that the illuminance around the vehicle at the current time is higher than the illuminance around the vehicle when the registered camera image was saved when the vehicle is entering an image update target area, which is an area around the registration point. Furthermore, the image update conditions include a condition that the vehicle speed of the host vehicle is equal to or less than an image update vehicle speed threshold.

[0008] In addition, in the vehicle driving assistance image display device of the present invention, the control device is configured to determine that the image update condition is met, for example, when the illuminance around the vehicle when the registered camera image is saved is equal to or less than a first illuminance threshold and when the illuminance around the vehicle when the vehicle enters the image update target area is equal to or greater than a second illuminance threshold that is set to a value greater than the first illuminance threshold.

[0009] According to the present invention, a brighter registration point image is displayed on the display, which makes it easier for a driver looking at the registration point image displayed on the display to distinguish the location indicated by the registration point image.

[0011] The slower the vehicle is traveling, the clearer the camera image can be acquired. According to the present invention, when the vehicle speed is equal to or lower than the image update vehicle speed threshold, the registered camera image is updated. Therefore, a clearer camera image can be displayed on the display as the registered point image.

[0012] In the vehicle driving assistance image display device according to the present invention, the driving assistance image display condition may include a condition that the vehicle speed of the host vehicle is equal to or less than an image display vehicle speed threshold.

[0013] When the vehicle speed is low, there is a high demand for displaying a driving assistance image on the display. According to the present invention, when the vehicle enters an automatic image display area and the vehicle speed is equal to or lower than an image display speed threshold, the driving assistance image is automatically displayed on the display. Therefore, only when there is a high demand for displaying a driving assistance image on the display, the driving assistance image can be automatically displayed on the display.

[0014] The components of the present invention are not limited to the embodiments of the present invention described below with reference to the drawings. Other objects, features, and attendant advantages of the present invention will be easily understood from the description of the embodiments of the present invention. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a diagram showing a vehicle driving assistance image display device according to an embodiment of the present invention and a vehicle (host vehicle) equipped with the device. [Figure 2] FIG. 2 is a diagram showing the imaging ranges of the front camera and the rear camera. [Figure 3] FIG. 3 is a diagram showing the imaging ranges of the left and right cameras. [Figure 4] FIG. 4A shows a scene in which the vehicle is traveling toward a T-junction, and FIG. 4B shows a scene in which the vehicle has stopped at the T-junction. [Figure 5] (A) of Figure 5 is a diagram showing a display displaying a navigation image, (B) of Figure 5 is a diagram showing a display displaying a display request image, etc., and (C) of Figure 5 is a diagram showing a display displaying a driving assistance image. [Figure 6]FIG. 6A is a diagram showing a scene in which the vehicle is traveling within the registered point range, and FIG. 6B is a diagram showing a scene in which the vehicle has left the registered point range. [Figure 7] FIG. 7(A) is a diagram showing a display that automatically displays a driving assistance image, and FIG. 7(B) is a diagram showing a display that displays a navigation image. [Figure 8] (A) of Figure 8 is a diagram showing a display displaying a navigation image, (B) of Figure 8 is a diagram showing a display displaying a delete operation image, and (C) of Figure 8 is a diagram showing a display displaying a delete confirmation image. [Figure 9] FIG. 9 is a flowchart showing a routine executed by the vehicle driving assistance image display device according to the embodiment of the present invention. [Figure 10] FIG. 10 is a flowchart showing a routine executed by the vehicle driving assistance image display device according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, a vehicle driving assistance image display device according to an embodiment of the present invention will be described with reference to the drawings. The vehicle driving assistance image display device according to the embodiment of the present invention is a device that provides a driver with an image for assisting the driver in driving a vehicle.

[0017] As shown in FIG. 1, a vehicle driving assistance image display device 10 according to an embodiment of the present invention is mounted on a vehicle (host vehicle 100). The vehicle driving assistance image display device 10 includes an ECU 90 as a control device. The ECU 90 includes a microcomputer as a main component. The ECU 90 includes a CPU, ROM, RAM, non-volatile memory, an interface, and the like. The CPU is configured to realize various functions by executing instructions (programs, routines) stored in the ROM.

[0018] <Vehicle driving device> Furthermore, the host vehicle 100 is equipped with a vehicle driving device 20. The vehicle driving device 20 is a device that drives, brakes, steers, and changes gears of the host vehicle 100, and in this example, includes a drive device 21, a braking device 22, a steering device 23, and a transmission device 24.

[0019] <Drive unit> The drive device 21 is a device that outputs a driving force that is applied to the host vehicle 100 to make the host vehicle 100 travel, and is, for example, an internal combustion engine and / or an electric motor. The drive device 21 is electrically connected to the ECU 90. The ECU 90 can control the driving force output from the drive device 21 by controlling the operation of the drive device 21.

[0020] <Brake device> The braking device 22 is a device that outputs a braking force to be applied to the host vehicle 100 in order to brake the host vehicle 100, and is, for example, a hydraulic brake device. The braking device 22 is electrically connected to the ECU 90. The ECU 90 can control the braking force output from the braking device 22 by controlling the operation of the braking device 22.

[0021] <Steering device> The steering device 23 is a device that outputs a steering force applied to the host vehicle 100 in order to steer the host vehicle 100, and is, for example, a power steering device. The steering device 23 is electrically connected to the ECU 90. The ECU 90 can control the steering force output from the steering device 23 by controlling the operation of the steering device 23.

[0022] <Transmission device> The transmission device 24 is a device that switches whether or not the driving force output from the drive device 21 is transmitted to the driving wheels of the vehicle 100, and switches whether the driving force is transmitted to the driving wheels so as to move the vehicle 100 forward or so as to move the vehicle 100 backward.

[0023] The transmission device 24 operates in one of the following states (shift states): a state in which the driving force is transmitted to the drive wheels to move the vehicle 100 forward (forward drive state), a state in which the driving force is transmitted to the drive wheels to move the vehicle 100 backward (reverse drive state), a state in which the driving force is not transmitted to the drive wheels of the vehicle 100 (neutral state), and a state in which the vehicle 100 is kept stopped (parking state).

[0024] The transmission device 24 is electrically connected to the ECU 90. The ECU 90 controls the operation of the transmission device 24 to set the shift state of the transmission device 24 to any one of a forward drive state, a reverse drive state, a neutral state, and a parking state.

[0025] <Sensors, etc.> Furthermore, the vehicle 100 is equipped with an accelerator pedal 31, an accelerator pedal operation amount sensor 32, a brake pedal 33, a brake pedal operation amount sensor 34, a steering wheel 35, a steering shaft 36, a steering angle sensor 37, a steering torque sensor 38, a vehicle speed detection device 40, a shift lever 41, a shift sensor 42, an illuminance sensor 50, a display device 60, a surrounding image acquisition device 70, and a GPS device 80.

[0026] <Accelerator pedal operation amount sensor> The accelerator pedal operation amount sensor 32 is a sensor that detects the amount of operation of the accelerator pedal 31. The accelerator pedal operation amount sensor 32 is electrically connected to the ECU 90. The accelerator pedal operation amount sensor 32 transmits information on the detected amount of operation of the accelerator pedal 31 to the ECU 90. Based on that information, the ECU 90 obtains the amount of operation of the accelerator pedal 31 as the accelerator pedal operation amount AP.

[0027] The ECU 90 calculates and obtains a required driving force (required driving torque) based on the accelerator pedal operation amount AP and the traveling speed (host vehicle speed) of the host vehicle 100. The ECU 90 controls the operation of the drive device 21 so that the required driving force is output.

[0028] <Brake pedal operation amount sensor> The brake pedal operation amount sensor 34 is a sensor that detects the operation amount of the brake pedal 33. The brake pedal operation amount sensor 34 is electrically connected to the ECU 90. The brake pedal operation amount sensor 34 transmits information on the detected operation amount of the brake pedal 33 to the ECU 90. Based on that information, the ECU 90 obtains the operation amount of the brake pedal 33 as the brake pedal operation amount BP.

[0029] The ECU 90 calculates a required braking force (required braking torque) from the brake pedal operation amount BP, and controls the operation of the braking device 22 so that the required braking force is output.

[0030] <Steering angle sensor> The steering angle sensor 37 is a sensor that detects the rotation angle of the steering shaft 36 relative to the neutral position, and is electrically connected to the ECU 90. The steering angle sensor 37 transmits information about the detected rotation angle of the steering shaft 36 to the ECU 90. Based on that information, the ECU 90 obtains the rotation angle of the steering shaft 36 as the steering angle θ.

[0031] <Steering torque sensor> The steering torque sensor 38 is a sensor that detects the torque input to the steering shaft 36 by the driver of the vehicle 100 via the steering wheel 35, and is electrically connected to the ECU 90. The steering torque sensor 38 transmits information on the detected torque to the ECU 90. Based on that information, the ECU 90 obtains the torque input to the steering shaft 36 by the driver via the steering wheel 35 (driver input torque).

[0032] The ECU 90 obtains the required steering force (required steering torque) based on the steering angle θ, the driver input torque, and the traveling speed (vehicle speed) of the vehicle 100, and controls the operation of the steering device 23 so that the required steering torque is output from the steering device 23.

[0033] <Vehicle speed detection device> The vehicle speed detection device 40 is a device that detects the traveling speed of the host vehicle 100, and is, for example, a wheel speed sensor. The vehicle speed detection device 40 is electrically connected to the ECU 90. The vehicle speed detection device 40 transmits information on the detected traveling speed of the host vehicle 100 to the ECU 90. The ECU 90 acquires the traveling speed of the host vehicle 100 (host vehicle speed V) based on the information.

[0034] <Shift sensor> The shift sensor 42 is a sensor that detects the setting position of the shift lever 41. The shift lever 41 is a device that is operated by the driver of the vehicle 100, and the setting positions of the shift lever 41 that can be set by the driver are a forward position (drive range), a reverse position (rearrange), a neutral position (neutral range), and a parking position (parking range). The shift sensor 42 is electrically connected to the ECU 90. The shift sensor 42 transmits a signal indicating the detected setting position of the shift lever 41 to the ECU 90.

[0035] When the shift lever 41 is set to the drive range, the shift sensor 42 transmits a signal indicating that the set position of the shift lever 41 is in the drive range to the ECU 90. Upon receiving this signal, the ECU 90 controls the shift state of the transmission device 24 so that the transmission device 24 is in a forward drive state.

[0036] Furthermore, when the shift lever 41 is set to the rearrange position, the shift sensor 42 transmits a signal indicating that the set position of the shift lever 41 is the rearrange position to the ECU 90. Upon receiving this signal, the ECU 90 controls the shift state of the transmission 24 so that the transmission 24 is in a reverse drive state.

[0037] Furthermore, when the shift lever 41 is set to the neutral range, the shift sensor 42 transmits a signal indicating that the set position of the shift lever 41 is the neutral range to the ECU 90. Upon receiving this signal, the ECU 90 controls the shift state of the transmission device 24 so that the transmission device 24 is in the neutral state.

[0038] Furthermore, when the shift lever 41 is set to the parking range, the shift sensor 42 transmits a signal indicating that the set position of the shift lever 41 is the parking range to the ECU 90. Upon receiving this signal, the ECU 90 controls the shift state of the transmission device 24 so that the transmission device 24 is in the parking state.

[0039] <Illuminance sensor> The illuminance sensor 50 is a sensor that detects the illuminance of light. The illuminance sensor 50 is mounted on the host vehicle 100 so as to be able to detect the illuminance around the host vehicle 100. The illuminance sensor 50 is electrically connected to the ECU 90. The illuminance sensor 50 transmits information on the detected illuminance to the ECU 90. Based on that information, the ECU 90 obtains the illuminance around the host vehicle 100 as the vehicle ambient illuminance LX.

[0040] <Display device> The display device 60 is a device that displays various images so that the driver can see them, and includes a display 61. The display 61 is, for example, a display of a navigation device. The display 61 is electrically connected to the ECU 90. The ECU 90 can display various images on the display 61.

[0041] <Peripheral information detection device> The surrounding image acquisition device 70 is a device that captures images of the surroundings of the host vehicle 100, and in this example, is equipped with a front camera 71, a rear camera 72, a left camera 73, and a right camera 74. As shown in FIG. 2, the front camera 71 is attached to the host vehicle 100 so as to be able to capture a predetermined range R71 in front of the host vehicle 100. The rear camera 72 is attached to the host vehicle 100 so as to be able to capture a predetermined range R72 behind the host vehicle 100. Also, as shown in FIG. 3, the left camera 73 is attached to the host vehicle 100 so as to be able to capture a predetermined range R73 to the left of the host vehicle 100. The right camera 74 is attached to the host vehicle 100 so as to be able to capture a predetermined range R74 to the right of the host vehicle 100.

[0042] The left area of ​​the predetermined range R71 photographed by the front camera 71 partially overlaps with the front area of ​​the predetermined range R73 photographed by the left camera 73. The right area of ​​the predetermined range R71 photographed by the front camera 71 partially overlaps with the front area of ​​the predetermined range R74 photographed by the right camera 74. Furthermore, the left area of ​​the predetermined range R72 photographed by the rear camera 72 partially overlaps with the rear area of ​​the predetermined range R73 photographed by the left camera 73. The right area of ​​the predetermined range R72 photographed by the rear camera 72 partially overlaps with the rear area of ​​the predetermined range R74 photographed by the right camera 74.

[0043] The surrounding image acquisition device 70 transmits front camera image data D1, rear camera image data D2, left camera image data D3, and right camera image data D4 to the ECU 90. The front camera image data D1 is data of an image captured by the front camera 71. The rear camera image data D2 is data of an image captured by the rear camera 72. The left camera image data D3 is data of an image captured by the left camera 73. The right camera image data D4 is data of an image captured by the right camera 74. Hereinafter, the front camera image data D1, rear camera image data D2, left camera image data D3, and right camera image data D4 will be collectively referred to as "camera image data D0."

[0044] <GPS device> The GPS device 80 is a device that receives GPS signals. The GPS device 80 is electrically connected to the ECU 90. The GPS device 80 transmits the received GPS signals to the ECU 90. The ECU 90 acquires the current position of the host vehicle 100 based on the GPS signals.

[0045] <Operation of the vehicle driving support image display device> Next, the operation of the vehicle driving support image display device 10 will be described.

[0046] <Display of the driving support image> As shown in (A) of FIG. 4, when the host vehicle 100 approaches a T-junction and a stop line 300 is provided at the T-junction, the driver stops the host vehicle 100 in front of the stop line 300 as shown in (B) of FIG. 4. At this time, if there are buildings, walls, etc. on the left and right of the host vehicle 100, the driver tries to visually check the left and right situations of the road ahead of the host vehicle 100 in order to safely turn right or left the host vehicle 100. However, the view is blocked by buildings, walls, etc., and it is difficult to check the left and right situations of the road ahead.

[0047] Such a situation can also occur not only when the host vehicle 100 approaches a T-junction but also in situations where, for example, the driver tries to park the host vehicle 100 in a home parking lot and the host vehicle 100 stops beside the home parking lot.

[0048] Therefore, the vehicle driving support image display device 10 is capable of displaying a driving support image G204 on the display 61 so that the driver can easily check the surroundings of the host vehicle 100.

[0049] Specifically, as shown in FIG. 5A, while the vehicle 100 is traveling, the vehicle driving assistance image display device 10 displays an image (navigation image G201) on the display 61, which image shows a map of the area in which the vehicle 100 is traveling and an image showing the position of the vehicle 100. At this time, the vehicle driving assistance image display device 10 also displays a menu image G202 on the display 61. The portion of the display 61 displaying the menu image G202 (menu image portion P202) is a portion that accepts touch operations. In this example, the touch operation is an operation in which a person, such as a driver, touches the display 61 with their finger.

[0050] Then, when a touch operation is applied to the menu image portion P202 while the vehicle 100 is stopped (or when a touch operation is applied to the menu image portion P202 and then a predetermined touch operation is applied to the display 61), the vehicle driving assistance image display device 10 displays a display request image G203 on the display 61, as shown in Fig. 5(B). The portion of the display 61 displaying the display request image G203 (the display request image portion P203) is a portion that accepts touch operations.

[0051] When a touch operation is performed on the display request image portion P203, the vehicle driving assistance image display device 10 displays a driving assistance image G204 on the display 61, as shown in (C) of FIG. 5. The driving assistance image G204 includes a front camera image G205 and an overhead image G206. The vehicle driving assistance image display device 10 displays the front camera image G205 in the left area of ​​the display 61, and also displays the overhead image G206 in the right area of ​​the display 61.

[0052] The front camera image G205 is an image captured by the front camera 71, and the overhead image G206 is an image that represents the vehicle and the surrounding conditions of the vehicle 100 when the vehicle 100 is viewed from above. The vehicle driving assistance image display device 10 can generate the overhead image G206 based on the front camera image data D1, the rear camera image data D2, the left camera image data D3, and the right camera image data D4.

[0053] By looking at the forward camera image G205 and the overhead image G206, the driver can check the situation around the vehicle 100 that cannot be confirmed visually and the situation around the vehicle 100 that is difficult to confirm visually.

[0054] <Registering front camera images> Furthermore, if the vehicle 100 stops while the transmission device 24 is in a forward drive state or a neutral state while the driving assistance image G204 is being displayed on the display 61, the vehicle driving assistance image display device 10 displays a registration request image G207 on the display 61. The portion of the display 61 displaying the registration request image G207 (registration request image portion P207) is a portion that accepts touch operations.

[0055] The vehicle driving assistance image display device 10 is configured to display the registration request image portion P207 on the display 61 when the vehicle 100 stops in a state where the shift state of the transmission device 24 is in a forward drive state or a neutral state, but when the driving assistance image G204 is displayed on the display 61, the registration request image G207 is displayed in a toned-down state and a state is set in which touch operations on the registration request image portion P207 are not accepted, and when the vehicle 100 stops in a state where the shift state of the transmission device 24 is in a state other than a forward drive state or a neutral state, the registration request image G207 is displayed in a toned-up state and a state is set in which touch operations on the registration request image portion P207 are accepted.

[0056] When a touch operation is performed on the registration request image portion P207, the vehicle driving assistance image display device 10 saves (stores) the position of the vehicle 100 at that time in a non-volatile memory as a registration point Preg. Specifically, the vehicle driving assistance image display device 10 acquires the position of the vehicle 100 based on a GPS signal, and saves the position (the position of the vehicle 100 in the GPS coordinates) in a non-volatile memory as a registration point Preg.

[0057] Furthermore, the vehicle driving assistance image display device 10 associates the image captured by the forward camera 71 at that time (forward camera image G205 at that time) with the saved registered point Preg and saves (stores) it in non-volatile memory as a registered camera image G208.

[0058] <Automatic display of driving assistance images> Thereafter, as shown in Fig. 6A, when the driving assistance image display condition is met that the vehicle 100 enters an area (registration point surrounding area ARreg) at a predetermined distance R from the registration point Preg and the vehicle speed V is equal to or lower than a predetermined vehicle speed (image display vehicle speed threshold Vth), the vehicle driving assistance image display device 10 automatically displays the driving assistance image G204 on the display 61, as shown in Fig. 7A, without any operation by the driver. In this example, the image display vehicle speed threshold Vth is set to a relatively small vehicle speed that is greater than zero.

[0059] At this time, if the driving assistance image display condition is met, the vehicle driving assistance image display device 10 also displays on the display 61 a message image G209 informing the driver that the vehicle 100 is approaching the registered point Preg.

[0060] Note that the condition that the host vehicle speed V is equal to or less than the image display vehicle speed threshold Vth may be excluded from the driving assistance image display conditions. In other words, the vehicle driving assistance image display device 10 may be configured to automatically display the driving assistance image G204 on the display 61 when the driving assistance image display condition that the host vehicle 100 is entering the registered point surrounding area ARreg is met.

[0061] Furthermore, a condition that the host vehicle 100 has entered the registration point surrounding area ARreg from the same direction as when the registration point Preg was saved in the non-volatile memory may be added to the driving assistance image display conditions. That is, the vehicle driving assistance image display device 10 may be configured to automatically display the driving assistance image G204 on the display 61 when the driving assistance image display condition that the host vehicle 100 has entered the registration point surrounding area ARreg from the same direction as when the registration point Preg was saved in the non-volatile memory and the host vehicle speed V is equal to or less than the image display vehicle speed threshold Vth is met.

[0062] Then, as shown in (B) of Figure 6, when the vehicle 100 moves out of the registered point surrounding area ARreg, the vehicle driving assistance image display device 10 ends the display of the driving assistance image G204 on the display 61, and displays the navigation image G201 on the display 61, as shown in (B) of Figure 7.

[0063] Of course, if the vehicle speed V becomes greater than the image display vehicle speed threshold Vth after the vehicle 100 enters the registered point surrounding area ARreg but before it leaves the registered point surrounding area ARreg, the display of the driving assistance image G204 on the display 61 is terminated and the navigation image G201 is displayed on the display 61 as shown in (B) of Figure 7.

[0064] <Delete registered location> Incidentally, there are cases where the driver wishes to erase the registered point Preg from the non-volatile memory. In this case, when a touch operation is applied to the menu image portion P202 (or when a predetermined touch operation is applied to the display 61 after a touch operation is applied to the menu image portion P202), the vehicle driving assistance image display device 10 displays a deletion request image G210 on the display 61, as shown in (A) of Fig. 8. The portion of the display 61 displaying the deletion request image G210 (deletion request image portion P210) is a portion that accepts touch operations.

[0065] When a touch operation is applied to the erase request image portion P210, the vehicle driving assistance image display device 10 displays an erase operation image G211 on the display 61, as shown in (B) of Fig. 8. (B) of Fig. 8 shows an example when there are three registered points Preg stored in the non-volatile memory, and therefore, in the example shown in (B) of Fig. 8, three erase operation images G211 (a first erase operation image G211_1, a second erase operation image G211_2, and a third erase operation image G211_3) are displayed on the display 61.

[0066] The erase operation image G211 includes a registered camera image G208 (thumbnail) as a registered point image indicating the registered point Preg and a erase selection image G212. In the example shown in Fig. 8(B), the first erase operation image G211_1 includes a first registered camera image G208_1 and a first erase selection image G212_1, the second erase operation image G211_2 includes a second registered camera image G208_2 and a second erase selection image G212_2, and the third erase operation image G211_3 includes a third registered camera image G208_3 and a third erase selection image G212_3.

[0067] As described above, the registered camera image G208 is the front camera image G205 when the vehicle driving assistance image display device 10 stores the position of the host vehicle 100 as a registered point Preg in a non-volatile memory. Furthermore, the portion of the display 61 displaying the delete selection image G212 (the delete selection image portion P212) is a portion that accepts a touch operation. In the example shown in FIG. 8B, the portions of the display 61 displaying the first delete selection image G212_1, the second delete selection image G212_2, and the third delete selection image G212_3 (the first delete selection image portion P212_1, the second delete selection image portion P212_2, and the third delete selection image portion P212_3) are each a portion that accepts a touch operation.

[0068] At this time, the vehicle driving assistance image display device 10 also displays on the display 61 a message image G213 that prompts the driver to select the registered point Preg to be deleted.

[0069] Furthermore, the vehicle driving assistance image display device 10 may be configured to not display the delete selection image G212 as the delete operation image G211, but to display only the registered camera image G208 as the delete operation image G211, and to make the part of the display 61 displaying the registered camera image G208 (registered camera image part) the part that accepts touch operations.

[0070] When a touch operation is performed on the deletion selected image portion P212, the vehicle driving assistance image display device 10 displays a deletion confirmation image G214 on the display 61, as shown in Fig. 8(C). The deletion confirmation image G214 includes a registered camera image G208, a deletion consent image G215, and a deletion refusal image G216. The registered camera image G208 displayed on the display 61 at this time is the registered camera image G208 corresponding to the deletion selected image portion P212 on which the touch operation was performed.

[0071] The part of the display 61 displaying the erasure consent image G215 (erasure consent image part P215) is an area that accepts touch operations. The part of the display 61 displaying the erasure refusal image G216 (erasure refusal image part P216) is also an area that accepts touch operations.

[0072] At this time, the vehicle driving assistance image display device 10 also displays on the display 61 a message image G217 that prompts the driver to decide whether or not to accept the deletion of the registered point Preg.

[0073] When a touch operation is performed on the deletion consent image portion P215, the vehicle driving assistance image display device 10 determines that the driver has consented to the deletion of the corresponding registered location Preg, and erases the registered location Preg corresponding to the deletion selection image portion P212 on which the touch operation was previously performed from the non-volatile memory.

[0074] On the other hand, when a touch operation is performed on the erasure refusal image portion P216, the vehicle driving assistance image display device 10 determines that the driver has refused to erase the corresponding registered point Preg, terminates the display of the erasure confirmation image G214 on the display 61, and displays the navigation image G201 on the display 61.

[0075] Furthermore, the vehicle driving assistance image display device 10 may be configured to erase the registered point Preg corresponding to the erase selection image portion P212 to which the touch operation was applied from the non-volatile memory without displaying the erase confirmation image G214 on the display 61 when a touch operation is applied to the erase selection image portion P212.

[0076] <Update registered camera images> However, if the time when the position of the vehicle 100 is saved in the non-volatile memory as the registered point Preg is nighttime, the front camera image G205 saved in the non-volatile memory as the registered camera image G208 at that time represents the state in front of the vehicle 100 at night, and therefore has low brightness (is dark). Such a registered camera image G208 is displayed on the display 61 as an erase operation image G211, and even if the driver looks at the registered camera image G208, it is difficult for the driver to determine which registered point Preg the registered camera image G208 represents.

[0077] Therefore, when the brightness of the registered camera image G208 is low, the vehicle driving assistance image display device 10 automatically updates the low-brightness registered camera image G208 with the high-brightness front camera image G205 (automatically replaces the low-brightness registered camera image G208 with the high-brightness front camera image G205). Next, the updating of the registered camera image G208 will be described with reference to the flowcharts shown in FIGS. 9 and 10.

[0078] The vehicle driving assistance image display device 10 is configured to execute the routine shown in Fig. 9 at a predetermined calculation cycle. Therefore, at a predetermined timing, the vehicle driving assistance image display device 10 starts processing from step 900 of the routine shown in Fig. 9, proceeds to step 905, and determines whether a touch operation has been applied to the registration request image portion P207 and whether a registration camera image G208 has been saved in non-volatile memory between the previous execution of this routine and the current execution of this routine.

[0079] If a touch operation is performed on the registration request image portion P207, the vehicle driving assistance image display device 10 judges "Yes" in step 905, proceeds to step 910, and determines whether the illuminance around the vehicle 100 (vehicle ambient illuminance LX) is equal to or less than a predetermined illuminance (first illuminance threshold LX1).

[0080] That is, when the vehicle driving assistance image display device 10 stores the position of the vehicle 100 as a registered point Preg in non-volatile memory and stores the front camera image G205 at that time as a registered camera image G208 in non-volatile memory, it acquires the illuminance around the vehicle 100 (vehicle ambient illuminance LX) based on information from the illuminance sensor 50 and determines whether the vehicle ambient illuminance LX is equal to or less than a predetermined illuminance (first illuminance threshold LX1).

[0081] If the vehicle ambient illuminance LX is equal to or less than the first illuminance threshold LX1, the vehicle driving assistance image display device 10 determines "Yes" in step 910, proceeds to step 915, and acquires the luminance of the registered camera image G208 stored in the nonvolatile memory at that time (registered camera image luminance BR) based on the front camera image data D1. In this example, the vehicle driving assistance image display device 10 acquires the average luminance of the registered camera image G208 stored in the nonvolatile memory at that time as the registered camera image luminance BR.

[0082] Next, the vehicle driving assistance image display device 10 proceeds to step 920 to determine whether the acquired registered camera image brightness BR is equal to or less than a predetermined brightness (predetermined brightness threshold BRth).

[0083] If the registered camera image brightness BR is equal to or less than the predetermined brightness threshold BRth, the vehicle driving assistance image display device 10 judges "Yes" in step 920, proceeds to step 925, and sets the value of the update target flag Xupdate linked to the registered camera image G208 stored in the non-volatile memory to "1".

[0084] That is, if the registered camera image G208 is a dark image, the vehicle driving assistance image display device 10 determines that the registered camera image G208 currently stored in the nonvolatile memory is an image that needs to be updated.

[0085] Then, the vehicle driving assistance image display device 10 advances the process to step 995 and temporarily ends the process of this routine.

[0086] If the vehicle driving assistance image display device 10 determines "No" in step 905, step 910, or step 920, it proceeds directly to step 995 and temporarily ends the processing of this routine.

[0087] Furthermore, the vehicle driving assistance image display device 10 is configured to execute the routine shown in Fig. 10 at a predetermined calculation cycle. Therefore, at a predetermined timing, the vehicle driving assistance image display device 10 starts processing from step 1000 of the routine shown in Fig. 10, and proceeds to step 1005 to determine whether or not the host vehicle 100 has entered the registered point surrounding area ARreg.

[0088] If the host vehicle 100 is entering the registration point surrounding area ARreg, the vehicle driving assistance image display device 10 determines "Yes" in step 1005, proceeds to step 1010, and determines whether the value of the update target flag Xupdate corresponding to the target registration point Preg (the registration point Preg corresponding to the registration point surrounding area ARreg into which the host vehicle 100 is entering) is "1." In other words, the vehicle driving assistance image display device 10 determines whether the registered camera image G208 corresponding to the target registration point Preg is an image that requires updating.

[0089] If the value of the update target flag Xupdate corresponding to the target registration point Preg is "1", the vehicle driving assistance image display device 10 determines "Yes" in step 1010, proceeds to step 1015, and determines whether the host vehicle 100 is stopped. That is, the vehicle driving assistance image display device 10 determines whether the host vehicle speed V is zero. Note that the vehicle driving assistance image display device 10 may be configured to determine whether the host vehicle speed V is equal to or lower than a predetermined vehicle speed greater than zero (a relatively small vehicle speed close to zero) rather than determining whether the host vehicle speed V is zero in step 1015. That is, the vehicle driving assistance image display device 10 may be configured to determine whether the host vehicle 100 is traveling at a relatively low speed in the registration point surrounding area ARreg in step 1015.

[0090] When the host vehicle 100 is stopped, the vehicle driving assistance image display device 10 determines "Yes" in step 1015, proceeds to step 1020, and determines whether the vehicle surrounding illuminance LX is equal to or greater than a predetermined illuminance (second illuminance threshold LX2). In this example, the second illuminance threshold LX2 is set to a value greater than the first illuminance threshold LX1. However, the second illuminance threshold LX2 may be set appropriately based on the vehicle surrounding illuminance LX when the registered camera image G208 (the target registered camera image G208) corresponding to the registration point surrounding area ARreg into which the host vehicle 100 is entering is stored in a non-volatile memory. For example, the second illuminance threshold LX2 may be set to a value greater than the vehicle surrounding illuminance LX when the target registered camera image G208 is stored in a non-volatile memory.

[0091] If the vehicle ambient illuminance LX is equal to or greater than the second illuminance threshold LX2, the vehicle driving assistance image display device 10 determines "Yes" in step 1020, proceeds to step 1025, and updates the target registered camera image G208 with the current front camera image G205. That is, the vehicle driving assistance image display device 10 replaces the target registered camera image G208 with the current front camera image G205 and saves it in non-volatile memory as a new registered camera image G208.

[0092] In this way, the vehicle driving assistance image display device 10 is configured to update the target registered camera image G208 with the current forward camera image G205 when the image update condition is met, that is, when the vehicle 100 is stopped while entering the registered point surrounding area ARreg and the vehicle ambient illuminance LX is higher than the vehicle ambient illuminance LX when the target registered camera image G208 was saved.

[0093] The condition that the host vehicle 100 is stopped may be excluded from the image update conditions. That is, the vehicle driving assistance image display device 10 may be configured to update the target registered camera image G208 with the current front camera image G205 when the image update condition is met that the current vehicle surrounding illuminance LX is higher than the vehicle surrounding illuminance LX when the target registered camera image G208 was saved while the host vehicle 100 is entering the registration point surrounding area ARreg (image update target area).

[0094] Furthermore, the image update condition that the host vehicle 100 is stopped may be replaced with a condition that the host vehicle speed V is equal to or less than a predetermined vehicle speed (image update vehicle speed threshold). That is, the vehicle driving assistance image display device 10 may be configured to update the target registered camera image G208 with the current front camera image G205 when the image update condition is met that the host vehicle speed V is equal to or less than the image update vehicle speed threshold when the host vehicle 100 is entering the registration point surrounding area ARreg (image update target area) and the current vehicle surrounding illuminance LX is higher than the vehicle surrounding illuminance LX when the target registered camera image G208 was saved.

[0095] Furthermore, the image update conditions may include a condition that the host vehicle 100 has entered the registration point surrounding area ARreg from the same direction as when the registration point Preg was saved in the nonvolatile memory. That is, the vehicle driving assistance image display device 10 may be configured to update the target registered camera image G208 with the current front camera image G205 when the image update condition is met, that is, when the host vehicle 100 is entering the registration point surrounding area ARreg from the same direction as when the registration point Preg was saved in the nonvolatile memory, the host vehicle 100 is stopped, and the vehicle surrounding illuminance LX is higher than the vehicle surrounding illuminance LX when the target registered camera image G208 was saved.

[0096] Furthermore, if the condition that the vehicle 100 has entered the registered point surrounding area ARreg from the same direction as when the registered point Preg was saved in non-volatile memory is not added to the image update conditions, and the vehicle 100 has entered the registered point surrounding area ARreg but has not entered the registered point surrounding area ARreg from the same direction as when the registered point Preg was saved in non-volatile memory, the vehicle driving assistance image display device 10 will update the registered camera image G208 with the image generated in this way if it can generate an image from the same angle as the registered camera image G208 from the front camera image data D1, rear camera image data D2, left camera image data D3 and right camera image data D4.

[0097] Furthermore, in this example, the registered point surrounding area ARreg (automatic image display area) used to determine the driving assistance image display conditions and the registered point surrounding area ARreg (image update target area) used to determine the image update conditions are the same area, but these areas may also be different areas.

[0098] Next, the vehicle driving assistance image display device 10 proceeds to step 1030, where it sets the value of the update target flag Xupdate associated with the updated registered camera image G208 to 0. Then, the vehicle driving assistance image display device 10 proceeds to step 1095, where it temporarily ends the processing of this routine.

[0099] If the vehicle driving assistance image display device 10 determines "No" in step 1005, step 1010, step 1015, or step 1020, the vehicle driving assistance image display device 10 proceeds directly to step 1095 and temporarily ends the processing of this routine.

[0100] In this example, when the registered camera image G208 can be updated, the vehicle driving assistance image display device 10 automatically updates the registered camera image G208 without confirming with the driver whether or not the registered camera image G208 is allowed (desired) to be updated. However, when the registered camera image G208 can be updated, an image is displayed on the display 61 to confirm with the driver whether or not the registered camera image G208 is allowed (desired) to be updated, and the registered camera image G208 is updated only when the driver performs a specified operation in response to that image.

[0101] In addition, there may be multiple opportunities to update the registered camera image G208 while the vehicle 100 is within the registration point surrounding area ARreg. In such a case, the vehicle driving assistance image display device 10 may be configured to update the registered camera image G208 each time an opportunity to update the registered camera image G208 arises, or may be configured to update the registered camera image G208 with the forward camera image G205 of the point closest to the registration point Preg out of the multiple forward camera images G205 acquired as images for updating the registered camera image G208, or may be configured to update the registered camera image G208 with the forward camera image G205 with the highest brightness out of the multiple forward camera images G205 acquired as images for updating the registered camera image G208.

[0102] <Effects> The above is the operation of the vehicle driving assistance image display device 10. According to the vehicle driving assistance image display device 10, if the registered camera image G208 is a dark image and the driver cannot clearly distinguish the location of the registered point Preg indicated by the registered camera image G208 even when the registered camera image G208 is displayed on the display 61 as the erase operation image G211, the dark registered camera image G208 is replaced with the brighter front camera image G205. Therefore, the registered camera image G208 displayed on the display 61 as the erase operation image G211 becomes a bright image, and the driver can clearly distinguish the registered point Preg indicated by the registered camera image G208.

[0103] The present invention is not limited to the above-described embodiment, and various modifications can be made within the scope of the present invention. [Explanation of symbols]

[0104] 10...vehicle driving assistance image display device, 50...illuminance sensor, 60...display device, 61...display, 70...peripheral image acquisition device, 71...front camera, 80...GPS device, 90...ECU, 100...own vehicle

Claims

1. A vehicle driving assistance image display device including: a camera mounted on a vehicle to capture an image of the surroundings of the vehicle; a display mounted on the vehicle to be visible to a driver of the vehicle; and a control device that displays, on the display, a driving assistance image that represents a situation around the vehicle using a camera image captured by the camera, the control device is configured to automatically display the driving assistance image on the display when a driving assistance image display condition is satisfied, the driving assistance image being an area around a pre-registered location that has entered an automatic image display area; the control device is configured to store the camera image taken when the registration point is registered as a registration point image showing the registration point as a registered camera image, and when the registration point image is to be displayed on the display, to display the registered camera image on the display as the registration point image; The control device is configured to update the registered camera image with the camera image currently captured by the camera when an image update condition is satisfied that the illuminance around the vehicle at the current time is higher than the illuminance around the vehicle when the registered camera image was saved, when the vehicle is entering an image update target area, which is an area around the registered point. In a vehicle driving assistance image display device, The image update condition includes a condition that the vehicle speed of the host vehicle is equal to or less than an image update vehicle speed threshold. Vehicle driving assistance image display device.

2. The vehicle driving assistance image display device according to claim 1, The control device is configured to determine that the image update condition is satisfied when the illuminance around the vehicle when the registered camera image is saved is equal to or less than a first illuminance threshold and when the illuminance around the vehicle when the vehicle enters the image update target area is equal to or more than a second illuminance threshold that is set to a value greater than the first illuminance threshold. Vehicle driving assistance image display device.

3. In the vehicle driving assistance image display device according to claim 1 or claim 2, The driving assistance image display condition includes a condition that the vehicle speed of the host vehicle is equal to or lower than an image display vehicle speed threshold. Vehicle driving assistance image display device.

Images