Vehicle surrounding image display device and method for displaying images from a vehicle surrounding image display device

Abstract

This can reduce the burden on drivers when making right or left turns at intersections. [Solution] A vehicle surrounding image display device that displays a right rear image, which is an image captured by a right rear camera that captures the right rear of a vehicle, on a display inside the vehicle cabin, comprising a right display located in front of the driver's seat inside the vehicle cabin and on the right side as seen from the driver's seat, and an image display unit that displays the right rear image on the right display, wherein when the vehicle turns right at an intersection, the image display unit displays a left front image, which is an image captured by a front camera of the vehicle that captures the left front of the vehicle, on the right display along with the right rear image.

Description

Technical Field

[0001] The present invention relates to a vehicle surrounding image display device and an image display method of the vehicle surrounding image display device.

Background Art

[0002] Conventionally, as a technical document related to a vehicle surrounding image display device, Japanese Patent Application Laid-Open No. 2020-037357 is known. This publication shows a vehicle peripheral display device that displays an image captured of the surrounding of a vehicle on a monitor in the vehicle interior, and that wide-angle displays an image of the rear side of the vehicle in conjunction with the operation of the vehicle's direction indicator and the vehicle speed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, when turning right or left at an intersection, a driver needs to check multiple directions, and further technological development for supporting the driver in checking the situation around the vehicle is required.

Means for Solving the Problems

[0005] One aspect of the present invention is a vehicle surrounding image display device that displays a right rear image, which is an image captured by a right rear camera that captures the right rear of the vehicle, on a display in the vehicle interior, including a right display provided in front of the driver's seat of the vehicle in the vehicle interior and provided on the right side as viewed from the driver in the driver's seat, and an image display unit that causes the right rear image to be displayed on the right display, and the image display unit causes a left front image, which is an image captured by the front camera of the vehicle and is an image of the left front of the vehicle, to be displayed on the right display in addition to the right rear image when the vehicle turns right at an intersection.

[0006] According to one aspect of the present invention, when a vehicle turns right at an intersection, the right rear image and the left front image, which is an image of the left front of the vehicle captured by the vehicle's front camera, are displayed on the right-side display. This allows the driver to check all the necessary information on a single screen, reducing eye movement. Furthermore, by displaying images from multiple directions in combination, the driver can efficiently grasp the surrounding situation, reducing the burden on the driver's checking work.

[0007] In a vehicle surrounding image display device according to one aspect of the present invention, the device further comprises a moving object detection unit for detecting moving objects around the vehicle, and the image display unit may increase the display area of ​​the captured image of the right rear of the vehicle on the right display when a moving object is detected to the right rear of the vehicle when the vehicle is turning right at an intersection, compared to when no moving object is detected to the right rear of the vehicle. According to this vehicle surrounding image display device, when a moving object is detected to the right rear when turning right at an intersection, the display area of ​​the right rear image is increased, allowing the driver to effectively obtain the necessary information and facilitating visual information processing.

[0008] In a vehicle surrounding image display device according to one aspect of the present invention, if the right-side turn signal of the vehicle is activated and the vehicle speed is below a slow-speed determination threshold, the device may determine that the vehicle is turning right at an intersection. According to this vehicle surrounding image display device, if the right turn signal of the vehicle is activated and the vehicle speed is below the slow-speed detection threshold, it is determined that the vehicle is turning right at an intersection, thereby accurately detecting a driver's right-turn action at an intersection.

[0009] Another aspect of the present invention is a vehicle surrounding image display device that displays a left rear image, which is an image captured by a left rear camera that captures the left rear of a vehicle, on a display inside the vehicle cabin, comprising: a left display located in front of the driver's seat of the vehicle inside the vehicle cabin and located to the left as seen from the driver's seat; and an image display unit that displays the left rear image on the left display, wherein when the vehicle turns left at an intersection, the image display unit displays a right front image, which is an image captured by a front camera of the vehicle that captures the right front of the vehicle, on the left display together with the left rear image.

[0010] According to another aspect of the present invention, when a vehicle turns left at an intersection, the right front image, which is an image of the vehicle's right front captured by the vehicle's front camera, is displayed on the left display along with the left rear image. This allows the driver to check all the necessary information on a single screen, reducing eye movement. Furthermore, by displaying images from multiple directions in combination, the driver can efficiently grasp the surrounding situation, reducing the burden on the driver's checking work.

[0011] In another aspect of the present invention, a vehicle surrounding image display device further comprises a moving object detection unit for detecting moving objects around the vehicle, and the image display unit may increase the display area of ​​the captured image of the left rear of the vehicle on the left display when a moving object is detected to the left rear of the vehicle when the vehicle is turning left at an intersection, compared to when no moving object is detected to the left rear of the vehicle. According to this vehicle surrounding image display device, when a moving object is detected to the left rear when turning left at an intersection, the display area of ​​the left rear image is increased, allowing the driver to effectively obtain the necessary information and facilitating visual information processing.

[0012] In another aspect of the present invention, a vehicle surrounding image display device may determine that the vehicle is turning left at an intersection if the left turn signal of the vehicle is activated and the vehicle speed is below a slow-speed determination threshold. According to this vehicle surrounding image display device, if the left turn signal of the vehicle is activated and the vehicle speed is below the slow-speed detection threshold, it can determine that the vehicle is turning left at an intersection, thereby accurately detecting a driver's left turn at an intersection.

[0013] A further aspect of the present invention is an image display method for a vehicle surrounding image display device that displays a right rear image, which is an image captured by a right rear camera that captures the right rear of a vehicle, on a display inside the vehicle, wherein the right rear image is displayed on a right display located in the vehicle interior in front of the driver's seat and on the right side as seen from the driver's seat, and when the vehicle turns right at an intersection, a left front image, which is an image captured by a front camera of the vehicle that captures the left front of the vehicle, is displayed on the right display along with the right rear image.

[0014] According to yet another aspect of the present invention, when a vehicle turns right at an intersection, the left front image, which is an image of the left front of the vehicle captured by the vehicle's front camera, is displayed on the right-side display along with the right rear image. This allows the driver to check all the necessary information on a single screen and reduces eye movement. Furthermore, by displaying images from multiple directions in combination, the driver can efficiently grasp the surrounding situation and reduce the burden of checking the surroundings.

[0015] A further aspect of the present invention is an image display method for a vehicle surrounding image display device that displays a left rear image, which is an image captured by a left rear camera that captures the left rear of a vehicle, on a display inside the vehicle, wherein the left rear image is displayed on a left display located in the vehicle interior in front of the driver's seat and on the left side as seen from the driver's seat, and when the vehicle turns left at an intersection, a right front image, which is an image captured by a front camera of the vehicle that captures the right front of the vehicle, is displayed on the left display along with the left rear image.

[0016] According to the image display method of the vehicle surrounding image display device according to another aspect of the present invention, when the vehicle makes a left turn at an intersection, the right front image, which is the captured image of the right front of the vehicle captured by the front camera of the vehicle together with the left rear image, is displayed on the left display. As a result, the driver can check the necessary information on one screen and reduce eye movement. In addition, by combining and displaying images from multiple directions, the driver can efficiently grasp the surrounding situation and reduce the burden of the driver's checking work.

Effects of the Invention

[0017] According to each aspect of the present invention, when the vehicle makes a right turn or a left turn at an intersection, the burden of the driver's checking work can be reduced.

Brief Description of the Drawings

[0018] [Figure 1] It is a block diagram showing a vehicle surrounding image display device according to an embodiment. [Figure 2] It is a diagram for explaining the driver's checking situation when making a right turn at an intersection. [Figure 3] It is a plan view showing an example of the situation where the host vehicle makes a right turn at an intersection. [Figure 4] It is a diagram for explaining an example of a composite image when making a right turn at an intersection when a two-wheeled vehicle exists on the right rear side of the host vehicle. [Figure 5] It is a diagram for explaining an example of a composite image when making a right turn at an intersection when no moving object exists on the right rear side of the host vehicle. [Figure 6] It is a plan view showing an example of the traveling situation of the host vehicle making a left turn at an intersection. [Figure 7] It is a diagram for explaining an example of a composite image when making a right turn at an intersection. [Figure 8] It is a flowchart showing an example of vehicle surrounding image display processing in a vehicle surrounding image display device. [Figure 9] It is a flowchart showing the continuation of vehicle surrounding image display processing.

Embodiments for Carrying Out the Invention

[0019] Embodiments of the present invention will be described below with reference to the drawings.

[0020] Figure 1 is a block diagram showing a vehicle surrounding image display device 100 according to one embodiment. The vehicle surrounding image display device 100 shown in Figure 1 is a device that is mounted on a vehicle such as a passenger car or a truck (hereinafter referred to as "the vehicle") and displays captured images of the area around the vehicle to the driver. The vehicle surrounding image display device 100 may also include a so-called digital outer mirror. The vehicle may be an autonomous vehicle with an autonomous driving function. In the case of an autonomous vehicle, the user sitting in a seat that can monitor the area around the vehicle corresponds to the driver. The following explanation will use the case of driving in a country or region with left-hand traffic as an example. The case of right-hand traffic will be described later.

[0021] [Configuration of the vehicle surrounding image display device] The vehicle surrounding image display device 100 includes a front camera 1 that captures images in front of the vehicle, a right rear camera 2 that captures images to the right rear of the vehicle, a left rear camera 3 that captures images to the left rear of the vehicle, a vehicle speed sensor 4 that detects vehicle speed, a radar sensor 5, a turn signal sensor 6, a right-side display 8, a multimedia display 9, a left-side display 10, and an ECU 20 that controls these.

[0022] Front camera 1 is a camera that captures images of the area in front of the vehicle. Front camera 1 is mounted on the front grille or the top of the windshield of the vehicle and is equipped with a lens with a wide field of view. Front camera 1 captures images of obstacles, other vehicles, pedestrians, etc., in the direction of travel of the vehicle and transmits the images to the ECU 20. The images from front camera 1 are used by the driver to check the situation ahead when turning right or left at intersections.

[0023] The right rear camera 2 is a camera that captures images of the right rear of the vehicle. The right rear camera 2 is installed on the right side of the vehicle's right mirror or rear bumper and has a lens with a wide field of view. The right rear camera 2 captures images of obstacles, other vehicles, motorcycles, etc., to the right rear of the vehicle and transmits the images to the ECU 20. Specifically, the images from the right rear camera 2 are used by the driver to check the situation to the right rear when turning right at an intersection.

[0024] The left rear camera 3 is a camera that captures images of the left rear of the vehicle. The left rear camera 3 is installed on the left side of the vehicle's left mirror or rear bumper and has a lens with a wide field of view. The left rear camera 3 captures images of obstacles, other vehicles, motorcycles, etc., to the left rear of the vehicle and transmits the images to the ECU 20. Specifically, the images from the left rear camera 3 are used by the driver to check the situation to the left rear when turning left at an intersection. The right rear camera 2 and the left rear camera 3 may also function as cameras for a digital outer mirror.

[0025] The vehicle speed sensor 4 is a sensor for detecting the vehicle's speed. The vehicle speed sensor 4 calculates the vehicle speed by measuring the rotation speed of the wheels, for example, using magnetic sensors or optical sensors attached to the vehicle's wheels. The signal from the vehicle speed sensor 4 is transmitted to the ECU 20 and used as vehicle speed information.

[0026] The radar sensor 5 is a sensor for detecting objects present around the vehicle. The radar sensor 5 measures the distance to an object and the vehicle speed by emitting radio waves and receiving the reflected waves. The radar sensor 5 may include millimeter-wave radar or LiDAR (Light Detection and Ranging). The detection results from the radar sensor 5 are transmitted to the ECU 20 and used, for example, to detect moving objects such as motorcycles or pedestrians located to the right rear or left rear of the vehicle.

[0027] The turn signal sensor 6 is a sensor for detecting the operating status and direction of the vehicle's turn signal. The turn signal sensor 6 monitors the state of the turn signal switch and detects whether the vehicle's turn signal is activated or not, and also detects whether the turn signal is activated to the right or left. The turn signal sensor 6 transmits data regarding the operating status and direction to the ECU 20.

[0028] The right-side display 8 is a display device located in the vehicle interior, in front of the driver's seat, and positioned to the right of the driver's perspective. "Right of the driver's perspective" includes not only cases where the entire display is located to the right of the driver's perspective, but also cases where a portion is directly in front of the driver, but the center of the display is to the right of the driver's front. The right-side display 8 is used to display the right rear image RRI from the right rear camera 2. The right rear image RRI will be described later.

[0029] The multimedia display 9 is a display device installed in the vehicle interior in front of the driver's seat and is used to display various multimedia content such as the navigation system, audio system, and vehicle information. The multimedia display 9 functions as an interface to provide visual information to the occupants of the vehicle. The multimedia display 9 is located, for example, in the center of the dashboard (left side from the driver's perspective) and corresponds to the center display.

[0030] The left-side display 10 is located in the vehicle interior, in front of the driver's seat, and is positioned to the left of the driver's seat. "To the left of the driver's seat" includes not only cases where the entire display is located to the left of the driver's seat, but also cases where a portion is directly in front of the driver, but the center of the display is to the left of the driver's front. The left-side display 10 is used to display the left rear image LRI from the left rear camera 3. Note that the vehicle surrounding image display device 100 does not need to include both the multimedia display 9 and the left-side display 10; it may include only one of them.

[0031] Next, the functional configuration of the ECU20 will be described. The ECU20 is an electronic control unit having a CPU (Central Processing Unit) and a memory unit. The memory unit consists of, for example, ROM (Read Only Memory), RAM (Random Access Memory), and EEPROM (Electrically Erasable Programmable Read-Only Memory). In the ECU20, various functions are realized by, for example, executing a program stored in the memory unit with the CPU. The ECU20 may be composed of multiple electronic units. As shown in Figure 1, the ECU20 has a turn signal determination unit 21, an intersection right / left turn determination unit 22, a moving object detection unit 23, and an image display unit 24.

[0032] The turn signal determination unit 21 determines the operating status of the vehicle's turn signals based on the signal from the turn signal sensor 6. Specifically, the turn signal sensor 6 detects the state of the turn signal switch and transmits this information to the turn signal determination unit 21. Based on this information, the turn signal determination unit 21 determines whether the turn signals are operating. If the turn signals are operating, the turn signal determination unit 21 determines whether the left or right turn signal is operating.

[0033] The intersection right / left turn determination unit 22 determines whether the vehicle is attempting to turn right or left at an intersection. For example, if the turn signal determination unit 21 determines that the turn signal is activated, the intersection right / left turn determination unit 22 determines whether the vehicle is attempting to turn right or left at an intersection based on the vehicle speed information from the vehicle speed sensor 4.

[0034] Specifically, the intersection right / left turn determination unit 22 determines that the vehicle is attempting to turn right or left at an intersection if the vehicle's speed is below the slow-down determination threshold when the turn signal is activated. The intersection right / left turn determination unit 22 determines whether the vehicle is attempting to turn right or left at an intersection based on the vehicle's speed while the turn signal is activated, since it is assumed that the vehicle's speed will be significantly reduced when turning right or left at an intersection compared to when changing lanes. The slow-down determination threshold is appropriately set as a threshold value that takes into account the vehicle's speed when turning right or left at an intersection. The distinction between a right turn and a left turn is determined according to the direction in which the turn signal is activated.

[0035] The intersection right / left turn determination unit 22 may perform the determination without using vehicle speed information. The intersection right / left turn determination unit 22 may determine whether the vehicle is attempting to turn right or left at an intersection from the image captured by the front camera 1 when the turn signal determination unit 21 has determined that the turn signal is activated.

[0036] Specifically, the intersection right / left turn determination unit 22 identifies road areas in the captured image by distinguishing between road areas and non-road areas using, for example, an edge detection algorithm or a segmentation algorithm on the image captured by the front camera 1. The intersection right / left turn determination unit 22 performs viewpoint transformation by applying homography transformation and detects intersections by analyzing the road surface from an overhead viewpoint. The intersection right / left turn determination unit 22 may also perform intersection detection by identifying lane markers such as white lines and yellow lines by utilizing hue and saturation information in the captured image and determining vanishing points from the extension of the lanes. The intersection right / left turn determination unit 22 determines whether the vehicle should turn right or left at an intersection based on the intersection detection result from the captured image and the operation of the turn signals. Note that intersection detection using captured images is not limited to the above method, and various well-known methods can be adopted.

[0037] Alternatively, the intersection right / left turn determination unit 22 may determine whether the vehicle is attempting to turn right or left at an intersection based on the determination result of the turn signal determination unit 21, the vehicle's position information, and map information. The vehicle's position information can be obtained using GNSS (Global Navigation Satellite System) or GPS (Global Positioning System). For example, if the vehicle is activating its right turn signal just before the intersection, the intersection right / left turn determination unit 22 determines that the vehicle is attempting to turn right at the intersection. If the vehicle is activating its left turn signal just before the intersection, the intersection right / left turn determination unit 22 determines that the vehicle is attempting to turn left at the intersection.

[0038] Furthermore, the intersection right / left turn determination unit 22 may determine whether the vehicle is about to turn right or left at an intersection based on the determination result of the turn signal determination unit 21 and the navigation information of the vehicle's navigation system. The intersection right / left turn determination unit 22 determines that the vehicle is about to turn right or left at an intersection if the turn signal for the direction of guidance is activated at the time when the vehicle is being guided to turn right or left at an intersection in order to reach its destination.

[0039] In addition, the intersection right / left turn determination unit 22 can determine whether the vehicle is turning right or left at an intersection using a well-known method. Alternatively, instead of the determination result of the turn signal determination unit 21 regarding the operation status of the turn signals, the unit may determine that the vehicle is turning right or left when the steering angle or yaw rate of the vehicle exceeds a threshold.

[0040] The moving object detection unit 23 detects whether a moving object exists around the vehicle based on the detection results of the radar sensor 5. A moving object is a movable object that obstructs the vehicle's movement.

[0041] The vehicle of movement may be a two-wheeled vehicle such as a motorcycle, a personal mobility device, or a pedestrian (e.g., a person running). Personal mobility devices include at least one of the following: electric kick scooters, two-wheeled self-supporting electric mobility devices, electric unicycles, electric skateboards, electric roller skates, electric wheelchairs, etc. Pedestrians may include people moving using bicycles, kickstarters, roller skates, skateboards, etc.

[0042] The moving object detection unit 23 may detect moving objects from images captured by the front camera 1, the right rear camera 2, and the left rear camera 3 without using the radar sensor 5, or it may detect moving objects by sensor fusion, which combines the detection results of the radar sensor 5 with the captured images.

[0043] The image display unit 24 processes images captured from the front camera 1, the right rear camera 2, and the left rear camera 3, and displays them on the screen of an appropriate display device from among the right display 8, the multimedia display 9, and the left display 10. For example, the image display unit 24 can display the image captured by the right rear camera 2 on the right display 8 and the image captured by the left rear camera 3 on the left display 10, acting as a digital outer mirror.

[0044] The image display unit 24 may display a narrow-angle field of view from the right rear camera 2 on the right display 8 when the vehicle is traveling straight, and a wide-angle field of view from the right rear camera 2 when the vehicle's right turn signal is activated. A narrow-angle field of view display is, for example, a display of a digital outer mirror image with a narrow field of view that can meet the magnification requirements of Japanese law. A wide-angle field of view display is a display of a digital outer mirror image with a wider field of view compared to a narrow-angle field of view display. With a wide-angle field of view display, for example, a display of a wide field of view that includes the boundary line on the opposite side of the adjacent lane from the vehicle can be used to simulate a lane change of the vehicle. Note that a narrow-angle field of view display does not necessarily have to meet the magnification requirements of Japanese law; it is sufficient to display a magnification that meets the regulations of each country. The same can be applied to the display on the left display 10 from the left rear camera 3.

[0045] Here, Figure 2 is a diagram illustrating the driver's confirmation process when turning right at an intersection. Figure 2 shows the driver 50, the vehicle's path C, another vehicle NR approaching from the right, another vehicle NL approaching from the left, the right-side display 8, the multimedia display 9, and the left-side display 10. Figure 2 shows the situation when the vehicle is turning right at a T-junction.

[0046] In Figure 2, the driver 50 is seated in the driver's seat, and the right-side display 8, multimedia display 9, and left-side display 10 are installed inside the vehicle. The right-side display 8 displays an image of the right rear of the vehicle captured by the right rear camera 2, allowing the driver to check the situation to the right rear of the vehicle.

[0047] The multimedia display 9 shows an image captured by the front camera 1 of the area in front of the vehicle, allowing the driver to check the situation in front of the vehicle. The multimedia display 9 also shows other vehicles approaching from the right (NR) and from the left (NL). Note that the multimedia display 9 does not necessarily have to display an image of the area in front of the vehicle; it may also display the navigation screen or other information. The left display 10 shows an image captured by the left rear camera 3 of the area behind the vehicle to the left, allowing the driver to check the situation behind the vehicle to the left.

[0048] However, in the situation shown in Figure 2, the driver 50 needs to visually check the front right of their vehicle to confirm the presence of another vehicle NR approaching from the right, visually check the rear right of their vehicle via the right-side display 8 to check for any other vehicles that might be involved when turning right, and visually check the front left of their vehicle to confirm the presence of another vehicle NL approaching from the left. For this reason, the driver 50 had to make large eye movements involving head movements as shown in Figure 2 when turning right or left at an intersection.

[0049] Therefore, in the image display unit 24 of the vehicle surrounding image display device 100 according to this embodiment, the driver's eye movement is reduced by displaying a composite image from multiple directions. This will be explained using Figures 3 and 4. Figure 3 is a plan view showing an example of a situation in which the vehicle is turning right at an intersection. Figure 3 shows the situation of the vehicle in Figure 2 as a plan view.

[0050] Figure 3 shows the vehicle M, the path of vehicle M C, another vehicle NR approaching from the right, another vehicle NL approaching from the left, and a motorcycle NB approaching from the left rear. It also shows the forward imaging range FV of the vehicle M's front camera 1, the left front region FLA within the forward imaging range FV, the right rear imaging range RRV of the vehicle M's right rear camera 2, and the close right rear region RRA within the right rear imaging range RRV. In the situation shown in Figure 3, the image display unit 24 reduces the driver's eye movement by displaying the image corresponding to the left front region FLA and the image corresponding to the close right rear region RRA together.

[0051] Figure 4 illustrates an example of a composite image taken when turning right at an intersection, with a motorcycle present to the right rear of the vehicle M. Figure 4 shows the left front image FLI, corresponding to the left front region FLA of the front camera 1, and the right rear image RRI, corresponding to the close right rear region RRA of the right rear camera 2. The left front image FLI shows the other vehicle NL, and the right rear image RRI shows the motorcycle NB.

[0052] As shown in Figure 4, the image display unit 24 displays a composite image on the right display 8, combining the left front image FLI and the right rear image RRI, in the situation shown in Figure 3. Here, since the motorcycle NB is visible in the right rear image RRI, the image display unit 24 increases the display area of ​​the right rear image RRI compared to the left front image FLI.

[0053] The image display unit 24 may have a ratio of 5:5 for the display area of ​​the right rear image RRI and the left front image FLI, or it may have a ratio of 6:4. The ratio of the display areas may be adjustable to any ratio. Images other than the left front image FLI and the right rear image RRI may be displayed on the right display 8.

[0054] As a result, in a situation like Figure 2 where the driver 50 is turning right at an intersection, the driver can check the situation of another vehicle NL approaching from the left of the vehicle M and the situation of the motorcycle NB approaching from the right rear of the vehicle M by looking at the right front display 8 located in front of the driver's right inside the cabin, while looking at the right front display 8 which is located in front of the driver inside the cabin, thus reducing the driver 50's eye movement. In this case, it is not necessary to display an image of what is in front of the vehicle M on the multimedia display 9. For example, a navigation screen can be displayed.

[0055] Furthermore, the image display unit 24 changes the ratio of the display area of ​​each image based on the detection result of the moving object by the moving object detection unit 23. Here, Figure 5 is a diagram illustrating an example of a composite image when turning right at an intersection when there is no moving object to the right rear of the vehicle M. Specifically, Figure 5 shows the image display on the right display 8 when no moving object is detected to the right rear of the vehicle M when turning right at an intersection.

[0056] As shown in Figure 5, if no moving object such as a motorcycle NB is detected to the right rear of the vehicle M, the image display unit 24 reduces the ratio of the display area of ​​the right rear image RRI on the right display 8 compared to Figure 4. Here, as an example, the image display unit 24 sets the ratio of the display area of ​​the right rear image RRI to the display area of ​​the left front image FLI to 5:5.

[0057] Similarly, the image display unit 24 may change the ratio of the display area of ​​the left front image FLI depending on whether or not a moving object approaching from the left side of the vehicle M is detected. When the moving object detection unit 23 detects a moving object approaching from the left side of the vehicle M, the image display unit 24 increases the ratio of the display area of ​​the left front image FLI compared to when no moving object is detected approaching from the left side of the vehicle M.

[0058] Furthermore, even when the moving object detection unit 23 detects a moving object to the right rear of the vehicle M, the image display unit 24 may be configured to increase the ratio of the display area of ​​the right rear image RRI when the moving object is located in a position visible in the right rear image RRI. In this case, the image display unit 24 does not increase the ratio of the display area of ​​the right rear image RRI even if the moving object is located in a position not visible in the right rear image RRI. The same can be applied to the left front image FLI.

[0059] The image display unit 24 may change the ratio of the display area in real time based on the detection results of the moving object detection unit 23. The image display unit 24 may highlight moving objects that appear in the left front image FLI or the right rear image RRI. The image display unit 24 may display a brightly colored border (for example, a red or yellow border) around the moving object as a highlight. The image display unit 24 may display icons indicating two-wheeled or four-wheeled vehicles near the moving object in the image as a highlight. The image display unit 24 may display the distance and speed of the moving object near the moving object in the image as a highlight.

[0060] Next, we will explain the case where vehicle M turns left at an intersection. Figure 6 is a plan view showing an example of the driving situation of vehicle M turning left at an intersection. Figure 6 shows the right front region FRA in the forward imaging range FV of vehicle M's front camera 1, the left rear imaging range RLV of vehicle M's left rear camera 3, and the close left rear region RLA in the left rear imaging range RLV. Components that are the same as in Figure 3 are denoted by the same reference numerals and their explanations are omitted.

[0061] In Figure 6, it is assumed that no moving object is detected to the left rear of the vehicle M. Also, as shown in Figure 6, in countries or regions with left-hand traffic, the space available for motorcycles and other vehicles to enter to the left of the vehicle is often narrow. Therefore, the left rear imaging range RLV may be a narrower angle range than the right rear imaging range RRV shown in Figure 3. Similarly, the close-range left rear region RLA may be a narrower angle range than the close-range right rear region RRA shown in Figure 3.

[0062] Figure 7 illustrates an example of a composite image taken when turning right at an intersection. Figure 7 shows the right front image FRI, which corresponds to the right front area FRA of the front camera 1, and the left rear image RLI, which corresponds to the close left rear area RLA of the left rear camera 3. Another vehicle NR is visible in the right front image FRI. No moving object is visible in the right rear image RRI.

[0063] As shown in Figure 7, the image display unit 24 displays a composite image on the left display 10, combining the right front image FRI and the left rear image RLI, in the situation shown in Figure 6. Here, the image display unit 24 increases the display area of ​​the right front image FRI compared to the left rear image RLI because the right front image FRI shows another vehicle NR, and the left rear image RLI does not show the motorcycle NB. The image display unit 24 may set the ratio of the display area of ​​the left rear image RLI to the display area of ​​the right front image FRI to 5:5 or 6:4. The ratio of the display areas may be adjustable to any ratio. Alternatively, the size of the area may be adjusted instead of the ratio.

[0064] As a result, in a situation like Figure 6 where the driver 50 is turning left at an intersection, the driver can check the status of other vehicles NR approaching from the right of the vehicle M and the presence or absence of moving objects approaching from the left rear of the vehicle M by looking at the left front, which is the direction of travel of the vehicle M, inside the vehicle cabin, and viewing the left display 10 located to the left front of the driver. This reduces the driver 50's eye movement. The image display unit 24 may display a composite image on the multimedia display 9 (a display device located to the left from the driver's perspective) instead of the left display 10.

[0065] Furthermore, the image display unit 24 may change the ratio of the display area of ​​each image based on the detection result of the moving object by the moving object detection unit 23, similar to when turning right at an intersection. For example, if no moving object such as another vehicle NL is detected to the left front of the vehicle M, the image display unit 24 will reduce the ratio of the display area of ​​the right front image FRI on the left display 10 compared to Figure 7. Similarly, if a moving object is detected to the left rear of the vehicle M, the image display unit 24 may increase the ratio of the display area of ​​the left rear image RLI on the left display 10 compared to Figure 7, where no moving object is detected. The same can be done for highlighting.

[0066] [Image display method for vehicle surrounding image display device] Next, the image display method of the vehicle surrounding image display device 100 according to this embodiment will be described with reference to the drawings. Figure 8 is a flowchart showing an example of the vehicle surrounding image display process in the vehicle surrounding image display device 100. Figure 8 shows the flow of the image display process when turning right at an intersection.

[0067] As shown in Figure 8, the ECU 20 of the vehicle surrounding image display device 100 first determines, in step S10, whether the right turn signal is activated by the turn signal determination unit 21. Specifically, the turn signal sensor 6 detects whether the switch for the right turn signal is on and transmits this information to the turn signal determination unit 21. If the right turn signal is not activated, the ECU 20 proceeds to step S20 in Figure 9. On the other hand, if the right turn signal is activated, the ECU 20 proceeds to step S11.

[0068] In step S11, the ECU 20 uses the intersection right / left turn determination unit 22 to determine whether the vehicle is attempting to turn right at the intersection. The intersection right / left turn determination unit 22 determines, for example, that the vehicle speed detected by the vehicle speed sensor 4 is below the slow-speed determination threshold if the vehicle is attempting to turn right at the intersection. If the ECU 20 does not determine that the vehicle is attempting to turn right at the intersection, it proceeds to step S12. If the ECU 20 determines that the vehicle is attempting to turn right at the intersection, it proceeds to step S13.

[0069] In step S12, the ECU 20 displays a wide-angle view of the right rear image captured by the right rear camera 2 on the right display 8 using the image display unit 24. The right rear image in this case may be different from or the same as the right rear image RRI in Figure 4. As a wide-angle view display, for example, an image of the range shown on the right display 8 in the upper right of Figure 4 can be displayed. After that, the ECU 20 terminates the current process.

[0070] In step S13, the ECU 20 determines whether the moving object detection unit 23 has detected a moving object to the right rear of the vehicle. The moving object detection unit 23, for example, analyzes the reflected waves of radio waves emitted by the radar sensor 5 to detect a moving object to the right rear of the vehicle M. If a moving object is detected, the ECU 20 proceeds to step S14. If no moving object is detected, the ECU 20 proceeds to step S15.

[0071] In step S14, the ECU 20 displays a composite image on the right-side display 8, combining the left-front image FLI from the front camera 1 and the right-rear image RRI from the right-rear camera 2, using the image display unit 24. This allows the driver 50 to view all the necessary information for turning right at an intersection on a single display, minimizing eye movement. In step S14, the display area ratio of the right-rear image RRI is increased. This makes it easier for the driver 50 to focus on moving objects approaching from the right rear of their vehicle M, thus preventing collisions with moving objects when turning right. The ECU 20 then terminates this process.

[0072] In step S15, the ECU 20 displays a composite image on the right-side display 8, combining the left-front image FLI from the front camera 1 and the right-rear image RRI from the right-rear camera 2, using the image display unit 24. In step S15, the left-front image FLI and the right-rear image RRI are displayed on the right-side display 8 at a predetermined ratio. The predetermined ratio may be 5:5 or 4:6. After that, the ECU 20 terminates the current process.

[0073] Next, Figure 9 is a flowchart showing the continuation of the vehicle surrounding image display processing. Figure 9 shows the flow of image display processing when turning left at an intersection. The processing shown in Figure 9 is executed if it is not determined that the right turn signal is activated in step S10 of Figure 8.

[0074] As shown in Figure 9, in step S20, the ECU 20 determines whether the left turn signal is activated using the turn signal determination unit 21. If the left turn signal is not activated, the ECU 20 terminates the process. On the other hand, if the left turn signal is activated, the ECU 20 proceeds to step S21.

[0075] In step S21, the ECU 20 uses the intersection right / left turn determination unit 22 to determine whether the vehicle is attempting to turn left at the intersection. The intersection right / left turn determination unit 22 determines, for example, that the vehicle's speed detected by the vehicle speed sensor 4 is below the slow-speed determination threshold. If the ECU 20 does not determine that the vehicle is attempting to turn left at the intersection, it proceeds to step S22. If the ECU 20 determines that the vehicle is attempting to turn left at the intersection, it proceeds to step S23.

[0076] In step S22, the ECU 20 displays a wide-angle view of the left rear image captured by the left rear camera 3 on the left display 10 using the image display unit 24. In this case, the left rear image may be different from or the same as the left rear image RLI in Figure 7. As a wide-angle view display, for example, an image of the range shown on the left display 10 in the upper left of Figure 7 can be displayed. After that, the ECU 20 terminates the current process.

[0077] In step S23, the ECU 20 determines whether the moving object detection unit 23 has detected a moving object to the left rear of the vehicle. If a moving object is detected, the ECU 20 proceeds to step S24. If no moving object is detected, the ECU 20 proceeds to step S25.

[0078] In step S24, the ECU 20 displays a composite image on the left display 10, combining the right front image FRI from the front camera 1 and the left rear image RLI from the left rear camera 3, using the image display unit 24. This allows the driver 50 to check the information necessary when turning left at an intersection on a single display, minimizing eye movement. In step S24, the display area ratio of the left rear image RLI is increased. This makes it easier for the driver 50 to focus on moving objects approaching from the left rear of their vehicle M, thus avoiding collisions with moving objects when turning left. After this, the ECU 20 terminates the current process.

[0079] In step S25, the ECU 20 displays a composite image on the left display 10, combining the right front image FRI from the front camera 1 and the left rear image RLI from the left rear camera 3, using the image display unit 24. In step S25, the right front image FRI and the left rear image RLI are displayed on the left display 10 at a predetermined ratio. The predetermined ratio may be 5:5 or 4:6. After that, the ECU 20 terminates the current process.

[0080] As described above, with the vehicle surrounding image display device 100 according to this embodiment, when the vehicle M turns right at an intersection, the left front image FLI captured by the vehicle M's front camera 1 is displayed on the right-side display 8 along with the right rear image RRI. This allows the driver 50 to check all the necessary information on a single screen, reducing eye movement. Furthermore, by displaying images from multiple directions in combination, the driver 50 can efficiently grasp the surrounding situation, reducing the burden of checking the surroundings.

[0081] Furthermore, according to the vehicle surrounding image display device 100, when a moving object is detected to the right rear when turning right at an intersection, the display area of ​​the right rear image RRI is increased, allowing the driver 50 to effectively acquire the necessary information and facilitating visual information processing.

[0082] Furthermore, the vehicle surrounding image display device 100 can accurately detect a right turn by the driver 50 at an intersection by determining that the vehicle M is turning right at an intersection when the right turn signal of the vehicle M is activated and the vehicle speed is below the slow-speed determination threshold. The same effects can be obtained when turning left at an intersection.

[0083] Although embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above. The present invention can be implemented in various forms, starting with the embodiments described above, by making various changes and improvements based on the knowledge of those skilled in the art.

[0084] In the above embodiment, the display of a composite image corresponding to both right and left turns at an intersection was described. However, the composite image may be displayed only when turning right or left at an intersection. Furthermore, in the above embodiment, the case of a country or region with left-hand traffic was described as an example, but the present invention is also applicable to countries or regions with right-hand traffic.

[0085] The vehicle surrounding image display device 100 does not necessarily need to change the ratio of the display area in the composite image according to the detection result of a moving object. The vehicle surrounding image display device 100 may display a composite image of the right rear image RRI and the left front image FLI at a predetermined ratio, regardless of whether or not there is a moving object. The same applies to a composite image of the right front image FRI and the left rear image RLI.

[0086] The vehicle surrounding image display device 100 does not need to include a left-side display 10 if the multimedia display 9 functions as a display device located to the left of the driver. Furthermore, at least one of the right-side display 8, the multimedia display 9, and the left-side display 10 may be a head-up display. The display area of ​​the head-up display located to the right of the driver may be treated as the right-side display 8, and the display area located to the left of the driver may be treated as the left-side display 10.

[0087] The vehicle surrounding image display device 100 may change the ratio of the display area of ​​the composite image according to the driver's (50's) viewing conditions when turning right or left at an intersection. When turning right at an intersection, if the driver (50) views the left front of their vehicle M, the vehicle surrounding image display device 100 may not increase the ratio of the display area of ​​the left front image FLI for a certain period of time after the driver (50) views the left front, even if a moving object is present in front of the left. Alternatively, when the driver (50) views the left front of their vehicle M, the vehicle surrounding image display device 100 may not display the left front image FLI as a composite image on the right-side display 8 for a certain period of time after the driver (50) views the left front. Similarly, when turning left at an intersection, if the driver (50) views the right front of their vehicle M, the vehicle surrounding image display device 100 may not increase the ratio of the display area of ​​the right front image FRI for a certain period of time after the driver (50) views the right front, even if a moving object is present in front of the right. The vehicle surrounding image display device 100 may be configured such that, when the driver 50 sees the area in front of the right side of the vehicle M, it does not display the front right image FRI as a composite image on the left display 10 for a certain period of time after the driver 50 sees the area in front of the right side. [Explanation of symbols]

[0088] 1...Front camera, 2...Right rear camera, 3...Left rear camera, 4...Vehicle speed sensor, 5...Radar sensor, 6...Turn signal sensor, 8...Right side display, 9...Multimedia display, 10...Left side display, 20...ECU, 21...Turn signal detection unit, 22...Intersection right / left turn detection unit, 23...Moving object detection unit, 24...Image display unit.

Claims

1. A vehicle surrounding image display device that displays a right rear image, which is an image captured by a right rear camera that captures the right rear of the vehicle, on a display inside the vehicle, A right-side display is provided in the vehicle interior, in front of the driver's seat, and located to the right of the driver's seat as viewed from the driver's perspective. An image display unit that displays the aforementioned right rear image on the right-side display, Equipped with, The image display unit is a vehicle surrounding image display device that, when the vehicle turns right at an intersection, displays the left front image, which is an image of the left front of the vehicle captured by the vehicle's front camera, on the right display along with the right rear image.

2. The vehicle further includes a moving object detection unit for detecting moving objects around the vehicle, The vehicle surrounding image display device according to claim 1, wherein when the vehicle is turning right at an intersection and the moving object is detected to the right rear of the vehicle, the display area of ​​the captured image of the right rear of the vehicle on the right display is increased compared to when the moving object is not detected to the right rear of the vehicle.

3. The vehicle surrounding image display device according to claim 1 or 2, which determines that the vehicle is turning right at an intersection when the right turn signal of the vehicle is activated and the vehicle speed is below a slow-speed determination threshold.

4. A vehicle surrounding image display device that displays a left rear image, which is an image captured by a left rear camera that captures the left rear of the vehicle, on a display inside the vehicle, A left-side display is provided in the vehicle interior, in front of the driver's seat, and located to the left of the driver's seat as viewed from the driver's perspective. An image display unit that displays the aforementioned left rear image on the left-side display, Equipped with, The image display unit is a vehicle surrounding image display device that, when the vehicle turns left at an intersection, displays the right front image, which is an image of the right front of the vehicle captured by the vehicle's front camera, on the left display along with the left rear image.

5. The vehicle further includes a moving object detection unit for detecting moving objects around the vehicle, The vehicle surrounding image display device according to claim 4, wherein when the vehicle is turning left at an intersection and the moving object is detected to the left rear of the vehicle, the display area of ​​the captured image of the left rear of the vehicle on the left display is increased compared to when the moving object is not detected to the left rear of the vehicle.

6. The vehicle surrounding image display device according to claim 4 or 5, which determines that the vehicle is turning left at an intersection when the left turn signal of the vehicle is activated and the vehicle speed is below a slow-speed determination threshold.

7. A method for displaying images on a vehicle surrounding image display device, which displays a right rear image captured by a right rear camera that captures the right rear of a vehicle on a display inside the vehicle, The right rear image is displayed on a right-side display located in the vehicle's interior, in front of the driver's seat, and positioned to the right of the driver's seat. An image display method for a vehicle surrounding image display device, wherein when the vehicle turns right at an intersection, the left front image, which is an image of the left front of the vehicle captured by the vehicle's front camera, is displayed on the right-side display along with the right rear image.

8. A method for displaying images on a vehicle surrounding image display device, which displays a left rear image, which is an image captured by a left rear camera that captures the left rear of a vehicle, on a display inside the vehicle, The left rear image is displayed on a left-side display located in front of the driver's seat of the vehicle within the vehicle interior, and positioned to the left of the driver's seat as viewed from the driver's perspective. A method for displaying images on a vehicle's surroundings display device, wherein, when the vehicle turns left at an intersection, the right front image, which is an image of the right front of the vehicle captured by the vehicle's front camera, is displayed on the left display along with the left rear image.

Images