Video display device and video display method

Abstract

To provide a technique that can appropriately achieve both convenience of video enlargement and an alert for a door open state when displaying an overhead view video of a vehicle.SOLUTION: A video display device includes: a camera that images a periphery of a vehicle; a control unit that outputs an overhead image in which the vehicle is viewed from a viewpoint above the vehicle, the overhead image being obtained by synthesizing images captured by the camera; a display unit that displays the overhead image output by the control unit; and a sensor that detects an open state of the vehicle door, The control unit is configured to: enlarge the overhead image and display the overhead image on the display unit; and cancel the enlargement of the overhead image in response to the detection of the open state of the vehicle door by the sensor.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an image display device and an image display method. [Background technology]

[0002] Patent Document 1 discloses an image display device for a vehicle. This device converts image signals from multiple cameras capturing images of the vehicle's surroundings into image signals viewed from a viewpoint above the vehicle, and then combines the converted image signals. This results in a bird's-eye view of the vehicle viewed from a viewpoint above the vehicle. This device also stores in advance the trajectories of the vehicle doors as they are opened and closed, and displays the stored door trajectories on the bird's-eye view when the vehicle is stopped. This allows occupants to check whether there is an area between the vehicle and adjacent parked vehicles where they can open the vehicle doors and exit the vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-51391 Summary of the Invention [Problem to be solved by the invention]

[0004] In the device described in Patent Document 1, it is conceivable to reflect the current door open state in the overhead image to alert the occupants. It is also conceivable to add a function to enlarge and display the overhead image to improve occupant convenience. However, when enlarging and displaying the overhead image, there is a risk that the door open state reflected for alerting the occupants may be located outside the screen and not be displayed. The present disclosure provides a technology that can appropriately achieve both the convenience of image enlargement and alerting the occupants to the door open state when displaying an overhead image of a vehicle. [Means for solving the problem]

[0005] An image display device according to one embodiment of the present disclosure includes a camera that captures images of the area around the vehicle, a control unit that outputs an overhead image of the vehicle from a viewpoint above the vehicle, obtained by synthesizing images captured by the camera, a display unit that displays the overhead image output by the control unit, and a sensor that detects whether the vehicle door is open, and the control unit is configured to enlarge the overhead image and display it on the display unit, and to cancel the enlargement of the overhead image when the sensor detects that the vehicle door is open.

[0006] In this image display device, images captured by cameras capturing images of the vehicle's surroundings are combined to produce a bird's-eye view image, which is then displayed on the display unit. The bird's-eye view image is then enlarged and displayed on the display unit. The enlargement of the bird's-eye view image is canceled when a sensor detects that a vehicle door is open. By canceling the enlarged view, a bird's-eye view image overlooking the entire vehicle is displayed on the display unit, allowing the image display device to draw attention to the open door on the bird's-eye view image. In this way, the image display device allows the enlarged display to continue, but cancels the enlarged display when the door is open, thereby appropriately achieving both the convenience of image enlargement and the ability to draw attention to the open door.

[0007] In one embodiment, the control unit may be configured to determine whether all doors detected as open by the sensors are within the enlarged overhead-view image displayed on the display unit, and not cancel the enlargement of the overhead-view image when it is determined that all open doors are within the overhead-view image. Since the enlargement of the overhead-view image is maintained when all open doors are within the overhead-view image, unnecessary cancellation of the enlargement of the overhead-view image is avoided. Therefore, this image display device can more appropriately achieve both the convenience of image enlargement and the alert for open door status.

[0008] A video display method according to another aspect of the present disclosure includes the steps of: generating an overhead video of a vehicle from a viewpoint above the vehicle by synthesizing videos captured by cameras capturing the surroundings of the vehicle; enlarging and displaying the generated overhead video on a display unit; detecting whether a vehicle door is open; and canceling the enlargement of the overhead video in response to the detection of the open state of the vehicle door. This video display method achieves the same effects as the video display device described above. [Effects of the Invention]

[0009] According to the present disclosure, a technology is provided that can appropriately balance the convenience of image enlargement and the need to draw attention to the door open state when displaying an overhead image of a vehicle. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of a vehicle equipped with an image display device according to an embodiment. [Figure 2] FIG. 2 is an example showing the imaging areas of a plurality of cameras provided on the vehicle 2. In FIG. [Figure 3] FIG. 3 is an example of an overhead image displayed on the display unit. [Figure 4] FIG. 4 is a diagram illustrating an example of a display unit having a display area for camera images and a display area for overhead images. [Figure 5] (A) of FIG. 5 is an example of an overhead view image, (B) of FIG. 5 is an example of an enlarged overhead view image, and (C) of FIG. 5 is another example of an enlarged overhead view image. [Figure 6] (A) of Figure 6 is an example of an enlarged overhead view image, and (B) of Figure 6 is an example of an overhead view image when the enlargement of the overhead view image is canceled in response to the door being opened. [Figure 7] FIG. 7 is a flowchart showing the operation of the video display device. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0012] [Vehicle configuration] Fig. 1 is a block diagram showing an example of the configuration of a vehicle equipped with an image display device according to an embodiment. As shown in Fig. 1, the image display device 1 is mounted on a vehicle 2, for example. The vehicle 2 may be a vehicle driven by a driver, or may be an autonomous vehicle.

[0013] The vehicle 2 includes a plurality of cameras 10, sensors 11, a control unit 12, a display unit 13, and an operation unit 14. The image display device 1 includes a plurality of cameras 10, sensors 11, a control unit 12, and a display unit 13.

[0014] The multiple cameras 10 are a group of cameras that capture images of the periphery of the vehicle 2. The multiple cameras 10 include, for example, a front camera 101, a right side camera 102, a left side camera 103, and a rear camera 104. FIG. 2 is an example showing the imaging areas of the multiple cameras provided on the vehicle 2. As shown in FIG. 2, the front camera 101 is provided on a front bumper or the like and captures an image of a front range 101A of the vehicle 2. The right side camera 102 is provided on a door mirror or the like and captures an image of a right side range 102A of the vehicle 2. The left side camera 103 is provided on a door mirror or the like and captures an image of a left side range 103A of the vehicle 2. The rear camera 104 is provided on a rear bumper or the like and captures an image of a rear range 104A of the vehicle 2. Each of the multiple cameras 10 outputs an image. The image includes video and still images.

[0015] The sensor 11 is a detector that detects the open state of a door of the vehicle 2. For example, the sensor 11 is a linear sensor that detects the door opening degree, or a sensor that detects the state of a courtesy switch. The door of the vehicle 2 may be not only a door disposed at each seat, but also a trunk door, a luggage room door, an engine room cover, etc. The type of door of the vehicle 2 is not particularly limited, and may be not only a general door for getting in and out with a vertically arranged hinge, but also a hatch type, a sliding type, a double door type, a gull-wing type door, etc.

[0016] The control unit 12 has a function of controlling the video display device. The control unit 12 is configured by an ECU (Electronic Control Unit), for example. The ECU is an electronic control unit that has a processor such as a CPU (Central Processing Unit), storage devices such as ROM and RAM, storage devices such as a CAN (Controller Area Network) communication circuit, and input / output circuits. The control unit 12 generates an overhead video by synthesizing videos captured by multiple cameras 10. The overhead video is a video looking down on the vehicle 2 from a viewpoint above the vehicle 2, and includes both moving and still images.

[0017] The display unit 13 displays the overhead image output by the control unit 12. The display unit 13 is, for example, a display device. FIG. 3 shows an example of an overhead image displayed on the display unit. As shown in FIG. 3, the display unit 13 displays the overhead image G generated by the control unit 12. The control unit 12 generates the overhead image G including a vehicle object OB corresponding to the vehicle 2. The vehicle object OB is pre-stored in a storage device of the ECU. The control unit 12 inputs image signals from the multiple cameras 10 and performs image processing to generate an overhead image of the vehicle 2 viewed from directly above. The viewpoint conversion is performed based on the respective positions and imaging directions of the multiple cameras 10, with a reference point O of the vehicle 2 as the reference point. The image processing for the viewpoint conversion may be performed by software or a hardware circuit.

[0018] The control unit 12 displays a front area AR1 of the vehicle 2 using an image signal captured by the front camera 101. The control unit 12 displays a right side area AR2 of the vehicle 2 using an image signal captured by the right side camera 102. The control unit 12 displays a left side area AR3 of the vehicle 2 using an image signal captured by the left side camera 103. The control unit 12 displays a rear area AR4 of the vehicle 2 using an image signal captured by the rear camera 104. As a result, an overhead image G is generated.

[0019] The control unit 12 may display the overhead image on the entire screen of the display unit 13 as shown in FIG. 3, or may divide the screen of the display unit 13 and display the overhead image in a partial display area. FIG. 4 is a diagram illustrating an example of a display unit having a display area for a camera image and a display area for an overhead image. As shown in FIG. 4, the display unit 13 may have a first display area 130 and a second display area 131. The first display area 130 is an area where the camera image is displayed. In the example of FIG. 4, an image from the front camera 101 is displayed, and a parking space demarcated by a white line is shown. The control unit 12 may switch the camera image displayed in the first display area 130 depending on the traveling direction. The second display area 131 is an area where the overhead image shown in FIG. 3 is displayed. In the example of FIG. 4, the positional relationship between the vehicle and the parking space can be grasped at a glance by checking the overhead image. The control unit 12 can appropriately notify the occupants of information about the surroundings of the vehicle 2 by displaying images of the surroundings of the vehicle 2 from various angles. Note that the occupants are people riding in the vehicle, and include the driver, passengers, etc.

[0020] The control unit 12 can enlarge the overhead view image and display it on the display unit 13. As shown in Fig. 3, the basic overhead view image before enlargement displays the entire vehicle object OB and includes an image of the entire periphery of the vehicle object OB. Enlarging the overhead view image means that a partial area included in the basic overhead view image is selected, and the image of the selected area is resized to fit the size of the display area of ​​the overhead view image (second display area 131 or the entire screen of the display unit 13).

[0021] The selection of a partial area is performed by the occupant. For example, the operation unit 14 shown in FIG. 1 accepts the operation of the occupant. The operation unit 14 may be a switch or a button, or may be a touch panel. When the operation unit 14 is configured as a touch panel, the operation unit 14 is configured integrally with the display unit 13.

[0022] The occupant's operation may be an operation to specify a predetermined partial area such as an "upper area" or a "lower area." The predetermined partial area may be, for example, two areas, an "upper area" and a "lower area," which are obtained by dividing the screen into two halves, or four areas, an "upper right area," an "upper left area," a "lower right area," and a "lower left area," which are obtained by dividing the screen into four quarters. The specifying operation may be an operation to select a presented partial area, or an operation to tap or pinch out with a finger on a part of the overhead image displayed on a touch panel that the occupant wants to enlarge. The operation unit 14 may be a device that allows input by voice or gesture.

[0023] The control unit 12 determines the area to be enlarged based on the occupant's operation received by the operation unit 14. The control unit 12 enlarges the determined area and displays it on the display unit 13. (A) of Fig. 5 is an example of an overhead view image, (B) of Fig. 5 is an example of an enlarged overhead view image, and (C) of Fig. 5 is another example of an enlarged overhead view image.

[0024] As shown in Fig. 5A, it is assumed that a bird's-eye view video is displayed in second display area 131. When the occupant selects the area above the vehicle as the area to be enlarged, control unit 12 enlarges first area R1 in the bird's-eye view video shown in Fig. 5A, which accommodates approximately the upper half of vehicle object OB, to the size of second display area 131 and displays it in second display area 131 (Fig. 5B). At this time, vehicle object OB1 corresponding to the upper half of vehicle object OB is displayed in second display area 131.

[0025] Furthermore, for example, if the occupant selects the upper right region of the vehicle as the region to be enlarged, the control unit 12 enlarges the second region R2, which fits approximately the upper right of the vehicle object OB in the overhead image shown in Fig. 5(A), to the size of the second display region 131 and displays it in the second display region 131 (Fig. 5(C)). At this time, the vehicle object OB2, which corresponds to the upper right of the vehicle object OB, is displayed in the second display region 131.

[0026] 5A and enlarge the size of the cropped image, or may enlarge the entire overhead image and then display the portion corresponding to the cropped image over the entire second display area 131. Furthermore, the control unit 12 may generate the vehicle objects OB1 and OB2 from the vehicle object OB, or may store the vehicle objects OB1 and OB2 in advance in a storage device of the ECU.

[0027] When sensor 11 detects that a door of vehicle 2 is open, control unit 12 cancels the enlargement of the overhead-view image. FIG. 6A shows an example of the enlarged overhead-view image, and FIG. 6B shows an example of the overhead-view image when the enlargement of the overhead-view image is canceled when the door is opened. As shown in FIG. 6A, control unit 12 enlarges an area of ​​the overhead-view image that fits approximately the upper half of vehicle object OB to the size of second display area 131 and displays it in second display area 131. In this state, it is assumed that sensor 11 detects that the rear right door has been opened. In this case, control unit 12 cancels the enlargement of the overhead-view image shown in FIG. 6A. When the enlargement of the overhead-view image is canceled, a basic overhead-view image is displayed in second display area 131, as shown in FIG. 6B. The basic overhead-view image includes the entire vehicle object and an image of the entire surroundings of the vehicle object.

[0028] Here, the control unit 12 may display a vehicle object OB3 for calling attention instead of the vehicle object OB shown in Fig. 3. The vehicle object OB3 includes a door object D1 indicating that the rear right door is open. This allows the control unit 12 to call attention to the open door state on the overhead view video. The vehicle object OB3 for calling attention is prepared in advance for each open door state and stored in the storage device of the ECU.

[0029] If a condition that does not require cancellation is satisfied when the open state of the doors of the vehicle 2 is detected by the sensor 11, the control unit 12 does not need to cancel the enlargement of the overhead image. The condition that does not require cancellation is that all doors that are detected to be open by the sensor 11 are within the enlarged overhead image displayed on the display unit 13.

[0030] When enlarging the overhead view image, the control unit 12 stores information about the enlarged area in the storage device of the ECU. The information about the enlarged area is information that can identify the door positions included in the enlarged overhead view image. When the sensor 11 detects that a door of the vehicle 2 is open, the control unit 12 determines the position of the open door based on the detection signal of the sensor 11. Then, the control unit 12 determines whether all open doors are included in the overhead view image based on the determined door positions and the information about the enlarged area. In other words, it determines whether a warning about the open door can be displayed in the enlarged overhead view image.

[0031] For example, in the enlarged overhead-view image shown in (A) of FIG. 6, the front door of the vehicle is displayed, and therefore a door object indicating an open state can be superimposed on the front door of the vehicle. Therefore, even if the sensor 11 detects that the front door of the vehicle 2 is open, the control unit 12 does not cancel the enlargement of the overhead-view image. In this way, by setting the condition that does not require cancellation, unnecessary cancellation of the enlargement of the overhead-view image is avoided. On the other hand, if the sensor 11 detects that the rear door of the vehicle 2 is open, the rear of the vehicle is not depicted on the vehicle object OB1, and therefore a door object indicating an open state cannot be superimposed. Therefore, when the sensor 11 detects that the rear door of the vehicle 2 is open, the control unit 12 cancels the enlargement of the overhead-view image. In this way, when the condition that does not require cancellation is not met, the enlargement of the overhead-view image is canceled.

[0032] The control unit 12 may emphasize the door object D1. For example, the control unit 12 may change the color of the door object D1. Furthermore, instead of synthesizing the door as an object, the control unit 12 may convert the viewpoint of the door captured in the camera image and synthesize it as an actual image. In this case, the control unit 12 does not need to store the attention-attracting vehicle object OB3 in advance, thereby reducing the calculation cost.

[0033] [Video display device operation (video display method)] Fig. 7 is a flowchart showing the operation of the video display device. The flowchart shown in Fig. 7 is executed by the control unit 12, for example, when an operation by the occupant is received.

[0034] 7, in step S10, the control unit 12 determines whether or not an enlarged overhead image is being displayed. If the overhead image is being enlarged and displayed based on an operation by the occupant received by the operation unit 14, the control unit 12 determines that the enlarged overhead image is being displayed (see, for example, (A) of FIG. 6).

[0035] If it is determined that the enlarged overhead image is being displayed (step S10: YES), the control unit 12 determines whether or not the door is open in step S12. The control unit 12 determines whether or not the door is open based on the detection result of the sensor 11.

[0036] If it is determined that the door is in an open state (step S12: YES), the control unit 12 determines in step S14 whether the open door is in the enlarged overhead-view image. When the sensor 11 detects that a door of the vehicle 2 is in an open state, the control unit 12 determines the position of the open door based on the detection signal of the sensor 11. Then, based on the determined door position and the door positions that can be drawn in the enlarged overhead-view image, the control unit 12 determines that the open door is in the enlarged overhead-view image if all of the open doors are in a door position that can be drawn. In the example shown in FIG. 6A, when the front door of the vehicle is in an open state, the control unit determines that the open door is in the enlarged overhead-view image. Based on the determined door position and the door positions that can be drawn in the enlarged overhead-view image, the control unit 12 determines that the open door is not in the enlarged overhead-view image if at least one open door is not in a door position that can be drawn. In the example shown in FIG. 6(A), when the rear door of the vehicle is in an open state, the control unit determines that the open door is not within the enlarged overhead image.

[0037] If it is determined that the open door is not within the enlarged overhead-view image (step S14: NO), the control unit 12 cancels the enlargement of the overhead-view image in step S16. As a result, the control unit 12 displays the basic overhead-view image in step S18 (see, for example, FIG. 6B). The basic overhead-view image includes the entire vehicle object and an image of the entire periphery of the vehicle object, so the open door will fit within the overhead-view image. At this time, the control unit 12 may display a vehicle object OB3 for calling attention to the open state of the door, or may display an overhead-view image in which the door captured in the camera image is converted from a viewpoint and synthesized as an actual image. In this way, the open door is called attention to in the overhead-view image.

[0038] The flowchart ends when it is determined that the enlarged overhead image is not being displayed (step S10: NO), when it is determined that the door is not open (step S12: NO), when it is determined that the open door is within the enlarged overhead image (step S14: YES), or when step S18 ends. The flowchart is repeatedly executed from the beginning until a driver operation to end the flowchart is received.

[0039] [Summary of the embodiment] According to the image display device 1, images captured by a plurality of cameras 10 capturing images of the periphery of the vehicle 2 are combined, and a bird's-eye view image is displayed on the display unit 13. The bird's-eye view image is enlarged and displayed on the display unit 13. The enlargement of the bird's-eye view image is canceled when the sensor 11 detects that a door of the vehicle 2 is open. By canceling the enlarged state, a bird's-eye view image overlooking the entire vehicle is displayed on the display unit 13, and the image display device 1 can call attention to the open door on the bird's-eye view image. In this way, the image display device 1 allows the enlarged display to continue, but cancels the enlarged display when the door is open, thereby appropriately achieving both the convenience of image enlargement and the ability to call attention to the open door.

[0040] According to the image display device 1, the enlargement of the overhead view image is maintained when all open doors are within the overhead view image, thereby avoiding unnecessary cancellation of the enlargement of the overhead view image. Thus, the image display device 1 can more appropriately achieve both the convenience of image enlargement and the alert for the open door state.

[0041] Although exemplary embodiments have been described above, various omissions, substitutions, and modifications may be made without being limited to the above-described exemplary embodiments.

[0042] For example, the control unit 12 may generate an overhead image using only a single camera, provided that the vehicle 2 is moving. The control unit 12 stores the camera images in the storage device of the ECU while the vehicle 2 is moving, and generates the overhead image using past images acquired as the vehicle 2 moves and real-time images. An overhead image generated using only a single camera can also be enlarged and displayed. Therefore, the image display device 1 is not limited by the number of cameras, and can appropriately achieve both the convenience of image enlargement and the ability to alert the driver to an open door state.

[0043] Furthermore, although the image display device 1 enlarges and displays the overhead image based on the occupant's operation received by the operation unit 14, the image display device 1 may also enlarge and display the overhead image based on information other than the occupant's operation. For example, the image display device 1 may be equipped with an external sensor such as a radar, and when an object such as a moving body or an obstacle is detected, the image display device 1 may enlarge and display the area where the detected object is located. Thus, the image display device 1 can appropriately achieve both the convenience of image enlargement and the ability to alert the driver that the door is open, regardless of whether or not the operation unit 14 is provided. [Explanation of symbols]

[0044] 1...image display device, 2...vehicle, 10...plurality of cameras, 11...sensor, 12...control unit, 13...display unit.

Claims

1. A camera that photographs the area around the vehicle; a control unit that outputs a bird's-eye view image of the vehicle from a viewpoint above the vehicle, the bird's-eye view image being obtained by combining the images captured by the cameras; and a display unit that displays the overhead image output by the control unit; a sensor for detecting an open state of a door of the vehicle; Equipped with The control unit enlarging the overhead image and displaying it on the display unit; and canceling the enlargement of the overhead view image in response to the sensor detecting an open state of the door of the vehicle. Video display device.

2. The control unit determining whether or not all doors detected as being open by the sensors are within the enlarged overhead image displayed on the display unit; and in response to a determination that all of the doors in an open state are present within the overhead image, the enlargement of the overhead image is not canceled.

2. The image display device according to claim 1.

3. a step of synthesizing images captured by cameras capturing images of the periphery of the vehicle to generate an overhead image overlooking the vehicle from a viewpoint above the vehicle, and enlarging and displaying the generated overhead image on a display unit; detecting an open state of a door of the vehicle; canceling the enlargement of the overhead view image in response to detection of an open state of a door of the vehicle; A video display method including:

Images