Indication device

Abstract

To provide a display device that makes it easy to recognize an opening degree of a door of a vehicle in a composite vehicle image displayed on a display unit.SOLUTION: A display device 100 includes: an image acquisition unit 11 that acquires a captured image of a camera group 1 that captures an image of surroundings of a vehicle; a composite image generation unit 12 that synthesizes a plurality of captured images to generate a composite vehicle image representing the vehicle viewed from an imaginary viewpoint; and a display control unit 13 that displays the composite vehicle image on an on-vehicle HMI3. Based on a direction of the imaginary viewpoint with respect to the vehicle, the composite image generation unit 12 generates a composite vehicle image so as to represent a door that is opened wider when the imaginary viewpoint is located in a predetermined first direction than when the imaginary viewpoint is located in a predetermined second direction in which an opening of the vehicle door is more easily seen from the imaginary viewpoint than in the first direction.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a display device.

Background Art

[0002] Conventionally, an image processing device is known that generates a composite image showing a vehicle as seen from a virtual viewpoint based on captured images of the periphery of a vehicle captured by a plurality of cameras, draws the opening and closing trajectories of the vehicle doors on the image of the vehicle included in the composite image, and outputs and displays the composite image on a display (for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] It has been considered to display a composite vehicle image representing a vehicle as seen from a virtual viewpoint on an in-vehicle display unit. In such a composite vehicle image, even if the actual opening state of the vehicle doors is the same, the opening state of the vehicle doors as seen from the virtual viewpoint may appear different. As a result, depending on the direction with respect to the vehicle at the virtual viewpoint, it may be difficult to recognize the opening state of the vehicle doors in the composite vehicle image displayed on the display unit.

Means for Solving the Problems

[0005] A display device according to one aspect of the present disclosure includes an image acquisition unit that acquires images captured by a plurality of cameras that capture images of the area around a vehicle, a composite image generation unit that combines the plurality of captured images to generate a composite vehicle image representing the vehicle as seen from a virtual viewpoint, and a display control unit that displays the composite vehicle image on an in-vehicle display unit. The composite image generation unit generates the composite vehicle image based on the direction of the virtual viewpoint relative to the vehicle, such that when the virtual viewpoint is located in a predetermined first direction, the image represents a door that is more widely open than when the virtual viewpoint is located in a predetermined second direction, which makes the door opening of the vehicle more visible from the virtual viewpoint than when the virtual viewpoint is located in the first direction.

[0006] In a display device according to one aspect of this disclosure, multiple captured images acquired using multiple cameras are combined to generate a composite vehicle image representing the vehicle as seen from a virtual viewpoint. The generated composite vehicle image is displayed on an in-vehicle display unit. When the virtual viewpoint is located in a predetermined first direction, the composite vehicle image is generated to represent a door that is more open than when the virtual viewpoint is located in a predetermined second direction. Therefore, the open door is more easily visible in the composite vehicle image as seen from a virtual viewpoint located in the first direction. Thus, according to a display device according to one aspect of this disclosure, it is easier to recognize the degree to which the vehicle door is open in the composite vehicle image displayed on the display unit.

[0007] In one embodiment, the vehicle door includes a hinged door that opens by rotating around a hinge fixed to the vehicle body, the first direction being the direction not facing the opening of the hinged door, and the second direction being the direction facing the opening of the hinged door, and the composite image generation unit may generate a composite vehicle image such that when the virtual viewpoint is located in the first direction, it represents the hinged door opened with a larger amount of rotation compared to when the virtual viewpoint is located in the second direction. In this case, it becomes easier to recognize the degree to which the vehicle's hinged door is open in the composite vehicle image displayed on the display unit.

[0008] In one embodiment, the vehicle door includes a sliding door that moves with a predetermined shift amount in the vehicle width direction relative to the vehicle body and also moves in the front-rear direction to open. The first direction is the direction not facing the opening of the sliding door, and the second direction is the direction facing the opening of the sliding door. The composite image generation unit may generate a composite vehicle image such that, when the virtual viewpoint is located in the first direction, it represents the sliding door that has moved with a larger shift amount in the vehicle width direction compared to when the virtual viewpoint is located in the second direction. In this case, it becomes easier to recognize the degree to which the vehicle's sliding door is open in the composite vehicle image displayed on the display unit. [Effects of the Invention]

[0009] According to one aspect of the present disclosure, the display device makes it easier to recognize the degree to which the vehicle doors are open in the composite vehicle image displayed on the display unit. [Brief explanation of the drawing]

[0010] [Figure 1] A block of diagrams showing a display device according to one embodiment. [Figure 2] (a) is an example of a composite vehicle image of a hinged door that opens to a degree corresponding to the actual opening of a vehicle door when the virtual viewpoint is located in the second direction. (b) is an example of a composite vehicle image of a hinged door that opens to a degree corresponding to the actual opening of a vehicle door when the virtual viewpoint is located in the first direction. (c) is an example of a composite vehicle image showing a hinged door that is opened wider when the virtual viewpoint is located in the first direction compared to when the virtual viewpoint is located in the second direction. [Figure 3] This diagram illustrates the trend in visibility of vehicle door openings. [Figure 4] (a) is a side view illustrating a sliding door that opens to a degree corresponding to the actual opening of a vehicle door. (b) is a top view illustrating a sliding door that opens to a degree corresponding to the actual opening of a vehicle door. (c) is a top view illustrating a sliding door that is opened wider than the degree corresponding to the actual opening of a vehicle door. [Figure 5]Figure 1 is a flowchart illustrating an example of ECU processing. [Figure 6] This flowchart shows another example of the ECU processing shown in Figure 1. [Modes for carrying out the invention]

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0012] Figure 1 is a block diagram illustrating a display device according to one embodiment. As shown in Figure 1, the display device 100 is mounted in a vehicle such as a passenger car. The display device 100 is a device for displaying a composite vehicle image (3D view) representing the vehicle as seen from a virtual viewpoint on a display unit such as an in-vehicle display. Some functions of the display device 100 may be executed on a server capable of communicating with the vehicle. The vehicle doors include hinged doors that open by rotating around a hinge fixed to the vehicle body as a pivot point.

[0013] [Display device configuration] The configuration of the display device 100 according to this embodiment will be described below with reference to Figure 1. As shown in Figure 1, the display device 100 includes an ECU [Electronic Control Unit] 10.

[0014] The ECU10 is an electronic control unit having a CPU (Central Processing Unit) and a memory unit. The memory unit consists of, for example, ROM (Read Only Memory), RAM (Random Access Memory), and EEPROM (Electrically Erasable Programmable Read-Only Memory). The ECU10 implements various functions by, for example, executing programs stored in the memory unit with the CPU. The ECU10 may be composed of multiple electronic units.

[0015] ECU 10 is connected to the camera group 1, the door opening degree sensor 2, and the HMI [Human Machine Interface] 3.

[0016] The camera group 1 is a plurality of cameras that image the periphery of the vehicle. The camera group 1 has, for example, a front camera, a rear camera, and a pair of left and right side view cameras. The front camera is provided, for example, on the front bumper of the vehicle and images the periphery in front of the vehicle. The rear camera is provided, for example, on the rear bumper of the vehicle and images the periphery behind the vehicle. The side view camera is provided, for example, near the door mirror of the vehicle and images the periphery on the side of the vehicle. The camera group 1 has, for example, a wide-angle imaging angle and can also image the vehicle body. The camera group 1 transmits an imaging image related to the periphery of the vehicle to the ECU 10.

[0017] The door opening degree sensor 2 is a detection device that detects the opening degree of the vehicle door. The door opening degree sensor 2 may be a linear sensor that detects the opening degree of the vehicle door. When the door is a hinged door, the linear sensor may detect the rotation amount of the hinged door or the angle of the door. When the door is a sliding door, the linear sensor may detect the shift amount, which is the amount of movement of the sliding door in the vehicle width direction. The door opening degree sensor 2 transmits information related to the opening degree or opening / closing of the door to the ECU 10.

[0018] The HMI 3 is an interface for performing information input / output between the ECU 10 and the occupant. The HMI 3 includes, for example, a display, a speaker, a microphone, etc. provided in the vehicle interior. The HMI 3 performs image output of the display and voice output from the speaker according to a control signal from the ECU 10. The display may function as a touch panel. The display may be a center display, a display for navigation, or a HUD [Head Up Display]. The HUD presents information to the occupant by projecting an image on the front windshield of the vehicle.

[0019] The HMI3 receives an operation by the occupant to select a virtual viewpoint. The operation to select a virtual viewpoint may be, for example, a touch operation on an image corresponding to the virtual viewpoint displayed on the touch panel, or an input operation of a value of the direction (angle) of the vehicle corresponding to the virtual viewpoint. The HMI3 displays the generated composite vehicle image to the occupant.

[0020] Next, the functional configuration of the ECU10 will be described. The ECU10 includes an image acquisition unit 11, a composite image generation unit 12, and a display control unit 13.

[0021] The image acquisition unit 11 acquires captured images of a plurality of cameras that image the periphery of the vehicle. The image acquisition unit 11 acquires the captured image captured by the camera group 1.

[0022] The composite image generation unit 12 synthesizes a plurality of captured images and generates a composite vehicle image representing the vehicle as seen from the virtual viewpoint. The composite image generation unit 12 generates, based on the captured images captured by the camera group 1, a composite vehicle image as seen from a virtual viewpoint, which is an arbitrary virtual viewpoint, of the periphery of the vehicle including the vehicle. The composite image generation unit 12 combines the captured images captured by the camera group 1 by a known method, except for the degree of opening of the door, to generate a composite vehicle image as seen from the virtual viewpoint.

[0023] Based on the direction of the vehicle with respect to the virtual viewpoint, when the virtual viewpoint is located in a predetermined first direction, the composite image generation unit 12 generates the composite vehicle image so as to represent a door that is more widely open than when the virtual viewpoint is located in a predetermined second direction where the opening of the vehicle door is more visible from the virtual viewpoint than the first direction. The first direction is a predetermined direction in which it is difficult to recognize the degree of opening of the vehicle door compared to the second direction. The second direction is a predetermined direction in which the opening of the vehicle door is more visible from the virtual viewpoint than the first direction.

[0024] Figure 2(a) is an example of a composite vehicle image of a hinged door that opens to a degree corresponding to the actual opening of a vehicle door when the virtual viewpoint is located in the second direction. Figure 2(b) is an example of a composite vehicle image of a hinged door that opens to a degree corresponding to the actual opening of a vehicle door when the virtual viewpoint is located in the first direction. Figure 2(c) is an example of a composite vehicle image showing a hinged door that is opened wider when the virtual viewpoint is located in the first direction compared to when the virtual viewpoint is located in the second direction.

[0025] Figure 2(a) shows a composite vehicle image 40A of vehicle 20 with its hinged door 21 open, viewed from the left rear of vehicle 20. The viewpoint in Figure 2(a) faces the opening 22 of the hinged door 21. From the viewpoint in Figure 2(a), the opening 22 of the hinged door 21 is easily visible. Therefore, even if the composite vehicle image 40A is generated and displayed on the HMI3 with the door in this position, the occupants of vehicle 20 can easily recognize the degree to which the vehicle 20's door is open in the composite vehicle image 40A displayed on the HMI3.

[0026] Figure 2(b) shows a composite vehicle image 40B of vehicle 20 with the hinged door 21 open to the same extent as in Figure 2(a), viewed from the front of vehicle 20. The viewpoint in Figure 2(b) does not face the opening 22 of the hinged door 21. From the viewpoint in Figure 2(b), it is difficult to see the opening 22 of the hinged door 21. As a result, even though the hinged door 21 of vehicle 20 is open to the same extent as in Figure 2(a), in Figure 2(b) the door opening of the hinged door 21 may appear smaller compared to Figure 2(a). Thus, if a composite vehicle image 40B is generated with the same degree of door opening as in Figure 2(a) and displayed on the HMI3, the occupants of vehicle 20 may have difficulty recognizing the degree of door opening in the composite vehicle image 40B displayed on the HMI3.

[0027] Therefore, the composite image generation unit 12 generates a composite vehicle image 40C such that when the virtual viewpoint is located in the first direction, it represents the hinged door 21 that has opened with a larger rotational amount compared to when the virtual viewpoint is located in the second direction. Here, the first direction is the direction that does not face the opening 22 of the hinged door 21, and the second direction is the direction that faces the opening 22 of the hinged door 21.

[0028] Figure 2(c) shows a composite vehicle image 40C from a frontal view of the vehicle 20, where the hinged door 21 of the vehicle 20 is actually open to the same extent as in Figure 2(a), but with a larger door opening than in Figure 2(a). This makes it easy to see that the hinged door of the vehicle 20 is open. The degree to which the composite vehicle image 40C is generated with a larger door opening than the actual opening can be determined, for example, by using a predetermined relationship between the visibility of the door opening 22 of the vehicle 20 and the direction relative to the vehicle 20.

[0029] Figure 3 illustrates the trend in visibility of a vehicle door opening. As shown in Figures 2 and 3, when the door of vehicle 20 is a hinged door 21, the trend in visibility of the door opening 22 of vehicle 20 is as shown in Figure 3. In the example of Figure 3, the direction relative to vehicle 20 is defined as, for example, a viewpoint located on the left side of vehicle 20, with the front of vehicle 20 being 0 degrees and the rear of vehicle 20 being 180 degrees, and viewpoints defined at 45-degree intervals. In the case of vehicle 20 in Figure 3, the door opening 22 of vehicle 20 is most visible from a viewpoint located on the left side of vehicle 20, when vehicle 20 is viewed from the left rear (135-degree direction), and becomes less visible in the order of 90 degrees and 45 degrees, and is least visible when vehicle 20 is viewed directly in front (0-degree direction).

[0030] Although not shown in Figure 3, a viewpoint located to the right of the vehicle 20 may be considered, with the rear of the vehicle 20 being 180 degrees and the front of the vehicle 20 being 360 degrees. The visibility of the door opening 22 of the vehicle 20 may be symmetrical between the viewpoint on the left side of the vehicle 20 and the viewpoint on the right side of the vehicle 20. In this case, specifically, the door opening 22 of the vehicle 20 is most visible from the viewpoint of the vehicle 20 viewed from the left rear (135-degree direction), becomes difficult to see from the viewpoint of the vehicle 20 viewed from the rear (180-degree direction), and becomes most visible again from the viewpoint of the vehicle 20 viewed from the right rear (225-degree direction). From there, it becomes difficult to see in the order of 270 degrees and 315 degrees, and is most difficult to see from the viewpoint of the vehicle 20 viewed directly in front (360-degree direction).

[0031] In Figure 3, the amount of additional door opening is defined in 45-degree increments from 0 to 180 degrees. The amount of additional door opening is a correction amount for the door opening on a composite vehicle image, which is set in advance according to how easily (or difficultly) the door opening 22 of the vehicle 20 is visible. The amount of additional door opening is smaller for viewpoints where the door opening 22 of the vehicle 20 is easily visible, and larger for viewpoints where the door opening 22 of the vehicle 20 is difficult to see. The amount of additional door opening may also increase monotonically with respect to the direction of the vehicle 20, from the viewpoint where the door opening 22 of the vehicle 20 is most easily visible to the viewpoint where it is least easily visible. In calculating the amount of additional door opening, the value of the amount of additional door opening may be determined once one virtual viewpoint is determined.

[0032] Furthermore, the angular ranges of the first and second directions relative to the vehicle do not need to be predetermined. The first and second directions only need to be directions that allow for a relative comparison of the visibility of the vehicle's door openings from two virtual viewpoints. For example, if the first direction is 90 degrees, the second direction may be 135 degrees, and if the first direction is 45 degrees, the second direction may be 90 degrees.

[0033] The vehicle doors may include sliding doors 31 that move in the vehicle width direction by a predetermined shift amount relative to the vehicle body 30 and also move in the front-rear direction to open. Figure 4(a) is a side view illustrating a sliding door that opens to a door opening degree corresponding to the actual opening of a vehicle door. Figure 4(b) is a top view illustrating a sliding door that opens to a door opening degree corresponding to the actual opening of a vehicle door. Figure 4(c) is a top view illustrating a sliding door that is opened wider than the door opening degree corresponding to the actual opening of a vehicle door.

[0034] Figure 4(a) shows a composite vehicle image 40D of vehicle 30 with its sliding door 31 open, viewed from the left side of vehicle 30. The viewpoint in Figure 4(a) faces the opening 32 of the sliding door 31. From the viewpoint in Figure 4(a), the opening 32 of the sliding door 31 is easily visible. Therefore, even if the composite vehicle image 40D is generated and displayed on the HMI3 with the door open to this extent, the occupants of vehicle 30 can easily recognize the degree to which the vehicle 30's door is open in the composite vehicle image 40D displayed on the HMI3.

[0035] Figure 4(b) shows a composite vehicle image 40E with the sliding door 31 of the vehicle 30 open to the same extent as in Figure 4(a), but for the sake of explanation, it is shown from a viewpoint above the vehicle 30. The viewpoint in Figure 4(b) does not face the opening 32 of the sliding door 31. From the viewpoint in Figure 4(b), it is difficult to see the opening 32 of the sliding door 31. As a result, even though the sliding door 31 of the vehicle 30 is open to the same extent as in Figure 4(a), it is more difficult to see that the sliding door 31 is open in Figure 4(b) compared to Figure 4(a). Specifically, the composite vehicle image 40E shows the state in which the sliding door 31 has moved by a predetermined shift amount SD1 in the vehicle width direction relative to the body of the vehicle 30, but because the viewpoint in Figure 4(b) does not face the opening 32 of the sliding door 31, it is more difficult to see that the sliding door 31 is open compared to Figure 4(a). Thus, if a composite vehicle image 40E is generated with a door opening degree similar to that of Figure 4(a) and displayed on the HMI3, the occupants of vehicle 30 may have difficulty recognizing the degree to which the vehicle's doors are open in the composite vehicle image 40E displayed on the HMI3.

[0036] Therefore, the composite image generation unit 12 generates a composite vehicle image 40F such that when the virtual viewpoint is located in the first direction, it represents the sliding door 31 that has moved in the vehicle width direction with a larger shift amount compared to when the virtual viewpoint is located in the second direction. Here, the first direction is the direction that does not face the opening 32 of the sliding door 31, and the second direction is the direction that faces the opening 32 of the sliding door 31.

[0037] Figure 4(c) shows a composite vehicle image 40F, for the sake of explanation, from a viewpoint above the vehicle 30, where the sliding door 31 of the vehicle 30 is actually open to the same extent as in Figure 4(a), but with a larger door opening than in Figure 4(a). The composite vehicle image 40F shows the state in which the sliding door 31 has moved relative to the vehicle body of the vehicle 30 by a predetermined shift amount SD2 in the vehicle width direction. In this way, by setting the shift amount SD2 to be larger than the shift amount SD1 when the virtual viewpoint is located in the second direction, it is easy to visually confirm that the sliding door 31 of the vehicle 30 is open.

[0038] The display control unit 13 displays the synthesized vehicle image on the display of the in-vehicle HMI 3. The display control unit 13 displays the generated synthesized vehicle image as a 3D view on the display of the HMI 3.

[0039] [Display device operation] Next, the operation of the display device 100 will be explained with reference to the drawings. Figure 5 is a flowchart showing an example of the processing of the ECU in Figure 1. The processing of the ECU in Figure 5 is executed, for example, when a request is made to draw a composite vehicle image while the vehicle is stopped with the doors open.

[0040] As shown in Figure 5, in step S11, the ECU 10 of the display device 100 acquires images captured by multiple cameras using the image acquisition unit 11. The image acquisition unit 11 acquires images captured by the camera group 1 that captures the area around the vehicle. In step S11, the ECU 10 acquires the door opening degree using the composite image generation unit 12. The composite image generation unit 12 acquires the door opening degree based on the detection result of the door opening degree sensor 2.

[0041] In step S12, the ECU 10 acquires a virtual viewpoint using the composite image generation unit 12. The composite image generation unit 12 acquires a virtual viewpoint in response to an operation performed by a vehicle occupant on the HMI 3 to select a virtual viewpoint.

[0042] In step S13, the ECU 10 uses the composite image generation unit 12 to calculate the door opening angle for the composite image, which corresponds to the actual degree to which the vehicle's door is open. The composite image generation unit 12 calculates the door opening angle for the composite image, for example, so that it is equal to the acquired door opening angle.

[0043] In step S14, the ECU 10 uses the composite image generation unit 12 to calculate the amount of additional door opening for the composite image based on the direction of the virtual viewpoint relative to the vehicle. The composite image generation unit 12, for example, obtains a preset amount of additional door opening according to the direction of the virtual viewpoint relative to the vehicle and calculates the amount of additional door opening for the composite image. The amount of additional door opening is smaller for viewpoints where the vehicle's door opening is easily visible and larger for viewpoints where the vehicle's door opening is difficult to see.

[0044] In step S15, the ECU 10 generates a composite vehicle image using the composite image generation unit 12 to represent a door opened at a door opening angle for the composite image. The composite image generation unit 12, for example, adds an additional amount of door opening angle to the door opening angle for the composite image, which is equal to the door opening angle, and generates a composite vehicle image to represent a door opened at the added door opening angle. At this time, the additional amount of door opening angle is smaller for viewpoints where the opening of the vehicle's door is easily visible, and larger for viewpoints where the opening of the vehicle's door is difficult to see. Therefore, the composite vehicle image is generated to represent a door that is more open when the virtual viewpoint is located in the first direction compared to when the virtual viewpoint is located in the second direction.

[0045] In step S16, the ECU 10 displays the composite vehicle image on the in-vehicle display unit via the display control unit 13. The display control unit 13 displays the composite vehicle image, for example, on the display of the HMI 3. After that, the ECU 10 terminates the process shown in Figure 5.

[0046] Furthermore, the angular ranges of the first and second directions relative to the vehicle may be predetermined. For example, for a viewpoint located on the left side of the vehicle, the first direction may be predetermined to be a direction between 0 and 45 degrees that does not face the opening of the hinged door, and the second direction may be predetermined to be a direction between 45 and 180 degrees that faces the opening of the hinged door. In this case, the ECU 10 can perform the processing shown in Figure 6.

[0047] Figure 6 is a flowchart showing another example of the ECU processing in Figure 1. The ECU processing in Figure 6 is performed when a request is made to draw a composite vehicle image while the vehicle is parked with the doors open, similar to Figure 5.

[0048] As shown in Figure 6, in step S21, the ECU 10 acquires images from multiple cameras using the image acquisition unit 11, similar to step S11. The ECU 10 acquires the door opening angle using the composite image generation unit 12. In step S22, the ECU 10 acquires a virtual viewpoint using the composite image generation unit 12, similar to step S12.

[0049] In step S23, the ECU 10 determines whether the virtual viewpoint is located in the first direction using the composite image generation unit 12. If the ECU 10 determines that the virtual viewpoint is located in the first direction using the composite image generation unit 12 (S23: YES), it proceeds to step S26. If the ECU 10 determines that the virtual viewpoint is not located in the first direction using the composite image generation unit 12 (S23: NO), it proceeds to step S24.

[0050] In step S24, the ECU 10 determines whether the virtual viewpoint is located in the second direction using the composite image generation unit 12. If the ECU 10 determines that the virtual viewpoint is located in the second direction (S24: YES), it proceeds to step S25. If the ECU 10 determines that the virtual viewpoint is not located in the second direction (S24: NO), it terminates the process shown in Figure 6.

[0051] In step S25, the ECU 10 generates a composite vehicle image using the composite image generation unit 12 to represent a door that has opened to a door opening corresponding to the actual door opening. The composite image generation unit 12, for example, calculates a door opening for the composite image that is equal to the door opening angle, and generates a composite vehicle image to represent a door that has opened to a door that is equal to the door opening angle without adding an additional amount for the door opening.

[0052] Meanwhile, in step S26, the ECU 10 uses the composite image generation unit 12 to generate a composite vehicle image that represents a door that is more open than when the virtual viewpoint is located in the second direction. The composite image generation unit 12, for example, adds an additional amount to the door opening angle for the composite image that is equal to the door opening angle, and generates a composite vehicle image that represents a door that is open at an angle greater than the door opening angle.

[0053] In step S27, the ECU 10 displays the synthesized vehicle image on the in-vehicle display unit via the display control unit 13. The display control unit 13 displays the synthesized vehicle image, for example, on the display of the HMI 3. After that, the ECU 10 terminates the process shown in Figure 6.

[0054] According to the display device 100 described above, multiple captured images acquired using the camera group 1 are combined by the composite image generation unit 12 to generate composite vehicle images 40C and 40F representing the vehicle as seen from a virtual viewpoint. The generated composite vehicle images 40C and 40F are displayed on the in-vehicle HMI 3. When the virtual viewpoint is located in a predetermined first direction, the composite vehicle images 40C and 40F are generated to represent a door that is more open compared to when the virtual viewpoint is located in a predetermined second direction. Therefore, in the composite vehicle images 40C and 40F viewed from a virtual viewpoint located in the first direction, the open door is more easily visible. In this way, the display device 100 makes it easier to recognize the degree to which the vehicle door is open in the composite vehicle images 40C and 40F displayed on the HMI 3.

[0055] The vehicle doors include hinged doors 21 that open by rotating around a hinge fixed to the body of the vehicle 20. The first direction is the direction not facing the opening 22 of the hinged door 21, and the second direction is the direction facing the opening 22 of the hinged door 21. The composite image generation unit 12 generates a composite vehicle image 40C such that when the virtual viewpoint is located in the first direction, it represents the hinged door 21 opened with a larger amount of rotation compared to when the virtual viewpoint is located in the second direction. In this case, it becomes easier to recognize the degree to which the hinged door 21 of the vehicle 20 is open in the composite vehicle image 40C displayed on the HMI 3.

[0056] The vehicle's doors include a sliding door 31 that moves in the vehicle width direction with a predetermined shift amount SD1 relative to the vehicle body 30, and also moves in the front-rear direction to open. The first direction is the direction not facing the opening 32 of the sliding door 31, and the second direction is the direction facing the opening 32 of the sliding door 31. The composite image generation unit 12 generates a composite vehicle image 40F such that when the virtual viewpoint is located in the first direction, it represents the sliding door 31 that has moved in the vehicle width direction with a larger shift amount SD2 compared to when the virtual viewpoint is located in the second direction. In this case, it becomes easier to recognize the degree to which the sliding door 31 of the vehicle 30 is open in the composite vehicle image 40F displayed on the HMI 3.

[0057] While embodiments of this disclosure have been described above, this disclosure is not limited to the embodiments described above. This disclosure can be implemented in various forms, including the embodiments described above, with various modifications and improvements based on the knowledge of those skilled in the art.

[0058] In the above embodiment, a composite vehicle image was generated based on the assumption that the door is open to an extent equivalent to the actual door opening, using the door opening sensor 2. However, the embodiment is not limited to this. The door opening sensor 2 may be a known courtesy switch that detects the opening and closing of a vehicle door. In this case, depending on whether the courtesy switch is ON or OFF, if the vehicle door is open, a composite vehicle image representing a door opened to a preset door opening may be generated based on this image, and an additional amount of door opening may be added.

[0059] In the above embodiment, a hinged door 21 and a sliding door 31 were given as examples of vehicle doors, but the vehicle is not limited to these examples. For example, vehicle doors may also be trunk or luggage compartment doors, hoods, or gullwing doors of sports vehicles. [Explanation of symbols]

[0060] 11...Image acquisition unit, 12...Composite image generation unit, 13...Display control unit, 20,30...Vehicle, 21...Hinged door, 22,32...Opening, 31...Sliding door, 40C,40F...Composite vehicle image, 100...Display device, SD1,SD2...Shift amount.

Claims

1. An image acquisition unit that acquires images captured by multiple cameras that capture images of the area around the vehicle, A composite image generation unit that combines multiple captured images to generate a composite vehicle image representing the vehicle as seen from a virtual viewpoint, The system includes a display control unit that displays the synthesized vehicle image on an in-vehicle display unit, The doors of the vehicle include hinged doors that open by rotating on a hinge fixed to the vehicle body, The composite image generation unit generates the composite vehicle image based on the orientation of the virtual viewpoint relative to the vehicle, such that when the virtual viewpoint is positioned in a direction not facing the opening of the hinged door, it represents the hinged door opened with a larger amount of rotation compared to when the virtual viewpoint is positioned in a direction facing the opening of the hinged door.

2. An image acquisition unit that acquires images captured by multiple cameras that capture images of the area around the vehicle, A composite image generation unit that combines multiple captured images to generate a composite vehicle image representing the vehicle as seen from a virtual viewpoint, The system includes a display control unit that displays the synthesized vehicle image on an in-vehicle display unit, The doors of the vehicle include sliding doors that move in the vehicle width direction with a predetermined shift amount relative to the vehicle body and move in the front-rear direction to open. The composite image generation unit generates the composite vehicle image based on the orientation of the virtual viewpoint relative to the vehicle, such that when the virtual viewpoint is positioned in a direction not facing the opening of the sliding door, it represents the sliding door that has moved in the vehicle width direction by a larger shift amount compared to when the virtual viewpoint is positioned in a direction facing the opening of the sliding door.

Images