Display device

Abstract

The invention relates to a display device. A display device (100) is provided with: an image acquisition unit (11) that acquires a captured image of a camera group (1) that captures an image of the surroundings of a vehicle; a composite image generation unit (12) that synthesizes the plurality of captured images and generates a composite vehicle image indicating the vehicle observed from the virtual viewpoint; and a display control unit (13) that displays the composite vehicle image on a vehicle-mounted HMI (3). On the basis of the direction of the virtual viewpoint with respect to the vehicle, the composite image generation unit (12) generates a composite vehicle image such that the door of the vehicle is opened larger when the virtual viewpoint is located in a prescribed first direction than when the virtual viewpoint is located in a prescribed second direction. The predetermined second direction is a direction in which the opening of the vehicle door is more easily observed from a virtual viewpoint than the first direction.

Description

Technical Field

[0001] The present disclosure relates to a display device. Background Art

[0002] In the past, an image processing device is known that generates a composite image representing a vehicle observed from a virtual viewpoint based on photographic images obtained by photographing the periphery of the vehicle by multiple cameras, depicts the opening and closing trajectory of the vehicle's doors in the image of the vehicle included in the composite image, outputs the composite image to a display, and displays the composite image (for example, Patent Document 1).

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2012-051391

[0006] A technology for displaying a synthetic vehicle image showing a vehicle viewed from a virtual viewpoint on a display unit mounted on a vehicle is studied. In such a synthetic vehicle image, even if the opening state of the vehicle doors is actually the same, the opening state of the vehicle doors viewed from the virtual viewpoint may appear different. As a result, depending on the direction of the virtual viewpoint relative to the vehicle, it may be difficult to recognize the opening state of the vehicle doors in the synthetic vehicle image displayed on the display unit. Summary of the invention

[0007] A display device of one embodiment of the present invention comprises: an image acquisition unit, which acquires images captured by multiple cameras that capture the periphery of a vehicle; a synthetic image generation unit, which synthesizes the multiple captured images to generate a synthetic vehicle image representing the vehicle observed from a virtual viewpoint; and a display control unit, which displays the synthetic vehicle image on a display unit mounted on a vehicle, wherein the synthetic image generation unit generates the synthetic vehicle image based on the direction of the virtual viewpoint relative to the vehicle in a manner that represents the vehicle's doors as being more open when the virtual viewpoint is located in a specified first direction than when the virtual viewpoint is located in a specified second direction, wherein the specified second direction is a direction in which the opening of the door is easier to observe from the virtual viewpoint than the first direction.

[0008] In a display device of one embodiment of the present invention, a plurality of captured images acquired using a plurality of cameras are synthesized to generate a synthesized vehicle image representing a vehicle observed from a virtual viewpoint. The generated synthesized vehicle image is displayed on a display unit mounted on a vehicle. The synthesized vehicle image is generated in a manner that represents a vehicle door as being more open when the virtual viewpoint is located in a specified first direction than when the virtual viewpoint is located in a specified second direction. Therefore, it is easy to visually confirm an open vehicle door in the synthesized vehicle image observed from a virtual viewpoint located in the first direction. In this way, according to a display device of one embodiment of the present invention, it is easy to identify the open state of a vehicle door in the synthesized vehicle image displayed on the display unit.

[0009] In one embodiment, the vehicle door may include a hinged door that is opened by rotating with a hinge fixed to the vehicle body as a fulcrum, the first direction is a direction not facing the opening of the hinged door, and the second direction is a direction facing the opening of the hinged door, and the synthetic image generation unit generates a synthetic vehicle image in a manner that represents the hinged door as being opened with a larger rotation amount when the virtual viewpoint is located in the first direction than when the virtual viewpoint is located in the second direction. In this case, it is easy to recognize the opening of the hinged door of the vehicle in the synthetic vehicle image displayed on the display unit.

[0010] In one embodiment, the door of the vehicle may include a sliding door that moves in a vehicle width direction with a predetermined offset relative to the vehicle body and in a front-rear direction to be opened, the first direction is a direction not facing the opening of the sliding door, and the second direction is a direction facing the opening of the sliding door, and the synthetic image generation unit generates a synthetic vehicle image in a manner that represents the sliding door as moving in the vehicle width direction with a larger offset when the virtual viewpoint is located in the first direction than when the virtual viewpoint is located in the second direction. In this case, it is easy to recognize the opening of the sliding door of the vehicle in the synthetic vehicle image displayed on the display unit.

[0011] Effects of the Invention

[0012] According to the display device of one aspect of the present disclosure, it is easy to recognize whether the door of the vehicle is open in the synthesized vehicle image displayed on the display unit. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is a block diagram showing a display device according to one embodiment.

[0014] Figure 2 (a) is an example of a synthetic vehicle image of a hinged door opened at a door opening corresponding to an actual opening state of a vehicle door when the virtual viewpoint is located in the second direction.

[0015] Figure 2 (b) is an example of a synthetic vehicle image of a hinged door opened at a door opening corresponding to an actual door opening state of the vehicle when the virtual viewpoint is located in the first direction.

[0016] Figure 2 (c) is an example of a synthetic vehicle image showing a hinged door that is opened wider when the virtual viewpoint is located in the first direction than when the virtual viewpoint is located in the second direction.

[0017] Figure 3 This is a diagram showing, by way of example, the trend of visibility of the opening of a vehicle door.

[0018] Figure 4 (a) is a diagram showing, in side view, an example of a sliding door opened at a door opening degree corresponding to an actual opening state of a vehicle door.

[0019] Figure 4 (b) is a plan view showing an example of a sliding door opened at a door opening degree corresponding to the actual opening state of a vehicle door.

[0020] Figure 4 (c) is a plan view showing an example of a sliding door opened to a greater degree than the door opening corresponding to the actual opening state of the vehicle door.

[0021] Figure 5 Yes means Figure 1 A flow chart of an example of the processing of an ECU.

[0022] Figure 6 Yes means Figure 1 A flowchart of another example of the processing of the ECU. DETAILED DESCRIPTION

[0023] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0024] Figure 1 FIG. 1 is a block diagram showing a display device according to one embodiment. Figure 1 As shown, the display device 100 is mounted on a vehicle such as a passenger vehicle. The display device 100 is a device for displaying a synthetic vehicle image (3D view) representing the vehicle observed from a virtual viewpoint on a display unit such as a display mounted on the vehicle. It should be noted that a solution in which part of the functions of the display device 100 are executed on a server that can communicate with the vehicle can also be adopted. The vehicle door includes a hinged door that is opened by rotating with a hinge fixed to the body of the vehicle as a fulcrum.

[0025] [Configuration of Display Device]

[0026] Below, refer to Figure 1 The structure of the display device 100 of this embodiment is described. Figure 1 As shown, the display device 100 includes an ECU (Electronic Control Unit) 10 .

[0027] ECU10 is an electronic control unit having a CPU [Central Processing Unit] and a storage unit. The storage unit is composed of, for example, ROM [Read Only Memory], RAM [Random Access Memory], EEPROM [Electrically Erasable Programmable Read-Only Memory], etc. In ECU10, various functions are realized by, for example, the CPU executing a program stored in the storage unit. ECU10 may also be composed of a plurality of electronic units.

[0028] The ECU 10 is connected to a camera group 1 , a door opening sensor 2 , and an HMI (Human Machine Interface) 3 .

[0029] The camera group 1 is a plurality of cameras for photographing the periphery of the vehicle. The camera group 1 has, for example, a front camera, a rear camera, and a pair of left and right side view cameras. The front camera is, for example, disposed on the front bumper of the vehicle to photograph the periphery in front of the vehicle. The rear camera is, for example, disposed on the rear bumper of the vehicle to photograph the periphery behind the vehicle. The side view camera is, for example, disposed near the door mirror of the vehicle to photograph the periphery on the side of the vehicle. The camera group 1 has, for example, a wide-angle field of view and can also photograph the vehicle body. The camera group 1 sends the photographed images related to the periphery of the vehicle to the ECU 10.

[0030] The door opening sensor 2 is a detection device for detecting the opening of the vehicle door. The door opening sensor 2 can also be a linear sensor for detecting the opening of the vehicle door. In the case where the door is a hinged door, the linear sensor can also detect the rotation amount of the hinged door or the angle of the door. In the case where the door is a sliding door, the linear sensor can also detect the offset of the sliding door, where the offset is the amount of movement in the vehicle width direction. The door opening sensor 2 sends information related to the opening or opening and closing of the door to the ECU 10.

[0031] HMI3 is an interface for inputting and outputting information between ECU10 and passengers. HMI3, for example, includes a display, a speaker, a microphone, etc., which are arranged in the vehicle cabin. HMI3 outputs images from the display and voices from the speaker in response to control signals from ECU10. The display can also function as a touch panel. The display can also be a central display, a display for navigation, or a HUD [Head Up Display]. HUD presents information to passengers by projecting images on the front windshield of the vehicle.

[0032] HMI3 receives an operation of selecting a virtual viewpoint from the passenger. The operation of selecting a virtual viewpoint may be, for example, a touch operation of an image corresponding to the virtual viewpoint displayed on a touch panel, or an input operation of a value of a direction (angle) relative to the vehicle corresponding to the virtual viewpoint. HMI3 displays the generated synthetic vehicle image to the passenger.

[0033] Next, the functional configuration of the ECU 10 will be described. The ECU 10 includes an image acquisition unit 11 , a synthetic image generation unit 12 , and a display control unit 13 .

[0034] The image acquisition unit 11 acquires images captured by a plurality of cameras that capture the periphery of the vehicle. The image acquisition unit 11 acquires images captured by the camera group 1 .

[0035] The synthetic image generation unit 12 synthesizes a plurality of captured images to generate a synthetic vehicle image representing the vehicle as viewed from a virtual viewpoint. The synthetic image generation unit 12 generates an image of the surroundings of the vehicle including the vehicle as viewed from an arbitrary virtual viewpoint as a synthetic vehicle image based on the captured images captured by the camera group 1. The synthetic image generation unit 12 combines the captured images captured by the camera group 1 by a known method to generate a synthetic vehicle image as viewed from a virtual viewpoint, excluding the opening of the vehicle door.

[0036] The synthetic image generation unit 12 generates a synthetic vehicle image based on the direction of the virtual viewpoint relative to the vehicle in such a manner that the door of the vehicle is shown to be more open when the virtual viewpoint is located in a predetermined first direction than when the virtual viewpoint is located in a predetermined second direction, wherein the predetermined second direction is a direction in which the opening of the door is easier to observe from the virtual viewpoint than the first direction. The first direction is a predetermined direction in which it is difficult to recognize the opening of the door of the vehicle compared to the second direction. The second direction is a predetermined direction in which the opening of the door of the vehicle is easier to observe from the virtual viewpoint than the first direction.

[0037] Figure 2(a) is an example of a synthetic vehicle image of a hinged door opened at a door opening corresponding to an actual opening state of a vehicle door when the virtual viewpoint is located in the second direction. Figure 2 (b) is an example of a synthetic vehicle image of a hinged door opened at a door opening corresponding to an actual door opening state of the vehicle when the virtual viewpoint is located in the first direction. Figure 2 (c) is an example of a synthetic vehicle image showing a hinged door that is opened wider when the virtual viewpoint is located in the first direction than when the virtual viewpoint is located in the second direction.

[0038] Figure 2 (a) shows a synthetic vehicle image 40A in a state where the hinged door 21 of the vehicle 20 is opened, from a viewpoint from the oblique rear left of the vehicle 20 . Figure 2 The viewpoint of (a) faces the opening 22 of the hinged door 21. Figure 2 The opening 22 of the hinged door 21 can be easily visually confirmed from the viewpoint (a) of FIG. Therefore, even if the synthesized vehicle image 40A to be displayed on the HMI 3 is generated according to the opening state of the door as it is and the synthesized vehicle image 40A is displayed on the HMI 3, the passengers of the vehicle 20 can easily recognize the opening state of the door of the vehicle 20 in the synthesized vehicle image 40A displayed on the HMI 3.

[0039] Figure 2 (b) shows the hinged door 21 of the vehicle 20 from the front side of the vehicle 20. Figure 2 A composite vehicle image 40B of the case where the vehicle is opened to the same degree as (a). Figure 2 The viewpoint of (b) does not face the opening 22 of the hinged door 21. Figure 2 The opening 22 of the hinged door 21 is difficult to visually confirm from the viewpoint (b) of FIG. Figure 2 (a) the same degree of opening, but in Figure 2 In (b), it may be visually confirmed that the door opening ratio of the hinged door 21 is Figure 2 The door opening of the hinged door 21 of (a) is small. Figure 2 When a synthetic vehicle image 40B to be displayed on HMI3 is generated and the synthetic vehicle image 40B is displayed on HMI3 with the same degree of door opening as (a), it may be difficult for passengers of vehicle 20 to recognize the door opening of vehicle 20 in the synthetic vehicle image 40B displayed on HMI3.

[0040] Therefore, the synthetic image generation unit 12 generates the synthetic vehicle image 40C in a manner that represents the hinged door 21 as being opened by a larger rotation amount when the virtual viewpoint is located in the first direction than when the virtual viewpoint is located in the second direction. Here, the first direction is a direction not facing the opening 22 of the hinged door 21, and the second direction is a direction facing the opening 22 of the hinged door 21.

[0041] Figure 2 (c) shows the following synthetic vehicle image 40C from the viewpoint of the front side of the vehicle 20: Although the hinged door 21 of the vehicle 20 is actually Figure 2 (a) the same degree of opening, but using a Figure 2 The door opening of the vehicle 20 is larger than the door opening of (a). Thus, it is easy to visually confirm that the hinged door of the vehicle 20 is opened. As for how much larger the door opening is than the actual door opening, the synthetic vehicle image 40C is generated, which can be determined by, for example, using the relationship between the ease of viewing the opening 22 of the door of the vehicle 20 and the direction set relative to the vehicle 20 in advance.

[0042] Figure 3 FIG. 2 is a diagram showing, by way of example, the trend of the ease of viewing of the opening of a vehicle door. Figure 2 and Figure 3 As shown, when the door of the vehicle 20 is a hinged door 21, the trend of the visibility of the opening 22 of the door of the vehicle 20 is, for example, Figure 3 This trend. Figure 3 In the example, for example, for a viewpoint on the left side of the vehicle 20, the front (front) of the vehicle 20 is set to 0 degrees and the back (rear) of the vehicle 20 is set to 180 degrees as the direction relative to the vehicle 20, and viewpoints are specified every 45 degrees. Figure 3 In the case of the vehicle 20, for the viewpoint located on the left side of the vehicle 20, the opening 22 of the door of the vehicle 20 becomes easiest to observe when the vehicle 20 is observed from the left oblique rear (in the direction of 135 degrees), and becomes difficult to observe in the order of 90 degrees and 45 degrees, and becomes most difficult to observe when the vehicle 20 is observed from the opposite side (in the direction of 0 degrees).

[0043] Although in Figure 3Although the illustration is omitted in the figure, a viewpoint located on the right side of the vehicle 20 may be assumed, wherein the back side (rear) of the vehicle 20 is set to 180 degrees and the front side (front) of the vehicle 20 is set to 360 degrees as the direction relative to the vehicle 20. The viewing ease of the opening 22 of the door of the vehicle 20 may be bilaterally symmetrical between the viewpoint located on the left side of the vehicle 20 and the viewpoint located on the right side of the vehicle 20. In this case, specifically, the opening 22 of the door of the vehicle 20 becomes most visible when the vehicle 20 is viewed from the left oblique rear (direction of 135 degrees), becomes temporarily difficult to view when the vehicle 20 is viewed from the back (direction of 180 degrees), and becomes most visible again when the vehicle 20 is viewed from the right oblique rear (direction of 225 degrees). From there, the opening 22 of the door of the vehicle 20 becomes difficult to see in the order of 270 degrees and 315 degrees, and becomes most difficult to see when the vehicle 20 is viewed from the opposite side (360 degrees).

[0044] exist Figure 3 In the figure, the additional door opening amount is specified every 45 degrees from 0 degrees to 180 degrees. The additional door opening amount is a correction amount of the door opening on the synthetic vehicle image that is pre-set according to the ease of observation (observation difficulty) of the opening 22 of the door of the vehicle 20. Regarding the additional door opening amount, the smaller the value at the viewpoint where the opening 22 of the door of the vehicle 20 is easy to observe, the larger the value at the viewpoint where the opening 22 of the door of the vehicle 20 is difficult to observe. Alternatively, the additional door opening amount may increase monotonically according to the direction relative to the vehicle 20 from the viewpoint where the opening 22 of the door of the vehicle 20 is easiest to observe to the viewpoint where the opening 22 of the door of the vehicle 20 is most difficult to observe. In the calculation of the additional door opening amount, as long as a virtual viewpoint is determined, the value of the additional door opening amount can be determined.

[0045] It should be noted that, regarding the first direction and the second direction, it is not necessary to predetermine which angle range the direction relative to the vehicle is in. The first direction and the second direction only need to be directions for relatively comparing the ease of observing the opening of the door of the vehicle at two virtual viewpoints. For example, when the first direction is a direction of 90 degrees, the second direction may also be a direction of 135 degrees, and when the first direction is a direction of 45 degrees, the second direction may also be a direction of 90 degrees.

[0046] The vehicle door may include a slide door 31 that moves in the vehicle width direction by a predetermined offset amount relative to the body of the vehicle 30 and moves in the front-rear direction to be opened. Figure 4 (a) is a diagram showing, in side view, an example of a sliding door opened at a door opening degree corresponding to an actual opening state of a vehicle door. Figure 4 (b) is a plan view showing an example of a sliding door opened at a door opening degree corresponding to the actual opening state of a vehicle door. Figure 4 (c) is a plan view showing an example of a sliding door opened to a greater degree than the door opening corresponding to the actual opening state of the vehicle door.

[0047] Figure 4 (a) shows a synthesized vehicle image 40D in which the sliding door 31 of the vehicle 30 is open, as viewed from the left side of the vehicle 30 . Figure 4 The viewpoint of (a) faces the opening 32 of the sliding door 31. Figure 4 The opening 32 of the sliding door 31 can be easily visually confirmed from the viewpoint (a) of FIG. Therefore, even if the synthetic vehicle image 40D to be displayed on the HMI 3 is generated according to the opening state of the door as it is and the synthetic vehicle image 40D is displayed on the HMI 3, the passengers of the vehicle 30 can easily recognize the opening state of the door of the vehicle 30 in the synthetic vehicle image 40D displayed on the HMI 3.

[0048] For ease of explanation, Figure 4 (b) shows the sliding door 31 of the vehicle 30 from a viewpoint above the vehicle 30. Figure 4 A composite vehicle image 40E showing the vehicle being opened to the same degree as (a). Figure 4 The viewpoint of (b) does not face the opening 32 of the sliding door 31. Figure 4 The opening 32 of the sliding door 31 is difficult to visually confirm from the viewpoint (b) of FIG. Figure 4 (a) the same degree of opening, but in Figure 4 In (b), Figure 4 Specifically, the synthetic vehicle image 40E shows that the sliding door 31 has moved relative to the body of the vehicle 30 in the vehicle width direction by a predetermined offset amount SD1. Figure 4 The viewpoint of (b) does not face the opening 32 of the sliding door 31, so Figure 4 Compared with (a), it is difficult to visually confirm that the sliding door 31 is opened. Figure 4 When a synthetic vehicle image 40E to be displayed on HMI3 is generated and the synthetic vehicle image 40E is displayed on HMI3 with the same degree of door opening as (a), it may be difficult for passengers of the vehicle 30 to recognize the opening state of the door of the vehicle 30 in the synthetic vehicle image 40E displayed on HMI3.

[0049] Therefore, the synthetic image generation unit 12 generates the synthetic vehicle image 40F in such a manner that the sliding door 31 is represented as being shifted in the vehicle width direction by a larger offset amount when the virtual viewpoint is located in the first direction than when the virtual viewpoint is located in the second direction. Here, the first direction is a direction not facing the opening 32 of the sliding door 31, and the second direction is a direction facing the opening 32 of the sliding door 31.

[0050] For ease of explanation, Figure 4 (c) shows the following synthetic vehicle image 40F from a viewpoint above the vehicle 30: Although the sliding door 31 of the vehicle 30 is actually Figure 4 (a) the same degree of opening, but using a Figure 4 The synthetic vehicle image 40F shows a state where the sliding door 31 has moved by a predetermined offset amount SD2 in the vehicle width direction relative to the body of the vehicle 30. In this way, by adopting an offset amount SD2 that is larger than the offset amount SD1 when the virtual viewpoint is located in the second direction, it is possible to easily visually confirm that the sliding door 31 of the vehicle 30 is opened.

[0051] The display control unit 13 displays the synthesized vehicle image on the display of the in-vehicle HMI 3. The display control unit 13 displays the generated synthesized vehicle image on the display of the HMI 3 in the form of a 3D view.

[0052] [Operation of the display device]

[0053] Next, the operation of the display device 100 will be described with reference to the drawings. Figure 5 Yes means Figure 1 A flow chart of an example of the processing of an ECU. Figure 5 The processing of the ECU is executed, for example, when a request is made to draw a composite vehicle image when the vehicle is stopped with its doors open.

[0054] like Figure 5 As shown, as step S11, the ECU 10 of the display device 100 acquires images captured by multiple cameras through the image acquisition unit 11. The image acquisition unit 11 acquires images captured by the camera group 1 that captures the surroundings of the vehicle. In step S11, the ECU 10 acquires the door opening through the synthetic image generation unit 12. The synthetic image generation unit 12 acquires the door opening based on the detection result of the door opening sensor 2.

[0055] In step S12, the ECU 10 acquires a virtual viewpoint through the synthetic image generation unit 12. The synthetic image generation unit 12 acquires a virtual viewpoint in response to, for example, an operation of selecting a virtual viewpoint performed on the HMI 3 by a passenger of the vehicle.

[0056] In step S13, the ECU 10 calculates the door opening for the synthetic image corresponding to the actual opening of the vehicle door through the synthetic image generating unit 12. The synthetic image generating unit 12 calculates the door opening for the synthetic image in such a manner that the door opening for the synthetic image is equal to the acquired door opening.

[0057] In step S14, the ECU 10 calculates the additional amount of door opening for the synthetic image based on the direction of the virtual viewpoint relative to the vehicle through the synthetic image generating unit 12. The synthetic image generating unit 12 obtains the preset additional amount of door opening based on the direction of the virtual viewpoint relative to the vehicle, and calculates the additional amount of door opening for the synthetic image. Regarding the additional amount of door opening, the smaller the value at the viewpoint where the opening of the vehicle door is easy to observe, the larger the value at the viewpoint where the opening of the vehicle door is difficult to observe.

[0058] In step S15, the ECU 10 generates a synthetic vehicle image through the synthetic image generation unit 12 in a manner that represents the door as being open at the door opening for the synthetic image. The synthetic image generation unit 12, for example, adds a door opening additional amount to the door opening for the synthetic image at an angle equal to the door opening, and generates a synthetic vehicle image in a manner that represents the door as being open at the door opening after the addition. At this time, regarding the door opening additional amount, the smaller the value at the viewpoint where the opening of the vehicle door is easy to observe, the larger the value at the viewpoint where the opening of the vehicle door is difficult to observe. Therefore, the synthetic vehicle image is generated in a manner that represents the door as being more open when the virtual viewpoint is in the first direction than when the virtual viewpoint is in the second direction.

[0059] In step S16, the ECU 10 displays the synthesized vehicle image on the in-vehicle display unit through the display control unit 13. The display control unit 13 displays the synthesized vehicle image on the display of the HMI 3, for example. Then, the ECU 10 ends this time. Figure 5 processing.

[0060] It should be noted that the first direction and the second direction may also be predetermined as the angle ranges of the directions relative to the vehicle. For example, for a viewpoint located on the left side of the vehicle, the first direction may be determined as a direction that is greater than 0 degrees and less than 45 degrees and does not face the opening of the hinged door, and the second direction may be determined as a direction that is greater than 45 degrees and less than 180 degrees and faces the opening of the hinged door. In this case, the ECU 10 may execute Figure 6 Such treatment.

[0061] Figure 6 Yes means Figure 1 Flowchart of another example of the processing of the ECU. Figure 5 Likewise, Figure 6 The processing of the ECU is executed when a request is made to draw a composite vehicle image when the vehicle is stopped with its doors open.

[0062] like Figure 6 As shown, as step S21, similarly to step S11, ECU 10 acquires images captured by multiple cameras through image acquisition unit 11. ECU 10 acquires the door opening through synthetic image generation unit 12. In step S22, similarly to step S12, ECU 10 acquires a virtual viewpoint through synthetic image generation unit 12.

[0063] In step S23, the ECU 10 determines whether the virtual viewpoint is located in the first direction through the synthetic image generation unit 12. When the synthetic image generation unit 12 determines that the virtual viewpoint is located in the first direction (S23: Yes), the ECU 10 transfers to step S26. When the synthetic image generation unit 12 determines that the virtual viewpoint is not located in the first direction (S23: No), the ECU 10 transfers to step S24.

[0064] In step S24, the ECU 10 determines whether the virtual viewpoint is located in the second direction through the synthetic image generation unit 12. If the synthetic image generation unit 12 determines that the virtual viewpoint is located in the second direction (S24: Yes), the ECU 10 transfers to step S25. If the synthetic image generation unit 12 determines that the virtual viewpoint is not located in the second direction (S24: No), the ECU 10 ends this time. Figure 6 processing.

[0065] As step S25, the ECU 10 generates a synthetic vehicle image by the synthetic image generating unit 12 in such a manner that the door is shown as being opened at a door opening corresponding to the actual door opening. The synthetic image generating unit 12 calculates the door opening for the synthetic image at an angle equal to the door opening, and generates the synthetic vehicle image in such a manner that the door is shown as being opened at a door opening equal to the door opening without adding the door opening additional amount.

[0066] On the other hand, as step S26, the ECU 10 generates a synthetic vehicle image by the synthetic image generating unit 12 in such a manner that the door is shown as being opened more than when the virtual viewpoint is located in the second direction. For example, the synthetic image generating unit 12 adds a door opening additional amount to the door opening for the synthetic image at an angle equal to the door opening, and generates the synthetic vehicle image in such a manner that the door is shown as being opened at an angle greater than the door opening equal to the door opening.

[0067] In step S27, the ECU 10 displays the synthesized vehicle image on the in-vehicle display unit through the display control unit 13. The display control unit 13 displays the synthesized vehicle image on the display of the HMI 3, for example. Then, the ECU 10 ends this Figure 6 processing.

[0068] According to the display device 100 described above, a plurality of captured images acquired using the camera group 1 are synthesized by the synthetic image generation unit 12 to generate synthetic vehicle images 40C and 40F representing the vehicle observed from a virtual viewpoint. The generated synthetic vehicle images 40C and 40F are displayed on the vehicle-mounted HMI 3. The synthetic vehicle images 40C and 40F are generated in a manner that represents the door as being opened more when the virtual viewpoint is located in a prescribed first direction than when the virtual viewpoint is located in a prescribed second direction. Therefore, it is easy to visually confirm the open door in the synthetic vehicle images 40C and 40F observed from the virtual viewpoint located in the first direction. In this way, according to the display device 100, it is easy to recognize the open state of the vehicle door in the synthetic vehicle images 40C and 40F displayed on the HMI 3.

[0069] The door of the vehicle includes a hinged door 21, which is opened by rotating around a hinge fixed to the body of the vehicle 20 as a fulcrum. The first direction is a direction not facing the opening 22 of the hinged door 21, and the second direction is a direction facing the opening 22 of the hinged door 21. The synthetic image generation unit 12 generates a synthetic vehicle image 40C in a manner that represents the hinged door 21 as being opened by a larger rotation amount when the virtual viewpoint is located in the first direction than when the virtual viewpoint is located in the second direction. In this case, the opening state of the hinged door 21 of the vehicle 20 is easily recognized in the synthetic vehicle image 40C displayed on the HMI 3.

[0070] The door of the vehicle includes a sliding door 31, which moves in the vehicle width direction by a predetermined offset amount SD1 relative to the body of the vehicle 30 and moves in the front-rear direction to open. The first direction is a direction not facing the opening portion 32 of the sliding door 31, and the second direction is a direction facing the opening portion 32 of the sliding door 31. The synthetic image generation unit 12 generates a synthetic vehicle image 40F in a manner that represents the sliding door 31 as moving in the vehicle width direction by an offset amount SD2 larger than the offset amount when the virtual viewpoint is located in the second direction when the virtual viewpoint is located in the first direction. In this case, the opening state of the sliding door 31 of the vehicle 30 can be easily recognized in the synthetic vehicle image 40F displayed on the HMI 3.

[0071] Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the above embodiments. The present disclosure can be implemented in various forms represented by the above embodiments and various changes and improvements based on the knowledge of those skilled in the art.

[0072] In the above embodiment, a synthetic vehicle image based on the door opening being opened to the same degree as the actual door opening obtained by using the door opening sensor 2 is generated, but the present invention is not limited thereto. The door opening sensor 2 may also be a well-known door control switch that detects the opening and closing of the vehicle door. In this case, it is also possible that, depending on whether the door control switch is turned on or off, when the vehicle door is opened, a synthetic vehicle image showing the door opened to a preset door opening degree is generated as a basis, and the door opening additional amount is added.

[0073] In the above embodiment, the hinged door 21 and the sliding door 31 are exemplified as the vehicle door, but the present invention is not limited to these examples. For example, the vehicle door may be a trunk or luggage compartment door, a hood, or a gull-wing door of a sports vehicle.

Claims

1. A display device comprising: An image acquisition unit that acquires images captured by a plurality of cameras that capture the periphery of the vehicle; a synthetic image generating unit that synthesizes the plurality of captured images to generate a synthetic vehicle image representing the vehicle viewed from a virtual viewpoint; and a display control unit, which displays the synthesized vehicle image on a display unit mounted on the vehicle, The synthetic image generation unit generates the synthetic vehicle image based on the direction of the virtual viewpoint relative to the vehicle in a manner that represents the door of the vehicle as being more open when the virtual viewpoint is located in a prescribed first direction than when the virtual viewpoint is located in a prescribed second direction, wherein The predetermined second direction is a direction in which the opening of the door is more easily visible from the virtual viewpoint than the first direction.

2. The display device according to claim 1, wherein: The vehicle door comprises a hinged door, which is opened by rotating with a hinge fixed to the vehicle body as a fulcrum. The first direction is a direction not facing the opening of the hinged door, The second direction is a direction facing the opening of the hinged door, The synthetic image generation unit generates the synthetic vehicle image in a manner showing the hinged door as opened by a larger rotation amount when the virtual viewpoint is located in the first direction than when the virtual viewpoint is located in the second direction.

3. The display device according to claim 1 or 2, wherein: The vehicle door includes a sliding door that moves in a vehicle width direction by a predetermined offset amount relative to a body of the vehicle and moves in a front-rear direction to open. The first direction is a direction not facing the opening of the sliding door, The second direction is a direction facing the opening of the sliding door, The synthetic image generation unit generates the synthetic vehicle image in a manner that represents the slide door as shifted in the vehicle width direction by the offset amount greater than the offset amount when the virtual viewpoint is located in the second direction when the virtual viewpoint is located in the first direction.

Images