Display device

Abstract

To provide a display device that makes it easy to recognize an opening degree of a door of a vehicle in a composite vehicle image displayed on a display unit.SOLUTION: A display device 100 includes: an image acquisition unit 11 that acquires a captured image of a camera group 1 that captures an image of surroundings of a vehicle; a composite image generation unit 12 that synthesizes a plurality of captured images to generate a composite vehicle image representing the vehicle viewed from an imaginary viewpoint; and a display control unit 13 that displays the composite vehicle image on an on-vehicle HMI3. Based on a direction of the imaginary viewpoint with respect to the vehicle, the composite image generation unit 12 generates a composite vehicle image so as to represent a door that is opened wider when the imaginary viewpoint is located in a predetermined first direction than when the imaginary viewpoint is located in a predetermined second direction in which an opening of the vehicle door is more easily seen from the imaginary viewpoint than in the first direction.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a display device. [Background technology]

[0002] Conventionally, there is known an image processing device that generates a composite image showing a vehicle viewed from a virtual viewpoint based on images captured by multiple cameras around the vehicle, draws the trajectory of the vehicle's door opening and closing on the image of the vehicle contained in the composite image, and outputs the composite image to a display for display (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2012-051391 A Summary of the Invention [Problem to be solved by the invention]

[0004] It is being considered to display a synthetic vehicle image showing a vehicle seen from a virtual viewpoint on a display unit mounted on a vehicle. In such synthetic vehicle images, even if the degree of opening of the vehicle doors is the same in reality, the degree of opening of the vehicle doors seen from the virtual viewpoint may appear different. As a result, depending on the direction of the virtual viewpoint relative to the vehicle, it may be difficult to recognize the degree of opening of the vehicle doors in the synthetic vehicle image displayed on the display unit. [Means for solving the problem]

[0005] A display device according to one embodiment of the present disclosure includes an image acquisition unit that acquires captured images from multiple cameras that capture images of the surroundings of the vehicle, a composite image generation unit that combines the multiple captured images to generate a composite vehicle image representing the vehicle as seen from a virtual viewpoint, and a display control unit that displays the composite vehicle image on an in-vehicle display unit, and the composite image generation unit generates the composite vehicle image based on the direction of the virtual viewpoint relative to the vehicle, so that when the virtual viewpoint is located in a predetermined first direction, the composite vehicle image represents the door as being wider open than when the virtual viewpoint is located in a predetermined second direction in which the opening of the vehicle door is easier to see from the virtual viewpoint than in the first direction.

[0006] In a display device according to an embodiment of the present disclosure, a plurality of captured images acquired using a plurality of cameras are composited to generate a composite vehicle image showing a vehicle as seen from a virtual viewpoint. The generated composite vehicle image is displayed on an in-vehicle display unit. When the virtual viewpoint is located in a predetermined first direction, a composite vehicle image is generated so as to show a door that is more widely open than when the virtual viewpoint is located in a predetermined second direction. Thus, an open door is more easily visible in the composite vehicle image as seen from the virtual viewpoint located in the first direction. Thus, according to the display device according to an embodiment of the present disclosure, it is easier to recognize the degree to which the vehicle door is open in the composite vehicle image displayed on the display unit.

[0007] In one embodiment, the vehicle door includes a hinge door that opens by rotating around a hinge fixed to the vehicle body, the first direction is a direction not facing the opening of the hinge door, and the second direction is a direction facing the opening of the hinge door, and the synthetic image generating unit may generate a synthetic vehicle image when the virtual viewpoint is located in the first direction so as to represent the hinge door opened with a larger amount of rotation than when the virtual viewpoint is located in the second direction. In this case, it is easy to recognize the degree to which the hinge door of the vehicle is opened in the synthetic vehicle image displayed on the display unit.

[0008] In one embodiment, the vehicle door includes a sliding door that moves a predetermined shift amount in the vehicle width direction relative to the vehicle body and moves in the front-rear direction to open, the first direction is a direction that does not face the opening of the sliding door, and the second direction is a direction that faces the opening of the sliding door, and the synthetic image generating unit may generate a synthetic vehicle image when the virtual viewpoint is located in the first direction so as to represent the sliding door that has moved in the vehicle width direction by a larger shift amount than when the virtual viewpoint is located in the second direction. In this case, it is easy to recognize the degree to which the vehicle's sliding door is open in the synthetic vehicle image displayed on the display unit. Effect of the Invention

[0009] According to the display device according to one aspect of the present disclosure, it is easy to recognize the degree to which the vehicle door is opened in the composite vehicle image displayed on the display unit. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a display device according to an embodiment; [Diagram 2] (a) is an example of a synthetic vehicle image of a hinged door that opens to a degree that corresponds to the degree to which a door of an actual vehicle is opened when the virtual viewpoint is located in the second direction. (b) is an example of a synthetic vehicle image of a hinged door that opens to a degree that corresponds to the degree to which a door of an actual vehicle is opened when the virtual viewpoint is located in the first direction. (c) is an example of a synthetic vehicle image showing a hinged door that opens more widely when the virtual viewpoint is located in the first direction than when the virtual viewpoint is located in the second direction. [Diagram 3] 1 is a diagram illustrating an example of a tendency of visibility of a vehicle door opening. [Figure 4] 1A is a diagram illustrating a side view of a sliding door that opens at a door opening angle corresponding to the opening degree of an actual vehicle door, FIG. 1B is a diagram illustrating a plan view of a sliding door that opens at a door opening angle corresponding to the opening degree of an actual vehicle door, and FIG. 1C is a diagram illustrating a plan view of a sliding door that opens widely compared to the door opening angle corresponding to the opening degree of an actual vehicle door. [Diagram 5]2 is a flowchart showing an example of a process of an ECU in FIG. 1; [Figure 6] 10 is a flowchart showing another example of the process of the ECU in FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0012] FIG. 1 is a block diagram showing a display device according to an embodiment. As shown in FIG. 1, the display device 100 is mounted on a vehicle such as a passenger car. The display device 100 is a device for displaying a synthetic vehicle image (3D view) showing the vehicle as seen from a virtual viewpoint on a display unit such as an in-vehicle display. Note that some functions of the display device 100 may be executed on a server capable of communicating with the vehicle. The vehicle door includes a hinge door that opens by rotating around a hinge fixed to the vehicle body as a fulcrum.

[0013] [Display device configuration] Hereinafter, a configuration of a display device 100 according to this embodiment will be described with reference to Fig. 1. As shown in Fig. 1, the display device 100 includes an ECU (Electronic Control Unit) 10.

[0014] The ECU 10 is an electronic control unit having a central processing unit (CPU) and a storage unit. The storage unit is composed of, for example, a read-only memory (ROM), a random access memory (RAM), and an electrically erasable programmable read-only memory (EEPROM). The ECU 10 realizes various functions by, for example, executing a program stored in the storage unit by the CPU. The ECU 10 may be composed of multiple electronic units.

[0015] The ECU 10 is connected to a camera group 1, a door opening sensor 2, and an HMI [Human Machine Interface] 3.

[0016] The camera group 1 is a plurality of cameras that capture images of the periphery of the vehicle. The camera group 1 includes, for example, a front camera, a rear camera, and a pair of left and right side view cameras. The front camera is provided, for example, on the front bumper of the vehicle, and captures images of the periphery in front of the vehicle. The rear camera is provided, for example, on the rear bumper of the vehicle, and captures images of the periphery behind the vehicle. The side view camera is provided, for example, near the door mirror of the vehicle, and captures images of the periphery on the sides of the vehicle. The camera group 1 has, for example, a wide angle of view, and can also capture images of the vehicle body. The camera group 1 transmits captured images of the periphery of the vehicle to the ECU 10.

[0017] The door opening sensor 2 is a detection device that detects the opening degree of a vehicle door. The door opening sensor 2 may be a linear sensor that detects the opening degree of the vehicle door. If the door is a hinged door, the linear sensor may detect the rotation amount of the hinged door or the angle of the door. If the door is a sliding door, the linear sensor may detect the shift amount, which is the amount of movement of the sliding door in the vehicle width direction. The door opening sensor 2 transmits information regarding the opening degree or opening / closing of the door to the ECU 10.

[0018] The HMI 3 is an interface for inputting and outputting information between the ECU 10 and the occupant. The HMI 3 includes, for example, a display, a speaker, a microphone, and the like provided in the vehicle cabin. The HMI 3 outputs images from the display and sounds from the speaker in response to a control signal from the ECU 10. The display may function as a touch panel. The display may be a center display, a display for navigation, or a HUD [Head Up Display]. The HUD presents information to the occupant by projecting an image onto the windshield of the vehicle.

[0019] The HMI 3 accepts an operation by the occupant to select a virtual viewpoint. The operation to select a virtual viewpoint may be, for example, a touch operation on an image corresponding to the virtual viewpoint displayed on a touch panel, or an input operation of a direction (angle) value with respect to the vehicle corresponding to the virtual viewpoint. The HMI 3 displays the generated composite vehicle image to the occupant.

[0020] Next, a description will be given of the functional configuration of the ECU 10. The ECU 10 includes an image acquisition unit 11, a composite image generation unit 12, and a display control unit 13.

[0021] The image acquisition unit 11 acquires images captured by a plurality of cameras capturing images of the periphery of the vehicle. The image acquisition unit 11 acquires the images captured by the camera group 1.

[0022] The composite image generating unit 12 generates a composite vehicle image that represents the vehicle as viewed from a virtual viewpoint by combining a plurality of captured images. The composite image generating unit 12 generates a composite vehicle image of the vehicle including its surroundings as viewed from a virtual viewpoint, which is an arbitrary virtual viewpoint, based on the captured images captured by the camera group 1. The composite image generating unit 12 generates a composite vehicle image as viewed from a virtual viewpoint by combining the captured images captured by the camera group 1 using a known method, except for the degree of door opening.

[0023] The synthetic image generating unit 12 generates a synthetic vehicle image based on the direction of the virtual viewpoint relative to the vehicle, so that when the virtual viewpoint is located in a predetermined first direction, the door is shown to be more widely open than when the virtual viewpoint is located in a predetermined second direction in which the opening of the vehicle door is more easily seen from the virtual viewpoint than in the first direction. The first direction is a predetermined direction in which it is more difficult to recognize the degree to which the vehicle door is open than in the second direction. The second direction is a predetermined direction in which the opening of the vehicle door is more easily seen from the virtual viewpoint than in the first direction.

[0024] Fig. 2(a) is an example of a synthetic vehicle image of a hinged door that opens at a door opening angle corresponding to the opening degree of a real vehicle door when the virtual viewpoint is located in the second direction. Fig. 2(b) is an example of a synthetic vehicle image of a hinged door that opens at a door opening angle corresponding to the opening degree of a real vehicle door when the virtual viewpoint is located in the first direction. Fig. 2(c) is an example of a synthetic vehicle image showing a hinged door that opens more widely when the virtual viewpoint is located in the first direction than when the virtual viewpoint is located in the second direction.

[0025] 2(a) shows a composite vehicle image 40A of the vehicle 20 with the hinged door 21 open, as viewed from the left rear of the vehicle 20. The viewpoint in FIG. 2(a) faces the opening 22 of the hinged door 21. From the viewpoint in FIG. 2(a), the opening 22 of the hinged door 21 can be easily seen. Therefore, even if a composite vehicle image 40A to be displayed on the HMI 3 is generated with the door open in this state and the composite vehicle image 40A is displayed on the HMI 3, the occupants of the vehicle 20 can easily recognize the degree to which the door of the vehicle 20 is open in the composite vehicle image 40A displayed on the HMI 3.

[0026] FIG. 2(b) shows a composite vehicle image 40B in which the hinge door 21 of the vehicle 20 is open to the same extent as in FIG. 2(a), from a viewpoint from the front side of the vehicle 20. The viewpoint in FIG. 2(b) does not face the opening 22 of the hinge door 21. It is difficult to visually recognize the opening 22 of the hinge door 21 from the viewpoint in FIG. 2(b). Due to this influence, even though the hinge door 21 of the vehicle 20 is open to the same extent as in FIG. 2(a), the door opening degree of the hinge door 21 may be visually recognized as being smaller in FIG. 2(b) than in FIG. 2(a). In this way, when the composite vehicle image 40B to be displayed on the HMI 3 with the door open to the same extent as in FIG. 2(a) is generated and the composite vehicle image 40B is displayed on the HMI 3, the occupant of the vehicle 20 may have difficulty recognizing the degree to which the door of the vehicle 20 is opened in the composite vehicle image 40B displayed on the HMI 3.

[0027] Therefore, when the virtual viewpoint is located in the first direction, the composite image generating unit 12 generates a composite vehicle image 40C so as to show the hinge door 21 opened with a larger rotation amount than when the virtual viewpoint is located in the second direction. Here, the first direction is a direction not facing the opening 22 of the hinge door 21, and the second direction is a direction facing the opening 22 of the hinge door 21.

[0028] 2(c) shows a composite vehicle image 40C from a viewpoint from the front side of the vehicle 20, in which the hinged door 21 of the vehicle 20 is actually opened to the same extent as in FIG. 2(a), but the door is opened to a greater extent than in FIG. 2(a). This allows the viewer to easily recognize that the hinged door of the vehicle 20 is open. The degree of opening relative to the actual door opening at which the composite vehicle image 40C is generated can be determined, for example, using a preset relationship between the visibility of the door opening 22 of the vehicle 20 and the direction relative to the vehicle 20.

[0029] FIG. 3 is a diagram illustrating the tendency of visibility of the opening of a door of a vehicle. As shown in FIG. 2 and FIG. 3, the tendency of visibility of the opening 22 of the door of the vehicle 20 when the door of the vehicle 20 is a hinged door 21 is, for example, as shown in FIG. 3. In the example of FIG. 3, for a viewpoint located on the left side of the vehicle 20, the front of the vehicle 20 is 0 degrees, the rear of the vehicle 20 is 180 degrees, and viewpoints are defined every 45 degrees. In the case of the vehicle 20 of FIG. 3, the opening 22 of the door of the vehicle 20 is most visible from a viewpoint located on the left side of the vehicle 20 when the vehicle 20 is viewed from the left rear (direction of 135 degrees), becomes less visible at 90 degrees and 45 degrees, and is least visible when the vehicle 20 is viewed from the front (direction of 0 degrees).

[0030] Although not shown in FIG. 3, a viewpoint located on the right side of the vehicle 20 may be assumed, with the rear of the vehicle 20 being 180 degrees and the front of the vehicle 20 being 360 degrees, as the direction relative to the vehicle 20. The visibility of the door opening 22 of the vehicle 20 may be symmetrical between a viewpoint located on the left side of the vehicle 20 and a viewpoint located on the right side of the vehicle 20. In this case, specifically, the door opening 22 of the vehicle 20 is most visible from a viewpoint (135 degrees) when the vehicle 20 is viewed diagonally from the left rear, becomes difficult to see at a viewpoint when the vehicle 20 is viewed from the rear (180 degrees), and becomes most visible again at a viewpoint (225 degrees) when the vehicle 20 is viewed diagonally from the right rear. From there, it becomes difficult to see at 270 degrees and 315 degrees, and becomes least visible when the vehicle 20 is viewed directly in front (360 degrees).

[0031] In FIG. 3, the additional door opening amount is specified in increments of 45 degrees from 0 degrees to 180 degrees. The additional door opening amount is a correction amount of the door opening on the synthetic vehicle image that is set in advance according to the visibility (difficulty of visibility) of the door opening 22 of the vehicle 20. The additional door opening amount is smaller at a viewpoint where the door opening 22 of the vehicle 20 is easier to see, and is larger at a viewpoint where the door opening 22 of the vehicle 20 is harder to see. The additional door opening amount may monotonically increase according to the direction with respect to the vehicle 20, from the viewpoint where the door opening 22 of the vehicle 20 is easiest to see to the viewpoint where it is hardest to see. In calculating the additional door opening amount, the value of the additional door opening amount may be determined once one virtual viewpoint is determined.

[0032] The first and second directions do not need to be predetermined in terms of the angle range relative to the vehicle. The first and second directions may be directions for relatively comparing the visibility of the vehicle door opening from two virtual viewpoints. For example, when the first direction is a 90-degree direction, the second direction may be a 135-degree direction, and when the first direction is a 45-degree direction, the second direction may be a 90-degree direction.

[0033] The vehicle door may include a sliding door 31 that moves in a vehicle width direction by a predetermined shift amount relative to the vehicle body of the vehicle 30 and moves in a front-rear direction to open. Fig. 4(a) is a diagram illustrating an example of a sliding door in a side view that opens at a door opening degree corresponding to the opening degree of the actual vehicle door. Fig. 4(b) is a diagram illustrating an example of a sliding door in a plan view that opens at a door opening degree corresponding to the opening degree of the actual vehicle door. Fig. 4(c) is a diagram illustrating an example of a sliding door in a plan view that opens widely compared to the door opening degree corresponding to the opening degree of the actual vehicle door.

[0034] Fig. 4(a) shows a composite vehicle image 40D of the vehicle 30 with the sliding door 31 open, as viewed from the left side of the vehicle 30. The viewpoint in Fig. 4(a) faces the opening 32 of the sliding door 31. From the viewpoint in Fig. 4(a), the opening 32 of the sliding door 31 can be easily seen. Therefore, even if a composite vehicle image 40D to be displayed on the HMI 3 is generated with the door open in this state and the composite vehicle image 40D is displayed on the HMI 3, the occupants of the vehicle 30 can easily recognize the degree to which the door of the vehicle 30 is opened in the composite vehicle image 40D displayed on the HMI 3.

[0035] FIG. 4(b) shows a composite vehicle image 40E in which the sliding door 31 of the vehicle 30 is open to the same extent as in FIG. 4(a), from a viewpoint above the vehicle 30, for convenience of explanation. The viewpoint in FIG. 4(b) does not face the opening 32 of the sliding door 31. It is difficult to visually recognize the opening 32 of the sliding door 31 from the viewpoint in FIG. 4(b). Due to this influence, even though the sliding door 31 of the vehicle 30 is open to the same extent as in FIG. 4(a), it is difficult to visually recognize that the sliding door 31 is open in FIG. 4(b) compared to FIG. 4(a). Specifically, the composite vehicle image 40E shows a state in which the sliding door 31 has moved by a predetermined shift amount SD1 in the vehicle width direction relative to the vehicle body of the vehicle 30, but since the viewpoint in FIG. 4(b) does not face the opening 32 of the sliding door 31, it is difficult to visually recognize that the sliding door 31 is open compared to FIG. 4(a). In this way, if a composite vehicle image 40E is generated and displayed on HMI 3 with the doors open to the same extent as in Figure 4(a), it may be difficult for occupants of the vehicle 30 to recognize the degree to which the doors of the vehicle 30 are open in the composite vehicle image 40E displayed on the HMI 3.

[0036] Therefore, when the virtual viewpoint is located in the first direction, the composite image generating unit 12 generates a composite vehicle image 40F so as to show the sliding door 31 moved in the vehicle width direction by a larger shift amount than when the virtual viewpoint is located in the second direction. Here, the first direction is a direction not facing the opening 32 of the sliding door 31, and the second direction is a direction facing the opening 32 of the sliding door 31.

[0037] 4(c) shows a composite vehicle image 40F in which the sliding door 31 of the vehicle 30 is actually opened to the same extent as in FIG. 4(a), but the door is opened to a greater extent than in FIG. 4(a) for convenience of explanation, from a viewpoint above the vehicle 30. The composite vehicle image 40F shows a state in which the sliding door 31 has moved in the vehicle width direction by a predetermined shift amount SD2 relative to the body of the vehicle 30. In this way, by setting the shift amount SD2 to be larger than the shift amount SD1 when the virtual viewpoint is located in the second direction, it is possible to easily visually confirm that the sliding door 31 of the vehicle 30 is open.

[0038] The display control unit 13 displays the composite vehicle image on the display of the in-vehicle HMI 3. The display control unit 13 displays the generated composite vehicle image on the display of the HMI 3 as a 3D view.

[0039] [Display device operation] Next, the operation of the display device 100 will be described with reference to the drawings. Fig. 5 is a flowchart showing an example of the processing of the ECU in Fig. 1. The processing of the ECU in Fig. 5 is executed, for example, when a request is made to draw a composite vehicle image while the vehicle is stopped with the door open.

[0040] 5, in step S11, the ECU 10 of the display device 100 causes the image acquisition unit 11 to acquire images captured by a plurality of cameras. The image acquisition unit 11 acquires images captured by a camera group 1 that captures images of the periphery of the vehicle. In step S11, the ECU 10 causes the composite image generation unit 12 to acquire the door opening degree. The composite image generation unit 12 acquires the door opening degree based on the detection result of the door opening sensor 2.

[0041] In step S12, the ECU 10 acquires a virtual viewpoint by the synthetic image generating unit 12. The synthetic image generating unit 12 acquires a virtual viewpoint in response to an operation performed by a vehicle occupant on the HMI 3 to select a virtual viewpoint, for example.

[0042] In step S13, the ECU 10 calculates a door opening degree for the composite image corresponding to the actual opening degree of the door of the vehicle by the composite image generating unit 12. The composite image generating unit 12 calculates the door opening degree for the composite image so that the door opening degree is equal to the acquired door opening degree, for example.

[0043] In step S14, the ECU 10 calculates the additional door opening degree for the composite image based on the direction of the virtual viewpoint with respect to the vehicle, using the composite image generating unit 12. For example, the composite image generating unit 12 obtains a preset additional door opening degree according to the direction of the virtual viewpoint with respect to the vehicle, and calculates the additional door opening degree for the composite image. The additional door opening degree is smaller for a viewpoint where the vehicle door opening is easier to see, and is larger for a viewpoint where the vehicle door opening is harder to see.

[0044] In step S15, the ECU 10 generates a composite vehicle image by the composite image generating unit 12 so as to show the door open at the door opening angle for the composite image. The composite image generating unit 12 adds the door opening addition amount to the door opening angle for the composite image, which is equal to the door opening angle, and generates a composite vehicle image so as to show the door open at the door opening angle after the addition. At this time, the door opening addition amount is smaller as the value at the viewpoint where the opening of the vehicle door is easily seen, and is larger as the value at the viewpoint where the opening of the vehicle door is less easily seen. Therefore, when the virtual viewpoint is located in the first direction, the composite vehicle image is generated so as to show the door open more widely than when the virtual viewpoint is located in the second direction.

[0045] In step S16, the ECU 10 causes the display control unit 13 to display the composite vehicle image on the in-vehicle display unit. The display control unit 13 displays the composite vehicle image on, for example, the display of the HMI 3. After that, the ECU 10 ends the current process of FIG.

[0046] The first direction and the second direction may have predetermined angular ranges relative to the vehicle. For example, for a viewpoint located on the left side of the vehicle, the first direction may be determined to be a direction between 0 degrees and less than 45 degrees that does not face the opening of the hinged door, and the second direction may be determined to be a direction between 45 degrees and 180 degrees that faces the opening of the hinged door. In this case, the ECU 10 can execute the process shown in FIG. 6.

[0047] Fig. 6 is a flowchart showing another example of the process of the ECU in Fig. 1. The process of the ECU in Fig. 6 is executed when a request to draw a composite vehicle image is made while the vehicle is stopped with the door open, similar to Fig. 5.

[0048] 6, in step S21, the ECU 10 acquires images captured by the multiple cameras using the image acquisition unit 11, similar to step S11. The ECU 10 acquires the door opening degree using the composite image generation unit 12. In step S22, the ECU 10 acquires a virtual viewpoint using the composite image generation unit 12, similar to step S12.

[0049] In step S23, the ECU 10 determines whether or not the virtual viewpoint is located in the first direction by the synthetic image generation unit 12. If the synthetic image generation unit 12 determines that the virtual viewpoint is located in the first direction (S23: YES), the ECU 10 proceeds to step S26. If the synthetic image generation unit 12 determines that the virtual viewpoint is not located in the first direction (S23: NO), the ECU 10 proceeds to step S24.

[0050] In step S24, the ECU 10 determines whether or not the virtual viewpoint is located in the second direction by the synthetic image generation unit 12. If the synthetic image generation unit 12 determines that the virtual viewpoint is located in the second direction (S24: YES), the ECU 10 proceeds to step S25. If the synthetic image generation unit 12 determines that the virtual viewpoint is not located in the second direction (S24: NO), the ECU 10 ends the current process of FIG. 6.

[0051] In step S25, the ECU 10 generates a composite vehicle image so as to show the door opened at a door opening degree corresponding to the actual door opening degree by the composite image generating unit 12. For example, the composite image generating unit 12 calculates a door opening degree for the composite image that is equal to the door opening degree, and generates a composite vehicle image so as to show the door opened at a door opening degree equal to the door opening degree without adding the additional door opening degree amount.

[0052] Meanwhile, in step S26, the ECU 10 causes the synthetic image generating unit 12 to generate a synthetic vehicle image so as to show the door opened wider than when the virtual viewpoint is located in the second direction. For example, the synthetic image generating unit 12 adds a door opening addition amount to the door opening angle for the synthetic image equal to the door opening angle, and generates a synthetic vehicle image so as to show the door opened at a door opening angle larger than the angle equal to the door opening angle.

[0053] In step S27, the ECU 10 causes the display control unit 13 to display the composite vehicle image on the in-vehicle display unit. The display control unit 13 displays the composite vehicle image on, for example, the display of the HMI 3. After that, the ECU 10 ends the current process of FIG.

[0054] According to the display device 100 described above, a plurality of captured images acquired using the camera group 1 are synthesized by the synthetic image generating unit 12, and synthetic vehicle images 40C, 40F representing a vehicle viewed from a virtual viewpoint are generated. The generated synthetic vehicle images 40C, 40F are displayed on the HMI 3 mounted on the vehicle. When the virtual viewpoint is located in a predetermined first direction, the synthetic vehicle images 40C, 40F are generated so as to represent a door that is more widely open than when the virtual viewpoint is located in a predetermined second direction. Therefore, the open door is more easily visible in the synthetic vehicle images 40C, 40F viewed from the virtual viewpoint located in the first direction. Thus, according to the display device 100, it is easier to recognize the degree to which the vehicle door is open in the synthetic vehicle images 40C, 40F displayed on the HMI 3.

[0055] The vehicle door includes a hinge door 21 that opens by rotating around a hinge fixed to the vehicle body of the vehicle 20. The first direction is a direction that does not face the opening 22 of the hinge door 21, and the second direction is a direction that faces the opening 22 of the hinge door 21. When the virtual viewpoint is located in the first direction, the synthetic image generating unit 12 generates a synthetic vehicle image 40C so as to represent the hinge door 21 opened with a larger amount of rotation compared to when the virtual viewpoint is located in the second direction. In this case, it is easy to recognize the degree to which the hinge door 21 of the vehicle 20 is opened in the synthetic vehicle image 40C displayed on the HMI 3.

[0056] The vehicle door includes a sliding door 31 that moves in the vehicle width direction with respect to the vehicle body of the vehicle 30 by a predetermined shift amount SD1 and moves in the front-rear direction to open. The first direction is a direction that does not face the opening 32 of the sliding door 31, and the second direction is a direction that faces the opening 32 of the sliding door 31. When the virtual viewpoint is located in the first direction, the synthetic image generating unit 12 generates a synthetic vehicle image 40F so as to represent the sliding door 31 that has moved in the vehicle width direction by a shift amount SD2 that is larger than when the virtual viewpoint is located in the second direction. In this case, it is easy to recognize the degree to which the sliding door 31 of the vehicle 30 is opened in the synthetic vehicle image 40F displayed on the HMI 3.

[0057] Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments. The present disclosure can be implemented in various forms including the above-described embodiments and various modifications and improvements based on the knowledge of those skilled in the art.

[0058] In the above embodiment, the synthetic vehicle image is generated based on the door opening angle sensor 2 opening the door to the same degree as the actual door opening angle, but is not limited to this. The door opening angle sensor 2 may be a known courtesy switch that detects the opening and closing of the vehicle door. In this case, when the vehicle door is open according to whether the courtesy switch is ON or OFF, a synthetic vehicle image showing the door opened to a preset door opening angle may be generated based on the door opening angle sensor 2 and an additional door opening angle may be added.

[0059] In the above embodiment, the hinge door 21 and the sliding door 31 are exemplified as the vehicle door, but the vehicle door is not limited to these examples. For example, the vehicle door may be a trunk or luggage room door, a hood of a bonnet, or a gull-wing door of a sports vehicle. [Explanation of symbols]

[0060] 11: image acquisition unit, 12: composite image generation unit, 13: display control unit, 20, 30: vehicle, 21: hinge door, 22, 32: opening, 31: sliding door, 40C, 40F: composite vehicle image, 100: display device, SD1, SD2: shift amount.

Claims

1. an image acquisition unit that acquires images captured by a plurality of cameras that capture images of the surroundings of the vehicle; a composite image generating unit that combines a plurality of the captured images to generate a composite vehicle image that represents the vehicle as seen from a virtual viewpoint; a display control unit that displays the composite vehicle image on an in-vehicle display unit, The display device, wherein the synthetic image generation unit generates the synthetic vehicle image based on a direction of the virtual viewpoint relative to the vehicle, so that when the virtual viewpoint is located in a predetermined first direction, the door is represented as being wider open than when the virtual viewpoint is located in a predetermined second direction in which the opening of the vehicle door is easier to see from the virtual viewpoint than in the first direction.

2. The vehicle door includes a hinge door that opens by pivoting about a hinge fixed to a body of the vehicle, The first direction is a direction not facing an opening of the hinge door, the second direction is a direction facing an opening of the hinge door, 2. The display device according to claim 1, wherein the synthetic image generation unit generates the synthetic vehicle image so as to represent the hinged door opened with a larger amount of rotation when the virtual viewpoint is located in the first direction compared to when the virtual viewpoint is located in the second direction.

3. the door of the vehicle includes a sliding door that moves by a predetermined shift amount in a vehicle width direction relative to a body of the vehicle and moves in a front-rear direction to open, the first direction is a direction not facing an opening of the sliding door, the second direction is a direction facing an opening of the sliding door, 3. The display device according to claim 1, wherein the synthetic image generation unit generates the synthetic vehicle image so as to represent the sliding door moved in the vehicle width direction by a shift amount that is larger when the virtual viewpoint is located in the first direction compared to when the virtual viewpoint is located in the second direction.

Images