Vehicle peripheral environment display device and control method of vehicle peripheral environment display device

Abstract

To prevent a user from having an uncomfortable feeling in recognition of an own vehicle peripheral environment using a virtual space.SOLUTION: In a vehicle peripheral environment display device that generates a virtual space corresponding to an own vehicle peripheral environment on the basis of detection information of an external sensor of an own vehicle and displays an image within the virtual space seen from a virtual viewpoint operated by a user of the own vehicle on a display, a three-dimensional own vehicle icon corresponding to the own vehicle is arranged in a virtual space, and performs extension display that extends the total length of the own vehicle icon compared with a case where the virtual viewpoint is not positioned in an icon deformation area when the virtual viewpoint is positioned in the icon deformation area which is set in a backward sky or a forward sky.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a vehicle surrounding environment display device and a control method for the vehicle surrounding environment display device.

Background Art

[0002] Conventionally, as a technical document related to a vehicle surrounding environment display device, Japanese Patent Application Laid-Open No. 2020-088697 is known. This publication discloses a peripheral monitoring device that generates a virtual space including a self-vehicle icon and projects the surrounding environment of the vehicle into the virtual space as a three-dimensional video. A user can freely view the environment around the vehicle by operating a virtual viewpoint in the virtual space.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, when displaying the virtual space viewed from the virtual viewpoint on the display for the user to view as in the above-described conventional device, it is difficult to grasp the positional relationship between the self-vehicle and the surrounding objects compared to the actual environment. For this reason, a technique for correcting the user's recognition so as to suppress the sense of incongruity in the user's recognition of the surrounding environment of the self-vehicle is desired.

Means for Solving the Problems

[0005] One aspect of the present invention is a vehicle surrounding environment display device that generates a virtual space corresponding to the surrounding environment of the host vehicle based on detection information of external sensors of the host vehicle and displays an image in the virtual space as seen from a virtual viewpoint operated by a user of the host vehicle on a display. In the virtual space, a three-dimensional host vehicle icon corresponding to the host vehicle is arranged. When the virtual viewpoint is located in an icon deformation area set above the rear or front of the host vehicle icon, an extension display is performed to extend the overall length of the host vehicle icon compared to when the virtual viewpoint is not located in the icon deformation area.

[0006] In the vehicle surrounding environment display device according to one aspect of the present invention, when the virtual viewpoint is not located in the icon deformation area, the overall length of the host vehicle icon approaches a preset initial setting length as the virtual viewpoint moves away from the icon deformation area, and the overall length of the host vehicle icon may be extended as the virtual viewpoint approaches the icon deformation area.

[0007] Another aspect of the present invention is a control method for a vehicle surrounding environment display device that generates a virtual space corresponding to the surrounding environment of the host vehicle based on detection information of external sensors of the host vehicle and displays an image in the virtual space as seen from a virtual viewpoint operated by a user of the host vehicle on a display. In the virtual space, a three-dimensional host vehicle icon corresponding to the host vehicle is arranged. When the virtual viewpoint is located in an icon deformation area set above the rear or front of the host vehicle icon, an extension display is performed to extend the overall length of the host vehicle icon compared to when the virtual viewpoint is not located in the icon deformation area.

Advantages of the Invention

[0008] According to each aspect of the present invention, it is possible to suppress a user from feeling uncomfortable when recognizing the surrounding environment of the host vehicle using the virtual space.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Embodiments for Carrying Out the Invention

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0011] FIG. 1 is a block diagram showing a vehicle surrounding environment display device 100 according to an embodiment. The vehicle surrounding environment display device 100 shown in FIG. 1 is mounted on a vehicle (hereinafter referred to as a host vehicle) such as a passenger car or a freight car, and is a device for supporting a user's recognition of the surrounding environment of the vehicle. The vehicle surrounding environment display device 100 generates a virtual space reflecting the surrounding environment of the host vehicle, and displays an image in the virtual space viewed from a virtual viewpoint operated by the user on a display. The vehicle surrounding environment display device 100 displays the surrounding environment of the host vehicle on the display as a so-called 3D view.

[0012] The user may be the driver of the own vehicle, a passenger in the own vehicle, or the owner of the own vehicle. The user may also be an operator who performs remote assistance for the own vehicle by a remote assistance system. In the remote assistance system, the operator can make a judgment on the running of the own vehicle (such as judgment of progress, turning right or left, stopping, etc.) or perform a driving operation of the own vehicle through remote assistance equipment provided outside the vehicle and capable of communicating with the own vehicle. The own vehicle is not limited to a vehicle that can be remotely assisted by the remote assistance system. The own vehicle may be a vehicle having an automatic driving function or a vehicle not having an automatic driving function.

[0013] [Configuration of Vehicle Surrounding Environment Display Device] As shown in FIG. 1, the vehicle surrounding environment display device 100 includes an ECU [Electronic Control Unit] 10 that comprehensively manages the device. The ECU 10 is an electronic control unit having a CPU [Central Processing Unit] and a storage unit. The storage unit is composed of, for example, a ROM [Read Only Memory], a RAM [Random Access Memory], an EEPROM [Electrically Erasable Programmable Read-Only Memory], etc. In the ECU 10, various functions are realized, for example, by the CPU executing a program stored in the storage unit. The ECU 10 may be composed of a plurality of electronic units. The ECU 10 is connected to an external camera 1 (external sensor), a radar sensor 2 (external sensor), a user operation reception unit 3, and a display 4.

[0014] The external camera 1 is an imaging device that images the external situation of the own vehicle. The external camera 1 includes, for example, a front camera that images the front of the own vehicle, a back camera that images the rear of the own vehicle, and a plurality of side cameras that image the left and right sides of the own vehicle respectively. The number of cameras of the external camera 1 is not particularly limited and may be one. The external camera 1 transmits imaging image information to the ECU 10.

[0015] The radar sensor 2 is a detection device that detects objects around the host vehicle using radio waves (e.g., millimeter waves) or light. The radar sensor 2 can be configured to include a millimeter wave radar or a lidar [LiDAR: Light Detection and Ranging]. The radar sensor 2 transmits object detection information regarding the detected objects to the ECU 10. Note that the radar sensor 2 and the external camera 1 constitute an external sensor for detecting the surrounding environment of the host vehicle. The object detection information of the radar sensor 2 or the imaging image information of the external camera 1 corresponds to the detection information of the external sensor.

[0016] The user operation reception unit 3 is a device that receives operations of a virtual viewpoint by the user. The user operation reception unit 3 can be, for example, an input unit of an HMI [Human Machine Interface] provided in the host vehicle. The input unit includes, for example, a touch panel display, buttons, levers, switches, etc. Further, the user operation reception unit 3 may be able to receive operations by voice recognition or gestures.

[0017] As the user operation reception unit 3, input devices of a mobile terminal or a computer communicatively connected to the host vehicle may be used. Further, a terminal for an operator of a remote support system may be used as the user operation reception unit 3.

[0018] The display 4 is, for example, a center display mounted on the dashboard of the host vehicle. The display 4 may be a display of a tablet-type computer installable in the host vehicle, or may be a HUD [Head Up Display]. The display 4 is not limited to being mounted on the host vehicle. The display 4 may be a display for an operator of a remote support system provided in a facility away from the host vehicle. The display 4 may be a display of a mobile terminal carried by the user, or may be a display of the user's tablet-type computer or desktop computer.

[0019] Next, the functional configuration of the ECU 10 will be described. As shown in FIG. 1, the ECU 10 has a virtual space generation unit 11 and an image display unit 12. Some of the functions of the ECU 10 described below may be executed by a server capable of communicating with the host vehicle (for example, a server of a remote assistance system), a mobile terminal, or a computer (for example, a tablet computer or a desktop computer).

[0020] The virtual space generation unit 11 generates a virtual space corresponding to the surrounding environment of the host vehicle based on, for example, the imaging image information of the external camera 1. The surrounding environment of the host vehicle includes, for example, the position of the white line of the lane in which the host vehicle is traveling. The surrounding environment of the host vehicle may include the situation (position, traveling direction, etc.) of other vehicles such as the preceding vehicle and the parallel vehicles of the adjacent vehicle.

[0021] The virtual space is generated as, for example, a 3D video synthesized from a plurality of images. The image synthesis method is not particularly limited. The virtual space generation unit 11 generates a virtual space as a 3D video by, for example, projecting each image onto a global coordinate system that is a reference for the virtual space and associating the overlapping pixels with each other.

[0022] The virtual space generation unit 11 arranges a host vehicle icon corresponding to the host vehicle in the virtual space. The host vehicle icon is arranged as a three-dimensional icon in the shape of a vehicle. The host vehicle icon can be formed by polygon, voxel, or other CG processing. The virtual space generation unit 11 may generate a host vehicle icon that reflects the state of the host vehicle. The virtual space generation unit 11 may reflect the lighting state of the lighting devices of the host vehicle (lighting states of headlights, direction indicators, brake lights, etc.) in the lighting state of the lighting devices in the host vehicle icon, and may also reflect the steering angle of the tires of the host vehicle in the tires of the host vehicle icon. The shape and size of the host vehicle icon are preset according to the vehicle type.

[0023] When the virtual space generation unit 11 recognizes an object based on the imaging image information of the external camera 1, it places an icon corresponding to the object in the virtual space. The object may be a tire stopper provided in a parking lot or the like, a curb, another vehicle, or a pedestrian. The virtual space generation unit 11 may recognize another vehicle or the like based on the object detection information of the radar sensor 2 instead of the imaging image information of the external camera 1, or may recognize another vehicle or the like using both the external camera 1 and the radar sensor 2.

[0024] Note that the virtual space generation unit 11 may recognize other vehicles or the like around the host vehicle using the information on the surrounding environment recognized by other vehicles through vehicle-to-vehicle communication. The virtual space generation unit 11 may, for example, communicate with a traffic information management server managed by a country to acquire image information of cameras installed on the road and various traffic information, and use these to recognize other vehicles or the like.

[0025] In addition, the virtual space generation unit 11 may predict the behavior of other vehicles based on the imaging image information of the external camera 1 or the object detection information of the radar sensor 2, and display the prediction result of the behavior of other vehicles in association with the other vehicle icon. The virtual space generation unit 11 may, for example, display the predicted travel route of another vehicle using an arrow icon or the like, or display the predicted stop position of another vehicle that is decelerating using a block-type icon that extends in the lane width direction. Similarly, the virtual space generation unit 11 may display the prediction result of the behavior of a pedestrian in association with the pedestrian icon.

[0026] The method of generating the virtual space is not limited to the method of synthesizing a plurality of images of the external camera 1, and other methods are also possible. The virtual space generation unit 11 does not need to generate the virtual space as a 3D video as long as the user can recognize the surrounding environment of the host vehicle. The virtual space generation unit 11 may generate a digital virtual space by arranging the host vehicle icon, the white line, and the other vehicle icon so that the positional relationship between the white line and the other vehicle with respect to the host vehicle can be understood, instead of a video.

[0027] The image display unit 12 displays, on the display 4, an image within the virtual space as seen from the virtual viewpoint that the user operates in the virtual space generated by the virtual space generation unit 11. The image display unit 12 moves the virtual viewpoint 50 according to the user operation input to the user operation reception unit 3.

[0028] The image display unit 12 corrects the position recognition of the own vehicle icon M and other vehicle icons by the user by changing the shape of the own vehicle icon M according to the position of the virtual viewpoint 50. Specifically, as an example, the image display unit 12 changes the overall length of the own vehicle icon M according to the position of the virtual viewpoint 50.

[0029] FIG. 2 is a diagram for explaining the own vehicle icon and the virtual viewpoint. FIG. 2 shows the own vehicle icon M, the virtual viewpoint 50, and the icon deformation area CA. The plane on which the own vehicle icon M is arranged corresponds to the horizontal plane of the global coordinate system. FIG. 2 shows the line of sight DA of the virtual viewpoint 50 and the depression angle α of the virtual viewpoint 50. In FIG. 2, the virtual viewpoint 50 is illustrated as an icon of a camera for ease of understanding. The depression angle α is the angle formed between the plane on which the own vehicle icon M is arranged in the vertical plane or the horizontal plane of the global coordinate system and the line of sight DA. Note that it is not essential to display the virtual viewpoint 50 as an icon on the virtual space.

[0030] The icon deformation area CA is an area preset for use in deforming the own vehicle icon M. As shown in FIG. 2, the icon deformation area CA is, as an example, a spherical area set above the rear of the own vehicle icon M. The icon deformation area CA may be a rectangular parallelepiped-shaped area, a cylindrical area, a triangular pyramid-shaped area, or a polygonal-shaped area. The icon deformation area CA may be an area that expands in a fan-shaped cross section from the own vehicle icon M upward obliquely rearward of the own vehicle icon M. The shape of the icon deformation area CA is not particularly limited.

[0031] The icon deformation region CA may be set to include the initial position of the virtual viewpoint 50. The initial position of the virtual viewpoint is the position in the virtual space where the virtual viewpoint 50 is pre-arranged when the image display function of the vehicle surrounding environment display device 100 is activated. The icon deformation region CA may be a region consisting of one coordinate point in the global coordinate system. The coordinate point may be the initial position of the virtual viewpoint 50.

[0032] The width of the icon deformation region CA may be set not to exceed the width of the host vehicle icon M. When the overall width of the host vehicle icon M changes due to deformation, the width of the icon deformation region CA can be set not to exceed the width of the host vehicle icon M when the width is the shortest.

[0033] Here, with reference to FIG. 3, the drawing state of the object in the virtual space will be described. FIG. 3 is a diagram for explaining the drawing state of the object in the virtual space. FIG. 3 shows the side camera Sc of the host vehicle, the object B1 located in front of the side of the host vehicle, the object B2 located on the side of the host vehicle, the object B3 located behind the side of the host vehicle, and the projection plane P corresponding to the display in the virtual space. The objects B1 to B3 are blocks having the same size and the same shape. Here, for ease of understanding, only the side end portions of the blocks are shown as square icons. The projection plane P is a bowl-shaped surface used to explain the size of the objects B1 to B3 drawn in the virtual space. The projection plane P is formed based on the side camera Sc. Note that the position of the side camera Sc is not limited to the side mirror of the host vehicle and can be set at any position.

[0034] In the situation shown in FIG. 3, the size (width of the upper surface) of object B1 in the virtual space generated from the captured image of object B1 as viewed from side camera Sc is indicated by reference numeral V1. The size (width of the upper surface) of object B2 in the virtual space generated from the captured image of object B2 as viewed from side camera Sc is indicated by reference numeral V2. Similarly, the size (width of the upper surface) of object B3 in the virtual space generated from the captured image of object B3 as viewed from side camera Sc is indicated by reference numeral V3. It can be considered that the size in the virtual space corresponds to the size projected onto projection plane P or the floor of the virtual space (corresponding to the road surface on which the own-vehicle icon M is located, etc.).

[0035] In this case, as shown in FIG. 3, due to the positional relationship between side camera Sc and objects B1 to B3, a difference occurs between the actual sizes of objects B1 to B3 and their sizes in the virtual space. In the case of object B3, the size Wv in the virtual space is nearly three times larger than the actual size Wr. For this reason, when the user attempts to recognize the positional relationship between the own vehicle and object B3 using the virtual space, there may be a recognition deviation.

[0036] Therefore, when virtual viewpoint 50 is located in icon deformation region CA, image display unit 12 performs an extension display that extends the overall length (length in the front-rear direction) of own-vehicle icon M as compared to the case where virtual viewpoint 50 is not located in icon deformation region CA. By performing the extension display of own-vehicle icon M, image display unit 12 suppresses the user from feeling a sense of discomfort when recognizing the surrounding environment of the own vehicle using the virtual space. Note that image display unit 12 grasps the position information of virtual viewpoint 50 in the virtual space.

[0037] In addition, own-vehicle icon M has an overall length (initial setting length) determined in advance as an initial setting. In the extension display, the overall length of own-vehicle icon M is extended so as to be longer than the initial setting length.

[0038] The image display unit 12 may uniformly stretch the entire self-vehicle icon M as a stretched display, or may stretch the rear overhang portion behind the rear axle of the self-vehicle icon M. The image display unit 12 may stretch the whole or the rear overhang portion so that the center position does not change with reference to the center of the self-vehicle icon M. The image display unit 12 may stretch the whole or the rear overhang portion so that the position of the rear axle does not change with reference to the rear axle of the self-vehicle icon M.

[0039] In addition, when the virtual viewpoint 50 is located in the icon deformation region CA when the icon deformation region CA is set above the front sky of the self-vehicle icon M, the image display unit 12 may stretch the front overhang portion in front of the front axle of the self-vehicle icon M. In this case, the image display unit 12 may perform stretching with reference to the center of the self-vehicle icon M, or may perform stretching with reference to the front axle of the self-vehicle icon M.

[0040] Even when the virtual viewpoint 50 is not located in the icon deformation region CA, the image display unit 12 may deform the self-vehicle icon M according to the change in the positional relationship between the virtual viewpoint 50 and the icon deformation region CA. That is, the image display unit 12 may be in a mode in which the self-vehicle icon M smoothly deforms as an animation according to the operation of the virtual viewpoint 50 by the user.

[0041] Specifically, outside the icon deformation region CA, the image display unit 12 may deform the self-vehicle icon M so that the overall length returns to the initial set length as the distance between the virtual viewpoint 50 and the icon deformation region CA increases. The distance may be a straight-line distance in the global coordinate system, or may be counted as a distance along the rotation trajectory in the case of rotational movement. When there are a plurality of icon deformation regions CA, the distance to the icon deformation region CA closest to the virtual viewpoint 50 is used.

[0042] The change of the host vehicle icon M when the virtual viewpoint 50 is not located in the icon deformation region CA will be described with reference to FIG. 4. FIG. 4(a) is a diagram for explaining an example of the deformation state of the host vehicle icon M when the virtual viewpoint 50 is located on the side of the host vehicle icon M outside the icon deformation region. FIG. 4(b) is a diagram for explaining an example of the deformation state of the host vehicle icon M when the virtual viewpoint 50 starts to rotate to the right and approaches the icon deformation region behind the host vehicle icon M. FIG. 4(c) is a diagram for explaining an example of the deformation state of the host vehicle icon M when the virtual viewpoint 50 further rotates to the right. FIG. 4(d) is a diagram for explaining an example of the deformation state of the host vehicle icon M when the virtual viewpoint 50 rotates to the right side of the host vehicle icon M.

[0043] In FIGS. 4(a) to 4(d), the deformation states seen from the side of the host vehicle icon M are shown as broken lines DM1 to DM4. In FIGS. 4(a) to 4(d), the host vehicle icon M is extended backward with reference to the front wheel axle of the host vehicle icon M.

[0044] As shown in FIGS. 4(a) to 4(d), the image display unit 12 extends the overall length of the host vehicle icon M as the virtual viewpoint 50 rotates to the right and approaches the icon deformation region CA. When the virtual viewpoint 50 rotates to the left and moves away from the icon deformation region CA by the user's operation, the image display unit 12 deforms the host vehicle icon M so that the overall length returns to the initial set length. That is, the image display unit 12 shrinks the overall length of the host vehicle icon M so as to approach the initial set length as the virtual viewpoint 50 moves away from the icon deformation region CA.

[0045] In this way, the image display unit 12 can suppress the user from feeling uncomfortable with the deformation of the host vehicle icon M by smoothly deforming the host vehicle icon M according to the positional relationship between the virtual viewpoint 50 and the icon deformation region CA.

[0046] Next, a method for deforming the host vehicle icon M according to the position of the object will be described. The image display unit 12 may change the deformation rate of the overall length of the host vehicle icon M according to the position of the object around the host vehicle.

[0047] As shown in FIG. 3, for objects B1 to B3 of the same size, the sizes V1 to V3 in the virtual space change according to their positions relative to the host vehicle icon M. The distances from the side camera Sc become longer in the order of object B2, object B1, and object B3. The sizes V1 to V3 in the virtual space become larger in the order of object B2, object B1, and object B3. Therefore, the image display unit 12 may change the deformation mode of the host vehicle icon M according to the position of the object existing around the host vehicle icon M.

[0048] Here, FIG. 5(a) is a diagram showing an example of area division according to the position of the object in a plan view. In FIG. 5(a), the area around the host vehicle icon M is divided into three areas in the front-rear direction of the host vehicle icon M. Areas A, B, and C in front of the host vehicle, in the center of the host vehicle, and behind the host vehicle are shown in FIG. 5(a). Areas A to C are set as rectangular regions having a certain width in the lateral direction, for example, centered on the host vehicle icon M. Objects B10 to B12 are objects located in areas A to C, respectively.

[0049] As shown in FIG. 5(a), the image display unit 12 may change the host vehicle icon deformation rate according to the area where the object exists. The host vehicle icon deformation rate corresponds to the degree of stretching the overall length of the host vehicle icon M. Specifically, when only the object B10 in area A exists, the image display unit 12 sets the host vehicle icon deformation rate to medium. When only the object B11 in area B exists, the image display unit 12 sets the host vehicle icon deformation rate to small. When only the object B12 in area C exists, the image display unit 12 sets the host vehicle icon deformation rate to large. As an example, the deformation rate can be a value around 100% for large, around 50% for medium, and less than 20% for small.

[0050] In this way, the image display unit 12 can correct the overall length of the host vehicle icon M so that the user can easily recognize the positional relationship between the object and the host vehicle icon M by changing the overall length of the host vehicle icon M according to the position of the object around the host vehicle.

[0051] The area division is not limited to the division method shown in Fig. 5(a), and other methods are also possible. The areas may be divided according to the distance from the left and right side cameras Sc of the host vehicle. The areas may be divided into not three but four or more.

[0052] When no object is detected in the preset object proximity determination area based on the captured image of the external camera 1 or the detection result of the radar sensor 2, the image display unit 12 may not change the overall length of the host vehicle icon M. The object proximity determination area is an area of the real space set to include the host vehicle. The object proximity determination area is set to determine whether it is necessary to change the overall length of the host vehicle icon M.

[0053] For example, the image display unit 12 may use the areas of the real space corresponding to area A and area C in Fig. 5(a) in the virtual space as the object proximity determination area. In this case, when there is no object in area A and area C and only object B11 in area B exists, even if the virtual viewpoint 50 is located in the icon deformation area CA, the image display unit 12 may not change the overall length of the host vehicle icon M from the initial set length. On the other hand, when object B10 in area A or object B12 in area C exists, the image display unit 12 changes the overall length of the host vehicle icon M according to the position of the virtual viewpoint 50.

[0054] The object proximity determination area is not limited to the areas of the real space corresponding to area A and area C. The object proximity determination area may be an area of the real space corresponding to either area A or area C, or may be an area of the real space corresponding to all of areas A to C. The object proximity determination area may be an area within a certain distance from the host vehicle. The object proximity determination area may be an area within a certain distance from the host vehicle in the lateral direction of the host vehicle, excluding the front and rear areas of the host vehicle.

[0055] Next, a method for deforming the own vehicle icon M according to the height of an object will be described. The image display unit 12 may change the overall length of the own vehicle icon M according to the height of an object around the own vehicle.

[0056] The image display unit 12 may deform the own vehicle icon M according to the height of an object around the own vehicle. FIG. 6 is a diagram for explaining a change in the drawing state in a virtual space due to a difference in the height of an object on the side of the own vehicle. FIG. 6 shows an object B4 that is higher than the object B3. The object B4 is a block having the same shape as the object B3 except for the height, and is assumed to be present at the same position as the object B3. In this case, as shown in FIG. 6, the size V4 (upper surface in the virtual space) of the object B4 in the virtual space becomes larger than the size V3 of the object B3 with a lower height in the virtual space.

[0057] Here, FIG. 5(b) is a diagram showing an example of area division according to the height of an object in a side view. In FIG. 5(b), areas D to F divided in the vertical direction of the own vehicle icon M are shown. Specifically, an area D above the own vehicle, an area E in the middle of the own vehicle, and an area F below the own vehicle are shown. In FIG. 5(b), the objects B10 to B12 are objects having different heights. The heights of the upper surfaces of the objects B10 to B12 respectively correspond to the areas D to F.

[0058] The image display unit 12 may change the own vehicle icon deformation rate according to the height of the object as shown in FIG. 5(b). Specifically, when only the object B10 corresponding to the height of the area E exists, the image display unit 12 sets the own vehicle icon deformation rate to medium. When only the object B11 corresponding to the height of the area F exists, the image display unit 12 sets the own vehicle icon deformation rate to small. When only the object B12 corresponding to the height of the area D exists, the image display unit 12 sets the own vehicle icon deformation rate to large.

[0059] In this way, the image display unit 12 can correct the overall length of the host vehicle icon M so that the user can more easily recognize the positional relationship between the object and the host vehicle icon M by changing the overall length of the host vehicle icon M according to the height of the object around the host vehicle.

[0060] The division of the vertical area is not limited to the division method shown in Fig. 5(a), and other methods are also possible. The area may be divided into four or more instead of three. Further, the image display unit 12 may change the overall length of the host vehicle icon M in consideration of both the position and height of the object. When the object only exists in area B in a plan view and the height of the object is included in area F in a side view, the image display unit 12 may not change the overall length of the host vehicle icon M.

[0061] [Program] The program causes the ECU 10 to function as the virtual space generation unit 11 and the image display unit 12 described above. The program is provided by a non-transitory recording medium such as a ROM or a semiconductor memory, for example. Further, the program may be provided via communication such as a network.

[0062] [Control Method of Vehicle Surrounding Environment Display Device] Next, a control method of the vehicle surrounding environment display device 100 according to the present embodiment will be described with reference to the drawings. Fig. 7 is a flowchart showing an example of the control method of the vehicle surrounding environment display device 100 according to the present embodiment.

[0063] As shown in Fig. 7, the ECU 10 of the vehicle surrounding environment display device 100 determines, as S10, whether the virtual viewpoint 50 is located in the icon deformation area CA by the image display unit 12. When the ECU 10 determines that the virtual viewpoint 50 is located in the icon deformation area CA (S10: YES), it proceeds to S11. When the ECU 10 does not determine that the virtual viewpoint 50 is located in the icon deformation area CA (S10: NO), it proceeds to S12.

[0064] In S11, the ECU 10 causes the image display unit 12 to transform the own-vehicle icon M and perform screen display. For example, the ECU 10 performs stretching display with a large deformation rate to stretch the entire length of the own-vehicle icon M on the display 4. Then, this process ends.

[0065] In S12, the ECU 10 determines whether the distance between the virtual viewpoint 50 and the icon deformation area CA is less than a certain distance by the image display unit 12. When the ECU 10 determines that the distance between the virtual viewpoint 50 and the icon deformation area CA is less than the certain distance (S12: YES), it proceeds to S13. When the ECU 10 does not determine that the distance between the virtual viewpoint 50 and the icon deformation area CA is less than the certain distance (S12: NO), it proceeds to S14.

[0066] In S13, the ECU 10 causes the image display unit 12 to transform the own-vehicle icon M at a deformation rate corresponding to the distance between the virtual viewpoint 50 and the icon deformation area CA and perform screen display. The ECU 10 performs image display so that the own-vehicle icon M deforms smoothly with less discomfort due to the change in the position of the virtual viewpoint 50 by the user. Then, this process ends.

[0067] In S14, the ECU 10 performs image display without deforming the own-vehicle icon M by the image display unit 12. The own-vehicle icon M is displayed, for example, in the initial set shape. Then, this process ends.

[0068] According to the vehicle surrounding environment display device 100 and its control method according to the present embodiment described above, when the virtual viewpoint 50 is located in the icon deformation area CA, compared with the case where the virtual viewpoint 50 is not located in the icon deformation area CA, by deforming the own-vehicle icon M so that the entire length of the own-vehicle icon M becomes longer, it is possible to suppress the user from having a sense of discomfort in recognizing the surrounding environment of the own vehicle using the virtual space compared to the real space.

[0069] In addition, the vehicle surrounding environment display device 100 performs animation control to smoothly deform the host vehicle icon M according to the distance between the virtual viewpoint 50 and the icon deformation area CA, thereby suppressing the user from feeling uncomfortable with the deformation of the host vehicle icon M.

[0070] Furthermore, when no object is detected within the object proximity determination area including the host vehicle, the vehicle surrounding environment display device 100 does not perform the stretching display of the host vehicle icon M regardless of the position of the virtual viewpoint 50, thereby avoiding unnecessary deformation of the host vehicle icon M.

[0071] As described above, the embodiments of the present invention have been described. However, the present invention is not limited to the above-described embodiments. The present invention can be implemented in various forms with various changes and improvements based on the knowledge of those skilled in the art, including the above-described embodiments.

[0072] The vehicle surrounding environment display device 100 may set the icon deformation area CA above the front of the host vehicle icon M. The vehicle surrounding environment display device 100 may set the icon deformation area CA above both the front and the rear of the host vehicle icon M.

[0073] The vehicle surrounding environment display device 100 does not necessarily need to smoothly deform the host vehicle icon M according to the change in the position of the virtual viewpoint 50. The vehicle surrounding environment display device 100 may be in a mode of deforming the shape of the host vehicle icon M at least when the virtual viewpoint 50 is located in the icon deformation area CA and when the virtual viewpoint 50 is not located in the icon deformation area CA.

Explanation of Reference Numerals

[0074] 1... External camera, 2... Radar sensor, 3... User operation reception unit, 4... Display, 10... ECU, 11... Virtual space generation unit, 12... Image display unit, 50... Virtual viewpoint, 100... Vehicle surrounding environment display device.

Claims

1. A vehicle surrounding environment display device that generates a virtual space corresponding to the surrounding environment of the vehicle based on detection information from external sensors of the vehicle and displays an image in the virtual space as seen from a virtual viewpoint operated by a user of the vehicle on a display, wherein a three-dimensional vehicle icon corresponding to the vehicle is arranged in the virtual space, and when the virtual viewpoint is located in an icon deformation area set above the rear or front of the vehicle icon, an extension display is performed to extend the overall length of the vehicle icon as compared with the case where the virtual viewpoint is not located in the icon deformation area.

2. When the virtual viewpoint is not located in the icon deformation area, the overall length of the vehicle icon approaches a preset initial setting length as the virtual viewpoint moves away from the icon deformation area, and the overall length of the vehicle icon is extended as the virtual viewpoint approaches the icon deformation area. The vehicle surrounding environment display device according to claim 1.

3. When no object is detected in an object proximity determination area including the preset vehicle, the extension display is not performed regardless of the position of the virtual viewpoint. The vehicle surrounding environment display device according to claim 1 or 2.

4. The vehicle surrounding environment display device according to any one of claims 1 to 3, wherein the deformation rate of the extension display is changed according to the position of an object around the vehicle.

5. A control method for a vehicle surrounding environment display device that generates a virtual space corresponding to the surrounding environment of the vehicle based on detection information from external sensors of the vehicle and displays an image in the virtual space as seen from a virtual viewpoint operated by a user of the vehicle on a display, wherein a three-dimensional vehicle icon corresponding to the vehicle is arranged in the virtual space, and when the virtual viewpoint is located in an icon deformation area set above the rear or front of the vehicle icon, an extension display is performed to extend the overall length of the vehicle icon as compared with the case where the virtual viewpoint is not located in the icon deformation area.

Images