Vehicle surroundings display device and method for controlling vehicle surroundings display device

Abstract

The purpose of the present invention is to suppress a feeling of discomfort of a user in recognizing the surroundings of a vehicle using a virtual space. A vehicle surroundings display device generates a virtual space corresponding to the surroundings of a host vehicle on the basis of detection information from an external sensor of the host vehicle, and displays, on a display, an image within the virtual space observed from a virtual viewpoint operated by a user of the host vehicle. A stereoscopic host vehicle icon corresponding to a host vehicle is disposed in a virtual space, and when a virtual viewpoint is located in an icon deformation region set above the rear or the front of the host vehicle icon, a stretching display is performed in which the entire length of the host vehicle icon is stretched compared to when the virtual viewpoint is not located in the icon deformation region.

Description

Technical Field

[0001] The present invention relates to a vehicle surrounding environment display device and a control method for the vehicle surrounding environment display device. Background Art

[0002] Conventionally, as a technical document related to a vehicle surrounding environment display device, Japanese Unexamined Patent Application Publication No. 2020-088697 is known. In this publication, a surrounding monitoring device is shown that generates a virtual space including an icon of the own vehicle and projects the surrounding environment of the vehicle as a three-dimensional image into the virtual space. In this device, the user can freely visually recognize the environment around the vehicle by operating a virtual viewpoint in the virtual space. Prior Art Documents Patent Documents

[0003] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2020-088697 Summary of the Invention Problems to be Solved by the Invention

[0004] In addition, in the case of displaying the virtual space observed from the virtual viewpoint on a display for the user to visually recognize as in the conventional device described above, it is difficult to grasp the positional relationship between the own vehicle and surrounding objects compared to the actual environment. Therefore, a technique for correcting the user's recognition in a manner that suppresses the sense of incongruity in the user's recognition of the surrounding environment of the own vehicle is desired. Means for Solving the Technical Problem

[0005] One aspect of the present invention is a vehicle surrounding environment display device that generates a virtual space corresponding to the surrounding environment of the own vehicle based on detection information from external sensors of the own vehicle, and displays an image in the virtual space observed from a virtual viewpoint operated by a user of the own vehicle on a display. A three-dimensional own vehicle icon corresponding to the own vehicle is arranged in the virtual space. When the virtual viewpoint is located in an icon deformation area set above the rear or front of the own vehicle icon, a stretching display that stretches the entire length of the own vehicle icon is performed compared to the case where the virtual viewpoint is not located in the icon deformation area.

[0006] In the vehicle surrounding environment display device according to one aspect of the present invention, it may also be that when the virtual viewpoint is not located in the icon deformation area, the closer the virtual viewpoint is to the icon deformation area, the more the entire length of the own vehicle icon is stretched, and the farther the virtual viewpoint is from the icon deformation area, the closer the entire length of the own vehicle icon is to a preset initial setting length.

[0007] Another aspect of the present invention is a control method for a vehicle surrounding environment display device. A virtual space corresponding to the surrounding environment of the vehicle is generated based on the detection information of external sensors of the vehicle, and an image in the virtual space observed from a virtual viewpoint operated by a user of the vehicle is displayed on a display. A three-dimensional vehicle icon corresponding to the vehicle is configured in the virtual space. When the virtual viewpoint is located in an icon deformation area set above the rear or front of the vehicle icon, a stretching display that stretches the entire length of the vehicle icon is performed as compared with the case where the virtual viewpoint is not located in the icon deformation area. Advantageous Effects of the Invention

[0008] According to each aspect of the present invention, it is possible to suppress a sense of discomfort of a user in recognizing the surrounding environment of the vehicle using the virtual space. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a block diagram showing a vehicle surrounding environment display device according to an embodiment. Figure 2 is a diagram for explaining the vehicle icon and the virtual viewpoint. Figure 3 is a diagram for explaining the drawing state of an object in the virtual space. Figure 4 (a) is a diagram showing an example of the deformation state of the vehicle icon when the virtual viewpoint is located on the side of the vehicle icon outside the icon deformation area. Figure 4 (b) is a diagram showing an example of the deformation state of the vehicle icon when the virtual viewpoint starts to rotate and move to the right and approaches the icon deformation area behind the vehicle icon. Figure 4 (c) is a diagram showing an example of the deformation state of the vehicle icon when the virtual viewpoint further rotates and moves to the right. Figure 4 (d) is a diagram showing an example of the deformation state of the vehicle icon when the virtual viewpoint rotates and moves to the right side of the vehicle icon. Figure 5 (a) is a diagram showing an example of area division corresponding to the position of an object in a top view. Figure 5 (b) is a diagram showing an example of area division corresponding to the height of an object in a side view. Figure 6 is a diagram for explaining the change in the drawing state in the virtual space due to the difference in the height of an object on the side of the vehicle. Figure 7 is a flowchart showing an example of the control method of the vehicle surrounding environment display device according to the present embodiment. Detailed implementation mode

[0010] Next, the implementation mode of the present invention will be described with reference to the accompanying drawings.

[0011] Figure 1 It is a block diagram showing a vehicle surrounding environment display device 100 according to an embodiment. Figure 1 The shown vehicle surrounding environment display device 100 is mounted on vehicles such as sedans and trucks. Hereinafter, the vehicle mounted with the vehicle surrounding environment display device 100 will be referred to as the present vehicle. The vehicle surrounding environment display device 100 is a device for supporting a user to recognize the vehicle surrounding environment. The vehicle surrounding environment display device 100 generates a virtual space reflecting the surrounding environment of the present vehicle, and displays an image in the virtual space observed from a virtual viewpoint operated by the user on a display. The vehicle surrounding environment display device 100 displays the surrounding environment of the present vehicle on the display as a so-called 3D view.

[0012] The user can be either the driver of the present vehicle, a passenger of the present vehicle, or the owner of the present vehicle. The user can also be an operator who performs remote assistance for the present vehicle through a remote assistance system. In the remote assistance system, the operator can judge the driving of the present vehicle (judgment of traveling, turning left or right, stopping, etc.) or perform driving operations of the present vehicle through a remote assistance device provided outside the vehicle and capable of communicating with the present vehicle. The present vehicle is not limited to a vehicle capable of being remotely assisted through a remote assistance system. The present vehicle can be a vehicle with an autonomous driving function or a vehicle without an autonomous driving function.

[0013] [Configuration of vehicle surrounding environment display device] As Figure 1As shown, the vehicle surrounding environment display device 100 includes an ECU [Electronic Control Unit] 10 that centrally manages the device. The ECU 10 is an electronic control unit having a CPU [Central Processing Unit] and a storage unit. The storage unit is constituted by, for example, a ROM [Read Only Memory], a RAM [Random Access Memory], an EEPROM [Electrically Erasable Programmable Read-Only Memory], etc. In the ECU 10, various functions are realized, for example, by the CPU executing a program stored in the storage unit. The ECU 10 may also be constituted by a plurality of electronic units. The ECU 10 is connected to an external camera 1 (external sensor), a radar sensor 2 (external sensor), a user operation reception unit 3, and a display 4.

[0014] The external camera 1 is a photographing device that photographs the external conditions of the vehicle. The external camera 1 includes, for example, a front camera that photographs the front of the vehicle, a rear camera that photographs the rear of the vehicle, and a plurality of side cameras that photograph the left and right sides of the vehicle, respectively. The number of cameras of the external camera 1 is not particularly limited and may also be one. The external camera 1 transmits photographed image information to the ECU 10.

[0015] The radar sensor 2 is a detection device that uses radio waves (e.g., millimeter waves) or light to detect objects around the vehicle. The radar sensor 2 may include a millimeter wave radar or a lidar [LIDAR: Light Detection and Ranging]. The radar sensor 2 transmits object detection information related to the detected objects to the ECU 10. In addition, the radar sensor 2 and the external camera 1 constitute an external sensor for detecting the surrounding environment of the vehicle. The object detection information of the radar sensor 2 or the photographed image information of the external camera 1 is equivalent to the detection information of the external sensor.

[0016] The user operation reception unit 3 is a device that receives user operations on the virtual viewpoint. The user operation reception unit 3 may be, for example, an input unit of an HMI [Human Machine Interface] provided on the vehicle. The input unit includes, for example, a touch panel display, buttons, a control lever, switches, etc. In addition, the user operation reception unit 3 may also be able to receive operations based on voice recognition or gestures.

[0017] As the user operation reception unit 3, the input device of a portable terminal or a computer communicatively connected to the vehicle may also be used. Additionally, as the user operation reception unit 3, the operator's terminal of a remote assistance system may also be used.

[0018] The display 4 is, for example, a center display mounted on the instrument panel of the vehicle. The display 4 may be the display of a tablet computer that can be installed on the vehicle, or may be a HUD [Head Up Display]. The display 4 does not need to be installed on the vehicle. The display 4 may also be the operator's display of a remote assistance system installed in a facility away from the vehicle. The display 4 may be the display of a portable terminal carried by the user, or may be the display of the user's tablet computer or desktop computer.

[0019] Next, the functional configuration of the ECU 10 will be described. As Figure 1 shown, the ECU 10 includes a virtual space generation unit 11 and an image display unit 12. A part of the functions of the ECU 10 described below may also be executed by a server, a portable terminal, or a computer that can communicate with the vehicle. The server that can communicate with the vehicle may be, for example, the server of a remote assistance system. The computer may be, for example, a tablet computer or a desktop computer.

[0020] The virtual space generation unit 11 generates a virtual space corresponding to the surrounding environment of the vehicle, for example, based on the captured image information of the external camera 1. The surrounding environment of the vehicle includes, for example, the position of the white lines of the lane in which the vehicle is traveling. The surrounding environment of the vehicle may also include the conditions (position, traveling direction, etc.) of other vehicles such as a preceding vehicle, a neighboring vehicle, and a parallel vehicle.

[0021] The virtual space is generated, for example, as a 3D image synthesized from a plurality of images. The method of synthesizing the images is not particularly limited. The virtual space generation unit 11 generates a virtual space as a 3D image, for example, by projecting each image in a global coordinate system that serves as a reference for the virtual space and associating overlapping pixels with each other.

[0022] The virtual space generation unit 11 arranges the vehicle icon corresponding to the own vehicle within the virtual space. The vehicle icon is arranged as a three-dimensional icon in the shape of a vehicle. The vehicle icon can be formed by polygons, voxels, or other CG processing. The virtual space generation unit 11 can also generate a vehicle icon that reflects the actual state of the own vehicle. The virtual space generation unit 11 can also reflect the lighting state of the actual lighting device of the own vehicle in the lighting state of the lighting device in the vehicle icon. The lighting state of the lighting device of the own vehicle is, for example, the lighting state of the headlight, turn indicator, or brake light of the own vehicle. The virtual space generation unit 11 can also reflect the steering angle of the actual tire of the own vehicle in the tire of the vehicle icon. The shape and size of the vehicle icon are preset according to the vehicle model.

[0023] When the virtual space generation unit 11 recognizes an object based on the captured image information of the external camera 1, it arranges the icon corresponding to the object within the virtual space. The object can be a wheel stopper installed in a parking lot or the like, a curb, another vehicle, or a pedestrian. The virtual space generation unit 11 can also recognize other vehicles or the like not based on the captured image information of the external camera 1 but based on the object detection information of the radar sensor 2, and can also use both the external camera 1 and the radar sensor 2 to recognize other vehicles or the like.

[0024] In addition, the virtual space generation unit 11 can also use the information on the surrounding environment recognized by other vehicles through vehicle-to-vehicle communication to recognize other vehicles or the like around the own vehicle. For example, the virtual space generation unit 11 can also obtain the image information of the cameras installed on the road and various traffic information by communicating with the traffic information management server managed by the state. The virtual space generation unit 11 can also use the image information of the cameras installed on the road and various traffic information to recognize other vehicles or the like.

[0025] In addition, the virtual space generation unit 11 can also predict the behavior of other vehicles based on the captured image information of the external camera 1 or the object detection information of the radar sensor 2. In this case, the virtual space generation unit 11 displays the prediction result of the behavior of the other vehicle in association with the other vehicle icon. The virtual space generation unit 11 can also display the predicted travel route of the other vehicle through an arrow icon or the like. The virtual space generation unit 11 can also display the predicted stop position of the other vehicle that is decelerating using an icon in the form of a block extending in the lane width direction. Similarly, the virtual space generation unit 11 can also display the prediction result of the behavior of the pedestrian in association with the pedestrian icon.

[0026] The method of generating the virtual space is not limited to the method of synthesizing multiple images of the external camera 1, and other methods can also be used. If it is a way for the user to recognize the surrounding environment of the vehicle, the virtual space generation unit 11 does not need to generate the virtual space as a 3D image. The virtual space generation unit 11 can also generate a digital virtual space by arranging the vehicle icon, the white line, and other vehicle icons in a way that the white line and the positional relationship of other vehicles relative to the vehicle can be known, rather than in the form of an image.

[0027] The image display unit 12 displays, on the display 4, an image within the virtual space observed from the virtual viewpoint operated by the user in the virtual space generated by the virtual space generation unit 11. The image display unit 12 moves the virtual viewpoint 50 according to the user operation input to the user operation reception unit 3.

[0028] The image display unit 12 changes the shape of the vehicle icon M according to the position of the virtual viewpoint 50. Thereby, the user's position recognition of the vehicle icon M and other vehicle icons is corrected. Specifically, as an example, the image display unit 12 changes the overall length of the vehicle icon M according to the position of the virtual viewpoint 50.

[0029] Figure 2 It is a diagram for explaining the vehicle icon and the virtual viewpoint. Figure 2 The vehicle icon M, the virtual viewpoint 50, and the icon deformation area CA are shown. The plane on which the vehicle icon M is arranged corresponds to the horizontal plane of the global coordinate system. Figure 2 The line of sight DA of the virtual viewpoint 50 and the depression angle α of the virtual viewpoint 50 are shown. In Figure 2 For easy understanding, the virtual viewpoint 50 is illustrated as the icon of a camera. The depression angle α is the angle formed by the plane on which the vehicle icon M is arranged in the vertical plane or the horizontal plane of the global coordinate system and the line of sight DA. In addition, it is not necessary to display the virtual viewpoint 50 as an icon on the virtual space.

[0030] The icon deformation area CA is an area preset for the deformation of the vehicle icon M. As Figure 2 shown, as an example, the icon deformation area CA is a spherical area set above the rear of the vehicle icon M. The icon deformation area CA can be a rectangular parallelepiped-shaped area, or a cylindrical area. The icon deformation area CA can be a triangular pyramid-shaped area, or a polygonal area. The icon deformation area CA can also be an area that expands in a sector shape in cross section from the vehicle icon M toward the upper rear of the vehicle icon M. The shape of the icon deformation area CA is not particularly limited.

[0031] The icon deformation area CA can be set to include the initial position of the virtual viewpoint 50. The initial position of the virtual viewpoint refers to the position in the virtual space where the virtual viewpoint 50 is preconfigured when the image display function of the vehicle surrounding environment display device 100 is started. The icon deformation area CA can also be an area composed of a coordinate point in the global coordinate system. This coordinate point can also be the initial position of the virtual viewpoint 50.

[0032] The width of the icon deformation area CA can also be set not to exceed the width of the own vehicle icon M. When the overall width of the own vehicle icon M changes due to deformation, the width of the icon deformation area CA can be set not to exceed the width of the own vehicle icon M when the width becomes the shortest.

[0033] Here, refer to Figure 3 to describe the depiction state of the object in the virtual space. Figure 3 It is a diagram for explaining the depiction state of the object in the virtual space. Figure 3 The side camera Sc of the own vehicle, the object B1 located in the front side of the own vehicle, the object B2 located in the side of the own vehicle, the object B3 located in the rear side of the own vehicle, and the projection plane P corresponding to the display in the virtual space are shown. The objects B1 to B3 are blocks having the same size and the same shape. Here, for the sake of easy understanding, only the side end portions of the blocks are shown as quadrilateral icons. The projection plane P is a bowl-shaped surface for explaining the size of the objects B1 to B3 depicted in the virtual space. The projection plane P is formed based on the side camera Sc. In addition, the position of the side camera Sc is not limited to the side rearview mirror of the own vehicle and can be any position.

[0034] In Figure 3 the shown state, the size (width of the upper surface) of the object B1 in the virtual space generated based on the captured image of the object B1 observed from the side camera Sc is shown by the reference numeral V1. The size (width of the upper surface) of the object B2 in the virtual space generated based on the captured image of the object B2 observed from the side camera Sc is shown by the reference numeral V2. Similarly, the size (width of the upper surface) of the object B3 in the virtual space generated based on the captured image of the object B3 observed from the side camera Sc is shown by the reference numeral V3. It can be considered that the size in the virtual space corresponds to the size projected on the projection plane P or the floor of the virtual space (equivalent to the road surface where the own vehicle icon M is located, etc.).

[0035] In this case, as Figure 3As shown, due to the positional relationship between the side camera Sc and the objects B1 to B3, there will be a difference between the actual sizes of the objects B1 to B3 and their sizes in the virtual space. In the case of object B3, the size Wv in the virtual space becomes nearly three times the actual size Wr. Therefore, if the user wants to use the virtual space to identify the positional relationship between the vehicle and object B3, there may be an identification deviation.

[0036] Then, when the virtual viewpoint 50 is located in the icon deformation area CA, compared with the case where the virtual viewpoint 50 is not located in the icon deformation area CA, the image display unit 12 performs a stretching display that stretches the entire length (the length in the front-rear direction) of the vehicle icon M. By performing the stretching display of the vehicle icon M, the image display unit 12 suppresses the user's sense of discomfort in recognizing the surrounding environment of the vehicle using the virtual space. In addition, the image display unit 12 grasps the position information of the virtual viewpoint 50 in the virtual space.

[0037] In addition, the vehicle icon M has an entire length (initial setting length) that is predetermined as an initial setting. In the stretching display, the entire length of the vehicle icon M is stretched to be longer than the initial setting length.

[0038] The image display unit 12 can also perform a stretching display by equally stretching the entire vehicle icon M. As the stretching display, the image display unit 12 can also stretch the rear overhang portion behind the rear axle of the vehicle icon M. The image display unit 12 can also perform the stretching of the entire vehicle icon M or the rear overhang portion of the vehicle icon M in such a way that the center position does not change with the center of the vehicle icon M as a reference. The image display unit 12 can also perform the stretching of the entire vehicle icon M or the rear overhang portion of the vehicle icon M in such a way that the position of the rear axle does not change with the rear axle of the vehicle icon M as a reference.

[0039] In addition, when the icon deformation area CA is set above the front of the vehicle icon M and the virtual viewpoint 50 is located in the icon deformation area CA, the image display unit 12 can also stretch the front overhang portion in front of the front axle of the vehicle icon M. In this case, the image display unit 12 can perform the stretching with the center of the vehicle icon M as a reference or with the front axle of the vehicle icon M as a reference.

[0040] Even when the virtual viewpoint 50 is not located in the icon deformation area CA, the image display unit 12 can also deform the vehicle icon M according to the change in the positional relationship between the virtual viewpoint 50 and the icon deformation area CA. That is, the image display unit 12 can also be such that the vehicle icon M smoothly deforms in the form of an animation according to the user's operation of the virtual viewpoint 50.

[0041] Specifically, when the virtual viewpoint 50 is located outside the icon deformation area CA, the image display unit 12 can also deform the own vehicle icon M in such a way that the farther the virtual viewpoint 50 is from the icon deformation area CA, the closer the overall length of the own vehicle icon M is to the initially set length. The distance between the virtual viewpoint 50 and the icon deformation area CA in this case can also be grasped as a straight-line distance in the global coordinate system. In the case of rotational movement, the distance between the virtual viewpoint 50 and the icon deformation area CA can be counted as the distance along the rotation trajectory. In the case where there are multiple icon deformation areas CA, the distance to the icon deformation area CA closest to the virtual viewpoint 50 is used.

[0042] Refer to Figure 4 A description will be given of the change in the own vehicle icon M when the virtual viewpoint 50 is not located in the icon deformation area CA. Figure 4 (a) of FIG. is an example of a diagram for explaining the deformation state of the own vehicle icon M when the virtual viewpoint 50 is located on the side of the own vehicle icon M outside the icon deformation area. Figure 4 (b) of FIG. is an example of a diagram for explaining the deformation state of the own vehicle icon M when the virtual viewpoint 50 starts to rotate and move to the right and approaches the icon deformation area behind the own vehicle icon M. Figure 4 (c) of FIG. is an example of a diagram for explaining the deformation state of the own vehicle icon M when the virtual viewpoint 50 further rotates and moves to the right. Figure 4 (d) of FIG. is an example of a diagram for explaining the deformation state of the own vehicle icon M when the virtual viewpoint 50 rotates and moves to the right side of the own vehicle icon M.

[0043] In Figure 4 (a) to Figure 4 (d) of FIG., the deformation states observed from the side of the own vehicle icon M are shown as dashed lines DM1 to DM4. In Figure 4 (a) to Figure 4 (d) of FIG., the own vehicle icon M is stretched toward the rear with the front wheel axle of the own vehicle icon M as a reference.

[0044] As Figure 4 (a) to Figure 4 (d) of FIG. show, the more the virtual viewpoint 50 rotates and moves to the right and approaches the icon deformation area CA, the more the image display unit 12 stretches the overall length of the own vehicle icon M. When the virtual viewpoint 50 rotates and moves to the left in a way that is away from the icon deformation area CA due to the user's operation, the image display unit 12 deforms the own vehicle icon M so that the overall length returns to the initially set length. That is, the farther the virtual viewpoint 50 is from the icon deformation area CA, the more the image display unit 12 contracts the overall length of the own vehicle icon M in a way that approaches the initially set length.

[0045] In this way, the image display unit 12 smoothly deforms the own vehicle icon M according to the positional relationship between the virtual viewpoint 50 and the icon deformation region CA. Thereby, the image display unit 12 can suppress the user from feeling a sense of incongruity with the deformation of the own vehicle icon M.

[0046] Next, a method of deforming the own vehicle icon M corresponding to the position of an object will be described. The image display unit 12 may also change the deformation rate of the entire length of the own vehicle icon M according to the position of an object around the own vehicle.

[0047] As Figure 3 shown, for objects B1 to B3 of the same size, their sizes V1 to V3 in the virtual space change according to their positions relative to the own vehicle icon M. The distances from the side camera Sc become longer in the order of object B2, object B1, and object B3. The sizes V1 to V3 in the virtual space become larger in the order of object B2, object B1, and object B3. Therefore, the image display unit 12 may also change the deformation mode of the own vehicle icon M according to the position of an object existing around the own vehicle icon M.

[0048] Here, Figure 5 (a) is a diagram showing an example of area division corresponding to the position of an object in a plan view. In Figure 5 (a), the periphery of the own vehicle icon M is divided into three areas in the front-rear direction of the own vehicle icon M. Figure 5 (a) shows an area A in front of the own vehicle, an area B in the center of the own vehicle, and an area C behind the own vehicle. The areas A to C are set as, for example, rectangular areas having a certain width in the horizontal direction with the own vehicle icon M as the center. Objects B10 to B12 are objects located in the areas A to C, respectively.

[0049] As Figure 5 (a) shows, the image display unit 12 may also change the own vehicle icon deformation rate according to the area where an object exists. The own vehicle icon deformation rate corresponds to the degree of stretching the entire length of the own vehicle icon M. Specifically, when only the object B10 in the area A exists, the image display unit 12 sets the own vehicle icon deformation rate to medium. When only the object B11 in the area B exists, the image display unit 12 sets the own vehicle icon deformation rate to small. When only the object B12 in the area C exists, the image display unit 12 sets the own vehicle icon deformation rate to large. As an example, the deformation rate may be set as follows: large is a value near 100%, medium is about 50%, and small is a value less than 20%.

[0050] In this way, the image display unit 12 changes the overall length of the own-vehicle icon M according to the position of an object around the own vehicle. Thus, the image display unit 12 can correct the overall length of the own-vehicle icon M in a manner that enables the user to easily recognize the positional relationship between the object and the own-vehicle icon M.

[0051] The area division is not limited to Figure 5 the division method shown in (a) of FIG. It may also be other methods. The area may also be divided according to the distance from the left and right side cameras Sc of the own vehicle. The area may be divided into four or more instead of three.

[0052] The image display unit 12 may also be configured not to change the overall length of the own-vehicle icon M when no object is detected within a preset object approach determination area. The detection of the object is performed based on the captured image of the external camera 1 or the detection result of the radar sensor 2. The object approach determination area is an area of the actual space set to include the own vehicle. The object approach determination area is set to determine whether it is necessary to change the overall length of the own-vehicle icon M.

[0053] The image display unit 12, for example, may also use the area of the actual space corresponding to Figure 5 area A and area C in (a) of FIG. as the object approach determination area. In this case, when there is no object in area A and area C and only the object B11 in area B exists, the image display unit 12 may not change the overall length of the own-vehicle icon M from the initial set length. Even if the virtual viewpoint 50 was once located in the icon deformation area CA, the image display unit 12 does not change the overall length of the own-vehicle icon M from the initial set length. On the other hand, in the case where there is an object B10 in area A or an object B12 in area C, the image display unit 12 changes the overall length of the own-vehicle icon M according to the position of the virtual viewpoint 50.

[0054] The object approach determination area is not limited to the area of the actual space corresponding to area A and area C. The object approach determination area may be the area of the actual space corresponding to either area A or area C, or may be the area of the actual space corresponding to all of area A to C. The object approach determination area may also be an area within a certain distance from the own vehicle. The object approach determination area may not include the areas in front of and behind the own vehicle but may be an area within a certain distance from the own vehicle in the lateral direction of the own vehicle.

[0055] Next, a method of deforming the own-vehicle icon M corresponding to the height of the object will be described. The image display unit 12 may also change the overall length of the own-vehicle icon M according to the height of an object around the own vehicle.

[0056] The image display unit 12 may also deform the own-vehicle icon M according to the height of an object around the own vehicle. Figure 6This is a diagram for explaining the change in the depiction state in the virtual space due to the difference in the height of an object on the side of the vehicle. Figure 6 An object B4 having a height higher than that of object B3 is shown. Object B4 is a block having the same shape as object B3 except for the height and exists at the same position as object B3. In this case, as Figure 6 shown, the dimension V4 (the upper surface in the virtual space) of object B4 in the virtual space is larger than the dimension V3 of object B3 having a lower height.

[0057] Here, Figure 5 (b) of Figure 5 is a diagram showing an example of the area division corresponding to the height of the object in a side view. In Figure 5 (b), areas D to F divided in the vertical direction of the vehicle icon M of the present vehicle are shown. Specifically, an area D above the vehicle of the present vehicle, an area E in the middle section of the vehicle of the present vehicle, and an area F below the vehicle of the present vehicle are shown. In Figure 5 (b), objects B10 to B12 are objects having different heights. The heights of the upper surfaces of objects B10 to B12 correspond to areas D to F, respectively.

[0058] The image display unit 12 may also change the vehicle icon deformation rate of the present vehicle according to the height of the object as shown in Figure 5 (b). Specifically, when only object B10 corresponding to the height of area E exists, the image display unit 12 sets the vehicle icon deformation rate of the present vehicle to medium. When only object B11 corresponding to the height of area F exists, the image display unit 12 sets the vehicle icon deformation rate of the present vehicle to small. When only object B12 corresponding to the height of area D exists, the image display unit 12 sets the vehicle icon deformation rate of the present vehicle to large.

[0059] In this way, the image display unit 12 changes the overall length of the vehicle icon M according to the height of the object around the vehicle. Thus, the image display unit 12 can correct the overall length of the vehicle icon M in a manner that enables the user to easily recognize the positional relationship between the object and the vehicle icon M.

[0060] The area division in the vertical direction is not limited to the division method shown in Figure 5 (a), and other methods may also be used. The area may be divided into four or more instead of three. In addition, the image display unit 12 may also change the overall length of the vehicle icon M in consideration of both the position and height of the object. The image display unit 12 may also be configured not to change the overall length of the vehicle icon M when the object exists only in area B in a top view and the height of the object is included in area F in a side view.

[0061] [Program] The program causes the ECU 10 to function as the virtual space generation unit 11 and the image display unit 12 described above. The program is provided by a non-transitory recording medium such as a ROM or a semiconductor memory. Additionally, the program can also be provided via communication such as a network.

[0062] [Control Method of Vehicle Surrounding Environment Display Device] Next, a control method for the vehicle surrounding environment display device 100 according to the present embodiment will be described with reference to the drawings. Figure 7 It is a flowchart showing an example of the control method for the vehicle surrounding environment display device 100 according to the present embodiment.

[0063] As Figure 7 shown, as S10, the ECU 10 of the vehicle surrounding environment display device 100 determines whether the virtual viewpoint 50 is located in the icon deformation area CA through the image display unit 12. When the ECU 10 determines that the virtual viewpoint 50 is located in the icon deformation area CA (S10: "Yes"), it proceeds to S11. When the ECU 10 does not determine that the virtual viewpoint 50 is located in the icon deformation area CA (S10: "No"), it proceeds to S12.

[0064] In S11, the ECU 10 causes the own vehicle icon M to deform through the image display unit 12 for screen display. For example, the ECU 10 performs a stretching display of the entire length of the own vehicle icon M on the display 4 in a manner with a large deformation rate. Then, this processing of the current time ends.

[0065] In S12, the ECU 10 determines through the image display unit 12 whether the distance between the virtual viewpoint 50 and the icon deformation area CA is less than a certain distance. When the ECU 10 determines that the distance between the virtual viewpoint 50 and the icon deformation area CA is less than a certain distance (S12: "Yes"), it proceeds to S13. When the ECU 10 does not determine that the distance between the virtual viewpoint 50 and the icon deformation area CA is less than a certain distance (S12: "No"), it proceeds to S14.

[0066] In S13, the ECU 10 causes the own vehicle icon M to deform at a deformation rate corresponding to the distance between the virtual viewpoint 50 and the icon deformation area CA through the image display unit 12 for screen display. The ECU 10 performs image display in such a way that the own vehicle icon M deforms smoothly with less sense of incongruity according to the change in the position of the virtual viewpoint 50 by the user. Then, this processing of the current time ends.

[0067] In S14, the ECU 10 performs screen display without deforming the own vehicle icon M through the image display unit 12. The own vehicle icon M is displayed, for example, in the initially set shape. Then, this processing of the current time ends.

[0068] According to the vehicle surrounding environment display device 100 and its control method according to the above-described embodiment, when the virtual viewpoint 50 is located in the icon deformation area CA, the full length of the own vehicle icon M is made longer than the case where the virtual viewpoint 50 is not located in the icon deformation area CA, so that the own vehicle icon M is deformed. Thus, according to the vehicle surrounding environment display device 100 and its control method, it is possible to suppress a sense of discomfort of the user in recognizing the surrounding environment of the own vehicle using the virtual space compared to the actual space.

[0069] In addition, the vehicle surrounding environment display device 100 performs animation control to smoothly deform the own vehicle icon M according to the distance between the virtual viewpoint 50 and the icon deformation area CA. Thus, it is possible to suppress a sense of discomfort of the user with respect to the deformation of the own vehicle icon M.

[0070] Furthermore, when the vehicle surrounding environment display device 100 does not detect an object in the object approach determination area including the own vehicle, regardless of the position of the virtual viewpoint 50, the stretching display of the own vehicle icon M is not performed. Thus, it is possible to avoid unnecessary deformation of the own vehicle icon M.

[0071] Although the embodiments of the present invention have been described above, the present invention is not limited to the above embodiments. The present invention can be implemented in various ways in which various changes and improvements are made based on the knowledge of those skilled in the art starting from the above embodiments.

[0072] The vehicle surrounding environment display device 100 may also set the icon deformation area CA above the front of the own vehicle icon M. The vehicle surrounding environment display device 100 may also set the icon deformation area CA above both the front and the rear of the own vehicle icon M.

[0073] The vehicle surrounding environment display device 100 does not necessarily need to smoothly deform the own vehicle icon M according to the change in the position of the virtual viewpoint 50. As long as the vehicle surrounding environment display device 100 deforms the shape of the own vehicle icon M when comparing the case where the virtual viewpoint 50 is located in the icon deformation area CA and the case where the virtual viewpoint 50 is not located in the icon deformation area CA, it is acceptable. Description of Reference Numerals

[0074] 1: External camera, 2: Radar sensor, 3: User operation reception unit, 4: Display, 10: ECU, 11: Virtual space generation unit, 12: Image display unit, 50: Virtual viewpoint, 100: Vehicle surrounding environment display device.

Claims

1. A vehicle surrounding environment display device, which generates a virtual space corresponding to the surrounding environment of the host vehicle based on detection information of an external sensor of the host vehicle, and displays an image in the virtual space observed from a virtual viewpoint operated by a user of the host vehicle on a display, A three-dimensional host vehicle icon corresponding to the host vehicle is arranged in the virtual space. When the virtual viewpoint is located in an icon deformation area set above the rear or front of the host vehicle icon, the host vehicle icon is displayed stretched in its entirety compared to a case where the virtual viewpoint is not located in the icon deformation area.

2. The vehicle surrounding environment display device according to claim 1, wherein: When the virtual viewpoint is not located in the icon deformation area, the farther the virtual viewpoint is from the icon deformation area, the closer the total length of the vehicle icon is to a preset initial set length; and the closer the virtual viewpoint is to the icon deformation area, the more the total length of the vehicle icon is stretched.

3. The vehicle surrounding environment display device according to claim 1 or 2, wherein: When no object is detected in a preset object approach determination area including the host vehicle, the stretched display is not performed regardless of the position of the virtual viewpoint.

4. The vehicle surrounding environment display device according to any one of claims 1 to 3, wherein: The deformation rate of the stretched display is changed according to the position of the object around the host vehicle.

5. A method for controlling a vehicle surrounding environment display device, comprising generating a virtual space corresponding to the surrounding environment of the vehicle based on detection information of an external sensor of the vehicle, and displaying an image in the virtual space observed from a virtual viewpoint operated by a user of the vehicle on a display, A three-dimensional host vehicle icon corresponding to the host vehicle is arranged in the virtual space. When the virtual viewpoint is located in an icon deformation area set above the rear or front of the host vehicle icon, the host vehicle icon is displayed stretched in its entirety compared to a case where the virtual viewpoint is not located in the icon deformation area.

Images