Method and apparatus for representing the environment of a vehicle

Abstract

Method for representing a setting (2) of a vehicle (1), comprising: - capturing the environment (2) of a vehicle (1) in camera images by means of at least one camera (3a, 3b, 3c, 3d) arranged on the vehicle (1), - determining the position of an object (7a, 7b, 7c) in the vicinity (2) of the vehicle (1) relative to the vehicle (1), - generating a view (4) of the environment (2) of the vehicle (1) from the perspective of a virtual camera (5), based on the captured camera images, wherein the vehicle (1) is represented in the view (4) of the environment (2) of the vehicle (1) by a vehicle model (6), - a modification of the visibility of the vehicle model (6) in the view of the environment (2) of the vehicle (1) based on the determined position of the object (7a, 7b, 7c) relative to the vehicle (1), and - a representation of the view (4) of the environment (2) of the vehicle (1) from the viewpoint of the virtual camera (5), characterized in that the vehicle model (6) is represented in different ways when the virtual camera (5) is moving and when the virtual camera (5) is stationary.

Description

State of the art

[0001] The present invention relates to a method and a device for displaying the environment of a vehicle. Driver assistance systems aim to support a driver when maneuvering a vehicle. For example, the driver can be supported by displaying an area in the immediate vicinity of the vehicle to prevent collisions with obstacles that are not within the driver's field of vision.

[0002] In so-called surround-view systems, the vehicle's surroundings are displayed to the driver from a bird's-eye view or another perspective showing the vehicle from the outside. The cameras of such a surround-view system are typically arranged on the vehicle in a way that enables a 360° view of the vehicle's surroundings.

[0003] Document JP 2007-158 426 A discloses an image display device for vehicle peripherals.

[0004] Document US 2014 / 0 085 466 A1 discloses an image-generating device.

[0005] Document JP 2006-238 131 A discloses an apparatus for monitoring the vehicle periphery. Disclosure of the invention

[0006] The inventive method for displaying a vehicle's environment comprises capturing a vehicle's environment in camera images using at least one camera arranged on the vehicle, determining the position of an object in the vehicle's environment relative to the vehicle, generating a view of the vehicle's environment from the perspective of a virtual camera based on the captured camera images, wherein the vehicle is represented in the view of the vehicle's environment by a vehicle model, modifying the visibility of the vehicle model in the view of the vehicle's environment based on the determined position of the object relative to the vehicle, and displaying the view of the vehicle's environment from the perspective of the virtual camera.

[0007] The device according to the invention for displaying the environment of a vehicle comprises at least one camera arranged on the vehicle, which is configured to capture the environment of the vehicle in camera images; vehicle sensors, which are configured to determine the position of an object in the environment of the vehicle relative to the vehicle; a computing unit, which is configured to generate a view of the environment of the vehicle from the perspective of a virtual camera, based on the captured camera images, wherein the vehicle is represented in the view of the environment of the vehicle by a vehicle model, and to modify the visibility of the vehicle model in the view of the environment of the vehicle, based on the determined position of the object relative to the vehicle; and a display device, which is configured to display the view of the environment of the vehicle from the perspective of the virtual camera.

[0008] When generating the view of the vehicle's surroundings from the perspective of the virtual camera, an external view of the vehicle is calculated. Specifically, camera images are projected in a virtual space onto a projection surface that surrounds the vehicle model. The vehicle model, representing the vehicle, is positioned within this projection surface. The projection surface, textured with the camera images, and the vehicle model are rendered by the virtual camera and made available to the driver. The vehicle model is a virtual representation of the vehicle. The generated view displays a representation of the vehicle model. Specifically, the vehicle model is a three-dimensional reconstruction of the vehicle. Thus, the vehicle model is displayed in a virtual environment. The virtual environment is a three-dimensional representation of the vehicle's actual surroundings.

[0009] Determining the object's position relative to the vehicle in its surroundings is preferably achieved using vehicle sensors. These sensors are typically comprised of an array of ultrasonic sensors and / or cameras. It is also advantageous if the vehicle sensors include a LiDAR system. The visibility of the vehicle model in the view is the degree to which the model is visible to an observer when viewing the vehicle's surroundings from the perspective of the virtual camera, while the vehicle model is within the camera's field of view. The visibility of the vehicle model can be modified in various ways, meaning it can be increased or decreased. Reduced visibility occurs when the vehicle model is less easily perceived by the observer and objects positioned behind the vehicle model become visible.Increased visibility occurs when the vehicle model is more easily perceived by the viewer and objects positioned behind the vehicle model become visible. The visibility of the vehicle model thus describes its opacity within the view of the vehicle's surroundings. The visibility of the vehicle model is inversely proportional to the visibility of objects located behind the vehicle model from the perspective of the virtual camera.

[0010] The virtual camera is not an actual camera, but merely a defined point in the vehicle's environment. This point serves as the basis for calculating a field of view. This field of view then corresponds to the visible area displayed in the vehicle's view. The virtual camera thus exists in a virtual space.

[0011] The method and device according to the invention are advantageous because they allow objects to become visible to a viewer of the vehicle's surroundings in certain situations, even if these objects are located behind the vehicle model in the displayed view (i.e., from the perspective of the virtual camera). At the same time, the vehicle model remains visible depending on the situation, thus providing the user with a reference point to correctly estimate the vehicle's position relative to the object in its actual surroundings. This prevents important information from being obscured by the vehicle model in the view.

[0012] The dependent claims describe preferred embodiments of the invention.

[0013] It is advantageous if modifying the visibility of the vehicle model depends on the position of the virtual camera, where the position of the virtual camera is chosen, in particular, based on the object's location in the vehicle's environment. In other words, it is advantageous if modifying the visibility of the vehicle model also depends on a displayed field of view of the virtual camera. Specifically, modifying the visibility of the vehicle model depends on the position of the virtual camera relative to the vehicle model. The position of the virtual camera is defined, in particular, by a distance that exists in a virtual space between the vehicle model and the virtual camera.In this way, the visibility of the vehicle model can be reduced in situations where it is positioned particularly unfavorably within the virtual camera's field of view. Furthermore, the position of the virtual camera generally indicates whether modifying the vehicle model's visibility is advantageous.

[0014] It is also advantageous if modifying the visibility of the vehicle model includes displaying the vehicle model in a transparent representation. This allows the vehicle model's visibility to be reduced. Displaying the vehicle model in a transparent representation is beneficial because, firstly, objects that are behind the vehicle model from the virtual camera's perspective become visible, while simultaneously ensuring the vehicle model remains fully recognizable.

[0015] It is also advantageous if modifying the visibility of the vehicle model includes displaying the vehicle model in a contour view, where an outer contour of the vehicle model is visible and areas within the outer contour are partially or completely transparent. The outer contour thus provides a point of reference by which a user can orient themselves within the displayed view of the vehicle's surroundings. Since a contour is perceived by a viewer as a complete object, this provides a particularly useful guide for user orientation.

[0016] According to the invention, the vehicle model is displayed in different modes depending on whether the virtual camera is moving or stationary. These modes include transparent representation and contour representation. It is particularly advantageous that the vehicle model is displayed transparently when the virtual camera is moving within the virtual space. When the virtual camera stops moving within the virtual space, the vehicle model is displayed in contour representation. This is advantageous because contour representation is less easily perceived in a view where the position of the vehicle model is changing. Orientation based on a transparent representation is therefore preferable.Conversely, it can be assumed that the viewer has a particular interest in a complete representation of objects that lie behind the vehicle model from the perspective of the virtual camera, provided the virtual camera is not moved. The user is thus supported, among other things, in moving the virtual camera within the virtual space, provided this is done through user input.

[0017] It is also advantageous if, when modifying the visibility of the vehicle model based on the determined position of the object relative to the vehicle, a traffic situation is recognized, and the visibility of the vehicle model is modified accordingly. A traffic situation is a situation in which the vehicle currently finds itself. Examples of traffic situations include parking, exiting a parking space, crossing an intersection, changing lanes, and similar situations. A traffic situation is a predefined situation, where the selection of whether a specific traffic situation exists is based on the determined position of the object relative to the vehicle.In particular, when recognizing the traffic situation, additional information is also accessed, such as sensor values ​​from the vehicle, like speed, direction of travel and steering angle.

[0018] It is also advantageous if, when determining the object's position in the vehicle's vicinity, the object's distance from the vehicle is calculated, and if, when modifying the vehicle's visibility, the visibility of the vehicle model is adjusted based on this calculated distance. Specifically, the visibility of the vehicle model is reduced when approaching the object and increased when moving away from it. This is beneficial because it is particularly important that the object is clearly visible when the vehicle is approaching it.Alternatively, the visibility of the vehicle model increases when approaching the object and decreases when moving away. This is advantageous because, especially when the vehicle is approaching the object, it is important that the vehicle model is recognizable as a point of reference. Specifically, the selection between these two alternatives is based on the position of the virtual camera relative to the vehicle model.

[0019] Furthermore, it is advantageous if individual components of the vehicle model are exempt from modifying the vehicle model's visibility. Individual components of the vehicle model are, in particular, those parts of the vehicle model that are perceived by an observer as structural elements of the vehicle. Advantageous individual components include, for example, the vehicle model's wheels, headlights, or bumpers. These individual components of the vehicle model are exempt from modifying the vehicle model's visibility.In particular, these vehicle model components are displayed fully visible, meaning that they completely obscure any object located behind them from the virtual camera's perspective. Therefore, the object is not visible in the vehicle's surroundings within the area obscured by the individual vehicle model component. Individual vehicle model components that are excluded from modifying the vehicle model's visibility are thus preferably displayed with full opacity.

[0020] Furthermore, it is advantageous to texture a ground area in the vehicle's environment view that lies beneath the vehicle in its actual environment. This texture is calculated based on camera images captured before the vehicle covered the ground area. When the visibility of the vehicle model decreases, the ground on which the vehicle is located becomes visible, depending on the orientation and position of the virtual camera. This ground is referred to as the ground area. This ground area cannot be textured using current camera images, as there are typically no cameras positioned beneath the vehicle. However, an area currently located beneath the vehicle was visible in the camera images as the vehicle approached its current position. At that time, the current ground area was captured in the camera images.The relevant area from the previously captured camera images is therefore distorted accordingly, temporarily stored, and applied as a texture to the current ground area. Alternatively, it is advantageous to display a previously provided texture of a road surface as a texture in the ground area. Brief description of the drawing(s)

[0021] Exemplary embodiments of the invention are described in detail below with reference to the accompanying drawing. The drawing shows: Fig. 1 a flowchart of an exemplary method according to the invention for representing the environment of a vehicle according to a first embodiment of the invention, Fig. 2 a representation of a vehicle comprising a device for representing the vehicle's surroundings according to the first embodiment of the invention, Fig. 3. A representation of the vehicle in an exemplary first situation, Fig. 4 a representation of a view of the vehicle's surroundings generated according to the invention in the first situation, Fig. 5. A representation of the vehicle in an exemplary second situation, Fig. 6 a representation of a view of the vehicle's surroundings generated according to the invention in the second situation, Fig. 7 a representation of a view of the vehicle's surroundings generated according to the invention in a third situation, and Fig. 8 a representation of a view of the environment of the vehicle generated according to the invention in a fourth situation. embodiment(s) of the invention

[0022] Fig. Figure 1 shows a flowchart of a method for displaying an environment 2 of a vehicle 1 according to a first embodiment of the invention. The method is started when a user requests a display of a view 4 of the environment 2 of the vehicle 1. Once the method has been started, a first method step S1 is executed.

[0023] In the first process step S1, the environment 2 of the vehicle 1 is captured in camera images using at least one camera 3a, 3b, 3c, 3d arranged on the vehicle 1.

[0024] Fig. Figure 2 is a representation of vehicle 1 in which the method for representing the environment 2 of vehicle 1 is carried out. A first camera 3a, a second camera 3b, a third camera 3c, and a fourth camera 3d are arranged on vehicle 1. Each of the cameras 3a, 3b, 3c, and 3d is equipped with a wide-angle lens. The first camera 3a is arranged at the front of the vehicle, the second camera 3b is arranged on the right side of vehicle 1, the third camera 3c is arranged at the rear of vehicle 1, and the fourth camera 3d is arranged on the left side of vehicle 1. Since the cameras 3a, 3b, 3c, and 3d are equipped with wide-angle lenses, they completely capture the environment 2 of vehicle 1. A 360° all-around view is thus obtained. After a camera image has been captured by each of the cameras 3a, 3b, 3c, 3d, the camera images are transmitted to a computing unit 9 of the vehicle, which is coupled to the cameras 3a, 3b, 3c, 3d.The computing unit 9 comprises a memory and a processor and is designed to control the method according to the invention.

[0025] In the first process step S1, a camera image is captured by each of the cameras 3a, 3b, 3c, 3d arranged on the vehicle 1. Due to the arrangement of the cameras 3a, 3b, 3c, 3d, the environment 2 of the vehicle 1 is depicted in the camera images of cameras 3a, 3b, 3c, 3d. Thus, the environment 2 of the vehicle 1 has been captured in camera images. After the first process step S1, a second process step S2 is carried out.

[0026] In the second procedural step S2, the position of an object 7a, 7b, 7c in the vicinity 2 of vehicle 1 relative to vehicle 1 is determined. Fig. Figure 2 shows an exemplary first object 7a, which is located in front of vehicle 1. It is further shown in Fig. Figure 2 shows that an ultrasonic sensor 8a, 8b, 8c, 8d is arranged on each side of the vehicle 1. A first ultrasonic sensor 8a is located at the front of the vehicle 1, a second ultrasonic sensor 8b is located on the right side of the vehicle 1, a third ultrasonic sensor 8d is located at the rear of the vehicle 1, and a fourth ultrasonic sensor 8d is located on the right side of the vehicle 1. Like the cameras 3a, 3b, 3c, 3d, the ultrasonic sensors 8a, 8b, 8c, 8d are also coupled to the processing unit 9. The ultrasonic sensors 8a, 8b, 8d, 8d form a vehicle sensor system configured to determine the position of the object 7a, 7b, 7c in the environment 2 of the vehicle 1 relative to the vehicle 1. Thus, in the Fig. In the example shown, a distance of 12 to the first object 7a was determined by the first ultrasonic sensor 8a. Thus, the position of the first object 7a in the vicinity 2 of the vehicle relative to vehicle 1 was determined. The first object 7a is located at a distance of 12 in front of vehicle 1. It should be noted that the Fig. The vehicle sensor system shown in Figure 2 can also be expanded by a large number of additional sensors. For example, it is advantageous if additional ultrasonic sensors are arranged on the vehicle 1. After the second process step S2, a third process step S3 is carried out.

[0027] In the third process step S3, view 4 of the environment 2 of vehicle 1 is generated from the perspective of a virtual camera 5, based on the captured camera images. Vehicle 1 is represented in view 4 of the environment 2 by a vehicle model 6. The generation of view 4 of the environment 2 is performed by the processing unit 9. The processing unit 9 provides a virtual space, which is a mathematical construct. This virtual space represents the environment 2 of vehicle 1. The vehicle model 6 is positioned within this virtual space. The vehicle model 6 is a virtual model of vehicle 1. It is positioned in the virtual space at a location that corresponds to a position of vehicle 1 in its actual environment.A projection surface in the form of a bowl is stretched around the vehicle model 6 in the virtual space, with the vehicle model 6 located inside the bowl on a base of the bowl. The camera images captured by cameras 3a, 3b, 3c, and 3d are arranged on the inner walls of the bowl-shaped projection surface. The camera images are arranged such that objects depicted in the camera images are placed on the projection surface at a position that corresponds to the actual position of the objects in the actual environment 2 of the vehicle 1, as seen from the vehicle 1's perspective. The virtual camera 5 is positioned in the virtual space. An image is calculated that represents the projection surface and the vehicle model 6 as viewed from the position of the virtual camera 5. This image is view 4 of the environment 2 of the vehicle 1.After the second process step S2, a third process step S3 is executed.

[0028] In the third process step S3, the visibility of vehicle model 6 in the view of the environment 2 of vehicle 1 is modified, based on the determined position of object 7a, 7b, 7c relative to vehicle 1. This modification of the visibility of vehicle model 6 takes place before the view 4 of the environment 2 of vehicle 1 is created.

[0029] In this first embodiment, the visibility of the vehicle model 6 is modified depending on the determined distance 12. The first object 7a is displayed fully visible when the distance 12 is greater than a threshold value. "Fully visible" means that objects located behind the vehicle model from the perspective of the virtual camera 5 are not displayed in the image of the virtual camera 5; only the vehicle model 6 is shown in this area. As the vehicle 1 approaches the first object 7a, the distance 12 decreases. If the distance 12 falls below the threshold value, the vehicle model 6 becomes transparent. Thus, the visibility of the vehicle model 6 is modified, which includes displaying the vehicle model 6 in a transparent representation.A transparent representation is one in which both the vehicle model 6 and an object located behind the vehicle model 6 from the perspective of the virtual camera 5 are visible. The transparent representation thus changes the opacity of the vehicle model 6.

[0030] When generating view 4 of the environment 2 of vehicle 1, vehicle model 6 is created in view 4 such that it is displayed with modified visibility. Assume that the distance 12 is below the specified threshold. The visibility of vehicle model 6 is therefore reduced. Vehicle model 6 is thus displayed transparently. For example, the opacity of vehicle model 6 is reduced to 50%. Vehicle model 6 is therefore displayed semi-transparently. In view 4 of the environment 2 of vehicle 1, vehicle model 6 is therefore also semi-transparent.

[0031] After the third process step S3, a fourth process step S4 is executed. In the fourth process step S4, view 4 of the environment 2 of the vehicle 1 is displayed from the perspective of the virtual camera 5. For this purpose, the generated view 4 is transmitted from the processing unit 9 to a display device 10, which is configured to display view 4 of the environment 2 of the vehicle 1 from the perspective of the virtual camera 5. The display device 10 is a display inside the vehicle 1, which is connected to the processing unit 9.

[0032] After the fourth process step S4, the process branches back to the first process step S1 and is executed in a loop. If vehicle 1 moves within the vicinity 2 of vehicle 1, the distance 12 changes. For example, if vehicle 1 approaches the first object 7a, vehicle model 6 is initially not displayed transparently, and as soon as the distance 12 falls below the threshold value, vehicle model 6 becomes transparent.

[0033] Fig. Figure 3 shows vehicle 1 in an exemplary initial situation. In this scenario, vehicle 1, which corresponds to the one in Fig. A method for displaying an environment 2 of the vehicle 1, as shown in Figure 2, is carried out according to a second embodiment of the invention. The second embodiment of the invention essentially corresponds to the first embodiment of the invention. In this second embodiment, modifying the visibility of the vehicle model 6 is additionally dependent on the position of the virtual camera 5. The position of the virtual camera 5 is selected, in particular, based on the position of the object 7a, 7b, 7c in the environment 2 of the vehicle 1. Fig. Figure 4 shows a view 4 of the environment 2 of the vehicle 1 created according to the invention, which can be seen by way of example in the Fig. 3 shows the first situation when carrying out the method according to the invention in accordance with the second embodiment of the invention.

[0034] As vehicle 1 approaches the first object 7a, which is located in Fig. Since the view 3 is exemplified by a second vehicle, it can be concluded that for the user, i.e., a viewer of the view 4 generated according to the invention, this area in front of the vehicle 1 is of interest. The virtual camera 5, which was initially arranged next to the vehicle 1 in the virtual space, is thus moved in front of and above the vehicle 1 and shows the front of the vehicle 1 from a bird's-eye view. The virtual camera 5 is also in Fig. 3 shown. Even if the virtual camera is 5 in Fig. As depicted in Figure 3, this camera is not actually located in the environment 2 of the vehicle 1. The virtual camera 5 is located at a corresponding point in the virtual space provided by the computing unit 9, which represents the actual environment 2 of the vehicle 1. Arrow 16 indicates this. Fig. Figure 3 shows the movement trajectory of the virtual camera 5. If the distance 12 between the vehicle 1 and the first object 7a falls below the threshold value, the virtual camera 5 is moved along trajectory 16 in front of and over the front of the vehicle 1, and the viewing direction of the virtual camera 5 is directed downwards. At the same time, the visibility of the vehicle model 6 is reduced.

[0035] In this second embodiment of the invention, the visibility of the vehicle model 6 is not reduced by making the vehicle model 6 transparent, but rather by showing the vehicle model 6 in a contour representation. In the contour representation, an outer contour 11 of the vehicle model 6 is shown visibly, and an area within the outer contour 11 is shown partially or completely transparently. In the Fig. In example 4, the area within the outer contour 11 is shown as completely transparent.

[0036] Reference is made to Fig. 4 taken. In the lower area of Fig. Figure 4 shows the vehicle model 6 depicted in the contour drawing. In an upper area of ​​the Fig. In view 4, the first object 7a is depicted, which is a vehicle. In a lower area of ​​the Fig. In view 4, the vehicle model 6 is depicted. It is evident that the vehicle model 6 is delimited within the environment 2 of the vehicle 1 by the outer contour 11. The outer contour 11 is a line that traces an outer edge of the vehicle model 6 from the perspective of the virtual camera 5. Within the outer contour 11, the vehicle model 6 is completely transparent.

[0037] In this second embodiment, individual components 13 of the vehicle model 6 are excluded from modifying the visibility of the vehicle model 6. In the embodiment described in Fig. In the example shown, the wheels of vehicle model 6 are components 13 that are excluded from modifying the visibility of vehicle model 6. Therefore, they are fully visible and have an opacity of 100%.

[0038] In this second embodiment of the invention, a ground area 14 in view 4 of the environment 2 of the vehicle 1, which lies beneath the vehicle 1 in the actual environment 2, is textured, the texture being calculated based on camera images captured before the ground area was covered by the vehicle 1. For this purpose, camera images are continuously captured by the first camera 3a and stored by the processing unit 9.

[0039] These are placed as textures on the virtual projection surface. If the vehicle 1 moves forward, the textures on the projection surface are shifted into the virtual space on the projection surface in the opposite direction of the vehicle 1's movement. A texture that was positioned in front of the vehicle model 6 on the projection surface is thus shifted underneath the vehicle model 6 when the vehicle 1 moves. In this way, it is possible to depict objects located under the vehicle 1 that are not currently captured by any of the cameras 3a, 3b, 3c, 3d in the view of the vehicle 1's environment 2 created according to the invention. Fig. 4 is shown as an example. A drain 15 is located under vehicle 1. This drain is located at the time when the in Fig. The area shown in Figure 4, the surroundings of vehicle 1, was calculated but is not within the field of view of cameras 3a, 3b, 3c, 3d. Nevertheless, the drain 15 is in the Fig. visible. This is due, firstly, to the fact that the vehicle model 6 is transparent within the outer contour 11 and the texture representing the drain 15 on the projection surface is thus captured by the virtual camera 5 arranged above the vehicle model 6, i.e., it is not optically obscured by the vehicle model 6, even though it is located between the virtual camera 5 and the part of the projection surface in which the drain 15 is represented in the texture.

[0040] Additionally, in view 4 of the environment 2 of the vehicle 1 in this second embodiment, surface elements 18 are shown in the area of ​​objects 7a, 7b, 7c, by which the distance 12 is color-coded. This means that the surface elements 18 change color according to the distance 12 between the vehicle 1 and the first object 7a. The visibility of the surface elements 18 is preferably chosen to correspond to the visibility of the vehicle model 6.

[0041] Fig. Figure 5 shows vehicle 1 in an exemplary second situation. In this situation, vehicle 1, which corresponds to the one in Fig. A method for representing an environment 2 of the vehicle 1, as depicted in Figure 2, is carried out according to a third embodiment of the invention. The third embodiment of the invention essentially corresponds to the first and second embodiments of the invention.

[0042] In this third embodiment of the invention, the visibility of vehicle model 6 is modified based on a traffic situation. The traffic situation is recognized based on the determined position of object 7a, 7b, 7c relative to vehicle 1. For this purpose, a database with defined traffic situations is used, for example. Fig. Example 5 represents a traffic situation, specifically a parking situation. The first object, 7a, and a second object, 7b, are present in the vicinity 2 of vehicle 1. Both objects, 7a and 7b, are parked vehicles. When determining the position of the object in the vicinity 2 of vehicle 1 relative to vehicle 1, both vehicles are detected. Based on their position and distance from vehicle 1, the parking situation is recognized. It should be noted that further parameters describing the current state of vehicle 1 or the vicinity 2 can also be accessed when recognizing the traffic situation. Since vehicle 1 is in a parking situation, the visibility of vehicle model 6 is modified.In this third embodiment, the vehicle model 6 is displayed transparently, just as it is in the first embodiment when the vehicle 1 approaches the first object 7a. However, in this third embodiment, the trigger for displaying the vehicle model 6 transparently is the detected traffic situation and not solely the distance 12 to the first object 7a. Fig. Figure 5 again shows the virtual camera 5, which is positioned at the height of vehicle 1 and is directed towards the right side of vehicle 1.

[0043] Fig. Figure 6 shows the view 4 of the environment 2 of the vehicle 1 from the perspective of the virtual camera 5, which was generated according to this third embodiment of the invention. The vehicle model 6 is shown transparently. Thus, the environment 2 of the vehicle 1 located behind the vehicle model 6 is fully visible. At the same time, the vehicle model 6 is visible. The wheels of the vehicle model 6, in turn, are excluded from modifying the visibility of the vehicle model 6 as individual components 13 of the vehicle model 6.

[0044] Fig. 7 and Fig. Figure 8 shows situations in which a representation of vehicle model 6 with modified visibility is advantageous. Both in Fig. 7 as well as in Fig. Figure 8 shows vehicle model 6 in outline view. Fig. Figure 7 shows that surface elements 18 are arranged behind the vehicle model 6, indicating that the vehicle 1 is approaching a third object 7c located behind the vehicle. The third object 7c is in Fig. 7 is chosen as an example of a tree. The surface elements 18 are arranged in the virtual space in the area of ​​the third object 7c, which is located in the vicinity 2 of the vehicle 1, and indicate the distance of the vehicle 1 from the third object 7c by color. The surface elements 18 are visible even though they are behind the vehicle model 6 from the perspective of the virtual camera 5, because the vehicle model 6 is displayed in outline view.

[0045] Furthermore, in Fig. 7 marks the ground area 14, which in the real world lies under vehicle 1 and is covered with a texture calculated based on camera images that were captured before the ground area was covered by vehicle 1.

[0046] In Fig. Figure 8 depicts a traffic situation in which vehicle 1 is located at a blind intersection. Because vehicle model 6 is shown in outline view, it is possible to see into a cross street 17. At the same time, vehicle model 6 remains recognizable in view 4, allowing an estimation of how far the front of the vehicle is already within the intersection.

[0047] Another traffic situation where modifying the visibility of vehicle model 6 proves particularly advantageous is when vehicle 1 is positioned over an inductive charging pad. In this scenario, it is especially beneficial to apply a texture to the ground area 14, calculated based on camera images captured before vehicle 1 covered the area. This allows the driver to orient themselves effectively using, for example, a transparent vehicle model 6, while simultaneously keeping the inductive charging pad, indicated by a road marking, visible as a reference point.

[0048] Based on sensor information from the vehicle's sensors, the device according to the invention knows where obstacles are located in the vicinity 2 of the vehicle 1. The field of view of the virtual camera 5 changes, for example, when an object is detected near the vehicle 1. The virtual camera 5 is moved accordingly to display the object. When the virtual camera 5 reaches its target position, the vehicle model 6 is displayed fully or partially transparently, allowing the user to better assess the situation and, in particular, to detect low-lying obstacles beneath the vehicle 1. Individual components of the vehicle model 6 can be displayed opaquely. Alternatively, they can be displayed transparently.

[0049] A transparent representation of the vehicle model 6 is advantageous for the visibility of objects located behind the vehicle model 6. However, this also has disadvantages, as the user loses the relationship between the vehicle shape and its edges. The driver, however, must have a clear understanding of the relationship between the vehicle model 6 and the vehicle's surroundings 2 in order to correctly judge distances in the displayed view 4 of the surroundings 2. Therefore, contour representation is also advantageous, whereby the vehicle model 6 is displayed as completely transparent or semi-transparent. Only the outer contour 11 of the vehicle model 6 is displayed with full opacity. The representation in the contour view can also depend on the field of view of the virtual camera 5. This prevents relevant objects, such as approaching vehicles, from being obscured by the vehicle model 6.Furthermore, in embodiments of the invention, it is possible to combine the transparent representation and the contour representation. For example, when the object is very close, the vehicle model 6 is displayed transparently and the outer contour fades.

[0050] In addition to the description above, explicit reference is made to the revelation of the Fig. Referenced to 1 to 8.

Claims

[1] Method for displaying an environment (2) of a vehicle (1), comprising: - capturing an environment (2) of a vehicle (1) in camera images by means of at least one camera (3a, 3b, 3c, 3d) arranged on the vehicle (1), - determining a position of an object (7a, 7b, 7c) in the environment (2) of the vehicle (1) relative to the vehicle (1), - generating a view (4) of the surroundings (2) of the vehicle (1) from the perspective of a virtual camera (5) based on the captured camera images, wherein the vehicle (1) is represented in the view (4) of the surroundings (2) of the vehicle (1) by a vehicle model (6), - modifying a visibility of the vehicle model (6) in the view of the surroundings (2) of the vehicle (1) based on the determined position of the object (7a, 7b, 7c) relative to the vehicle (1), and - displaying the view (4) of the surroundings (2) of the vehicle (1) from the perspective of the virtual camera (5), characterized by that the vehicle model (6) is displayed in different ways when the virtual camera (5) is moving and when the virtual camera (5) is stationary. [2] Method according to claim 1, characterized by that the modification of the visibility of the vehicle model (6) depends on a position of the virtual camera (5), wherein the position of the virtual camera is selected in particular based on the position of the object (7a, 7b, 7c) in the environment (2) of the vehicle (1). [3] Method according to one of the preceding claims, characterized by that modifying the visibility of the vehicle model (6) comprises displaying the vehicle model (6) in a transparent representation. [4] Method according to one of the preceding claims, characterized bythat modifying the visibility of the vehicle model (6) comprises displaying the vehicle model (6) in a contour representation in which an outer contour (11) of the vehicle model (6) is displayed visibly and surface areas within the outer contour (11) are displayed partially or completely transparently. [5] Method according to one of the preceding claims, characterized by that when modifying the visibility of the vehicle model (6) based on the determined position of the object (7a, 7b, 7c) relative to the vehicle (1), a traffic situation is detected and the visibility of the vehicle model (6) is modified based on which traffic situation was detected. [6] Method according to one of the preceding claims, characterized bythat when determining the position of the object (7a, 7b, 7c) in the environment (2) of the vehicle (1), a distance (12) of the object (7a, 7b, 7c) relative to the vehicle (1) is determined, and when modifying the visibility of the vehicle model (6), the visibility of the vehicle model (6) is modified depending on the determined distance (12). [7] Method according to one of the preceding claims, characterized by that individual components (13) of the vehicle model (6) are excluded from modifying the visibility of the vehicle model (6). [8] Method according to one of the preceding claims, characterized by that a ground area (14) in the view (4) of the surroundings (2) of the vehicle (1), which lies below the vehicle (1) in the actual surroundings (2) of the vehicle (1), is covered with a texture, wherein the texture is calculated based on camera images that were captured before the ground area was covered by the vehicle (1). [9] Device for displaying an environment (2) of a vehicle (1), comprising: - at least one camera (3a, 3b, 3c, 3d) arranged on the vehicle (1), which is designed to capture an environment (2) of the vehicle (1) in camera images, - a vehicle sensor system (8a, 8b, 8c, 8d) which is designed to determine a position of an object (7a, 7b, 7c) in the environment (2) of the vehicle (1) relative to the vehicle (1), - a computing unit (9) which is configured to generate a view (4) of the surroundings (2) of the vehicle (1) from the perspective of a virtual camera (5) based on the captured camera images, wherein the vehicle (1) is represented in the view (4) of the surroundings (2) of the vehicle (1) by a vehicle model (6), and to modify a visibility of the vehicle model (6) in the view (4) of the surroundings (2) of the vehicle (1) based on the determined position of the object (7a, 7b, 7c) relative to the vehicle (1), and - a display device (10) which is designed to display the view (4) of the surroundings (2) of the vehicle (1) from the perspective of the virtual camera (5), characterized by that the vehicle model (6) is displayed in different ways when the virtual camera (5) is moving and when the virtual camera (5) is stationary.

Images