Method for generating a perspective image of an environment of a motor vehicle, computer program product, computer-readable storage medium, as well as electronic computing device

Abstract

The invention relates to a method for generating a perspective image (10) of an environment (11) of a motor vehicle (1) by means of an electronic computing device (4) of an assistance system (2) of the motor vehicle (1), comprising the steps: receiving at least a first image of the environment (11) from at least a first perspective by means of the electronic computing device (4); and generating the perspective image (10) depending on the first image by means of a generative network (5) of the electronic computing device (4), wherein topographic information (H) determined by means of the electronic computing device (4) from the at least one image is taken into account in the perspective image (10). Further, the invention relates to a computer program product, to a computer-readable storage medium as well as to an electronic computing device (4).

Description

[0001] 2024PF00234 clean copy

[0002] 1

[0003] Method for generating a perspective image of an environment of a motor vehicle, computer program product, computer-readable storage medium, as well as electronic computing device

[0004] The following invention relates to a method for generating a perspective image of an environment of a motor vehicle by means of an electronic computing device of an assistance system of the motor vehicle according to the applicable claim 1 . Further, the invention relates to a corresponding computer program product, to a corresponding computer-readable storage medium as well as to an electronic computing device.

[0005] Current high-class utility vehicles are equipped with parking assistance systems, which display images for example from the bird's eye view (BEV) to a driver of the motor vehicle. In order to generate such a view, a widespread perspective transformation is used. It is also referred to as inverse perspective mapping (IPM), to project the pixels of environmental images, which are captured by on-board cameras, to a flat plane in that a homography matrix is calculated, which is based on the calibration of the camera.

[0006] An essential disadvantage of IPM is in the assumption that the world is flat since objects with heights, for example vehicles, trees and poles, are severely distorted and thus mask other objects, which are not directly in the line of sight of the camera. Moreover, errors in the camera calibration contribute to the fact that IPM cannot seamlessly merge the projected perspective views, which results in misalignments at the transition zones.

[0007] Due to the technical restrictions, IPM is more often employed in parking assistance systems, in particular in contrast to general driving aids, and these systems offer either fixed representations of the environment, wherein real bird's eye perspective images are restricted to a projection range of about + / - five meters around the motor vehicle, or they reproduce a virtual 360 degree 3D reconstruction of the scene around the vehicle, wherein the more flexible, interactive and dynamic controllability of the displayed view by the user would considerably improve the experience of the user.

[0008] US 9961259 B2 describes an image generating unit, which uses a plurality of captured image data and vehicle body data of a motor vehicle to continuously generate a virtual perspective image, which displays the vehicle body of the motor vehicle and the periphery 2024PF00234 clean copy

[0009] 2 of the motor vehicle from a virtual perspective, which is positioned in the cabin of the vehicle. An image control unit changes the angle of the line of sight of the virtual perspective in a top view such that the line of sight encircles the environment of the vehicle, and shifts the position of the virtual perspective in the front-back direction of the vehicle.

[0010] It is the object of the present invention to provide a method, a computer program product as well as an electronic computing device, by means of which an improved perspective view or an improved perspective image of an environment of the motor vehicle can be generated.

[0011] This object is solved by a method, a computer program product, a computer-readable storage medium as well as an electronic computing device according to the independent claims. Advantageous forms of configuration are specified in the dependent claims.

[0012] An aspect of the invention relates to a method for generating a perspective image of an environment of a motor vehicle by means of an electronic computing device of an assistance system of the motor vehicle. At least a first image of the environment from at least a first perspective is received by means of the electronic computing device. Generating the perspective image depending on the first image is effected by means of a generative network of the electronic computing device, wherein topographic information determined by means of the electronic computing device from the at least one image is taken into account in the perspective image.

[0013] In particular to overcome the limits of the IPM and to improve the experience and safety of a user in the motor vehicle for example in parking as well as in driving, the generative network is in particular proposed, which generates a perspective view in relation to a certain translation and rotation parameter in real time in that it for example uses so-called surround view images, which have been captured by means of multiple capturing devices, such that a multimodal generation is advantageous for the coverage of near and distant perspectives or scenes.

[0014] The strength of the generative network is in particular in that a consistent and coherent representation of the environment can be generated. In particular, the generated images do not have distortions or misalignments. 2024PF00234 clean copy

[0015] 3

[0016] Thus, the invention in particular contributes to the development of innovative parking assistance systems with visually attractive display, which offers a very truthful and accurate representation of the real scene or environment, since there are no distortions or misalignments in the generated perspective images, in particular in contrast to the IPM case. In addition, the capturing range of the generated images is considerably increased, which allows a more extensive comprehension of the environment to a driver in connection with the usage-interactive controllability of the view.

[0017] In particular, to generate a view of a certain position and orientation, the generative network is used, which can generate the images attributable to the images of the environment and the translation and rotation parameters. Therein, a generative network can in particular be composed of two networks, namely in particular a generator and a discriminator, which are collectively trained in a manner. The generator learns to approximate the distribution of the training data, while the discriminator learns to distinguish between the generated data and the corresponding ground truth. In the image generation, generative networks have proven that they efficiently generate completely new patterns, which correlate with the distribution of the training data.

[0018] First, the electronic computing device in turn for example informs the driver about his environment with a perspective image, for example a bird's eye perspective image, which is generated using a predefined position and orientation and has a certain capturing range, for example 12.5 meters around the motor vehicle. Then, an adaptation of the represented perspective image can in turn be performed.

[0019] According to an advantageous form of configuration, at least two images, in particular four images, are received and used for generating the perspective image. For example, the electronic computing device can receive the corresponding four images from four cameras. Therein, a front camera, a first side camera, a rear camera as well as a second side camera can be provided. Based on the two respectively four received images, the perspective image, for example a bird's eye perspective image, can now be generated. By using the generative network, thus, topographic information, for example height information, can be used such that distortions do not occur within the perspective image. Thus, a perspective image of the environment can be generated in improved manner.

[0020] A further advantageous form of configuration provides that the at least two images are received from at least two different capturing devices. The different capturing devices can for example be different cameras of different locations at the motor vehicle. Further, the 2024PF00234 clean copy

[0021] 4 capturing devices can also be different in their type, one capturing device can for example be formed as a camera, the other capturing device as a LIDAR sensor and receive a corresponding image as a point cloud. An ultrasonic sensor device and a radar sensor device can also be used. Subsequently, a sensor fusion can occur to be able to generate the perspective image. Thus, a comprehensive capture of the environment and a corresponding perspective image can be generated. In a further advantageous form of configuration, it can be provided that a bird's eye perspective image is generated as the perspective image. The bird's eye perspective image is also referred to as so-called bird's eye view image (BEV). Thus, the motor vehicle can in particular for example be substantially centrally represented and the environment be represented in a top view around the motor vehicle. In particular, the bird's eye perspective has proven to be very advantageous for example in a parking assistant, since the vehicle driver can reliably capture the environment from the bird's eye perspective. This assists the driver for example in the parking operation to be able to observe the environment in improved manner.

[0022] Further, it has proven to be advantageous if the at least two capturing devices are provided at two different installation locations at the motor vehicle. Therein, a first capturing device can for example be formed in a front area of the motor vehicle and a second capturing device in a rear area of the motor vehicle. Further, capturing devices can also be correspondingly used in the lateral areas of the motor vehicle.

[0023] It is further advantageous if the generative network is provided as a generative adversarial network. It is in particular a network with a generator and a discriminator, as already described. In particular, the generative adversarial network has proven to be advantageous to merge corresponding images and to process corresponding topographic information. Thus, an independent aspect of the invention in particular also relates to a method for training a generative adversarial network, which can then in turn generate the corresponding bird's eye perspective images or perspective images.

[0024] In a further advantageous form of configuration, it can be provided that the perspective image is communicated to a display device of the assistance system for displaying. The display device can for example be provided as a central display device in the motor vehicle. Thus, the perspective image, for example in the form of a bird's eye perspective image, can be displayed on the display device. In particular in corresponding assistance systems, for example a parking assistant, thus, the perspective image can be reliably displayed to a vehicle driver of the motor vehicle. Thus, he can reliably observe the 2024PF00234 clean copy

[0025] 5 environment to be able to correspondingly observe the environment and intervene for example in an at least partially automated operated parking maneuver or fully assisted operated parking maneuver, if it should for example be required. Thus, a safe operation of the motor vehicle can be realized.

[0026] It has further proven to be advantageous if a view of the perspective image is adapted in automated manner depending on an operating mode of the motor vehicle. For example, in a parking maneuver as the operating mode, a corresponding adaptation can be performed. For example, the near range of the motor vehicle can then be correspondingly displayed since only low speeds and the near range are of interest in particular in the parking maneuver. If the motor vehicle should for example be in a driving mode, thus, the perspective image can in particular also be adapted in automated manner depending on speed. For example, at low speeds, a near range can be displayed, while larger ranges can also be represented at higher speeds and thus it can in particular be "zoomed out". Therein, the perspective image can additionally also be adapted depending on a location. For example, a corresponding wider range can be displayed on a highway. Furthermore, the front area is for example of greater interest than a rear area on the highway, such that the image can also be shifted such that a wider range is displayed in a front area than in a rear area. Thus, the perspective image is not centered to the motor vehicle. Thus, the perspective image can be reliably generated and displayed.

[0027] A further advantageous form of configuration provides that a view of the perspective image is adapted in automated manner depending on a speed of the motor vehicle. In particular, a speed signal of the motor vehicle can for example be communicated to the electronic computing device such that the electronic computing device can adapt the perspective image in automated manner. Therein, it can in particular be provided at higher speeds that a wider range is displayed than at correspondingly lower speeds. Thus, a perspective image can be generated and displayed in improved manner.

[0028] Further, it has proven to be advantageous if a view of the perspective image is adapted in automated manner depending on the object capture of an object in the environment. Hereto, an object recognition algorithm can in particular be provided to be able to recognize a corresponding object. For example, if an object should appear, thus, it can for example be zoomed or centered to the object such that a person is made aware of the fact that a new object has occurred in the environment. Thus, a safe operation of the motor vehicle can be realized. 2024PF00234 clean copy

[0029] 6

[0030] Therein, a further advantageous form of configuration provides that the view is adapted such that the perspective image is centered to a captured object. Thus, it can for example be decentered from the motor vehicle and centered to the newly captured object. Thus, the driver can be informed that a new object has been found in the environment, which is now represented in centered manner, whereby the driver can be made aware of the object in improved manner.

[0031] It is further advantageous if the captured object is optically highlighted in the perspective image. For example, the captured object can obtain a red color or be represented in flashing manner in the perspective image. Thus, the newly captured object can be conveyed to the vehicle driver in correspondingly improved manner such that he can for example react to it if a potential collision with the object should occur.

[0032] It has further proven to be advantageous if the perspective image is adapted depending on an input of a person. For example, the display device can be formed as a touch- sensitive display device. The person can then initiate a corresponding shift of the perspective image by an input on the touch-sensitive display device. Furthermore, zooming in and zooming out, respectively, can for example also be correspondingly performed by the input. A rotation of the image by the user can also be initiated. Thus, the perspective image can be adapted in user-specific or driver-specific manner.

[0033] The presented method is substantially a computer-implemented method. Therefore, a further aspect of the invention relates to a computer program product with program code means, which cause an electronic computing device, when the program code means are processed by the electronic computing device, to perform a method according to the preceding aspect.

[0034] Furthermore, the invention therefore also relates to a computer-readable storage medium with at least the computer program product according to the preceding aspect.

[0035] A still further aspect of the invention relates to an electronic computing device for generating a perspective image of an environment of a motor vehicle, wherein the electronic computing device is formed for performing a method according to the preceding aspect. In particular, the method is performed by means of the electronic computing device. 2024PF00234 clean copy

[0036] 7

[0037] Further, the invention also relates to an assistance system with an electronic computing device according to the preceding aspect. For example, the assistance system can be formed as a parking assistance system.

[0038] A still further aspect of the invention also relates to a motor vehicle with at least one assistance system according to the preceding aspect. Therein, the motor vehicle can be formed at least partially assisted or fully assisted.

[0039] Advantageous forms of configuration of the method are to be regarded as advantageous forms of configuration of the computer program product, of the computer-readable storage medium, of the electronic computing device, of the assistance system as well as of the motor vehicle. Hereto, the electronic computing device, the assistance system as well as the motor vehicle comprise concrete features to be able to perform corresponding method steps.

[0040] Here, and in the following, an artificial neural network can be understood as a software code, which is stored on a computer-readable storage medium and represents one or more linked artificial neurons or can emulate their function. Therein, the software code can also include multiple software code components, which can for example have different functions. In particular, an artificial neural network can implement a non-linear model or a non-linear algorithm, which maps an input to an output, wherein the input is given by an input feature vector or an input sequence and the output can for example include an output category for a classification task, one or more predicated values or a predicated sequence.

[0041] Within the scope of the present disclosure, an object recognition algorithm can be understood as a computer algorithm, which is capable of identifying and localizing one or more objects within a provided input dataset, for example input image, for example in that it determines corresponding bounding boxes or regions of interest (ROI) and in particular associates a corresponding object class with each of the bounding boxes, wherein the object classes can be selected from a predefined set of object classes. Therein, the assignment of an object class to a bounding box can be understood such that a corresponding confidence value or a probability that the object identified within the bounding box belongs to the corresponding object class, is provided. For example, the algorithm can provide such a confidence value or a probability for each of the object classes for a given bounding box. The assignment of the object class can for example include the selection or provision of the object class with the greatest confidence value or 2024PF00234 clean copy

[0042] 8 the greatest probability. Alternatively, the algorithm can also only determine the bounding boxes without associating a corresponding object class.

[0043] In the present disclosure, a computing unit / electronic computing device can for example be understood as a data processing apparatus with processing circuits. Thus, a computing unit can perform computing operations to process data. The computing operations can also include indexed accesses to a data structure, for example a look-up table, LUT.

[0044] In particular, a computing unit can include one or more computers, one or more microcontrollers and / or one or more integrated circuits, for example one or more application-specific integrated circuits, ASIC, one or more field-programmable gate arrays, FPGA, and / or one or more systems on a chip, SoC. The computing unit can also include one or more processors, for example one or more microprocessors, one or more central processing units, CPU, one or more graphics processing units, GPU, and / or one or more signal processors, in particular one or more digital signal processors, DSP. The computing unit can also include a physical or virtual cluster of computers or others of the mentioned units.

[0045] A computing unit can also include one or more hardware and / or software interfaces and / or one or more storage units. Therein, a storage unit can be designed as a volatile data memory, for example as a dynamic random access memory, DRAM, or static random access memory, SRAM, or as a non-volatile data memory, for example as a read-only memory, ROM, as a programmable read-only memory, PROM, as an erasable programmable read-only memory, EPROM, as an electrically erasable programmable read-only memory, EEPROM, as a flash memory or flash EEPROM, as a ferroelectric random access memory, FRAM, as a magnetoresistive random access memory, MRAM, or as a phase-change random access memory, PCRAM.

[0046] Further features of the invention are apparent from the claims, the figures and the description of figures. The features and feature combinations mentioned above in the description as well as the features and feature combinations mentioned below in the description of figures and / or shown in the figures alone are usable not only in the respectively specified combination, but also in other combinations without departing from the scope of the invention. Thus, implementations are also to be considered as encompassed and disclosed by the invention, which are not explicitly shown in the figures and explained, but arise from and can be generated by separated feature combinations 2024PF00234 clean copy

[0047] 9 from the explained implementations. Implementations and feature combinations are also to be considered as disclosed, which thus do not comprise all of the features of an originally formulated independent claim. Moreover, implementations and feature combinations are to be considered as disclosed, in particular by the implementations set out above, which extend beyond or deviate from the feature combinations set out in the relations of the claims.

[0048] There show:

[0049] Fig. 1 a schematic top view to an embodiment of a motor vehicle;

[0050] Fig. 2 a schematic flow diagram according to an embodiment of a method;

[0051] Fig. 3 a schematic perspective image generated by an embodiment of the method;

[0052] Fig. 4 a schematic block diagram according to an embodiment of a generative network;

[0053] Fig. 5 a further schematic perspective image generated by an embodiment of the method;

[0054] Fig. 6 a still further schematic perspective image generated by an embodiment of the method; and

[0055] Fig. 7 another schematic perspective view generated with an embodiment of the method according to Fig. 6 from a different perspective.

[0056] In the figures, identical or functionally identical elements are provided with the same reference characters.

[0057] Fig. 1 shows a schematic top view to an embodiment of a motor vehicle 1 . Therein, the motor vehicle 1 is formed at least partially assisted or fully assisted. The motor vehicle 1 comprises an assistance system 2. The assistance system 2 can for example correspondingly act on a longitudinal acceleration device 3 and lateral acceleration 2024PF00234 clean copy

[0058] 10 device, respectively. In particular, corresponding control signals can be communicated from the assistance system 2 to the longitudinal acceleration device 3 and to the lateral acceleration device, respectively. Further, the assistance system 2 comprises at least one electronic computing device 4. The electronic computing device 4 comprises at least one generative network 5.

[0059] Further, Fig. 1 shows that the motor vehicle 1 comprises at least two capturing devices 6, 7, 8, 9. In the following embodiment, the motor vehicle 1 comprises at least four capturing devices 6, 7, 8, 9. In particular, a first capturing device 6, a second capturing device 7, a third capturing device 8 as well as a fourth capturing device 9 are shown. The first capturing device 6 can for example be formed as a front camera. The second capturing device 7 can be formed as a side camera. The third capturing device 8 can be formed as a rear camera. The fourth capturing device 9 can be formed as a side camera.

[0060] According to an embodiment of the method, it can in particular be provided that at least a first image of an environment 1 1 of the motor vehicle 1 from at least a first perspective is received by means of the electronic computing device 4 and optional at least a second image of the environment 11 from at least a second perspective different from the first perspective is received by means of the electronic computing device 4 for generating a perspective image 10 (Fig. 3). Then, generating the perspective image 10 depending on the first image and optional the second image is effected by means of the generative network 5 of the electronic computing device 4, wherein topographic information H, for example a height information, determined by means of the electronic computing device 4 from the at least one image is taken into account in the perspective image 10.

[0061] Thus, it can in particular be provided that at least two or preferably four images are received and used for generating the perspective image 10. Therein, at least two images can in particular be received from at least two different capturing devices 6, 7, 8, 9. Therein, the at least two capturing devices 6, 7, 8, 9 can in particular be provided on at least two different installation locations at the motor vehicle 1 .

[0062] Fig. 2 in turn shows a schematic flow diagram according to an embodiment of the method. In a first step S1 , the start of the method is effected. In a second step S2, receiving the captured images is effected. In an alternative third step S3, it can be provided that additional information, for example a speed of the motor vehicle 1 , the position of the motor vehicle 1 as well as a traffic situation, is correspondingly received. In a fourth step S4, an initial perspective image 10 is generated. In a fifth step S5, this image can then be 2024PF00234 clean copy

[0063] 11 correspondingly displayed on a display device 12 (Fig. 1 ) of the assistance system 2. In a sixth step S6, dynamic or manual changes can be adapted in the view, for example the orientation, and / or the position as well as corresponding objects 13 (Fig. 3) can be correspondingly detected. In a seventh step S7, it can be determined if a corresponding viewing angle has changed. If this should not be the case, thus, it is transitioned into the fourth step S4. If the viewing angle should be changed, thus, it is transitioned into an eighth step S8, which displays an adapted view of the perspective image 10. Starting from the eighth step S8, it can then again be transitioned into the fifth step S5.

[0064] Fig. 3 in turn shows a corresponding perspective image 10 of the motor vehicle 1 . In particular, a so-called bird's eye perspective image is shown. In particular, Fig. 3 shows that a view of the perspective image 10 is adapted for example depending on an operating mode of the motor vehicle 1 . For example, the view of the perspective image 10 can be adapted in automated manner depending on an object capture of the object 13 in the environment 11 . Further, the view can be adapted such that the perspective image 10 is centered to a captured object 13. Furthermore, the captured object 13 can be optically highlighted in the perspective image 10.

[0065] Fig. 4 in turn shows a schematic block diagram according to an embodiment of a generative network 5, in particular in the form of configuration of an adversarial network. Therein, the generative network 5 in particular comprises a generator 14 as well as a discriminator 15.

[0066] Fig. 5 and Fig. 6 in turn show different perspective images 10. In Fig. 5, it is in particular shown that the perspective image 10 for example has been correspondingly shifted to the right. For example, this can have been manually performed by a vehicle driver or user of the motor vehicle 1 . Fig. 6 shows substantially the same image as Fig. 5, but the motor vehicle 1 is further centered in Fig. 6, but was zoomed out of the image.

[0067] Fig. 7 shows another schematic perspective view generated with an embodiment of the method according to Fig. 6 from a different perspective. Fig. 7 shows that, in addition to the bird's eye view, a lateral perspective of the motor vehicle 1 can also be generated.

[0068] Thus, the Figs, in particular show that visually attractive displays can be represented for example for a parking assistance system as the assistance system 2, which offer a very truthful and accurate representation of the real environment 11 , since distortions or misalignments in particular do not occur in the generated perspective images 10. In 2024PF00234 clean copy

[0069] 12 addition, the capturing range of the generated perspective image 10 is correspondingly larger, which allows a more extensive comprehension of the environment 1 1 to a driver in connection with for example the usage-interactive controllability of the view.

[0070] Moreover, the electronic computing device 4 can assist the driver since the displayed view focuses on the representation of changes in the environment 1 1 in real time and warns him of changes, which could compromise his safety. For example, if an object 13 unexpectedly manifests, the view is centered around the object 13 such that the driver can timely bypass it. Another example relates to the change of the view depending on the speed of the motor vehicle 1 . If the driver for example wants to have a wider view to the environment 11 on highways, while he is in a jam, thus, the view can focus on the motor vehicle 1 .

[0071] Parking of the motor vehicle 1 can be regarded as a further example, in which the user has the possibility of changing the view of the central cluster, in particular of the bird's eye perspective image, to focus on the distance between wheel and curb. Therein, the user has the possibility of adjusting the view as it results from the view of another person, who looks at this area.

[0072] In particular to generate a view of a certain position and orientation, the generative network 5 is trained and generated, which establishes the position and orientation by images of the environment 11 and conditioned by translation and rotation parameters. Herein, a similarity to the use of conditioned GAN is in particular provided.

[0073] Therein, a generative adversarial network GAN is in particular composed of two networks. In particular of the generator 14 and the discriminator 15, which are collectively trained in an adversarial manner. The generator 14 learns to approximate the distribution of the training data, while the discriminator 15 learns to distinguish between the generated data and the corresponding ground truth. In the image generation, GANs have proven that they efficiently generate completely new patterns, which correlate with the distribution of the training data.

[0074] First, the electronic computing device 4 for example informs the user about his environment 11 with a bird's eye perspective image, which is generated using a predefined position and orientation and is generated in a certain capturing range, for example 12.5 meters around the motor vehicle 1 . 2024PF00234 clean copy

[0075] 13

[0076] Therein, two possibilities are in particular proposed, which cause the capturing device 4 to perform a view adaptation.

[0077] On the one hand, a user can change the position and / or the orientation of the currently displayed view on the screen, for example via a touchpad, which comprises corresponding arrow buttons on the screen, voice control, gaze control or the like.

[0078] Further, the module can perform an adaptation of the view in automated manner in a second possibility, for example if a shift in the environment 11 should be identified, which would result in a poor visualization of the environment 11 with the currently displayed view. Therein, the following shift can be taken into account. For example, the speed of the motor vehicle 1 can change. If the motor vehicle 1 becomes slower, the field of view can for example be correspondingly reduced, since only peripheral objects 13 have to be identified in the scene, in that the corresponding view position changes along the Z-axis (vertical axis). At corresponding higher speeds, zooming out can in particular be realized. Furthermore, for example if a speed bump is discovered as the object 13, thus, the view can be adapted such that it is displayed around the speed bump or the speed bump is highlighted. Upon appearance of sudden objects 13 or other obstacles, the current view can be adapted in relation to the visible areas, and the electronic computing device 4 for example sends visual or acoustic signals to the user to increase his attention. Thus, the bounding lines of the closest obstacles can for example be displayed or highlighted as it is in particular for example illustrated in Fig. 3.

[0079] For example, if a change in the environment 11 should be determined at the same time as the action of the user, however, the changes initiated by the user in particular thus have priority.

[0080] The transformations of the currently displayed view can be described as follows. The left- to-right and right-to-left transformation is performed with a shift along the Y-axis. The forward- rearward transformation is performed with a shift along the X-axis. The enlargement and reduction are effected by the shift along the Z-axis. The rotation is performed along all of the axes. The transformations can be formulated with the aid of a projection matrix. According to current view, the transformation can in particular be limited to a certain area. For example, a boundary can be defined from the depth estimation with a LIDAR sensor, if not present, the surround view images can be defined with computer vision tools in order that the image generator does not generate non-existing scenes. 2024PF00234 clean copy

[0081] 14

[0082] In particular to for example perform the shift due to suddenly appearing objects 13, an object recognition module can in turn be used, which localizes the scene of present objects and indicates to the module for adapting the viewing angle if there are obstacles or interfering objects 13.

Claims

2024PF00234 clean copyClaims1 . A method for generating a perspective image (10) of an environment (11 ) of a motor vehicle (1 ) by means of an electronic computing device (4) of an assistance system (2) of the motor vehicle (1 ), comprising the steps:- receiving at least a first image of the environment (11) from at least a first perspective by means of the electronic computing device (4); and- generating the perspective image (10) depending on the first image by means of a generative network (5) of the electronic computing device (4), wherein topographic information (H) determined by means of the electronic computing device (4) from the at least one image is taken into account in the perspective image (10).

2. The method according to claim 1 , characterized in that at least two images, in particular four images, are received and used for generating the perspective image (10) and / or a bird's eye perspective image is generated as the perspective image (10).

3. The method according to claim 2, characterized in that the at least two images are received from at least two different capturing devices (6, 7, 8, 9).

4. The method according to claim 3, characterized in that the at least two capturing devices (6, 7, 8, 9) are provided at two different installation locations at the motor vehicle (1 ).

5. The method according to any one of the preceding claims, characterized in that the generative network (5) is provided as a generative adversarial network.2024PF00234 clean copy166. The method according to any one of the preceding claims, characterized in that the perspective image (10) is communicated to a display device (12) of the assistance system (2) for displaying.

7. The method according to claim 6, characterized in that a view of the perspective image (10) is adapted in automated manner depending on an operating mode of the motor vehicle (1 ).

8. The method according to claim 6 or 7, characterized in that a view of the perspective image (10) is adapted in automated manner depending on a speed of the motor vehicle (1 ).

9. The method according to claim 6 to 8, characterized in that a view of the perspective image (10) is adapted in automated manner depending on an object capture of an object (13) in the environment (1 1 ).

10. The method according to claim 9, characterized in that the view is adapted such that the perspective image (10) is centered to a captured object (13).11 . The method according to claim 9 or 10, characterized in that the captured object (13) is optically highlighted in the perspective image (10).

12. The method according to any one of claims 6 to 12, characterized in that the perspective image (10) is adapted depending on an input of a person.2024PF00234 clean copy1713. A computer program product with program code means, which cause an electronic computing device (4), when the program code means are processed by the electronic computing device (4), to perform a method according to any one of claims 1 to 12.

14. A computer-readable storage medium with at least a computer program product according to claim 13.

15. An electronic computing device (4) for generating a perspective image (10) of an environment (11 ) of a motor vehicle (1 ), wherein the electronic computing device (4) is formed for performing a method according to any one of claims 1 to 12.

Images