Display of image data in vehicle with varying projection surface

Abstract

To display image data, initial and final projection surfaces (9a, 9b) are provided, and a video stream is received and buffered in a buffer memory that is updated at a frame rate. The image data is read out from the buffer at a read-out rate at the initial read-out instance, the intermediate read-out instance, and the final read-out instance. A respective set of initial and final sampling points (11a, 11b) on the projection surface (9a, 9b) are determined. For each intermediate readout instance, intermediate sampling points (12, 13) interpolated between the final sampling point (11b) and the corresponding initial sampling point (11a) and an intermediate projection surface (10) containing the intermediate sampling points (12, 13) are determined. The read-out image data is projected onto the respective intermediate projection surface (10) and the respective image is displayed on the display device (5).

Description

Technical Field

[0001] The invention relates to a computer-implemented method for displaying image data in a vehicle, an electronic vehicle navigation system for a vehicle, a computer program and a computer-readable storage medium. Background Art

[0002] The vehicle's camera system (which may include one or more cameras arranged at different locations of the vehicle, such as a vehicle surround view system) can be used for driver assistance functions or other functions for autonomous or semi-autonomous driving. Camera images or multiple camera images from different cameras (these images are stitched into a combined view) can be projected onto a predefined projection surface, which is a part of a two-dimensional manifold in three-dimensional space, such as a bowl shape, etc. In addition, the projected image or multiple projected images can be transformed according to the corresponding observation parameters of a virtual observer (also represented as a virtual camera) so that the images appear as if they have been observed by the virtual observer or captured by the virtual camera, respectively. The position and / or orientation of the virtual observer can be set or modified, for example, by a user or automatically set or modified by the vehicle.

[0003] In the application note “360° surround video imaging technology is ready for integration with Fujitsu graphics SoCs” (retrieved on September 12, 2022 at URL: https: / / www.fujitsu.com / us / imagesgig5 / 360_OmniView_AppNote.pdf), a video imaging technology capable of real-time 360° surround viewing of the vehicle’s surroundings is described.

[0004] Document DE 10 2015 105 529 A1 relates to a method for transforming an image which represents an area around a motor vehicle from the perspective of a virtual camera. The image is represented by a transformation from a plurality of real images which were generated with the aid of a plurality of real cameras of the motor vehicle.

[0005] Document US 2021 / 0125401 A1 relates to a method for representing an environment of a motor vehicle in an image, wherein real images of the environment are captured by a real camera of the motor vehicle and the image is generated from these real images. The image is represented from the perspective of a virtual camera in the environment and the image is generated in the shape of a bowl.

[0006] In case the position and / or orientation of the virtual observer is changed while displaying the video stream, the shape of the projection surface may also change. The change may be performed as a hard switch, i.e. in particular the projection surface changes from an initial shape to a final shape without transition. However, the change of the projection surface is not necessarily associated with a change in the position and / or orientation of the virtual observer, but may be caused, for example, by a change in user settings or the like. Summary of the invention

[0007] In view of this, it is an object of the present invention to provide the possibility of achieving a smooth transition from one projection surface to another.

[0008] This object is achieved by the respective subject matter of the independent claims. Further embodiments and preferred embodiments are subject matter of the dependent claims.

[0009] The invention is based on the idea of ​​sampling an initial projection surface and a final projection surface and determining a corresponding intermediate projection surface for each of a plurality of consecutive intermediate readout instances of a buffer memory for reading out a buffered video stream. Each intermediate projection surface comprises intermediate sampling points interpolated between sampling points on the final projection surface and corresponding sampling points on the initial projection surface.

[0010] According to one aspect of the invention, a computer-implemented method for displaying image data in a vehicle, in particular a motor vehicle, is provided. Therein, a predefined initial projection surface and a predefined final projection surface are provided. A video stream depicting the environment of the vehicle, in particular the external environment, is received from a camera system of the vehicle and the video stream is buffered in a buffer memory, wherein the buffer memory is updated at a predefined frame rate. Image data, in particular updated image data, are read out from the buffer memory at a predefined readout rate. Therein, the readout occurs at an initial readout instance, at a plurality of consecutive intermediate readout instances after the initial readout instance, and at a final readout instance after a plurality of intermediate readout instances. A set of initial sampling points on the initial projection surface and a set of final sampling points on the final projection surface are determined. Therein, each final sampling point in the set of final sampling points is assigned to exactly one initial sampling point in the set of initial sampling points.

[0011] For each of the intermediate readout instances, a set of intermediate sampling points is determined, wherein each intermediate sampling point in the set of intermediate sampling points is interpolated between one of the final sampling points and a corresponding assigned initial sampling point. For each of the intermediate readout instances, an intermediate projection surface containing the set of intermediate sampling points is determined. For each of the intermediate readout instances, the readout image data is projected onto the corresponding intermediate projection surface, and an intermediate image dependent on the projected image data is displayed on a display device of the vehicle.

[0012] Unless otherwise stated, all steps of the computer-implemented method may be performed by at least one computing unit, in particular of a vehicle, which may also be denoted as a data processing device. In particular, the at least one computing unit comprises at least one processing circuit, which is configured or adapted to perform the steps of the computer-implemented method. For this purpose, the at least one computing unit may, for example, store a computer program comprising instructions, which, when executed by the at least one computing unit, cause the at least one computing unit to perform the computer-implemented method. The step of displaying the corresponding intermediate image on a display device may be understood as causing the at least one computing unit to control the display device to display the corresponding intermediate image. Then, the display is not necessarily part of the at least one computing unit. Alternatively, the display may be part of the at least one computing unit.

[0013] A computing unit may be understood in particular as a data processing device, which comprises a processing circuit. Thus, a computing unit may in particular process data to perform computing operations. This may also include operations for performing index access to a data structure (eg a lookup table LUT).

[0014] In particular, the computing unit may include one or more computers, one or more microcontrollers and / or one or more integrated circuits, for example, one or more application specific integrated circuits ASIC, one or more field programmable gate arrays FPGA and / or one or more systems on chip SoC. The computing unit may also include one or more processors, for example, one or more microprocessors, one or more central processing units CPU, one or more graphics processing units GPU and / or one or more signal processors, in particular one or more digital signal processors DSP. The computing unit may also include a physical or virtual cluster of computers or other such units.

[0015] In various embodiments, a computing unit includes one or more hardware and / or software interfaces and / or one or more memory units.

[0016] The memory unit can be implemented as a volatile data memory, such as a dynamic random access memory DRAM or a static random access memory SRAM, or as a non-volatile data memory, such as a read-only memory ROM, a programmable read-only memory PROM, an erasable programmable read-only memory EPROM, an electrically erasable programmable read-only memory EEPROM, a flash memory or a flash EEPROM, a ferroelectric random access memory FRAM, a magnetoresistive random access memory MRAM, or a phase change random access memory PCRAM.

[0017] The initial projection surface and the final projection surface may be provided, for example, by storing them on a memory unit of the at least one computing unit, in particular by storing a corresponding mathematical description, a look-up table or another computer-readable data set.

[0018] The video stream comprises a plurality of consecutive frames, each of which comprises corresponding image data stored one by one in a buffer memory. In particular, when the corresponding image data of a frame is stored in the buffer memory, the corresponding image data of the previous frame can be deleted or overwritten. However, the buffer memory can also be a circular buffer. In any case, the latest image data stored in the buffer memory changes at a predefined frame rate and is read out at a predefined readout rate. Therefore, the inverse readout rate defines the time difference between the initial readout instance and the first intermediate readout instance, the time difference between each pair of consecutive intermediate readout instances, and the time difference between the last intermediate readout instance and the final readout instance. The frame does not necessarily constitute the entire video stream, but can be a part thereof.

[0019] For example, for an initial readout instance, the readout image data may be projected onto an initial projection surface, and an initial image depending on the projected image data is displayed on the display device, in particular before any intermediate images are displayed. Alternatively or additionally, for a final readout instance, the readout image data may be projected onto a final projection surface, and a final image depending on the projected image data is displayed on the display device, in particular after all intermediate images are displayed.

[0020] Specifically, the corresponding projection image data according to the intermediate readout instance can be directly displayed on the display device, or can be further processed and then displayed on the display device. In other words, the intermediate image can be given directly by the corresponding projection image data or by the further processed projection image data. The further processing can, for example, include filtering steps or processing steps for stitching together image data from different cameras, transforming image data according to the observation position and / or orientation of the virtual observer or other observation parameters, etc.

[0021] The projection surface can, for example, be considered as a two-dimensional manifold in a three-dimensional space. For example, the projection surface can be defined as Z=f(X,Y) in a Cartesian coordinate system with coordinates X, Y, Z by means of a function f. However, the projection surface may not necessarily be described by such a function. In particular, in some embodiments, the projection surface can also be defined in a segmented manner, with different functions for different parts of the three-dimensional space. The projection surface is defined in particular in a vehicle coordinate system, in which the position and orientation of the vehicle are known, in particular fixed. For example, a specified reference point of the vehicle can be located in the origin of the coordinate system.

[0022] The projection surface may be a portion of a two-dimensional manifold. Thus, different projection surfaces may correspond to different portions of the same two-dimensional manifold. Alternatively, different projection surfaces may also correspond to portions of different two-dimensional manifolds, for example defined by different functions f(X, Y).

[0023] Given the corresponding intermediate sampling points, intermediate projective surfaces can be constructed in various ways. For example, they can be given by the corresponding polygon meshes, where the intermediate sampling points represent the corners of the polygons. Intermediate projective surfaces can also be given by the corresponding spline surfaces, where the intermediate sampling points serve as control points. Intermediate projective surfaces can also be given by other surfaces interpolated between the intermediate sampling points.

[0024] The number of final sampling points may be less than or equal to the number of initial sampling points. The number of intermediate sampling points for a given intermediate readout instance is equal to the number of final sampling points. The intermediate projection surface describes a smooth transition from the initial projection surface to the final projection surface, and for each intermediate readout instance, a corresponding intermediate image is displayed. Thus, the resulting sequence of displayed images describes a smooth transition from the initial view to the final view. Thus, a user of a vehicle viewing the display device does not see a sudden change in the displayed image due to a sudden change in the projection surface.

[0025] In particular, all initial projection surfaces, intermediate projection surfaces and final projection surfaces may be different from each other. Thus, the displayed image changes for each of the readout instances.

[0026] Since a corresponding intermediate image is displayed for each intermediate readout instance, the readout rate is in particular equal to or less than the refresh rate of the display device, preferably equal to the refresh rate. It should be noted that the readout rate may be greater than the frame rate. In other words, the corresponding readout image data may remain unchanged for two or more consecutive intermediate readout instances. However, the displayed image typically still changes because the intermediate projection surface typically changes. However, in other embodiments, the readout rate may also be equal to or less than the frame rate, which means that the readout image data is different for each pair of subsequent intermediate readout instances.

[0027] According to several embodiments, predefined initial observation parameters of the virtual observer (including the initial position of the virtual observer) and predefined final observation parameters of the virtual observer (including the final position of the virtual observer) are provided, in particular stored on a memory unit of at least one computing unit. For each of the intermediate readout instances, intermediate observation parameters of the virtual observer including the corresponding intermediate position of the virtual observer are determined. Wherein the intermediate position is located on a predetermined curve connecting the initial position to the final position. For each of the intermediate readout instances, the readout image data are projected onto a corresponding intermediate projection surface and transformed according to the corresponding intermediate observation parameters. The corresponding intermediate image depends on (e.g. corresponds to) the projected and transformed image data.

[0028] In such an embodiment, the change of the projection surface from the initial projection surface via the intermediate projection surface to the final projection surface is accompanied by a change in the position of the virtual observer. In addition, the change in the position of the virtual observer occurs in a smooth transition. However, the dynamics of the change in the projection surface does not necessarily match the dynamics of the change in the position of the virtual observer. In other words, the position of the virtual observer can remain unchanged for two or more consecutive readout instances of the initial, intermediate and final readout instances.

[0029] In addition to the position of the virtual observer, the viewing parameters may for example include the orientation of the virtual observer. The viewing parameters may also include or define the field of view of the virtual observer, which is particularly also represented as a vehicle camera. The viewing parameters may also include virtual mapping parameters describing a mapping function of the virtual camera.

[0030] The virtual observer can be located at any position in the vehicle environment. Depending on the position of the virtual observer and / or other observation parameters, the size of the projection surface can vary. In particular, the image data displayed on the display device appear as if they have been captured by a virtual camera or viewed by a virtual observer, respectively.

[0031] The curve on which the intermediate position is located may be a straight line connecting the initial position to the final position or any other curve. A user viewing the displayed image has an impression as if the observation point is flying along the curve.

[0032] According to several embodiments, the initial viewing parameters include an initial orientation of the virtual observer and the final viewing parameters include a final orientation of the virtual observer.For each of the intermediate readout instances, the respective intermediate viewing parameters include a respective intermediate orientation of the virtual observer.

[0033] The orientation of the virtual observer may be given, for example, by three orientation angles (eg Euler angles). Thus, the intermediate orientations may be considered to lie on corresponding curves in the parameter space of orientation angles connecting the initial orientation to the final orientation.

[0034] According to several embodiments, the initial observation parameters include an initial field of view of the virtual observer, and the final observation parameters include a final field of view of the virtual observer.For each of the intermediate readout instances, the respective intermediate observation parameters include a respective intermediate field of view of the virtual observer.

[0035] The field of view of the virtual observer may be defined, for example, by two field of view angles, such as a horizontal opening angle and a vertical opening angle. Thus, the intermediate field of view may be considered to be located on a corresponding curve in the parameter space of field of view angles connecting the initial field of view to the final field of view.

[0036] According to several embodiments, for each of the final sampling points, intermediate sampling points interpolated between the final sampling point and the assigned initial sampling point for all intermediate readout instances lie on a straight line connecting the final sampling point and the assigned initial sampling point.

[0037] In this way, the shortest connection between the corresponding final and initial points is used for interpolation. As a result, smooth transitions may require fewer intermediate projection surfaces.

[0038] According to several embodiments, for each of the final sampling points, the corresponding distances between intermediate sampling points interpolated between the final sampling point and the assigned initial sampling point are the same for each pair of consecutive intermediate readout instances.

[0039] In particular, the distance between the initial sampling point and the first intermediate sampling point and the distance between the final sampling point and the last intermediate sampling point are also equal to the distances between consecutive pairs of intermediate sampling points.

[0040] In other words, the transition is performed in a linear manner, which reduces the computational effort for calculating the intermediate projection surface.

[0041] According to several embodiments, the readout rate is greater than the frame rate.

[0042] Thus, for at least two consecutive intermediate readout instances, the same image data may be read out from the buffer memory. Thus, for an observing user, the transition appears smoother.

[0043] Preferably, the readout rate is at least twice the frame rate, such as at least five times the frame rate.For example, the ratio of the readout rate to the frame rate may lie in the interval [5, 20].

[0044] Thus, for each camera frame, the same buffered image data is read out at least twice or at least five times or at least the number given by the ratio at respective consecutive intermediate readout instances. Thus, for an observing user, the transition appears smoother.

[0045] In some embodiments, the total number of intermediate readout instances can be adjusted to the ratio so that the initial readout instance, all intermediate readout instances, and the final readout instance occur during a single frame, and the image data read out by each is the same. Thus, the entire transition can occur during a single frame, which makes the transition appear even smoother. However, the total number of intermediate readout instances can also be adjusted to the ratio so that the initial readout instance, all intermediate readout instances, and the final readout instance occur during a predefined maximum number of frames.

[0046] According to several embodiments, the initial projection surface is given by the points satisfying the following equation:

[0047]

[0048] In particular, the initial sampling points satisfy the equation: Where X, Y, and Z represent the Cartesian coordinates of the corresponding points on the initial projection surface, Z corresponds to the height above the predefined ground plane on which the vehicle is located, n is an even integer equal to or greater than 4, and ai, bi, and c are predefined real coefficients.

[0049] In other words, the initial projection surface is given by a polynomial of degree n in two variables X, Y, preferably n=4. Thus, the shape of the initial projection surface can be represented as a bowl, having a relatively flat portion in the area where the vehicle is located, and rising relatively steeply further away from the vehicle. In particular, the vehicle is centered at the origin of the coordinate system, and the XY plane corresponds to the ground plane. This represents a suitable approximation to the distance of the depicted object.

[0050] For n=4, the above equation for Z can also be rewritten as

[0051]

[0052] In some embodiments, the same applies to the final projection surface. In other words, in such embodiments, the final projection surface is given by a point satisfying the following equation:

[0053]

[0054] In particular, the final sampling point satisfies the equation: wherein X, Y and Z represent the Cartesian coordinates of the corresponding point on the final projection surface, Z corresponds to the height above the ground plane, n' is an even integer equal to or greater than 4, and a'i, b'i and c'i are predefined real coefficients. Preferably, n'=n.

[0055] Note that the intermediate projection surface, and in particular the intermediate sampling points of a given intermediate projection surface, do not necessarily satisfy such equations of a polynomial in two variables.

[0056] According to several embodiments, the initial projection surface comprises an initial base and an initial raised portion, the initial base being given by an initial portion of the ground plane on which the vehicle is located, wherein the vehicle is located within the initial base, and the initial raised portion adjoining the initial base at its outer boundary. Alternatively or additionally, the final projection surface comprises a final base and a final raised portion, the final base being given by a final portion of the ground plane, wherein the vehicle is located within the final base, and the final raised portion adjoining the final base at its outer boundary.

[0057] In particular, the initial base and / or the final base portion is convex, or in other words, the initial base and / or the final base portion or the corresponding outer boundary is a convex geometric figure, in particular a convex polygon. In contrast to the corresponding base, a point on the corresponding convex portion has a non-zero height above the ground plane.

[0058] In some implementations, the configuration using bases and protrusions as described can also be used for intermediate raised surfaces.

[0059] For use cases or usage scenarios that may arise in the computer-implemented method and are not explicitly described herein, it may be provided that, according to the method, error messages and / or prompts for user feedback are output and / or default settings and / or predetermined initial states are set.

[0060] According to another aspect of the invention, an electronic vehicle navigation system for a vehicle is provided. The electronic vehicle navigation system comprises at least one computing unit, in particular for the vehicle, which stores a predefined initial projection surface and a predefined final projection surface, in particular on one or more memory units of the at least one computing unit. The at least one computing unit is configured to receive a video stream depicting the environment of the vehicle from a camera system of the vehicle and to buffer the video stream, for example in a buffer memory of the at least one computing unit, wherein the buffer memory is updated at a predefined frame rate. The at least one computing unit is configured to read out image data from the buffer memory at a predefined readout rate at an initial readout instance, a plurality of consecutive intermediate readout instances and a final readout instance.

[0061] The at least one computing unit is configured to determine a set of initial sampling points on an initial projection surface and a set of final sampling points on a final projection surface, wherein each final sampling point is assigned to exactly one of the initial sampling points. The at least one computing unit is configured to determine a set of intermediate sampling points for each of the intermediate readout instances, wherein each intermediate sampling point is interpolated between one of the final sampling points and the corresponding assigned initial sampling point, and to determine an intermediate projection surface containing the set of intermediate sampling points. The at least one computing unit is configured to project the readout image data onto the corresponding intermediate projection surface for each of the intermediate readout instances, and to control a display device of the vehicle to display an intermediate image according to the projected image data.

[0062] An electronic vehicle guidance system may be understood as an electronic system that is configured to guide the vehicle in a fully automated or fully autonomous manner and, in particular, does not require manual intervention or control by the driver or user of the vehicle. The vehicle performs all required functions, such as steering maneuvers, deceleration maneuvers and / or acceleration maneuvers and automatically monitors and records road traffic and corresponding reactions. In particular, the electronic vehicle navigation system may implement a fully automated or fully autonomous driving mode according to level 5 of the SAE J3016 classification. The electronic vehicle navigation system may also be implemented as an advanced driver assistance system ADAS that assists the driver in partially automated or partially autonomous driving. In particular, the electronic vehicle navigation system may implement a partially automated or partially autonomous driving mode according to levels 1 to 4 of the SAE J3016 classification. Here and hereinafter, SAE J3016 refers to the corresponding standard dated June 2018.

[0063] Thus, at least partially automatically guiding the vehicle may include guiding the vehicle according to a fully automatic or fully autonomous driving mode according to level 5 of the SAE J3016 classification. At least partially automatically guiding the vehicle may also include guiding the vehicle according to a partially automatic or partially autonomous driving mode according to levels 1 to 4 of the SAE J3016 classification.

[0064] In some embodiments, the electronic vehicle navigation system includes a display device and / or a camera system.

[0065] According to several embodiments, the readout rate is equal to or less than a refresh rate of the display device, and / or preferably, the readout rate is greater than the frame rate, such as at least twice the frame rate, such as at least five times the frame rate.

[0066] Further implementations of the electronic vehicle navigation system according to the invention directly follow from the various embodiments of the computer-implemented method according to the invention, and vice versa. In particular, the individual features and corresponding explanations as well as the advantages associated with the various embodiments of the computer-implemented method according to the invention can be transferred analogously to the corresponding embodiments of the electronic vehicle navigation system according to the invention. In particular, the electronic vehicle navigation system according to the invention is designed or programmed to perform the computer-implemented method according to the invention. In particular, the electronic vehicle navigation system according to the invention performs the computer-implemented method according to the invention.

[0067] According to another aspect of the invention, there is provided a computer program product comprising instructions which, when executed by at least one computing unit, for example by the at least one computing unit of the electronic vehicle guidance system according to the invention, cause the at least one computing unit to perform a computer-implemented method according to the invention.

[0068] According to a further aspect of the present invention, there is provided a computer-readable storage medium storing a computer program according to the present invention.

[0069] Other features of the invention are apparent from the claims, the drawings and the description of the drawings. The features and feature combinations mentioned above in the specification and the features and feature combinations mentioned below in the description of the drawings and / or shown in the drawings may be included by the invention not only in the respective combinations described, but also in other combinations. In particular, embodiments and feature combinations that do not have all the features of the initially formulated claims may also be included in the invention. In addition, the invention may include embodiments and feature combinations that exceed or deviate from the combinations of features set forth in the description of the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In the following, the present invention will be explained in detail with reference to specific exemplary embodiments and corresponding schematic diagrams. In the accompanying drawings, identical or functionally identical elements may be represented by the same reference numerals. The description of identical or functionally identical elements is not necessarily repeated with respect to different drawings.

[0071] In the accompanying drawings,

[0072] Figure 1 schematically illustrates a vehicle having an exemplary embodiment of an electronic vehicle navigation system according to the present invention;

[0073] Figure 2 An example of a projection surface is schematically shown;

[0074] Figure 3 Another example of a projection surface is schematically shown;

[0075] Figure 4 Schematically illustrating exemplary positions of a vehicle and a virtual observer in a top view;

[0076] Figure 5 An exemplary position of a vehicle and a virtual observer is schematically shown in a side view;

[0077] Figure 6 schematically illustrates the transformation of a projection surface in an exemplary embodiment of a computer-implemented method according to the present invention;

[0078] Figure 7 Other exemplary positions of the vehicle and the virtual observer are schematically shown in side view;

[0079] Figure 8 A flowchart showing another exemplary implementation of the computer-implemented method according to the present invention; and

[0080] Fig. 9 A flow chart showing another exemplary implementation of the computer-implemented method according to the present invention is shown. DETAILED DESCRIPTION

[0081] Figure 1 Schematically, a vehicle 1 is shown with an exemplary embodiment of an electronic vehicle navigation system 2 according to the invention. The electronic vehicle navigation system 2 comprises a computing unit 3, which in some embodiments may also represent two or more computing units. The vehicle 1, in particular the electronic vehicle navigation system 2, comprises a camera 4, for example a front camera, a rear camera or a side camera, and a display device 5.

[0082] The camera 4 is configured to generate a video stream depicting the environment of the vehicle 1 and provide it to the computing unit 3. The computing unit 3 is configured to control the display device 5 to display images according to the video stream. To this end, the computing unit 3 can perform a computer-implemented method for displaying image data in the vehicle 1 according to the present invention.

[0083] The calculation unit 3 stores a predefined initial projection surface 9a and a predefined final projection surface 9b (see Figure 6 ). Figure 2 An example of the shape of the projection surface 6a according to a bowl shape is shown, and Figure 3 Another example is shown, in which the projection surface 6b consists of adjacent planes.

[0084] The computing unit 3 buffers the video stream in a buffer memory, wherein the buffer memory is updated at a predefined frame rate. Note that in some embodiments, parts of the computing unit 3, such as for example the buffer memory, may also be included in the camera 4, in particular since the computing unit 3 may represent two or more computing units. Alternatively, the buffer memory may be a part of the computing unit 3 arranged outside the camera 4.

[0085] The computing unit 3 reads out the image data from the buffer memory at a predefined readout rate. In particular, the computing unit 3 reads out the corresponding image data at an initial readout instance, a plurality of consecutive intermediate readout instances, and a final readout instance. The computing unit 3 samples the initial projection surface 9a to determine a set of initial sampling points 11a on the initial projection surface 9a, and samples the final projection surface 9b to determine a set of final sampling points 11b on the final projection surface 9b. Wherein, each final sampling point 11b is assigned to exactly one of the initial sampling points 11a. For each of the intermediate readout instances, a set of intermediate sampling points 12, 13 is determined, wherein each intermediate sampling point 12, 13 is interpolated between one of the final sampling points 11b and the corresponding assigned initial sampling point 11a, and an intermediate projection surface 10 containing the set of intermediate sampling points 12, 13 is determined. For each of the intermediate readout instances, the computing unit 3 projects the respectively readout image data to the corresponding intermediate projection surface 10, and controls the display device 5 to display the intermediate image according to the projected image data.

[0086] In some embodiments, the computing unit 3 may store virtual observers 7a, 7b, 7c, 7d, 7e, 7f, 7g, 7h, 7i (see Figure 4 ), which includes the initial position of the virtual observer 7a, 7b, 7c, 7d, 7e, 7f, 7g, 7h, 7i, and predefined final observation parameters of the virtual observer 7a, 7b, 7c, 7d, 7e, 7f, 7g, 7h, 7i, which includes the final position of the virtual observer 7a, 7b, 7c, 7d, 7e, 7f, 7g, 7h, 7i. For each of the intermediate readout instances, the calculation unit 3 determines the intermediate observation parameters of the virtual observer 7a, 7b, 7c, 7d, 7e, 7f, 7g, 7h, 7i including the corresponding intermediate position of the virtual observer 7a, 7b, 7c, 7d, 7e, 7f, 7g, 7h, 7i, wherein the intermediate position is located on a predetermined curve connecting the initial position to the final position. For each of the intermediate readout instances, a corresponding intermediate image can be given by the projected and transformed image data.

[0087] like Figure 4As shown, the virtual observers 7a, 7b, 7c, 7d, 7e, 7f, 7g, 7h, 7i, which can also be represented as virtual cameras, can be located at any position, in particular around the vehicle 1, including the position of the real camera 4. The position of the virtual observers 7a, 7b, 7c, 7d, 7e, 7f, 7g, 7h, 7i can be, for example, located in a distance range of 1 to 3 meters from the vehicle 1. When the position of the virtual observers 7a, 7b, 7c, 7d, 7e, 7f, 7g, 7h, 7i changes, in particular from an initial position via an intermediate position to a final position, this can also be represented as the virtual observers 7a, 7b, 7c, 7d, 7e, 7f, 7g, 7h, 7i flying from one position to another.

[0088] Figure 5 The vehicle 1 and one of the positions of a virtual observer 7e and the corresponding projection surface 8a are shown in a side view. Figure 7 Additionally shown is another position of the virtual observer 7i, which corresponds to the position of the camera 4, and the corresponding projection surface 8c. For example, the initial position may be the position of the virtual observer 7i, and the final position may be the position of the virtual observer 7e. Alternatively, the position of the virtual observer 7i may correspond to one of the intermediate positions.

[0089] Figure 8 A flow chart of another exemplary implementation of a computer-implemented method according to the present invention is shown. In step 800, a user may generate a view switch request. In step 810, a virtual observer 7a, 7b, 7c, 7d, 7e, 7f, 7g, 7h, 7i flies from an initial position via an intermediate position and arrives at a final intermediate position in step 820. In step 830, an initial projection surface 9a is sampled, and in step 840, a final projection surface 9b is sampled, as described above. In step 850, the initial projection surface 9a is changed via an intermediate projection surface 10 as described, and arrives at the final projection surface 9b in step 860. In addition, in step 860, the virtual observer 7a, 7b, 7c, 7d, 7e, 7f, 7g, 7h, 7i flies from the final intermediate position to the final position. In an optional step 870, the orientation of the virtual observer 7a, 7b, 7c, 7d, 7e, 7f, 7g, 7h, 7i is adjusted. The described sequence can be understood so that each time the position of the projection surface and / or the virtual observer 7a, 7b, 7c, 7d, 7e, 7f, 7g, 7h, 7i changes, this corresponds to a readout instance and a corresponding image is displayed by the display device 5.

[0090] Fig. 9A high-level flow chart of another exemplary implementation of a computer-implemented method according to the present invention is shown, wherein the readout rate is at least twice the frame rate. In step 900, the buffer memory is updated. In step 920, the image data is read out at the initial readout instance. In step 920, the projection surface is set to the initial projection surface 9a, the readout image data is projected onto the initial projection surface 9a as described and the initial image is displayed. In step 930, it is checked whether the buffer memory has been updated again. If this is the case, the updated image data is read out in step 900 in the first intermediate readout instance, the projection surface is changed to the first intermediate projection surface 10 in step 910, and the first intermediate image is displayed in step 920. Otherwise, the previously readout image data is obtained at the first intermediate readout instance, the projection surface is changed to the first intermediate projection surface 10 in step 910, and the first intermediate image is displayed in step 920. These steps are repeated until the final image is displayed.

Claims

1. A computer-implemented method for displaying image data in a vehicle (1), wherein: - providing a predefined initial projection surface (9a) and a predefined final projection surface (9b); - receiving a video stream depicting the environment of the vehicle (1) from a camera system (4) of the vehicle (1), and the video stream is buffered in a buffer memory, wherein the buffer memory is updated at a predefined frame rate; - reading out image data from said buffer memory at a predetermined readout rate at an initial readout instance, a plurality of consecutive intermediate readout instances and a final readout instance; - determining a set of initial sampling points (11a) on the initial projection surface (9a) and a set of final sampling points (11b) on the final projection surface (9b), wherein each final sampling point (11b) is assigned to exactly one of the initial sampling points (11a); - for each of the intermediate readout instances, determining a set of intermediate sampling points (12, 13), wherein each intermediate sampling point (12, 13) is interpolated between one of the final sampling points (11b) and the corresponding assigned initial sampling point (11a), and determining an intermediate projection surface (10) comprising the set of intermediate sampling points (12, 13); For each intermediate readout instance, the readout image data are projected onto a corresponding intermediate projection surface (10) and an intermediate image depending on the projected image data is displayed on a display device (5) of the vehicle (1).

2. The computer-implemented method of claim 1, It is characterized in that - providing predetermined initial observation parameters of a virtual observer (7a, 7b, 7c, 7d, 7e, 7f, 7g, 7h, 7i) including an initial position of the virtual observer (7a, 7b, 7c, 7d, 7e, 7f, 7g, 7h, 7i) and predetermined final observation parameters of a virtual observer (7a, 7b, 7c, 7d, 7e, 7f, 7g, 7h, 7i) including a final position of the virtual observer (7a, 7b, 7c, 7d, 7e, 7f, 7g, 7h, 7i); - determining, for each of said intermediate readout instances, intermediate observation parameters of said virtual observer (7a, 7b, 7c, 7d, 7e, 7f, 7g, 7h, 7i) comprising a respective intermediate position of said virtual observer (7a, 7b, 7c, 7d, 7e, 7f, 7g, 7h, 7i), wherein said intermediate position is located on a predefined curve connecting said initial position to said final position; and - For each intermediate readout instance, the readout image data are projected onto a corresponding intermediate projection surface (10) and transformed according to corresponding intermediate observation parameters, and the corresponding intermediate image depends on the projected and transformed image data.

3. The computer-implemented method of claim 2, It is characterized in that - the initial observation parameters comprise an initial orientation of the virtual observer (7a, 7b, 7c, 7d, 7e, 7f, 7g, 7h, 7i), and the final observation parameters comprise a final orientation of the virtual observer (7a, 7b, 7c, 7d, 7e, 7f, 7g, 7h, 7i), and for each of the intermediate readout instances, the respective intermediate observation parameters comprise a respective intermediate orientation of the virtual observer (7a, 7b, 7c, 7d, 7e, 7f, 7g, 7h, 7i); and / or - the initial observation parameters include the initial field of view of the virtual observer (7a, 7b, 7c, 7d, 7e, 7f, 7g, 7h, 7i), and the final observation parameters include the final field of view of the virtual observer (7a, 7b, 7c, 7d, 7e, 7f, 7g, 7h, 7i), and for each of the intermediate readout instances, the corresponding intermediate observation parameters include the corresponding intermediate field of view of the virtual observer (7a, 7b, 7c, 7d, 7e, 7f, 7g, 7h, 7i).

4. A computer-implemented method according to any one of the preceding claims, It is characterized in that - for an initial readout instance, the readout image data are projected onto an initial projection surface (9a) and an initial image dependent on the projected image data is displayed on a display device (5); and / or - For the final readout instance, the readout image data are projected onto a final projection surface (9b) and a final image depending on the projected image data is displayed on the display device (5).

5. A computer-implemented method according to any one of the preceding claims, It is characterized in that For each of the final sampling points (11b), for all intermediate readout instances, the intermediate sampling points (12, 13) interpolated between the final sampling point (11b) and the assigned initial sampling point (11a) are located on a straight line connecting the final sampling point (11b) and the assigned initial sampling point.

6. A computer-implemented method according to any one of the preceding claims, It is characterized in that For each final sampling point (11b), the corresponding distances between the intermediate sampling points (12, 13) interpolated between the final sampling point (11b) and the assigned initial sampling point (11a) are the same for each pair of consecutive intermediate readout instances.

7. A computer-implemented method according to any one of the preceding claims, It is characterized in that The readout rate is greater than the frame rate.

8. A computer-implemented method according to any one of the preceding claims, It is characterized in that The readout rate is at least twice the frame rate, for example at least five times the frame rate.

9. A computer-implemented method according to any one of the preceding claims, It is characterized in that The readout rate is equal to or less than the refresh rate of the display device (5).

10. A computer-implemented method according to any one of the preceding claims. It is characterized in that -The initial projection surface (9a) is given by the points satisfying the following equation: ， wherein X, Y and Z represent the Cartesian coordinates of corresponding points on the initial projection surface (9a), Z corresponds to the height above a predefined ground plane on which the vehicle (1) is located, n is an even integer equal to or greater than 4, and ai, bi and c are predefined real coefficients; and / or -The final projection surface (9b) is given by the points satisfying the following equation: ， where X, Y and Z represent the Cartesian coordinates of the corresponding points on the final projection surface (9b), Z corresponds to the height above the ground plane, n' is an even integer equal to or greater than 4, and ai', bi' and c' are predefined real coefficients.

11. A computer-implemented method according to any one of claims 1 to 9, It is characterized in that - the initial projection surface (9a) comprises an initial base and an initial raised portion, the initial base being given by an initial portion of a predetermined ground plane, the vehicle (1) being located on the initial base, wherein the vehicle (1) is located within the initial base and the initial raised portion adjoining the initial base at an outer boundary of the initial base; and / or - the final projection surface (9b) comprises a final base given by a final portion of the ground plane, wherein the vehicle (1) is located within the final base, and a final raised portion adjoining the final base at its outer boundary.

12. An electronic vehicle navigation system (2) for a vehicle (1), comprising at least one computing unit (3), the computing unit storing a predefined initial projection surface (9a) and a predefined final projection surface (9b) and being configured to: - receiving a video stream depicting the environment of the vehicle (1) from a camera system (4) of the vehicle (1) and buffering the video stream in a buffer memory, wherein, The buffer memory is updated at a predefined frame rate; - reading out image data from said buffer memory at a predetermined readout rate at an initial readout instance, a plurality of consecutive intermediate readout instances and a final readout instance; - determining a set of initial sampling points (11a) on the initial projection surface (9a) and a set of final sampling points (11b) on the final projection surface (9b), wherein each final sampling point (11b) is assigned to exactly one of the initial sampling points (11a); - for each of the intermediate readout instances, determining a set of intermediate sampling points (12, 13), wherein each intermediate sampling point is interpolated between one of the final sampling points (11b) and the corresponding assigned initial sampling point (11a), and determining an intermediate projection surface (10) containing the set of intermediate sampling points (12, 13); - for each of the intermediate readout instances, projecting the readout image data onto a corresponding intermediate projection surface (10), and controlling a display device (5) of the vehicle (1) to display an intermediate image in dependence on the projected image data.

13. The electronic vehicle navigation system (2) according to claim 12, It is characterized in that - the readout rate is equal to or less than the refresh rate of the display device (5); and / or - The readout rate is greater than the frame rate, such as at least twice the frame rate, such as at least five times the frame rate.

14. A computer program comprising instructions which, when executed by at least one computing unit (3), cause the at least one computing unit (3) to perform a computer-implemented method according to any one of claims 1 to 11.

15. A computer-readable storage medium storing the computer program according to claim 14.

Images