Method for operating an image recording device of a vehicle, control unit, image recording device, vehicle and computer program

Abstract

The invention relates to a method for operating an image recording device (2) of a vehicle (1), wherein the image recording device (2) comprises at least one camera (3) and at least one display device (9), wherein the camera (3) captures image data from a partial area of ​​a vehicle's surroundings and an environmental view (16) generated from the image data is displayed on the display device (9), wherein the environmental view (16) is generated from the image data by means of a projection of the image data onto a virtual projection surface (12), wherein an elliptically curved projection surface (12) is used, which is elliptically curved in a first spatial direction (x) corresponding to the vehicle's longitudinal direction and a second spatial direction (y) corresponding to the vehicle's width direction.

Description

[0001] The invention relates to a method for operating an image recording device of a vehicle, wherein the image recording device comprises at least one camera and at least one display device. The camera captures image data from a partial region of a vehicle's surroundings and displays an environmental view generated from the image data on the display device. The environmental view is generated from the image data by projecting the image data onto a virtual projection surface. Furthermore, the invention relates to a control unit, an image recording device, a vehicle, and a computer program.

[0002] In vehicles, image recording devices that capture part of the vehicle's surroundings can be used to provide driver assistance functions. For example, images recorded by one or more of the vehicle's cameras and / or representations synthesized from these images can be displayed on a display device in the vehicle. This can improve the driver's perception of the surroundings and thus also the maneuverability of the vehicle. In particular, areas that are not directly visible from inside the vehicle can be made visible to the driver. This makes maneuvering in tight spaces and / or parking the vehicle easier, for example.

[0003] In order to capture the largest possible environmental area, it is common practice to use wide-angle cameras with a large field of view. However, the camera images can exhibit significant distortion, which can be reduced by projecting the image data. The image data is projected from the plane of an image sensor onto a virtual projection surface.

[0004] The image presented to the driver can then be derived from the projection surface.

[0005] US 2019 / 0001887 A1 describes a method for generating a view of an area in front of a vehicle. Camera images captured with a fisheye lens camera of the vehicle are projected in parallel onto a virtual, flat projection surface, with the projection surface parallel to a plane of an image sensor of the camera.

[0006] The invention is based on the object of specifying a method which is improved compared to the method and which in particular enables an improved adaptability of the environmental display to a current driving operation.

[0007] To achieve this object, it is provided according to the invention in a method of the type mentioned at the outset that an elliptically curved projection surface is used which is elliptically curved in a first spatial direction corresponding to the vehicle longitudinal direction and a second spatial direction corresponding to the vehicle width direction.

[0008] The vehicle can be, for example, a passenger car, a truck, a trailer, or a rail-bound vehicle. However, the invention is not limited to specific vehicle types, so other vehicle configurations are also possible. The vehicle comprises an image recording device with one or more cameras, each of which captures a partial area of the vehicle's surroundings. When using multiple cameras, it is possible for the recording areas of two or more cameras to partially overlap.

[0009] The camera or cameras of the image recording device can be mounted on the vehicle body, for example, on a bumper, a radiator grille, a tailgate, or on the side mirrors. Other camera locations are also possible depending on the vehicle type.

[0010] The image data or camera images from at least one of the cameras are used to generate an environmental view that represents at least a portion of the vehicle's surroundings. The environmental view is then displayed on the vehicle's display device. The display device can be, for example, a screen or display that is arranged in an interior of the vehicle, for example, in a center console or the like, and that can be viewed by a driver of the vehicle.

[0011] According to the invention, the environmental view is generated by projecting the image data onto a virtual, elliptically curved projection surface. The image data is transferred from an image sensor plane of an image sensor of at least one camera to the projection surface. The environmental view can then be generated starting from a virtual camera position.

[0012] The elliptically curved projection surface is a virtual structure arranged in a virtual space relative to a virtual position of the vehicle, for example, in a vehicle-fixed coordinate system. The virtual camera position from which the virtual projection surface is viewed to derive the surrounding view can correspond to or differ from the actual camera position in the virtual space.

[0013] The elliptically curved projection surface comprises two elliptically curved edges. The elliptically curved edges of the projection surface lie in or parallel to a plane corresponding to the vehicle's base area or a road surface or the like in virtual space. This plane extends in the vehicle's longitudinal direction and in the vehicle's transverse direction, so that the projection surface is elliptically curved in a first spatial direction corresponding to the vehicle's longitudinal direction and a second spatial direction corresponding to the vehicle's width direction.

[0014] The elliptically curved edges or the elliptical curvature of the projection surface can each be described by an ellipse or an elliptical arc segment and can be expressed accordingly by two radii or semi-axes, whereby, for example, a first radius or a first semi-axis lies in the first spatial direction and a second radius or a second semi-axis lies in the second spatial direction.

[0015] The advantage of the elliptically curved projection surface is that, particularly in an edge region of the projection surface or in an edge region of the detection range of the at least one camera, objects can be displayed larger than would be possible using a planar projection surface or a cylindrically curved projection surface having two circular arc-segment-shaped edges. Compared to a planar or cylindrical projection surface, a different type of distortion correction can thus be achieved for the image data of the at least one camera, which particularly highlights objects in an edge region of the detection range of the at least one camera.

[0016] For example, this is advantageous if the at least one camera is a rear-view camera that captures image data in an area behind the vehicle. For example, when the vehicle is reversing out of a gate, road users and / or objects located to the side of the rear of the vehicle captured by the rear-view camera can be displayed in an enlarged format. This makes it easier for the driver to perceive these road users and / or objects and to take them into account when maneuvering the vehicle.

[0017] In a preferred embodiment of the invention, it can be provided that a projection surface is used that is straight in a third spatial direction corresponding to the vehicle's vertical direction. The third spatial direction is, in particular, orthogonal to the first spatial direction and the second spatial direction, resulting in a projection surface that is curved in the vehicle's longitudinal direction.

[0018] According to the invention, it can be provided that, depending on a user selection of a user of the vehicle, the radius of the elliptical curvature in the first spatial direction is smaller or larger than the radius of the elliptical curvature in the second spatial direction and / or that, by means of a user selection, a pair of values for the first radius and the second radius can be selected from a plurality of different predetermined pairs of values for the first radius and the second radius.

[0019] By changing the ratio between the radius of the elliptical curvature in the first spatial direction and the radius of the elliptical curvature in the second spatial direction, the distortion correction of the image data and thus the appearance of the surrounding view can be changed. The larger the radius in the second spatial direction compared to the radius in the first spatial direction, the more the projection surface approximates a plane. Accordingly, the appearance of the surrounding view approaches that of a view generated using a flat projection surface. If the radius in the first spatial direction is selected to be larger than the radius in the second spatial direction, the result is a surrounding view that has less compression in the lateral areas.

[0020] Preferably, it can be provided that, by means of user input, it is possible to switch between several predefined ratios for the radius in the first spatial direction and the second spatial direction. This allows the driver to adapt the appearance of the surrounding view to their preferences and / or to the current driving or traffic situation.

[0021] The user input can, for example, be entered via a user interface located in the interior of the vehicle. The user interface can be a control element, such as a switch, a rotary control, or a touch display.

[0022] In a preferred embodiment of the invention, it can be provided that, depending on a further user selection, the generation of the surrounding view based on the elliptically curved projection surface is terminated, with the surrounding view subsequently being generated based on another projection surface, in particular a planar or cylindrically curved projection surface. This also enables the driver to set a suitable surrounding view according to their preferences or the current driving situation.

[0023] According to the invention, at least one camera with a wide-angle lens, in particular a camera with a fisheye lens, can be used as the camera. The camera can, for example, have a field of view with a horizontal aperture angle of at least 180°. This allows a large portion of the vehicle's surroundings to be captured with a single camera. Alternatively, multiple cameras, each with a smaller horizontal aperture angle, can be used.

[0024] In the method according to the invention, a front camera, a side camera, and / or a rear-view camera of the vehicle can preferably be used as the camera. The surroundings view can be generated based on the image data from a single one of these cameras. Alternatively, the surroundings view can also be generated from the image data of two or more cameras, wherein in particular the image data from cameras with at least partially overlapping detection areas are used. In this way, a combined surroundings view can be generated, by means of which a larger sub-area of the vehicle's surroundings can be imaged than when using the image data from a single camera.

[0025] A control device according to the invention is configured to execute a method according to the invention upon provision of image data from at least one camera. The control device can be configured to be connected to the camera via a wired or wireless communication link for transmitting the image data.

[0026] For an image recording device according to the invention for a vehicle, it is provided that it comprises at least one camera, at least one display device and a control device according to the invention.

[0027] A vehicle according to the invention comprises an image recording device according to the invention.

[0028] A computer program according to the invention is provided to include instructions that configure a control unit to execute a method according to the invention based on image data provided by a camera. The computer program can be stored on a non-transient data storage device and / or be retrievable from a data storage device such as a server via a communication connection, such as the Internet.

[0029] All advantages and embodiments described above in relation to the method according to the invention apply accordingly to the control unit according to the invention, the image recording device according to the invention, the vehicle according to the invention and the computer program according to the invention and vice versa.

[0030] Further advantages and details of the invention will become apparent from the exemplary embodiments described below and from the drawings. These are schematic representations and show: Fig. 1 an embodiment of a vehicle according to the invention, Fig. 2 a block diagram of an embodiment of a method according to the invention, Fig. 3 a representation of an elliptically curved projection surface to explain the embodiment of the method according to the invention, Fig. 4 shows an environmental view generated using the method according to the invention and Fig. 5 an environmental view generated by means of a cylindrically curved projection surface.

[0031] In Fig. 1 shows an embodiment of a vehicle 1. The vehicle 1 can be, for example, a motor vehicle, in particular a passenger car, a truck, or another type of commercial vehicle. It is also possible for the vehicle 1 to be a non-motorized vehicle such as a trailer or a combination consisting of a towing vehicle and a trailer. The vehicle 1 can also be a rail-bound vehicle, such as a tram or the like, or a robot.

[0032] The vehicle 1 comprises an exemplary embodiment of an image recording device 2, which comprises one or more cameras 3 and a control unit 4. In the present exemplary embodiment, the vehicle 1 comprises four surroundings cameras 3, which form a camera arrangement designed as a surround view system. A first camera 5 is arranged as a front camera of the vehicle 1, a second camera 6 as a rear-view camera of the vehicle 1, and a third camera 7 and a fourth camera 8 are each arranged as a side camera of the vehicle 1. By means of the cameras 3, image data can be generated from a partial area of the surroundings of the vehicle 1. The cameras 3 are each designed as a wide-angle camera, for example as cameras each with a fisheye lens, and preferably have a horizontal detection range of 180° or more.The control unit 4 of the image recording device 2 can be designed, for example, as a microcontroller, as a processor or as another type of computing device.

[0033] The vehicle 1 further comprises a display device 9, via which graphic information can be presented to a driver or a user of the vehicle 1. The display device 9 can, for example, be one or more screens or displays arranged in an interior of the vehicle 1.

[0034] The cameras 3 are connected to the control unit 4 via a communication connection 10. The communication connection 10 can, for example, comprise a plurality of point-to-point connections or be a bus connection such as a CAN bus or the like. Camera images or image data are transmitted from the cameras 3 to the control unit 4 via the communication connection 10, in particular continuously as a video stream. At least some of the camera images and / or a view of the surroundings generated from the image data of at least one of the cameras 3 can then be displayed to a driver of the vehicle 1 on the display device 9.

[0035] Furthermore, the vehicle 1 comprises a user interface 11, via which one or more different user inputs can be made. The user interface can be a control element, for example a switch, a rotary control, or a touch display. The user interface 11 is preferably arranged in the interior of the vehicle 1.

[0036] The control unit 4 is designed to carry out an embodiment of a method for operating the image recording device 2. In Fig. 2 shows a block diagram of an embodiment of the method.

[0037] In step S1, one or more of the cameras 3 each capture image data of a sub-area of the surroundings of the vehicle 1 and transmit them to the control unit 4. For example, the camera 6 arranged as a rear-view camera can capture image data from a sub-area of the vehicle's surroundings located behind the vehicle and transmit them via the communication connection to the control unit 4. The image data can be recorded and transmitted, in particular, as a video stream, so that the generation of an environmental view that reproduces the surroundings at least essentially in real time is possible.

[0038] In step S2, the control unit 4 generates the surrounding view from the image data of the camera 6. For this purpose, the image data of the camera 6 are projected onto a virtual, elliptically curved projection surface 12.

[0039] In Fig. 3 shows the geometry of the virtual projection surface 12 relative to a virtual counterpart 13 of the vehicle position in a virtual space. The virtual space is defined by a coordinate system having a first spatial direction x extending in the vehicle's longitudinal direction. Furthermore, the coordinate system includes a second spatial direction y extending in the vehicle's transverse direction and a third spatial direction z extending in the vehicle's vertical direction. The three spatial directions are each orthogonal to one another.

[0040] The elliptically curved projection surface 12 is elliptically curved in the first spatial direction x and in the second spatial direction y. In the third spatial direction z, the projection surface is straight. In other words, the projection surface 12 has two edges 14 lying in or parallel to the xy plane corresponding to a road surface, which edges have the shape of an elliptical arc segment. The other two edges, which extend in the third spatial direction z, are not curved and are therefore straight.

[0041] The elliptical curvature of the projection surface can be defined using two radii, with a first radius r1 in the first spatial direction x and a second radius r2 in the second spatial direction y. Alternatively, the elliptical curvature can also be defined using two semi-axes. In addition, further parameters, such as the height of the projection surface in the third spatial direction z and / or the position of the focal points, the center point and / or the vertices, or the like, of an ellipse having the elliptical arc segment can be used to define the unique position of the projection surface 12 in the coordinate system.

[0042] The image data generated, for example, by camera 6, arranged as a rear-view camera, is projected onto the projection surface 12. This is not a physical projection, but rather a computational operation performed by the control unit 4. Subsequently, starting from a virtual camera position 15, the surrounding view can be derived using the projection surface 12 and the image data projected onto it. The virtual camera position 15 can correspond to the actual position of the camera 6 in the coordinate system or, in order to generate a surrounding view from a different angle, can also be different from it.

[0043] In a step S3 of the Fig. In the exemplary embodiment of the method shown in Figure 2, the surroundings representation generated in this way is displayed on the display device 9 of the vehicle 1. Due to the use of the elliptically curved projection surface 12, objects and / or road users located in a lateral partial area of the detection range of the camera 6 are displayed in an enlarged manner compared to the use of a cylindrically curved or planar projection surface.

[0044] In Fig. 4 shows an embodiment of an environmental view 16 generated with the aid of the elliptically curved projection surface 12. For comparison, Fig. 5 shows an example of a further environmental view 17 generated using a cylindrically curved projection surface. The elliptical curvature of the projection surface 12 clearly causes a further vehicle 18 located at the lateral edge of the detection area to be displayed in an enlarged manner. Advantageously, the use of the elliptically curved projection surface 12 allows road users approaching from the side to be better detected, for example, when the driver maneuvers the vehicle 1 backward out of a gate entrance.

[0045] In order to generate an environmental view adapted to a user request and / or to a current traffic situation, the radius r1 of the elliptical curvature in the first spatial direction x can be smaller or larger than the radius r2 of the elliptical curvature in the second spatial direction y, depending on a user selection by a user of the vehicle 1. For example, a value pair r1, r2 can be selected from a plurality of different predefined value pairs r1, r2 through the user selection. In this way, the geometry of the projection surface 12 and consequently the appearance of the environmental view 16 can be changed.

[0046] Additionally or alternatively, depending on the user selection or a further selection, the generation of the environmental view based on the elliptical projection surface can be terminated, with the environmental view subsequently being generated based on another projection surface, in particular a planar or cylindrical projection surface. The appearance of the environmental view 16 can also be adapted by switching between differently shaped projection surfaces. For example, by switching to a cylindrical projection surface, as used for the further environmental view 17 in Fig.4 was used, distortions can be reduced, which could be preferable, for example, for straight reversing in an open area. The transition between different projection surfaces and / or different pairs of radii r1, r2 can be smooth, so that a smooth transition can also be seen in the surrounding view 16 shown on the display device 9. Alternatively, the transition can also be abrupt.

[0047] In addition to the surroundings view 17 described above as an example based on the images from a single camera 3 or the camera 6 arranged as a rear-view camera, the surroundings view 16 can also be generated based on the image data from two or more of the cameras 3. For this purpose, in particular, the image data from cameras 3 with at least pairwise overlapping detection areas can be combined. The use of the image data from all four cameras 3 is also possible. In this way, larger sub-areas of the vehicle's surroundings can be displayed in the surroundings view 16. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] US 2019 / 0001887 A1

[0005]

Claims

[1] Method for operating an image recording device (2) of a vehicle (1), wherein the image recording device (2) comprises at least one camera (3) and at least one display device (9), wherein the camera (3) captures image data from a partial area of a vehicle's surroundings and an environmental view (16) generated from the image data is displayed on the display device (9), wherein the environmental view (16) is generated from the image data by means of a projection of the image data onto a virtual projection surface (12), characterized by that an elliptically curved projection surface (12) is used which is elliptically curved in a first spatial direction (x) corresponding to the vehicle longitudinal direction and a second spatial direction (y) corresponding to the vehicle width direction. [2] Method according to claim 1, characterized bythat a projection surface (12) is used which is straight in a third spatial direction (z) corresponding to the vehicle height direction. [3] Method according to claim 1 or 2, characterized by that, depending on a user selection of a user of the vehicle (1), the radius (r1) of the elliptical curvature in the first spatial direction (x) is smaller or larger than the radius (r2) of the elliptical curvature in the second spatial direction (y) and / or that, by means of a user selection, a value pair (r1, r2) for the first radius (r1) and the second radius (r2) can be selected from a plurality of different predetermined value pairs (r1, r2) for the first radius (r1) and the second radius (r2). [4] Method according to one of the preceding claims, characterized bythat, depending on a further user selection, the generation of the environmental view (16) based on the elliptically curved projection surface (12) is terminated, wherein the environmental view (16) is subsequently generated based on a further projection surface, in particular based on a planar or cylindrically curved projection surface. [5] Method according to one of the preceding claims, characterized by that at least one camera with a wide-angle lens, in particular a camera with a fisheye lens, is used as the camera (3). [6] Method according to one of the preceding claims, characterized by that a front camera, a side camera and / or a rear view camera of the vehicle (1) is used as the camera (3). [7] Control device which is designed to carry out a method according to one of the preceding claims when image data from at least one camera (3) are provided. [8] Image recording device for a vehicle, comprising at least one camera (3), at least one display device (9) and a control device (4) according to claim 7. [9] Vehicle comprising an image recording device (2) according to claim 8. [10] Computer program comprising instructions which set up a control device (4) to carry out a method according to one of claims 1 to 6 on the basis of image data provided by a camera (3).

Images