Method for operating an image capturing device of a vehicle, control device, image capturing device, vehicle, and computer program

Abstract

The invention relates to a method for operating an image capturing device (2) of a vehicle (1), said image capturing device (2) comprising at least one camera (3) and at least one display device (9). The camera (3) detects image data from a sub-region of the surroundings of the vehicle, and a view (16) of the surroundings generated from the image data is displayed on the display device (9). The view (16) of the surroundings is generated from the image data using a projection of the image data onto a virtual projection surface (12), and an elliptically curved projection surface (12) is used which is elliptically curved in a first spatial direction (x) corresponding to the vehicle longitudinal direction and in a second spatial direction (y) corresponding to the vehicle width direction.

Description

[0001] Description

[0002] Method for operating an image recording device of a vehicle, control unit, image recording device, vehicle and computer program

[0003] The invention relates to a method for operating an image recording device of a vehicle, wherein the image recording device comprises at least one camera and at least one display device. The camera captures image data from a partial area of ​​a vehicle's surroundings, and a view of the surroundings generated from the image data is displayed on the display device. The view of the surroundings is generated from the image data by projecting the image data onto a virtual projection surface. Furthermore, the invention relates to a control unit, an image recording device, a vehicle, and a computer program.

[0004] In vehicles, image recording devices that capture part of the vehicle's surroundings can be used to provide driver assistance functions. For example, images captured by one or more of the vehicle's cameras and / or representations synthesized from these images can be displayed on a vehicle's display. This can improve the driver's perception of the surroundings and thus also the vehicle's maneuverability. In particular, areas that are not directly visible from inside the vehicle can be made visible to the driver. This makes maneuvering in tight spaces and / or parking the vehicle easier, for example.

[0005] In order to capture the largest possible area of ​​the surroundings, it is common practice to use wide-angle cameras with a large field of view. However, these camera images can exhibit significant distortion, which can be reduced by projecting the image data. The image data is projected from the plane of an image sensor onto a virtual projection surface. The image presented to the driver can then be derived from the projection surface.

[0006] US 2019 / 0001887 A1 describes a method for generating a view of an area in front of a vehicle. Images captured by a fisheye lens camera of the vehicle are projected in parallel onto a virtual, flat projection surface, with the projection surface being parallel to the plane of an image sensor of the camera.

[0007] The invention is based on the object of specifying a method which is improved compared to the method and which in particular enables an improved adaptability of the environmental display to a current ferry operation.

[0008] To achieve this object, it is provided according to the invention in a method of the type mentioned at the outset that an elliptically curved projection surface is used which is elliptically curved in a first spatial direction corresponding to the vehicle longitudinal direction and a second spatial direction corresponding to the vehicle width direction.

[0009] The vehicle can be, for example, a passenger car, a truck, a trailer, or a rail-bound vehicle. However, the invention is not limited to specific vehicle types, so other vehicle configurations are also possible. The vehicle comprises an image recording device with one or more cameras, each of which captures a partial area of ​​the vehicle's surroundings. When using multiple cameras, it is possible that the recording areas of two or more of the cameras may partially overlap.

[0010] The camera or cameras of the image recording device can be arranged, for example, on the vehicle body of the vehicle, for example on a bumper, a radiator grille, a tailgate or on the side mirrors of the vehicle. Other arrangement positions for the cameras are also possible depending on the type of vehicle. The image data or camera images from at least one of the cameras are used to generate a view of the surroundings which represents at least a partial area of ​​the vehicle's surroundings. The view of the surroundings is then shown on the display device of the vehicle. The display device can be, for example, a screen or a display which is arranged in an interior of the vehicle, for example in a center console or the like, and which can be viewed by a driver of the vehicle.

[0011] According to the invention, the environmental view is generated by projecting the image data onto a virtual, elliptically curved projection surface. The image data is transferred from an image sensor plane of an image sensor of at least one camera to the projection surface. The environmental view can then be generated starting from a virtual camera position.

[0012] The elliptically curved projection surface is a virtual structure that is arranged in a virtual space relative to a virtual position of the vehicle, for example, in a vehicle-fixed coordinate system. The virtual camera position from which the virtual projection surface is viewed to derive the surrounding view can correspond to or differ from the actual camera position in the virtual space.

[0013] The elliptically curved projection surface comprises two elliptically curved edges. The elliptically curved edges of the projection surface lie in or parallel to a plane corresponding to the vehicle's base area or a road surface or the like in virtual space. This plane extends in the vehicle's longitudinal direction and in the vehicle's transverse direction, so that the projection surface is elliptically curved in a first spatial direction corresponding to the vehicle's longitudinal direction and a second spatial direction corresponding to the vehicle's width direction. The elliptically curved edges or the elliptical curvature of the projection surface can each be described by an ellipse or an elliptical arc segment and can be expressed accordingly by two radii or semi-axes, where, for example, a first radius or a first semi-axis lies in the first spatial direction and a second radius or a second semi-axis lies in the second spatial direction.

[0014] The advantage of the elliptically curved projection surface is that, particularly in an edge region of the projection surface or in an edge region of the detection range of the at least one camera, objects can be displayed larger than would be possible using a planar projection surface or a cylindrically curved projection surface having two circular arc-shaped edges. Compared to a planar or cylindrical projection surface, a different type of distortion correction can thus be achieved for the image data of the at least one camera, which particularly highlights objects in an edge region of the detection range of the at least one camera.

[0015] For example, this is advantageous if the at least one camera is a rear-view camera that captures image data in an area behind the vehicle. For example, when the vehicle is reversing out of a gate, road users and / or objects located to the side of the rear of the vehicle captured by the rear-view camera can be displayed in an enlarged format. This makes it easier for the driver to perceive these road users and / or objects and to take them into account when maneuvering the vehicle.

[0016] In a preferred embodiment of the invention, it can be provided that a projection surface is used which is straight in a third spatial direction corresponding to the vehicle height direction. The third spatial direction is in particular orthogonal to the first spatial direction and the second spatial direction, resulting in a projection surface that is curved in the vehicle longitudinal direction. According to the invention, it can be provided that, depending on a user selection of a user of the vehicle, the radius of the elliptical curvature in the first spatial direction is smaller or larger than the radius of the elliptical curvature in the second spatial direction and / or that, through a user selection, a pair of values ​​for the first radius and the second radius can be selected from a plurality of different predetermined pairs of values ​​for the first radius and the second radius.

[0017] By changing the ratio between the radius of the elliptical curvature in the first spatial direction and the radius of the elliptical curvature in the second spatial direction, the distortion correction of the image data and thus the appearance of the surrounding view can be changed. The larger the radius in the second spatial direction compared to the radius in the first spatial direction, the more the projection surface approximates a plane. Accordingly, the appearance of the surrounding view approaches that of a view generated using a flat projection surface. If the radius in the first spatial direction is selected to be larger than the radius in the second spatial direction, the result is a surrounding view that has less compression in the lateral areas.

[0018] Preferably, it can be provided that, by means of user input, switching between several predefined ratios for the radius in the first spatial direction and the second spatial direction can be performed. This allows the driver to adapt the appearance of the surrounding view to their preferences and / or to the current driving or traffic situation.

[0019] The user input can be entered, for example, via a user interface arranged in the interior of the vehicle. The user interface can be, for example, an operating element, such as a switch, a rotary control, or a touch display. In a preferred embodiment of the invention, it can be provided that, depending on a further user selection, the generation of the environmental view based on the elliptically curved projection surface is terminated, with the environmental view subsequently being generated using a further projection surface, in particular a planar or cylindrically curved projection surface. This also enables the driver to set a suitable environmental view according to their preferences or the current driving situation.

[0020] According to the invention, at least one camera with a wide-angle lens, in particular a camera with a fisheye lens, can be used as the camera. The camera can, for example, have a field of view with a horizontal aperture angle of at least 180°. This allows a large portion of the vehicle's surroundings to be captured with a single camera. Alternatively, multiple cameras, each with a smaller horizontal aperture angle, can be used.

[0021] In the method according to the invention, a front camera, a side camera, and / or a rearview camera of the vehicle can preferably be used as the camera. The surroundings view can be generated based on the image data from a single one of these cameras. Alternatively, the surroundings view can also be generated from the image data of two or more cameras, in particular using the image data from cameras with at least partially overlapping detection areas. In this way, a combined surroundings view can be generated, by means of which a larger sub-area of ​​the vehicle's surroundings can be imaged than when using the image data from a single camera.

[0022] A control unit according to the invention is configured to execute a method according to the invention upon provision of image data from at least one camera. The control unit can be configured to be connected to the camera via a wired or wireless communication link for transmitting the image data. An image recording device for a vehicle according to the invention is configured to comprise at least one camera, at least one display device, and a control unit according to the invention.

[0023] A vehicle according to the invention comprises an image recording device according to the invention.

[0024] A computer program according to the invention is provided to include instructions that configure a control unit to execute a method according to the invention based on image data provided by a camera. The computer program can be stored on a non-transient data storage device and / or be retrievable from a data storage device such as a server via a communications connection, such as the Internet.

[0025] All advantages and embodiments described above in relation to the method according to the invention apply accordingly to the control unit according to the invention, the image recording device according to the invention, the vehicle according to the invention and the computer program according to the invention and vice versa.

[0026] Further advantages and details of the invention will become apparent from the exemplary embodiments described below and from the drawings. These are schematic representations and show:

[0027] Fig. 1 shows an embodiment of a vehicle according to the invention,

[0028] Fig. 2 is a block diagram of an embodiment of a method according to the invention,

[0029] Fig. 3 shows an elliptically curved projection surface for

[0030] Explanation of the embodiment of the method according to the invention, Fig. 4 shows an environmental view generated using the method according to the invention and

[0031] Fig. 5 shows an environmental view generated by means of a cylindrically curved projection surface.

[0032] Fig. 1 shows an embodiment of a vehicle 1. The vehicle 1 can be, for example, a motor vehicle, in particular a passenger car, a truck, or another type of commercial vehicle. It is also possible for the vehicle 1 to be a non-motorized vehicle such as a trailer or a combination consisting of a towing vehicle and a trailer. The vehicle 1 can also be a rail-bound vehicle, such as a tram or the like, or a robot.

[0033] The vehicle 1 comprises an exemplary embodiment of an image recording device 2, which comprises one or more cameras 3 and a control unit 4. In the present exemplary embodiment, the vehicle 1 comprises four surroundings cameras 3, which form a camera arrangement designed as a surround view system. A first camera 5 is arranged as a front camera of the vehicle 1, a second camera 6 as a rear-view camera of the vehicle 1, and a third camera 7 and a fourth camera 8 each as a side camera of the vehicle 1. By means of the cameras 3, image data can be generated from a partial area of ​​the surroundings of the vehicle 1. The cameras 3 are each designed as a wide-angle camera, for example as cameras each with a fisheye lens, and preferably have a horizontal detection range of 180° or more.The control unit 4 of the image recording device 2 can be designed, for example, as a microcontroller, as a processor or as another type of computing device.

[0034] The vehicle 1 further comprises a display device 9, via which graphic information can be presented to a driver or a user of the vehicle 1. The display device 9 can, for example, be one or more screens or displays arranged in an interior of the vehicle 1.

[0035] The cameras 3 are connected to the control unit 4 via a communication connection 10. The communication connection 10 can, for example, comprise a plurality of point-to-point connections or be a bus connection such as a CAN bus or the like. Camera images or video streams are transmitted via the communication connection 10, in particular continuously as a video stream.

[0036] Image data from the cameras 3 is transmitted to the control unit 4. At least a portion of the camera images and / or an environmental view generated from the image data of at least one of the cameras 3 can then be displayed to a driver of the vehicle 1 on the display device 9.

[0037] Furthermore, the vehicle 1 comprises a user interface 11, via which one or more different user inputs can be made. The user interface can be a control element, for example a switch, a rotary control, or a touch display. The user interface 11 is preferably arranged in the interior of the vehicle 1.

[0038] The control unit 4 is designed to carry out an embodiment of a method for operating the image recording device 2. Fig. 2 shows a block diagram of an embodiment of the method.

[0039] In step S1, one or more of the cameras 3 each capture image data from a partial area of ​​the surroundings of the vehicle 1 and transmit them to the control unit 4. For example, the camera 6 arranged as a rear-view camera can capture image data from a partial area of ​​the vehicle's surroundings located behind the vehicle and transmit them to the control unit 4 via the communication connection. The image data can in particular be recorded and transmitted as a video stream, so that the generation of an environmental view that reproduces the surroundings at least essentially in real time is possible. In step S2, the control unit 4 generates the environmental view from the image data of the camera 6. For this purpose, the image data from the camera 6 is projected onto a virtual, elliptically curved projection surface 12.

[0040] Fig. 3 shows the geometry of the virtual projection surface 12 relative to a virtual counterpart 13 of the vehicle position in a virtual space. The virtual space is defined by a coordinate system having a first spatial direction x extending in the vehicle's longitudinal direction. Furthermore, the coordinate system includes a second spatial direction y extending in the vehicle's transverse direction and a third spatial direction z extending in the vehicle's vertical direction. The three spatial directions are each orthogonal to one another.

[0041] The elliptically curved projection surface 12 is elliptically curved in the first spatial direction x and in the second spatial direction y. In the third spatial direction z, the projection surface is straight. In other words, the projection surface 12 has two edges 14 lying in or parallel to the xy plane corresponding to a road surface, which edges have the shape of an elliptical arc segment. The other two edges, which extend in the third spatial direction z, are not curved and are therefore straight.

[0042] The elliptical curvature of the projection surface can be defined using two radii, with a first radius n in the first spatial direction x and a second radius r2 in the second spatial direction y. Alternatively, the elliptical curvature can also be defined using two semi-axes. In addition, further parameters, such as the height of the projection surface in the third spatial direction z and / or the position of the focal points, the center point and / or the vertices or the like, of an ellipse having the elliptical arc segment can be used to define the unique position of the projection surface 12 in the coordinate system. The image data generated, for example, by the camera 6 arranged as a rear-view camera, are projected onto the projection surface 12.This is not a physical projection, but rather a computational operation performed by the control unit 4. Subsequently, starting from a virtual camera position 15, the surrounding view can be derived using the projection surface 12 and the image data projected onto it. The virtual camera position 15 can correspond to the actual position of the camera 6 in the coordinate system or, in order to generate a view of the surroundings from a different angle, can also be different from it.

[0043] In a step S3 of the exemplary embodiment of the method shown in Fig. 2, the environmental representation generated in this way is displayed on the display device 9 of the vehicle 1. Due to the use of the elliptically curved projection surface 12, objects and / or road users located in a lateral portion of the detection range of the camera 6 are displayed in an enlarged manner compared to the use of a cylindrically curved or planar projection surface.

[0044] Fig. 4 shows an embodiment of an environmental view 16 generated using the elliptically curved projection surface 12. For comparison, Fig. 5 shows an example of a further environmental view 17 generated using a cylindrically curved projection surface. The elliptical curvature of the projection surface 12 clearly causes an additional vehicle 18 located at the lateral edge of the detection area to be displayed in an enlarged manner. Advantageously, the use of the elliptically curved projection surface 12 allows road users approaching from the side to be better detected, for example, when the driver is reversing the vehicle 1 out of a gate entrance.

[0045] In order to generate an environmental view adapted to a user request and / or to a current traffic situation, the radius n of the elliptical curvature in the first spatial direction x can be smaller or larger than the radius r2 of the elliptical curvature in the second spatial direction y, depending on a user selection by a user of the vehicle 1. For example, a value pair n, rc can be selected from a plurality of different predefined value pairs n, r2 through the user selection. In this way, the geometry of the projection surface 12 and consequently the appearance of the environmental view 16 can be changed.

[0046] Additionally or alternatively, depending on the user's selection or another selection, the generation of the environmental view based on the elliptical projection surface can be terminated, with the environmental view subsequently being generated based on another projection surface, in particular a planar or cylindrical projection surface. The appearance of the environmental view 16 can also be adjusted by switching between differently shaped projection surfaces. For example, switching to a cylindrical projection surface, such as that used for the additional environmental view 17 in Fig. 4, can reduce distortions, which could be preferable, for example, for reversing straight ahead on an open surface.The transition between different projection surfaces and / or different pairs of radii n, r2 can be smooth, so that a smooth transition can also be seen in the surrounding view 16 displayed on the display device 9. Alternatively, the transition can also be abrupt.

[0047] In addition to the surroundings view 17 described above as an example based on the images from a single camera 3 or the camera 6 arranged as a rear-view camera, the surroundings view 16 can also be generated based on the image data from two or more of the cameras 3. For this purpose, in particular, the image data from cameras 3 with at least pairwise overlapping detection areas can be combined. The use of the image data from all four cameras 3 is also possible. In this way, larger sub-areas of the vehicle's surroundings can be displayed in the surroundings view 16.

Claims

Patent claims 1. A method for operating an image recording device (2) of a vehicle (1), wherein the image recording device (2) comprises at least one camera (3) and at least one display device (9), wherein the camera (3) captures image data from a partial area of ​​a vehicle's surroundings and an environmental view (16) generated from the image data is displayed on the display device (9), wherein the environmental view (16) is generated from the image data by means of a projection of the image data onto a virtual projection surface (12), characterized in that an elliptically curved projection surface (12) is used, which is elliptically curved in a first spatial direction (x) corresponding to the vehicle longitudinal direction and a second spatial direction (y) corresponding to the vehicle width direction.

2. Method according to claim 1, characterized in that a projection surface (12) is used which is straight in a third spatial direction (z) corresponding to the vehicle height direction.

3. Method according to claim 1 or 2, characterized in that, depending on a user selection of a user of the vehicle (1), the radius (n) of the elliptical curvature in the first spatial direction (x) is smaller or larger than the radius (r2) of the elliptical curvature in the second spatial direction (y) and / or that, by means of a user selection, a value pair (n, rc) for the first radius (n) and the second radius (ra) can be selected from a plurality of different predetermined value pairs (n, ) for the first radius (n) and the second radius ( ).

4. Method according to one of the preceding claims, characterized in that, depending on a further user selection, the generation of the environmental view (16) based on the elliptically curved projection surface (12) is terminated, wherein the environmental view (16) is subsequently generated based on a further projection surface, in particular based on a planar or cylindrically curved projection surface.

5. Method according to one of the preceding claims, characterized in that at least one camera with a wide-angle lens, in particular a camera with a fisheye lens, is used as the camera (3).

6. Method according to one of the preceding claims, characterized in that a front camera, a side camera and / or a rear view camera of the vehicle (1) is used as the camera (3).

7. Control device which is configured to carry out a method according to one of the preceding claims when image data from at least one camera (3) is provided.

8. Image recording device for a vehicle, comprising at least one camera (3), at least one display device (9) and a control unit (4) according to claim 7.

9. Vehicle comprising an image recording device (2) according to claim 8.

10. Computer program comprising instructions which set up a control device (4) to carry out a method according to one of claims 1 to 6 on the basis of image data provided by a camera (3).

Images