Method for operating an image recording device of a vehicle, control unit, image recording device, vehicle and computer program

Abstract

The invention relates to a method for operating an image recording device (2) of a vehicle (1), wherein the image recording device (2) comprises at least one camera (3) and at least one display device (9), wherein the camera (3) captures image data from a partial area of ​​a vehicle's surroundings and an environmental view (18) generated from the image data is displayed on the display device (9), wherein the environmental view (18) is generated from the image data by means of a projection of the image data, wherein, depending on a speed of the vehicle (1), a change is made between at least two different projection methods for projecting the image data, wherein the different projection methods each use a differently shaped projection surface (12, 13, 14) for projecting the image data.

Description

The invention relates to a method for operating an image recording device of a vehicle, wherein the image recording device comprises at least one camera and at least one display device, wherein the camera acquires image data from a partial region of a vehicle environment and an environment view generated from the image data is displayed on the display device, wherein the environment view is generated from the image data by means of a projection of the image data. The invention further relates to a control device, an image recording device, a vehicle and a computer program.In vehicles, image recording devices which record a part of the vehicle environment can be used to provide driver assistance functions. Images of the vehicle environment recorded by one or more cameras of the vehicle and / or representations synthesized from these images can be displayed on a display device for a driver of the vehicle. By providing such an environmental view, the environmental portability for the driver and thus also the maneuverability of the vehicle can be improved. In particular, regions which cannot be seen directly from the interior of the motor vehicle can also be made visible to the driver. This facilitates, for example, maneuvering in a confined space and / or parking operations to be carried out with the vehicle.For example, in order to equalize image data generated by wide-angle cameras and / or to display the surrounding view from a virtual viewpoint different from the actual viewpoint of the camera, it is known to perform projection of the image data. In this case, the image data are projected from the plane of an image sensor onto a virtual projection surface. Subsequently, the representation presented to the driver can then be derived from the projection surface starting from the virtual camera position.US 2019 / 0001887 A1 describes a method for generating a view of an area lying in front of a vehicle. Camera images recorded with a camera of the vehicle are projected in parallel onto a virtual, planar projection surface, wherein the projection surface is parallel to a plane of an image sensor of the camera.It is known from the prior art to adapt the camera field of view of the surroundings cameras of a vehicle as a function of a current operating state of the vehicle, in order that the driver of the vehicle can be permitted an improved perception of the surroundings.For example, the publication US 2015 / 0350607 A1 describes a device for displaying a vehicle environment, which comprises a plurality of cameras. The surrounding areas captured by the cameras can be changed depending on the speed in this case by the cameras being mechanically tilted and / or rotated. Tilting or rotation of the cameras takes place by tilting or rotating an image sensor of the camera and / or a lens of the camera optics and brings about a change in the detection range of the camera with respect to the vehicle environment.The invention is based on the object of specifying a method which is improved in comparison therewith and which in particular enables improved adaptability of the environmental representation to a current driving mode.To achieve this object, in a method of the type mentioned at the beginning, it is provided according to the invention that, depending on a speed of the vehicle, a switch is made between at least two different projection methods for the projection of the image data, wherein the different projection methods each use a differently shaped projection surface for the projection of the image data.By means of the image recording device of the vehicle, camera images from the environment of the vehicle are recorded, in particular continuously in the form of an image data stream or a video stream. For this purpose, the image recording device comprises at least one camera which is arranged on the vehicle and can capture a partial region of the vehicle environment, for example in front of, behind or next to the vehicle, depending on its arrangement position on the vehicle.From the image data recorded by means of the one or more cameras, an environmental view is generated, which is displayed to the driver on the display device of the image recording device. The display device can be arranged in particular in an interior of the vehicle such that it can be seen by the driver and can be embodied, for example, as a screen, a touch display or the like.The surrounding view is generated from the image data by means of a projection of the image data onto a projection surface. In this case, the image data that were recorded by an image sensor of the at least one camera are transmitted to the projection surface by means of an assignment rule. Subsequently, the surrounding view can be derived from the image data transmitted to the projection surface. In particular, the surrounding view can be generated starting from a virtual camera position, which can either correspond to the actual camera position or differ from the actual position of the camera. When using a virtual camera position different from the actual camera position, the surrounding view may be displayed from a perspective different from the perspective from which the image data was captured by the camera.The projection surface used for the projection is a virtual construct to which the image data are transmitted, for example, by means of an assignment rule. The subsequently derived environmental view depends, inter alia, on the geometry of the projection surface. Depending on how the projection surface is shaped, environmental views with different image properties result.According to the invention, depending on the speed of the vehicle, a switch is made between two different projection methods, which differ with regard to the geometry of the projection surface used. This advantageously makes it possible for a suitable environmental view to be generated and presented to the driver as a function of the vehicle speed and thus also for different driving situations. The maneuvering of the vehicle is advantageously facilitated as a result, since an advantageous environmental view can be provided for the respective speeds or driving situations. Mechanical actuators for changing the viewing range of the camera can advantageously be dispensed with here.As different projection methods, for example, a planar projection using a planar projection surface and / or a cylindrical projection using a cylindrical curved projection surface or an elliptical projection using an elliptically curved projection surface can be employed. In this context, cylindrical curved projection surface means that the projection surface in virtual space has in particular two edges which are in the form of a segment of a circular arc. The projection surface preferably comprises two further edges which are not curved or straight. This yields a projection surface which corresponds to a segment of a cylinder jacket, wherein the projection is effected in particular onto the concave side or onto the inner side of the cylinder jacket segment.Accordingly, in this context, an elliptically curved projection surface is to be understood as a projection surface which in virtual space has in particular two edges curved in accordance with an ellipse segment. Here, too, the projection surface can have two further, straight edges and the projection can take place in particular onto the concave side of the projection surface.The change between the projection methods or between the different geometries of the projection surface can take place in particular automatically, i.e. without previous user input or the like. This can be realized, for example, in that a control device configured to carry out the method determines and uses a projection surface for generating the surrounding view as a function of a speed measurement value of the vehicle.Changing the geometry of the projection surface as a function of the speed makes it possible to adapt the image information represented in the surrounding view to the current vehicle state. In this case, for example, the proportion of the partial region of the vehicle environment depicted in the environmental view or the field of view (FOV) can be adapted. Additionally or alternatively, the distortion of the surrounding view can also be influenced by the selection of the geometry of the projection surface, i.e. the image data can be reproduced with a lower distortion and / or with different magnifications in partial regions of the displayed vehicle environment.According to the invention, it can be provided that a projection method is used at lower speeds for imaging a larger horizontal field of view of the camera than at higher speeds. This can be achieved, for example, by using a cylindrical projection or an elliptical projection at lower speeds.As a result, at low speeds, a portion of the horizontal field of view that is as large as possible can be imaged, even if said field of view has greater distortions compared to an environmental view that was generated, for example, with the aid of a planar projection. At higher speeds, a smaller horizontal field of view is also generally sufficient, since the vehicle is generally moved substantially straight ahead at higher speeds or is generally no longer maneuvered at higher speeds.In a preferred embodiment of the invention, it can be provided that a projection method with a lower distortion of the image data than at lower speeds is used at higher speeds. This can be achieved, for example, by using planar projection at the higher speeds. As a result, at higher speeds, an image display with low distortion or without distortion can advantageously be carried out.In order to trigger the change between the projection methods, it can be provided according to the invention that a first projection method is used at a speed below a limit value and a second projection method is used at a speed above the limit value or above a further limit value. Below the speed limit value, the first projection method can thus be used to generate an environmental representation desired for lower speeds. Above the limit value, or also above a further limit value different from the limit value, a second projection method can be changed to achieve a preferred display for higher speeds.According to the invention, it can be provided that, when the vehicle speed exceeds or falls below the limit value or at speeds between the limit value and the further limit value, an, in particular continuous, transition between the projection methods is used for generating the surrounding view. The continuous transition can be used both during an acceleration of the vehicle, i.e. with an increasing speed of the vehicle, and during a braking or with a decreasing speed of the vehicle.The continuous transition between the projection methods can be generated, for example, by calculating and displaying intermediate images of the environmental view between an environmental view which was generated by means of the first projection method and a further environmental view which was generated by means of the second projection method. In particular, a smooth transition between the projection methods can thereby be generated. The smooth transition can be triggered by exceeding or falling below the limit value, depending on whether the vehicle is accelerating or decelerating. When using a further limit value, the flowing transition can be represented in particular in the speed range between the limit value and the further limit value.As an alternative to the generation of the intermediate images, it is possible for the geometry of the projection surface to be adapted, so that a smooth course results from the geometry of the projection surface of the first projection method to the geometry of the projection surface of the second projection method. This has the effect that the environmental view generated with the aid of the projection surface also has a continuous course over time, so that the environmental view can be displayed to the driver with a continuous change or with a morphing between the two different presentation modes. Here too, the display of the continuous transition can be triggered when the limit value is exceeded or undershot or can be displayed in the speed range between the limit value and the further limit value.According to the invention, it can be provided that a different limit value and / or a different further limit value is used when the vehicle is accelerating than when the vehicle is braking. In this way, the behavior of the projection of the image data or the image properties of the environmental representation can be further adapted to different driving situations.In a preferred embodiment of the invention, it can be provided that a cylindrical projection with a cylindrically curved projection surface or an elliptical projection with an elliptically curved projection surface is used as the or a first projection method and a planar projection with a planar projection surface is used as the or a second projection method.According to the invention, it can be provided that a front camera, a side camera and / or a rear camera of the vehicle is used as at least one camera. The surrounding view can be generated from the image data of an individual one of the cameras, in particular a front camera or a rear camera. However, it is also possible for the image data to be used by a plurality of cameras, for example by combining the image data from cameras arranged adjacent to the vehicle or cameras with capture regions which overlap at least partially, in order to be able to cover a greater proportion of the vehicle environment.The cameras are preferably designed as cameras with a wide-angle lens, for example a fish-eye lens or the like, and have a horizontal field of view of at least 180°. Alternatively, it is possible that a plurality of cameras with a smaller field of view are used on the front of the vehicle, the sides of the vehicle and / or on the rear of the vehicle, in order to capture the entire vehicle environment in particular.For a control device according to the invention, it is provided that it is configured to execute a method according to the invention when providing image data of at least one camera.For an image recording device according to the invention for a vehicle, it is provided that it comprises at least one camera, at least one display device and a control device according to the invention.For a vehicle according to the invention, it is provided that it comprises an image recording device according to the invention.For a computer program according to the invention, it is provided that it comprises instructions which set up a control device to execute a method according to the invention on the basis of provided image data of a camera. The computer program can be stored on an in particular non-transient data carrier. It is also possible for the computer program to be retrievable from a data storage device, for example a server, via a communication connection, such as the Internet.For the control device according to the invention, the image recording device according to the invention, the vehicle according to the invention and the computer program according to the invention, the advantages and embodiments described above with reference to the method according to the invention apply accordingly.The advantages and details described for the control device according to the invention, the image recording device according to the invention, the vehicle according to the invention and the computer program according to the invention can also be applied analogously to the respective other subject matters.Further advantages and details of the invention are evident from the exemplary embodiments described below and on the basis of the drawings. These are schematic representations and show: FIG. 1 : an exemplary embodiment of a vehicle according to the invention, FIG. 2 : shows a block diagram of an exemplary embodiment of a method according to the invention, FIG. 3 : several examples of differently shaped projection surfaces and their arrangement in virtual space, FIG. 4 is an environmental view when using a first projection method; and FIG. 5 : shows the environmental view when using a second projection method.FIG. 1 shows an exemplary embodiment of a vehicle 1. The vehicle 1 can be, for example, a motor vehicle, in particular a passenger car, a truck or another type of commercial vehicle. It is also possible that the vehicle 1 is an unmotorized vehicle such as a trailer, or that it is a combination of a towing vehicle and a trailer. The vehicle 1 may also be a rail-bound vehicle such as a roadway or the like, or a robot.The vehicle 1 comprises an exemplary embodiment of an image recording device 2 which comprises one or more cameras 3 and a control unit 4. In the present exemplary embodiment, the vehicle 1 comprises four surroundings cameras 3, which form a camera arrangement embodied as a surround view system. A first camera 5 is arranged as a front camera of the vehicle 1, a second camera 6 as a rear camera of the vehicle 1, and a third camera 7 and a fourth camera 8 are each arranged as a side camera of the vehicle 1. By means of the cameras 3, image data from a partial area of the surroundings of the vehicle 1 can be generated in each case.The cameras 3 are each designed as a wide-angle camera, for example as cameras each having a fish-eye lens, and preferably have a detection range in the horizontal of 180° or more. The control unit 4 of the image recording device 2 can be designed, for example, as a microcontroller, as a processor or as another type of computing device.The vehicle 1 further comprises a display device 9, via which graphical information can be displayed for a driver or a user of the vehicle 1. The display device 9 can be, for example, one or more screens or displays arranged in an interior of the vehicle 1.The cameras 3 are connected to the control device 4 via a communication link 10. The communication link 10 may include, for example, a plurality of point-to-point connections or may be a bus connection such as a CAN bus or the like. Via the communication link 10, camera images or image data are transmitted, in particular continuously as a video stream, from the cameras 3 to the control device 4. At least a part of the camera images and / or an environmental view generated from the image data of at least one of the cameras 3 can then be displayed to a driver of the vehicle 1 on the display device 9.Furthermore, the vehicle 1 comprises at least one speed sensor 11, by means of which a measured value describing a speed of the vehicle 1 can be ascertained continuously. The continuously determined measured values can be transmitted to the control device 4 in this case in particular via the communication connection 10. The speed sensor 11 may be configured as a wheel speed sensor or as any other type of speed sensor, for example.The control device 4 is designed to carry out an exemplary embodiment of a method for operating the image recording device 2. FIG. 2 shows a block diagram of an exemplary embodiment of the method.In a first step S 1 of the method, image data is recorded using at least one of the cameras 3 of the vehicle 1. The camera 3 can be, for example, the camera 6 arranged as a rear view camera. The image data is transmitted from the camera 6 to the control device 4 via the communication link 10. The image data comprise, for example, a continuous series of camera images which reproduce a partial region of the vehicle environment, more precisely the partial region behind the vehicle 1.In step S 2 of the method, the control device 4 generates an environmental view from the image data. For this purpose, the image data are projected onto a virtual projection surface. The projection is therefore a computing operation and not a projection in the physical sense. During the projection, the image data or the individual image points of at least a part of the image data generated by the camera are imaged onto the projection surface by means of an assignment rule, as is described in more detail below with reference to FIGS. 3 to 5.In step S 3 of the method, the generated environmental view is finally displayed on the display device 9 of the vehicle 1. The surrounding view can be displayed in particular in an updated manner several times per second, so that at least substantially a live reproduction of the surrounding area of the vehicle 1 in the form of a video results. For this purpose, the control device 4 can continuously calculate the environmental representation or an image of the environmental representation from the camera images, or from at least a part of the camera images, of the cameras 3 as image data and transmit it to the display device 9.FIG. 3 schematically shows different projection surfaces 12, 13, 14. The projection surfaces 12, 13, 14 differ in their geometry. In a virtual space defined, for example, by a vehicle-fixed coordinate system 15, the projection surfaces 12, 13, 14 are arranged relative to a virtual vehicle model 16. With the aid of the projection surfaces 12, 13, 14, the environmental representation can be generated starting from a virtual camera position 17, which can correspond to the actual position of the camera 6 in the coordinate system 15 or can differ from this position.The projection surface 12 is a planar projection surface which does not have curved edges. In other words, both the edges extending in the y direction corresponding to the vehicle width direction and the edges extending in the z direction corresponding to the vehicle height direction are straight.The projection surface 13 is a cylindrically curved projection surface which has two circular arc segment-shaped edges in the xy-plane. The edges extending in the z-direction are likewise straight, such that the projection surface 13 has the shape of a cylinder jacket segment.The projection surface 14 is an elliptical projection surface which has two elliptically curved edges in the xy plane, i.e. two edges with the shape of an ellipsoidal arc segment. Here too, the edges are straight in the z direction, so that a surface bent in the x direction and in the y direction results. The image data of the camera six are projected, for example, onto the side of the respective projection surface 12, 13, 14 facing the virtual vehicle model 16 or the concavely curved side.Depending on the speed of the vehicle 1 determined by the speed sensor 11, for example, when the environmental view is generated by the control device 4, a switch is made between at least two different projection methods for the projection of the image data. The projection methods differ in this case by the use of differently shaped projection surfaces 12, 13, 14, as described above with reference to FIG. 3, for example.At low speeds, a projection method is used to image a larger horizontal field of view of the camera 3 than at higher speeds. This projection method may be a cylindrical projection using, for example, the cylindrically curved projection surface 13. Alternatively, an elliptical projection, which uses, for example, the elliptically curved projection surface 14, can also be used.At higher speeds, a projection method with less distortion than at the low speeds is used. Preferably, at higher speeds, a planar projection utilizing the planar projection surface 12 is employed.In FIG. 4, an environmental view 18 is exemplarily shown, which uses a cylindrical projection as a projection method for imaging the image data. For comparison, in FIG. 5, the environmental view 18 is shown using a planar projection. It can be seen that by using the cylindrical projection, the field of view represented in the surrounding view 18 is greater than when using the planar projection. In contrast, the surrounding view 18 has less distortions when using the planar projection than when using the cylindrical projection.Preferably, at a speed of the vehicle 1 below a limit value, the cylindrical or elliptical projection can be used as a first projection method. At a speed above the threshold, the planar projection may be used as a second projection method. Alternatively, the second projection method can also be used only at a speed above a further limit value, which differs from the limit value.The limit value or the further limit value is in each case a speed limit value which can in each case be, for example, 7 km / h, 30 km / h, 60 km / h, 90 km / h, 120 km / h, 130 km / h or the like. When using a further limit value, this is in particular higher than the limit value. It is possible that a speed of 0 km / h is also used as a limit value.By using cylindrical or elliptical projection for the low speeds, in particular when maneuvering the vehicle to parking spaces or the like, an increased field of view is provided, thereby facilitating maneuvering in particular in a narrow space. At higher speeds, a lower angle of view is sufficient, for example for a representation of the environment in front of or behind the vehicle. By using the planar projection, the distortion can be reduced, which enables a more realistic reproduction of the vehicle environment and thereby facilitates a quick perception of the vehicle environment by the driver, in particular at higher speeds.In order to avoid abrupt changes in the representation of the surrounding view 18 on the display device 9, an in particular continuous transition between the projection methods is preferably used for the generation of the surrounding view 18 when the vehicle speed exceeds or falls below the limit value or at speeds between the limit value and the further limit value.The continuous transition between the two projection methods can be generated, for example, by calculating and displaying intermediate images of the surrounding view 18 between the first projection method and the second projection method. In other words, the intermediate images can correspond to a projection by means of a projection surface which is an intermediate shape between the projection surface of the first projection method and the projection surface of the second projection method.With an increasing speed of the vehicle 1, a transition from the cylindrical projection or an elliptical projection to the planar projection can take place, for example, when the limit value for the vehicle speed is reached. For this purpose, after the limit value has been exceeded or at speeds which are located between the limit value and the further limit value, intermediate images can be calculated by the control unit 4, which images represent the transition between the cylindrical projection and the planar projection in particular continuously. In this case, for example, the effect of a successively changing curvature of the projection surface can be added to the surrounding view 18 subsequently on the basis of a computing rule in the case of images generated by means of the first or the second projection method. The display device 9 then displays a changing environmental view to the user of the vehicle 1, in which the type of display changes continuously or in which morphing takes place between the different projection methods.As an alternative to the generation of the intermediate images, it is possible for the geometry of the projection surface 12, 13, 14 to be adapted, so that, for example, with an increasing vehicle speed, a smooth profile results from the geometry of the cylindrical or elliptical projection surface 13, 14 of the first projection method to the planar geometry of the projection surface 12 of the second projection method. Conversely, with a decreasing vehicle speed, a transition can take place from the projection surface 12 of the second projection method to the projection surface 13, 14 of the first projection method. The respective change in the geometry of the projection surface 12, 13, 14 also changes the environmental representation derived therefrom, so that the effects caused by the change in the projection method, such as the larger field of view, for example, continuously increase.The limit value and / or the further limit value can have the same values or different values for the acceleration and for the deceleration of the vehicle 1, or for increasing speeds and decreasing speeds of the vehicle 1.In addition to using the image data of the camera 6 arranged as a rear view camera, it is possible to also use the image data of a plurality of the cameras 3. In particular, the image data from two or more adjacently arranged cameras 3 or cameras 3 at least partially overlapping capture regions can be merged and used to generate an environmental representation.References included in the specificationThis list of documents cited by the applicant has been produced in an automated manner and is only included for the better information of the reader. The list is not part of the German patent application or utility model application. The DPMA does not take any adhesion for any faults or omissions.Patent Literature citedUS 2019 / 0001887 A1

[0004] US 2015 / 0350607 A1

[0006]

Claims

Method for operating an image recording device (2) of a vehicle (1), wherein the image recording device (2) comprises at least one camera (3) and at least one display device (9), wherein the camera (3) acquires image data from a partial region of a vehicle environment and an environment view (18) generated from the image data is displayed on the display device (9), wherein the environment view (18) is generated from the image data by means of a projection of the image data, characterized in that a switch is made between at least two different projection methods for the projection of the image data depending on a speed of the vehicle (1), wherein the different projection methods each use a differently shaped projection surface (12, 13, 14) for the projection of the image data.Method according to claim 1, characterised in that a projection method is used at lower speeds for imaging a greater horizontal field of view of the camera (3) than at higher speeds.Method according to claim 1 or 2, characterized in that a projection method with a lower distortion of the image data than at lower speeds is used at higher speeds.Method according to one of the preceding claims, characterized in that a first projection method is used at a speed below a limit value and a second projection method is used at a speed above the limit value or above a further limit value.Method according to Claim 4, characterized in that, if the vehicle speed exceeds or falls below the limit value or if speeds between the limit value and the further limit value are used, an, in particular continuous, transition between the projection methods is used for generating the surrounding view.Method according to Claim 4 or 5, characterized in that, when the vehicle is accelerating, a different limiting value and / or a different further limiting value is used than when the vehicle is decelerating.Method according to one of the preceding claims, characterized in that a cylindrical projection with a cylindrically curved projection surface (13) or an elliptical projection with an elliptically curved projection surface (14) is used as the or a first projection method, and a planar projection with a planar projection surface (12) is used as the or a second projection method.Method according to one of the preceding claims, characterized in that a front camera, a side camera and / or a rear camera of the vehicle (1) is used as at least one camera (3).Control device which is configured to execute a method according to one of the preceding claims when image data is provided to at least one camera (3).Image recording device for a vehicle, comprising at least one camera (3), at least one display device (9) and a control device (4) according to Claim 9.Vehicle comprising an image recording device (2) according to claim 10.Computer program comprising instructions which configure a control device (4) to execute a method according to one of Claims 1 to 8 on the basis of provided image data of a camera (3).

Images