Method and camera system for generating defined output images for further processing

Abstract

The invention relates to a method for generating an output image (A) of a camera system (2), which camera system (2) is configured to generate an input image (E), a camera system (2) for carrying out the method according to the invention, a vehicle (1) comprising a camera system (2) according to the invention, a computer program, and a computer program product. The method according to the invention comprises the following process steps: - Providing at least one input image (E), and - Generating an output image (A) based on the input image (E), wherein the output image (A) is generated such that it has at least one predefinable image parameter (p).

Description

[0001] The present invention relates to a method for generating an output image of a camera system, a camera system which is suitable for carrying out the method according to the invention, a vehicle with a camera system according to the invention, as well as a computer program and computer program product.

[0002] Today's advanced driver-assistance systems (ADAS) offer the driver a wide variety of ADAS functions, which can be used to support the driver while maintaining control of the vehicle. Depending on the level of automation, fully automated driving can also be achieved. Examples of such ADAS functions include various methods for detecting objects or obstacles on the road, methods for detecting lane boundaries and / or keeping the vehicle in a lane, methods for detecting rain on the windshield, or methods for assisting with or executing a parking maneuver. These and other functionalities are often based, at least in part, on images captured by cameras attached to the vehicle.

[0003] Three-dimensional detection of the vehicle's surroundings is a necessary prerequisite for many common ADAS functions. Sensor systems used for this task include, for example, camera systems with one or more cameras for detecting the vehicle's surroundings. In addition to mono cameras, camera systems with multiple cameras, such as stereo camera systems, are often used.

[0004] Suitable image processing is a key aspect in providing the various ADAS functions using images of the vehicle's surroundings captured by a camera system. In order to capture as much information as possible about the vehicle's surroundings, even at great distances from the vehicle, the camera systems used are equipped with cameras with the highest possible resolution and maximum light sensitivity. However, it is equally important that the camera systems used can reliably capture the vehicle's surroundings even in poor visibility conditions, particularly in bad weather or at night. In this respect, a high dynamic range and high light sensitivity are crucial. In this regard, it has become known, for example, to combine color pixels and panchromatic or clear pixels in image generation. The panchromatic pixels are particularly light-sensitive but do not provide any color information.

[0005] However, most of the approaches mentioned to increase image quality and thus improve the detection of the vehicle's surroundings lead to ever larger amounts of data that must be processed and transmitted by the camera systems.

[0006] Therefore, the object of the present invention is to provide a possibility for reducing the amount of data to be transmitted without loss of information.

[0007] This object is achieved by the method according to claim 1, the camera system according to claim 10, the vehicle according to claim 11, the computer program according to claim 12 and the computer program product according to claim 13.

[0008] With regard to the method, the object underlying the invention is achieved by a method for generating an output image from a camera system, which camera system is configured to generate an input image. The method comprises the following method steps: - Providing at least one input image, and - generating an output image based on the input image, wherein the output image is generated such that it has at least one predefinable image parameter.

[0009] Based on at least one input image from at least one camera of a camera system, a suitable output image with at least one predeterminable image parameter is generated. The output image can then be used for further, particularly downstream, processing and, for this purpose, forwarded, for example, to a central processing unit, in which processing unit various ADAS functions or other algorithms based on input information in the form of image data can be provided. The method according to the invention is advantageously carried out within the camera system, so that only the generated output images need to be transmitted.

[0010] According to an advantageous embodiment, the image parameter is an image resolution, a color space, a contrast, an image brightness, a dynamic range, a color saturation, or an image error rate, in particular a motion blur or signal noise. The output image can thus be suitably preprocessed and optimally adapted for subsequent image analysis.

[0011] An advantageous embodiment includes the image resolution of the at least one input image being greater than the image resolution of the output image. In this case, the at least one image parameter is therefore the image resolution. In addition to the image resolution, however, at least one further image parameter can also be specified for the output image. Specifying a specific image resolution that is lower than that of the input image leads to a significant reduction in the amount of data to be transmitted and processed.

[0012] It is advantageous if adjacent image elements, in particular pixels, adjacent image regions, in particular image sections comprising a plurality of pixels and / or image data from adjacent image sensors, in particular with different light sensitivities or different detectable frequency ranges, are combined. These image elements, image regions or image data are preferably combined into smaller units, ie, for example, a plurality of image elements are each combined to form a image element, a plurality of image regions are combined to form one image region or image element and / or the image data from a plurality of image sensors are combined to form one image or, in the case of at least partial overlap of the detection areas of the different image sensors, to form individual images, image regions or image elements. In this regard, the method of binning may be mentioned as an example. However, the invention is by no means limited to combination by binning.

[0013] According to an advantageous embodiment of the method according to the invention, the output image is generated such that it has an image brightness that differs from the at least one input image, at least in partial areas. It is thus possible to brighten the image both in partial areas of the at least one input image and in the entire input image. This allows the respective lighting conditions to be specifically addressed when capturing the input images using the camera system.

[0014] According to a further advantageous embodiment, the output image is generated based on at least two input images, wherein the at least two input images differ from one another in particular with regard to at least one recording parameter. The at least two input images can be images from the same camera taken at different times, or images from at least two different cameras of the camera system. When generating the output image based on the at least two input images, a wide variety of options are conceivable, all of which fall within the scope of the present invention. For example, different input images can be used for different image elements or image regions of the output image. A weighted combination of the input images is also conceivable, for example.The specific weighting can also be selected differently for different image elements or image areas.

[0015] In this respect, it is advantageous if at least one recording parameter is an exposure time, a grain, or a sensitivity or gain.

[0016] In one embodiment of the method according to the invention, different image elements and / or image regions are provided with at least partially different weighting factors during the generation of the output image. A wide variety of variants are also conceivable in this context and fall within the scope of the present invention. On the one hand, different image elements or image regions can be emphasized by specific weighting. It is also possible to use different weightings when combining neighboring image elements, image regions, and / or image data.

[0017] In this respect, it is advantageous if the weighting factors are made available together with the output image, for example, transmitted together to the central processing unit.

[0018] The object underlying the invention is further achieved by a camera system comprising at least one camera, in particular with an optical unit and an image sensor, and a computing unit, wherein the computing unit is designed to carry out the method according to the invention. The camera system therefore comprises at least one camera with an optical unit and an image sensor as well as a, in particular local, computing unit for carrying out the method according to the invention. The generation of the appropriate output image therefore takes place locally in the camera system. The output image can then, in a further step, be forwarded to a central computing unit for suitable further processing of the information it contains, for example to provide various ADAS functions. In this way, a suitable output image can be generated independently of the respective optical units and image sensors used.This has the advantage that the processes implemented in the central processing unit do not need to be specifically adapted to the respective camera system. Instead, the adaptation takes place directly and locally within the camera system.

[0019] The present invention can generally be used for any image capture using cameras. A preferred application relates to the automotive sector, and within this, in particular, to the field of driver assistance systems. Thus, the object underlying the invention is also achieved by a vehicle comprising a camera system according to the invention.

[0020] Furthermore, the object underlying the invention is achieved by a computer program with instructions which, when the computer program is executed by a computer, cause the computer to carry out the method according to the invention and by a computer program on which the computer program according to the invention is stored.

[0021] The invention and its advantageous embodiments are explained in more detail with reference to the following figures. They show: Fig. 1: a vehicle with a camera system according to the invention; and Fig. 2: a schematic flow diagram for the method according to the invention.

[0022] In the figures, identical elements are each provided with the same reference numeral.

[0023] In Fig. 1 illustrates a vehicle 1 with a camera system 2 according to the invention. In this embodiment, the camera system 2 has a camera 3 with an optical unit 4 and an image sensor 5. The camera system 2 further has a computing unit 6 for carrying out the method according to the invention. In the present case, this is a camera 3 installed towards the front, for example, inside the vehicle in front of the windshield. In other embodiments, the respective camera system 2 can also be arranged at other positions relative to the vehicle 1, and several cameras 3 arranged at the same position or at different positions can also be provided.

[0024] In Fig. 2a, the method according to the invention is illustrated by means of a flowchart. Based on or based on one or more input images E or input images E1-E captured by the camera system 2 nAn output image A with at least one predeterminable image parameter p is generated, for example in the processing unit 6. The image parameter can be, for example, an image resolution a. Multiple image parameters p can also be specified.

[0025] Preferably, this is specified in such a way that the output image A has a lower image resolution a than the input image E or the input images E1-E n , i.e. a A >a E . It is an advantage of the present invention that a reduction of the data volume, ie the file sizes of the output images A compared to the input images E, is possible without loss of the relevant information in the data.

[0026] In Fig.2b illustrates an embodiment in which adjacent image areas b1-b4 of the input image E are each combined to form an image area b in the generated output image A. Alternatively or additionally, in the case of several input images E1-E n to generate the output image A, also image areas b of the various input images E1-E n The different image areas b1-b4 can also optionally be assigned at least partially different weighting factors g1-g4.

[0027] Instead of image areas b, individual image elements, especially pixels, can also be combined. The latter is advantageous, for example, when a combination of color pixels and panchromatic pixels is used for the input images E. In this case, the panchromatic pixels do not provide any new information but are used primarily due to their high light sensitivity.

Claims

[1] Method for generating an output image (A) of a camera system (2), which camera system (2) is designed to generate an input image (E), the method comprising the following method steps: - providing at least one input image (E), and - generating an output image (A) based on the input image (E), wherein the output image (A) is generated such that it has at least one predefinable image parameter (p). [2] Method according to claim 1, wherein the image parameter (p) is an image resolution (a), a color space, a contrast, an image brightness, a dynamic range, a color saturation, or an image error rate, in particular a motion blur or a signal noise. [3] Method according to claim 1 or 2, wherein an image resolution (a E ) of the at least one input image (E) is greater than an image resolution (a A ) of the output image (A). [4] Method according to one of the preceding claims, wherein adjacent image elements, adjacent image areas (b1-b4) and / or image data of adjacent image sensors, in particular with different light sensitivity or different detectable frequency range, are combined. [5] Method according to at least one of the preceding claims, wherein the output image (A) is generated such that it has an image brightness which differs from the at least one input image (E) at least in partial areas. [6] Method according to one of the preceding claims, wherein the output image (A) is generated based on at least two input images (E1-E n ), and wherein the at least two input images (E1-E n ) differ from each other, particularly with regard to at least one recording parameter. [7] The method of claim 6, wherein the at least one recording parameter is an exposure time, a grain, or a sensitivity. [8] Method according to one of the preceding claims, wherein different image elements and / or image areas (b1-b4) are provided with at least partially different weighting factors (g1-g4) during the generation of the output image. [9] Method according to claim 8, wherein the weighting factors (g1-g4) are provided together with the output image (A). [10] Camera system (2) comprising at least one camera (3), in particular with an optical unit (4) and an image sensor (5), and a computing unit (6), wherein the computing unit (6) is designed to carry out the method according to at least one of the preceding claims. [11] Vehicle (1) comprising a camera system according to claim 10. [12] A computer program comprising instructions which, when the computer program is executed by a computer, cause the computer to carry out the method according to any one of claims 1-9. [13] A computer program product on which the computer program according to claim 12 is stored.

Images