METHOD FOR DETECTING AND CORRECTING IMAGE ERROR, AS WELL AS IMAGE PROCESSING UNIT AND COMPUTER PROGRAM FOR THIS PURPOSE

Description

[0001] The invention relates to a method for detecting and correcting image errors caused by flickering light information in images generated by an image sensor.

[0002] The invention further relates to an image processing unit and a computer program for carrying out the method.

[0003] Many light sources today operate in pulsed mode. When switched on, the light source alternates rapidly between an on and off state, or its brightness changes rapidly. Such pulsed light sources can be, in particular, pulsed light-emitting diodes (LEDs), which have become extremely widespread in many areas in recent years. The flickering associated with pulsed operation is not visible to the human eye due to its high frequency. Therefore, when the light source is switched on, the desired impression is created for the human observer that the light source is continuously switched on.

[0004] To capture and record images or image sequences containing light emitted by LEDs or other pulsed light sources, image sensors are typically used whose sampling frequencies are significantly lower than the pulse frequencies of the pulsed light sources themselves. The latter can be many times higher than the sampling frequency. Furthermore, the exposure times of the image sensors are generally much shorter than the periods of the pulsed light sources. This is particularly relevant in the automotive sector, where equipping vehicles with simple and cost-effective image sensors is desirable.

[0005] This can lead to the image sensor capturing an image showing the pulsed light source in the off state, even though the light source is switched on. When capturing image sequences, the image sensor may capture long sequences of images that exclusively show the light source in the off state, even though it is switched on. The playback of the image sequence captured by the image sensor therefore creates the impression that the captured light source is switched off, even though it is actually switched on. The captured images or image sequences thus contain image errors caused by flickering light information.

[0006] Flickering light information within the meaning of the present invention is understood to mean that the light source periodically changes its brightness, as is the case when controlling LED light sources or other pulsed light sources. Such flickering of the light source or of the light information it emits is also referred to as flicker. For the purposes of this application, the terms flicker and flicker are therefore used synonymously.

[0007] For the purposes of this application, the term exposure time is used synonymously with the term integration time, which is particularly common in the field of semiconductor image sensors.

[0008] Of particular practical importance is the previously described problem of capturing and recording images or image sequences containing light emitted by pulsed LEDs. Since brake and turn signal lights in modern vehicles also use pulsed LEDs, image errors of the type described above can create unacceptable safety risks when image sensors are used in the automotive sector. This is especially critical when an electronic camera takes over the function of the rearview mirror in a vehicle and the recorded image is displayed on a screen after image processing. In such cases, for example, a turn signal from a nearby vehicle might not be displayed at all or not in time.

[0009] Cameras integrated into motor vehicles, i.e., vehicle cameras, are used for a variety of tasks. They serve, for example, as front cameras, rear cameras (including reversing cameras), or side cameras for the driver to monitor the vehicle's surroundings. They also function, in the form of monocular or stereo cameras, as optical sensors used to collect sensor data for driver assistance systems such as emergency braking systems, adaptive cruise control, lane keeping assist, and other assistance systems. In the context of such vehicle-integrated cameras, incomplete and / or faulty reproduction of image sequences containing flickering light information can lead to an increased risk of accidents.

[0010] Avoiding image errors caused by pulsed LEDs is complicated by the fact that commercially available pulsed LEDs operate at a wide variety of pulse frequencies. Particularly in the vicinity of vehicles, pulsed light sources with a broad spectrum of different pulse frequencies are now commonly observed, and all light sources within a single image must be captured by the image sensor. Therefore, synchronizing the image sensor's sampling intervals with the LED's on-times is not a viable solution to this problem.

[0011] From US patent 2004 / 0012692 A1, a method for flicker detection is known in which images are captured with two or more different exposure times, the longer of the exposure times being sufficiently long to prevent any flicker-related image defects. By comparing the images captured with different exposure times, it is determined whether or not flicker is present. If flicker is detected, flicker compensation is performed by selecting the exposure time to be an integer multiple of the flicker period, i.e., the reciprocal of the pulse frequency of the pulsed light source.

[0012] The method known from US 2004 / 0012692 A1 yields good results if, firstly, the exposure time can be sufficiently long and, secondly, the pulse frequency of the pulsed light sources is known and uniform. However, these conditions are not met when images or image sequences containing light emitted by pulsed LEDs are to be captured with simple and inexpensive image sensors. This is because the exposure times of such image sensors are short and can only be extended to a limited extent without overexposure. This is due to the fact that simple and inexpensive image sensors, such as those used particularly in the automotive sector, do not have an aperture to adjust the light transmission to the exposure time.Furthermore, the pulse frequencies of the pulsed LEDs in the vicinity of the image sensor, particularly in the vicinity of a vehicle, are unknown and, moreover, not uniform, but can exhibit completely different values ​​across a wide spectrum. Therefore, the method known from US 2004 / 0012692 A1 cannot provide satisfactory results when capturing and recording images or image sequences that include light emitted by pulsed LEDs. Other methods for flicker detection are known from US 2014 / 0153839 A1 and US 2015 / 0253198 A1.

[0013] The object of the present invention is therefore to provide an improved method for detecting and correcting image defects caused by flickering light information, which is suitable for images with short exposure times and allows detection and correction of image defects caused by pulsed LEDs.

[0014] The problem is solved by the method with the features of claim 1, by the image processing unit configured for carrying out the method with the features of claim 16, and by the computer program with the features of claim 17. Advantageous embodiments are described in the dependent claims.

[0015] To detect and correct image errors caused by flickering light information in images generated by an image sensor, it is proposed to read in an exposure series comprising at least one reference image and at least one comparison image, i.e., at least two images. According to the invention, the exposure time with which the at least one reference image was taken is longer than the exposure time with which the at least one comparison image was taken; i.e., the exposure time of the reference image is longer than the exposure time of the comparison image.

[0016] The exposure series can, for example, be captured by an image sensor and temporarily stored in a data storage device. Such an image sensor could be, for example, a video camera, especially a digital video camera, another type of camera, or a component of such a camera.

[0017] According to the invention, several comparison images can be read in, which have been produced with different exposure times, each shorter than the exposure time of the reference image. For example, it is conceivable that the exposure time of the reference image is long, while the exposure time of one comparison image is short, and the exposure time of another comparison image is very short. The number of comparison images with different or identical exposure times is, in principle, unlimited.

[0018] According to the invention, it is necessary that at least one reference image is read in, the exposure time of which is longer than the exposure time of the at least one comparison image. The image of the read-in exposure series that was produced with the longest exposure time, wherein this time is longer than the exposure time of at least one further image, is a reference image within the meaning of the present invention. The image of the read-in exposure series that was produced with the shortest exposure time, wherein this time is shorter than the exposure time of at least one further image, is a comparison image within the meaning of the present invention.

[0019] According to the invention, the images in the exposure series show the same scene. A scene is understood to be a section of the visible environment of the image sensor at a specific time. The same scene is understood to be an identical or approximately identical section of the visible environment at an identical or approximately identical time.

[0020] According to the invention, the different images of an exposure series can either have been taken simultaneously, for example with the help of a split-pixel sensor, or they can have been taken one after the other at very short intervals.

[0021] In a further step of the method, according to the invention, local deviations of the image data of the comparison image from the image data of the reference image are determined by comparing the image data of the comparison image with the image data of the reference image at a specific location. Local deviations are understood to be deviations at a specific position within the image.

[0022] Local comparison refers to comparing image data at a specific location within the image. This local comparison, and / or the resulting local deviations of the image data, can relate to the positions of individual pixels when comparing individual pixels. This achieves particularly high accuracy. However, larger areas of the image, such as image blocks, can also be compared. In this case, the local comparison and / or the resulting local deviations of the image data relate to the positions of image blocks within the image. This reduces the computational and storage requirements associated with determining the local deviations. It is also possible to compare entire images, which can further reduce the computational and storage requirements.Local deviations can therefore be determined, for example, in pixel-accurate, block-accurate, or image-accurate resolution.

[0023] Due to the longer exposure time of the reference image, the image data of the reference image generally differs from the image data of the comparison image, even though both images depict the same scene. In particular, the reference image typically exhibits greater brightness compared to the comparison image because of its longer exposure time. To compare the image data of images taken with different exposure times, the image data can be normalized, for example, based on absolute brightness and / or a specific exposure time. This will be discussed in more detail later.

[0024] According to the invention, by determining local deviations by comparing images taken with different exposure times, the presence of flickering light information, i.e., the presence of flicker, is detected. The flickering light information can be in the form of light emitted by pulsed light sources, e.g., LEDs. The probability that a captured image shows the off state of a pulsed light source, even though the light source is switched on, decreases with increasing exposure time. Therefore, by using a longer exposure time for the reference image, it can be achieved that the reference image at least partially shows the on state of the pulsed light source, while the comparison image, taken with a shorter exposure time, shows the off state of the same pulsed light source.Based on the resulting local deviations between the image data of the reference image and the image data of the comparison image, flickering light information and the resulting image errors are detected according to the invention.

[0025] To reliably detect flickering light information and / or the resulting image defects, the exposure time of at least one reference image can be significantly longer than the exposure time of at least one comparison image. This advantageously ensures that the reference image, taken with a longer exposure time, shows at least part of the pulsed light source in its on state with sufficient probability, while the comparison image shows the off state of the same pulsed light source.

[0026] In order to reliably detect flickering light information and / or image errors caused by it, the exposure time of the reference image can advantageously be chosen to be so long that the on state of the switched-on pulsed light sources, e.g. switched-on LEDs, is highly likely - or even in every case - to fall completely or at least partially within the exposure time and thus be captured by the reference image.

[0027] For example, the exposure time of the reference image can be chosen so that it corresponds at least to the reciprocal of the lowest pulse frequency of the pulsed light sources expected in the scene. Then the on state of the switched-on pulsed light sources falls entirely within the exposure time of the reference image and is thus reliably captured by the reference image.

[0028] Furthermore, the exposure time of the reference image can be chosen to be at least 50% of the reciprocal of the image acquisition frequency (e.g., 10 ms at 50 Hz). This ensures that the on-state of the switched-on pulsed light sources falls partially (e.g., at least every second pulse at a 100 Hz pulse frequency) within the exposure time of the reference image and is thus reliably captured by the reference image.

[0029] However, a comparison image taken at the same or nearly the same time, but with a shorter exposure time, may show the off state of a pulsed light source due to the problem explained above, even though the light source is switched on.

[0030] In such cases, the reference image exhibits significantly greater brightness at the positions showing an activated pulsed light source or the light it emits than the shorter-exposure comparison image, which shows the activated pulsed light source in its off state. Therefore, a spatial comparison of the comparison image data with the reference image data reveals clear deviations that exceed those expected solely due to the different exposure times. Similarly, the normalized image data also exhibits local deviations, which are discernible by comparing the normalized image data and indicate the presence of flickering light information.

[0031] By determining local deviations, it is thus determined whether, to what extent and at which positions there are deviations between the image data of the comparison image and the image data of the reference image.

[0032] In a further step of the process, an error mask is generated based on the determined local deviations of the image data. This error mask displays the positions of image defects within the comparison image.

[0033] The error positions of image defects within the comparison image can, for example, refer to the positions of individual pixels in the image, resulting in particularly high accuracy of the defect mask. Alternatively, the error positions can refer to the positions of larger areas of the image, such as the positions of image blocks. This reduces the computational and storage requirements associated with generating the defect mask. The defect mask can also simply indicate whether the entire image contains defects or not. This further reduces the computational and storage requirements. The defect mask can therefore be generated, for example, at pixel-accurate, block-accurate, or frame-accurate resolution.

[0034] To generate the error mask based on the determined local deviations, it can be checked whether these deviations are so large that they can be assumed to have been caused by a pulsed light source. This is particularly the case if the deviations are significantly larger than would be expected solely based on the different exposure times or light sensitivities of the reference image and the comparison image. If the local deviations are so large that they can be assumed to have been caused by a pulsed light source, the position of the local deviation is recorded as the error position in the error mask. In addition to the exposure time, the sensor sensitivity setting plays a role here. For sensors with different cells for the reference image and the comparison image, their relative sensitivity, i.e.,The relationship between their sensitivities plays a role.

[0035] In this way, the defect mask displays the positions of image defects within the comparison image. If the defect mask is generated at pixel-perfect resolution, which advantageously achieves particularly high accuracy, it can display the position of each detected defective pixel. The defect mask can be structured, for example, to include an entry for each pixel position within the comparison image, with a status indicator (flag) for each entry showing whether the pixel contains a defect or not.

[0036] After generating the error mask according to the invention, it is thus known at which positions the at least one comparison image exhibits image defects caused by flickering light information. On this basis, it is possible to correct the image defects of the comparison image.

[0037] For this purpose, the invention provides that local correction values ​​are determined for the image data of the reference image. This can advantageously be done, in particular, by estimation. Each local correction value is assigned to a correction position within the reference image, wherein the correction positions can have a pixel-accurate, block-accurate, or frame-accurate resolution. The assignment of the local correction value to a correction position within the reference image allows the correction of an image defect in the reference image at a specific defect position, which corresponds to the correction position.

[0038] To determine local correction values ​​by estimation, the image data of the reference image can be used. Using the reference image data is advantageous in this regard because it can be assumed that, in the case of an image error caused by a pulsed light source captured in the off state, the comparison image will show the pulsed light source in the on state. This will be discussed in more detail later.

[0039] Alternatively or additionally, to determine local correction values ​​by estimation, image data from other reference images can be used, particularly those acquired at different times, especially earlier times, than the current reference image whose image errors are to be corrected. Using image data from other reference images is advantageous in this regard because reference images acquired at other times, despite their shorter exposure time compared to the reference image, may have accidentally captured the on state of the pulsed light source. This will be discussed in more detail later.

[0040] According to the invention, local correction values ​​are thus known which are each assigned to a correction position within the comparison image and can be used to correct the comparison image if necessary, i.e., if the error mask indicates the presence of image errors within the comparison image.

[0041] For this purpose, the invention provides that a corrected comparison image is generated by inserting the local correction values ​​at the error positions displayed by the error mask. For each error position displayed by the error mask, the local correction value is used whose associated correction position corresponds to the error position. In this way, the respective erroneous image data of the comparison image can be replaced by the correction value, and the image error can thus be corrected.

[0042] The result is a corrected comparison image in which the image errors caused by flickering light information have been selectively corrected, i.e., at the error positions affected by the image errors, using correction values. Thus, the method according to the invention can provide a flicker-reduced or even a completely flicker-free image despite the presence of flickering light information.

[0043] In an advantageous embodiment of the invention, the image data are brightness values. In this case, brightness values ​​are used for spatial comparison of the image data and / or the spatial deviations of the image data relate to brightness values ​​and / or the spatial correction values ​​relate to brightness values. A brightness value in this sense can, for example, be the luminance or luma value of digital or analog image data, preferably digital image data.

[0044] Brightness values ​​in this sense can be, in particular, the direct color components (RGGB) of a Bayer sensor. Brightness values ​​in this sense can also be the Y-value in the YCbCr color model or a Y-value in the YUV color model. Brightness values ​​in this sense can also be the values ​​of the color components R, G, and B after debayering a Bayer sensor. A brightness value in this sense can also be any other brightness value. For the purposes of the inventive method, the brightness values ​​can also be calculated from the raw data of the images in the acquired exposure series, for example, from such raw data as provided by an image sensor at its output interface.

[0045] The use of brightness values ​​as image data for the purposes of the method according to the invention offers the advantage that it is particularly easy to detect and correct image errors caused by flickering light information based on the brightness values. This is because the change of a light source from an on state to an off state or a change in the brightness of the light source is directly reflected in the brightness values ​​and thus in the image data.

[0046] In a further advantageous embodiment of the invention, the image sensor is a high-dynamic-range image sensor, or HDR image sensor for short, and the exposure series is a high-dynamic-range exposure series, or HDR exposure series, generated by the HDR image sensor.

[0047] HDR image sensors are capable of generating an HDR exposure series through multiple exposures. This series comprises several images acquired simultaneously, e.g., when using a split-pixel sensor, or at very short intervals, so that the images depict the same scene as defined in the present invention. The images in an HDR exposure series are captured by the HDR image sensor with different exposure times to provide a greater dynamic range of brightness compared to a single image.

[0048] This functionality of an HDR image sensor can be advantageously used to provide the exposure series required for carrying out the method according to the invention. The use of an HDR image sensor that generates an HDR exposure series offers the advantage that multiple images of the same scene with different exposure times can be provided quickly and automatically and used as a reference image or comparison image within the meaning of the present application.

[0049] The HDR image sensor can, in particular, be a split-pixel sensor. Image sensors typically have a large number of sensor cells (referred to simply as cells), with each sensor cell being responsible for generating one pixel of the image to be produced. While conventional image sensors have only one sensor cell per pixel, a split-pixel sensor has at least two sensor cells per pixel, with the different sensor cells of each pixel being used to generate different images in the HDR exposure series.

[0050] In this way, for example, it is possible for a split-pixel sensor to generate each pixel of the reference image using the first sensor cell and each pixel of the at least one comparison image using the second sensor cell.

[0051] The light sensitivities (also referred to as sensor sensitivities or simply sensitivities) of the at least two sensor cells of each pixel of a split-pixel sensor can differ. For example, the split-pixel sensor can have one sensor cell with lower light sensitivity and one sensor cell with higher light sensitivity per pixel. Typically, such split-pixel sensors produce a brighter image with a longer exposure time, i.e., a reference image as defined in the present invention, using the sensor cells with higher light sensitivity, and a darker image with a shorter exposure time, i.e., a comparison image as defined in the present invention, using the sensor cells with lower light sensitivity.Such split-pixel sensors therefore have two types of sensor cells for generating the images of the exposure series, with the light sensitivity of the sensor cells of a first type being greater than the light sensitivity of the sensor cells of a second type.

[0052] In particular, it is possible to select which type of sensor cells are used to generate each image in the exposure series. This allows, for example, the selection of whether the reference image is generated with the more sensitive or less sensitive sensor cells, and whether the at least one comparison image is generated with the more sensitive or less sensitive sensor cells. Thus, such image sensors are configured to generate the images in the exposure series with a selectable light sensitivity.

[0053] In a further advantageous embodiment of the invention, the image sensor that generates the images of the exposure series is configured to generate the images of the exposure series with a selectable light sensitivity. The light sensitivity is selectable between at least a first light sensitivity and a second light sensitivity, the second being lower than the first. The method includes an additional step of generating the exposure series with the image sensor, wherein at least the reference image is generated with the second light sensitivity. The reference image is thus generated with a lower light sensitivity.

[0054] This offers the advantage that a long exposure time can be selected for the reference image without the image data in the bright areas of the scene reaching saturation and causing an undesirable loss of dynamic range. The long exposure time also increases the likelihood that the reference image at least partially shows the on state of the pulsed light source, while the comparison image, taken with a shorter exposure time, shows the off state of the same pulsed light source. This improves the reliability with which flickering light information and the resulting image errors can be detected and corrected.

[0055] In a further advantageous embodiment of the invention, the image sensor that generates the images of the exposure series is also configured to generate the images of the exposure series with a selectable light sensitivity. The light sensitivity is again selectable between at least a first light sensitivity and a second light sensitivity, the second being lower than the first. According to this advantageous embodiment, the method comprises two additional steps.

[0056] In an additional step, the light sensitivity used to generate the reference image is selected. This selection depends on the brightness of the scene, i.e., the brightness of the visible area captured by the image sensor. According to the invention, the first light sensitivity (i.e., a higher light sensitivity) is selected when the scene is less bright, and the second light sensitivity (i.e., a lower light sensitivity) is selected when the scene is brighter. Furthermore, the light sensitivity used to generate the at least one comparison image can also be selected. In a further additional step, the exposure series is generated using the image sensor, with at least the reference image being generated using the light sensitivity selected in the previously described step.

[0057] This further development of the method according to the invention offers the advantage that, in a comparatively bright scene, a long exposure time of the reference image can be selected without the image data of the reference image reaching saturation in the bright areas of the scene and thus avoiding an undesirable loss of dynamic range. In this way, the previously described advantages of a longer exposure time of the reference image can be achieved in the case of a comparatively bright scene. Conversely, in the case of a comparatively dark scene, a long exposure time of the reference image can also be achieved in conjunction with a higher sensitivity without the image data of the reference image reaching saturation.Using a higher sensitivity for the reference image is advantageous for relatively dark scenes, as it avoids unwanted image noise that could occur if the image were generated with a lower light sensitivity. By selecting the light sensitivity used to generate the reference image for the exposure series, as described above, the advantages of lower light sensitivity for brighter scenes and those of higher light sensitivity for darker scenes can be combined.

[0058] In a further advantageous embodiment of the invention, the image sensor can be an HDR image sensor configured as a split-pixel sensor and comprising at least two types of sensor cells for generating the images of the HDR exposure series, wherein the light sensitivity of the sensor cells of a first type is greater than the light sensitivity of the sensor cells of a second type. According to this embodiment, the method comprises an additional step of generating the HDR exposure series with the HDR image sensor, wherein at least the reference image is generated by the sensor cells of the second type.

[0059] Split-pixel sensors, which comprise one sensor cell with lower light sensitivity and one sensor cell with higher light sensitivity per pixel, typically produce a brighter image with a longer exposure time, i.e., a reference image as defined in the present invention, using the sensor cells with higher light sensitivity, and a darker image with a shorter exposure time, i.e., a comparison image as defined in the present invention, using the sensor cells with lower light sensitivity. According to the invention, it is therefore proposed that this assignment be reversed and that the sensor cells with lower sensitivity be used to produce the reference image.

[0060] This, in turn, offers the advantage that a long exposure time for the reference image can be selected without the image data in the bright areas of the scene reaching saturation and resulting in an undesirable loss of dynamic range. In this way, the previously explained advantages of a longer exposure time for the reference image can be achieved.

[0061] In a further advantageous embodiment of the invention, the image sensor can be an HDR image sensor configured as a split-pixel sensor and comprising at least two types of sensor cells for generating the images of the HDR exposure series. Furthermore, it is configured to generate each image of the HDR exposure series using a selectable type of sensor cells. The light sensitivity of the sensor cells of a first type is, in turn, greater than the light sensitivity of the sensor cells of a second type. According to this advantageous embodiment, the method according to the invention comprises two additional steps.

[0062] In an additional step, at least the type of sensor cells used to generate the reference image is selected. This selection is again dependent on the scene brightness, with the first type of sensor cells (i.e., those with higher light sensitivity) being selected in lower scene brightness and the second type (i.e., those with lower light sensitivity) being selected in higher scene brightness. Furthermore, the light sensitivity used to generate at least one comparison image can also be selected. In a further additional step, the HDR exposure series is generated using the HDR image sensor, with at least the reference image being produced by the selected type of sensor cells.

[0063] According to the invention, it is therefore proposed that the previously described assignment, which is usual for pixel sensors, between the images of the exposure series and the types of sensor cells is reversed only in the case of a comparatively bright scene, so that only in the case of a comparatively bright scene are the sensor cells of lower sensitivity used to generate the reference image.

[0064] This advantageous further development of the inventive method makes it advantageously possible, with a split-pixel sensor of the type described above, to generate the reference image with lower light sensitivity in the case of a bright scene and with higher light sensitivity in the case of a dark scene. Thus, when using such a split-pixel sensor to generate the exposure series, the advantages of lower light sensitivity for brighter scenes, namely in particular a longer exposure time, and those of higher light sensitivity for darker scenes, namely in particular the avoidance of image noise, can be combined in the manner described above.

[0065] To determine local deviations, in a further advantageous embodiment of the invention, the image data are normalized and the normalized image data at a position in the comparison image are compared with the normalized image data at the same position in the reference image. In order to compare the normalized image data, a difference can be determined, in particular, between the normalized image data at a position in the comparison image and the normalized image data at the same position in the reference image.

[0066] Due to the longer exposure time of the reference image, the image data of the reference image fundamentally differs from the image data of the comparison image, even though both images depict the same scene. In particular, the reference image exhibits greater brightness compared to the comparison image because of its longer exposure time. Therefore, normalizing the image data is advisable in order to determine local deviations between the image data of the comparison image and the image data of the reference image.

[0067] Image data normalization can refer, for example, to absolute brightness and / or a specific exposure time. When normalizing image data, the exposure parameters selected during image creation and the operating characteristics of the image sensor, such as exposure time, sensitivity, aperture, and / or the camera response curve of the image sensor, can be taken into account.

[0068] If, for example, the exposure parameters and operating characteristics of the image sensor are the same for all images in the exposure series, with the exception of the exposure time, then the images in the exposure series can be normalized to an arbitrarily selectable reference exposure time based on the respective exposure time. For example, if the exposure series includes a reference image L with an exposure time TL, a comparison image S with an exposure time TS = TL / 4, and another comparison image VS with an exposure time TV VS = TL / 16 and if the exposure time TL is selected as the reference exposure time, the image data available as brightness values ​​can be normalized to the reference exposure time by multiplying the brightness values ​​of the comparison image S by four and the brightness values ​​of the comparison image VS by sixteen.

[0069] After normalization, the image data are directly comparable, allowing for the determination of local deviations. Specifically, a difference can be calculated between the normalized image data at a given position in the comparison image and the normalized image data at the same position in the reference image. The difference between the normalized image data at a specific position can thus quantify the local deviations.

[0070] The positions of the normalized image data can be determined at pixel-accurate, block-accurate, or frame-accurate resolution. Advantageously, a difference between the normalized image data of the reference image and the comparison image can be determined based on pixel-accurate positions for each individual pixel of the images. This achieves a particularly high accuracy of the determined spatial deviations. By analyzing the difference in image data at a specific pixel position, the spatial deviation at that pixel position can be precisely quantified.

[0071] In a further advantageous embodiment of the invention, a relative light sensitivity ratio between the light sensitivity of the reference image and the light sensitivity of the comparison image is used to normalize the image data. Alternatively or additionally, an exposure time ratio between the exposure time of the reference image and the exposure time of the comparison image can be used to normalize the image data. A relative light sensitivity ratio is understood to be the ratio of the light sensitivity of one image to the light sensitivity of another image. An exposure time ratio is understood to be the ratio of the exposure time of one image to the exposure time of another image.

[0072] In a further advantageous embodiment of the invention, the determined local deviations of the image data are compared with a threshold value to generate the defect mask. If the local deviations exceed the threshold value, it can be assumed that an image defect caused by flickering light information is present at the position of the local deviation. In this case, the position of the local deviation can be entered or marked as defective in the defect mask, so that the defect position of the image defect is displayed within the comparison image.

[0073] Comparing with a threshold value offers the advantage that generating the error mask in this way can be achieved with low computational effort.

[0074] To determine local correction values ​​for the image data of the comparison image by estimation, the image data of the reference image can be used, for example. For correcting image errors caused by a pulsed light source captured in the off state, using the image data of the reference image is advantageous, since it can be assumed that the reference image, due to its longer exposure time, shows the pulsed light source in the on state. However, due to the longer exposure time, the image data of the reference image generally exhibits greater brightness than the image data of the comparison image, so the image data of the reference image cannot be directly used as correction values ​​for the image data of the comparison image.

[0075] In a further advantageous embodiment of the invention, it is therefore particularly provided that a simple estimation is used to determine the local correction values ​​for the image data of the comparison image, which is based on the image data of the reference image and the ratio between the exposure time of the reference image and the exposure time of the comparison image.

[0076] To determine a local correction value assigned to a specific correction position within the comparison image, the image data, for example, a brightness value, at the position of the reference image corresponding to the correction position is used. If the ratio between the exposure time of the reference image and the exposure time of the comparison image is known, an estimate of the corresponding brightness value of the shorter-exposure comparison image can be calculated from the brightness value of the reference image and the ratio of the exposure times, and this can then be used as the correction value.For example, if the exposure series includes a reference image L with an exposure time TL and a comparison image S with an exposure time TS = TL / 4, then the brightness value of the reference image can be divided by four and this value can be used as a local correction value for the image data of the comparison image at the assigned correction position.

[0077] However, a problem arises because in the bright areas of the scene, the image data of the reference image may already have reached saturation due to the long exposure time. This means that the reference image data may exhibit undesirable clipping effects as a result of the image sensor being overdriven. This particularly affects those areas of the scene containing flickering light information emitted by a pulsed light source, as these areas are very bright. Therefore, when estimating the local correction value from the ratio between the exposure time of the reference image and the exposure time of the comparison image, it is advantageous to apply an additional correction factor. The value of this additional correction factor can be estimated, for example, using the image data from the area surrounding the correction position in the reference image.If the image data of the reference image has already reached the saturation range in a large area around the correction position, it makes sense to choose a large additional correction factor.

[0078] The method of a simple estimation, based on the ratio between the exposure time of the reference image and the exposure time of the comparison image, offers the advantage that the local correction values ​​can be determined with very little computational and storage effort.

[0079] The invention provides that, to determine the local correction values ​​for the image data of the reference image, an estimation is applied that is based on the image data of a reference image time series stored in a data storage device. The reference image time series comprises a number of stored images that were generated by the image sensor with the same exposure time as the reference image, but at different times than the reference image. The stored images of the reference image time series can, in particular, have been generated by the image sensor at earlier times than the reference image. Advantageously, the times at which the various stored images of the reference image time series were generated by the image sensor can also differ from one another.The data storage can in particular be a ring buffer in which, when the data storage is full and / or a fixed number of stored images is reached, the oldest stored image is replaced by a new stored image.

[0080] The comparison image time series, which is used to determine the local correction values ​​for the image data of the comparison image, may also include an unmodified copy or a modified copy of the current, i.e., last read, comparison image itself.

[0081] The comparison image time series can, for example, comprise 30 stored images. The times at which the stored images were generated by the image sensor can be different from one another, in particular immediately or indirectly successive times. Time steps of a certain length can lie between these times.

[0082] The stored images of the comparison image time series can, in particular, be copies of the comparison images from exposure series read in at earlier times for carrying out the method according to the invention, in unchanged or modified form. In particular, an unchanged or modified copy of the comparison image of the read-in exposure series can be stored in the data storage after the exposure series has been read in. In the case of a ring buffer, this newly added comparison image can then replace the oldest previously stored image of the comparison image time series when the data storage is full and / or a predetermined number of stored images has been reached.

[0083] In this way, a comparison image time series is provided in the data storage, comprising temporally successive images produced by the image sensor with the same exposure time as a current comparison image whose image errors are to be corrected using the local correction values.

[0084] Applying an estimation based on the image data of such a comparison image time series stored in a data storage device is useful for determining the local correction values ​​for the image data of the comparison image, since comparison images generated at other times may have captured the on state of the pulsed light source by chance, despite their shorter exposure time compared to the reference image. If the number of stored images of the comparison image time series is chosen to be sufficiently large, it can be assumed that at least one of the stored images of the comparison image time series has captured the on state of the pulsed light source.

[0085] From the image data of one or more stored images in the comparison time series, which captured the on state of the pulsed light source, a suitable local correction value for the image data of the comparison image can therefore be determined. If the image data is available, for example, in the form of brightness values, then the brightness value of a stored image that captured the on state of the pulsed light source can be used as the correction value.

[0086] Applying an estimation method to determine the local correction values ​​for the image data of the reference image, based on the image data of a reference image time series of the type described, stored in a data storage device, offers the advantage of enabling a particularly precise determination of the local correction values. This is because the stored images of the reference image time series were created with the same exposure time as the reference image, so that the image data of the stored images have a brightness that corresponds to the exposure time of the reference image.

[0087] In a further advantageous embodiment of the invention, the images of the comparison time series are subjected to dilation and / or downscaling and / or compression before being stored in the data storage. These operations can be performed in any order and multiple times. For example, downscaling can be performed first, followed by dilation, and then downscaling again. This advantageously reduces both the computational effort associated with dilation and the size of the data storage required for storing the images.

[0088] In these cases, what is stored are therefore not unaltered copies, but rather copies of images that have been modified by dilation and / or downscaling and / or compression. These copies were created by the image sensor at different times than the reference image. The images are thus stored in a modified form. In particular, the stored images of the reference image time series can be advantageously provided by storing a copy of the reference image from the exposure series, modified in the manner described above, in the data storage after the exposure series has been read.

[0089] The term "dilation" is to be understood in the context of image processing. Dilation widens, or dilates, specific areas of the image. According to the invention, it is particularly advantageous to widen bright areas of the image by means of dilation. Widening bright areas offers the advantage that the quality of the local correction values, which are determined based on the stored images of the comparison time series, can be improved, since image data with high brightness are of particular relevance for correcting image errors caused by flickering light information.

[0090] This advantage becomes particularly apparent when a pulsed light source, or the flickering light information it emits, moves relative to the image sensor over time, for example, due to movement of the pulsed light source and / or the image sensor. If, under these circumstances, a local correction value corresponding to a specific correction position is sought for the image data of the comparison image within the image data of the stored images, it may happen that, due to the movement, the appropriate (bright) correction value within the stored image is not found at the correction position, but rather in the vicinity of this position. This effect can be compensated for by widening the bright areas within the stored images that are relevant for correcting the image errors.

[0091] The images in the comparison time series can be advantageously reduced in size by downscaling before being stored in the data storage. This downscaling can include subsampling and / or downsampling and / or subsample. Downscaling the images before storage offers the advantage of easily reducing the storage space required for keeping the comparison images.

[0092] The images in the comparison time series can also be advantageously compressed before being stored in the data storage. Compressing the images before storage offers the benefit of significantly reducing the storage space required for retaining the comparison images.

[0093] In a further advantageous embodiment of the invention, the image data of the stored images of the comparison image time series are motion-compensated with respect to the comparison image. The images of the comparison image time series are thus stored in the data storage in a motion-compensated form. Advantageously, alternatively or additionally, before determining the spatial correction values ​​for the image data of the comparison image, motion compensation of the image data of the stored images of the comparison image time series with respect to the comparison image can be performed. The motion compensation of the image data of the images of the comparison image time series can therefore take place before and / or after storing the images in the data storage. Similarly, the comparison image can also be motion-compensated before determining the spatial correction values ​​with respect to the images of the comparison image time series.Motion compensation can be based in particular on motion estimation using motion vectors.

[0094] The application of motion compensation is advantageous because the times at which the stored images of the comparison time series were generated by the image sensor differ from the times at which at least one comparison image of the imported exposure series was generated. Furthermore, the times at which the various stored images of the comparison time series were generated by the image sensor may also differ from each other.

[0095] Relative movements of an object between the scenes captured by the different images can, due to the different image generation times, lead to the at least one comparison image and the various images of the comparison image time series showing the moving object in different positions. Such a moving object could, for example, be a pulsed light source that shows the comparison image as off, even though the pulsed light source is actually switched on. In this case, the method according to the invention detects a number of image defects at associated defect positions within the comparison image, which must be corrected using local correction values.The accuracy of the local correction values, which are determined from the stored images of the comparison image time series, can be affected by the relative movement of the pulsed light source, since it may happen that the stored images of the comparison image time series show the pulsed light source not at the error position, but at a position offset from the error position due to the movement.

[0096] The application of motion compensation between the stored images of the comparison image time series on the one hand and the comparison image whose image errors are to be corrected on the other hand therefore has the advantage that the negative effects of relative movements can be offset by the motion compensation and the accuracy of the local correction values, which are determined by an estimate from the stored images of the comparison image time series, can be improved.

[0097] In a further advantageous embodiment of the invention, it is provided that, in order to determine the local correction values ​​for the image data of the comparison image, a maximum local brightness value is determined from the image data of the images of the comparison image time series and the maximum local brightness value is used as the local correction value.

[0098] A local brightness value is understood to be a brightness value at a specific location within the image. This location can, for example, refer to the position of a single pixel in the image, resulting in particularly high accuracy of the local correction value. Alternatively, the location can refer to the position of a larger area of ​​the image, such as the position of an image block. This reduces the computational and storage requirements associated with determining the local correction values. The local brightness value can also refer to the entire image. This further reduces the computational and storage requirements. A local brightness value, especially a maximum local brightness value, can therefore have, for example, pixel-accurate, block-accurate, or frame-accurate resolution.

[0099] To determine a local correction value associated with a specific correction position within the comparison image, it can be provided that the maximum local brightness value, i.e., the image datum with the highest brightness, for the correction position is determined by comparing the brightness values ​​of the images in the comparison image time series at that correction position. This maximum local brightness value can then be used as the local correction value for the correction position.

[0100] Using maximum local brightness values ​​as local correction values ​​is advantageous because it can be assumed that one or more of the stored images in the comparison time series captured the flickering light information that caused the image error to be corrected in the "on" state or in a state of high brightness. By determining maximum local brightness values, those image data from the comparison time series can be identified simply and efficiently that show the pulsed light source or the flickering light information it emits in the "on" state or in a state of high brightness.

[0101] In a particularly advantageous embodiment of the invention, all maximum local brightness values ​​for all possible correction positions can be determined and stored from the image data of the comparison image time series. For example, the maximum brightness value for each individual pixel position can be determined from the image data of the comparison image time series. In this way, a maximum brightness image is generated whose image data exhibits the maximum brightness value of the comparison image time series for each correction position. If necessary, i.e., if the generated error mask indicates an image defect of the comparison image at at least one error position, the correction value for the comparison image can then be extracted from the maximum brightness image at the correction position corresponding to the error position.

[0102] The image data of the maximum brightness image can also be advantageously subjected to dilation, in which the bright areas of the maximum brightness image are widened. This dilation can advantageously compensate for relative movements of the pulsed light sources or the flickering light information they emit, thus achieving improved accuracy of the local correction values ​​derived from the maximum brightness image.

[0103] In a further advantageous embodiment of the invention, it is provided that an output image for the viewer is generated from the exposure series, which can in particular be an HDR exposure series generated by an HDR image sensor, by means of HDR post-processing (high-dynamic-range post-processing) using at least one corrected reference image. First, one or more reference images that are part of the imported exposure series are corrected by inserting local correction values ​​at the error positions indicated by the error mask. This generates a corrected exposure series containing images with different exposure times, namely at least one reference image and at least one corrected reference image. From the image data of the images in this corrected exposure series, an output image for the viewer can be generated by HDR post-processing in a manner known to those skilled in the art.

[0104] Generating such an output image from the (corrected) exposure series using HDR post-processing offers two advantages. First, the different exposure times of the images in the series provide a greater dynamic range compared to a single image, thus improving the quality of the output image. Second, it avoids image errors caused by short exposure times, as the corrections made in the respective reference images can be applied to the output image. In this way, a flicker-reduced or even flicker-free output image with improved dynamic range can be produced.

[0105] In a further advantageous embodiment of the invention, the acquired exposure series is provided that it is part of an exposure series sequence generated by the image sensor, wherein the exposure series sequence comprises a succession of exposure series. The sequence of exposure series shows temporally successive scenes that may contain flickering light information.

[0106] Such an exposure bracketing sequence can, in particular, be a video sequence generated by the image sensor. In this case, each of the temporally successive scenes is not merely represented by a single frame, as is the case with conventional image sequences for displaying moving images, but rather each of the successive scenes is represented by an exposure bracket comprising several images with different exposure times.

[0107] The method according to the invention can thus be applied to the individual temporally successive exposure series of an exposure series sequence, in particular a video sequence, so that image errors caused by flickering light information in the exposure series sequence can be advantageously detected and corrected. The method according to the invention is therefore particularly suitable for the detection and correction of image errors caused by flickering light information in video sequences.

[0108] In a further advantageous embodiment of the invention, the exposure time of at least one reference image is between 50% and 100% of the image duration of the exposure series sequence. For a frame rate of 60 frames per second, i.e., an image duration of 1 / 60 s (-16.67 ms), the exposure time of at least one reference image is between 1 / 120 s (~8.33 ms) and 1 / 60 s (-16.67 ms). The exposure time of at least one comparison image can, in particular, be 1 ms or less.

[0109] In the case of a conventional image sequence, where a series of temporally successive images captured and displayed at a specific frame rate is used to represent moving images, the frame duration is the reciprocal of the frame rate, i.e., the time interval between two immediately successive images. Similarly, the frame duration of an exposure bracket sequence is understood to be the time interval between two immediately successive exposure brackets.

[0110] The image sensor can, for example, generate exposure series at a frequency of 60 Hz. In this case, the image duration of the exposure series sequence generated by the image sensor is 1000 / 60 ms = 50 / 3 ms ≈ 16.67 ms. If the exposure time of at least one reference image in this case is to be between 50% and 100% of the image duration of the exposure series sequence, the resulting exposure times are between 25 / 3 ms and 50 / 3 ms. However, the image sensor can also generate exposure series at a higher or lower frequency, e.g., at a frequency of 50 Hz, 30 Hz, 25 Hz, 100 Hz, or at another frequency. The method according to the invention is fundamentally independent of the frequency at which the image sensor generates the exposure series of the exposure series sequence and can be applied to exposure series sequences of any image duration.

[0111] If the exposure time of at least one reference image is chosen to be between 50% and 100% of the exposure duration of the exposure series sequence, this offers the advantage that the on state of the switched-on pulsed light sources, e.g., switched-on LEDs, is highly likely to fall completely or at least partially within the exposure time and thus be captured by the reference image. If the exposure time of at least one reference image is chosen to be 100% of the exposure duration of the exposure series sequence, it can be assumed that the on state of the switched-on pulsed light sources, e.g., switched-on LEDs, will definitely be captured by the reference image.If the exposure time of at least one reference image is chosen to be between 50% and 100% of the image duration of the exposure series sequence, it can therefore be advantageously ensured that image errors in the image data of the comparison image, which were caused by flickering light information, can be reliably detected by comparison with the image data of the reference image.

[0112] If the exposure time of at least one comparison image is 1 ms or less, this offers the advantage that the comparison image will have sufficient sharpness even if the scene depicted contains rapid relative movements. With a longer exposure time, the parts of the scene affected by the relative movement could be rendered blurry. Furthermore, this short exposure time of the comparison image ensures that even very bright areas of the scene can be reproduced without errors, as the image sensor does not reach saturation in these very bright areas due to the short exposure time. Undesirable clipping effects in the image data of the comparison image, caused by overdriving the image sensor, are thus avoided.

[0113] In a further advantageous embodiment of the invention, the previously described process steps of the inventive method are repeated. The inventive method is applied to a plurality of exposure series of the exposure series sequence or to each exposure series of the exposure series sequence. This generates a corrected exposure series sequence comprising corrected comparison images.

[0114] The exposure series sequence can, in particular, be a video sequence. In this way, the method according to the invention can advantageously be applied, for example, to a video sequence that contains image errors caused by flickering light information, in order to detect and correct the image errors of the video sequence, so that a flicker-reduced or even a flicker-free video sequence can be provided.

[0115] In a further advantageous embodiment of the invention, a correction method for correcting remaining image errors caused by flickering light information is applied to the image data of the images from the corrected exposure series sequence. This correction method can be based on motion estimation and motion compensation of the image data from the corrected exposure series sequence. Additionally or alternatively, the correction method can be based on mixing the image data of different images from the corrected exposure series sequence.

[0116] Even after applying the previously described correction of image errors according to the invention by inserting local correction values, the exposure series sequence, which can be a video sequence in particular, may still exhibit residual image errors caused by flickering light information. By mixing the image data of different images in the corrected exposure series sequence, the images of the corrected exposure series sequence are essentially interpolated. If flickering light information was correctly captured in one of the images whose image data is being mixed, but not captured in another image due to an image error, this correctly captured light information is retained after mixing. Although the light information is attenuated by the mixing, it is still present.The images from the exposure series sequence can be motion-compensated for mixing after motion estimation, so that image content shifted by an intermediate relative movement is recombined for quasi-interpolation. Thus, methods for motion estimation and motion compensation known to those skilled in the art can be used for this process.

[0117] In this way, remaining image errors caused by flickering light information can be advantageously corrected in the images of the corrected exposure series sequence.

[0118] As explained at the beginning, flickering light information in image sequences generated by vehicle cameras can pose safety risks, e.g., an increased risk of accidents, if the image sequences are displayed incorrectly or incompletely in the vehicle due to the flickering light information.

[0119] The method according to the invention is therefore advantageously carried out in a motor vehicle.

[0120] In an advantageous embodiment of the invention, the method according to the invention is carried out by an image processing unit integrated into a motor vehicle. The method according to the invention can, in particular, be carried out by a data processing unit that is part of or connected to the image processing unit. The exposure series can, in particular, be an exposure series generated by an image sensor integrated into the motor vehicle. The image sensor integrated into the motor vehicle can be part of the image processing unit or connected to the image processing unit.

[0121] Such an image sensor integrated into the motor vehicle can, in particular, be a camera integrated into the motor vehicle or a component of a camera integrated into the motor vehicle. Such a camera integrated into the motor vehicle is also referred to as a vehicle camera. Such a camera integrated into the motor vehicle can, in particular, be a front camera, a rear camera, a reversing camera, a side camera, or another type of camera integrated into the motor vehicle. Such a camera integrated into the motor vehicle can, in particular, be a camera that serves the driver to observe the vehicle's surroundings. Such a camera integrated into the motor vehicle can also, in particular, be a monocular camera or a stereo camera that serves as an optical sensor for acquiring sensor data for one or more driver assistance systems of the motor vehicle.

[0122] Such a further development of the inventive method is particularly advantageous because it can reduce or even completely eliminate the aforementioned safety risks associated with the use of cameras in motor vehicles. This is of particular importance since LEDs, which cause flickering light signals, are becoming increasingly common in the automotive environment.

[0123] The flickering light information can therefore be, in particular, flickering light information caused by light emitted by light-emitting diodes.

[0124] In an advantageous further development of the invention, the image processing unit set up for carrying out the method according to the invention and / or a camera which is part of the image processing unit or is connected to the image processing unit is integrated into a motor vehicle.

[0125] In this way, it is advantageously possible to use the inventive method in motor vehicles and to reduce or even completely eliminate the aforementioned safety risks in the context of the use of cameras in motor vehicles.

[0126] The invention is explained in more detail below by way of example embodiments with the accompanying drawings. These show: Figure 1 - schematic representation of an exemplary exposure series with one reference image and two comparison images, as well as two error masks generated therefrom; Figure 2 - schematic representation of a first embodiment of the method according to the invention; Figure 3 - block diagram of a second embodiment of the method according to the invention; Figure 4 - block diagram of an embodiment of an error correction according to the invention using maximum brightness values; Figure 5 - block diagram of an image processing unit for carrying out the method.

[0127] Figure 1Figure 1 shows a schematic representation of an exemplary exposure series as it is read in during the first step of the method according to the invention, as well as two error masks E(Si) and E(VSi) generated therefrom. The exposure series consists of a reference image Li, a first comparison image Si, and a second comparison image VSi. The exposure series is part of a video sequence generated by an image sensor, which comprises successive exposure series produced at a frame rate of 60 Hz. The image sensor generates an exposure series in each time step, the length of which corresponds to the frame duration of the video sequence of approximately 16.67 ms. The index i indicates that it is an exposure series generated by the image sensor in time step i, i.e., the i-th exposure series of the video sequence.

[0128] Images L i, Si, and VSi of the exposure series show the same scene 4 with a vehicle 3 whose headlights 1, 2 use pulsed LEDs and therefore emit flickering light information. The images in the exposure series were taken with different exposure times. The exposure time of the reference image L i is TL = 16 ms, the exposure time of the first comparison image Si is Ts = TL / 4 = 4 ms, and the exposure time of the second comparison image VSi is Tvs = Ts / 4 = TL / 16 = 1 ms. The images therefore exhibit different brightness levels. While image Si reproduces the brightness largely realistically, image VSi is underexposed due to its short exposure time, and image L i is overexposed due to its long exposure time.

[0129] Both headlights 1 and 2 of the vehicle are switched on. All images, i.e., both the reference image L i and the comparison images Si and VSi, therefore show the on state of the first headlight 1. However, due to their short exposure times, the comparison images Si and VSi do not show the on state, but rather the off state of headlight 2. This is because the exposure times of the comparison images Si and VSi fall entirely within the off phase of the pulsed LEDs of headlight 2. This effect occurs quasi-randomly; that is, an immediately preceding comparison image Si-1 or VS i-1, or an immediately following comparison image Si+1 or VS i+1 with the same exposure time, could already show the on state of headlight 2. The two comparison images Si and VSi thus exhibit image errors caused by the flickering light information emitted by headlight 2.

[0130] Due to its long exposure time, reference image L i shows both spotlights 1 and 2 in their on state; that is, reference image L i exhibits no image errors caused by flickering light information. This fact can be used to identify and correct image errors caused by flickering light information in the comparison images.

[0131] For this purpose, the image data of the respective comparison image Si or VSi are compared with the image data of the reference image L i spatially, i.e., positionally, and in this way, local deviations between the image data are determined. In the example shown, the comparison is performed pixel by pixel, i.e., the image data is compared for each individual pixel of the images. The image data is in the form of brightness values ​​with an exemplary resolution of 8 bits, i.e., in a value range of 0 to 255.

[0132] Due to the long exposure time, the image data of the reference image Li has reached saturation in the area of ​​the two spotlights 1 and 2. As a result of the image sensor being overdriven, the corresponding pixels of the reference image at positions x1, y1 of the first spotlight 1 and positions x2, y2 of the second spotlight 2 exhibit maximum brightness values ​​Li(x1, y1) = Li(x2, y2) = 255. Relative to an absolute brightness scale, the reference image in the area of ​​the spotlights is much too dark due to this saturation.

[0133] To compare the image data of images taken with different exposure times, these are first normalized to a uniform reference value. The reference value for the normalized image data can, in principle, be freely chosen. In the Figure 1In the example shown, the brightness value at an exposure time of TL = 16 ms is defined as normalized image data, i.e., as normalized brightness. The image data L i (x,y) of the comparison image L i, which was taken with an exposure time of TL = 16 ms, is therefore already in normalized form, so no further normalization is necessary.

[0134] The image data Si(x,y) and VSi(x,y) of the first and second comparison images, however, are based on a shorter exposure time and are therefore normalized to the exposure time TL of the reference image for comparison purposes by multiplying the brightness values ​​by a factor ks or kv, which results from the ratio of the exposure times. The following applies: k S = T L / T S = 16 ms / 4 ms = 4 for the first comparison image Si and k V = T L / T VS = 16 ms / 1 ms = 16 for the second comparison image VSi. This yields the normalized brightness values ​​ks · Si(x,y) and kv VSi(x,y), which can be directly compared with the image data L i (x,y) of the reference image to determine local deviations between the image data.

[0135] It is immediately apparent that the reference image at the pixel positions showing the first spotlight 1 has a significantly lower brightness L i (x 1 ,y 1 ) = 255 due to saturation than the first comparison image and the second comparison image, whose normalized brightnesses at these positions of the image are ks · S i (x 1 ,y 1 ) = 4 · 200 = 800 and kv VS i (x 1 ,y 1 ) = 16· 50 = 800 respectively.

[0136] Furthermore, by comparing the image data locally, it becomes apparent that the reference image has a significantly higher brightness L i (x 2 ,y 2 ) = 255 at the pixel positions showing the second spotlight 2 than the first comparison image and the second comparison image, whose normalized brightnesses at these positions of the image are only k S · S i (x 2 ,y 2 ) = 4 · 20 = 80 and kv VS i (x 2 ,y 2 ) = 16· 5 = 80 respectively.

[0137] The normalized brightness of the comparison images is therefore significantly lower in the area of ​​spotlight 2 than the brightness of the reference image. This results in local deviations at the positions of the comparison images whose pixels show the off state of the pulsed LEDs of spotlight 2, even though it is actually switched on. These deviations can be quantified, for example, by a difference ΔS(x,y) or ΔV(x,y) between the normalized image data at the pixel position (x,y). In the example shown, this results in Δ S x 2 / y 2 = L i x 2 y 2 − k S ⋅ S i x 2 y 2 = 255 − 80 = 175 for the first comparison image Si and Δ V x 2 / y 2 = L i x 2 y 2 − k V ⋅ VS i x 2 y 2 = 255 − 80 = 175 for the second comparison image VSi.

[0138] These local deviations between the image data of the reference image Li and the image data of the comparison images Si and VSi indicate that the image data of the reference images in the area of ​​the second headlight contain image errors caused by flickering light information. Depending on the local deviations, error masks E(Si) and E(VSi) can therefore be generated, which show the error positions of image errors within the comparison images Si and VSi, respectively.

[0139] To generate such an error mask E(Si(x,y)) or E(VS i (x,y)), the following conditions can be set for the first comparison image Si, for example E S i x y = 1 , wenn L i x y > k S ⋅ S i x y , E S i x y = 0 , sonst or E S i x y = 1 , wenn L i x y > > k S ⋅ S i x y , E S i x y = 0 , sonst can be applied. For the second comparison image VSi, the conditions can be adjusted accordingly. E VS i x y = 1 , wenn L i x y > k V ⋅ VS i x y , E VS i x y = 0 , sonst or E VS i x y = 1 , wenn L i x y ≫ k V ⋅ VS i x y , E VS i x y = 0 , sonst be applied.

[0140] Alternatively, to generate the error mask, the local deviations quantified by the differences ΔS(x,y) and ΔV(x,y) can be compared with a threshold value Eth. In this case, an image error at the error position (x,y) is assumed if the threshold value is reached or exceeded, i.e., the following conditions apply. E S i x y = 1 , wenn Δs x y ≥ E th , E S i x y = 0 , wenn Δs x y < E th for the first comparison image Si and E VS i x y = 1 , wenn Δv x y ≥ E th , E VS i x y = 0 , wenn Δv x y < E th for the second comparison image VSi.

[0141] In this way, an error mask can be generated which displays the error positions of image defects within the respective comparison image that were caused by flickering light information.

[0142] In the example shown, the Figure 1The resulting error mask E(Si) for the first comparison image Si (consistently, regardless of which of the conditions listed above is used to generate the error mask) indicates an image error for the pixel positions (x 2 ,y 2 ) that represent the second spotlight 2, i.e., E(Si(x 2 ,y 2 )) = 1. For all other pixel positions, E(Si(x,y)) = 0. The resulting error mask E(VSi) for the second comparison image VSi contains, accordingly, the value E(VSi(x 2 ,y 2 )) = 1 for the area of ​​the second spotlight 2 and E(VSi(x,y)) = 0 for all other pixel positions.

[0143] Using the generated error masks, it is now possible to correct the image errors of the comparison images at the error positions indicated by the error mask using correction values. The steps according to the invention of determining local correction values ​​and generating a corrected comparison image are described below. Figures 2 to 5explained.

[0144] In contrast to the simplified example shown here, when normalizing image data, other exposure parameters and operating characteristics of the image sensor can be taken into account in addition to the exposure time. These include, for example, the sensor sensitivity, the analog gain in the sensor A / D converter, the aperture value, and / or the camera response curve of the image sensor.

[0145] Figure 2 Figure 1 shows a schematic representation of a first embodiment of the method according to the invention. In accordance with the Figure 1The example shown again depicts a reference image Li and a comparison image Si, which are components of an exposure series read in during the first step of the procedure. Images Li and Si of the exposure series show the same scene 4 with a vehicle 3 whose headlights 1, 2 use pulsed LEDs and therefore emit flickering light information. The exposure time of the reference image Li is TL = 16 ms, and the exposure time of the comparison image Si is Ts = TL / 4 = 4 ms. The image data is again in the form of brightness values.

[0146] The imported exposure series, which includes images L i and Si, is, analogous to the preceding explanations, the i-th exposure series of a video sequence generated by an image sensor, i.e., an exposure series sequence that includes exposure series generated in successive time steps.

[0147] Both headlights 1 and 2 of the vehicle are switched on. While the reference image L i, due to its long exposure time, shows both headlights 1 and 2 in their on state, the comparison image Si shows not the on state, but the off state of headlights 1 and 2. This is because the short exposure times of the comparison image Si fall entirely within the off phase of the pulsed LEDs of headlights 1 and 2. The comparison image Si therefore exhibits image errors caused by the flickering light information emitted by headlights 1 and 2.

[0148] Analogous to the explanations regarding the one in Figure 1In the example shown, local deviations between the image data of the reference image L i and the comparison image Si are determined by comparing the image data of the two images pixel by pixel. These local deviations occur at those pixel positions in the images that show the flickering light information emitted by the headlights 1, 2 of the vehicle 3. Depending on the determined local deviations, analogous to the explanations in Figure 1 In the example shown, an error mask E(Si) is generated. The error mask E(Si) shows the error positions of the image errors caused by flickering light information within the comparison image Si. These are precisely the pixel positions where the images show either spotlight 1 or spotlight 2, since the reference image Li has a significantly higher brightness at these positions than the comparison image Si due to the pulsed light emitted by the LEDs.

[0149] This identifies the positions where the comparison image Si exhibits image errors caused by flickering light information, which require correction. Using the generated error masks, it is now possible to correct the image errors of the comparison images at the error positions indicated by the error mask using local correction values.

[0150] To determine the local correction values, the following is used in the Figure 2In the illustrated embodiment, an estimation is applied that is based on a comparison image time series S' iN,..., S' i-1 , S' i stored in a data storage device. The comparison image time series contains N+1 stored images and can, for example, contain a total of 30 stored images. The stored images S' iN ,... S' i-1 are modified copies of comparison images S iN ,... S i-1 that belong to immediately preceding exposure series of the video sequence, which the image sensor generated in earlier time steps iN, ..., i-1 immediately preceding the current time step i. The stored images S' iN ,... S' i-1 were generated with the same exposure time as the currently read comparison image S. In addition, the comparison image time series S' iN ,..., S' i-1 , S' i contains a modified copy of the currently read comparison image Si. The stored images S' iN ,...S'i-1, S'i represent modified copies of the comparison images SiN,... Si-1, Si in that they were reduced in size by downscaling before being stored in the data storage. In this way, the size of the data storage required for storing the comparison image time series S'iN,... S'i-1, S'i can be reduced. Advantageously, the stored images S'iN,... S'i-1, S'i can also be stored in motion-compensated form to compensate for relative movements of the vehicle 3 to the image sensor.

[0151] To determine the local correction values ​​for the image data of the comparison image Si, the fact that image errors caused by flickering light information occur quasi-randomly is exploited. The stored images S' iN ,... S' i-1, taken at earlier times, can therefore, unlike the comparison image Si, show the on state of the headlight 1 or 2 if the on phase of the pulsed LEDs of the headlight has fallen at least partially within the exposure time of one or more of the stored images S' iN ,... S' i-1. In the exemplary embodiment of the Figure 2 For example, image S' iN shows the on state of headlight 1 and image S' i-1 shows the on state of headlight 2.

[0152] To determine the local correction values ​​for the image data of the comparison image, the following is done according to the method described in Figure 2In the illustrated embodiment, the maximum local brightness value is determined and stored for each individual pixel position, i.e., for all possible correction positions, from the image data of the comparison image time series S' iN ,... S' i-1 , S'i. In this way, a maximum brightness image max(S'i) can be generated, whose image data exhibits the maximum brightness value of the comparison image time series S' iN ,... S' i-1 , S'i for each correction position. The image data of the maximum brightness image max(S'i) is also subjected to dilation, in which the bright areas of the maximum brightness image are expanded.

[0153] The maximum local brightness values ​​of the generated maximum brightness image can now be used as local correction values ​​for the image errors of the comparison image Si at the corresponding error positions. As a result of the dilation of the maximum brightness image, the downscaling of the stored images, and relative movements of the vehicle 3 to the image sensor between the time steps iN, ..., i-1, i of the video sequence, the maximum local brightness values ​​of the maximum brightness image max(S' i ) do not exhibit high positional accuracy. However, high accuracy is not necessary in this respect, since the highly accurate error positions of the image errors present in the comparison image can be obtained from the error mask E(Si).

[0154] Using the comparison image Si, the error mask E(Si), and the maximum brightness values ​​of the maximum brightness image max(S'i), a corrected comparison image S i,cor is generated in which the image errors caused by the pulsed LEDs of the headlights 1, 2 have been corrected. For this purpose, at each error position indicated by the error mask E(Si), the brightness value of the comparison image Si is replaced by the maximum brightness value of the maximum brightness image max(S'i); that is, the maximum brightness values ​​are inserted into the comparison image as local correction values. In the exemplary embodiment of the Figure 2This results in the brightness values ​​of all pixels in the comparison image Si, which show the pulsed LED-operated spotlights 1, 2, being replaced by the maximum brightness values ​​from the maximum brightness image max(S'i). The corrected comparison image S i,cor therefore shows both spotlights in the on state and thus no longer exhibits any image errors.

[0155] By repeatedly performing the in Figure 2 According to the invention, by applying the described method to a plurality of successive exposure series of the video sequence, it is possible to generate a corrected video sequence with corrected comparison images S i,cor.

[0156] Figure 3 shows a block diagram of a second embodiment of the method according to the invention.

[0157] Shown is an image sensor 10, which is an HDR image sensor and has generated an HDR exposure series. This differs from the one in Figure 2The first embodiment shown comprises the Figure 3 The exposure series shown includes, in addition to the reference image L and the comparison image S, another comparison image VS. To simplify the representation, the time index i of images Li, Si, and VSi, which indicates the time step in which the image was taken, is shown in the Figure 3 Omitted. Regarding the properties of the images. L, S and VS, particularly regarding their different exposure times, can be found in the corresponding explanations in Figure 1 Reference is made to the example shown. In this embodiment as well, the image data is in the form of brightness values.

[0158] The block diagram of Figure 3The figure also shows a flicker detection unit 21 for detecting image errors caused by flickering image information, a flicker correction unit 22 for correcting image errors caused by flickering image information, and an HDR post-processing unit 24, which generates an output image through HDR post-processing. The flicker detection unit 21, the flicker correction unit 22, and the HDR post-processing unit 24 can be implemented in software and / or hardware as components of a data processing unit 11, which is located in Figure 3 not shown, but in Figure 5 shown.

[0159] The image data of the exposure series with the three images L,S and VS are provided by the image sensor 10 via corresponding interfaces to both the flicker detection unit 21 and the flicker correction unit 22. The image data of the reference image L is also provided without further processing via an interface to the HDR post-processing unit 24.

[0160] The flicker detection unit 21 in functional block 25 first performs a multiplication of the brightness values ​​of the reference image L by the quotient 1 / ks to account for the different exposure times and sensor sensitivities of images L and S and to enable a direct comparison of the image data of these two images. ks denotes the value associated with the Figure 1The explained factor, which results from the ratio of the exposure times of both images as ks = TL / Ts, assuming equal sensor sensitivities, is then compared in function block 27 with the image data of the comparison image S. This determines local deviations between the image data of both images. Depending on the determined local deviations of the image data, an error mask E(S) is generated, which displays the error positions of image defects within the comparison image S.

[0161] Similarly, in function block 26, the flicker detection unit 21 multiplies the brightness values ​​of the reference image L by the quotient 1 / kv to account for the different exposure times of images L and VS and to enable a direct comparison of the image data of these two images. Kv is derived from the ratio of the exposure times as kv = TL / Tv. Alternatively, function block 26 can perform the comparison using the already corrected image Scor and the quotient kvs. Instead of the reference image L, the already corrected image Scor is then used, and the quotient is 1 / kvs instead of 1 / kv, with kvs = Ts / Tv. The resulting image data VS CMP from the multiplication are then compared in function block 28 with the image data of the comparison image VS, and in this way, local deviations between the image data of both images are determined.Depending on the determined local deviations of the image data, an error mask E(VS) is generated, which displays the error positions of image errors within the comparison image VS.

[0162] The generated error masks E(S) and E(VS) are passed on to the flicker correction unit 22.

[0163] The flicker correction unit 22 has a functional block 231 which, with the aid of an associated data memory 121, performs a maximum-value-based image error correction of the image data of the comparison image S. The flicker correction unit 22 further has a functional block 232 which, with the aid of an associated data memory 122, performs a maximum-value-based image error correction of the image data of the comparison image VS. The data memory 121 and the data memory 122 can be memory areas in a common memory 12, which is located in the Figure 3 not shown, but in the Figure 5As shown, to perform the image error correction, the image data of the comparison image S and the error mask E(S) are provided to function block 231, and the image data of the comparison image VS and the error mask E(VS) are provided to function block 232. Based on this, the flicker correction unit 22 generates a corrected comparison image S cor and a corrected comparison image VS cor, the image data of which are provided to the HDR post-processing unit 24 via corresponding interfaces.

[0164] The HDR post-processing unit 24 ultimately generates an output image HDR-OUT, which is an HDR output image, by means of HDR post-processing using the reference image L, the corrected comparison image S cor, and the corrected comparison image VS cor. The output image HDR-OUT can be displayed directly to a viewer, stored, transmitted over a network, and / or further processed in subsequent image processing steps.

[0165] Figure 4 Figure 1 shows a block diagram of an embodiment of an error correction according to the invention using maximum brightness values. This is a detailed representation of the already described in the Figure 3 shown functional block 231 and the associated data storage 121.

[0166] The image data of the comparison image Si, read in with the exposure series, form the input data IN of the function block 231, which is responsible for the maximum value-based image error correction of the image data. The image data is in the form of brightness values.

[0167] The image data can also be in the form of values ​​for the RGGB color components, such as those output by a Bayer sensor. In this case, blocks 31, 32, 33, 121, 35, and 34 are present once for each color component. Advantageously, only the maximum of the two G-values ​​in the Bayer pattern is processed, rather than processing the two G-values ​​separately. Following block 34, this processed G-value is used via multiplexer 36 for both G-values ​​of the output signal Si,cor, provided the error mask E(Si) indicates an image error at the image position.

[0168] As further input data, the function block 231 is provided with the data from the error mask E(Si), which displays the error positions of image defects within the comparison image Si. In the detailed view of the Figure 4It can be seen that the image data of the comparison image Si in functional block 31 are first reduced in size by downscaling. Advantageously, this downscaling can be performed, for example, as a 1:4 subsampling. Subsequently, the image data undergoes dilation in functional block 32, whereby the bright areas of the comparison image Si are widened. After dilation, the image data is again reduced in size by downscaling in functional block 33. Advantageously, this downscaling can be performed, for example, as a 1:8 subsampling.

[0169] In this way, a modified copy S'i of the comparison image Si is created and stored in data storage 121. Data storage 121 is a ring buffer configured to hold N+1 stored images S'iN, ..., S'i-1, S'i. When the modified copy S'i of the currently (i.e., the last read) comparison image Si is stored in ring buffer 121, the oldest image stored therein, S'iN-1, is deleted, so that the ring buffer always contains the most recent N+1 stored images. If the image data consists of values ​​for the color components R, G, and B of an RGB color space, then the individual values ​​of the color components R, G, and B are stored in the ring buffer.

[0170] In function block 35, the maximum local brightness value for all possible correction positions is then determined and stored from the image data of all images in the comparison image time series S' iN ,... S' i-1 , S'i. This generates a maximum brightness image, which is then enlarged by upscaling in function block 34. The image data of the resulting maximum brightness image exhibits the maximum brightness value of the comparison image time series S' iN ,... S' i-1 , S'i for each correction position.

[0171] Finally, in function block 36, a corrected comparison image Si,cor is generated using the error mask E(Si), the image data of the maximum brightness image obtained by upscaling (reversal of function blocks 31 and 33, upscaling factor for example 32:1), and the image data of the comparison image Si. This is achieved by inserting the maximum brightness values ​​at those positions in the comparison image Si for which the error mask E(Si) indicates an image error. The image data of the corrected comparison image Si,cor forms the output data OUT of function block 231.

[0172] In this way, within functional block 231, the image errors of the comparison image Si caused by flickering light information are corrected by a maximum value-based image error correction.

[0173] The explanations regarding the exemplary implementation of the Figure 4 apply accordingly to the in Figure 3shown and responsible for the image error correction of the comparison image VS, functional block 232 and the data storage 122 assigned to it.

[0174] Figure 5 Figure 1 shows a block diagram of an image processing unit for carrying out the method according to the invention. The image processing unit has a data processing unit 11 and a data storage unit 12 for storing image data. The data processing unit can be, for example, a suitably programmed microprocessor, a digital signal processor (DSP), a specialized image processing processor, an FPGA (field-programmable gate array), an ASIC (custom integrated circuit), or the like. The data processing unit 11 accesses the data storage unit 12 for reading and writing. The data storage unit 12 can be fragmented or consist of several storage units.

[0175] Images and / or exposure series comprising images are generated by an image sensor 10, which can be a video camera, in particular a digital video camera, another type of camera, or a component of such a camera. The image sensor 10 is directly or indirectly connected to the image processing unit 13, so that the image data of the images or the exposure series can be read by the image processing unit 13 in this way. The image data can then be temporarily stored in the data memory 12.

[0176] After the image data has been processed by the data processing unit 11 in the manner described above, output images can be transferred to a display unit 14, which may be, for example, a screen or other display, for playback. The display unit 14 is connected directly or indirectly to the image processing unit 13 for this purpose.

[0177] It is also conceivable that the original image data is stored on a suitable data carrier and / or transmitted further via a data connection.

[0178] The image processing unit 13 is integrated into a vehicle, in particular a motor vehicle. The image sensor 10 and the display unit 14 can, in particular, replace or supplement conventional rearview mirrors. The image sensor 10 is aligned with the field of view to be observed, and the display unit 14 is positioned within the driver's suitable field of vision.

Claims

1. Method for detecting and correcting image errors caused by flickering light information (1, 2) in images (Si, VSi) generated by an image sensor (10), the method being carried out by an image processing unit (13) integrated in a motor vehicle, wherein the image sensor (10) is integrated in the motor vehicle, and having the steps: - reading in an exposure series comprising at least one reference image (Li) and at least one comparison image (Si, VSi), wherein the images of the exposure series show the same scene (4) and the exposure time of the reference image (Li) is longer than the exposure time of the comparison image (Si, VSi), - determining local deviations of the image data of the comparison image (Si, VSi) from the image data of the reference image (Li) by locally comparing the image data of the comparison image (Si, VSi) with the image data of the reference image (Li), - generating an error mask (E(Si), E(VSi)) as a function of the determined local deviations of the image data, wherein the error mask (E(Si), E(VSi)) indicates the error positions of image errors within the comparison image (Si, VSi), - providing a comparison image time series (S'i-N, ..., S'i) in a data memory (12, 121, 122), wherein the comparison image time series comprises a number of stored images (S'i-N, ..., S'i-1) which were generated by the image sensor (10) with the same exposure time as the comparison image (Si) at different times than the comparison image (Si), in particular at earlier times than the comparison image (Si), - determining local correction values for the image data of the comparison image (Si, VSi) by an estimation in which, for each local correction value, a maximum local brightness value is determined from the image data of the images of the comparison image time series (S'i-N, ..., S'i) and the maximum local brightness value is used as the local correction value, wherein each local correction value is assigned to a correction position within the comparison image (Si, VSi), - generating a corrected comparison image (Si, cor, VSi,cor) by inserting the local correction values at the error positions of the comparison image (Si, VSi) indicated by the error mask (E(Si), E(VSi)).

2. Method according to Claim 1, characterized in that the image data are brightness values.

3. Method according to Claim 1 or 2, characterized in that the image sensor (10) is an HDR image sensor and the exposure series is an HDR exposure series generated by the HDR image sensor.

4. Method according to any one of the preceding claims, characterized in that the image sensor (10) is configured to generate the images (Li, Si, VSi) of the exposure series with a selectable light sensitivity, wherein the light sensitivity is at least selectable between a first light sensitivity and a second light sensitivity which is lower than the first light sensitivity, and the method comprises the following additional step: - generating the exposure series with the image sensor (10), wherein at least the reference image (Li) is generated with the second light sensitivity.

5. Method according to any one of Claims 1 to 3, characterized in that the image sensor (10) is configured to generate the images (Li, Si, VSi) of the exposure series with a selectable light sensitivity, wherein the light sensitivity is at least selectable between a first light sensitivity and a second light sensitivity which is lower than the first light sensitivity, and the method comprises the following additional steps: - selecting at least the light sensitivity with which the reference image (Li) is generated as a function of the brightness of the scene (4), wherein the first light sensitivity is selected in the case of lower brightness of the scene (4) and the second light sensitivity is selected in the case of higher brightness of the scene (4), - generating the exposure series with the image sensor (10), wherein at least the reference image (Li) is generated with the selected light sensitivity.

6. Method according to any one of the preceding claims, characterized in that, in order to determine the local deviations, the image data are normalized and the normalized image data at a position of the comparison image (Si, VSi) are compared with the normalized image data at the same position of the reference image (Li), in particular by determining a difference between the normalized image data at a position of the comparison image (Si, VSi) and the normalized image data at the same position of the reference image (Li).

7. Method according to any one of claims 4 and 5 and according to claim 6, characterized in that, in order to normalize the image data, a relative light sensitivity ratio between the light sensitivity of the reference image (Li) and the light sensitivity of the comparison image (Si, VSi) and / or an exposure time ratio between the exposure time of the reference image (Li) and the exposure time of the comparison image (Si, VSi) is used.

8. Method according to any one of the preceding claims, characterized in that the images of the comparison image time series (S'i-N, ..., S'i) are subjected to a dilatation before being stored in the data memory (12, 121, 122) and / or are reduced in size by downscaling and / or are compressed and / or the image data of the stored images of the comparison image time series (S'i-N, ..., S'i) are motion-compensated with respect to the comparison image (Si) and / or a motion compensation of the image data of the stored images of the comparison image time series (S'i-N, ..., S'i) is carried out with respect to the comparison image (Si) before determining the local correction values for the image data of the comparison image.

9. Method according to any one of the preceding claims, characterized in that the exposure series is part of an exposure series sequence generated by the image sensor (10), wherein the exposure series sequence comprises a sequence of exposure series and the sequence of exposure series shows temporally successive scenes (4) which may contain flickering light information (1, 2).

10. Method according to Claim 9, characterized in that the exposure time of at least one reference image (Li) is between 50% and 100% of the image duration of the exposure series sequence, in particular between 25 / 3 ms and 50 / 3 ms, and / or the exposure time of at least one comparison image (Si) is 1 ms or less.

11. Method according to Claim 9 or 10, characterized in that the method steps are repeated, wherein the method is applied to a plurality of exposure series of the exposure series sequence or to each exposure series of the exposure series sequence in order to generate a corrected exposure series sequence comprising corrected comparison images (Si, cor, VSi,cor).

12. Method according to one of Claims 9 to 11, characterized in that a correction method for correcting remaining image errors caused by flickering light information (1, 2) is applied to the image data of the images of the corrected exposure series sequence, wherein the correction method is based on a motion estimation and a motion compensation of the image data of the images of the corrected exposure series sequence and / or a mixing of the image data of different images of the corrected exposure series sequence.

13. Image processing unit (13) having an image sensor (10), a data memory (12) for storing image data and a data processing unit (11), characterized in that the image processing unit (13) is integrated in a motor vehicle and is configured to carry out the method according to any one of the preceding claims.

14. Computer program having program code means which are configured to carry out the method according to any one of Claims 1 to 12 when the computer program is executed with an image processing unit (13) according to claim 13.